mirror of
https://github.com/larvit/adf-codec.git
synced 2026-11-12 19:58:28 +01:00
Answer architecture round 2: every builder joins its pieces through one bounded join, the readers take the engine guard, every refusal stands at the document's path, and a portable list's refusal names its own place
This commit is contained in:
+4
-2
@@ -85,9 +85,11 @@
|
||||
where all three hung.
|
||||
- Fix a document whose markdown would be longer than 536,870,888 characters, V8's longest string:
|
||||
`adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape` on
|
||||
every engine, before building it, where they threw or ran out of memory.
|
||||
every engine, where they threw or ran out of memory.
|
||||
- Fix a directive whose CommonMark spelling would be that long: `losslessMarkdownToAdf` refuses it
|
||||
as `unsupported-node-shape`, where a few megabytes of nested list lines ran it out of memory.
|
||||
as `unsupported-node-shape`, where a few megabytes of nested list lines ran it out of memory. The
|
||||
markdown readers return `unsupported-node-shape` where they threw past the longest string or
|
||||
array the JavaScript engine builds.
|
||||
- Fix deeply nested compact JSON in an `adf:` fence: `losslessMarkdownToAdf` refuses it as
|
||||
`unsupported-node-shape`, as it refuses any compact JSON there, where about a megabyte of it threw.
|
||||
- Add `adfToPortableMarkdown` and `portableMarkdownToAdf`, a lossy pair converting ADF to and from
|
||||
|
||||
@@ -212,9 +212,9 @@ Each call's name says which markdown it reads or writes:
|
||||
| --- | --- | --- |
|
||||
| `adfToLosslessMarkdown` | writes the lossless flavour: CommonMark plus GFM's pipe tables and `~~`, with `!adf:` directives and `adf:` fences for everything else | `not-an-adf-document`, `unsupported-document-version`, `unsupported-nesting-depth`, `unsupported-node-shape` |
|
||||
| `adfToPortableMarkdown` | writes markdown GitHub, GitLab and Obsidian render; drops what they cannot show, and writes no directive | `not-an-adf-document`, `unsupported-document-version`, `unsupported-nesting-depth`, `unsupported-node-shape` |
|
||||
| `commonMarkToAdf` | reads strict CommonMark as its spec says: a directive, a pipe table, `~~`, `==` and an `adf:` fence read as text and code | `not-a-string`, `unmappable-html`, `unsupported-nesting-depth` |
|
||||
| `commonMarkToAdf` | reads strict CommonMark as its spec says: a directive, a pipe table, `~~`, `==` and an `adf:` fence read as text and code | `not-a-string`, `unmappable-html`, `unsupported-nesting-depth`, `unsupported-node-shape` |
|
||||
| `losslessMarkdownToAdf` | reads the lossless flavour, markdown `adfToLosslessMarkdown` wrote, edited or not | every code in [The errors](#the-errors) except `not-an-adf-document` and `unsupported-document-version` |
|
||||
| `portableMarkdownToAdf` | reads CommonMark, GFM's pipe tables and `~~`, alerts and task markers, Obsidian's callouts and `==`; a directive and an `adf:` fence read as text and code | `malformed-pipe-table`, `not-a-string`, `unmappable-html`, `unsupported-nesting-depth` |
|
||||
| `portableMarkdownToAdf` | reads CommonMark, GFM's pipe tables and `~~`, alerts and task markers, Obsidian's callouts and `==`; a directive and an `adf:` fence read as text and code | `malformed-pipe-table`, `not-a-string`, `unmappable-html`, `unsupported-nesting-depth`, `unsupported-node-shape` |
|
||||
|
||||
`isAdfDocument` is the type guard for the boundary you receive JSON at. It checks the shape the
|
||||
converters require: `type: 'doc'`, a finite `version` (the converters take only 1), nodes holding
|
||||
@@ -445,7 +445,7 @@ The emitter raises these, and so does a parse, which asks the emitter how to spe
|
||||
| Code | Fires when | What you can do |
|
||||
| --- | --- | --- |
|
||||
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
|
||||
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters, keep the ADF and pass the document over, or show it read-only |
|
||||
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node whose CommonMark spelling would be longer than 536,870,888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536,870,888 characters, V8's longest string, on every engine. In every call: a conversion that passes the longest string or array the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536,870,888 characters or the engine's limit, set the document aside: keep the ADF and pass it over, or show it read-only |
|
||||
|
||||
## The guarantees
|
||||
|
||||
|
||||
+17
-14
@@ -462,11 +462,11 @@ handles one cause alike whichever node, attribute or direction raised it.
|
||||
- A non-finite number takes two codes: `unsupported-node-shape` parsing, `not-an-adf-document`
|
||||
emitting — no document holds one, so no round-trip crosses them (2026-09-23).
|
||||
- A document whose markdown would be longer than V8's longest string is `unsupported-node-shape`
|
||||
at the block that passes it, in either writer, and so is a directive `losslessMarkdownToAdf`
|
||||
reads whose CommonMark spelling would be: the document does not convert, and no spelling in the
|
||||
input causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested
|
||||
JSON, since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in
|
||||
a length linear in its JSON (2026-10-11).
|
||||
at the document's path, in either writer, and so is a directive `losslessMarkdownToAdf` reads
|
||||
whose CommonMark spelling would be: the document does not convert, and no spelling in the input
|
||||
causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested JSON,
|
||||
since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in a
|
||||
length linear in its JSON (2026-10-11).
|
||||
- The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at
|
||||
position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and
|
||||
`not-an-adf-document` names the emitters' input (2026-10-05).
|
||||
@@ -665,15 +665,18 @@ or pins it.
|
||||
is the shortest of the engines the library claims.
|
||||
|
||||
Every writer, and the reader's CommonMark ask, refuses markdown longer than 2^29 − 24 characters
|
||||
(`longestOutput`) before building it, so the same document converts alike everywhere and no
|
||||
amplification — an indent on every line of a deep subtree, the carry's two-space JSON, text one
|
||||
string shared by many nodes — builds a string no engine was asked for. Each walk of blocks projects
|
||||
their length with the indent their containers will prefix every line with, and a container joining
|
||||
several walks — a list's items, a table's rows — sums their projections before it joins them, so the
|
||||
bound trips at the block that passes it, before any container copies it. The projection never
|
||||
undercounts: a list giving way to the directive form counts its items' marker indent, so markdown a
|
||||
few characters a line under the bound may be refused. The engines' own throws past their longest
|
||||
string or array stay refused as `unsupported-node-shape` by `withinEngineLimits`, behind this bound.
|
||||
(`longestMarkdown`) as `unsupported-node-shape` at the document's path, so the same document
|
||||
converts alike everywhere. Each builder that joins pieces — blocks, quoted and list lines, directive
|
||||
pairs and openers, fences, pipe rows, the carry's JSON — joins them through `joinedWithin`, which
|
||||
stops once they pass the bound, so none builds a string no engine was asked for. A walk of blocks
|
||||
also forecasts each block's length with the indent its containers will prefix every line with, so a
|
||||
deep subtree refuses before any container copies it.
|
||||
|
||||
A line is held to its text's length alone; what its spelling adds — escapes, link targets,
|
||||
attributes — passes V8's longest string only by throwing, and `withinEngineLimits` refuses that
|
||||
throw, and SpiderMonkey's, as the same refusal: both mean a string past the bound. `todo.md` item
|
||||
108 bounds the line itself. The engines' throws past their own longest array, and JavaScriptCore's
|
||||
`Out of memory`, which also reports a heap running short, stay refused with the engine's message.
|
||||
|
||||
## Nothing recurses unbounded
|
||||
|
||||
|
||||
+1
-1
@@ -86,7 +86,7 @@ export function textLength(node: AdfNode): number {
|
||||
return node.text?.length ?? 0
|
||||
}
|
||||
|
||||
// A line spells every character of its nodes' text at least once, so this much text cannot fit.
|
||||
// The length of the text the nodes hold, the nodes below them included.
|
||||
export function heldTextLength(nodes: readonly AdfNode[]): number {
|
||||
let length = 0
|
||||
const pending = [...nodes]
|
||||
|
||||
@@ -16,16 +16,21 @@ export function closingBacktickRun(text: string, from: number, opener: number):
|
||||
}
|
||||
|
||||
export function fencedCodeBlock(info: string, body: string): string {
|
||||
return fencePieces(info, body).join('')
|
||||
}
|
||||
|
||||
export function fencePieces(info: string, body: string): string[] {
|
||||
const fence = '`'.repeat(Math.max(3, longestBacktickRun(body) + 1))
|
||||
return body === '' ? `${fence}${info}\n${fence}` : `${fence}${info}\n${body}\n${fence}`
|
||||
return body === '' ? [fence, info, '\n', fence] : [fence, info, '\n', body, '\n', fence]
|
||||
}
|
||||
|
||||
export function longestBacktickRun(text: string): number {
|
||||
let longest = 0
|
||||
let current = 0
|
||||
for (const character of text) {
|
||||
current = character === '`' ? current + 1 : 0
|
||||
longest = Math.max(longest, current)
|
||||
for (let start = text.indexOf('`'); start !== -1; ) {
|
||||
let end = start + 1
|
||||
while (text.charAt(end) === '`') end += 1
|
||||
longest = Math.max(longest, end - start)
|
||||
start = text.indexOf('`', end)
|
||||
}
|
||||
return longest
|
||||
}
|
||||
|
||||
@@ -112,9 +112,18 @@ export function slotFault(type: string, text: string): ConvertFault | undefined
|
||||
}
|
||||
|
||||
export function spellAttributes(pairs: readonly (readonly [string, string])[]): string {
|
||||
if (pairs.length === 0) return ''
|
||||
const spelled = [...pairs].sort(([left], [right]) => keyOrder(left, right)).map(([key, value]) => `${key}=${value}`)
|
||||
return `{${spelled.join(' ')}}`
|
||||
return [...attributePieces(pairs)].join('')
|
||||
}
|
||||
|
||||
export function* attributePieces(pairs: readonly (readonly [string, string])[]): Generator<string> {
|
||||
if (pairs.length === 0) return
|
||||
yield '{'
|
||||
for (const [index, [key, value]] of [...pairs].sort(([left], [right]) => keyOrder(left, right)).entries()) {
|
||||
if (index > 0) yield ' '
|
||||
yield `${key}=`
|
||||
yield value
|
||||
}
|
||||
yield '}'
|
||||
}
|
||||
|
||||
export function spellDirectiveCloser(name: string): string {
|
||||
@@ -122,7 +131,18 @@ export function spellDirectiveCloser(name: string): string {
|
||||
}
|
||||
|
||||
export function spellDirectiveOpener(name: string, argument: string | undefined, attributes: string): string {
|
||||
return `${directivePrefix}${name}${argument === undefined ? '' : ` ${argument}`}${attributes === '' ? '' : ` ${attributes}`}`
|
||||
return [...directiveOpenerPieces(name, argument, attributes === '' ? [] : [attributes])].join('')
|
||||
}
|
||||
|
||||
export function* directiveOpenerPieces(name: string, argument: string | undefined, attributes: Iterable<string>): Generator<string> {
|
||||
yield `${directivePrefix}${name}`
|
||||
if (argument !== undefined) yield ` ${argument}`
|
||||
let first = true
|
||||
for (const piece of attributes) {
|
||||
if (first) yield ' '
|
||||
first = false
|
||||
yield piece
|
||||
}
|
||||
}
|
||||
|
||||
export function spellJsonAttribute(value: JsonValue): string {
|
||||
|
||||
@@ -44,7 +44,7 @@ test('refuses a value that is not an ADF document, naming the check it failed',
|
||||
assert.equal(markdown(adfToLosslessMarkdown({ type: 'doc', version: Number.NaN })), 'not-an-adf-document: an ADF document holds a version number: found NaN')
|
||||
})
|
||||
|
||||
const tooLong = 'unsupported-node-shape: the markdown would be longer than 536870888 characters, the longest string V8 holds'
|
||||
const tooLong = 'unsupported-node-shape: spelled as markdown, this would be longer than 536870888 characters, the longest string V8 holds'
|
||||
|
||||
// About 556 million characters of carry JSON: past V8's longest string, short of JavaScriptCore's and SpiderMonkey's.
|
||||
test('refuses a carry whose JSON would run past V8\'s longest string, on every engine', () => {
|
||||
@@ -52,15 +52,13 @@ test('refuses a carry whose JSON would run past V8\'s longest string, on every e
|
||||
for (let level = 0; level < 495; level += 1) value = [value]
|
||||
const result = adfToLosslessMarkdown(document({ attrs: { value }, type: 'unknownBlock' }))
|
||||
assert.equal(markdown(result), tooLong)
|
||||
assert.deepEqual(path(result), ['content', 0])
|
||||
})
|
||||
|
||||
test('refuses lines their containers would prefix past V8\'s longest string, at the block holding them', () => {
|
||||
test('refuses lines their containers would prefix past V8\'s longest string, before any container copies them', () => {
|
||||
let quote: AdfNode = { attrs: {}, content: [{ text: 'a\n'.repeat(1_100_000), type: 'text' }], type: 'codeBlock' }
|
||||
for (let level = 0; level < 245; level += 1) quote = { content: [quote], type: 'blockquote' }
|
||||
const result = adfToLosslessMarkdown(document(quote))
|
||||
assert.equal(markdown(result), tooLong)
|
||||
assert.equal(path(result).length, 2 * 246)
|
||||
})
|
||||
|
||||
test('refuses a directive whose opener and body together would pass V8\'s longest string', () => {
|
||||
@@ -73,7 +71,6 @@ test('refuses a list whose items together would pass V8\'s longest string, befor
|
||||
const item: AdfNode = { content: [{ attrs: {}, content: [{ text: 'a'.repeat(2 ** 28), type: 'text' }], type: 'codeBlock' }], type: 'listItem' }
|
||||
const result = adfToLosslessMarkdown(document({ content: [item, item], type: 'bulletList' }))
|
||||
assert.equal(markdown(result), tooLong)
|
||||
assert.deepEqual(path(result), ['content', 0, 'content', 1])
|
||||
})
|
||||
|
||||
test('refuses a pipe table whose rows their containers would prefix past V8\'s longest string', () => {
|
||||
@@ -84,11 +81,21 @@ test('refuses a pipe table whose rows their containers would prefix past V8\'s l
|
||||
assert.equal(markdown(adfToLosslessMarkdown(document(quote))), tooLong)
|
||||
})
|
||||
|
||||
test('refuses a code block whose fences together would pass V8\'s longest string', () => {
|
||||
const long: AdfNode = { text: 'a'.repeat(2 ** 28), type: 'text' }
|
||||
assert.equal(markdown(adfToLosslessMarkdown(document({ attrs: {}, content: [long, long, long], type: 'codeBlock' }))), tooLong)
|
||||
})
|
||||
|
||||
test('refuses a directive opener whose attributes together would pass V8\'s longest string', () => {
|
||||
const long = 'a'.repeat(2 ** 28)
|
||||
const panel: AdfNode = { attrs: { panelColor: long, panelIcon: long, panelIconText: long, panelType: 'custom' }, content: [paragraph({ text: 'x', type: 'text' })], type: 'panel' }
|
||||
assert.equal(markdown(adfToLosslessMarkdown(document(panel))), tooLong)
|
||||
})
|
||||
|
||||
test('refuses text past V8\'s longest string before spelling it', () => {
|
||||
const long = 'a'.repeat(2 ** 28)
|
||||
const result = adfToLosslessMarkdown(document(paragraph({ text: long, type: 'text' }, { text: long, type: 'text' }, { text: long, type: 'text' })))
|
||||
assert.equal(markdown(result), tooLong)
|
||||
assert.deepEqual(path(result), ['content', 0])
|
||||
})
|
||||
|
||||
test('refuses a document version the markdown cannot carry', () => {
|
||||
|
||||
@@ -7,12 +7,12 @@ import { blockDirectiveForm, documentSpelling } from '../block-directive.ts'
|
||||
import { blockNodeModel, blockNodes, isTaskItem } from '../../adf/block-nodes.ts'
|
||||
import { carriedBlock } from '../opaque-carry.ts'
|
||||
import { emitInlineLine } from './inline-line.ts'
|
||||
import { failure, faulted, success, withinEngineLimits, type ConvertErrorPath, type Result } from '../../result.ts'
|
||||
import { fencedCodeBlock } from '../commonmark/backtick-runs.ts'
|
||||
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
|
||||
import { fencePieces } from '../commonmark/backtick-runs.ts'
|
||||
import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts'
|
||||
import { boundedThen, joinedWithin, longestMarkdown, separatedBy, tooLong, withinEngineLimits } from '../length-bound.ts'
|
||||
import { languageSlot, type LanguageSlot } from '../code-language.ts'
|
||||
import { largestNesting } from '../../nesting.ts'
|
||||
import { longestOutput, tooLong } from '../../output-length.ts'
|
||||
import { spellBlockDirectiveOpener } from './block-directive-spelling.ts'
|
||||
import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts'
|
||||
import { tryImage } from './image.ts'
|
||||
@@ -24,8 +24,7 @@ type Delimiters = readonly [string, string]
|
||||
type EmittedBlock = SpelledBlock & { headroom: number }
|
||||
type KeptSpelling = { block: EmittedBlock | undefined; depth: number }
|
||||
type PlacedBlock = SpelledBlock & { node: AdfNode }
|
||||
// projected: the blocks' length once joined and their containers prefix their lines; it never undercounts.
|
||||
type PlacedBlocks = { blocks: readonly PlacedBlock[]; headroom: number; projected: number }
|
||||
type PlacedBlocks = { blocks: readonly PlacedBlock[]; headroom: number }
|
||||
// CommonMark starts a new list where the marker changes, so every second list in a run of one kind takes `alternateText`.
|
||||
type SpelledBlock = { spelling: 'commonmark' | 'directive'; text: string } | { alternateText: string; spelling: 'list'; text: string }
|
||||
// Keyed by reference: only a caller building one object per position (the parse, the portable reduction) passes one; a consumer's document may share a node.
|
||||
@@ -35,12 +34,12 @@ type WalkedItem = { node: AdfNode; walk: PlacedBlocks }
|
||||
export type Writing = { flavour: WrittenFlavour; indent: number; memo: SpellingMemo | undefined }
|
||||
|
||||
const bulletDelimiters: Delimiters = ['-', '*']
|
||||
const bulletWidth = 2
|
||||
const bulletWidth = '- '.length
|
||||
export const largestListMarker = 999999999
|
||||
// Bare because tryList admits no item carrying attributes, marks or text.
|
||||
const listItemOpener = spellDirectiveOpener('listItem', undefined, '')
|
||||
const listItemCloser = spellDirectiveCloser('listItem')
|
||||
const orderedDelimiters: Delimiters = ['.', ')']
|
||||
const quoteWidth = '> '.length
|
||||
|
||||
export function adfToLosslessMarkdown(document: AdfDocument): Result<string> {
|
||||
return withinEngineLimits(() => writeMarkdown(document, 'lossless'))
|
||||
@@ -53,29 +52,25 @@ export function writeMarkdown(document: AdfDocument, flavour: WrittenFlavour): R
|
||||
if (document.content === undefined) return success(`${documentSpelling}\n`)
|
||||
const walk = walkBlocks(document.content, [], 0, { flavour, indent: 0, memo: undefined })
|
||||
if (!walk.ok) return walk
|
||||
const text = joinBlocks(walk.value.blocks, 'document')
|
||||
return success(text === '' ? '' : `${text}\n`)
|
||||
const text = joinedWithin(walk.value.blocks.length === 0 ? [] : [...blockPieces(walk.value.blocks, 'document'), '\n'])
|
||||
return boundedThen(text, success)
|
||||
}
|
||||
|
||||
function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> {
|
||||
let headroom = largestNesting - depth
|
||||
if (headroom < 0) return tooDeep(path)
|
||||
const blocks: PlacedBlock[] = []
|
||||
let projected = 0
|
||||
// A forecast, so a deep subtree refuses where its containers would prefix its lines past the bound, before any of them copies those lines.
|
||||
let forecast = 0
|
||||
for (const [index, node] of nodes.entries()) {
|
||||
const block = emitBlock(node, [...path, 'content', index], depth, writing)
|
||||
if (!block.ok) return block
|
||||
projected += (index === 0 ? 0 : separatorLength(writing)) + prefixedLength(block.value.text, writing.indent) + 1
|
||||
if (projected > longestOutput) return tooLong([...path, 'content', index])
|
||||
forecast += prefixedLength(block.value.text, writing.indent)
|
||||
if (forecast > longestMarkdown) return tooLong()
|
||||
headroom = Math.min(headroom, block.value.headroom)
|
||||
blocks.push({ ...block.value, node })
|
||||
}
|
||||
return success({ blocks, headroom, projected })
|
||||
}
|
||||
|
||||
// A block counts the newline ending it; a blank line between two is prefixed as their lines are.
|
||||
function separatorLength(writing: Writing): number {
|
||||
return 1 + writing.indent
|
||||
return success({ blocks, headroom })
|
||||
}
|
||||
|
||||
function prefixedLength(text: string, indent: number): number {
|
||||
@@ -92,16 +87,18 @@ function tooDeep(path: ConvertErrorPath): Result<never> {
|
||||
return failure('unsupported-nesting-depth', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path)
|
||||
}
|
||||
|
||||
function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): string {
|
||||
let text = ''
|
||||
function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): string | undefined {
|
||||
return joinedWithin(blockPieces(blocks, container))
|
||||
}
|
||||
|
||||
function* blockPieces(blocks: readonly PlacedBlock[], container: BlockContainer): Generator<string> {
|
||||
let alternated = false
|
||||
for (const [index, block] of blocks.entries()) {
|
||||
const previous = blocks[index - 1]
|
||||
if (previous !== undefined) text += separationBetween(previous, block, container)
|
||||
if (previous !== undefined) yield separationBetween(previous, block, container)
|
||||
alternated = !alternated && previous !== undefined && sharesMarker(previous, block)
|
||||
text += alternated && block.spelling === 'list' ? block.alternateText : block.text
|
||||
yield alternated && block.spelling === 'list' ? block.alternateText : block.text
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
// A task list is spelled as a bullet list.
|
||||
@@ -156,13 +153,13 @@ function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
|
||||
if (portable !== undefined) return portable
|
||||
if (node.type === 'blockquote') return tryBlockquote(node, path, depth, writing)
|
||||
if (node.type === 'bulletList' || node.type === 'orderedList') return tryList(node, path, depth, writing)
|
||||
if (node.type === 'codeBlock') return readableText(tryCodeBlock(node, writing.flavour))
|
||||
if (node.type === 'codeBlock') return tryCodeBlock(node, writing.flavour)
|
||||
if (node.type === 'heading') return tryHeading(node, path, writing.flavour)
|
||||
if (node.type === 'mediaSingle') return readableText(tryImage(node))
|
||||
if (node.type === 'paragraph') return tryParagraph(node, path, writing.flavour)
|
||||
if (node.type === 'rule') return readableText(tryRule(node))
|
||||
if (node.type === 'table') {
|
||||
const table = tryPipeTable(node, path, writing)
|
||||
const table = tryPipeTable(node, path, writing.flavour)
|
||||
return table === undefined || !table.ok ? table : success(commonMarkText(table.value))
|
||||
}
|
||||
return undefined
|
||||
@@ -180,27 +177,31 @@ function spellPortableBlock(node: AdfNode, path: ConvertErrorPath, depth: number
|
||||
}
|
||||
|
||||
function quotedUnder(head: string, node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
|
||||
const quotedWriting = indented(writing, quoteWidth)
|
||||
const inner = walkBlocks(nodeContent(node), path, depth + 1, quotedWriting)
|
||||
const inner = walkBlocks(nodeContent(node), path, depth + 1, indented(writing, quoteWidth))
|
||||
if (!inner.ok) return inner
|
||||
if (prefixedLength(head, quotedWriting.indent) + 1 + separatorLength(quotedWriting) + inner.value.projected > longestOutput) return tooLong(path)
|
||||
const body = joinBlocks(inner.value.blocks, 'document')
|
||||
return success(commonMarkText(quoted(body === '' ? head : `${head}\n\n${body}`), inner.value.headroom))
|
||||
const text = joinedWithin(inner.value.blocks.length === 0 ? [head] : [head, '\n\n', ...blockPieces(inner.value.blocks, 'document')])
|
||||
return quotedBlock(text, inner.value.headroom)
|
||||
}
|
||||
|
||||
const quoteWidth = 2
|
||||
function quotedBlock(text: string | undefined, headroom: number): Result<EmittedBlock> {
|
||||
return boundedThen(text, (whole) => boundedThen(joinedWithin(quotedPieces(whole)), (quotedText) => success(commonMarkText(quotedText, headroom))))
|
||||
}
|
||||
|
||||
function quoted(text: string): string {
|
||||
return text
|
||||
.split('\n')
|
||||
.map((line) => (line === '' ? '>' : `> ${line}`))
|
||||
.join('\n')
|
||||
|
||||
function* quotedPieces(text: string): Generator<string> {
|
||||
for (let start = 0; ; ) {
|
||||
const end = text.indexOf('\n', start)
|
||||
const line = text.slice(start, end === -1 ? text.length : end)
|
||||
yield line === '' ? '>' : '> '
|
||||
yield line
|
||||
if (end === -1) return
|
||||
yield '\n'
|
||||
start = end + 1
|
||||
}
|
||||
}
|
||||
|
||||
function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
|
||||
const items: PlacedBlock[][] = []
|
||||
// Each walk alone stayed within the bound, so the join is checked before it builds.
|
||||
let projected = 0
|
||||
let headroom = largestNesting - depth - 1
|
||||
// A child other than a task nests in the task before it.
|
||||
for (const [index, child] of nodeContent(node).entries()) {
|
||||
@@ -208,26 +209,31 @@ function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writi
|
||||
const itemWriting = indented(writing, bulletWidth)
|
||||
const walk = task ? taskBlocks(child, [...path, 'content', index], depth + 1, itemWriting) : placedBlock(child, [...path, 'content', index], depth + 1, itemWriting)
|
||||
if (!walk.ok) return walk
|
||||
projected += (index === 0 ? 0 : separatorLength(writing)) + walk.value.projected
|
||||
if (projected > longestOutput) return tooLong([...path, 'content', index])
|
||||
headroom = Math.min(headroom, walk.value.headroom)
|
||||
const previous = items.at(-1)
|
||||
if (task || previous === undefined) items.push([...walk.value.blocks])
|
||||
else for (const block of walk.value.blocks) previous.push(block)
|
||||
}
|
||||
const list = spelledList(
|
||||
items.map((blocks) => joinBlocks(blocks, 'list-item')),
|
||||
() => '',
|
||||
bulletDelimiters,
|
||||
headroom,
|
||||
)
|
||||
// The portable reduction leaves no task a list item cannot hold: a directive here would break the flavour.
|
||||
return list === undefined ? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path) : success(list)
|
||||
return boundedThen(joinedItems(items.map((blocks) => blockPieces(blocks, 'list-item'))), (inners) => spelledList(inners, () => '', bulletDelimiters, headroom) ?? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path))
|
||||
}
|
||||
|
||||
// Each item's text, or `undefined` once all of them together pass the bound; each alone stayed within the forecast.
|
||||
function joinedItems(items: readonly Iterable<string>[]): string[] | undefined {
|
||||
const inners: string[] = []
|
||||
let length = 0
|
||||
for (const pieces of items) {
|
||||
const inner = [...pieces].join('')
|
||||
length += inner.length
|
||||
if (length > longestMarkdown) return undefined
|
||||
inners.push(inner)
|
||||
}
|
||||
return inners
|
||||
}
|
||||
|
||||
function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<PlacedBlocks> {
|
||||
const block = emitBlock(node, path, depth, writing)
|
||||
return block.ok ? success({ blocks: [{ ...block.value, node }], headroom: block.value.headroom, projected: prefixedLength(block.value.text, writing.indent) + 1 }) : block
|
||||
return block.ok ? success({ blocks: [{ ...block.value, node }], headroom: block.value.headroom }) : block
|
||||
}
|
||||
|
||||
// The marker leads the first paragraph, or stands as one where the blocks open with another.
|
||||
@@ -238,15 +244,15 @@ function taskBlocks(task: AdfNode, path: ConvertErrorPath, depth: number, writin
|
||||
const content = nodeContent(task)
|
||||
const line = content.length === 0 ? success('') : emitInlineLine(content, 'paragraph', path, writing.flavour)
|
||||
if (!line.ok) return line
|
||||
const text = line.value === '' ? marker : `${marker} ${line.value}`
|
||||
return success({ blocks: [{ ...markerBlock, text }], headroom: Number.POSITIVE_INFINITY, projected: prefixedLength(text, writing.indent) + 1 })
|
||||
const text = joinedWithin(line.value === '' ? [marker] : [marker, ' ', line.value])
|
||||
return boundedThen(text, (spelled) => success({ blocks: [{ ...markerBlock, text: spelled }], headroom: Number.POSITIVE_INFINITY }))
|
||||
}
|
||||
const walk = walkBlocks(nodeContent(task), path, depth, writing)
|
||||
if (!walk.ok) return walk
|
||||
const [first, ...rest] = walk.value.blocks
|
||||
const blocks = first?.node.type === 'paragraph' ? [{ ...first, text: `${marker} ${first.text}` }, ...rest] : [markerBlock, ...walk.value.blocks]
|
||||
// The marker either leads the first line or stands as a block of its own.
|
||||
return success({ blocks, headroom: walk.value.headroom, projected: walk.value.projected + marker.length + separatorLength(writing) + writing.indent + 1 })
|
||||
if (first?.node.type !== 'paragraph') return success({ blocks: [markerBlock, ...walk.value.blocks], headroom: walk.value.headroom })
|
||||
const text = joinedWithin([marker, ' ', first.text])
|
||||
return boundedThen(text, (spelled) => success({ blocks: [{ ...first, text: spelled }, ...rest], headroom: walk.value.headroom }))
|
||||
}
|
||||
|
||||
function readableText(text: string | undefined): Result<EmittedBlock> | undefined {
|
||||
@@ -262,8 +268,10 @@ function commonMarkText(text: string, headroom: number = Number.POSITIVE_INFINIT
|
||||
return { headroom, spelling: 'commonmark', text }
|
||||
}
|
||||
|
||||
function directivePair(node: AdfNode, opener: string, body: string, headroom: number = Number.POSITIVE_INFINITY): EmittedBlock {
|
||||
return { headroom, spelling: 'directive', text: `${opener}\n${body === '' ? '' : `${body}\n`}${spellDirectiveCloser(node.type)}` }
|
||||
function directivePair(node: AdfNode, opener: string, body: string, headroom: number = Number.POSITIVE_INFINITY): Result<EmittedBlock> {
|
||||
const closer = spellDirectiveCloser(node.type)
|
||||
const text = joinedWithin(body === '' ? [opener, '\n', closer] : [opener, '\n', body, '\n', closer])
|
||||
return boundedThen(text, (spelled) => success({ headroom, spelling: 'directive', text: spelled }))
|
||||
}
|
||||
|
||||
function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number, walkBody: () => Result<PlacedBlocks>): Result<EmittedBlock> {
|
||||
@@ -281,12 +289,12 @@ function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string,
|
||||
if (model.contentModel === 'inline') {
|
||||
const line = emitInlineLine(nodeContent(node), 'paragraph', path, 'lossless')
|
||||
if (!line.ok) return line
|
||||
return success(directivePair(node, opener, line.value))
|
||||
return directivePair(node, opener, line.value)
|
||||
}
|
||||
const walk = walkBody()
|
||||
if (!walk.ok) return walk
|
||||
if (opener.length + walk.value.projected > longestOutput) return tooLong(path)
|
||||
return success(directivePair(node, opener, joinBlocks(walk.value.blocks, 'directive'), walk.value.headroom))
|
||||
const body = joinBlocks(walk.value.blocks, 'directive')
|
||||
return boundedThen(body, (spelled) => directivePair(node, opener, spelled, walk.value.headroom))
|
||||
}
|
||||
|
||||
function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
|
||||
@@ -294,17 +302,18 @@ function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, wri
|
||||
const inner = walkBlocks(nodeContent(node), path, depth + 1, indented(writing, quoteWidth))
|
||||
if (!inner.ok) return inner
|
||||
const text = joinBlocks(inner.value.blocks, 'document')
|
||||
const alert = flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined
|
||||
return success(commonMarkText(quoted(alert ? `\\${text}` : text), inner.value.headroom))
|
||||
const alert = text !== undefined && flavourClaims[writing.flavour].alerts && leadingMarker(text, readAlertMarker) !== undefined
|
||||
return quotedBlock(alert ? joinedWithin(['\\', text]) : text, inner.value.headroom)
|
||||
}
|
||||
|
||||
function tryCodeBlock(node: AdfNode, flavour: WrittenFlavour): string | undefined {
|
||||
function tryCodeBlock(node: AdfNode, flavour: WrittenFlavour): Result<EmittedBlock> | undefined {
|
||||
if (!holdsOnlyAttributes(node, ['language'])) return undefined
|
||||
const slot = languageSlot(nodeAttrs(node)['language'], flavourClaims[flavour])
|
||||
if (slot.kind === 'attribute') return undefined
|
||||
const [only, ...others] = fencedTexts(node) ?? []
|
||||
if (only === undefined || others.length > 0) return undefined
|
||||
return fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', only)
|
||||
const text = joinedWithin(fencePieces(slot.kind === 'fence' ? slot.info : '', only))
|
||||
return boundedThen(text, (spelled) => success(commonMarkText(spelled)))
|
||||
}
|
||||
|
||||
// spec/flavour.md, The CommonMark blocks: one fence per text node, the language on each; with no fence to carry it, the attribute does.
|
||||
@@ -316,7 +325,8 @@ function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertEr
|
||||
if (opener === undefined) return carriedFence(carriedBlock(node, path, depth))
|
||||
if (!opener.ok) return opener
|
||||
const info = slot.kind === 'fence' ? slot.info : ''
|
||||
return success(directivePair(node, opener.value, texts.map((text) => fencedCodeBlock(info, text)).join('\n')))
|
||||
const body = joinedWithin(separatedBy('\n', texts.map((text) => fencePieces(info, text))))
|
||||
return boundedThen(body, (spelled) => directivePair(node, opener.value, spelled))
|
||||
}
|
||||
|
||||
// The text of each fence, or `undefined` where a child holds what no fence keeps, so the block carry takes the node.
|
||||
@@ -350,8 +360,6 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing:
|
||||
if (start === undefined || items.length === 0) return undefined
|
||||
if (items.some((item) => item.type !== 'listItem' || !holdsOnlyAttributes(item, []))) return undefined
|
||||
const walked: WalkedItem[] = []
|
||||
// Each walk alone stayed within the bound, so the join is checked before it builds; the directive form adds an opener and a closer line to each item.
|
||||
let projected = 0
|
||||
let headroom = Number.POSITIVE_INFINITY
|
||||
const itemWriting = indented(writing, ordered ? `${start + items.length - 1}. `.length : bulletWidth)
|
||||
for (const [offset, item] of items.entries()) {
|
||||
@@ -359,43 +367,45 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing:
|
||||
if (!walk.ok) return walk
|
||||
headroom = Math.min(headroom, walk.value.headroom)
|
||||
walked.push({ node: item, walk: walk.value })
|
||||
projected += (offset === 0 ? 0 : separatorLength(writing)) + walk.value.projected + directiveItemLength(writing)
|
||||
if (projected > longestOutput) return tooLong([...path, 'content', offset])
|
||||
}
|
||||
const inners = walked.map((item) => {
|
||||
const inner = joinBlocks(item.walk.blocks, 'list-item')
|
||||
return boundedThen(joinedItems(walked.map((item) => blockPieces(item.walk.blocks, 'list-item'))), (joined) => {
|
||||
// GitHub reads a task marker opening any item's first paragraph as a checkbox, whatever its siblings hold.
|
||||
return flavourClaims[writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined ? `\\${inner}` : inner
|
||||
const inners = joined.map((inner) => (flavourClaims[writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined ? `\\${inner}` : inner))
|
||||
const list = ordered ? spelledList(inners, (offset) => `${start + offset}`, orderedDelimiters, headroom) : spelledList(inners, () => '', bulletDelimiters, headroom)
|
||||
if (list !== undefined) return list
|
||||
// The directive form spends a level the walk did not count.
|
||||
if (headroom < 1) return tooDeep(path)
|
||||
return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, path, depth, () => directiveItems(walked, headroom - 1))
|
||||
})
|
||||
const list = ordered ? spelledList(inners, (offset) => `${start + offset}`, orderedDelimiters, headroom) : spelledList(inners, () => '', bulletDelimiters, headroom)
|
||||
if (list !== undefined) return success(list)
|
||||
// The directive form spends a level the walk did not count.
|
||||
if (headroom < 1) return tooDeep(path)
|
||||
return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, path, depth, () => success({ blocks: directiveItems(walked), headroom: headroom - 1, projected }))
|
||||
}
|
||||
|
||||
function spelledList(inners: readonly string[], number: (offset: number) => string, [usual, alternate]: Delimiters, headroom: number): EmittedBlock | undefined {
|
||||
const text = listLines(inners, number, usual)
|
||||
const alternateText = listLines(inners, number, alternate)
|
||||
return text === undefined || alternateText === undefined ? undefined : { alternateText, headroom, spelling: 'list', text }
|
||||
function spelledList(inners: readonly string[], number: (offset: number) => string, [usual, alternate]: Delimiters, headroom: number): Result<EmittedBlock> | undefined {
|
||||
const pieces = listLines(inners, number, usual)
|
||||
const alternatePieces = listLines(inners, number, alternate)
|
||||
if (pieces === undefined || alternatePieces === undefined) return undefined
|
||||
// The alternate delimiter is as wide as the usual one, so its text is as long.
|
||||
return boundedThen(joinedWithin(pieces), (text) => success({ alternateText: [...alternatePieces].join(''), headroom, spelling: 'list', text }))
|
||||
}
|
||||
|
||||
function listLines(inners: readonly string[], number: (offset: number) => string, delimiter: string): string | undefined {
|
||||
const lines: string[] = []
|
||||
// `undefined` where an item holds a line no list item spells.
|
||||
function listLines(inners: readonly string[], number: (offset: number) => string, delimiter: string): Iterable<string> | undefined {
|
||||
const items: Iterable<string>[] = []
|
||||
for (const [offset, inner] of inners.entries()) {
|
||||
const line = tryListItemLines(inner, `${number(offset)}${delimiter} `)
|
||||
if (line === undefined) return undefined
|
||||
lines.push(line)
|
||||
const pieces = listItemPieces(inner, `${number(offset)}${delimiter} `)
|
||||
if (pieces === undefined) return undefined
|
||||
items.push(pieces)
|
||||
}
|
||||
return lines.join('\n')
|
||||
return separatedBy('\n', items)
|
||||
}
|
||||
|
||||
function directiveItemLength(writing: Writing): number {
|
||||
return listItemOpener.length + listItemCloser.length + 2 * (1 + writing.indent)
|
||||
}
|
||||
|
||||
function directiveItems(items: readonly WalkedItem[]): PlacedBlock[] {
|
||||
return items.map((item) => ({ ...directivePair(item.node, listItemOpener, joinBlocks(item.walk.blocks, 'directive')), node: item.node }))
|
||||
function directiveItems(items: readonly WalkedItem[], headroom: number): Result<PlacedBlocks> {
|
||||
const blocks: PlacedBlock[] = []
|
||||
for (const item of items) {
|
||||
const pair = boundedThen(joinBlocks(item.walk.blocks, 'directive'), (body) => directivePair(item.node, listItemOpener, body))
|
||||
if (!pair.ok) return pair
|
||||
blocks.push({ ...pair.value, node: item.node })
|
||||
}
|
||||
return success({ blocks, headroom })
|
||||
}
|
||||
|
||||
function listStart(node: AdfNode, items: number): number | undefined {
|
||||
@@ -405,16 +415,25 @@ function listStart(node: AdfNode, items: number): number | undefined {
|
||||
return start + items - 1 > largestListMarker ? undefined : start
|
||||
}
|
||||
|
||||
function tryListItemLines(inner: string, marker: string): string | undefined {
|
||||
if (inner === '') return marker.trimEnd()
|
||||
const body = inner.split('\n')
|
||||
if (body.some((line) => line !== '' && isBlankLine(line))) return undefined
|
||||
const indent = ' '.repeat(marker.length)
|
||||
const lines = body.map((line, index) => (index === 0 ? `${marker}${line}` : line === '' ? '' : `${indent}${line}`))
|
||||
const first = lines[0] ?? ''
|
||||
if (!isThematicBreak(first)) return lines.join('\n')
|
||||
function listItemPieces(inner: string, marker: string): Iterable<string> | undefined {
|
||||
if (inner === '') return [marker.trimEnd()]
|
||||
const [first = '', ...rest] = inner.split('\n')
|
||||
if (rest.some((line) => line !== '' && isBlankLine(line)) || (first !== '' && isBlankLine(first))) return undefined
|
||||
// An item opens with `*` only as text, a list there taking `- ` and a rule `---`; a backslash keeps a first line reading as a rule text.
|
||||
return inner.startsWith('*') ? [`${marker}\\${first.slice(marker.length)}`, ...lines.slice(1)].join('\n') : undefined
|
||||
const rule = isThematicBreak(`${marker}${first}`)
|
||||
if (rule && !inner.startsWith('*')) return undefined
|
||||
return itemLinePieces(rule ? `${marker}\\` : marker, first, rest, ' '.repeat(marker.length))
|
||||
}
|
||||
|
||||
function* itemLinePieces(opening: string, first: string, rest: readonly string[], indent: string): Generator<string> {
|
||||
yield opening
|
||||
yield first
|
||||
for (const line of rest) {
|
||||
yield '\n'
|
||||
if (line === '') continue
|
||||
yield indent
|
||||
yield line
|
||||
}
|
||||
}
|
||||
|
||||
function tryParagraph(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedBlock> | undefined {
|
||||
|
||||
@@ -3,7 +3,8 @@ import type { BlockNodeModel } from '../../adf/block-nodes.ts'
|
||||
import { attributeNestingMessage, nodeAttrs, nodeMarks } from '../../adf/document.ts'
|
||||
import { blockArgument, markValues, marksAttribute } from '../block-directive.ts'
|
||||
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
|
||||
import { isBareToken, spellAttributes, spellDirectiveOpener, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts'
|
||||
import { joinedWithin, tooLong } from '../length-bound.ts'
|
||||
import { attributePieces, directiveOpenerPieces, isBareToken, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts'
|
||||
import { overNested } from '../../json-value.ts'
|
||||
import { spellEmptyKeys } from '../empty-keys.ts'
|
||||
import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts'
|
||||
@@ -22,7 +23,8 @@ export function spellBlockDirectiveOpener(node: AdfNode, model: BlockNodeModel,
|
||||
if (overNested(values)) return failure('unsupported-nesting-depth', attributeNestingMessage(marksAttribute, node.type), path)
|
||||
spelledPairs.push([marksAttribute, spellJsonAttribute(values)])
|
||||
}
|
||||
return success(spellDirectiveOpener(node.type, slot.argument, spellAttributes(spelledPairs)))
|
||||
const opener = joinedWithin(directiveOpenerPieces(node.type, slot.argument, attributePieces(spelledPairs)))
|
||||
return opener === undefined ? tooLong() : success(opener)
|
||||
}
|
||||
|
||||
// `undefined` where the argument slot holds a value no bare token spells.
|
||||
|
||||
@@ -9,13 +9,13 @@ import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../
|
||||
import { escapeUnbalanced, spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts'
|
||||
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
|
||||
import { flavourClaims, highlightDelimiter } from '../portable/conventions.ts'
|
||||
import { holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts'
|
||||
import { heldTextLength, identicalMark, isSpellableText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
|
||||
import { holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts'
|
||||
import { inlineNodeModel } from '../../adf/inline-nodes.ts'
|
||||
import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts'
|
||||
import { largestNesting } from '../../nesting.ts'
|
||||
import { longestBacktickRun } from '../commonmark/backtick-runs.ts'
|
||||
import { longestOutput, tooLong } from '../../output-length.ts'
|
||||
import { longestMarkdown, tooLong } from '../length-bound.ts'
|
||||
import { slotFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts'
|
||||
import { spellInlineNodeAttributes } from './inline-directive-spelling.ts'
|
||||
import { spellTextDirective } from '../text-directive.ts'
|
||||
@@ -81,7 +81,7 @@ export function tryImageLine(alt: string | undefined, url: string, link?: LinkDe
|
||||
}
|
||||
|
||||
function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: WrittenFlavour): Result<EmittedLine> {
|
||||
if (heldTextLength(nodes) > longestOutput) return tooLong(path)
|
||||
if (heldTextLength(nodes) > longestMarkdown) return tooLong()
|
||||
const fallbacks: LineFallbacks = { carried: new Set(), flavour, openingLinkAsDirective: false }
|
||||
// Terminates because takeFallback refuses a pass that took no new fallback.
|
||||
for (;;) {
|
||||
|
||||
@@ -1,33 +1,38 @@
|
||||
import type { AdfNode } from '../../adf/document.ts'
|
||||
import type { WrittenFlavour } from '../portable/conventions.ts'
|
||||
import { delimiterCells, pipeRowPieces } from '../pipe-table-syntax.ts'
|
||||
import { holdsOnlyAttributes, nodeContent } from '../../adf/document.ts'
|
||||
import { longestOutput, tooLong } from '../../output-length.ts'
|
||||
import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts'
|
||||
import { boundedThen, joinedWithin, longestMarkdown, separatedBy, tooLong } from '../length-bound.ts'
|
||||
import { success, type ConvertErrorPath, type Result } from '../../result.ts'
|
||||
import { tryPipeCell } from './inline-line.ts'
|
||||
|
||||
// indent: the characters the containers around the table prefix each of its lines with.
|
||||
export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, { flavour, indent }: { flavour: WrittenFlavour; indent: number }): Result<string> | undefined {
|
||||
export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: WrittenFlavour): Result<string> | undefined {
|
||||
const rows = pipeRows(node)
|
||||
if (rows === undefined) return undefined
|
||||
const lines: string[] = []
|
||||
// A row spells ` | ` beside each cell, and a newline and the indent after it.
|
||||
let projected = 0
|
||||
const spelled: string[][] = []
|
||||
// The cells alone must fit before the rows around them are built.
|
||||
let length = 0
|
||||
for (const [rowIndex, row] of rows.entries()) {
|
||||
const cells: string[] = []
|
||||
for (const [cellIndex, paragraph] of row.entries()) {
|
||||
const content = nodeContent(paragraph)
|
||||
const line = content.length === 0 ? '' : tryPipeCell(content, [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0], flavour)
|
||||
if (line === undefined) return undefined
|
||||
length += line.length
|
||||
if (length > longestMarkdown) return tooLong()
|
||||
cells.push(line)
|
||||
projected += line.length + 3
|
||||
}
|
||||
projected += 2 + indent + (rowIndex === 0 ? spellPipeDelimiter(cells.length).length + 1 + indent : 0)
|
||||
if (projected > longestOutput) return tooLong([...path, 'content', rowIndex])
|
||||
lines.push(spellPipeRow(cells))
|
||||
if (rowIndex === 0) lines.push(spellPipeDelimiter(cells.length))
|
||||
spelled.push(cells)
|
||||
}
|
||||
const text = joinedWithin(separatedBy('\n', tableRows(spelled)))
|
||||
return boundedThen(text, success)
|
||||
}
|
||||
|
||||
function* tableRows(rows: readonly string[][]): Generator<Iterable<string>> {
|
||||
for (const [index, cells] of rows.entries()) {
|
||||
yield pipeRowPieces(cells)
|
||||
if (index === 0) yield pipeRowPieces(delimiterCells(cells.length))
|
||||
}
|
||||
return success(lines.join('\n'))
|
||||
}
|
||||
|
||||
function pipeRows(node: AdfNode): AdfNode[][] | undefined {
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
import assert from 'node:assert/strict'
|
||||
import test from 'node:test'
|
||||
|
||||
import type { Result } from '../result.ts'
|
||||
import { boundedThen, joinedWithin, separatedBy, tooLong, withinEngineLimits } from './length-bound.ts'
|
||||
import { success } from '../result.ts'
|
||||
|
||||
function thrown(name: string, message: string): () => Result<string> {
|
||||
return () => {
|
||||
const error = new Error(message)
|
||||
error.name = name
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
test('joins pieces up to the bound, and builds nothing past it', () => {
|
||||
assert.equal(joinedWithin(separatedBy(', ', [['a', 'b'], ['c']])), 'ab, c')
|
||||
const half = 'a'.repeat(2 ** 28)
|
||||
assert.equal(joinedWithin([half, half]), undefined)
|
||||
})
|
||||
|
||||
test('refuses where a bounded builder stopped, and goes on with what it built', () => {
|
||||
assert.deepEqual(boundedThen(undefined, success), tooLong())
|
||||
assert.deepEqual(boundedThen('x', (text) => success(`${text}!`)), success('x!'))
|
||||
})
|
||||
|
||||
test('refuses a string past the bound as too long on every engine, and names any other engine limit', () => {
|
||||
assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Invalid string length')), tooLong())
|
||||
assert.deepEqual(withinEngineLimits(thrown('InternalError', 'allocation size overflow')), tooLong())
|
||||
assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Out of memory')), {
|
||||
error: { code: 'unsupported-node-shape', message: 'the output would be longer than the longest string or array this JavaScript engine builds (Out of memory)', path: [] },
|
||||
ok: false,
|
||||
})
|
||||
assert.equal(withinEngineLimits(thrown('RangeError', 'Invalid array length')).ok, false)
|
||||
assert.deepEqual(
|
||||
withinEngineLimits(() => success('x')),
|
||||
{ ok: true, value: 'x' },
|
||||
)
|
||||
})
|
||||
|
||||
test('lets a stack overflow or any other throw through', () => {
|
||||
assert.throws(() => withinEngineLimits(thrown('RangeError', 'Maximum call stack size exceeded')))
|
||||
assert.throws(() => withinEngineLimits(thrown('TypeError', 'a bug')))
|
||||
assert.throws(() =>
|
||||
withinEngineLimits(() => {
|
||||
throw 'a string'
|
||||
}),
|
||||
)
|
||||
})
|
||||
@@ -0,0 +1,52 @@
|
||||
import { failure, type Result } from '../result.ts'
|
||||
|
||||
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
|
||||
export const longestMarkdown = 2 ** 29 - 24
|
||||
|
||||
// Measured on Firefox 155, Node 24 and Bun 1.4. The first two say a string passed longestMarkdown, so they refuse as tooLong; the others name the engine's own limit.
|
||||
const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length']
|
||||
const pastEngineLimit = ['RangeError: Invalid array length', 'RangeError: Out of memory']
|
||||
|
||||
export function tooLong<T>(): Result<T> {
|
||||
return failure('unsupported-node-shape', `spelled as markdown, this would be longer than ${longestMarkdown} characters, the longest string V8 holds`, [])
|
||||
}
|
||||
|
||||
// The pieces joined, or `undefined` once their length passes longestMarkdown, so nothing past it is built.
|
||||
export function joinedWithin(pieces: Iterable<string>): string | undefined {
|
||||
const kept: string[] = []
|
||||
let length = 0
|
||||
for (const piece of pieces) {
|
||||
length += piece.length
|
||||
if (length > longestMarkdown) return undefined
|
||||
kept.push(piece)
|
||||
}
|
||||
return kept.join('')
|
||||
}
|
||||
|
||||
// The refusal where a bounded builder stopped, else what `next` makes of what it built.
|
||||
export function boundedThen<T, U>(built: T | undefined, next: (built: T) => Result<U>): Result<U> {
|
||||
return built === undefined ? tooLong() : next(built)
|
||||
}
|
||||
|
||||
export function* separatedBy(separator: string, parts: Iterable<Iterable<string>>): Generator<string> {
|
||||
let first = true
|
||||
for (const part of parts) {
|
||||
if (!first) yield separator
|
||||
first = false
|
||||
yield* part
|
||||
}
|
||||
}
|
||||
|
||||
// A builder this module does not bound can still pass an engine's longest string or array, so this wraps a whole conversion.
|
||||
export function withinEngineLimits<T>(convert: () => Result<T>): Result<T> {
|
||||
try {
|
||||
return convert()
|
||||
} catch (error) {
|
||||
// A stack overflow is a bug (docs/decisions.md §Nothing recurses unbounded), so it stays a throw.
|
||||
if (!(error instanceof Error)) throw error
|
||||
const thrown = `${error.name}: ${error.message}`
|
||||
if (pastLongestString.includes(thrown)) return tooLong()
|
||||
if (!pastEngineLimit.includes(thrown)) throw error
|
||||
return failure('unsupported-node-shape', `the output would be longer than the longest string or array this JavaScript engine builds (${error.message})`, [])
|
||||
}
|
||||
}
|
||||
@@ -4,12 +4,12 @@ import type { JsonSpelling } from '../canonical-json.ts'
|
||||
import type { JsonValue } from '../json-value.ts'
|
||||
import { directiveEscape, malformedDirective, readSoleStringAttribute, spellAttributes, spellDirectiveOpener, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts'
|
||||
import { failure, success, type ConvertErrorPath, type Result } from '../result.ts'
|
||||
import { fencedCodeBlock } from './commonmark/backtick-runs.ts'
|
||||
import { fencePieces } from './commonmark/backtick-runs.ts'
|
||||
import { infoStringCarries } from './commonmark/grammar.ts'
|
||||
import { isAdfNode } from '../adf/document.ts'
|
||||
import { isJsonValue, nestingDepth, overNested } from '../json-value.ts'
|
||||
import { boundedThen, joinedWithin, longestMarkdown } from './length-bound.ts'
|
||||
import { largestNesting } from '../nesting.ts'
|
||||
import { longestOutput, tooLong } from '../output-length.ts'
|
||||
import { parseJsonText } from '../json-text.ts'
|
||||
import { serializeCanonicalJsonWithin, spellsCanonicalJson } from '../canonical-json.ts'
|
||||
|
||||
@@ -25,7 +25,8 @@ export function carriedBlock(node: AdfNode, path: ConvertErrorPath, depth: numbe
|
||||
const named = infoStringCarries(type)
|
||||
const json = carriedJson(node, named ? untyped : node, 'two-space', path, largestNesting - depth)
|
||||
if (!json.ok) return json
|
||||
return success({ headroom: json.value.headroom, text: fencedCodeBlock(`${carryFencePrefix}${named ? type : ''}`, json.value.json) })
|
||||
const text = joinedWithin(fencePieces(`${carryFencePrefix}${named ? type : ''}`, json.value.json))
|
||||
return boundedThen(text, (fence) => success({ headroom: json.value.headroom, text: fence }))
|
||||
}
|
||||
|
||||
export function carriedInline(node: AdfNode, path: ConvertErrorPath): Result<string> {
|
||||
@@ -59,8 +60,8 @@ function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, pat
|
||||
if (!isJsonValue(spelled) || headroom < 0) {
|
||||
return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path)
|
||||
}
|
||||
const json = serializeCanonicalJsonWithin(spelled, spelling, longestOutput)
|
||||
return json === undefined ? tooLong(path) : success({ headroom, json })
|
||||
const json = serializeCanonicalJsonWithin(spelled, spelling, longestMarkdown)
|
||||
return boundedThen(json, (spelled) => success({ headroom, json: spelled }))
|
||||
}
|
||||
|
||||
// `type` is the one the fence's info string names, `undefined` for the inline carry.
|
||||
|
||||
@@ -458,7 +458,7 @@ test('refuses compact JSON in the carry fence without building its two-space spe
|
||||
|
||||
test('refuses a directive whose CommonMark spelling would pass V8\'s longest string', () => {
|
||||
const markdown = `!adf:blockquote\n${'- '.repeat(490)}a\\\n${'b\\\n'.repeat(560_000)}c\n!adf:/blockquote\n`
|
||||
assert.equal(content(losslessMarkdownToAdf(markdown)), 'unsupported-node-shape: the markdown would be longer than 536870888 characters, the longest string V8 holds')
|
||||
assert.equal(content(losslessMarkdownToAdf(markdown)), 'unsupported-node-shape: spelled as markdown, this would be longer than 536870888 characters, the longest string V8 holds')
|
||||
})
|
||||
|
||||
test('names the node JSON an opaque carry restores alone', () => {
|
||||
|
||||
@@ -20,6 +20,7 @@ import { readBlockDirectiveNode } from './directive-nodes.ts'
|
||||
import { spellStringAttribute, unsupportedNodeShape } from '../directive-syntax.ts'
|
||||
import { spellsEmpty } from '../empty-keys.ts'
|
||||
import { splitPlace, splitTaskList } from '../portable/task-list.ts'
|
||||
import { withinEngineLimits } from '../length-bound.ts'
|
||||
|
||||
type Paragraph = Extract<Block, { kind: 'paragraph' }>
|
||||
|
||||
@@ -44,6 +45,13 @@ export function portableMarkdownToAdf(markdown: string): Result<AdfDocument, Par
|
||||
}
|
||||
|
||||
function readDocument(markdown: string, claims: Claims): Result<AdfDocument, ParseError> {
|
||||
return positioned(
|
||||
withinEngineLimits(() => readMarkdown(markdown, claims)),
|
||||
{ line: 1, offset: 0 },
|
||||
)
|
||||
}
|
||||
|
||||
function readMarkdown(markdown: string, claims: Claims): Result<AdfDocument, ParseError> {
|
||||
if (typeof markdown !== 'string') return positioned(failure('not-a-string', `markdown is a string: found ${typeName(markdown)}`, []), { line: 1, offset: 0 })
|
||||
const parsed = parseBlocks(markdown, claims)
|
||||
const [only] = parsed.blocks
|
||||
|
||||
@@ -36,12 +36,17 @@ export function pipeCells(line: string): string[] | undefined {
|
||||
return claimsPipeLine(line) ? splitPipeCells(line, 1) : undefined
|
||||
}
|
||||
|
||||
export function spellPipeDelimiter(columns: number): string {
|
||||
return spellPipeRow(Array.from({ length: columns }, () => '---'))
|
||||
export function delimiterCells(columns: number): string[] {
|
||||
return Array.from({ length: columns }, () => '---')
|
||||
}
|
||||
|
||||
export function spellPipeRow(cells: readonly string[]): string {
|
||||
return `| ${cells.join(' | ')} |`
|
||||
export function* pipeRowPieces(cells: readonly string[]): Generator<string> {
|
||||
yield '|'
|
||||
for (const cell of cells) {
|
||||
yield ' '
|
||||
yield cell
|
||||
yield ' |'
|
||||
}
|
||||
}
|
||||
|
||||
function splitPipeCells(line: string, start: number): string[] {
|
||||
|
||||
@@ -73,13 +73,13 @@ test('refuses what the document guard refuses, and nothing else', () => {
|
||||
test('refuses a paragraph whose text would pass V8\'s longest string, joining none of it', () => {
|
||||
const long = 'a'.repeat(2 ** 28)
|
||||
const merged = paragraph(...Array.from({ length: 9 }, () => text(long)))
|
||||
assert.equal(portableDocument(document(merged)), 'unsupported-node-shape at /content/0')
|
||||
assert.equal(portableDocument(document(merged)), 'unsupported-node-shape at /')
|
||||
})
|
||||
|
||||
test('refuses an expand whose title and body together would pass V8\'s longest string', () => {
|
||||
let quote: AdfNode = node('expand', { title: 'a'.repeat(1_000_000) }, { attrs: {}, content: [text('a\n'.repeat(1_108_000))], type: 'codeBlock' })
|
||||
for (let level = 0; level < 240; level += 1) quote = { content: [quote], type: 'blockquote' }
|
||||
assert.equal(portableDocument(document(quote)), `unsupported-node-shape at /${Array.from({ length: 241 }, () => 'content/0').join('/')}`)
|
||||
assert.equal(portableDocument(document(quote)), 'unsupported-node-shape at /')
|
||||
})
|
||||
|
||||
test('refuses nesting past 500 levels wherever the reduction walks', () => {
|
||||
@@ -106,6 +106,12 @@ test('refuses nesting past 500 levels wherever the reduction walks', () => {
|
||||
for (const bottom of bottoms) assert.equal(lowest(bottom), 'unsupported-nesting-depth', bottom.type)
|
||||
})
|
||||
|
||||
test('names a list nested too deep for its spelling at its place in the caller\'s document', () => {
|
||||
let deep: AdfNode = bulletList(item(said('x')))
|
||||
for (let level = 0; level < largestNesting; level += 1) deep = bulletList(item(deep))
|
||||
assert.match(portableDocument(document(said('a'), deep)), /^unsupported-nesting-depth at \/content\/1\/content\/0\/content\/0\//)
|
||||
})
|
||||
|
||||
test('spells a panel as an alert in the GitHub word for its colour', () => {
|
||||
const panel = (panelType: string | undefined): string =>
|
||||
portable(node('panel', panelType === undefined ? {} : { localId: '01a0d99b-1f56-7a50-889a-f4375f09ee05', panelType }, said('Check it.')))
|
||||
|
||||
@@ -3,7 +3,7 @@ import { adfDocumentFault, nodeAttrs, nodeContent, nodeMarks } from '../../adf/d
|
||||
import { blockNodeModel, isTaskItem } from '../../adf/block-nodes.ts'
|
||||
import { centeredImage, externalMedia } from '../external-image.ts'
|
||||
import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from '../emit/adf-to-markdown.ts'
|
||||
import { failure, faulted, success, withinEngineLimits, type ConvertErrorPath, type Result } from '../../result.ts'
|
||||
import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts'
|
||||
import { flavourClaims, taskMarker } from './conventions.ts'
|
||||
import { inlineLeaves, isBlockNodeType, oneLine, portableLink, reduceInline, writableHref } from './inline-reduction.ts'
|
||||
import { inlineNodeModel } from '../../adf/inline-nodes.ts'
|
||||
@@ -12,6 +12,7 @@ import { largestNesting } from '../../nesting.ts'
|
||||
import { linkDefinition } from '../mark-spellings.ts'
|
||||
import { splitTaskList } from './task-list.ts'
|
||||
import { toEditorNormal } from './editor-normal.ts'
|
||||
import { withinEngineLimits } from '../length-bound.ts'
|
||||
|
||||
// depth: the level the node reduced stands at, counted as the emitter counts it.
|
||||
type Reduction = { depth: number; memo: SpellingMemo; path: ConvertErrorPath }
|
||||
@@ -22,6 +23,9 @@ type PlacedCell = { colspan: number; paragraph: AdfNode; rowspan: number }
|
||||
|
||||
type Placed = { index: number; loose: AdfNode[] } | { index: number; loose?: undefined; node: AdfNode }
|
||||
|
||||
// path: the caller's node the block was reduced from.
|
||||
type Sourced = { block: AdfNode; path: ConvertErrorPath }
|
||||
|
||||
const blockReducers: Readonly<Record<string, BlockReducer>> = {
|
||||
blockCard: paragraphOfNode,
|
||||
blockquote: (node, reduction) => contained({ type: 'blockquote' }, node, reduction),
|
||||
@@ -74,13 +78,14 @@ function reduceBlocks(nodes: readonly AdfNode[], reduction: Reduction): Result<A
|
||||
else if (previous?.loose !== undefined) previous.loose.push(node)
|
||||
else placed.push({ index, loose: [node] })
|
||||
}
|
||||
const blocks = concatenated(
|
||||
placed.map((entry) => {
|
||||
const at = { ...reduction, path: [...reduction.path, 'content', entry.index] }
|
||||
return entry.loose === undefined ? reduceNode(entry.node, at) : paragraphOf(entry.loose, reduction)
|
||||
}),
|
||||
)
|
||||
return blocks.ok ? portableSequence(blocks.value, reduction) : blocks
|
||||
const sourced: Sourced[] = []
|
||||
for (const entry of placed) {
|
||||
const at = { ...reduction, path: [...reduction.path, 'content', entry.index] }
|
||||
const blocks = entry.loose === undefined ? reduceNode(entry.node, at) : paragraphOf(entry.loose, reduction)
|
||||
if (!blocks.ok) return blocks
|
||||
for (const block of blocks.value) sourced.push({ block, path: at.path })
|
||||
}
|
||||
return portableSequence(sourced, reduction)
|
||||
}
|
||||
|
||||
function reduceNode(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
|
||||
@@ -91,7 +96,7 @@ function reduceNode(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
|
||||
|
||||
function reduceStanding(node: AdfNode, reduction: Reduction): Result<AdfNode[]> {
|
||||
const blocks = standsInline(node) ? paragraphOf([node], reduction) : reduceNode(node, reduction)
|
||||
return blocks.ok ? portableSequence(blocks.value, reduction) : blocks
|
||||
return blocks.ok ? portableSequence(blocks.value.map((block) => ({ block, path: reduction.path })), reduction) : blocks
|
||||
}
|
||||
|
||||
function standsInline(node: AdfNode): boolean {
|
||||
@@ -117,21 +122,22 @@ function concatenated(results: readonly Result<AdfNode[]>[]): Result<AdfNode[]>
|
||||
return success(blocks)
|
||||
}
|
||||
|
||||
// A list still taking the directive form gives way to its items' blocks; one whose spelling is refused, too deep or too long, refuses the reduction (the maintainer, 2026-10-12).
|
||||
function portableSequence(blocks: readonly AdfNode[], reduction: Reduction): Result<AdfNode[]> {
|
||||
const pending = blocks.filter((block) => block.type !== 'paragraph' || nodeContent(block).length > 0).reverse()
|
||||
// A list still taking the directive form gives way to its items' blocks; one whose spelling is refused, too deep or too long, refuses the reduction.
|
||||
function portableSequence(blocks: readonly Sourced[], reduction: Reduction): Result<AdfNode[]> {
|
||||
const pending = blocks.filter(({ block }) => block.type !== 'paragraph' || nodeContent(block).length > 0).reverse()
|
||||
const sequence: AdfNode[] = []
|
||||
for (let next = pending.pop(); next !== undefined; next = pending.pop()) {
|
||||
const block = numberedPastMarkers(next)
|
||||
const block = numberedPastMarkers(next.block)
|
||||
const listed = block.type === 'bulletList' || block.type === 'orderedList'
|
||||
const spelled = listed && block === next ? commonMarkSpelling(block, reduction.path, reduction.depth, { flavour: 'portable', indent: 0, memo: reduction.memo }) : undefined
|
||||
const spelled = listed && block === next.block ? commonMarkSpelling(block, next.path, reduction.depth, { flavour: 'portable', indent: 0, memo: reduction.memo }) : undefined
|
||||
if (spelled !== undefined && !spelled.ok) return spelled
|
||||
if (!listed || spelled !== undefined) {
|
||||
sequence.push(block)
|
||||
continue
|
||||
}
|
||||
// Back on `pending`: a replacement may take the directive form itself.
|
||||
const replacement = block === next ? nodeContent(block).flatMap(nodeContent) : [block]
|
||||
const { path } = next
|
||||
const replacement = block === next.block ? nodeContent(block).flatMap((item, index) => nodeContent(item).map((node, offset) => ({ block: node, path: [...path, 'content', index, 'content', offset] }))) : [{ block, path }]
|
||||
for (const node of replacement.reverse()) pending.push(node)
|
||||
}
|
||||
return success(sequence)
|
||||
|
||||
@@ -2,7 +2,7 @@ import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/doc
|
||||
import type { JsonValue } from '../../json-value.ts'
|
||||
import { identicalMark, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks, textLength } from '../../adf/document.ts'
|
||||
import { joinsWhenRead } from '../adjacent-text.ts'
|
||||
import { longestOutput } from '../../output-length.ts'
|
||||
import { longestMarkdown } from '../length-bound.ts'
|
||||
|
||||
type JsonContainer = JsonValue[] | { [key: string]: JsonValue }
|
||||
|
||||
@@ -11,8 +11,6 @@ type NodeHolder = { content?: AdfNode[] }
|
||||
// The editor joins two text nodes where the reader would join their editor-normal forms.
|
||||
export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean {
|
||||
if (previous.type !== 'text' || node.type !== 'text') return false
|
||||
// Text this long is refused as markdown either way; joining it first would build the string.
|
||||
if (textLength(previous) + textLength(node) > longestOutput) return false
|
||||
return joinsWhenRead(normalNode(previous), normalNode(node))
|
||||
}
|
||||
|
||||
@@ -24,7 +22,8 @@ export function toEditorNormal(document: AdfDocument): AdfDocument {
|
||||
const normal: AdfDocument = { type: document.type, version: Object.is(document.version, -0) ? 0 : document.version }
|
||||
const pending: { holder: NodeHolder; source: NodeHolder }[] = [{ holder: normal, source: document }]
|
||||
for (let entry = pending.pop(); entry !== undefined; entry = pending.pop()) {
|
||||
const content = mergeAdjacentText(nodeContent(entry.source), joinsWhenEditorNormal)
|
||||
// Text past the bound is refused as markdown either way; joining it first would build the string.
|
||||
const content = mergeAdjacentText(nodeContent(entry.source), (previous, node) => joinsWhenEditorNormal(previous, node) && textLength(previous) + textLength(node) <= longestMarkdown)
|
||||
if (content.length === 0) continue
|
||||
entry.holder.content = content.map((source) => {
|
||||
const holder = normalNode(source)
|
||||
|
||||
@@ -3,10 +3,10 @@ import type { LineContainer } from '../line-container.ts'
|
||||
import type { MarkRun } from '../emit/line-escaping.ts'
|
||||
import { blockNodeModel } from '../../adf/block-nodes.ts'
|
||||
import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts'
|
||||
import { heldTextLength, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
|
||||
import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from './editor-normal.ts'
|
||||
import { largestNesting } from '../../nesting.ts'
|
||||
import { longestOutput, tooLong } from '../../output-length.ts'
|
||||
import { heldTextLength, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts'
|
||||
import { longestMarkdown, tooLong } from '../length-bound.ts'
|
||||
import { portableLineFallback, type PortableLineFallback } from '../emit/inline-line.ts'
|
||||
import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts'
|
||||
|
||||
@@ -18,7 +18,7 @@ const keptMarks: readonly string[] = [...edgeStrippingMarks, 'code', 'link']
|
||||
export function reduceInline(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, depth: number): Result<AdfNode[]> {
|
||||
const leaves = inlineLeaves(nodes, container, path, depth)
|
||||
if (!leaves.ok) return leaves
|
||||
if (heldTextLength(leaves.value) > longestOutput) return tooLong(path)
|
||||
if (heldTextLength(leaves.value) > longestMarkdown) return tooLong()
|
||||
return spellableLine(trimmedEdges(highlightsInsideSharedMarks(trimmedEdges(leaves.value))), container, path)
|
||||
}
|
||||
|
||||
|
||||
@@ -1,8 +0,0 @@
|
||||
import { failure, type ConvertErrorPath, type Result } from './result.ts'
|
||||
|
||||
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
|
||||
export const longestOutput = 2 ** 29 - 24
|
||||
|
||||
export function tooLong<T>(path: ConvertErrorPath): Result<T> {
|
||||
return failure('unsupported-node-shape', `the markdown would be longer than ${longestOutput} characters, the longest string V8 holds`, path)
|
||||
}
|
||||
@@ -4,12 +4,6 @@ import { fileURLToPath } from 'node:url'
|
||||
import { readFileSync, readdirSync } from 'node:fs'
|
||||
import test from 'node:test'
|
||||
|
||||
import { success, withinEngineLimits, type Result } from './result.ts'
|
||||
|
||||
function code(result: Result<string>): string {
|
||||
return result.ok ? `converted ${result.value}` : result.error.code
|
||||
}
|
||||
|
||||
const sourceRoot = dirname(fileURLToPath(import.meta.url))
|
||||
const union = /export type ConvertErrorCode =\n((?:\s+\| '[a-z-]+'\n)+)/
|
||||
const declared = /'([a-z-]+)'/g
|
||||
@@ -35,36 +29,3 @@ function calledCodes(): string[] {
|
||||
test('every ConvertErrorCode is the code of a production call site, and every call site names a declared one', () => {
|
||||
assert.deepEqual(calledCodes(), declaredCodes())
|
||||
})
|
||||
|
||||
test('refuses where the engine runs out of string or array, and lets a stack overflow or any other throw through', () => {
|
||||
const overflow = new Error('allocation size overflow')
|
||||
overflow.name = 'InternalError'
|
||||
assert.deepEqual(
|
||||
withinEngineLimits(() => {
|
||||
throw overflow
|
||||
}),
|
||||
{ error: { code: 'unsupported-node-shape', message: 'the output would be longer than the longest string or array this JavaScript engine builds (allocation size overflow)', path: [] }, ok: false },
|
||||
)
|
||||
assert.equal(
|
||||
code(
|
||||
withinEngineLimits(() => {
|
||||
throw new RangeError('Invalid array length')
|
||||
}),
|
||||
),
|
||||
'unsupported-node-shape',
|
||||
)
|
||||
assert.throws(() =>
|
||||
withinEngineLimits(() => {
|
||||
throw new RangeError('Maximum call stack size exceeded')
|
||||
}),
|
||||
)
|
||||
assert.throws(() =>
|
||||
withinEngineLimits(() => {
|
||||
throw new TypeError('a bug')
|
||||
}),
|
||||
)
|
||||
assert.deepEqual(
|
||||
withinEngineLimits(() => success('x')),
|
||||
{ ok: true, value: 'x' },
|
||||
)
|
||||
})
|
||||
|
||||
@@ -41,17 +41,3 @@ export function positioned<T>(result: Result<T>, position: SourcePosition): Resu
|
||||
export function success<T>(value: T): { ok: true; value: T } {
|
||||
return { ok: true, value }
|
||||
}
|
||||
|
||||
// Past the longest string or array, measured on Firefox 155, Node 24 and Bun 1.4: SpiderMonkey throws the first, V8 the next two, JavaScriptCore the last.
|
||||
const engineLimitThrows = ['InternalError: allocation size overflow', 'RangeError: Invalid array length', 'RangeError: Invalid string length', 'RangeError: Out of memory']
|
||||
|
||||
// Any string concatenation can throw one, so this wraps a whole conversion.
|
||||
export function withinEngineLimits<T>(convert: () => Result<T>): Result<T> {
|
||||
try {
|
||||
return convert()
|
||||
} catch (error) {
|
||||
// A stack overflow is a bug (docs/decisions.md §Nothing recurses unbounded), so it stays a throw.
|
||||
if (!(error instanceof Error) || !engineLimitThrows.includes(`${error.name}: ${error.message}`)) throw error
|
||||
return failure('unsupported-node-shape', `the output would be longer than the longest string or array this JavaScript engine builds (${error.message})`, [])
|
||||
}
|
||||
}
|
||||
|
||||
@@ -81,7 +81,9 @@ The line escaper keeps an array element per character twice (`escapings`, `place
|
||||
50 bytes a character: one 20-million-character text node took 1.2 GB on Node 24, and one of 80
|
||||
million exhausted its 2.2 GB default heap. Twenty million `*` throw `RangeError: Set maximum size
|
||||
exceeded`, which `withinEngineLimits` lets through. `losslessMarkdownToAdf` reaches both through a
|
||||
directive's CommonMark ask. Found while shipping item 106, 2026-10-11.
|
||||
directive's CommonMark ask. Bound the line as the blocks are (`joinedWithin`): its segments and its
|
||||
escaped output, which today pass V8's longest string only by throwing. Found while shipping item
|
||||
106, 2026-10-11.
|
||||
|
||||
### 103. Fold `unspellable-whitespace` into `unsupported-node-shape`, so a line ending and a null character in a content slot take one code.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user