3
0
mirror of https://github.com/larvit/adf-codec.git synced 2026-11-12 11:48:29 +01:00

Answer architecture round 1: 2 🔴, 5 🟡; readableBlock alone fills and reads the spelling memo, the escaper's steps hand on what they claim rather than write a shared buffer, engine-limit names only the throws an engine returns, and the docs agree on it

This commit is contained in:
2026-10-13 17:32:36 +02:00
parent ecd695a6d1
commit 025ccc0cf8
10 changed files with 110 additions and 83 deletions
+4 -3
View File
@@ -9,9 +9,10 @@
code. `MIGRATION.md` says which call to take. code. `MIGRATION.md` says which call to take.
- **Breaking:** the markdown readers refuse a value that is not a string with the new code - **Breaking:** the markdown readers refuse a value that is not a string with the new code
`not-a-string`; `MIGRATION.md` lists what each value did before. `not-a-string`; `MIGRATION.md` lists what each value did before.
- **Breaking:** every call returns the new code `engine-limit` where the JavaScript engine runs - **Breaking:** every call returns the new code `engine-limit` where the JavaScript engine passes
short of memory or passes its longest array, and, on JavaScriptCore, its longest string. A retry its longest array, `Map` or `Set`, or finds no memory for a buffer, and, on JavaScriptCore, where
may succeed. Every other refusal stays deterministic. its heap runs short or a string passes its longest. A retry may succeed. Every other refusal stays
deterministic.
- **Breaking:** an attribute value that is not a plain object, array or JSON primitive, such as a - **Breaking:** an attribute value that is not a plain object, array or JSON primitive, such as a
`Date`, `Map`, typed array or class instance, makes `isAdfDocument` return `false` and the `Date`, `Map`, typed array or class instance, makes `isAdfDocument` return `false` and the
converters refuse it as `not-an-adf-document`. Before, the converters wrote it as `{}` or its own converters refuse it as `not-an-adf-document`. Before, the converters wrote it as `{}` or its own
+3 -3
View File
@@ -88,8 +88,8 @@ Markdown the spelling table leaves alone, which `0.2.0` reads as a different doc
### Error codes ### Error codes
`unmappable-image`, `unspellable-character`, `unspellable-line-start` and `unspellable-link` leave `unmappable-image`, `unspellable-character`, `unspellable-line-start` and `unspellable-link` leave
`ConvertErrorCode` and `engine-limit` and `not-a-string` join it: a `switch` naming one of the four `ConvertErrorCode`, and `engine-limit` and `not-a-string` join it. A `switch` naming one of the four
stops compiling, and one naming every code needs the new cases. A document `adfToMarkdown` refused with one of the stops compiling, and one naming every code needs the two new cases. A document `adfToMarkdown` refused with one of the
three `unspellable-` codes now converts in `adfToLosslessMarkdown`; markdown the parser refused three `unspellable-` codes now converts in `adfToLosslessMarkdown`; markdown the parser refused
with one of the four now reads, or refuses with the code the table gives. with one of the four now reads, or refuses with the code the table gives.
@@ -114,4 +114,4 @@ with one of the four now reads, or refuses with the code the table gives.
| `null`, `undefined` or a non-empty array, on parse | throws a `TypeError` | `not-a-string` | | `null`, `undefined` or a non-empty array, on parse | throws a `TypeError` | `not-a-string` |
| a `String` object, on parse | the document its text spells | `not-a-string`; pass `String(value)` | | a `String` object, on parse | the document its text spells | `not-a-string`; pass `String(value)` |
| any other value that is not a string, such as a number or `{}`, on parse | an empty document | `not-a-string` | | any other value that is not a string, such as a number or `{}`, on parse | an empty document | `not-a-string` |
| a conversion that runs the JavaScript engine short of memory or past its longest array | throws a `RangeError`, or the engine ends the process | `engine-limit`; retry it once | | a conversion past the JavaScript engine's longest array, `Map` or `Set`, or finding no memory for a buffer | throws a `RangeError` | `engine-limit`; retry it once |
+7 -6
View File
@@ -411,7 +411,9 @@ typed on `ConvertError` takes both.
A call reports the first refusal in document order and stops, so a document with several surfaces A call reports the first refusal in document order and stops, so a document with several surfaces
them one per call. Every refusal is deterministic — there is no I/O anywhere — so a retry returns them one per call. Every refusal is deterministic — there is no I/O anywhere — so a retry returns
the identical error: fix the input, or set the document aside. The exception is `engine-limit`, the identical error: fix the input, or set the document aside. The exception is `engine-limit`,
whose causes include memory running short: retry it once, and treat a repeat as lasting. whose causes include memory running short: retry it once, and treat a repeat as lasting. On V8
(Node, Deno, Chrome) a heap that runs out ends the process instead, so convert input of unbounded
size in a worker.
### Parsing ### Parsing
@@ -443,9 +445,9 @@ The emitter raises these, and so does a parse, which asks the emitter how to spe
| Code | Fires when | What you can do | | Code | Fires when | What you can do |
| --- | --- | --- | | --- | --- | --- |
| `engine-limit` | in every call: a conversion that passes the longest array the JavaScript engine builds or runs it short of memory, and on JavaScriptCore (Bun, Safari) one that passes its longest string, 2^31 − 1 characters | retry once; if it repeats, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply | | `engine-limit` | in every call: a conversion that passes the longest array, `Map` or `Set` the JavaScript engine builds, or that finds no memory for a buffer; on JavaScriptCore (Bun, Safari), also one that runs its heap short or passes its longest string, 2^31 − 1 characters | retry once; if it repeats, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply |
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it | | `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or one that, written out as plain CommonMark, would be longer than 536870888 characters, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536870888 characters, V8's longest string, on every engine; in `adfToPortableMarkdown`, also a document whose text alone is that long, edge whitespace included. In every call: a conversion that passes the longest string the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536870888 characters or the engine's longest string, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply | | `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or one that, written out as plain CommonMark, would be longer than 536870888 characters, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536870888 characters, V8's longest string, on every engine; in `adfToPortableMarkdown`, also a document whose text alone is that long, edge whitespace included. In every call: a conversion that passes V8's or SpiderMonkey's longest string | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536870888 characters or the engine's longest string, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply |
## The guarantees ## The guarantees
@@ -459,8 +461,7 @@ Serves [Goals](#goals) 1, 4 and 5.
- Markdown this library reads, and markdown it writes, means what the CommonMark spec says. The - Markdown this library reads, and markdown it writes, means what the CommonMark spec says. The
bullets below name every exception. bullets below name every exception.
- CommonMark is valid input to `commonMarkToAdf`, apart from the raw HTML `unmappable-html` names, - CommonMark is valid input to `commonMarkToAdf`, apart from the raw HTML `unmappable-html` names,
nesting past 500 levels, and input whose conversion passes the longest string or array the nesting past 500 levels, and input whose conversion `engine-limit` or the length bound refuses.
JavaScript engine builds.
- The other readers claim shapes CommonMark reads as text or code. `losslessMarkdownToAdf` claims - The other readers claim shapes CommonMark reads as text or code. `losslessMarkdownToAdf` claims
four: literal text matching directive, pipe-table or strikethrough syntax, and a code fence whose four: literal text matching directive, pipe-table or strikethrough syntax, and a code fence whose
info string opens `adf:`. `portableMarkdownToAdf` claims what [Portable info string opens `adf:`. `portableMarkdownToAdf` claims what [Portable
@@ -487,7 +488,7 @@ Serves [Goals](#goals) 1, 4 and 5.
the outer brackets literal text, which the spec requires and the reference itself breaks, nesting the outer brackets literal text, which the spec requires and the reference itself breaks, nesting
one `<a>` in the other. one `<a>` in the other.
- Not every document converts back: `adfToLosslessMarkdown` refuses a document nesting past 500 - Not every document converts back: `adfToLosslessMarkdown` refuses a document nesting past 500
levels, and the node shapes the `unsupported-node-shape` row lists. Show the refusal and keep the levels, the node shapes the `unsupported-node-shape` row lists, and what `engine-limit` names. Show the refusal and keep the
document read-only; saving markdown you could not produce is the loss the round-trip exists to document read-only; saving markdown you could not produce is the loss the round-trip exists to
stop. stop.
- The pipe table that `losslessMarkdownToAdf` and `portableMarkdownToAdf` read narrows GFM's twice: - The pipe table that `losslessMarkdownToAdf` and `portableMarkdownToAdf` read narrows GFM's twice:
+31
View File
@@ -680,3 +680,34 @@ Maintainability senior, hardest first:
Inherited architect, would restructure first: Inherited architect, would restructure first:
- Locality: `src/markdown/emit/adf-to-markdown.ts:126` `readableBlock`: the memo is mutated and deleted across parse, emit and portable, and the length bound is applied at five sites - Locality: `src/markdown/emit/adf-to-markdown.ts:126` `readableBlock`: the memo is mutated and deleted across parse, emit and portable, and the length bound is applied at five sites
## 2026-10-13T14:48:38Z, PR #37 at ecd695a, against 7a27f6b
Ruling: worse
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | worse | better | worse | worse |
| Inherited architect | better | worse | same | worse | worse |
Mid A, decided by:
- Locality, Self-sufficiency: `src/markdown/emit/adf-to-markdown.ts:116` `commonMarkSpelling` / `readableBlock` / `takeKeptSpelling`: only commonMarkSpelling fills the memo and a read empties it, so whether an entry exists depends on which caller ran first; the reason lives only behind a pointer to docs/decisions.md
- Locality: `src/markdown/emit/line-escaping.ts:229` `escapeClosedRuns`: both steps write one shared Uint8Array, and correctness rests on the scan order a second comment line must defend
- Navigation, Shape: `src/markdown/length-bound.ts:30` `withinEngineLimits`: the engine-limit catch moves out of result.ts into one well-named file, with string-length and engine limits apart and `engine-limit` a code of its own
- Locality: `src/markdown/emit/adf-to-markdown.ts:51` `writeMarkdown` / `walkBlocks` / `commonMarkSpelling`: the length bound is enforced in layers a reader changing it must hold together
- Shape: `src/markdown/parse/markdown-to-adf.ts:266` `directiveNode`: undefined is wrapped into a Result to pass the engine guard, then unpicked
- Shape: `src/markdown/emit/line-escaping.ts:35` `Escapings`: the type is a lookup function, its comment describes the Uint8Array behind it; `backslashed` adds 4096-piece chunking
- Shape, Locality: `src/canonical-json.ts:23` `canonicalJsonPieces`: a generator replaces a boolean-continue callback
- Shape: `src/markdown/emit/inline-line.ts:235` `emitLeaf`: the content-or-text refusal moves into one place
- Locality: `src/markdown/portable/adf-to-portable-markdown.ts:128` `portableSequence`: a spelling refusal propagates where it was swallowed, at the cost of Sourced path plumbing
Inherited architect, decided by:
- Locality, Self-sufficiency: `src/markdown/emit/adf-to-markdown.ts:139` `takeKeptSpelling`: reading deletes the entry and only commonMarkSpelling fills it, so readableBlock is right only if another file's caller asked first, once per node; explained only by a pointer to docs/decisions.md
- Locality, Shape: `src/markdown/parse/markdown-to-adf.ts:266` `directiveNode`: a second withinEngineLimits inside readDocument's, fed `?? success(undefined)`, so null and undefined carry meaning the reader must remember
- Locality: `src/markdown/emit/adf-to-markdown.ts:58` `walkBlocks`: the length bound is enforced at about six sites across three files
- Navigation: `src/markdown/length-bound.ts:30` `withinEngineLimits`: engine-limit handling sits in a file named for it, and the engine-limit code parts memory from the length bound
- Shape, Locality: `src/markdown/emit/line-escaping.ts:229` `escapeClosedRuns`: mutates its input in place and needs a two-line comment arguing it is safe; output joined in batches of 4096
- Locality, Shape: `src/markdown/portable/adf-to-portable-markdown.ts:128` `portableSequence`: a refusal propagates with its source path, where it was swallowed
- Shape: `src/markdown/emit/inline-line.ts:235` `emitLeaf`: one place refuses content and text, and a generator replaces the boolean-callback JSON writer
+5 -5
View File
@@ -467,11 +467,11 @@ handles one cause alike whichever node, attribute or direction raised it.
causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested JSON, causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested JSON,
since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in a since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in a
length linear in its JSON (2026-10-11). length linear in its JSON (2026-10-11).
- A conversion the engine stops, by running short of memory or passing its longest array, is - A conversion the engine stops with a throw of its own, past its longest array, `Map` or `Set`
`engine-limit`, at the empty path. A retry may pass it, so `unsupported-node-shape`, which never or short of memory, is `engine-limit`, at the empty path. A retry may pass the memory case, so
does, cannot hold it. JavaScriptCore throws the same `Out of memory` past its longest string, `unsupported-node-shape`, which no retry passes, cannot hold the class. JavaScriptCore throws the
which takes `engine-limit` too: telling the two apart would mean reading the engine's message same `Out of memory` past its longest string, which takes `engine-limit` too, since only the
(2026-10-13, the maintainer). message would tell the two apart (2026-10-13, the maintainer).
- The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at - The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at
position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and
`not-an-adf-document` names the emitters' input (2026-10-05). `not-an-adf-document` names the emitters' input (2026-10-05).
+12 -17
View File
@@ -108,39 +108,34 @@ function interruptsParagraph(node: AdfNode): boolean {
function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> { function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
const model = blockNodeModel(node.type) const model = blockNodeModel(node.type)
if (model === undefined) return carriedFence(carriedBlock(node, path, depth)) if (model === undefined) return carriedFence(carriedBlock(node, path, depth))
const readable = readableBlock(node, path, depth, writing) const readable = readableBlock(node, path, depth, writing, false)
if (readable !== undefined) return readable if (readable !== undefined) return readable
return emitDirectiveBlock(node, model, path, depth, () => walkBlocks(nodeContent(node), path, depth + 1, writing)) return emitDirectiveBlock(node, model, path, depth, () => walkBlocks(nodeContent(node), path, depth + 1, writing))
} }
export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing & { memo: SpellingMemo }): Result<null> | undefined { export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing & { memo: SpellingMemo }): Result<null> | undefined {
const readable = readableBlock(node, path, depth, writing) const readable = readableBlock(node, path, depth, writing, true)
if (readable === undefined) { if (readable === undefined) return undefined
writing.memo.set(node, { block: undefined, depth })
return undefined
}
if (!readable.ok) return readable if (!readable.ok) return readable
const fits = withinLongest(readable.value.text) const fits = withinLongest(readable.value.text)
if (!fits.ok) return fits if (!fits.ok) return fits
writing.memo.set(node, { block: readable.value, depth })
return readable.value.spelling === 'directive' ? undefined : success(null) return readable.value.spelling === 'directive' ? undefined : success(null)
} }
function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined { // Only an asked node's spelling is kept, and reading it drops it: the walk above spells the node into its own text, so no level of a subtree holds that text twice.
const kept = takeKeptSpelling(writing.memo, node) function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing, asked: boolean): Result<EmittedBlock> | undefined {
const { memo } = writing
const kept = memo?.get(node)
memo?.delete(node)
if (kept !== undefined) { if (kept !== undefined) {
if (kept.block === undefined) return undefined if (kept.block === undefined) return undefined
// An entry holds the spelling at its fill depth. A read at or above it rebases headroom; a read below re-spells, because a hit skips the walk's depth guards. // An entry holds the spelling at its fill depth. A read at or above it rebases headroom; a read below re-spells, because a hit skips the walk's depth guards.
if (depth <= kept.depth) return success({ ...kept.block, headroom: kept.block.headroom + kept.depth - depth }) if (depth <= kept.depth) return success({ ...kept.block, headroom: kept.block.headroom + kept.depth - depth })
} }
return spellReadableBlock(node, path, depth, writing) const spelled = spellReadableBlock(node, path, depth, writing)
} if (asked && spelled === undefined) memo?.set(node, { block: undefined, depth })
if (asked && spelled?.ok === true) memo?.set(node, { block: spelled.value, depth })
// docs/decisions.md §The spelling memo return spelled
function takeKeptSpelling(memo: SpellingMemo | undefined, node: AdfNode): KeptSpelling | undefined {
const kept = memo?.get(node)
memo?.delete(node)
return kept
} }
function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined { function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
+36 -36
View File
@@ -11,7 +11,7 @@ import { opensLinkDefinition } from '../commonmark/link-reference-definitions.ts
export type DelimiterRole = 'close' | 'open' export type DelimiterRole = 'close' | 'open'
// A segment's kind: `backslash` text, `bracketed` text inside `[…]` (link text, alt, a directive's content), `bracketed-link-target` syntax: a link target inside `[…]`, `none` syntax written as is. // A segment's kind: `backslash` text, `bracketed` text inside `[…]` (link text, alt, a directive's content), `bracketed-link-target` syntax: a link target inside `[…]`, `none` syntax written as is.
export type InlineEscaping = 'backslash' | 'bracketed' | 'bracketed-link-target' | 'none' export type InlineEscaping = (typeof escapingCodes)[number]
export type NodeRange = { first: number; last: number } export type NodeRange = { first: number; last: number }
@@ -32,9 +32,14 @@ type EmittedDelimiter = { closes: boolean; offset: number; pair: number; width:
type EmittedRun = { canClose: boolean; canOpen: boolean; character: string; delimiters: EmittedDelimiter[]; length: number; start: number } type EmittedRun = { canClose: boolean; canOpen: boolean; character: string; delimiters: EmittedDelimiter[]; length: number; start: number }
// One byte a character, never an array element or Set entry: a line may run to hundreds of millions of characters.
type Escapings = (index: number) => InlineEscaping | undefined type Escapings = (index: number) => InlineEscaping | undefined
// Where a segment's first character landed in the output.
type PlacedSegment = { segment: InlineSegment; start: number }
// The run's escapable characters, or its characters before `end`.
type RunEscapes = { end: number; escapable: false } | { escapable: true }
type EscapeWalk = { type EscapeWalk = {
readonly claims: Claims readonly claims: Claims
readonly container: LineContainer readonly container: LineContainer
@@ -44,7 +49,7 @@ type EscapeWalk = {
readonly scan: string readonly scan: string
} }
const escapingCodes: readonly InlineEscaping[] = ['backslash', 'bracketed', 'bracketed-link-target', 'none'] const escapingCodes: readonly ['backslash', 'bracketed', 'bracketed-link-target', 'none'] = ['backslash', 'bracketed', 'bracketed-link-target', 'none']
const followsLinkText = /[([]/ const followsLinkText = /[([]/
@@ -85,6 +90,7 @@ function resolveEmphasis(segments: readonly InlineSegment[]): InlineSegment[] {
function escape(segments: readonly InlineSegment[], container: LineContainer, claims: Claims): AssembledLine { function escape(segments: readonly InlineSegment[], container: LineContainer, claims: Claims): AssembledLine {
const scan = segments.map((segment) => segment.text).join('') const scan = segments.map((segment) => segment.text).join('')
// One byte a character, never an array element or Set entry: a line may run to hundreds of millions of characters.
const codes = new Uint8Array(scan.length) const codes = new Uint8Array(scan.length)
let start = 0 let start = 0
for (const segment of segments) { for (const segment of segments) {
@@ -96,16 +102,15 @@ function escape(segments: readonly InlineSegment[], container: LineContainer, cl
return code === undefined ? undefined : escapingCodes[code] return code === undefined ? undefined : escapingCodes[code]
} }
const escaped = escapeClosedRuns(scan, escapings, escapeClaims(scan, escapings, container, claims)) const escaped = escapeClosedRuns(scan, escapings, escapeClaims(scan, escapings, container, claims))
const { output, placements } = backslashed(segments, scan, escaped) const { output, placed } = backslashed(segments, scan, escaped)
if (container === 'paragraph' && opensLinkDefinition(output)) { if (container === 'paragraph' && opensLinkDefinition(output)) {
if (segments[0]?.nodes !== undefined) return { line: output, openingLinkAsDirective: true, unspellableRuns: [] } if (segments[0]?.nodes !== undefined) return { line: output, openingLinkAsDirective: true, unspellableRuns: [] }
return { line: `\\${output}`, unspellableRuns: unspellableRuns(segments, output, placements) } return { line: `\\${output}`, unspellableRuns: unspellableRuns(placed, output) }
} }
return { line: output, unspellableRuns: unspellableRuns(segments, output, placements) } return { line: output, unspellableRuns: unspellableRuns(placed, output) }
} }
// The output, and where each segment's first character landed in it. function backslashed(segments: readonly InlineSegment[], scan: string, escaped: Uint8Array): { output: string; placed: PlacedSegment[] } {
function backslashed(segments: readonly InlineSegment[], scan: string, escaped: Uint8Array): { output: string; placements: number[] } {
// Joined a few thousand pieces at a time: an array element per escape would cost more than the character it escapes. // Joined a few thousand pieces at a time: an array element per escape would cost more than the character it escapes.
const chunks: string[] = [] const chunks: string[] = []
let pieces: string[] = [] let pieces: string[] = []
@@ -119,7 +124,7 @@ function backslashed(segments: readonly InlineSegment[], scan: string, escaped:
} }
pieces.push(scan.slice(from)) pieces.push(scan.slice(from))
chunks.push(pieces.join('')) chunks.push(pieces.join(''))
const placements: number[] = [] const placed: PlacedSegment[] = []
let escapes = 0 let escapes = 0
let next = escaped.indexOf(1) let next = escaped.indexOf(1)
let start = 0 let start = 0
@@ -128,10 +133,10 @@ function backslashed(segments: readonly InlineSegment[], scan: string, escaped:
escapes += 1 escapes += 1
next = escaped.indexOf(1, next + 1) next = escaped.indexOf(1, next + 1)
} }
placements.push(start + escapes) placed.push({ segment, start: start + escapes })
start += segment.text.length start += segment.text.length
} }
return { output: chunks.join(''), placements } return { output: chunks.join(''), placed }
} }
function escapeClaims(scan: string, escapings: Escapings, container: LineContainer, claims: Claims): Uint8Array { function escapeClaims(scan: string, escapings: Escapings, container: LineContainer, claims: Claims): Uint8Array {
@@ -150,7 +155,9 @@ function escapeClaims(scan: string, escapings: Escapings, container: LineContain
while (index > line.start + line.text.length) line = scanLine(scan, line.start + line.text.length + 1) while (index > line.start + line.text.length) line = scanLine(scan, line.start + line.text.length + 1)
if (runCharacters.includes(scan.charAt(index))) { if (runCharacters.includes(scan.charAt(index))) {
const end = index + runLength(scan, index) const end = index + runLength(scan, index)
runEscapes(walk, index, end, line, escaped) const run = runEscapes(walk, index, end, line)
if (!run.escapable) escaped.fill(1, index, run.end)
else for (let at = index; at < end; at += 1) if (isEscapable(escapings(at))) escaped[at] = 1
index = end index = end
continue continue
} }
@@ -164,18 +171,13 @@ function escapeClaims(scan: string, escapings: Escapings, container: LineContain
} }
// A text character beside one of the emitter's delimiters would lengthen that delimiter's run, so every text character in such a run escapes. // A text character beside one of the emitter's delimiters would lengthen that delimiter's run, so every text character in such a run escapes.
function runEscapes(walk: EscapeWalk, start: number, end: number, line: ScanLine, escaped: Uint8Array): void { function runEscapes(walk: EscapeWalk, start: number, end: number, line: ScanLine): RunEscapes {
const { escapings, scan } = walk const { escapings, scan } = walk
if (runHolds(escapings, start, end, isSyntax) && runHolds(escapings, start, end, isEscapable)) { if (runHolds(escapings, start, end, isSyntax) && runHolds(escapings, start, end, isEscapable)) return { escapable: true }
for (let index = start; index < end; index += 1) if (isEscapable(escapings(index))) escaped[index] = 1 let index = start
return // A backtick run escapes whole: a bare rest would be a shorter backtick string, which can close a span an earlier run opens.
} while (index < end && ((index > start && scan.charAt(index) === '`') || claimsRunRest(walk, index, end, line))) index += 1
for (let index = start; index < end; index += 1) { return { end: index, escapable: false }
// A backtick run escapes whole: a bare rest would be a shorter backtick string, which can close a span an earlier run opens.
const claimed = (index > start && scan.charAt(index) === '`') || claimsRunRest(walk, index, end, line)
if (!claimed) break
escaped[index] = 1
}
} }
function runHolds(escapings: Escapings, start: number, end: number, kind: (escaping: InlineEscaping | undefined) => boolean): boolean { function runHolds(escapings: Escapings, start: number, end: number, kind: (escaping: InlineEscaping | undefined) => boolean): boolean {
@@ -225,8 +227,8 @@ function claimsCharacter({ container, escapings, headingCloser, linkClose, scan
} }
// CommonMark reads no escape inside a code span, so a backtick string left beside an escape still closes a span that an earlier bare run of its length opens. // CommonMark reads no escape inside a code span, so a backtick string left beside an escape still closes a span that an earlier bare run of its length opens.
// The walk runs backwards and escapes only the run it just read, so every index it reads still holds the claims alone. function escapeClosedRuns(scan: string, escapings: Escapings, claimed: Uint8Array): Uint8Array {
function escapeClosedRuns(scan: string, escapings: Escapings, escaped: Uint8Array): Uint8Array { const escaped = claimed.slice()
// The lengths of the backtick strings escapes leave: a bare run of one of these lengths would open a span one of them closes, so it is escaped too. // The lengths of the backtick strings escapes leave: a bare run of one of these lengths would open a span one of them closes, so it is escaped too.
const formed = new Set<number>() const formed = new Set<number>()
let end = scan.length - 1 let end = scan.length - 1
@@ -239,11 +241,11 @@ function escapeClosedRuns(scan: string, escapings: Escapings, escaped: Uint8Arra
while (scan.charAt(start - 1) === '`') start -= 1 while (scan.charAt(start - 1) === '`') start -= 1
let segmentEnd = end let segmentEnd = end
for (let index = end; index > start; index -= 1) { for (let index = end; index > start; index -= 1) {
if (escaped[index] !== 1) continue if (claimed[index] !== 1) continue
formed.add(segmentEnd - index + 1) formed.add(segmentEnd - index + 1)
segmentEnd = index - 1 segmentEnd = index - 1
} }
if (segmentEnd !== end || escaped[start] === 1) formed.add(segmentEnd - start + 1) if (segmentEnd !== end || claimed[start] === 1) formed.add(segmentEnd - start + 1)
else if (escapings(start) !== 'none' && formed.has(end - start + 1)) { else if (escapings(start) !== 'none' && formed.has(end - start + 1)) {
escaped.fill(1, start, end + 1) escaped.fill(1, start, end + 1)
// Each escaped backtick then stands alone, a string of length one. // Each escaped backtick then stands alone, a string of length one.
@@ -255,18 +257,17 @@ function escapeClosedRuns(scan: string, escapings: Escapings, escaped: Uint8Arra
} }
// One emphasis run, the innermost, or every highlight run the line cannot spell. // One emphasis run, the innermost, or every highlight run the line cannot spell.
function unspellableRuns(segments: readonly InlineSegment[], output: string, placements: readonly number[]): MarkRun[] { function unspellableRuns(placed: readonly PlacedSegment[], output: string): MarkRun[] {
const { nodes, runs } = emittedRuns(segments, placements, output) const { nodes, runs } = emittedRuns(placed, output)
const pair = misflanked(runs) ?? unpaired(runs) const pair = misflanked(runs) ?? unpaired(runs)
const run = pair === undefined ? undefined : nodes[pair] const run = pair === undefined ? undefined : nodes[pair]
return run === undefined ? unreadHighlights(segments, output, placements) : [run] return run === undefined ? unreadHighlights(placed, output) : [run]
} }
// No `==` in text can open or close, and highlights never nest, so a pair reads back where each delimiter flanks. // No `==` in text can open or close, and highlights never nest, so a pair reads back where each delimiter flanks.
function unreadHighlights(segments: readonly InlineSegment[], output: string, placements: readonly number[]): MarkRun[] { function unreadHighlights(placed: readonly PlacedSegment[], output: string): MarkRun[] {
const unread: MarkRun[] = [] const unread: MarkRun[] = []
for (const [index, segment] of segments.entries()) { for (const { segment, start } of placed) {
const start = placements[index] ?? 0
if (segment.highlight === undefined) continue if (segment.highlight === undefined) continue
const flanking = highlightFlanking(output, start) const flanking = highlightFlanking(output, start)
const flanks = segment.highlight === 'open' ? flanking.opens : flanking.closes const flanks = segment.highlight === 'open' ? flanking.opens : flanking.closes
@@ -305,12 +306,11 @@ function delimiterAt(run: EmittedRun, closes: boolean, offset: number, width: nu
return run.delimiters.find((delimiter) => delimiter.closes === closes && delimiter.offset === offset && delimiter.width === width) return run.delimiters.find((delimiter) => delimiter.closes === closes && delimiter.offset === offset && delimiter.width === width)
} }
function emittedRuns(segments: readonly InlineSegment[], placements: readonly number[], output: string): { nodes: MarkRun[]; runs: EmittedRun[] } { function emittedRuns(placed: readonly PlacedSegment[], output: string): { nodes: MarkRun[]; runs: EmittedRun[] } {
const runs: EmittedRun[] = [] const runs: EmittedRun[] = []
const nodes: MarkRun[] = [] const nodes: MarkRun[] = []
const open: number[] = [] const open: number[] = []
for (const [index, segment] of segments.entries()) { for (const { segment, start } of placed) {
const start = placements[index] ?? 0
if (segment.emphasis === undefined) continue if (segment.emphasis === undefined) continue
const closes = segment.emphasis === 'close' const closes = segment.emphasis === 'close'
const pair = closes ? (open.pop() ?? nodes.length) : nodes.length const pair = closes ? (open.pop() ?? nodes.length) : nodes.length
+3
View File
@@ -36,6 +36,9 @@ test('refuses a string past the bound as too long on every engine, and any other
} }
assert.equal(code('Invalid array length'), 'engine-limit') assert.equal(code('Invalid array length'), 'engine-limit')
assert.equal(code('Array buffer allocation failed'), 'engine-limit') assert.equal(code('Array buffer allocation failed'), 'engine-limit')
assert.equal(code('Map maximum size exceeded'), 'engine-limit')
assert.equal(code('Set maximum size exceeded'), 'engine-limit')
assert.equal(code('invalid array length'), 'engine-limit')
assert.deepEqual( assert.deepEqual(
withinEngineLimits(() => success('x')), withinEngineLimits(() => success('x')),
{ ok: true, value: 'x' }, { ok: true, value: 'x' },
+9 -2
View File
@@ -3,9 +3,16 @@ import { failure, success, type Result } from '../result.ts'
// V8's longest string (docs/decisions.md §Markdown past V8's longest string is refused on every engine). // V8's longest string (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
export const longestMarkdown = 2 ** 29 - 24 export const longestMarkdown = 2 ** 29 - 24
// Measured on Firefox 155, Node 24 and Bun 1.4. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the others name an engine's own limit or its memory running short. // Measured on Firefox 155, Node 24 and Bun 1.4, and SpiderMonkey's array length from its js.msg. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the others name an engine's own limit or its memory running short.
const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length'] const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length']
const pastEngineLimit = ['RangeError: Array buffer allocation failed', 'RangeError: Invalid array length', 'RangeError: Out of memory'] const pastEngineLimit = [
'RangeError: Array buffer allocation failed',
'RangeError: Invalid array length',
'RangeError: Map maximum size exceeded',
'RangeError: Out of memory',
'RangeError: Set maximum size exceeded',
'RangeError: invalid array length',
]
export function tooLong<T>(): Result<T> { export function tooLong<T>(): Result<T> {
return failure('unsupported-node-shape', `the markdown for this document, or for a directive's plain CommonMark, would be longer than ${longestMarkdown} characters, the longest this library writes`, []) return failure('unsupported-node-shape', `the markdown for this document, or for a directive's plain CommonMark, would be longer than ${longestMarkdown} characters, the longest this library writes`, [])
-11
View File
@@ -32,7 +32,6 @@
| 89 | 0.2.1 | | **Build a `date`, `emoji`, `inlineCard`, `mention` or `status` holding no mark Atlassian's schema withholds from it, in every reader.** | 4 | 4 | 5 | 6 | 4 | 13.0 | | 89 | 0.2.1 | | **Build a `date`, `emoji`, `inlineCard`, `mention` or `status` holding no mark Atlassian's schema withholds from it, in every reader.** | 4 | 4 | 5 | 6 | 4 | 13.0 |
| 99 | 0.2.1 | defect | **Write a linked external image standing inline or in a table cell in the spelling a reader panel picks, in `adfToPortableMarkdown`.** | 3 | 3 | 6 | 7 | 7 | 12.9 | | 99 | 0.2.1 | defect | **Write a linked external image standing inline or in a table cell in the spelling a reader panel picks, in `adfToPortableMarkdown`.** | 3 | 3 | 6 | 7 | 7 | 12.9 |
| 100 | 0.2.1 | decision | **Put the reading of an image's title as its caption back to a reader panel, now that a writer panel reads a title as hover text.** | 3 | 3 | 4 | 5 | 4, 6 | 10.9 | | 100 | 0.2.1 | decision | **Put the reading of an image's title as its caption back to a reader panel, now that a writer panel reads a title as hover text.** | 3 | 3 | 4 | 5 | 4, 6 | 10.9 |
| 110 | 0.2.1 | decision | **Fill and read the spelling memo in `readableBlock` alone, dropping a spelled node's children's entries so memory stays linear.** | 3 | 3 | 2 | 6 | 2 | 10.9 |
| 60 | 0.2.1 | decision | **Collect the questions `parse/` asks `emit/` into one named module.** | 3 | 4 | 2 | 6 | 2 | 10.7 | | 60 | 0.2.1 | decision | **Collect the questions `parse/` asks `emit/` into one named module.** | 3 | 4 | 2 | 6 | 2 | 10.7 |
| 59 | 0.2.1 | decision | **Group the directive grammar into `src/markdown/directive/`, move `Read<T>` to `result.ts`, and move `Flavour` and `flavourClaims` to `markdown/flavour.ts`.** | 3 | 5 | 2 | 6 | 2 | 10.4 | | 59 | 0.2.1 | decision | **Group the directive grammar into `src/markdown/directive/`, move `Read<T>` to `result.ts`, and move `Flavour` and `flavourClaims` to `markdown/flavour.ts`.** | 3 | 5 | 2 | 6 | 2 | 10.4 |
| 62 | 0.2.1 | decision | **Move the portable flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/portable/`, so `emit/` no longer imports `portable/`.** | 4 | 5 | 2 | 6 | 2 | 9.4 | | 62 | 0.2.1 | decision | **Move the portable flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/portable/`, so `emit/` no longer imports `portable/`.** | 4 | 5 | 2 | 6 | 2 | 9.4 |
@@ -126,16 +125,6 @@ Every reader reads `![a](u "t")` as an image captioned `t` (`docs/decisions.md`
image only alone in its paragraph, 3 of 3 on 2026-10-09). On 2026-10-11 a writer panel of the same image only alone in its paragraph, 3 of 3 on 2026-10-09). On 2026-10-11 a writer panel of the same
audience kept the captioned image's `!adf:caption` directive, 3 of 3, reading a title as hover text. audience kept the captioned image's `!adf:caption` directive, 3 of 3, reading a title as hover text.
### 110. Fill and read the spelling memo in `readableBlock` alone, dropping a spelled node's children's entries so memory stays linear.
Item 106 made the CommonMark ask fill the memo and the walk above drop each entry as it reads it, so
a deep subtree holds no level's text twice. Every comprehension seat read that split, across
`commonMarkSpelling`, `readableBlock` and their three callers, as order-dependent state, and the
chunk shipped under a worse ruling on Locality. Give the memo one reader and writer again: when
`readableBlock` keeps a node's spelling, it drops the entries of the node's children and
grandchildren, since the node's own spelling now holds their text. Found by the comprehension check
of item 106, 2026-10-13.
### 60. Collect the questions `parse/` asks `emit/` into one named module. ### 60. Collect the questions `parse/` asks `emit/` into one named module.
`parse/` asks `emit/` through `commonMarkSpelling` and `openingLinkTakesDirective`, each imported `parse/` asks `emit/` through `commonMarkSpelling` and `openingLinkTakesDirective`, each imported