mirror of
https://github.com/larvit/adf-codec.git
synced 2026-11-12 19:58:28 +01:00
Answer architecture round 1: 2 🔴, 5 🟡; readableBlock alone fills and reads the spelling memo, the escaper's steps hand on what they claim rather than write a shared buffer, engine-limit names only the throws an engine returns, and the docs agree on it
This commit is contained in:
+4
-3
@@ -9,9 +9,10 @@
|
|||||||
code. `MIGRATION.md` says which call to take.
|
code. `MIGRATION.md` says which call to take.
|
||||||
- **Breaking:** the markdown readers refuse a value that is not a string with the new code
|
- **Breaking:** the markdown readers refuse a value that is not a string with the new code
|
||||||
`not-a-string`; `MIGRATION.md` lists what each value did before.
|
`not-a-string`; `MIGRATION.md` lists what each value did before.
|
||||||
- **Breaking:** every call returns the new code `engine-limit` where the JavaScript engine runs
|
- **Breaking:** every call returns the new code `engine-limit` where the JavaScript engine passes
|
||||||
short of memory or passes its longest array, and, on JavaScriptCore, its longest string. A retry
|
its longest array, `Map` or `Set`, or finds no memory for a buffer, and, on JavaScriptCore, where
|
||||||
may succeed. Every other refusal stays deterministic.
|
its heap runs short or a string passes its longest. A retry may succeed. Every other refusal stays
|
||||||
|
deterministic.
|
||||||
- **Breaking:** an attribute value that is not a plain object, array or JSON primitive, such as a
|
- **Breaking:** an attribute value that is not a plain object, array or JSON primitive, such as a
|
||||||
`Date`, `Map`, typed array or class instance, makes `isAdfDocument` return `false` and the
|
`Date`, `Map`, typed array or class instance, makes `isAdfDocument` return `false` and the
|
||||||
converters refuse it as `not-an-adf-document`. Before, the converters wrote it as `{}` or its own
|
converters refuse it as `not-an-adf-document`. Before, the converters wrote it as `{}` or its own
|
||||||
|
|||||||
+3
-3
@@ -88,8 +88,8 @@ Markdown the spelling table leaves alone, which `0.2.0` reads as a different doc
|
|||||||
### Error codes
|
### Error codes
|
||||||
|
|
||||||
`unmappable-image`, `unspellable-character`, `unspellable-line-start` and `unspellable-link` leave
|
`unmappable-image`, `unspellable-character`, `unspellable-line-start` and `unspellable-link` leave
|
||||||
`ConvertErrorCode` and `engine-limit` and `not-a-string` join it: a `switch` naming one of the four
|
`ConvertErrorCode`, and `engine-limit` and `not-a-string` join it. A `switch` naming one of the four
|
||||||
stops compiling, and one naming every code needs the new cases. A document `adfToMarkdown` refused with one of the
|
stops compiling, and one naming every code needs the two new cases. A document `adfToMarkdown` refused with one of the
|
||||||
three `unspellable-` codes now converts in `adfToLosslessMarkdown`; markdown the parser refused
|
three `unspellable-` codes now converts in `adfToLosslessMarkdown`; markdown the parser refused
|
||||||
with one of the four now reads, or refuses with the code the table gives.
|
with one of the four now reads, or refuses with the code the table gives.
|
||||||
|
|
||||||
@@ -114,4 +114,4 @@ with one of the four now reads, or refuses with the code the table gives.
|
|||||||
| `null`, `undefined` or a non-empty array, on parse | throws a `TypeError` | `not-a-string` |
|
| `null`, `undefined` or a non-empty array, on parse | throws a `TypeError` | `not-a-string` |
|
||||||
| a `String` object, on parse | the document its text spells | `not-a-string`; pass `String(value)` |
|
| a `String` object, on parse | the document its text spells | `not-a-string`; pass `String(value)` |
|
||||||
| any other value that is not a string, such as a number or `{}`, on parse | an empty document | `not-a-string` |
|
| any other value that is not a string, such as a number or `{}`, on parse | an empty document | `not-a-string` |
|
||||||
| a conversion that runs the JavaScript engine short of memory or past its longest array | throws a `RangeError`, or the engine ends the process | `engine-limit`; retry it once |
|
| a conversion past the JavaScript engine's longest array, `Map` or `Set`, or finding no memory for a buffer | throws a `RangeError` | `engine-limit`; retry it once |
|
||||||
|
|||||||
@@ -411,7 +411,9 @@ typed on `ConvertError` takes both.
|
|||||||
A call reports the first refusal in document order and stops, so a document with several surfaces
|
A call reports the first refusal in document order and stops, so a document with several surfaces
|
||||||
them one per call. Every refusal is deterministic — there is no I/O anywhere — so a retry returns
|
them one per call. Every refusal is deterministic — there is no I/O anywhere — so a retry returns
|
||||||
the identical error: fix the input, or set the document aside. The exception is `engine-limit`,
|
the identical error: fix the input, or set the document aside. The exception is `engine-limit`,
|
||||||
whose causes include memory running short: retry it once, and treat a repeat as lasting.
|
whose causes include memory running short: retry it once, and treat a repeat as lasting. On V8
|
||||||
|
(Node, Deno, Chrome) a heap that runs out ends the process instead, so convert input of unbounded
|
||||||
|
size in a worker.
|
||||||
|
|
||||||
### Parsing
|
### Parsing
|
||||||
|
|
||||||
@@ -443,9 +445,9 @@ The emitter raises these, and so does a parse, which asks the emitter how to spe
|
|||||||
|
|
||||||
| Code | Fires when | What you can do |
|
| Code | Fires when | What you can do |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| `engine-limit` | in every call: a conversion that passes the longest array the JavaScript engine builds or runs it short of memory, and on JavaScriptCore (Bun, Safari) one that passes its longest string, 2^31 − 1 characters | retry once; if it repeats, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply |
|
| `engine-limit` | in every call: a conversion that passes the longest array, `Map` or `Set` the JavaScript engine builds, or that finds no memory for a buffer; on JavaScriptCore (Bun, Safari), also one that runs its heap short or passes its longest string, 2^31 − 1 characters | retry once; if it repeats, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply |
|
||||||
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
|
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
|
||||||
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or one that, written out as plain CommonMark, would be longer than 536870888 characters, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536870888 characters, V8's longest string, on every engine; in `adfToPortableMarkdown`, also a document whose text alone is that long, edge whitespace included. In every call: a conversion that passes the longest string the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536870888 characters or the engine's longest string, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply |
|
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or one that, written out as plain CommonMark, would be longer than 536870888 characters, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536870888 characters, V8's longest string, on every engine; in `adfToPortableMarkdown`, also a document whose text alone is that long, edge whitespace included. In every call: a conversion that passes V8's or SpiderMonkey's longest string | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536870888 characters or the engine's longest string, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply |
|
||||||
|
|
||||||
## The guarantees
|
## The guarantees
|
||||||
|
|
||||||
@@ -459,8 +461,7 @@ Serves [Goals](#goals) 1, 4 and 5.
|
|||||||
- Markdown this library reads, and markdown it writes, means what the CommonMark spec says. The
|
- Markdown this library reads, and markdown it writes, means what the CommonMark spec says. The
|
||||||
bullets below name every exception.
|
bullets below name every exception.
|
||||||
- CommonMark is valid input to `commonMarkToAdf`, apart from the raw HTML `unmappable-html` names,
|
- CommonMark is valid input to `commonMarkToAdf`, apart from the raw HTML `unmappable-html` names,
|
||||||
nesting past 500 levels, and input whose conversion passes the longest string or array the
|
nesting past 500 levels, and input whose conversion `engine-limit` or the length bound refuses.
|
||||||
JavaScript engine builds.
|
|
||||||
- The other readers claim shapes CommonMark reads as text or code. `losslessMarkdownToAdf` claims
|
- The other readers claim shapes CommonMark reads as text or code. `losslessMarkdownToAdf` claims
|
||||||
four: literal text matching directive, pipe-table or strikethrough syntax, and a code fence whose
|
four: literal text matching directive, pipe-table or strikethrough syntax, and a code fence whose
|
||||||
info string opens `adf:`. `portableMarkdownToAdf` claims what [Portable
|
info string opens `adf:`. `portableMarkdownToAdf` claims what [Portable
|
||||||
@@ -487,7 +488,7 @@ Serves [Goals](#goals) 1, 4 and 5.
|
|||||||
the outer brackets literal text, which the spec requires and the reference itself breaks, nesting
|
the outer brackets literal text, which the spec requires and the reference itself breaks, nesting
|
||||||
one `<a>` in the other.
|
one `<a>` in the other.
|
||||||
- Not every document converts back: `adfToLosslessMarkdown` refuses a document nesting past 500
|
- Not every document converts back: `adfToLosslessMarkdown` refuses a document nesting past 500
|
||||||
levels, and the node shapes the `unsupported-node-shape` row lists. Show the refusal and keep the
|
levels, the node shapes the `unsupported-node-shape` row lists, and what `engine-limit` names. Show the refusal and keep the
|
||||||
document read-only; saving markdown you could not produce is the loss the round-trip exists to
|
document read-only; saving markdown you could not produce is the loss the round-trip exists to
|
||||||
stop.
|
stop.
|
||||||
- The pipe table that `losslessMarkdownToAdf` and `portableMarkdownToAdf` read narrows GFM's twice:
|
- The pipe table that `losslessMarkdownToAdf` and `portableMarkdownToAdf` read narrows GFM's twice:
|
||||||
|
|||||||
@@ -680,3 +680,34 @@ Maintainability senior, hardest first:
|
|||||||
Inherited architect, would restructure first:
|
Inherited architect, would restructure first:
|
||||||
|
|
||||||
- Locality: `src/markdown/emit/adf-to-markdown.ts:126` `readableBlock`: the memo is mutated and deleted across parse, emit and portable, and the length bound is applied at five sites
|
- Locality: `src/markdown/emit/adf-to-markdown.ts:126` `readableBlock`: the memo is mutated and deleted across parse, emit and portable, and the length bound is applied at five sites
|
||||||
|
|
||||||
|
## 2026-10-13T14:48:38Z, PR #37 at ecd695a, against 7a27f6b
|
||||||
|
|
||||||
|
Ruling: worse
|
||||||
|
|
||||||
|
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| Mid A | better | worse | better | worse | worse |
|
||||||
|
| Inherited architect | better | worse | same | worse | worse |
|
||||||
|
|
||||||
|
Mid A, decided by:
|
||||||
|
|
||||||
|
- Locality, Self-sufficiency: `src/markdown/emit/adf-to-markdown.ts:116` `commonMarkSpelling` / `readableBlock` / `takeKeptSpelling`: only commonMarkSpelling fills the memo and a read empties it, so whether an entry exists depends on which caller ran first; the reason lives only behind a pointer to docs/decisions.md
|
||||||
|
- Locality: `src/markdown/emit/line-escaping.ts:229` `escapeClosedRuns`: both steps write one shared Uint8Array, and correctness rests on the scan order a second comment line must defend
|
||||||
|
- Navigation, Shape: `src/markdown/length-bound.ts:30` `withinEngineLimits`: the engine-limit catch moves out of result.ts into one well-named file, with string-length and engine limits apart and `engine-limit` a code of its own
|
||||||
|
- Locality: `src/markdown/emit/adf-to-markdown.ts:51` `writeMarkdown` / `walkBlocks` / `commonMarkSpelling`: the length bound is enforced in layers a reader changing it must hold together
|
||||||
|
- Shape: `src/markdown/parse/markdown-to-adf.ts:266` `directiveNode`: undefined is wrapped into a Result to pass the engine guard, then unpicked
|
||||||
|
- Shape: `src/markdown/emit/line-escaping.ts:35` `Escapings`: the type is a lookup function, its comment describes the Uint8Array behind it; `backslashed` adds 4096-piece chunking
|
||||||
|
- Shape, Locality: `src/canonical-json.ts:23` `canonicalJsonPieces`: a generator replaces a boolean-continue callback
|
||||||
|
- Shape: `src/markdown/emit/inline-line.ts:235` `emitLeaf`: the content-or-text refusal moves into one place
|
||||||
|
- Locality: `src/markdown/portable/adf-to-portable-markdown.ts:128` `portableSequence`: a spelling refusal propagates where it was swallowed, at the cost of Sourced path plumbing
|
||||||
|
|
||||||
|
Inherited architect, decided by:
|
||||||
|
|
||||||
|
- Locality, Self-sufficiency: `src/markdown/emit/adf-to-markdown.ts:139` `takeKeptSpelling`: reading deletes the entry and only commonMarkSpelling fills it, so readableBlock is right only if another file's caller asked first, once per node; explained only by a pointer to docs/decisions.md
|
||||||
|
- Locality, Shape: `src/markdown/parse/markdown-to-adf.ts:266` `directiveNode`: a second withinEngineLimits inside readDocument's, fed `?? success(undefined)`, so null and undefined carry meaning the reader must remember
|
||||||
|
- Locality: `src/markdown/emit/adf-to-markdown.ts:58` `walkBlocks`: the length bound is enforced at about six sites across three files
|
||||||
|
- Navigation: `src/markdown/length-bound.ts:30` `withinEngineLimits`: engine-limit handling sits in a file named for it, and the engine-limit code parts memory from the length bound
|
||||||
|
- Shape, Locality: `src/markdown/emit/line-escaping.ts:229` `escapeClosedRuns`: mutates its input in place and needs a two-line comment arguing it is safe; output joined in batches of 4096
|
||||||
|
- Locality, Shape: `src/markdown/portable/adf-to-portable-markdown.ts:128` `portableSequence`: a refusal propagates with its source path, where it was swallowed
|
||||||
|
- Shape: `src/markdown/emit/inline-line.ts:235` `emitLeaf`: one place refuses content and text, and a generator replaces the boolean-callback JSON writer
|
||||||
|
|||||||
+5
-5
@@ -467,11 +467,11 @@ handles one cause alike whichever node, attribute or direction raised it.
|
|||||||
causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested JSON,
|
causes the bound (§The code list). A carried node passes it on about 1 MB of deeply nested JSON,
|
||||||
since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in a
|
since the block carry indents two spaces a level; `todo.md` item 107 respells the carry in a
|
||||||
length linear in its JSON (2026-10-11).
|
length linear in its JSON (2026-10-11).
|
||||||
- A conversion the engine stops, by running short of memory or passing its longest array, is
|
- A conversion the engine stops with a throw of its own, past its longest array, `Map` or `Set`
|
||||||
`engine-limit`, at the empty path. A retry may pass it, so `unsupported-node-shape`, which never
|
or short of memory, is `engine-limit`, at the empty path. A retry may pass the memory case, so
|
||||||
does, cannot hold it. JavaScriptCore throws the same `Out of memory` past its longest string,
|
`unsupported-node-shape`, which no retry passes, cannot hold the class. JavaScriptCore throws the
|
||||||
which takes `engine-limit` too: telling the two apart would mean reading the engine's message
|
same `Out of memory` past its longest string, which takes `engine-limit` too, since only the
|
||||||
(2026-10-13, the maintainer).
|
message would tell the two apart (2026-10-13, the maintainer).
|
||||||
- The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at
|
- The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at
|
||||||
position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and
|
position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and
|
||||||
`not-an-adf-document` names the emitters' input (2026-10-05).
|
`not-an-adf-document` names the emitters' input (2026-10-05).
|
||||||
|
|||||||
@@ -108,39 +108,34 @@ function interruptsParagraph(node: AdfNode): boolean {
|
|||||||
function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
|
function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> {
|
||||||
const model = blockNodeModel(node.type)
|
const model = blockNodeModel(node.type)
|
||||||
if (model === undefined) return carriedFence(carriedBlock(node, path, depth))
|
if (model === undefined) return carriedFence(carriedBlock(node, path, depth))
|
||||||
const readable = readableBlock(node, path, depth, writing)
|
const readable = readableBlock(node, path, depth, writing, false)
|
||||||
if (readable !== undefined) return readable
|
if (readable !== undefined) return readable
|
||||||
return emitDirectiveBlock(node, model, path, depth, () => walkBlocks(nodeContent(node), path, depth + 1, writing))
|
return emitDirectiveBlock(node, model, path, depth, () => walkBlocks(nodeContent(node), path, depth + 1, writing))
|
||||||
}
|
}
|
||||||
|
|
||||||
export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing & { memo: SpellingMemo }): Result<null> | undefined {
|
export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing & { memo: SpellingMemo }): Result<null> | undefined {
|
||||||
const readable = readableBlock(node, path, depth, writing)
|
const readable = readableBlock(node, path, depth, writing, true)
|
||||||
if (readable === undefined) {
|
if (readable === undefined) return undefined
|
||||||
writing.memo.set(node, { block: undefined, depth })
|
|
||||||
return undefined
|
|
||||||
}
|
|
||||||
if (!readable.ok) return readable
|
if (!readable.ok) return readable
|
||||||
const fits = withinLongest(readable.value.text)
|
const fits = withinLongest(readable.value.text)
|
||||||
if (!fits.ok) return fits
|
if (!fits.ok) return fits
|
||||||
writing.memo.set(node, { block: readable.value, depth })
|
|
||||||
return readable.value.spelling === 'directive' ? undefined : success(null)
|
return readable.value.spelling === 'directive' ? undefined : success(null)
|
||||||
}
|
}
|
||||||
|
|
||||||
function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
|
// Only an asked node's spelling is kept, and reading it drops it: the walk above spells the node into its own text, so no level of a subtree holds that text twice.
|
||||||
const kept = takeKeptSpelling(writing.memo, node)
|
function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing, asked: boolean): Result<EmittedBlock> | undefined {
|
||||||
|
const { memo } = writing
|
||||||
|
const kept = memo?.get(node)
|
||||||
|
memo?.delete(node)
|
||||||
if (kept !== undefined) {
|
if (kept !== undefined) {
|
||||||
if (kept.block === undefined) return undefined
|
if (kept.block === undefined) return undefined
|
||||||
// An entry holds the spelling at its fill depth. A read at or above it rebases headroom; a read below re-spells, because a hit skips the walk's depth guards.
|
// An entry holds the spelling at its fill depth. A read at or above it rebases headroom; a read below re-spells, because a hit skips the walk's depth guards.
|
||||||
if (depth <= kept.depth) return success({ ...kept.block, headroom: kept.block.headroom + kept.depth - depth })
|
if (depth <= kept.depth) return success({ ...kept.block, headroom: kept.block.headroom + kept.depth - depth })
|
||||||
}
|
}
|
||||||
return spellReadableBlock(node, path, depth, writing)
|
const spelled = spellReadableBlock(node, path, depth, writing)
|
||||||
}
|
if (asked && spelled === undefined) memo?.set(node, { block: undefined, depth })
|
||||||
|
if (asked && spelled?.ok === true) memo?.set(node, { block: spelled.value, depth })
|
||||||
// docs/decisions.md §The spelling memo
|
return spelled
|
||||||
function takeKeptSpelling(memo: SpellingMemo | undefined, node: AdfNode): KeptSpelling | undefined {
|
|
||||||
const kept = memo?.get(node)
|
|
||||||
memo?.delete(node)
|
|
||||||
return kept
|
|
||||||
}
|
}
|
||||||
|
|
||||||
function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
|
function spellReadableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result<EmittedBlock> | undefined {
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ import { opensLinkDefinition } from '../commonmark/link-reference-definitions.ts
|
|||||||
export type DelimiterRole = 'close' | 'open'
|
export type DelimiterRole = 'close' | 'open'
|
||||||
|
|
||||||
// A segment's kind: `backslash` text, `bracketed` text inside `[…]` (link text, alt, a directive's content), `bracketed-link-target` syntax: a link target inside `[…]`, `none` syntax written as is.
|
// A segment's kind: `backslash` text, `bracketed` text inside `[…]` (link text, alt, a directive's content), `bracketed-link-target` syntax: a link target inside `[…]`, `none` syntax written as is.
|
||||||
export type InlineEscaping = 'backslash' | 'bracketed' | 'bracketed-link-target' | 'none'
|
export type InlineEscaping = (typeof escapingCodes)[number]
|
||||||
|
|
||||||
export type NodeRange = { first: number; last: number }
|
export type NodeRange = { first: number; last: number }
|
||||||
|
|
||||||
@@ -32,9 +32,14 @@ type EmittedDelimiter = { closes: boolean; offset: number; pair: number; width:
|
|||||||
|
|
||||||
type EmittedRun = { canClose: boolean; canOpen: boolean; character: string; delimiters: EmittedDelimiter[]; length: number; start: number }
|
type EmittedRun = { canClose: boolean; canOpen: boolean; character: string; delimiters: EmittedDelimiter[]; length: number; start: number }
|
||||||
|
|
||||||
// One byte a character, never an array element or Set entry: a line may run to hundreds of millions of characters.
|
|
||||||
type Escapings = (index: number) => InlineEscaping | undefined
|
type Escapings = (index: number) => InlineEscaping | undefined
|
||||||
|
|
||||||
|
// Where a segment's first character landed in the output.
|
||||||
|
type PlacedSegment = { segment: InlineSegment; start: number }
|
||||||
|
|
||||||
|
// The run's escapable characters, or its characters before `end`.
|
||||||
|
type RunEscapes = { end: number; escapable: false } | { escapable: true }
|
||||||
|
|
||||||
type EscapeWalk = {
|
type EscapeWalk = {
|
||||||
readonly claims: Claims
|
readonly claims: Claims
|
||||||
readonly container: LineContainer
|
readonly container: LineContainer
|
||||||
@@ -44,7 +49,7 @@ type EscapeWalk = {
|
|||||||
readonly scan: string
|
readonly scan: string
|
||||||
}
|
}
|
||||||
|
|
||||||
const escapingCodes: readonly InlineEscaping[] = ['backslash', 'bracketed', 'bracketed-link-target', 'none']
|
const escapingCodes: readonly ['backslash', 'bracketed', 'bracketed-link-target', 'none'] = ['backslash', 'bracketed', 'bracketed-link-target', 'none']
|
||||||
|
|
||||||
const followsLinkText = /[([]/
|
const followsLinkText = /[([]/
|
||||||
|
|
||||||
@@ -85,6 +90,7 @@ function resolveEmphasis(segments: readonly InlineSegment[]): InlineSegment[] {
|
|||||||
|
|
||||||
function escape(segments: readonly InlineSegment[], container: LineContainer, claims: Claims): AssembledLine {
|
function escape(segments: readonly InlineSegment[], container: LineContainer, claims: Claims): AssembledLine {
|
||||||
const scan = segments.map((segment) => segment.text).join('')
|
const scan = segments.map((segment) => segment.text).join('')
|
||||||
|
// One byte a character, never an array element or Set entry: a line may run to hundreds of millions of characters.
|
||||||
const codes = new Uint8Array(scan.length)
|
const codes = new Uint8Array(scan.length)
|
||||||
let start = 0
|
let start = 0
|
||||||
for (const segment of segments) {
|
for (const segment of segments) {
|
||||||
@@ -96,16 +102,15 @@ function escape(segments: readonly InlineSegment[], container: LineContainer, cl
|
|||||||
return code === undefined ? undefined : escapingCodes[code]
|
return code === undefined ? undefined : escapingCodes[code]
|
||||||
}
|
}
|
||||||
const escaped = escapeClosedRuns(scan, escapings, escapeClaims(scan, escapings, container, claims))
|
const escaped = escapeClosedRuns(scan, escapings, escapeClaims(scan, escapings, container, claims))
|
||||||
const { output, placements } = backslashed(segments, scan, escaped)
|
const { output, placed } = backslashed(segments, scan, escaped)
|
||||||
if (container === 'paragraph' && opensLinkDefinition(output)) {
|
if (container === 'paragraph' && opensLinkDefinition(output)) {
|
||||||
if (segments[0]?.nodes !== undefined) return { line: output, openingLinkAsDirective: true, unspellableRuns: [] }
|
if (segments[0]?.nodes !== undefined) return { line: output, openingLinkAsDirective: true, unspellableRuns: [] }
|
||||||
return { line: `\\${output}`, unspellableRuns: unspellableRuns(segments, output, placements) }
|
return { line: `\\${output}`, unspellableRuns: unspellableRuns(placed, output) }
|
||||||
}
|
}
|
||||||
return { line: output, unspellableRuns: unspellableRuns(segments, output, placements) }
|
return { line: output, unspellableRuns: unspellableRuns(placed, output) }
|
||||||
}
|
}
|
||||||
|
|
||||||
// The output, and where each segment's first character landed in it.
|
function backslashed(segments: readonly InlineSegment[], scan: string, escaped: Uint8Array): { output: string; placed: PlacedSegment[] } {
|
||||||
function backslashed(segments: readonly InlineSegment[], scan: string, escaped: Uint8Array): { output: string; placements: number[] } {
|
|
||||||
// Joined a few thousand pieces at a time: an array element per escape would cost more than the character it escapes.
|
// Joined a few thousand pieces at a time: an array element per escape would cost more than the character it escapes.
|
||||||
const chunks: string[] = []
|
const chunks: string[] = []
|
||||||
let pieces: string[] = []
|
let pieces: string[] = []
|
||||||
@@ -119,7 +124,7 @@ function backslashed(segments: readonly InlineSegment[], scan: string, escaped:
|
|||||||
}
|
}
|
||||||
pieces.push(scan.slice(from))
|
pieces.push(scan.slice(from))
|
||||||
chunks.push(pieces.join(''))
|
chunks.push(pieces.join(''))
|
||||||
const placements: number[] = []
|
const placed: PlacedSegment[] = []
|
||||||
let escapes = 0
|
let escapes = 0
|
||||||
let next = escaped.indexOf(1)
|
let next = escaped.indexOf(1)
|
||||||
let start = 0
|
let start = 0
|
||||||
@@ -128,10 +133,10 @@ function backslashed(segments: readonly InlineSegment[], scan: string, escaped:
|
|||||||
escapes += 1
|
escapes += 1
|
||||||
next = escaped.indexOf(1, next + 1)
|
next = escaped.indexOf(1, next + 1)
|
||||||
}
|
}
|
||||||
placements.push(start + escapes)
|
placed.push({ segment, start: start + escapes })
|
||||||
start += segment.text.length
|
start += segment.text.length
|
||||||
}
|
}
|
||||||
return { output: chunks.join(''), placements }
|
return { output: chunks.join(''), placed }
|
||||||
}
|
}
|
||||||
|
|
||||||
function escapeClaims(scan: string, escapings: Escapings, container: LineContainer, claims: Claims): Uint8Array {
|
function escapeClaims(scan: string, escapings: Escapings, container: LineContainer, claims: Claims): Uint8Array {
|
||||||
@@ -150,7 +155,9 @@ function escapeClaims(scan: string, escapings: Escapings, container: LineContain
|
|||||||
while (index > line.start + line.text.length) line = scanLine(scan, line.start + line.text.length + 1)
|
while (index > line.start + line.text.length) line = scanLine(scan, line.start + line.text.length + 1)
|
||||||
if (runCharacters.includes(scan.charAt(index))) {
|
if (runCharacters.includes(scan.charAt(index))) {
|
||||||
const end = index + runLength(scan, index)
|
const end = index + runLength(scan, index)
|
||||||
runEscapes(walk, index, end, line, escaped)
|
const run = runEscapes(walk, index, end, line)
|
||||||
|
if (!run.escapable) escaped.fill(1, index, run.end)
|
||||||
|
else for (let at = index; at < end; at += 1) if (isEscapable(escapings(at))) escaped[at] = 1
|
||||||
index = end
|
index = end
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
@@ -164,18 +171,13 @@ function escapeClaims(scan: string, escapings: Escapings, container: LineContain
|
|||||||
}
|
}
|
||||||
|
|
||||||
// A text character beside one of the emitter's delimiters would lengthen that delimiter's run, so every text character in such a run escapes.
|
// A text character beside one of the emitter's delimiters would lengthen that delimiter's run, so every text character in such a run escapes.
|
||||||
function runEscapes(walk: EscapeWalk, start: number, end: number, line: ScanLine, escaped: Uint8Array): void {
|
function runEscapes(walk: EscapeWalk, start: number, end: number, line: ScanLine): RunEscapes {
|
||||||
const { escapings, scan } = walk
|
const { escapings, scan } = walk
|
||||||
if (runHolds(escapings, start, end, isSyntax) && runHolds(escapings, start, end, isEscapable)) {
|
if (runHolds(escapings, start, end, isSyntax) && runHolds(escapings, start, end, isEscapable)) return { escapable: true }
|
||||||
for (let index = start; index < end; index += 1) if (isEscapable(escapings(index))) escaped[index] = 1
|
let index = start
|
||||||
return
|
// A backtick run escapes whole: a bare rest would be a shorter backtick string, which can close a span an earlier run opens.
|
||||||
}
|
while (index < end && ((index > start && scan.charAt(index) === '`') || claimsRunRest(walk, index, end, line))) index += 1
|
||||||
for (let index = start; index < end; index += 1) {
|
return { end: index, escapable: false }
|
||||||
// A backtick run escapes whole: a bare rest would be a shorter backtick string, which can close a span an earlier run opens.
|
|
||||||
const claimed = (index > start && scan.charAt(index) === '`') || claimsRunRest(walk, index, end, line)
|
|
||||||
if (!claimed) break
|
|
||||||
escaped[index] = 1
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
function runHolds(escapings: Escapings, start: number, end: number, kind: (escaping: InlineEscaping | undefined) => boolean): boolean {
|
function runHolds(escapings: Escapings, start: number, end: number, kind: (escaping: InlineEscaping | undefined) => boolean): boolean {
|
||||||
@@ -225,8 +227,8 @@ function claimsCharacter({ container, escapings, headingCloser, linkClose, scan
|
|||||||
}
|
}
|
||||||
|
|
||||||
// CommonMark reads no escape inside a code span, so a backtick string left beside an escape still closes a span that an earlier bare run of its length opens.
|
// CommonMark reads no escape inside a code span, so a backtick string left beside an escape still closes a span that an earlier bare run of its length opens.
|
||||||
// The walk runs backwards and escapes only the run it just read, so every index it reads still holds the claims alone.
|
function escapeClosedRuns(scan: string, escapings: Escapings, claimed: Uint8Array): Uint8Array {
|
||||||
function escapeClosedRuns(scan: string, escapings: Escapings, escaped: Uint8Array): Uint8Array {
|
const escaped = claimed.slice()
|
||||||
// The lengths of the backtick strings escapes leave: a bare run of one of these lengths would open a span one of them closes, so it is escaped too.
|
// The lengths of the backtick strings escapes leave: a bare run of one of these lengths would open a span one of them closes, so it is escaped too.
|
||||||
const formed = new Set<number>()
|
const formed = new Set<number>()
|
||||||
let end = scan.length - 1
|
let end = scan.length - 1
|
||||||
@@ -239,11 +241,11 @@ function escapeClosedRuns(scan: string, escapings: Escapings, escaped: Uint8Arra
|
|||||||
while (scan.charAt(start - 1) === '`') start -= 1
|
while (scan.charAt(start - 1) === '`') start -= 1
|
||||||
let segmentEnd = end
|
let segmentEnd = end
|
||||||
for (let index = end; index > start; index -= 1) {
|
for (let index = end; index > start; index -= 1) {
|
||||||
if (escaped[index] !== 1) continue
|
if (claimed[index] !== 1) continue
|
||||||
formed.add(segmentEnd - index + 1)
|
formed.add(segmentEnd - index + 1)
|
||||||
segmentEnd = index - 1
|
segmentEnd = index - 1
|
||||||
}
|
}
|
||||||
if (segmentEnd !== end || escaped[start] === 1) formed.add(segmentEnd - start + 1)
|
if (segmentEnd !== end || claimed[start] === 1) formed.add(segmentEnd - start + 1)
|
||||||
else if (escapings(start) !== 'none' && formed.has(end - start + 1)) {
|
else if (escapings(start) !== 'none' && formed.has(end - start + 1)) {
|
||||||
escaped.fill(1, start, end + 1)
|
escaped.fill(1, start, end + 1)
|
||||||
// Each escaped backtick then stands alone, a string of length one.
|
// Each escaped backtick then stands alone, a string of length one.
|
||||||
@@ -255,18 +257,17 @@ function escapeClosedRuns(scan: string, escapings: Escapings, escaped: Uint8Arra
|
|||||||
}
|
}
|
||||||
|
|
||||||
// One emphasis run, the innermost, or every highlight run the line cannot spell.
|
// One emphasis run, the innermost, or every highlight run the line cannot spell.
|
||||||
function unspellableRuns(segments: readonly InlineSegment[], output: string, placements: readonly number[]): MarkRun[] {
|
function unspellableRuns(placed: readonly PlacedSegment[], output: string): MarkRun[] {
|
||||||
const { nodes, runs } = emittedRuns(segments, placements, output)
|
const { nodes, runs } = emittedRuns(placed, output)
|
||||||
const pair = misflanked(runs) ?? unpaired(runs)
|
const pair = misflanked(runs) ?? unpaired(runs)
|
||||||
const run = pair === undefined ? undefined : nodes[pair]
|
const run = pair === undefined ? undefined : nodes[pair]
|
||||||
return run === undefined ? unreadHighlights(segments, output, placements) : [run]
|
return run === undefined ? unreadHighlights(placed, output) : [run]
|
||||||
}
|
}
|
||||||
|
|
||||||
// No `==` in text can open or close, and highlights never nest, so a pair reads back where each delimiter flanks.
|
// No `==` in text can open or close, and highlights never nest, so a pair reads back where each delimiter flanks.
|
||||||
function unreadHighlights(segments: readonly InlineSegment[], output: string, placements: readonly number[]): MarkRun[] {
|
function unreadHighlights(placed: readonly PlacedSegment[], output: string): MarkRun[] {
|
||||||
const unread: MarkRun[] = []
|
const unread: MarkRun[] = []
|
||||||
for (const [index, segment] of segments.entries()) {
|
for (const { segment, start } of placed) {
|
||||||
const start = placements[index] ?? 0
|
|
||||||
if (segment.highlight === undefined) continue
|
if (segment.highlight === undefined) continue
|
||||||
const flanking = highlightFlanking(output, start)
|
const flanking = highlightFlanking(output, start)
|
||||||
const flanks = segment.highlight === 'open' ? flanking.opens : flanking.closes
|
const flanks = segment.highlight === 'open' ? flanking.opens : flanking.closes
|
||||||
@@ -305,12 +306,11 @@ function delimiterAt(run: EmittedRun, closes: boolean, offset: number, width: nu
|
|||||||
return run.delimiters.find((delimiter) => delimiter.closes === closes && delimiter.offset === offset && delimiter.width === width)
|
return run.delimiters.find((delimiter) => delimiter.closes === closes && delimiter.offset === offset && delimiter.width === width)
|
||||||
}
|
}
|
||||||
|
|
||||||
function emittedRuns(segments: readonly InlineSegment[], placements: readonly number[], output: string): { nodes: MarkRun[]; runs: EmittedRun[] } {
|
function emittedRuns(placed: readonly PlacedSegment[], output: string): { nodes: MarkRun[]; runs: EmittedRun[] } {
|
||||||
const runs: EmittedRun[] = []
|
const runs: EmittedRun[] = []
|
||||||
const nodes: MarkRun[] = []
|
const nodes: MarkRun[] = []
|
||||||
const open: number[] = []
|
const open: number[] = []
|
||||||
for (const [index, segment] of segments.entries()) {
|
for (const { segment, start } of placed) {
|
||||||
const start = placements[index] ?? 0
|
|
||||||
if (segment.emphasis === undefined) continue
|
if (segment.emphasis === undefined) continue
|
||||||
const closes = segment.emphasis === 'close'
|
const closes = segment.emphasis === 'close'
|
||||||
const pair = closes ? (open.pop() ?? nodes.length) : nodes.length
|
const pair = closes ? (open.pop() ?? nodes.length) : nodes.length
|
||||||
|
|||||||
@@ -36,6 +36,9 @@ test('refuses a string past the bound as too long on every engine, and any other
|
|||||||
}
|
}
|
||||||
assert.equal(code('Invalid array length'), 'engine-limit')
|
assert.equal(code('Invalid array length'), 'engine-limit')
|
||||||
assert.equal(code('Array buffer allocation failed'), 'engine-limit')
|
assert.equal(code('Array buffer allocation failed'), 'engine-limit')
|
||||||
|
assert.equal(code('Map maximum size exceeded'), 'engine-limit')
|
||||||
|
assert.equal(code('Set maximum size exceeded'), 'engine-limit')
|
||||||
|
assert.equal(code('invalid array length'), 'engine-limit')
|
||||||
assert.deepEqual(
|
assert.deepEqual(
|
||||||
withinEngineLimits(() => success('x')),
|
withinEngineLimits(() => success('x')),
|
||||||
{ ok: true, value: 'x' },
|
{ ok: true, value: 'x' },
|
||||||
|
|||||||
@@ -3,9 +3,16 @@ import { failure, success, type Result } from '../result.ts'
|
|||||||
// V8's longest string (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
|
// V8's longest string (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
|
||||||
export const longestMarkdown = 2 ** 29 - 24
|
export const longestMarkdown = 2 ** 29 - 24
|
||||||
|
|
||||||
// Measured on Firefox 155, Node 24 and Bun 1.4. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the others name an engine's own limit or its memory running short.
|
// Measured on Firefox 155, Node 24 and Bun 1.4, and SpiderMonkey's array length from its js.msg. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the others name an engine's own limit or its memory running short.
|
||||||
const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length']
|
const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length']
|
||||||
const pastEngineLimit = ['RangeError: Array buffer allocation failed', 'RangeError: Invalid array length', 'RangeError: Out of memory']
|
const pastEngineLimit = [
|
||||||
|
'RangeError: Array buffer allocation failed',
|
||||||
|
'RangeError: Invalid array length',
|
||||||
|
'RangeError: Map maximum size exceeded',
|
||||||
|
'RangeError: Out of memory',
|
||||||
|
'RangeError: Set maximum size exceeded',
|
||||||
|
'RangeError: invalid array length',
|
||||||
|
]
|
||||||
|
|
||||||
export function tooLong<T>(): Result<T> {
|
export function tooLong<T>(): Result<T> {
|
||||||
return failure('unsupported-node-shape', `the markdown for this document, or for a directive's plain CommonMark, would be longer than ${longestMarkdown} characters, the longest this library writes`, [])
|
return failure('unsupported-node-shape', `the markdown for this document, or for a directive's plain CommonMark, would be longer than ${longestMarkdown} characters, the longest this library writes`, [])
|
||||||
|
|||||||
@@ -32,7 +32,6 @@
|
|||||||
| 89 | 0.2.1 | | **Build a `date`, `emoji`, `inlineCard`, `mention` or `status` holding no mark Atlassian's schema withholds from it, in every reader.** | 4 | 4 | 5 | 6 | 4 | 13.0 |
|
| 89 | 0.2.1 | | **Build a `date`, `emoji`, `inlineCard`, `mention` or `status` holding no mark Atlassian's schema withholds from it, in every reader.** | 4 | 4 | 5 | 6 | 4 | 13.0 |
|
||||||
| 99 | 0.2.1 | defect | **Write a linked external image standing inline or in a table cell in the spelling a reader panel picks, in `adfToPortableMarkdown`.** | 3 | 3 | 6 | 7 | 7 | 12.9 |
|
| 99 | 0.2.1 | defect | **Write a linked external image standing inline or in a table cell in the spelling a reader panel picks, in `adfToPortableMarkdown`.** | 3 | 3 | 6 | 7 | 7 | 12.9 |
|
||||||
| 100 | 0.2.1 | decision | **Put the reading of an image's title as its caption back to a reader panel, now that a writer panel reads a title as hover text.** | 3 | 3 | 4 | 5 | 4, 6 | 10.9 |
|
| 100 | 0.2.1 | decision | **Put the reading of an image's title as its caption back to a reader panel, now that a writer panel reads a title as hover text.** | 3 | 3 | 4 | 5 | 4, 6 | 10.9 |
|
||||||
| 110 | 0.2.1 | decision | **Fill and read the spelling memo in `readableBlock` alone, dropping a spelled node's children's entries so memory stays linear.** | 3 | 3 | 2 | 6 | 2 | 10.9 |
|
|
||||||
| 60 | 0.2.1 | decision | **Collect the questions `parse/` asks `emit/` into one named module.** | 3 | 4 | 2 | 6 | 2 | 10.7 |
|
| 60 | 0.2.1 | decision | **Collect the questions `parse/` asks `emit/` into one named module.** | 3 | 4 | 2 | 6 | 2 | 10.7 |
|
||||||
| 59 | 0.2.1 | decision | **Group the directive grammar into `src/markdown/directive/`, move `Read<T>` to `result.ts`, and move `Flavour` and `flavourClaims` to `markdown/flavour.ts`.** | 3 | 5 | 2 | 6 | 2 | 10.4 |
|
| 59 | 0.2.1 | decision | **Group the directive grammar into `src/markdown/directive/`, move `Read<T>` to `result.ts`, and move `Flavour` and `flavourClaims` to `markdown/flavour.ts`.** | 3 | 5 | 2 | 6 | 2 | 10.4 |
|
||||||
| 62 | 0.2.1 | decision | **Move the portable flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/portable/`, so `emit/` no longer imports `portable/`.** | 4 | 5 | 2 | 6 | 2 | 9.4 |
|
| 62 | 0.2.1 | decision | **Move the portable flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/portable/`, so `emit/` no longer imports `portable/`.** | 4 | 5 | 2 | 6 | 2 | 9.4 |
|
||||||
@@ -126,16 +125,6 @@ Every reader reads `` as an image captioned `t` (`docs/decisions.md`
|
|||||||
image only alone in its paragraph, 3 of 3 on 2026-10-09). On 2026-10-11 a writer panel of the same
|
image only alone in its paragraph, 3 of 3 on 2026-10-09). On 2026-10-11 a writer panel of the same
|
||||||
audience kept the captioned image's `!adf:caption` directive, 3 of 3, reading a title as hover text.
|
audience kept the captioned image's `!adf:caption` directive, 3 of 3, reading a title as hover text.
|
||||||
|
|
||||||
### 110. Fill and read the spelling memo in `readableBlock` alone, dropping a spelled node's children's entries so memory stays linear.
|
|
||||||
|
|
||||||
Item 106 made the CommonMark ask fill the memo and the walk above drop each entry as it reads it, so
|
|
||||||
a deep subtree holds no level's text twice. Every comprehension seat read that split, across
|
|
||||||
`commonMarkSpelling`, `readableBlock` and their three callers, as order-dependent state, and the
|
|
||||||
chunk shipped under a worse ruling on Locality. Give the memo one reader and writer again: when
|
|
||||||
`readableBlock` keeps a node's spelling, it drops the entries of the node's children and
|
|
||||||
grandchildren, since the node's own spelling now holds their text. Found by the comprehension check
|
|
||||||
of item 106, 2026-10-13.
|
|
||||||
|
|
||||||
### 60. Collect the questions `parse/` asks `emit/` into one named module.
|
### 60. Collect the questions `parse/` asks `emit/` into one named module.
|
||||||
|
|
||||||
`parse/` asks `emit/` through `commonMarkSpelling` and `openingLinkTakesDirective`, each imported
|
`parse/` asks `emit/` through `commonMarkSpelling` and `openingLinkTakesDirective`, each imported
|
||||||
|
|||||||
Reference in New Issue
Block a user