3
0
mirror of https://github.com/larvit/adf-codec.git synced 2026-11-12 11:48:29 +01:00

Answer prose round 1: the refusal messages name what passed the bound, the decisions read one idea a sentence, and the changelog gives the readers' fix its own bullet

This commit is contained in:
2026-10-13 01:17:36 +02:00
committed by Lillem4n
parent 142ed310c2
commit 7bc69355c3
11 changed files with 60 additions and 59 deletions
+8 -7
View File
@@ -83,13 +83,14 @@
- Fix a cyclic document, in which an object holds itself: `adfToLosslessMarkdown` and
`adfToPortableMarkdown` refuse it as `not-an-adf-document` and `isAdfDocument` returns `false`,
where all three hung.
- Fix a document whose markdown would be longer than 536,870,888 characters, V8's longest string:
`adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape` on
every engine, where they threw or ran out of memory.
- Fix a directive that, written out as plain CommonMark, would be that long: `losslessMarkdownToAdf`
refuses it as `unsupported-node-shape`, where a few megabytes of nested list lines ran it out of
memory. The markdown readers return `unsupported-node-shape` where they threw past the longest
string or array the JavaScript engine builds.
- Fix a document whose markdown would be longer than 536870888 characters, V8's longest string:
`adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape` on every
engine, where they threw or ran out of memory.
- Fix a directive that, written out as plain CommonMark, would be longer than 536870888 characters:
`losslessMarkdownToAdf` refuses it as `unsupported-node-shape`, where a few megabytes of nested
list lines ran the call out of memory.
- Fix the markdown readers past the longest string or array the JavaScript engine builds: they
return `unsupported-node-shape` where they threw.
- Fix deeply nested compact JSON in an `adf:` fence: `losslessMarkdownToAdf` refuses it as
`unsupported-node-shape`, as it refuses any compact JSON there, where about a megabyte of it threw.
- Add `adfToPortableMarkdown` and `portableMarkdownToAdf`, a lossy pair converting ADF to and from
+6 -5
View File
@@ -410,9 +410,9 @@ typed on `ConvertError` takes both.
A call reports the first refusal in document order and stops, so a document with several surfaces
them one per call. Every refusal is deterministic — there is no I/O anywhere — so a retry returns
the identical error: fix the input, or set the document aside. The one exception is JavaScriptCore
(Bun, Safari), which reports a heap running short as it reports a string past its longest: there,
retry once a call's `unsupported-node-shape` at the empty path.
the identical error: fix the input, or set the document aside. On JavaScriptCore (Bun, Safari),
retry once any call that returns `unsupported-node-shape` at the empty path: that engine reports a
heap running short the same way as a string too long to build.
### Parsing
@@ -445,7 +445,7 @@ The emitter raises these, and so does a parse, which asks the emitter how to spe
| Code | Fires when | What you can do |
| --- | --- | --- |
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node that, written out as plain CommonMark, would be longer than 536870888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536870888 characters, V8's longest string, on every engine; in `adfToPortableMarkdown`, also a document whose text alone is that long, edge whitespace included. In every call: a conversion that passes the longest string or array the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536870888 characters or the engine's limit, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply |
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or one that, written out as plain CommonMark, would be longer than 536870888 characters, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536870888 characters, V8's longest string, on every engine; in `adfToPortableMarkdown`, also a document whose text alone is that long, edge whitespace included. In every call: a conversion that passes the longest string or array the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536870888 characters or the engine's limit, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply |
## The guarantees
@@ -459,7 +459,8 @@ Serves [Goals](#goals) 1, 4 and 5.
- Markdown this library reads, and markdown it writes, means what the CommonMark spec says. The
bullets below name every exception.
- CommonMark is valid input to `commonMarkToAdf`, apart from the raw HTML `unmappable-html` names,
nesting past 500 levels, and input past what the JavaScript engine holds.
nesting past 500 levels, and input whose conversion passes the longest string or array the
JavaScript engine builds.
- The other readers claim shapes CommonMark reads as text or code. `losslessMarkdownToAdf` claims
four: literal text matching directive, pipe-table or strikethrough syntax, and a code fence whose
info string opens `adf:`. `portableMarkdownToAdf` claims what [Portable
+25 -27
View File
@@ -617,9 +617,9 @@ builder as a gain. The scores at be23835 were Junior 6.5, Mid 6, Senior 6 and Ar
6.13. `todo.md` item 102 is to place that path without the replay. Item 106, 2026-10-13, shipped
after three rounds under a worse ruling, the maintainer's call: both paired seats read the overall
as the same, but Locality fell to a mean of 5.13, laid on the spelling memo the chunk splits between
the CommonMark ask, which fills it, and the walk, which drops each entry as it reads it. The scores
at 1f4551a were Junior 5.5, Mid 5.5, Senior 6 and Architect 5.5. `todo.md` item 110 is to fill and
read the memo in one place.
the CommonMark ask, which fills it, and the walk, which drops each entry as it reads it. The overall
scores at 1f4551a were Junior 5.5, Mid 5.5, Senior 6 and Architect 5.5, mean 5.63.
`todo.md` item 110 is to fill and read the memo in one place.
## Properties on a fixed seed
@@ -669,22 +669,20 @@ or pins it.
2026-10-11, the maintainer. Goals 1 and 6. Valid while V8's longest string, 2^29 − 24 characters,
is the shortest of the engines the library claims.
Every writer, and the reader's CommonMark ask, refuses markdown longer than 2^29 − 24 characters
(`longestMarkdown`) as `unsupported-node-shape` at the document's path, so the same document
converts alike everywhere. A writer first refuses a document whose text alone passes the bound: the
lossless writer spells every character of it, and the portable reduction refuses on the caller's
text, edge whitespace and all, since merging that text first could build past the bound. The
builders join plain strings, and each block, the writer's result and the CommonMark ask's spelling
are held to the bound as they are built. V8 throws building the same string past the bound,
SpiderMonkey past its own longest string, and `withinEngineLimits` refuses both throws as the same
`tooLong`, so every engine stops at the same block with the same answer. The carry's two-space JSON,
which indents every value its depth over, stops at the bound unbuilt (`joinedWithin`). A deep list
or quote re-indents its whole text at each level before the bound refuses it (`todo.md` item 109).
Every writer, and `losslessMarkdownToAdf`'s `commonMarkSpelling` ask, refuses markdown longer than
2^29 − 24 characters (`longestMarkdown`), so the same document converts alike everywhere. A writer
first refuses a document whose text alone passes the bound: the lossless writer spells every
character of it, and the portable reduction refuses on the caller's text, edge whitespace and all,
since merging that text first could build past the bound. Each block, the writer's result and the
CommonMark ask's spelling are held to the bound as they are built. V8 throws building the same
string past the bound, SpiderMonkey past its own longest string, and `withinEngineLimits` refuses
both throws as the same `tooLong`, so every engine stops at the same block with the same answer. The
carry's two-space JSON, which indents every value its depth over, stops at the bound unbuilt
(`joinedWithin`).
JavaScriptCore builds a joined block up to its own longest string, 2^31 − 1, before the check. A
throw past that, past any engine's longest array, or JavaScriptCore's `Out of memory`, which also
reports a heap running short, is refused with the engine's message. `todo.md` item 108 bounds the
line's own memory.
JavaScriptCore builds a joined block up to its own longest string, 2^31 − 1, before the check. Past
that, JavaScriptCore throws `Out of memory`, as it does when its heap runs short. That throw, and
any engine's throw past its longest array, is refused with the engine's message.
## Nothing recurses unbounded
@@ -750,15 +748,15 @@ two cannot disagree — which is what makes the kept value a memo rather than a
2026-09-19, the maintainer. Goal 9. Valid while the `commonMarkSpelling` ask spells a node once
per level above it otherwise.
The parse and the portable reduction keep the readable spelling of each node they put the
`commonMarkSpelling` ask to in a memo, so the ask stops spelling a node once per level above it. A
walk that reads an entry drops it, and a walk fills none for the nodes below the asked one: the walk
above spells them into its own text, and kept, every level of a deep subtree would hold its whole
text (2026-10-11). `text`, `alternateText` and `spelling` carry no depth and `headroom` is affine in
it, so a read at or above the depth that filled the entry rebases; a read below re-spells, because a
hit skips the depth guards the walk it replaces runs and an ordered list past the marker cap gives
way, spending two emitter levels where the parser spent one. Only what succeeded is kept, so no path
minted at another position is ever read.
When the parse or the portable reduction asks `commonMarkSpelling` about a node, the node's readable
spelling goes into a memo, so the ask stops spelling a node once per level above it. Reading an
entry drops it, and no entry is kept for the nodes below the asked one: the walk above spells those
nodes into its own text, so keeping them would make every level of a deep subtree hold the subtree's
whole text (2026-10-11). `text`, `alternateText` and `spelling` carry no depth and `headroom` is
affine in it, so a read at or above the depth that filled the entry rebases; a read below re-spells,
because a hit skips the depth guards the walk it replaces runs and an ordered list past the marker
cap gives way, spending two emitter levels where the parser spent one. Only what succeeded is kept,
so no path minted at another position is ever read.
## Cost fixes are measured, never timed
+2 -2
View File
@@ -44,7 +44,7 @@ test('refuses a value that is not an ADF document, naming the check it failed',
assert.equal(markdown(adfToLosslessMarkdown({ type: 'doc', version: Number.NaN })), 'not-an-adf-document: an ADF document holds a version number: found NaN')
})
const tooLong = 'unsupported-node-shape: spelled as markdown, this would be longer than 536870888 characters, the longest markdown this library writes'
const tooLong = 'unsupported-node-shape: the markdown for this document, or for a directive\'s plain CommonMark, would be longer than 536870888 characters, the longest this library writes'
// A code block of 300 lines; two of these together fall some 1000 characters short of V8's longest string, so only what their spelling adds passes it.
function nearlyHalfTheBound(): AdfNode {
@@ -59,7 +59,7 @@ test('refuses a carry whose JSON would run past V8\'s longest string, on every e
assert.equal(markdown(result), tooLong)
})
test('refuses lines their quotes would prefix past V8\'s longest string', () => {
test('refuses a quote whose prefixed lines would pass V8\'s longest string', () => {
const code = nearlyHalfTheBound()
assert.equal(markdown(adfToLosslessMarkdown(document({ content: [code, code], type: 'blockquote' }))), tooLong)
})
+2 -2
View File
@@ -62,7 +62,7 @@ function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: nu
for (const [index, node] of nodes.entries()) {
const block = emitBlock(node, [...path, 'content', index], depth, writing)
if (!block.ok) return block
// V8 throws building a block past the bound; an engine holding longer strings stops at the same block.
// Refused here, so an engine holding longer strings stops at the block where V8 throws.
const fits = withinLongest(block.value.text)
if (!fits.ok) return fits
headroom = Math.min(headroom, block.value.headroom)
@@ -136,7 +136,7 @@ function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, wri
return spellReadableBlock(node, path, depth, writing)
}
// The CommonMark ask fills an entry; the walk above reads it once and spells it into its own text, so kept, every level of a deep subtree would hold its whole text.
// docs/decisions.md §The spelling memo
function takeKeptSpelling(memo: SpellingMemo | undefined, node: AdfNode): KeptSpelling | undefined {
const kept = memo?.get(node)
memo?.delete(node)
+1 -1
View File
@@ -27,7 +27,7 @@ test('refuses a string past the bound as too long on every engine, and names any
assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Invalid string length')), tooLong())
assert.deepEqual(withinEngineLimits(thrown('InternalError', 'allocation size overflow')), tooLong())
assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Out of memory')), {
error: { code: 'unsupported-node-shape', message: 'the output would be longer than the longest string or array this JavaScript engine builds (Out of memory)', path: [] },
error: { code: 'unsupported-node-shape', message: 'the conversion passed this JavaScript engine\'s longest string or array, or its heap (Out of memory)', path: [] },
ok: false,
})
assert.equal(withinEngineLimits(thrown('RangeError', 'Invalid array length')).ok, false)
+5 -5
View File
@@ -1,14 +1,14 @@
import { failure, success, type Result } from '../result.ts'
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
// V8's longest string (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
export const longestMarkdown = 2 ** 29 - 24
// Measured on Firefox 155, Node 24 and Bun 1.4. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the other two name the engine's own limit.
// Measured on Firefox 155, Node 24 and Bun 1.4. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the other two name an engine's own limit, and JavaScriptCore's also a heap running short.
const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length']
const pastEngineLimit = ['RangeError: Invalid array length', 'RangeError: Out of memory']
export function tooLong<T>(): Result<T> {
return failure('unsupported-node-shape', `spelled as markdown, this would be longer than ${longestMarkdown} characters, the longest markdown this library writes`, [])
return failure('unsupported-node-shape', `the markdown for this document, or for a directive's plain CommonMark, would be longer than ${longestMarkdown} characters, the longest this library writes`, [])
}
export function withinLongest(text: string): Result<string> {
@@ -31,11 +31,11 @@ export function withinEngineLimits<T>(convert: () => Result<T>): Result<T> {
try {
return convert()
} catch (error) {
// A stack overflow is a bug (docs/decisions.md §Nothing recurses unbounded), so it stays a throw.
if (!(error instanceof Error)) throw error
const thrown = `${error.name}: ${error.message}`
if (pastLongestString.includes(thrown)) return tooLong()
// A stack overflow is a bug (docs/decisions.md §Nothing recurses unbounded), so it stays a throw.
if (!pastEngineLimit.includes(thrown)) throw error
return failure('unsupported-node-shape', `the output would be longer than the longest string or array this JavaScript engine builds (${error.message})`, [])
return failure('unsupported-node-shape', `the conversion passed this JavaScript engine's longest string or array, or its heap (${error.message})`, [])
}
}
+1 -1
View File
@@ -59,7 +59,7 @@ function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, pat
if (!isJsonValue(spelled) || headroom < 0) {
return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path)
}
// Two-space JSON indents every value its depth over, so it stops at the bound unbuilt.
// Two-space JSON indents each value by its depth, so a short compact value can spell past the bound: join only up to it.
const json = joinedWithin(canonicalJsonPieces(spelled, spelling))
return json.ok ? success({ headroom, json: json.value }) : json
}
+1 -1
View File
@@ -262,7 +262,7 @@ function directiveNode(block: DirectiveBlock, reading: Reading, path: ConvertErr
if (!read.ok) return read
const built = directiveBody(read.value, block.blocks, reading, path, depth)
if (!built.ok) return built
// Guarded here, so V8's throw past the bound refuses at the directive as another engine's check does.
// Refused here so V8's throw past the bound is placed at this directive, like the bound check on other engines.
const readable = withinEngineLimits<null | undefined>(() => commonMarkSpelling(built.value, path, depth, { flavour: 'lossless', memo: reading.memo }) ?? success(undefined))
if (!readable.ok) return readable
if (readable.value === undefined) return built
@@ -60,7 +60,7 @@ export function reduceToPortable(document: AdfDocument): Result<AdfDocument> {
const fault = adfDocumentFault(document)
if (fault !== undefined) return faulted(fault, [])
if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, [])
// Refused on the caller's text, edge whitespace and all: merging it first would build past the bound.
// Measured on the caller's text, edge whitespace and all: `toEditorNormal`'s merge could build past the bound.
if (heldTextLength(nodeContent(document)) > longestMarkdown) return tooLong()
// Two documents the editor holds equal write the same portable markdown (docs/decisions.md §Equality is deep).
const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] })
+8 -7
View File
@@ -143,9 +143,10 @@ audience kept the captioned image's `!adf:caption` directive, 3 of 3, reading a
Item 106 made the CommonMark ask fill the memo and the walk above drop each entry as it reads it, so
a deep subtree holds no level's text twice. Every comprehension seat read that split, across
`commonMarkSpelling`, `readableBlock` and their three callers, as order-dependent state, and the
chunk shipped under a worse ruling on Locality. Keep main's single reader and writer instead: when
`readableBlock` keeps a node's spelling, it drops its children's and grandchildren's entries, whose
text the node's now holds. Found by the comprehension check of item 106, 2026-10-13.
chunk shipped under a worse ruling on Locality. Give the memo one reader and writer again: when
`readableBlock` keeps a node's spelling, it drops the entries of the node's children and
grandchildren, since the node's own spelling now holds their text. Found by the comprehension check
of item 106, 2026-10-13.
### 60. Collect the questions `parse/` asks `emit/` into one named module.
@@ -223,10 +224,10 @@ the reason the project read worse.
Each level of a nested list or quote re-indents its whole text, so the work is the depth times the
text. `losslessMarkdownToAdf` on 3 MB of lazy lines in a list nested 490 levels inside
`!adf:blockquote` spends about 5 seconds re-spelling it. The same 2 MB split over two items of a
list nested 480 levels took 11 minutes to refuse on Node 24: a deep subtree past the length bound
refuses only once every level below has re-indented its text. The writers do the same on such a
document. Found while shipping item 106, 2026-10-11.
`!adf:blockquote` spends about 5 seconds re-spelling it. 2 MB of the same lines, split over two
items of a list nested 480 levels, took 11 minutes to refuse on Node 24: a deep subtree past the
length bound refuses only once every level below has re-indented its text. The writers do the same
on such a document. Found while shipping item 106, 2026-10-11.
### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and markdown to and from HTML composed through ADF, each call named by its flavour.