mirror of
https://github.com/larvit/adf-codec.git
synced 2026-11-12 11:48:29 +01:00
Answer prose round 1: the refusal messages name what passed the bound, the decisions read one idea a sentence, and the changelog gives the readers' fix its own bullet
This commit is contained in:
+8
-7
@@ -83,13 +83,14 @@
|
||||
- Fix a cyclic document, in which an object holds itself: `adfToLosslessMarkdown` and
|
||||
`adfToPortableMarkdown` refuse it as `not-an-adf-document` and `isAdfDocument` returns `false`,
|
||||
where all three hung.
|
||||
- Fix a document whose markdown would be longer than 536,870,888 characters, V8's longest string:
|
||||
`adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape` on
|
||||
every engine, where they threw or ran out of memory.
|
||||
- Fix a directive that, written out as plain CommonMark, would be that long: `losslessMarkdownToAdf`
|
||||
refuses it as `unsupported-node-shape`, where a few megabytes of nested list lines ran it out of
|
||||
memory. The markdown readers return `unsupported-node-shape` where they threw past the longest
|
||||
string or array the JavaScript engine builds.
|
||||
- Fix a document whose markdown would be longer than 536870888 characters, V8's longest string:
|
||||
`adfToLosslessMarkdown` and `adfToPortableMarkdown` refuse it as `unsupported-node-shape` on every
|
||||
engine, where they threw or ran out of memory.
|
||||
- Fix a directive that, written out as plain CommonMark, would be longer than 536870888 characters:
|
||||
`losslessMarkdownToAdf` refuses it as `unsupported-node-shape`, where a few megabytes of nested
|
||||
list lines ran the call out of memory.
|
||||
- Fix the markdown readers past the longest string or array the JavaScript engine builds: they
|
||||
return `unsupported-node-shape` where they threw.
|
||||
- Fix deeply nested compact JSON in an `adf:` fence: `losslessMarkdownToAdf` refuses it as
|
||||
`unsupported-node-shape`, as it refuses any compact JSON there, where about a megabyte of it threw.
|
||||
- Add `adfToPortableMarkdown` and `portableMarkdownToAdf`, a lossy pair converting ADF to and from
|
||||
|
||||
@@ -410,9 +410,9 @@ typed on `ConvertError` takes both.
|
||||
|
||||
A call reports the first refusal in document order and stops, so a document with several surfaces
|
||||
them one per call. Every refusal is deterministic — there is no I/O anywhere — so a retry returns
|
||||
the identical error: fix the input, or set the document aside. The one exception is JavaScriptCore
|
||||
(Bun, Safari), which reports a heap running short as it reports a string past its longest: there,
|
||||
retry once a call's `unsupported-node-shape` at the empty path.
|
||||
the identical error: fix the input, or set the document aside. On JavaScriptCore (Bun, Safari),
|
||||
retry once any call that returns `unsupported-node-shape` at the empty path: that engine reports a
|
||||
heap running short the same way as a string too long to build.
|
||||
|
||||
### Parsing
|
||||
|
||||
@@ -445,7 +445,7 @@ The emitter raises these, and so does a parse, which asks the emitter how to spe
|
||||
| Code | Fires when | What you can do |
|
||||
| --- | --- | --- |
|
||||
| `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it |
|
||||
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document, or writes as a directive a node that, written out as plain CommonMark, would be longer than 536870888 characters. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536870888 characters, V8's longest string, on every engine; in `adfToPortableMarkdown`, also a document whose text alone is that long, edge whitespace included. In every call: a conversion that passes the longest string or array the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536870888 characters or the engine's limit, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply |
|
||||
| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or one that, written out as plain CommonMark, would be longer than 536870888 characters, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document. Emitting, in `adfToLosslessMarkdown`: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content`. Emitting, in `adfToPortableMarkdown`: a line or task list the portable writer cannot spell, a bug in this library. Emitting, in both writers: a document whose markdown would be longer than 536870888 characters, V8's longest string, on every engine; in `adfToPortableMarkdown`, also a document whose text alone is that long, edge whitespace included. In every call: a conversion that passes the longest string or array the JavaScript engine builds | fix what the message names; [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md) lists every type's attributes and body. For a portable line or task list, report it as a bug. Past 536870888 characters or the engine's limit, writing: set the document aside, keeping the ADF and passing it over or showing it read-only; reading: shorten the markdown, or nest its lists and quotes less deeply |
|
||||
|
||||
## The guarantees
|
||||
|
||||
@@ -459,7 +459,8 @@ Serves [Goals](#goals) 1, 4 and 5.
|
||||
- Markdown this library reads, and markdown it writes, means what the CommonMark spec says. The
|
||||
bullets below name every exception.
|
||||
- CommonMark is valid input to `commonMarkToAdf`, apart from the raw HTML `unmappable-html` names,
|
||||
nesting past 500 levels, and input past what the JavaScript engine holds.
|
||||
nesting past 500 levels, and input whose conversion passes the longest string or array the
|
||||
JavaScript engine builds.
|
||||
- The other readers claim shapes CommonMark reads as text or code. `losslessMarkdownToAdf` claims
|
||||
four: literal text matching directive, pipe-table or strikethrough syntax, and a code fence whose
|
||||
info string opens `adf:`. `portableMarkdownToAdf` claims what [Portable
|
||||
|
||||
+25
-27
@@ -617,9 +617,9 @@ builder as a gain. The scores at be23835 were Junior 6.5, Mid 6, Senior 6 and Ar
|
||||
6.13. `todo.md` item 102 is to place that path without the replay. Item 106, 2026-10-13, shipped
|
||||
after three rounds under a worse ruling, the maintainer's call: both paired seats read the overall
|
||||
as the same, but Locality fell to a mean of 5.13, laid on the spelling memo the chunk splits between
|
||||
the CommonMark ask, which fills it, and the walk, which drops each entry as it reads it. The scores
|
||||
at 1f4551a were Junior 5.5, Mid 5.5, Senior 6 and Architect 5.5. `todo.md` item 110 is to fill and
|
||||
read the memo in one place.
|
||||
the CommonMark ask, which fills it, and the walk, which drops each entry as it reads it. The overall
|
||||
scores at 1f4551a were Junior 5.5, Mid 5.5, Senior 6 and Architect 5.5, mean 5.63.
|
||||
`todo.md` item 110 is to fill and read the memo in one place.
|
||||
|
||||
## Properties on a fixed seed
|
||||
|
||||
@@ -669,22 +669,20 @@ or pins it.
|
||||
2026-10-11, the maintainer. Goals 1 and 6. Valid while V8's longest string, 2^29 − 24 characters,
|
||||
is the shortest of the engines the library claims.
|
||||
|
||||
Every writer, and the reader's CommonMark ask, refuses markdown longer than 2^29 − 24 characters
|
||||
(`longestMarkdown`) as `unsupported-node-shape` at the document's path, so the same document
|
||||
converts alike everywhere. A writer first refuses a document whose text alone passes the bound: the
|
||||
lossless writer spells every character of it, and the portable reduction refuses on the caller's
|
||||
text, edge whitespace and all, since merging that text first could build past the bound. The
|
||||
builders join plain strings, and each block, the writer's result and the CommonMark ask's spelling
|
||||
are held to the bound as they are built. V8 throws building the same string past the bound,
|
||||
SpiderMonkey past its own longest string, and `withinEngineLimits` refuses both throws as the same
|
||||
`tooLong`, so every engine stops at the same block with the same answer. The carry's two-space JSON,
|
||||
which indents every value its depth over, stops at the bound unbuilt (`joinedWithin`). A deep list
|
||||
or quote re-indents its whole text at each level before the bound refuses it (`todo.md` item 109).
|
||||
Every writer, and `losslessMarkdownToAdf`'s `commonMarkSpelling` ask, refuses markdown longer than
|
||||
2^29 − 24 characters (`longestMarkdown`), so the same document converts alike everywhere. A writer
|
||||
first refuses a document whose text alone passes the bound: the lossless writer spells every
|
||||
character of it, and the portable reduction refuses on the caller's text, edge whitespace and all,
|
||||
since merging that text first could build past the bound. Each block, the writer's result and the
|
||||
CommonMark ask's spelling are held to the bound as they are built. V8 throws building the same
|
||||
string past the bound, SpiderMonkey past its own longest string, and `withinEngineLimits` refuses
|
||||
both throws as the same `tooLong`, so every engine stops at the same block with the same answer. The
|
||||
carry's two-space JSON, which indents every value its depth over, stops at the bound unbuilt
|
||||
(`joinedWithin`).
|
||||
|
||||
JavaScriptCore builds a joined block up to its own longest string, 2^31 − 1, before the check. A
|
||||
throw past that, past any engine's longest array, or JavaScriptCore's `Out of memory`, which also
|
||||
reports a heap running short, is refused with the engine's message. `todo.md` item 108 bounds the
|
||||
line's own memory.
|
||||
JavaScriptCore builds a joined block up to its own longest string, 2^31 − 1, before the check. Past
|
||||
that, JavaScriptCore throws `Out of memory`, as it does when its heap runs short. That throw, and
|
||||
any engine's throw past its longest array, is refused with the engine's message.
|
||||
|
||||
## Nothing recurses unbounded
|
||||
|
||||
@@ -750,15 +748,15 @@ two cannot disagree — which is what makes the kept value a memo rather than a
|
||||
2026-09-19, the maintainer. Goal 9. Valid while the `commonMarkSpelling` ask spells a node once
|
||||
per level above it otherwise.
|
||||
|
||||
The parse and the portable reduction keep the readable spelling of each node they put the
|
||||
`commonMarkSpelling` ask to in a memo, so the ask stops spelling a node once per level above it. A
|
||||
walk that reads an entry drops it, and a walk fills none for the nodes below the asked one: the walk
|
||||
above spells them into its own text, and kept, every level of a deep subtree would hold its whole
|
||||
text (2026-10-11). `text`, `alternateText` and `spelling` carry no depth and `headroom` is affine in
|
||||
it, so a read at or above the depth that filled the entry rebases; a read below re-spells, because a
|
||||
hit skips the depth guards the walk it replaces runs and an ordered list past the marker cap gives
|
||||
way, spending two emitter levels where the parser spent one. Only what succeeded is kept, so no path
|
||||
minted at another position is ever read.
|
||||
When the parse or the portable reduction asks `commonMarkSpelling` about a node, the node's readable
|
||||
spelling goes into a memo, so the ask stops spelling a node once per level above it. Reading an
|
||||
entry drops it, and no entry is kept for the nodes below the asked one: the walk above spells those
|
||||
nodes into its own text, so keeping them would make every level of a deep subtree hold the subtree's
|
||||
whole text (2026-10-11). `text`, `alternateText` and `spelling` carry no depth and `headroom` is
|
||||
affine in it, so a read at or above the depth that filled the entry rebases; a read below re-spells,
|
||||
because a hit skips the depth guards the walk it replaces runs and an ordered list past the marker
|
||||
cap gives way, spending two emitter levels where the parser spent one. Only what succeeded is kept,
|
||||
so no path minted at another position is ever read.
|
||||
|
||||
## Cost fixes are measured, never timed
|
||||
|
||||
|
||||
@@ -44,7 +44,7 @@ test('refuses a value that is not an ADF document, naming the check it failed',
|
||||
assert.equal(markdown(adfToLosslessMarkdown({ type: 'doc', version: Number.NaN })), 'not-an-adf-document: an ADF document holds a version number: found NaN')
|
||||
})
|
||||
|
||||
const tooLong = 'unsupported-node-shape: spelled as markdown, this would be longer than 536870888 characters, the longest markdown this library writes'
|
||||
const tooLong = 'unsupported-node-shape: the markdown for this document, or for a directive\'s plain CommonMark, would be longer than 536870888 characters, the longest this library writes'
|
||||
|
||||
// A code block of 300 lines; two of these together fall some 1000 characters short of V8's longest string, so only what their spelling adds passes it.
|
||||
function nearlyHalfTheBound(): AdfNode {
|
||||
@@ -59,7 +59,7 @@ test('refuses a carry whose JSON would run past V8\'s longest string, on every e
|
||||
assert.equal(markdown(result), tooLong)
|
||||
})
|
||||
|
||||
test('refuses lines their quotes would prefix past V8\'s longest string', () => {
|
||||
test('refuses a quote whose prefixed lines would pass V8\'s longest string', () => {
|
||||
const code = nearlyHalfTheBound()
|
||||
assert.equal(markdown(adfToLosslessMarkdown(document({ content: [code, code], type: 'blockquote' }))), tooLong)
|
||||
})
|
||||
|
||||
@@ -62,7 +62,7 @@ function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: nu
|
||||
for (const [index, node] of nodes.entries()) {
|
||||
const block = emitBlock(node, [...path, 'content', index], depth, writing)
|
||||
if (!block.ok) return block
|
||||
// V8 throws building a block past the bound; an engine holding longer strings stops at the same block.
|
||||
// Refused here, so an engine holding longer strings stops at the block where V8 throws.
|
||||
const fits = withinLongest(block.value.text)
|
||||
if (!fits.ok) return fits
|
||||
headroom = Math.min(headroom, block.value.headroom)
|
||||
@@ -136,7 +136,7 @@ function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, wri
|
||||
return spellReadableBlock(node, path, depth, writing)
|
||||
}
|
||||
|
||||
// The CommonMark ask fills an entry; the walk above reads it once and spells it into its own text, so kept, every level of a deep subtree would hold its whole text.
|
||||
// docs/decisions.md §The spelling memo
|
||||
function takeKeptSpelling(memo: SpellingMemo | undefined, node: AdfNode): KeptSpelling | undefined {
|
||||
const kept = memo?.get(node)
|
||||
memo?.delete(node)
|
||||
|
||||
@@ -27,7 +27,7 @@ test('refuses a string past the bound as too long on every engine, and names any
|
||||
assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Invalid string length')), tooLong())
|
||||
assert.deepEqual(withinEngineLimits(thrown('InternalError', 'allocation size overflow')), tooLong())
|
||||
assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Out of memory')), {
|
||||
error: { code: 'unsupported-node-shape', message: 'the output would be longer than the longest string or array this JavaScript engine builds (Out of memory)', path: [] },
|
||||
error: { code: 'unsupported-node-shape', message: 'the conversion passed this JavaScript engine\'s longest string or array, or its heap (Out of memory)', path: [] },
|
||||
ok: false,
|
||||
})
|
||||
assert.equal(withinEngineLimits(thrown('RangeError', 'Invalid array length')).ok, false)
|
||||
|
||||
@@ -1,14 +1,14 @@
|
||||
import { failure, success, type Result } from '../result.ts'
|
||||
|
||||
// V8's longest string, held on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
|
||||
// V8's longest string (docs/decisions.md §Markdown past V8's longest string is refused on every engine).
|
||||
export const longestMarkdown = 2 ** 29 - 24
|
||||
|
||||
// Measured on Firefox 155, Node 24 and Bun 1.4. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the other two name the engine's own limit.
|
||||
// Measured on Firefox 155, Node 24 and Bun 1.4. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the other two name an engine's own limit, and JavaScriptCore's also a heap running short.
|
||||
const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length']
|
||||
const pastEngineLimit = ['RangeError: Invalid array length', 'RangeError: Out of memory']
|
||||
|
||||
export function tooLong<T>(): Result<T> {
|
||||
return failure('unsupported-node-shape', `spelled as markdown, this would be longer than ${longestMarkdown} characters, the longest markdown this library writes`, [])
|
||||
return failure('unsupported-node-shape', `the markdown for this document, or for a directive's plain CommonMark, would be longer than ${longestMarkdown} characters, the longest this library writes`, [])
|
||||
}
|
||||
|
||||
export function withinLongest(text: string): Result<string> {
|
||||
@@ -31,11 +31,11 @@ export function withinEngineLimits<T>(convert: () => Result<T>): Result<T> {
|
||||
try {
|
||||
return convert()
|
||||
} catch (error) {
|
||||
// A stack overflow is a bug (docs/decisions.md §Nothing recurses unbounded), so it stays a throw.
|
||||
if (!(error instanceof Error)) throw error
|
||||
const thrown = `${error.name}: ${error.message}`
|
||||
if (pastLongestString.includes(thrown)) return tooLong()
|
||||
// A stack overflow is a bug (docs/decisions.md §Nothing recurses unbounded), so it stays a throw.
|
||||
if (!pastEngineLimit.includes(thrown)) throw error
|
||||
return failure('unsupported-node-shape', `the output would be longer than the longest string or array this JavaScript engine builds (${error.message})`, [])
|
||||
return failure('unsupported-node-shape', `the conversion passed this JavaScript engine's longest string or array, or its heap (${error.message})`, [])
|
||||
}
|
||||
}
|
||||
|
||||
@@ -59,7 +59,7 @@ function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, pat
|
||||
if (!isJsonValue(spelled) || headroom < 0) {
|
||||
return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path)
|
||||
}
|
||||
// Two-space JSON indents every value its depth over, so it stops at the bound unbuilt.
|
||||
// Two-space JSON indents each value by its depth, so a short compact value can spell past the bound: join only up to it.
|
||||
const json = joinedWithin(canonicalJsonPieces(spelled, spelling))
|
||||
return json.ok ? success({ headroom, json: json.value }) : json
|
||||
}
|
||||
|
||||
@@ -262,7 +262,7 @@ function directiveNode(block: DirectiveBlock, reading: Reading, path: ConvertErr
|
||||
if (!read.ok) return read
|
||||
const built = directiveBody(read.value, block.blocks, reading, path, depth)
|
||||
if (!built.ok) return built
|
||||
// Guarded here, so V8's throw past the bound refuses at the directive as another engine's check does.
|
||||
// Refused here so V8's throw past the bound is placed at this directive, like the bound check on other engines.
|
||||
const readable = withinEngineLimits<null | undefined>(() => commonMarkSpelling(built.value, path, depth, { flavour: 'lossless', memo: reading.memo }) ?? success(undefined))
|
||||
if (!readable.ok) return readable
|
||||
if (readable.value === undefined) return built
|
||||
|
||||
@@ -60,7 +60,7 @@ export function reduceToPortable(document: AdfDocument): Result<AdfDocument> {
|
||||
const fault = adfDocumentFault(document)
|
||||
if (fault !== undefined) return faulted(fault, [])
|
||||
if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, [])
|
||||
// Refused on the caller's text, edge whitespace and all: merging it first would build past the bound.
|
||||
// Measured on the caller's text, edge whitespace and all: `toEditorNormal`'s merge could build past the bound.
|
||||
if (heldTextLength(nodeContent(document)) > longestMarkdown) return tooLong()
|
||||
// Two documents the editor holds equal write the same portable markdown (docs/decisions.md §Equality is deep).
|
||||
const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] })
|
||||
|
||||
@@ -143,9 +143,10 @@ audience kept the captioned image's `!adf:caption` directive, 3 of 3, reading a
|
||||
Item 106 made the CommonMark ask fill the memo and the walk above drop each entry as it reads it, so
|
||||
a deep subtree holds no level's text twice. Every comprehension seat read that split, across
|
||||
`commonMarkSpelling`, `readableBlock` and their three callers, as order-dependent state, and the
|
||||
chunk shipped under a worse ruling on Locality. Keep main's single reader and writer instead: when
|
||||
`readableBlock` keeps a node's spelling, it drops its children's and grandchildren's entries, whose
|
||||
text the node's now holds. Found by the comprehension check of item 106, 2026-10-13.
|
||||
chunk shipped under a worse ruling on Locality. Give the memo one reader and writer again: when
|
||||
`readableBlock` keeps a node's spelling, it drops the entries of the node's children and
|
||||
grandchildren, since the node's own spelling now holds their text. Found by the comprehension check
|
||||
of item 106, 2026-10-13.
|
||||
|
||||
### 60. Collect the questions `parse/` asks `emit/` into one named module.
|
||||
|
||||
@@ -223,10 +224,10 @@ the reason the project read worse.
|
||||
|
||||
Each level of a nested list or quote re-indents its whole text, so the work is the depth times the
|
||||
text. `losslessMarkdownToAdf` on 3 MB of lazy lines in a list nested 490 levels inside
|
||||
`!adf:blockquote` spends about 5 seconds re-spelling it. The same 2 MB split over two items of a
|
||||
list nested 480 levels took 11 minutes to refuse on Node 24: a deep subtree past the length bound
|
||||
refuses only once every level below has re-indented its text. The writers do the same on such a
|
||||
document. Found while shipping item 106, 2026-10-11.
|
||||
`!adf:blockquote` spends about 5 seconds re-spelling it. 2 MB of the same lines, split over two
|
||||
items of a list nested 480 levels, took 11 minutes to refuse on Node 24: a deep subtree past the
|
||||
length bound refuses only once every level below has re-indented its text. The writers do the same
|
||||
on such a document. Found while shipping item 106, 2026-10-11.
|
||||
|
||||
### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and markdown to and from HTML composed through ADF, each call named by its flavour.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user