diff --git a/CHANGELOG.md b/CHANGELOG.md index cc66599..ed289be 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,9 +9,9 @@ code. `MIGRATION.md` says which call to take. - **Breaking:** the markdown readers refuse a value that is not a string with the new code `not-a-string`; `MIGRATION.md` lists what each value did before. -- **Breaking:** every call returns the new code `engine-limit` where the JavaScript engine passes - its longest array, `Map` or `Set`, or reports itself out of memory, and, on JavaScriptCore, where - a string passes its longest. A retry may succeed. Every other refusal stays deterministic. +- **Breaking:** every call returns the new code `engine-limit` where a conversion passes the + JavaScript engine's longest array, `Map` or `Set`, or runs it out of memory, and, on + JavaScriptCore, its longest string. A retry may succeed. Every other refusal stays deterministic. - **Breaking:** an attribute value that is not a plain object, array or JSON primitive, such as a `Date`, `Map`, typed array or class instance, makes `isAdfDocument` return `false` and the converters refuse it as `not-an-adf-document`. Before, the converters wrote it as `{}` or its own @@ -92,14 +92,13 @@ - Fix a directive that, written out as plain CommonMark, would be longer than 536870888 characters: `losslessMarkdownToAdf` refuses it as `unsupported-node-shape`, where a few megabytes of nested list lines ran the call out of memory. -- Fix the markdown readers past the longest string or array the JavaScript engine builds: they - return `unsupported-node-shape` past V8's or SpiderMonkey's longest string, and `engine-limit` - past the rest, where they threw. +- Fix the markdown readers past V8's or SpiderMonkey's longest string: they return + `unsupported-node-shape`, where they threw. - Fix a paragraph of tens of millions of characters: `adfToLosslessMarkdown` and - `adfToPortableMarkdown` write it in a few bytes of memory a character beyond the document and its - markdown, where they took about 50, so 80 million characters ran Node's default heap out of memory - and 20 million `*` threw `RangeError`. `losslessMarkdownToAdf` returns a result for such a - paragraph inside a directive, where it threw `RangeError`. + `adfToPortableMarkdown` need a few bytes a character on top of the document and its markdown, + where they needed about 50. Before, 80 million characters ran Node's default heap out of memory + and 20 million `*` threw `RangeError`. `losslessMarkdownToAdf` no longer throws `RangeError` on + such a paragraph inside a directive. - Fix deeply nested compact JSON in an `adf:` fence: `losslessMarkdownToAdf` refuses it as `unsupported-node-shape`, as it refuses any compact JSON there, where about a megabyte of it threw. - Add `adfToPortableMarkdown` and `portableMarkdownToAdf`, a lossy pair converting ADF to and from diff --git a/MIGRATION.md b/MIGRATION.md index 51537d4..4ddb0e2 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -89,9 +89,9 @@ Markdown the spelling table leaves alone, which `0.2.0` reads as a different doc `unmappable-image`, `unspellable-character`, `unspellable-line-start` and `unspellable-link` leave `ConvertErrorCode`, and `engine-limit` and `not-a-string` join it. A `switch` naming one of the four -stops compiling, and one naming every code needs the two new cases. A document `adfToMarkdown` refused with one of the -three `unspellable-` codes now converts in `adfToLosslessMarkdown`; markdown the parser refused -with one of the four now reads, or refuses with the code the table gives. +stops compiling, and one naming every code needs the two new cases. A document `adfToMarkdown` +refused with one of the three `unspellable-` codes now converts in `adfToLosslessMarkdown`; markdown +the parser refused with one of the four now reads, or refuses with the code the table gives. | Input | `0.1.0` | `0.2.0` | | --- | --- | --- | diff --git a/README.md b/README.md index 31666fa..fb3e17d 100644 --- a/README.md +++ b/README.md @@ -182,7 +182,8 @@ which is free text. - **Bot posting content** — turns generated markdown into ADF. Relies on CommonMark being valid input, so nothing upstream has to learn a flavour. - **Export/indexing tool** — converts ADF to markdown or HTML in bulk. Relies on readable output - and on every refusal being deterministic, so a document that fails fails the same way next run. + and on every refusal but `engine-limit` being deterministic, so a document that fails fails the + same way next run. - **LLM/agent pipeline** — hands documents to a model as markdown and writes the edits back. Relies on the round-trip and on markdown a reader half-knowing the lossless flavour can still edit. @@ -409,11 +410,12 @@ A parse always names a position, so the three markdown readers return `ParseErro typed on `ConvertError` takes both. A call reports the first refusal in document order and stops, so a document with several surfaces -them one per call. Every refusal is deterministic — there is no I/O anywhere — so a retry returns -the identical error: fix the input, or set the document aside. The exception is `engine-limit`, -whose causes include memory running short: retry it once, and treat a repeat as lasting. On V8 -(Node, Deno, Chrome) a heap that runs out ends the process instead. In Node, a `worker_threads` -worker given `resourceLimits` ends alone, so convert input of unbounded size in one. +them one per call. Every refusal but `engine-limit` is deterministic — there is no I/O anywhere — +so a retry returns the identical error: fix the input, or set the document aside. `engine-limit` +can mean memory ran short: retry it once, and treat a repeat as lasting. On V8 (Node, Deno, +Chrome), running out of JavaScript heap ends the process. In Node, convert input of unbounded size +in a `worker_threads` worker given `resourceLimits`: running out of heap there ends only that +worker. ### Parsing @@ -441,7 +443,7 @@ Which reader raises which code: [The calls](#the-calls). Codes every direction r ### Either direction -The emitter raises these, and so does a parse, which asks the emitter how to spell each node: +Every call can raise these: | Code | Fires when | What you can do | | --- | --- | --- | @@ -461,7 +463,8 @@ Serves [Goals](#goals) 1, 4 and 5. - Markdown this library reads, and markdown it writes, means what the CommonMark spec says. The bullets below name every exception. - CommonMark is valid input to `commonMarkToAdf`, apart from the raw HTML `unmappable-html` names, - nesting past 500 levels, and input whose conversion `engine-limit` or the length bound refuses. + nesting past 500 levels, and input whose conversion passes the engine's longest string, refused + as `unsupported-node-shape`, or another engine limit, refused as `engine-limit`. - The other readers claim shapes CommonMark reads as text or code. `losslessMarkdownToAdf` claims four: literal text matching directive, pipe-table or strikethrough syntax, and a code fence whose info string opens `adf:`. `portableMarkdownToAdf` claims what [Portable @@ -488,8 +491,8 @@ Serves [Goals](#goals) 1, 4 and 5. the outer brackets literal text, which the spec requires and the reference itself breaks, nesting one `` in the other. - Not every document converts back: `adfToLosslessMarkdown` refuses a document nesting past 500 - levels, the node shapes the `unsupported-node-shape` row lists, and what `engine-limit` names. Show the refusal and keep the - document read-only; saving markdown you could not produce is the loss the round-trip exists to + levels, the node shapes the `unsupported-node-shape` row lists, and a conversion `engine-limit` + stops. Show the refusal and keep the document read-only; saving markdown you could not produce is the loss the round-trip exists to stop. - The pipe table that `losslessMarkdownToAdf` and `portableMarkdownToAdf` read narrows GFM's twice: every row opens with a pipe, so GFM's bare form is an error result, and an alignment colon in the diff --git a/docs/decisions.md b/docs/decisions.md index 9ef6f8d..2f8b090 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -470,8 +470,8 @@ handles one cause alike whichever node, attribute or direction raised it. - A conversion the engine stops with a throw of its own, past its longest array, `Map` or `Set` or short of memory, is `engine-limit`, at the empty path. A retry may pass the memory case, so `unsupported-node-shape`, which no retry passes, cannot hold the class. JavaScriptCore throws the - same `Out of memory` past its longest string, which takes `engine-limit` too, since only the - message would tell the two apart (2026-10-13, the maintainer). + same `Out of memory` past its longest string as when its heap runs short. Nothing tells the two + apart, so both are `engine-limit` (2026-10-13, the maintainer). - The markdown readers and `htmlToAdf` refuse a value that is not a string as `not-a-string`, at position `{ line: 1, offset: 0 }` with an empty path: no existing code reads true of it, and `not-an-adf-document` names the emitters' input (2026-10-05). @@ -624,7 +624,6 @@ after three rounds under a worse ruling, the maintainer's call: both paired seat as the same, but Locality fell to a mean of 5.13, laid on the spelling memo the chunk splits between the CommonMark ask, which fills it, and the walk, which drops each entry as it reads it. The overall scores at 1f4551a were Junior 5.5, Mid 5.5, Senior 6 and Architect 5.5, mean 5.63. -Item 108's chunk, 2026-10-13, fills and reads the memo in `readableBlock` alone. ## Properties on a fixed seed @@ -686,9 +685,7 @@ carry's two-space JSON, which indents every value its depth over, stops at the b (`joinedWithin`). JavaScriptCore builds a joined block up to its own longest string, 2^31 − 1, before the check. Past -that, JavaScriptCore throws `Out of memory`, as it does when its heap runs short. That throw, and -any engine's throw past its longest array or out of memory, is `engine-limit` (§Which code a cause -takes). +that, JavaScriptCore throws `Out of memory`, which is `engine-limit` (§Which code a cause takes). ## Nothing recurses unbounded diff --git a/src/markdown/emit/adf-to-markdown.test.ts b/src/markdown/emit/adf-to-markdown.test.ts index 389c015..26e0382 100644 --- a/src/markdown/emit/adf-to-markdown.test.ts +++ b/src/markdown/emit/adf-to-markdown.test.ts @@ -70,8 +70,7 @@ test('refuses text past V8\'s longest string', () => { assert.equal(markdown(result), tooLong) }) -// One escape past the most entries V8's Set holds. -test('escapes more characters in a line than a Set holds', () => { +test('escapes more characters in a line than V8\'s Set holds', () => { const stars = '*'.repeat(2 ** 24 + 1) assert.equal(markdown(adfToLosslessMarkdown(document(paragraph({ text: stars, type: 'text' })))), `${'\\*'.repeat(2 ** 24 + 1)}\n`) }) diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index b0b58fb..1a2e09e 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -123,7 +123,7 @@ export function commonMarkSpelling(node: AdfNode, path: ConvertErrorPath, depth: return success(readable.value.spelling !== 'directive') } -// Only an asked node's spelling is kept, and reading it drops it: the walk above spells the node into its own text, so no level of a subtree holds that text twice. +// Only an `asked` call keeps a spelling, and any read drops it: the ancestor that reads it spells it into its own text, so no two levels hold that text. function readableBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing, caller: 'asked' | 'walked'): Result | undefined { const { memo } = writing const kept = memo?.get(node) diff --git a/src/markdown/emit/line-escaping.ts b/src/markdown/emit/line-escaping.ts index 651504e..3e6fd74 100644 --- a/src/markdown/emit/line-escaping.ts +++ b/src/markdown/emit/line-escaping.ts @@ -37,7 +37,7 @@ type Escapings = (index: number) => InlineEscaping | undefined // Where a segment's first character landed in the output. type PlacedSegment = { segment: InlineSegment; start: number } -// The run's escapable characters, or its characters before `end`. +// Which of a run's characters escape: every escapable one, or every one before `end`. type RunEscapes = { end: number; escapable: false } | { escapable: true } type EscapeWalk = { @@ -111,7 +111,7 @@ function escape(segments: readonly InlineSegment[], container: LineContainer, cl } function backslashed(segments: readonly InlineSegment[], scan: string, escaped: Uint8Array): { output: string; placed: PlacedSegment[] } { - // Joined a few thousand pieces at a time: an array element per escape would cost more than the character it escapes. + // The output joins 4096 pieces at a time: an array element per escape would cost more than the character it escapes. const chunks: string[] = [] let pieces: string[] = [] let from = 0 diff --git a/src/markdown/length-bound.test.ts b/src/markdown/length-bound.test.ts index fb7d14f..47eeaf1 100644 --- a/src/markdown/length-bound.test.ts +++ b/src/markdown/length-bound.test.ts @@ -27,7 +27,7 @@ test('refuses a string past the bound as too long on every engine, and any other assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Invalid string length')), tooLong()) assert.deepEqual(withinEngineLimits(thrown('InternalError', 'allocation size overflow')), tooLong()) assert.deepEqual(withinEngineLimits(thrown('RangeError', 'Out of memory')), { - error: { code: 'engine-limit', message: 'the conversion passed this JavaScript engine\'s longest string or array, or ran it out of memory (Out of memory)', path: [] }, + error: { code: 'engine-limit', message: 'the conversion passed a limit of this JavaScript engine on a string, array, Map or Set, or ran it out of memory (Out of memory)', path: [] }, ok: false, }) const code = (message: string): string | undefined => { @@ -43,7 +43,7 @@ test('refuses a string past the bound as too long on every engine, and any other withinEngineLimits(() => { throw 'out of memory' }), - { error: { code: 'engine-limit', message: 'the conversion passed this JavaScript engine\'s longest string or array, or ran it out of memory (out of memory)', path: [] }, ok: false }, + { error: { code: 'engine-limit', message: 'the conversion passed a limit of this JavaScript engine on a string, array, Map or Set, or ran it out of memory (out of memory)', path: [] }, ok: false }, ) assert.deepEqual( withinEngineLimits(() => success('x')), diff --git a/src/markdown/length-bound.ts b/src/markdown/length-bound.ts index cef69ca..b535593 100644 --- a/src/markdown/length-bound.ts +++ b/src/markdown/length-bound.ts @@ -3,7 +3,7 @@ import { failure, success, type Result } from '../result.ts' // V8's longest string, refused on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine). export const longestMarkdown = 2 ** 29 - 24 -// Measured on Firefox 155, Node 24 and Bun 1.4, and SpiderMonkey's array length from its js.msg. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the others name an engine's own limit or its memory running short. +// Measured on Firefox 155, Node 24 and Bun 1.4; SpiderMonkey's `invalid array length` is taken from its js.msg. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the others name an engine's own limit or its memory running short. const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length'] const pastEngineLimit = [ 'RangeError: Array buffer allocation failed', @@ -50,5 +50,5 @@ export function withinEngineLimits(convert: () => Result): Result { } function engineLimit(thrown: string): Result { - return failure('engine-limit', `the conversion passed this JavaScript engine's longest string or array, or ran it out of memory (${thrown})`, []) + return failure('engine-limit', `the conversion passed a limit of this JavaScript engine on a string, array, Map or Set, or ran it out of memory (${thrown})`, []) } diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index c3580cc..2b5ec91 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -341,7 +341,7 @@ test('refuses the directive form of a blockquote opening with a code span', () = assert.equal(content(losslessMarkdownToAdf('!adf:blockquote\n` `` `\n!adf:/blockquote\n')), 'unsupported-node-shape: blockquote takes the CommonMark spelling, not the directive form') }) -// Some 1000 characters short of V8's longest string, which the quote's `> ` on each of its 600 lines passes; reading that much outlasts Bun's 5 s default timeout. +// The input is some 1000 characters short of V8's longest string; the `> ` the quote adds to each of its 600 lines passes it. Reading this much outlasts Bun's 5 s default timeout. test('refuses a blockquote directive whose CommonMark spelling would pass V8\'s longest string', { timeout: 120_000 }, () => { const fence = `\`\`\`\n${`${'a'.repeat(894_782)}\n`.repeat(300)}\`\`\`\n` assert.equal( diff --git a/todo.md b/todo.md index 031a540..b66001e 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 111` +`Next ID = 112` | Goal | W | |---|---| @@ -29,6 +29,7 @@ | 84 | 0.2.0 | | **Read GFM's pipe table without leading pipes, and with alignment colons, in `portableMarkdownToAdf`, dropping the alignment.** | 3 | 4 | 7 | 7 | 4, 7 | 19.4 | | 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 6 | 11.5 | | 53 | 0.2.0 | | **Put a block's `marks` spelling to the writer panel and adopt its pick.** | 4 | 5 | 5 | 6 | 6 | 10.0 | +| 111 | 0.2.1 | | **Split README's `unsupported-node-shape` error-table row into one bullet per cause group, each carrying its remedy.** | 2 | 2 | 5 | 5 | 3 | 15.3 | | 89 | 0.2.1 | | **Build a `date`, `emoji`, `inlineCard`, `mention` or `status` holding no mark Atlassian's schema withholds from it, in every reader.** | 4 | 4 | 5 | 6 | 4 | 13.0 | | 99 | 0.2.1 | defect | **Write a linked external image standing inline or in a table cell in the spelling a reader panel picks, in `adfToPortableMarkdown`.** | 3 | 3 | 6 | 7 | 7 | 12.9 | | 100 | 0.2.1 | decision | **Put the reading of an image's title as its caption back to a reader panel, now that a writer panel reads a title as hover text.** | 3 | 3 | 4 | 5 | 4, 6 | 10.9 | @@ -104,6 +105,13 @@ Today `marks="[{\"attrs\":{\"mode\":\"wide\"},\"type\":\"breakout\"}]"`, the mar escaped JSON. Breaking where the panel picks another spelling: `MIGRATION.md`'s Spellings table gains its row. +### 111. Split README's `unsupported-node-shape` error-table row into one bullet per cause group, each carrying its remedy. + +The row's "Fires when" cell holds about a dozen causes in four groups (parsing, each writer, both +writers, every call) and its "What you can do" cell mixes three remedies, so a reader searches it. +List the groups under the table, each with its remedy. Found by the prose review of item 108, +2026-10-13. + ### 89. Build a `date`, `emoji`, `inlineCard`, `mention` or `status` holding no mark Atlassian's schema withholds from it, in every reader. `*!adf:mention[@a]{id=…}*` and `[!adf:status[Done]{…}](/u)` build a node holding `em` or `link`.