diff --git a/AGENTS.md b/AGENTS.md index 5e12b2d..059d422 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,6 +55,7 @@ In `docs/decisions.md`: - The flavour spec is read as a source - The node tables answer to Atlassian's schema - Nothing recurses unbounded +- JSON text is read by `parseJsonText` - No value `JSON.parse` or `structuredClone` builds loops a call - Nothing spreads an unbounded array - A retry loop checks its own termination diff --git a/CHANGELOG.md b/CHANGELOG.md index 6b02b28..6f6426b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -51,9 +51,10 @@ `collection`, `id` and `occurrenceKey` directly where they rode the opaque carry. - Fix an image inside another image's description: it flattens into the alt text, where it was refused. -- Fix a JSON key holding an escape, in a carried node or a JSON attribute such as `parameters`: on - Node 24, Deno and Chrome, `markdownToAdf` could read it as another key once the process had parsed - JSON holding a key with a backslash ([V8 bug](https://issues.chromium.org/issues/521080746)). +- Fix a JSON key holding an escape, in a carried node or a JSON attribute such as `parameters`: + `markdownToAdf` and `plainMarkdownToAdf` read it as the key it spells, where on V8 13.6 and later + (Node 24, Deno, Chrome) they could refuse canonical markdown as `unsupported-node-shape` once the + process had read a key holding a backslash ([V8 bug](https://issues.chromium.org/issues/521080746)). - Fix a cyclic document, in which an object holds itself: `adfToMarkdown` and `adfToPlainMarkdown` refuse it as `not-an-adf-document` and `isAdfDocument` returns `false`, where all three hung. diff --git a/docs/decisions.md b/docs/decisions.md index 5641cfa..c9f2c8e 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -508,17 +508,17 @@ exits 0 on. while `todo.md` items 59, 60, 61 and 62 are open. A four-seat comprehension panel scored the project under the floor of 7. Its scoring run of -2026-10-05, the third round of item 78's chunk, is the baseline. Since 2026-10-04, a chunk whose -panel scores any dimension lower improves what the seats named and scores again, and after three -such rounds asks the maintainer. +2026-10-08, item 64's chunk, is the baseline. Since 2026-10-04, a chunk whose panel scores any +dimension lower improves what the seats named and scores again, and after three such rounds asks +the maintainer. | Seat | Navigation | Locality | Shape | Self-sufficiency | Overall | |---|---|---|---|---|---| -| Junior | 7 | 5 | 6 | 6 | 6 | -| Mid | 6.5 | 5 | 6 | 5 | 5.5 | -| Senior | 6.5 | 5 | 6 | 5.5 | 6 | -| Architect | 6 | 5.5 | 5.5 | 6 | 6 | -| Mean | 6.50 | 5.13 | 5.88 | 5.63 | 5.88 | +| Junior | 7 | 5.5 | 6.5 | 6 | 6 | +| Mid | 7 | 5.5 | 6.5 | 5.5 | 6 | +| Senior | 7 | 5.5 | 6.5 | 6 | 6 | +| Architect | 6.5 | 6 | 6 | 6 | 6.5 | +| Mean | 6.88 | 5.63 | 6.38 | 5.88 | 6.13 | Panels on near-identical code scored overall means of 5.75 to 6.00, and a seat moves ±0.5 between runs. The maintainer shipped under a fall twice after three rounds: the move to GitHub, @@ -579,6 +579,16 @@ format, as its parser does. A list giving way to the directive form refuses at z than walking again; counting every list twice halved the list limit, counting the directive form once doubled the parser's frames per level. +## JSON text is read by `parseJsonText` + +2026-10-08, item 64, whose remedy the maintainer planned. Goal 1. Valid while engines on a V8 with +https://issues.chromium.org/issues/521080746 open run the library. + +The library reads JSON text with `parseJsonText`, never `JSON.parse`: from V8 13.6, `JSON.parse` +can read an escaped key as another key it read before, and the round-trip then refuses its own +output. A test fails on `JSON.parse(` in a shipped file. The tests' own oracles keep `JSON.parse`, +since a misread there fails the gate and never hides a bug. + ## No value `JSON.parse` or `structuredClone` builds loops a call 2026-10-04, the maintainer. Goals 1 and 9. Valid until `todo.md` item 77 lands. diff --git a/src/json-text.test.ts b/src/json-text.test.ts index 93da487..67f287f 100644 --- a/src/json-text.test.ts +++ b/src/json-text.test.ts @@ -1,5 +1,8 @@ import assert from 'node:assert/strict' +import { dirname, join } from 'node:path' import fc from 'fast-check' +import { fileURLToPath } from 'node:url' +import { readFileSync, readdirSync } from 'node:fs' import test from 'node:test' import type { JsonSpelling } from './canonical-json.ts' @@ -13,48 +16,50 @@ test('reads every JSON value kind', () => { assert.deepEqual(parseJsonText('[ ]'), { value: [] }) assert.deepEqual(parseJsonText('{ }'), { value: {} }) assert.deepEqual(parseJsonText('[[],{}]'), { value: [[], {}] }) - assert.ok(Object.is(parseJsonText('-0')?.value, -0)) + assert.ok(Object.is(parseJsonText('-0').value, -0)) }) test('decodes every string escape', () => { assert.deepEqual(parseJsonText('"\\"\\\\\\/\\b\\f\\n\\r\\t\\u00e9\\uD83C\\uDF89\\ud800"'), { value: '"\\/\b\f\n\r\té\u{1f389}\ud800' }) }) -test('keys stay the keys the text spells after V8 has read a colliding key (https://issues.chromium.org/issues/521080746)', () => { +test('keys stay the keys the text spells after JSON.parse has read a key holding a backslash (https://issues.chromium.org/issues/521080746)', () => { const canary = '{"\\r":1,"\\f":2}' JSON.parse(canary) JSON.parse('{"\\r":3,"\\\\":4}') - assert.deepEqual(Object.keys(parseJsonText(canary)?.value ?? {}), ['\r', '\f']) + assert.deepEqual(Object.keys(parseJsonText(canary).value ?? {}), ['\r', '\f']) }) test('a __proto__ key is an own key', () => { - const read = parseJsonText('{"__proto__":{"a":1}}')?.value + const read = parseJsonText('{"__proto__":{"a":1}}').value assert.deepEqual(Object.keys(read ?? {}), ['__proto__']) assert.equal(Object.getPrototypeOf(read), Object.prototype) }) test('a repeated key keeps its first position and its last value, as JSON.parse does', () => { - const read = parseJsonText('{"a":1,"b":2,"a":3}')?.value + const read = parseJsonText('{"a":1,"b":2,"a":3}').value assert.deepEqual(read, { a: 3, b: 2 }) assert.deepEqual(Object.keys(read ?? {}), ['a', 'b']) }) -test('a number past the double range reads as JSON.parse reads it', () => { - assert.deepEqual(parseJsonText('1e999'), { value: Infinity }) +test('refuses a number past the double range, which JSON.parse reads as Infinity', () => { + assert.deepEqual(parseJsonText('1e999'), { refusal: 'non-finite' }) + assert.deepEqual(parseJsonText('[-1e999]'), { refusal: 'non-finite' }) + assert.deepEqual(parseJsonText('[1e999,'), { refusal: 'syntax' }) }) test('refuses text JSON.parse refuses', () => { - const refused = ['', ' ', '{', '[', '[1', '[1,', '[1,]', '{"a"}', '{"a":}', '{"a":1,}', '{a:1}', '{"a" 1}', '{,}', '01', '1.', '.5', '-', '1e', '+1', '"a', '"\u0001"', '"\\x"', '"\\u12"', '"\\u12g4"', 'tru', 'nul', '1 2', '[1]]', '}', "'a'", 'NaN', ' 1'] + const refused = ['', ' ', '{', '[', '[1', '[1,', '[1,]', '{"a"}', '{"a":}', '{"a":1,}', '{a:1}', '{"a" 1}', '{,}', '01', '1.', '.5', '-', '1e', '+1', '"a', '"\u0001"', '"\\x"', '"\\u12"', '"\\u12g4"', 'tru', 'nul', '1 2', '[1]]', '}', "'a'", 'NaN', '\u00a01'] for (const text of refused) { assert.throws(() => JSON.parse(text), SyntaxError, text) - assert.equal(parseJsonText(text), undefined, text) + assert.deepEqual(parseJsonText(text), { refusal: 'syntax' }, text) } }) test('reads a value nested far past the call stack without recursing', () => { const depth = 100_000 - assert.ok(parseJsonText(`${'['.repeat(depth)}${']'.repeat(depth)}`) !== undefined) - assert.ok(parseJsonText(`${'{"a":'.repeat(depth)}1${'}'.repeat(depth)}`) !== undefined) + assert.ok(parseJsonText(`${'['.repeat(depth)}${']'.repeat(depth)}`).value !== undefined) + assert.ok(parseJsonText(`${'{"a":'.repeat(depth)}1${'}'.repeat(depth)}`).value !== undefined) }) test('reads back every value the canonical serializer writes', () => { @@ -76,8 +81,16 @@ test('accepts exactly the text JSON.parse accepts', () => { } catch { accepted = false } - assert.equal(parseJsonText(text) !== undefined, accepted) + assert.equal(parseJsonText(text).refusal !== 'syntax', accepted) }), propertyRuns(2000), ) }) + +test('no shipped file reads JSON with JSON.parse, which parseJsonText stands in for', () => { + const sourceRoot = dirname(fileURLToPath(import.meta.url)) + const readers = readdirSync(sourceRoot, { encoding: 'utf8', recursive: true }).filter( + (name) => name.endsWith('.ts') && !name.endsWith('.test.ts') && !name.startsWith('conformance') && readFileSync(join(sourceRoot, name), 'utf8').includes('JSON.parse('), + ) + assert.deepEqual(readers, []) +}) diff --git a/src/json-text.ts b/src/json-text.ts index ec5eefb..af614cd 100644 --- a/src/json-text.ts +++ b/src/json-text.ts @@ -2,8 +2,16 @@ import type { JsonValue } from './json-value.ts' type Frame = { entries: [string, JsonValue][]; key: string; kind: 'object' } | { items: JsonValue[]; kind: 'array' } +type Item = { end: number; frame: Frame; kind: 'opened' } | { end: number; kind: 'value'; value: JsonValue } | { kind: 'refused' } + +export type JsonReading = { refusal: JsonRefusal; value?: undefined } | { refusal?: undefined; value: JsonValue } + +type JsonRefusal = 'non-finite' | 'syntax' + type Scanned = { end: number; value: T } | undefined +type State = { expect: 'separator'; index: number; value: JsonValue } | { expect: 'value'; index: number } + const escapes = new Map([ ['"', '"'], ['/', '/'], @@ -21,57 +29,52 @@ const literals = new Map([ ['true', true], ]) const numberSyntax = /-?(?:0|[1-9][0-9]*)(?:\.[0-9]+)?(?:[Ee][+-]?[0-9]+)?/y +const syntaxRefusal: Item = { kind: 'refused' } -// In place of JSON.parse: V8 reads an escaped key against keys earlier parses saw (https://issues.chromium.org/issues/521080746). -export function parseJsonText(text: string): { value: JsonValue } | undefined { +// In place of JSON.parse: V8 can read an escaped key as another key it read before (https://issues.chromium.org/issues/521080746). +export function parseJsonText(text: string): JsonReading { const frames: Frame[] = [] - let index = skipWhitespace(text, 0) + let overflowed = false + let state: State = { expect: 'value', index: skipWhitespace(text, 0) } for (;;) { - const opened = openValue(text, index, frames) - if (opened === undefined) return undefined - let { end, value } = opened - if (value === undefined) { - index = end + if (state.expect === 'value') { + const item = readItem(text, state.index) + if (item.kind === 'refused') return { refusal: 'syntax' } + if (item.kind === 'opened') frames.push(item.frame) + else if (typeof item.value === 'number' && !Number.isFinite(item.value)) overflowed = true + state = item.kind === 'opened' ? { expect: 'value', index: item.end } : { expect: 'separator', index: skipWhitespace(text, item.end), value: item.value } continue } - for (;;) { - index = skipWhitespace(text, end) - const frame = frames[frames.length - 1] - if (frame === undefined) return index === text.length ? { value } : undefined - if (frame.kind === 'array') frame.items.push(value) - else frame.entries.push([frame.key, value]) - const separator = text.charAt(index) - if (separator === ',') break - if (separator !== (frame.kind === 'array' ? ']' : '}')) return undefined - frames.pop() - end = index + 1 - value = frame.kind === 'array' ? frame.items : Object.fromEntries(frame.entries) - } + const index: number = state.index + const value = state.value const frame = frames[frames.length - 1] - index = skipWhitespace(text, index + 1) - if (frame?.kind === 'object') { - const key = readKey(text, index) - if (key === undefined) return undefined - frame.key = key.value - index = key.end + if (frame === undefined) return index !== text.length ? { refusal: 'syntax' } : overflowed ? { refusal: 'non-finite' } : { value } + if (frame.kind === 'array') frame.items.push(value) + else frame.entries.push([frame.key, value]) + const separator = text.charAt(index) + if (separator === (frame.kind === 'array' ? ']' : '}')) { + frames.pop() + state = { expect: 'separator', index: skipWhitespace(text, index + 1), value: frame.kind === 'array' ? frame.items : Object.fromEntries(frame.entries) } + continue } + if (separator !== ',') return { refusal: 'syntax' } + state = { expect: 'value', index: skipWhitespace(text, index + 1) } + if (frame.kind === 'array') continue + const key = readKey(text, state.index) + if (key === undefined) return { refusal: 'syntax' } + frame.key = key.value + state = { expect: 'value', index: key.end } } } -// A value whose `value` is undefined opened a container, and `end` is where its first member starts. -function openValue(text: string, index: number, frames: Frame[]): Scanned { +function readItem(text: string, index: number): Item { const opener = text.charAt(index) if (opener !== '[' && opener !== '{') return readScalar(text, index) const first = skipWhitespace(text, index + 1) - if (text.charAt(first) === (opener === '[' ? ']' : '}')) return { end: first + 1, value: opener === '[' ? [] : {} } - if (opener === '[') { - frames.push({ items: [], kind: 'array' }) - return { end: first, value: undefined } - } + if (text.charAt(first) === (opener === '[' ? ']' : '}')) return { end: first + 1, kind: 'value', value: opener === '[' ? [] : {} } + if (opener === '[') return { end: first, frame: { items: [], kind: 'array' }, kind: 'opened' } const key = readKey(text, first) - if (key === undefined) return undefined - frames.push({ entries: [], key: key.value, kind: 'object' }) - return { end: key.end, value: undefined } + return key === undefined ? syntaxRefusal : { end: key.end, frame: { entries: [], key: key.value, kind: 'object' }, kind: 'opened' } } function readKey(text: string, index: number): Scanned { @@ -81,12 +84,15 @@ function readKey(text: string, index: number): Scanned { return text.charAt(colon) === ':' ? { end: skipWhitespace(text, colon + 1), value: key.value } : undefined } -function readScalar(text: string, index: number): Scanned { - if (text.charAt(index) === '"') return readString(text, index) - for (const [spelling, value] of literals) if (text.startsWith(spelling, index)) return { end: index + spelling.length, value } +function readScalar(text: string, index: number): Item { + if (text.charAt(index) === '"') { + const string = readString(text, index) + return string === undefined ? syntaxRefusal : { end: string.end, kind: 'value', value: string.value } + } + for (const [spelling, value] of literals) if (text.startsWith(spelling, index)) return { end: index + spelling.length, kind: 'value', value } numberSyntax.lastIndex = index const number = numberSyntax.exec(text)?.[0] - return number === undefined ? undefined : { end: index + number.length, value: Number(number) } + return number === undefined ? syntaxRefusal : { end: index + number.length, kind: 'value', value: Number(number) } } function readString(text: string, start: number): Scanned { diff --git a/src/markdown/directive-syntax.ts b/src/markdown/directive-syntax.ts index 7e5c28a..3b75314 100644 --- a/src/markdown/directive-syntax.ts +++ b/src/markdown/directive-syntax.ts @@ -3,7 +3,7 @@ import type { ConvertFault } from '../result.ts' import type { JsonValue } from '../json-value.ts' import { backslashEscape, holdsNullCharacter } from './commonmark/grammar.ts' import { backtickRun, closingBacktickRun } from './commonmark/backtick-runs.ts' -import { isJsonValue, overNested } from '../json-value.ts' +import { overNested } from '../json-value.ts' import { largestNesting } from '../nesting.ts' import { parseJsonText } from '../json-text.ts' import { serializeCanonicalJson } from '../canonical-json.ts' @@ -61,7 +61,7 @@ const prefixFault = `an unescaped ${directivePrefix} completes no directive; ${d export function attributeValue(text: string, kind: AttributeKind): AttributeReading { if (kind === 'string') return { value: { kind, value: text } } if (kind === 'boolean') return text === 'true' || text === 'false' ? { value: { kind, value: text === 'true' } } : { refusal: 'kind' } - const parsed = parseJson(text) + const parsed = parseJsonText(text).value if (parsed === undefined) return { refusal: 'kind' } if (kind === 'number') return typeof parsed === 'number' ? { value: { kind, value: parsed } } : { refusal: 'kind' } return overNested(parsed) ? { refusal: 'nesting' } : { value: { kind, value: parsed } } @@ -325,12 +325,8 @@ function readQuotedValue(text: string, index: number): Read<{ end: number; value if (character !== undefined) { return { fault: malformedDirective(`a raw ${character} inside {attrs} breaks the directive: spell it \\u${escapeDigits(character)}`) } } - const parsed = parseJson(spelling) + const parsed = parseJsonText(spelling).value if (typeof parsed !== 'string') return { fault: malformedDirective('the {attrs} quoted value is not a JSON string') } return { value: { end: cursor + 1, value: { decoded: parsed, spelling } } } } -function parseJson(raw: string): JsonValue | undefined { - const value = parseJsonText(raw)?.value - return value !== undefined && isJsonValue(value) ? value : undefined -} diff --git a/src/markdown/opaque-carry.ts b/src/markdown/opaque-carry.ts index 6f00b52..ef32c59 100644 --- a/src/markdown/opaque-carry.ts +++ b/src/markdown/opaque-carry.ts @@ -65,9 +65,10 @@ function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, pat function readCarriedJson(raw: string, spelling: JsonSpelling, levels: number, type?: string): Read { const wayOut = type === undefined ? directiveEscape : fenceEscape(type) const parsed = parseJsonText(raw) - if (parsed === undefined) return { fault: malformedDirective(`the opaque carry holds invalid JSON; ${wayOut}`) } + if (parsed.refusal !== undefined) { + return { fault: parsed.refusal === 'syntax' ? malformedDirective(`the opaque carry holds invalid JSON; ${wayOut}`) : unsupportedNodeShape('the opaque carry holds a number JSON cannot spell') } + } const { value } = parsed - if (!isJsonValue(value)) return { fault: unsupportedNodeShape('the opaque carry holds a number JSON cannot spell') } const typed = type === undefined || type === '' ? { value } : typedValue(value, type) if (typed.fault !== undefined) return typed const held = typed.value diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index fea1acb..c3e9624 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -405,6 +405,13 @@ test('reads the inline carry back to the node its json attribute holds', () => { ]) }) +test('reads an escaped JSON key as the key it spells after reading a key holding a backslash (https://issues.chromium.org/issues/521080746)', () => { + const extension = (parameters: string): string => `!adf:extension {extensionKey=toc extensionType=x parameters=${JSON.stringify(parameters)}}\n` + const read = content(markdownToAdf([extension('{"\\b":1,"\\f":2}'), extension('{"\\b":3,"\\\\":4}'), extension('{"\\b":1,"\\f":2}')].join('\n'))) + const node = (parameters: AdfAttributes): AdfNode => ({ attrs: { extensionKey: 'toc', extensionType: 'x', parameters }, type: 'extension' }) + assert.deepEqual(read, [node({ '\b': 1, '\f': 2 }), node({ '\b': 3, '\\': 4 }), node({ '\b': 1, '\f': 2 })]) +}) + test('names the invalid JSON no opaque carry holds, and the spelling that keeps it literal', () => { const invalid = 'malformed-directive: the opaque carry holds invalid JSON; ' const fence = `${invalid}!adf:codeBlock {language="adf:x"} around a bare fence keeps it a code block` diff --git a/todo.md b/todo.md index f202678..2f09cda 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 83` +`Next ID = 84` | Goal | W | |---|---| @@ -36,6 +36,7 @@ | 45 | 0.3.0 | | **Replace `isAdfDocument` with a reader returning `Result`.** | 2 | 3 | 6 | 8 | 1 | 25.2 | | 6 | 0.3.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 4 | 24.7 | | 77 | 0.3.0 | decision | **Read the input once into a plain copy, so an accessor property or a throwing Proxy yields `not-an-adf-document` and every call returns a result.** | 4 | 4 | 5 | 9 | 1 | 23.0 | +| 83 | 0.3.0 | defect | **Refuse with a result, never a `RangeError`, a document whose markdown would pass the engine's string limit.** | 3 | 4 | 5 | 8 | 1, 9 | 22.0 | | 71 | 0.3.0 | | **Add a README table comparing the package with the other ADF converters, each cell checked against that package's own docs or behaviour.** | 1 | 3 | 6 | 7 | 3 | 21.2 | | 80 | 0.3.0 | | **Name the node's path and the attribute key when `not-an-adf-document` refuses a node inside the document.** | 2 | 3 | 5 | 6 | 1 | 19.2 | | 68 | 0.3.0 | | **Carry a non-text inline node holding `content` or `text`, a block holding `text`, and a leaf block holding `content`.** | 3 | 3 | 4 | 7 | 1 | 18.2 | @@ -169,6 +170,14 @@ left, so a Proxy claiming 2³² elements is refused at once. The chunk asks whet shares the 500-level guard's `unsupported-nesting-depth` or takes another code. Found by the technical-principles audit, 2026-10-04. +### 83. Refuse with a result, never a `RangeError`, a document whose markdown would pass the engine's string limit. + +The block carry indents its JSON two spaces a level, so one unknown block holding 497 nested arrays +around 540,000 numbers, about 1.1 MB as JSON, spells about 540 million characters, past V8's +2^29 − 24. `serializeCanonicalJson`'s `join` throws, and `adfToMarkdown` throws with it. Any output +past the limit does the same. Refuse it with a code from the closed list, and add a fixture. Found +by the README-goals audit, 2026-10-08. + ### 71. Add a README table comparing the package with the other ADF converters, each cell checked against that package's own docs or behaviour. Lands after items 7 and 46: the table names HTML and compares item 46's size figure. Its columns: