From bfc4b2ce00b9f10ddfe9fc7b823765db78cd880d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:03:20 +0200 Subject: [PATCH 01/67] Name the markdown's readers and writers in the audience, and the panel rule --- AGENTS.md | 6 +++++- README.md | 4 ++++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index 25442ed..e954f90 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -162,12 +162,16 @@ nearest text, a candidate entry in that file's voice, and the instance it yields keeps collecting instances is wrong: rewrite it. Which output the audience expects — README goal 5 — is settled by a reader panel rather than -asked: three fresh-context readers, one per README persona the conversion serves, each given only +asked: three fresh-context readers, one per README persona the question serves, each given only `## Audience` and the input, writing what they expect before picking among outputs the goals allow, rendered, shuffled, with no rationale and nothing saying what is implemented. Three agreeing settle it; otherwise four more read, five of seven settle it, and less is a missing goal, asked. The verdict lands in `docs/decisions.md`. +Every new or changed markdown or HTML spelling goes to such a panel, seated by the people who read +and write that format (README `## Audience`); a question about what an app relies on seats the +developer personas. + ### Stated numbers A stated number — 500 levels, the branch floor — is kept; a chunk that cannot keep it asks, naming diff --git a/README.md b/README.md index 13cac24..f164f6b 100644 --- a/README.md +++ b/README.md @@ -52,6 +52,10 @@ which is free text. - **LLM/agent pipeline** — hands documents to a model as markdown and writes the edits back. Relies on the round-trip and on markdown a reader half-knowing the lossless flavour can still edit. +Behind those apps, the people who read and write the markdown, and later the HTML: product +managers, engineers and support agents working in Atlassian products through a plugin or another +UI. They know markdown and not ADF, and rely on every spelling saying what it means to them. + ## The shape ```sh -- 2.52.0 From 9ab7286e6d5f48b678f0261ba3a68b0025d5f2dd Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:25:01 +0200 Subject: [PATCH 02/67] Make markdownToAdf(adfToMarkdown(doc)) deep-equal doc: textBreak, empty keys, -0, typed carry fences, code blocks of several text nodes --- browser-tests/convert-corpus.js | 9 +- browser-tests/run.js | 7 +- .../errors/carry-fence-type-spellable.error | 1 + corpus/errors/carry-fence-type-spellable.md | 5 + corpus/errors/carry-fence-type.error | 1 + corpus/errors/carry-fence-type.md | 5 + corpus/errors/carry-invalid-json.md | 4 +- .../errors/code-block-fence-languages.error | 1 + corpus/errors/code-block-fence-languages.md | 8 + corpus/errors/document-directive-bare.error | 1 + corpus/errors/document-directive-bare.md | 1 + .../errors/document-directive-misplaced.error | 1 + corpus/errors/document-directive-misplaced.md | 3 + .../errors/empty-attrs-with-attributes.error | 1 + corpus/errors/empty-attrs-with-attributes.md | 3 + corpus/errors/empty-content-with-body.error | 1 + corpus/errors/empty-content-with-body.md | 3 + corpus/errors/empty-key-value.error | 1 + corpus/errors/empty-key-value.md | 3 + corpus/errors/empty-marks-wrapped.error | 1 + corpus/errors/empty-marks-wrapped.md | 1 + corpus/errors/text-break-attributes.error | 1 + corpus/errors/text-break-attributes.md | 1 + corpus/errors/text-break-content.error | 1 + corpus/errors/text-break-content.md | 1 + corpus/errors/text-break-marks-differ.error | 1 + corpus/errors/text-break-marks-differ.md | 1 + corpus/errors/text-break-misplaced.error | 1 + corpus/errors/text-break-misplaced.md | 1 + .../block-nodes/code-block-carry.json | 14 +- .../block-nodes/code-block-carry.md | 12 +- .../block-nodes/code-block-text-nodes.json | 56 +++++++ .../block-nodes/code-block-text-nodes.md | 31 ++++ .../block-nodes/document-without-content.json | 4 + .../block-nodes/document-without-content.md | 1 + corpus/round-trip/block-nodes/empty-keys.json | 89 ++++++++++ corpus/round-trip/block-nodes/empty-keys.md | 32 ++++ .../combinations/carry-attributes.md | 10 +- .../combinations/negative-zero.json | 83 ++++++++++ .../round-trip/combinations/negative-zero.md | 21 +++ .../commonmark-subset/document-empty.json | 1 + .../round-trip/inline-nodes/empty-keys.json | 98 +++++++++++ corpus/round-trip/inline-nodes/empty-keys.md | 9 + .../round-trip/inline-nodes/text-break.json | 154 ++++++++++++++++++ corpus/round-trip/inline-nodes/text-break.md | 13 ++ .../opaque-carry/carry-in-wrappers.md | 10 +- .../opaque-carry/code-block-children.json | 35 ++++ .../opaque-carry/code-block-children.md | 32 ++++ .../round-trip/opaque-carry/unknown-block.md | 10 +- .../opaque-carry/unknown-type-spelling.json | 21 +++ .../opaque-carry/unknown-type-spelling.md | 24 +++ src/adf/document.ts | 28 +++- src/adf/editor-normal.ts | 4 +- src/canonical-json.ts | 2 +- src/conformance/adf-property.test.ts | 3 +- src/conformance/corpus.test.ts | 9 +- src/conformance/markdown-property.test.ts | 3 +- src/conformance/property-harness.ts | 69 +++++--- src/markdown/block-directive.ts | 14 +- src/markdown/code-language.ts | 8 +- src/markdown/commonmark/grammar.ts | 7 +- src/markdown/directive-syntax.ts | 2 +- src/markdown/emit/adf-to-markdown.test.ts | 85 +++++----- src/markdown/emit/adf-to-markdown.ts | 54 +++--- src/markdown/emit/block-directive-spelling.ts | 3 +- .../emit/inline-directive-spelling.ts | 3 +- src/markdown/emit/inline-line.ts | 36 ++-- src/markdown/emit/plain-reduction.test.ts | 2 +- src/markdown/emit/plain-reduction.ts | 9 +- src/markdown/empty-keys.ts | 30 ++++ src/markdown/mark-spellings.ts | 10 +- src/markdown/opaque-carry.ts | 43 +++-- src/markdown/parse/directive-marks.ts | 8 +- src/markdown/parse/directive-nodes.ts | 41 +++-- src/markdown/parse/inline-content.ts | 86 +++++++--- src/markdown/parse/markdown-to-adf.test.ts | 57 +++++-- src/markdown/parse/markdown-to-adf.ts | 48 ++++-- src/markdown/text-break.ts | 16 ++ 78 files changed, 1263 insertions(+), 246 deletions(-) create mode 100644 corpus/errors/carry-fence-type-spellable.error create mode 100644 corpus/errors/carry-fence-type-spellable.md create mode 100644 corpus/errors/carry-fence-type.error create mode 100644 corpus/errors/carry-fence-type.md create mode 100644 corpus/errors/code-block-fence-languages.error create mode 100644 corpus/errors/code-block-fence-languages.md create mode 100644 corpus/errors/document-directive-bare.error create mode 100644 corpus/errors/document-directive-bare.md create mode 100644 corpus/errors/document-directive-misplaced.error create mode 100644 corpus/errors/document-directive-misplaced.md create mode 100644 corpus/errors/empty-attrs-with-attributes.error create mode 100644 corpus/errors/empty-attrs-with-attributes.md create mode 100644 corpus/errors/empty-content-with-body.error create mode 100644 corpus/errors/empty-content-with-body.md create mode 100644 corpus/errors/empty-key-value.error create mode 100644 corpus/errors/empty-key-value.md create mode 100644 corpus/errors/empty-marks-wrapped.error create mode 100644 corpus/errors/empty-marks-wrapped.md create mode 100644 corpus/errors/text-break-attributes.error create mode 100644 corpus/errors/text-break-attributes.md create mode 100644 corpus/errors/text-break-content.error create mode 100644 corpus/errors/text-break-content.md create mode 100644 corpus/errors/text-break-marks-differ.error create mode 100644 corpus/errors/text-break-marks-differ.md create mode 100644 corpus/errors/text-break-misplaced.error create mode 100644 corpus/errors/text-break-misplaced.md create mode 100644 corpus/round-trip/block-nodes/code-block-text-nodes.json create mode 100644 corpus/round-trip/block-nodes/code-block-text-nodes.md create mode 100644 corpus/round-trip/block-nodes/document-without-content.json create mode 100644 corpus/round-trip/block-nodes/document-without-content.md create mode 100644 corpus/round-trip/block-nodes/empty-keys.json create mode 100644 corpus/round-trip/block-nodes/empty-keys.md create mode 100644 corpus/round-trip/combinations/negative-zero.json create mode 100644 corpus/round-trip/combinations/negative-zero.md create mode 100644 corpus/round-trip/inline-nodes/empty-keys.json create mode 100644 corpus/round-trip/inline-nodes/empty-keys.md create mode 100644 corpus/round-trip/inline-nodes/text-break.json create mode 100644 corpus/round-trip/inline-nodes/text-break.md create mode 100644 corpus/round-trip/opaque-carry/code-block-children.json create mode 100644 corpus/round-trip/opaque-carry/code-block-children.md create mode 100644 corpus/round-trip/opaque-carry/unknown-type-spelling.json create mode 100644 corpus/round-trip/opaque-carry/unknown-type-spelling.md create mode 100644 src/markdown/empty-keys.ts create mode 100644 src/markdown/text-break.ts diff --git a/browser-tests/convert-corpus.js b/browser-tests/convert-corpus.js index 28a6808..3b535bc 100644 --- a/browser-tests/convert-corpus.js +++ b/browser-tests/convert-corpus.js @@ -1,16 +1,19 @@ try { const { adfToMarkdown, isAdfDocument, markdownToAdf } = await import('/dist/index.js') + const { serializeCanonicalJson } = await import('/dist/canonical-json.js') + // WebDriver's JSON reads -0 back as 0, so a document crosses as its canonical spelling. + const spelled = (result) => (result.ok ? { ok: true, value: `${serializeCanonicalJson(result.value, 'two-space')}\n` } : result) window.convertCorpus = (corpus) => ({ errors: corpus.errors.map(({ markdown, name }) => ({ name, parsed: markdownToAdf(markdown) })), - normalization: corpus.normalization.map(({ markdown, name }) => ({ name, parsed: markdownToAdf(markdown) })), + normalization: corpus.normalization.map(({ markdown, name }) => ({ name, parsed: spelled(markdownToAdf(markdown)) })), realPayloads: corpus.realPayloads.map(({ json, name }) => { const adf = JSON.parse(json) const emitted = adfToMarkdown(adf) - return { emitted, isDocument: isAdfDocument(adf), name, parsed: emitted.ok ? markdownToAdf(emitted.value) : undefined } + return { emitted, isDocument: isAdfDocument(adf), name, parsed: emitted.ok ? spelled(markdownToAdf(emitted.value)) : undefined } }), roundTrip: corpus.roundTrip.map(({ json, markdown, name }) => { const adf = JSON.parse(json) - return { emitted: adfToMarkdown(adf), isDocument: isAdfDocument(adf), name, parsed: markdownToAdf(markdown) } + return { emitted: adfToMarkdown(adf), isDocument: isAdfDocument(adf), name, parsed: spelled(markdownToAdf(markdown)) } }), }) } catch (cause) { diff --git a/browser-tests/run.js b/browser-tests/run.js index 0369402..6739064 100644 --- a/browser-tests/run.js +++ b/browser-tests/run.js @@ -2,7 +2,6 @@ import assert from 'node:assert/strict' import { createServer } from 'node:http' import { extname, join } from 'node:path' import { readFileSync, readdirSync } from 'node:fs' -import { toEditorNormal } from '../dist/adf/editor-normal.js' const contentTypes = { '.html': 'text/html; charset=utf-8', '.js': 'text/javascript' } const driver = 'http://127.0.0.1:4444' @@ -107,7 +106,7 @@ for (const [index, { json, markdown, name }] of corpus.roundTrip.entries()) { assert.ok(result.emitted.ok, `it did not emit — ${refusal(result.emitted)}`) assert.equal(result.emitted.value, markdown) assert.ok(result.parsed.ok, `it did not parse — ${refusal(result.parsed)}`) - assert.deepEqual(toEditorNormal(result.parsed.value), JSON.parse(json)) + assert.equal(result.parsed.value, json) }) } @@ -115,7 +114,7 @@ for (const [index, { name }] of corpus.normalization.entries()) { const result = results.normalization[index] checking(name, result, () => { assert.ok(result.parsed.ok, `it did not parse — ${refusal(result.parsed)}`) - assert.deepEqual(toEditorNormal(result.parsed.value), JSON.parse(fixture(name, '.json'))) + assert.equal(result.parsed.value, fixture(name, '.json')) }) } @@ -125,7 +124,7 @@ for (const [index, { json, name }] of corpus.realPayloads.entries()) { assert.ok(result.isDocument, `${name}.json is no ADF document`) assert.ok(result.emitted.ok, `it did not emit — ${refusal(result.emitted)}`) assert.ok(result.parsed.ok, `it did not parse back — ${refusal(result.parsed)}`) - assert.deepEqual(toEditorNormal(result.parsed.value), JSON.parse(json)) + assert.equal(result.parsed.value, json) }) } diff --git a/corpus/errors/carry-fence-type-spellable.error b/corpus/errors/carry-fence-type-spellable.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/carry-fence-type-spellable.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/carry-fence-type-spellable.md b/corpus/errors/carry-fence-type-spellable.md new file mode 100644 index 0000000..1b73fc1 --- /dev/null +++ b/corpus/errors/carry-fence-type-spellable.md @@ -0,0 +1,5 @@ +```adf: +{ + "type": "blockCard" +} +``` diff --git a/corpus/errors/carry-fence-type.error b/corpus/errors/carry-fence-type.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/carry-fence-type.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/carry-fence-type.md b/corpus/errors/carry-fence-type.md new file mode 100644 index 0000000..7a266ef --- /dev/null +++ b/corpus/errors/carry-fence-type.md @@ -0,0 +1,5 @@ +```adf:blockCard +{ + "type": "blockCard" +} +``` diff --git a/corpus/errors/carry-invalid-json.md b/corpus/errors/carry-invalid-json.md index 1cd3435..febfe45 100644 --- a/corpus/errors/carry-invalid-json.md +++ b/corpus/errors/carry-invalid-json.md @@ -1,3 +1,3 @@ -```carry -{"type": +```adf:blockCard +{"attrs": ``` diff --git a/corpus/errors/code-block-fence-languages.error b/corpus/errors/code-block-fence-languages.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/code-block-fence-languages.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/code-block-fence-languages.md b/corpus/errors/code-block-fence-languages.md new file mode 100644 index 0000000..077c903 --- /dev/null +++ b/corpus/errors/code-block-fence-languages.md @@ -0,0 +1,8 @@ +!adf:codeBlock +```js +a +``` +```ts +b +``` +!adf:/codeBlock diff --git a/corpus/errors/document-directive-bare.error b/corpus/errors/document-directive-bare.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/document-directive-bare.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/document-directive-bare.md b/corpus/errors/document-directive-bare.md new file mode 100644 index 0000000..82cf095 --- /dev/null +++ b/corpus/errors/document-directive-bare.md @@ -0,0 +1 @@ +!adf:doc diff --git a/corpus/errors/document-directive-misplaced.error b/corpus/errors/document-directive-misplaced.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/document-directive-misplaced.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/document-directive-misplaced.md b/corpus/errors/document-directive-misplaced.md new file mode 100644 index 0000000..f3158b4 --- /dev/null +++ b/corpus/errors/document-directive-misplaced.md @@ -0,0 +1,3 @@ +!adf:doc {content=none} + +Text. diff --git a/corpus/errors/empty-attrs-with-attributes.error b/corpus/errors/empty-attrs-with-attributes.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/empty-attrs-with-attributes.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/empty-attrs-with-attributes.md b/corpus/errors/empty-attrs-with-attributes.md new file mode 100644 index 0000000..560ecc7 --- /dev/null +++ b/corpus/errors/empty-attrs-with-attributes.md @@ -0,0 +1,3 @@ +!adf:panel info {attrs=empty} +Text. +!adf:/panel diff --git a/corpus/errors/empty-content-with-body.error b/corpus/errors/empty-content-with-body.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/empty-content-with-body.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/empty-content-with-body.md b/corpus/errors/empty-content-with-body.md new file mode 100644 index 0000000..648b150 --- /dev/null +++ b/corpus/errors/empty-content-with-body.md @@ -0,0 +1,3 @@ +!adf:paragraph {content=empty} +Text. +!adf:/paragraph diff --git a/corpus/errors/empty-key-value.error b/corpus/errors/empty-key-value.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/empty-key-value.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/empty-key-value.md b/corpus/errors/empty-key-value.md new file mode 100644 index 0000000..eadca54 --- /dev/null +++ b/corpus/errors/empty-key-value.md @@ -0,0 +1,3 @@ +!adf:paragraph {attrs=none} +Text. +!adf:/paragraph diff --git a/corpus/errors/empty-marks-wrapped.error b/corpus/errors/empty-marks-wrapped.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/empty-marks-wrapped.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/empty-marks-wrapped.md b/corpus/errors/empty-marks-wrapped.md new file mode 100644 index 0000000..8824f00 --- /dev/null +++ b/corpus/errors/empty-marks-wrapped.md @@ -0,0 +1 @@ +**!adf:date{marks=empty}** diff --git a/corpus/errors/text-break-attributes.error b/corpus/errors/text-break-attributes.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/text-break-attributes.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/text-break-attributes.md b/corpus/errors/text-break-attributes.md new file mode 100644 index 0000000..d4f7006 --- /dev/null +++ b/corpus/errors/text-break-attributes.md @@ -0,0 +1 @@ +a!adf:textBreak{x=y}b diff --git a/corpus/errors/text-break-content.error b/corpus/errors/text-break-content.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/text-break-content.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/text-break-content.md b/corpus/errors/text-break-content.md new file mode 100644 index 0000000..c5b44ef --- /dev/null +++ b/corpus/errors/text-break-content.md @@ -0,0 +1 @@ +a!adf:textBreak[x]b diff --git a/corpus/errors/text-break-marks-differ.error b/corpus/errors/text-break-marks-differ.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/text-break-marks-differ.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/text-break-marks-differ.md b/corpus/errors/text-break-marks-differ.md new file mode 100644 index 0000000..4a8f869 --- /dev/null +++ b/corpus/errors/text-break-marks-differ.md @@ -0,0 +1 @@ +**a**!adf:textBreak{}b diff --git a/corpus/errors/text-break-misplaced.error b/corpus/errors/text-break-misplaced.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/text-break-misplaced.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/text-break-misplaced.md b/corpus/errors/text-break-misplaced.md new file mode 100644 index 0000000..e84475f --- /dev/null +++ b/corpus/errors/text-break-misplaced.md @@ -0,0 +1 @@ +!adf:textBreak{}Hello diff --git a/corpus/round-trip/block-nodes/code-block-carry.json b/corpus/round-trip/block-nodes/code-block-carry.json index a5bbd7d..9981d4e 100644 --- a/corpus/round-trip/block-nodes/code-block-carry.json +++ b/corpus/round-trip/block-nodes/code-block-carry.json @@ -14,7 +14,19 @@ }, { "attrs": { - "language": "carry" + "language": "adf:blockCard" + }, + "content": [ + { + "text": "{\n \"attrs\": {}\n}", + "type": "text" + } + ], + "type": "codeBlock" + }, + { + "attrs": { + "language": "adf:" }, "content": [ { diff --git a/corpus/round-trip/block-nodes/code-block-carry.md b/corpus/round-trip/block-nodes/code-block-carry.md index 5a032f7..665bd3c 100644 --- a/corpus/round-trip/block-nodes/code-block-carry.md +++ b/corpus/round-trip/block-nodes/code-block-carry.md @@ -1,12 +1,18 @@ -!adf:codeBlock {language=carry} -``` +```carry { "type": "blockCard" } ``` + +!adf:codeBlock {language="adf:blockCard"} +``` +{ + "attrs": {} +} +``` !adf:/codeBlock -!adf:codeBlock {language=carry} +!adf:codeBlock {language="adf:"} ```` ``` ```` diff --git a/corpus/round-trip/block-nodes/code-block-text-nodes.json b/corpus/round-trip/block-nodes/code-block-text-nodes.json new file mode 100644 index 0000000..37554b6 --- /dev/null +++ b/corpus/round-trip/block-nodes/code-block-text-nodes.json @@ -0,0 +1,56 @@ +{ + "content": [ + { + "attrs": { + "language": "js" + }, + "content": [ + { + "text": "const a = 1\n", + "type": "text" + }, + { + "text": "const b = 2", + "type": "text" + } + ], + "type": "codeBlock" + }, + { + "content": [ + { + "text": "a", + "type": "text" + }, + { + "text": "```", + "type": "text" + }, + { + "text": "\n", + "type": "text" + } + ], + "type": "codeBlock" + }, + { + "attrs": { + "language": "has`tick", + "wrap": true + }, + "content": [ + { + "text": "x", + "type": "text" + }, + { + "text": " y ", + "type": "text" + } + ], + "type": "codeBlock" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/block-nodes/code-block-text-nodes.md b/corpus/round-trip/block-nodes/code-block-text-nodes.md new file mode 100644 index 0000000..cb87dbf --- /dev/null +++ b/corpus/round-trip/block-nodes/code-block-text-nodes.md @@ -0,0 +1,31 @@ +!adf:codeBlock +```js +const a = 1 + +``` +```js +const b = 2 +``` +!adf:/codeBlock + +!adf:codeBlock +``` +a +``` +```` +``` +```` +``` + + +``` +!adf:/codeBlock + +!adf:codeBlock {language="has\u0060tick" wrap=true} +``` +x +``` +``` + y +``` +!adf:/codeBlock diff --git a/corpus/round-trip/block-nodes/document-without-content.json b/corpus/round-trip/block-nodes/document-without-content.json new file mode 100644 index 0000000..6591ef5 --- /dev/null +++ b/corpus/round-trip/block-nodes/document-without-content.json @@ -0,0 +1,4 @@ +{ + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/block-nodes/document-without-content.md b/corpus/round-trip/block-nodes/document-without-content.md new file mode 100644 index 0000000..e23047f --- /dev/null +++ b/corpus/round-trip/block-nodes/document-without-content.md @@ -0,0 +1 @@ +!adf:doc {content=none} diff --git a/corpus/round-trip/block-nodes/empty-keys.json b/corpus/round-trip/block-nodes/empty-keys.json new file mode 100644 index 0000000..3fcc6d2 --- /dev/null +++ b/corpus/round-trip/block-nodes/empty-keys.json @@ -0,0 +1,89 @@ +{ + "content": [ + { + "content": [], + "type": "paragraph" + }, + { + "attrs": {}, + "content": [ + { + "text": "Plain words.", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [], + "type": "rule" + }, + { + "attrs": { + "level": 2 + }, + "content": [ + { + "text": "Title", + "type": "text" + } + ], + "marks": [], + "type": "heading" + }, + { + "attrs": { + "panelType": "info" + }, + "content": [], + "type": "panel" + }, + { + "content": [ + { + "attrs": {}, + "content": [ + { + "content": [ + { + "text": "Item", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + }, + { + "content": [ + { + "content": [ + { + "text": "Next", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "bulletList" + }, + { + "content": [], + "type": "codeBlock" + }, + { + "attrs": { + "language": "js" + }, + "marks": [], + "type": "codeBlock" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/block-nodes/empty-keys.md b/corpus/round-trip/block-nodes/empty-keys.md new file mode 100644 index 0000000..04299da --- /dev/null +++ b/corpus/round-trip/block-nodes/empty-keys.md @@ -0,0 +1,32 @@ +!adf:paragraph {content=empty} +!adf:/paragraph + +!adf:paragraph {attrs=empty} +Plain words. +!adf:/paragraph + +!adf:rule {content=empty} + +!adf:heading {level=2 marks=empty} +Title +!adf:/heading + +!adf:panel info {content=empty} +!adf:/panel + +!adf:bulletList +!adf:listItem {attrs=empty} +Item +!adf:/listItem +!adf:listItem +Next +!adf:/listItem +!adf:/bulletList + +!adf:codeBlock {content=empty} +!adf:/codeBlock + +!adf:codeBlock {marks=empty} +```js +``` +!adf:/codeBlock diff --git a/corpus/round-trip/combinations/carry-attributes.md b/corpus/round-trip/combinations/carry-attributes.md index 088dbe9..605ffbd 100644 --- a/corpus/round-trip/combinations/carry-attributes.md +++ b/corpus/round-trip/combinations/carry-attributes.md @@ -1,4 +1,4 @@ -```carry +```adf:panel { "attrs": { "rounded": true @@ -13,12 +13,11 @@ ], "type": "paragraph" } - ], - "type": "panel" + ] } ``` -```carry +```adf:panel { "attrs": { "panelType": "extra info" @@ -33,8 +32,7 @@ ], "type": "paragraph" } - ], - "type": "panel" + ] } ``` diff --git a/corpus/round-trip/combinations/negative-zero.json b/corpus/round-trip/combinations/negative-zero.json new file mode 100644 index 0000000..2ffed45 --- /dev/null +++ b/corpus/round-trip/combinations/negative-zero.json @@ -0,0 +1,83 @@ +{ + "content": [ + { + "attrs": { + "order": -0 + }, + "content": [ + { + "content": [ + { + "content": [ + { + "text": "First", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "orderedList" + }, + { + "attrs": { + "level": -0 + }, + "content": [ + { + "text": "Title", + "type": "text" + } + ], + "type": "heading" + }, + { + "attrs": { + "extensionKey": "toc", + "extensionType": "com.atlassian.confluence.macro.core", + "parameters": { + "maxLevel": -0 + } + }, + "type": "extension" + }, + { + "content": [ + { + "attrs": { + "data": [ + -0 + ], + "url": "https://example.com/" + }, + "type": "inlineCard" + }, + { + "marks": [ + { + "attrs": { + "color": "#000000", + "size": -0 + }, + "type": "border" + } + ], + "text": "x", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "attrs": { + "x": -0 + }, + "type": "blockCard" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/combinations/negative-zero.md b/corpus/round-trip/combinations/negative-zero.md new file mode 100644 index 0000000..766f709 --- /dev/null +++ b/corpus/round-trip/combinations/negative-zero.md @@ -0,0 +1,21 @@ +!adf:orderedList {order=-0} +!adf:listItem +First +!adf:/listItem +!adf:/orderedList + +!adf:heading {level=-0} +Title +!adf:/heading + +!adf:extension {extensionKey=toc extensionType="com.atlassian.confluence.macro.core" parameters="{\"maxLevel\":-0}"} + +!adf:inlineCard{data="[-0]" url="https://example.com/"}!adf:border[x]{color="#000000" size=-0} + +```adf:blockCard +{ + "attrs": { + "x": -0 + } +} +``` diff --git a/corpus/round-trip/commonmark-subset/document-empty.json b/corpus/round-trip/commonmark-subset/document-empty.json index 6591ef5..317b19d 100644 --- a/corpus/round-trip/commonmark-subset/document-empty.json +++ b/corpus/round-trip/commonmark-subset/document-empty.json @@ -1,4 +1,5 @@ { + "content": [], "type": "doc", "version": 1 } diff --git a/corpus/round-trip/inline-nodes/empty-keys.json b/corpus/round-trip/inline-nodes/empty-keys.json new file mode 100644 index 0000000..a5b1623 --- /dev/null +++ b/corpus/round-trip/inline-nodes/empty-keys.json @@ -0,0 +1,98 @@ +{ + "content": [ + { + "content": [ + { + "attrs": {}, + "type": "date" + }, + { + "text": " ", + "type": "text" + }, + { + "content": [], + "type": "date" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "text": "a", + "type": "text" + }, + { + "marks": [], + "type": "hardBreak" + }, + { + "text": "b", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "attrs": { + "id": "x", + "text": "@M" + }, + "content": [], + "type": "mention" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "text": "bare", + "type": "text" + }, + { + "marks": [], + "text": "marked", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "attrs": {}, + "type": "em" + } + ], + "text": "x", + "type": "text" + }, + { + "text": " and ", + "type": "text" + }, + { + "attrs": { + "shortName": ":a:" + }, + "marks": [ + { + "attrs": {}, + "type": "strong" + } + ], + "type": "emoji" + } + ], + "type": "paragraph" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/inline-nodes/empty-keys.md b/corpus/round-trip/inline-nodes/empty-keys.md new file mode 100644 index 0000000..db86fe8 --- /dev/null +++ b/corpus/round-trip/inline-nodes/empty-keys.md @@ -0,0 +1,9 @@ +!adf:date{attrs=empty} !adf:date{content=empty} + +a!adf:hardBreak{marks=empty}b + +!adf:mention[@M]{content=empty id=x} + +bare!adf:carry{json="{\"marks\":[],\"text\":\"marked\",\"type\":\"text\"}"} + +!adf:carry{json="{\"marks\":[{\"attrs\":{},\"type\":\"em\"}],\"text\":\"x\",\"type\":\"text\"}"} and !adf:carry{json="{\"attrs\":{\"shortName\":\":a:\"},\"marks\":[{\"attrs\":{},\"type\":\"strong\"}],\"type\":\"emoji\"}"} diff --git a/corpus/round-trip/inline-nodes/text-break.json b/corpus/round-trip/inline-nodes/text-break.json new file mode 100644 index 0000000..f7e56f4 --- /dev/null +++ b/corpus/round-trip/inline-nodes/text-break.json @@ -0,0 +1,154 @@ +{ + "content": [ + { + "content": [ + { + "text": "Hello, ", + "type": "text" + }, + { + "text": "world", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "type": "strong" + } + ], + "text": "Hello, ", + "type": "text" + }, + { + "marks": [ + { + "type": "strong" + } + ], + "text": "world", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "type": "code" + } + ], + "text": "a", + "type": "text" + }, + { + "marks": [ + { + "type": "code" + } + ], + "text": "b", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "attrs": { + "href": "https://example.com/" + }, + "type": "link" + } + ], + "text": "a", + "type": "text" + }, + { + "marks": [ + { + "attrs": { + "href": "https://example.com/" + }, + "type": "link" + } + ], + "text": "b", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "text": "Line\n", + "type": "text" + }, + { + "text": "next", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "type": "underline" + } + ], + "text": "a", + "type": "text" + }, + { + "marks": [ + { + "type": "underline" + } + ], + "text": "b", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "attrs": {}, + "type": "underline" + } + ], + "text": "a", + "type": "text" + }, + { + "marks": [ + { + "type": "underline" + } + ], + "text": "b", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/inline-nodes/text-break.md b/corpus/round-trip/inline-nodes/text-break.md new file mode 100644 index 0000000..71144ea --- /dev/null +++ b/corpus/round-trip/inline-nodes/text-break.md @@ -0,0 +1,13 @@ +Hello, !adf:textBreak{}world + +**Hello, !adf:textBreak{}world** + +`a`!adf:textBreak{}`b` + +[a!adf:textBreak{}b](https://example.com/) + +Line!adf:text{text="\n"}!adf:textBreak{}next + +!adf:underline[a!adf:textBreak{}b] + +!adf:underline[a]{attrs=empty}!adf:underline[b] diff --git a/corpus/round-trip/opaque-carry/carry-in-wrappers.md b/corpus/round-trip/opaque-carry/carry-in-wrappers.md index df32328..fb4de61 100644 --- a/corpus/round-trip/opaque-carry/carry-in-wrappers.md +++ b/corpus/round-trip/opaque-carry/carry-in-wrappers.md @@ -1,17 +1,15 @@ -> ```carry +> ```adf:blockCard > { > "attrs": { > "url": "https://example.com/quoted" -> }, -> "type": "blockCard" +> } > } > ``` -- ```carry +- ```adf:blockCard { "attrs": { "url": "https://example.com/listed" - }, - "type": "blockCard" + } } ``` diff --git a/corpus/round-trip/opaque-carry/code-block-children.json b/corpus/round-trip/opaque-carry/code-block-children.json new file mode 100644 index 0000000..b849d02 --- /dev/null +++ b/corpus/round-trip/opaque-carry/code-block-children.json @@ -0,0 +1,35 @@ +{ + "content": [ + { + "attrs": { + "language": "js" + }, + "content": [ + { + "marks": [ + { + "type": "strong" + } + ], + "text": "const a = 1", + "type": "text" + } + ], + "type": "codeBlock" + }, + { + "content": [ + { + "text": "a", + "type": "text" + }, + { + "type": "hardBreak" + } + ], + "type": "codeBlock" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/opaque-carry/code-block-children.md b/corpus/round-trip/opaque-carry/code-block-children.md new file mode 100644 index 0000000..4749c97 --- /dev/null +++ b/corpus/round-trip/opaque-carry/code-block-children.md @@ -0,0 +1,32 @@ +```adf:codeBlock +{ + "attrs": { + "language": "js" + }, + "content": [ + { + "marks": [ + { + "type": "strong" + } + ], + "text": "const a = 1", + "type": "text" + } + ] +} +``` + +```adf:codeBlock +{ + "content": [ + { + "text": "a", + "type": "text" + }, + { + "type": "hardBreak" + } + ] +} +``` diff --git a/corpus/round-trip/opaque-carry/unknown-block.md b/corpus/round-trip/opaque-carry/unknown-block.md index a88fe07..5469fb5 100644 --- a/corpus/round-trip/opaque-carry/unknown-block.md +++ b/corpus/round-trip/opaque-carry/unknown-block.md @@ -1,23 +1,21 @@ -```carry +```adf:blockCard { "attrs": { "url": "https://example.com/roadmap" - }, - "type": "blockCard" + } } ``` !adf:panel info The card below has no spelling yet. -```carry +```adf:embedCard { "attrs": { "layout": "wide", "url": "https://example.com/board", "width": 100 - }, - "type": "embedCard" + } } ``` !adf:/panel diff --git a/corpus/round-trip/opaque-carry/unknown-type-spelling.json b/corpus/round-trip/opaque-carry/unknown-type-spelling.json new file mode 100644 index 0000000..45a747a --- /dev/null +++ b/corpus/round-trip/opaque-carry/unknown-type-spelling.json @@ -0,0 +1,21 @@ +{ + "content": [ + { + "attrs": { + "url": "https://example.com/" + }, + "type": "has`tick" + }, + { + "type": "" + }, + { + "type": " padded" + }, + { + "type": "two words" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/opaque-carry/unknown-type-spelling.md b/corpus/round-trip/opaque-carry/unknown-type-spelling.md new file mode 100644 index 0000000..1988b18 --- /dev/null +++ b/corpus/round-trip/opaque-carry/unknown-type-spelling.md @@ -0,0 +1,24 @@ +```adf: +{ + "attrs": { + "url": "https://example.com/" + }, + "type": "has`tick" +} +``` + +```adf: +{ + "type": "" +} +``` + +```adf: +{ + "type": " padded" +} +``` + +```adf:two words +{} +``` diff --git a/src/adf/document.ts b/src/adf/document.ts index 490c5ac..370876c 100644 --- a/src/adf/document.ts +++ b/src/adf/document.ts @@ -1,6 +1,7 @@ import type { ConvertFault } from '../result.ts' import { isJsonValue, overNested, type JsonValue } from '../json-value.ts' import { largestNesting } from '../nesting.ts' +import { serializeCanonicalJson } from '../canonical-json.ts' export type AdfAttributes = { [key: string]: JsonValue } @@ -17,6 +18,8 @@ export type AdfNode = { type: string } +export type EmptyKey = 'attrs' | 'content' | 'marks' + export type AdfDocument = { content?: AdfNode[] type: 'doc' @@ -49,10 +52,26 @@ export function attributeNestingMessage(key: string, type: string): string { } export function carriesOnly(node: AdfNode, attributes: readonly string[]): boolean { - if (nodeMarks(node).length > 0 || node.text !== undefined) return false + if (nodeMarks(node).length > 0 || node.text !== undefined || emptyKeys(node).length > 0) return false return holdsOnly(nodeAttrs(node), attributes) } +export function emptyKeys(held: { attrs?: AdfAttributes; content?: AdfNode[]; marks?: AdfMark[] }): EmptyKey[] { + const keys: EmptyKey[] = [] + if (held.attrs !== undefined && Object.keys(held.attrs).length === 0) keys.push('attrs') + if (held.content?.length === 0) keys.push('content') + if (held.marks?.length === 0) keys.push('marks') + return keys +} + +export function identicalMark(left: AdfMark, right: AdfMark): boolean { + return marksKey([left]) === marksKey([right]) +} + +export function identicalMarks(left: AdfNode, right: AdfNode): boolean { + return marksKey(nodeMarks(left)) === marksKey(nodeMarks(right)) +} + // Depth is the walks' business, not the shape's: the guard waves a deep document through as blocks and marks do. export function isAdfDocument(value: unknown): value is AdfDocument { const fault = adfDocumentFault(value) @@ -99,6 +118,13 @@ function isNodeArray(value: readonly unknown[]): value is readonly AdfNode[] { return true } +function marksKey(marks: readonly AdfMark[]): string { + return serializeCanonicalJson( + marks.map((mark) => (mark.attrs === undefined ? [mark.type] : [mark.type, mark.attrs])), + 'compact', + ) +} + function nestingFault(nodes: readonly AdfNode[]): ConvertFault | undefined { const pending: AdfNode[] = [...nodes] while (pending.length > 0) { diff --git a/src/adf/editor-normal.ts b/src/adf/editor-normal.ts index b8a299b..cd37b1f 100644 --- a/src/adf/editor-normal.ts +++ b/src/adf/editor-normal.ts @@ -77,7 +77,7 @@ function mergesText(node: AdfNode): boolean { return node.type === 'text' && Object.keys(nodeAttrs(node)).length === 0 } -export function sameMarks(previous: AdfNode, node: AdfNode): boolean { +function sameMarks(previous: AdfNode, node: AdfNode): boolean { return marksKey(nodeMarks(previous)) === marksKey(nodeMarks(node)) } @@ -86,5 +86,5 @@ function marksKey(marks: readonly AdfMark[]): string { } function markKey(mark: AdfMark): string { - return `${mark.type} ${serializeCanonicalJson(nodeAttrs(mark), 'compact')}` + return `${mark.type} ${serializeCanonicalJson(normalAttributes(nodeAttrs(mark)) ?? {}, 'compact')}` } diff --git a/src/canonical-json.ts b/src/canonical-json.ts index 487dd99..a1fd6ec 100644 --- a/src/canonical-json.ts +++ b/src/canonical-json.ts @@ -18,7 +18,7 @@ export function serializeCanonicalJson(value: JsonValue, spelling: JsonSpelling) const { depth, value: held } = next if (Array.isArray(held)) schedule(pending, '[', held.map((item) => ({ label: '', value: item })), ']', indent, depth) else if (held !== null && typeof held === 'object') schedule(pending, '{', objectMembers(held, indent), '}', indent, depth) - else text.push(JSON.stringify(held)) + else text.push(Object.is(held, -0) ? '-0' : JSON.stringify(held)) } return text.join('') } diff --git a/src/conformance/adf-property.test.ts b/src/conformance/adf-property.test.ts index 16de9d7..0061ea9 100644 --- a/src/conformance/adf-property.test.ts +++ b/src/conformance/adf-property.test.ts @@ -8,7 +8,6 @@ import { adfToMarkdown } from '../markdown/emit/adf-to-markdown.ts' import { adfToPlainMarkdown, reduceToPlain } from '../markdown/emit/plain-reduction.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' import { markdownToAdf, plainMarkdownToAdf } from '../markdown/parse/markdown-to-adf.ts' -import { toEditorNormal } from '../adf/editor-normal.ts' const gateRuns = 1600 const renamedPrefix = '!adg:' @@ -41,7 +40,7 @@ test('a generated document refuses to emit, or its markdown reads back to it', { if (!emitted.ok) return const read = markdownToAdf(emitted.value) assert.ok(read.ok, read.ok ? '' : `${read.error.code}: ${read.error.message} — reading ${JSON.stringify(emitted.value)}`) - assert.deepEqual(toEditorNormal(read.value), document, `reading ${JSON.stringify(emitted.value)}`) + assert.deepEqual(read.value, document, `reading ${JSON.stringify(emitted.value)}`) }), propertyRuns(gateRuns), ) diff --git a/src/conformance/corpus.test.ts b/src/conformance/corpus.test.ts index 22ff052..86fe0f6 100644 --- a/src/conformance/corpus.test.ts +++ b/src/conformance/corpus.test.ts @@ -9,7 +9,6 @@ import { isAdfDocument } from '../adf/document.ts' import { isJsonValue } from '../json-value.ts' import { markdownToAdf } from '../markdown/parse/markdown-to-adf.ts' import { serializeCanonicalJson } from '../canonical-json.ts' -import { toEditorNormal } from '../adf/editor-normal.ts' const corpusRoot = join(dirname(fileURLToPath(import.meta.url)), '..', '..', 'corpus') const errorsRoot = join(corpusRoot, 'errors') @@ -97,7 +96,7 @@ for (const directory of roundTripDirectories) { assert.ok(isAdfDocument(expected), `${name}.json is not an ADF document`) const result = markdownToAdf(readFileSync(join(roundTripRoot, directory, `${name}.md`), 'utf8')) assert.ok(result.ok, result.ok ? '' : `${result.error.code}: ${result.error.message}`) - assert.deepEqual(toEditorNormal(result.value), expected) + assert.deepEqual(result.value, expected) }) } } @@ -125,12 +124,12 @@ for (const name of pairedNames(normalizationRoot, '.md', '.json')) { assert.ok(isAdfDocument(expected), `${name}.json is not an ADF document`) const result = markdownToAdf(readFileSync(join(normalizationRoot, `${name}.md`), 'utf8')) assert.ok(result.ok, result.ok ? '' : `${result.error.code}: ${result.error.message}`) - assert.deepEqual(toEditorNormal(result.value), expected) + assert.deepEqual(result.value, expected) const emitted = adfToMarkdown(result.value) assert.ok(emitted.ok, emitted.ok ? '' : `${emitted.error.code}: ${emitted.error.message}`) const again = markdownToAdf(emitted.value) assert.ok(again.ok, again.ok ? '' : `${again.error.code}: ${again.error.message}`) - assert.deepEqual(toEditorNormal(again.value), expected) + assert.deepEqual(again.value, expected) }) } @@ -146,7 +145,7 @@ for (const name of names(realPayloadsRoot, '.json')) { assert.ok(emitted.ok, emitted.ok ? '' : `${emitted.error.code}: ${emitted.error.message}`) const parsed = markdownToAdf(emitted.value) assert.ok(parsed.ok, parsed.ok ? '' : `${parsed.error.code}: ${parsed.error.message}`) - assert.deepEqual(toEditorNormal(parsed.value), payload) + assert.deepEqual(parsed.value, payload) }) } diff --git a/src/conformance/markdown-property.test.ts b/src/conformance/markdown-property.test.ts index 580dce9..64a435f 100644 --- a/src/conformance/markdown-property.test.ts +++ b/src/conformance/markdown-property.test.ts @@ -31,7 +31,6 @@ import { markdownToAdf } from '../markdown/parse/markdown-to-adf.ts' import { nodeContent, nodeMarks } from '../adf/document.ts' import { serializeCanonicalJson } from '../canonical-json.ts' import { textDirectiveName } from '../markdown/text-directive.ts' -import { toEditorNormal } from '../adf/editor-normal.ts' import { vocabularyPairs } from '../adf/attribute-vocabulary.ts' type Choice = { arbitrary: Arbitrary; hostile?: true; weight: number } @@ -433,7 +432,7 @@ test('generated markdown refuses, or what it parses to refuses to emit, or its s if (holdsDirectiveShape(parsed.value)) directiveShaped += 1 const read = markdownToAdf(emitted.value) assert.ok(read.ok, read.ok ? '' : `${read.error.code}: ${read.error.message} — reading ${JSON.stringify(emitted.value)}`) - assert.deepEqual(toEditorNormal(read.value), toEditorNormal(parsed.value), `reading ${JSON.stringify(emitted.value)}`) + assert.deepEqual(read.value, parsed.value, `reading ${JSON.stringify(emitted.value)}`) const respelled = adfToMarkdown(read.value) assert.ok(respelled.ok, respelled.ok ? '' : `${respelled.error.code}: ${respelled.error.message} — spelling ${JSON.stringify(emitted.value)} again`) assert.equal(respelled.value, emitted.value) diff --git a/src/conformance/property-harness.ts b/src/conformance/property-harness.ts index 5013bfa..cd935f8 100644 --- a/src/conformance/property-harness.ts +++ b/src/conformance/property-harness.ts @@ -11,7 +11,7 @@ import { blockNodes } from '../adf/block-nodes.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' import { inlineNodes } from '../adf/inline-nodes.ts' import { markAttributes } from '../adf/mark-attributes.ts' -import { toEditorNormal } from '../adf/editor-normal.ts' +import { mergeAdjacentText } from '../adf/editor-normal.ts' type Positions = { block: AdfNode; inline: AdfNode } @@ -36,7 +36,11 @@ export function textOf(minLength: number): Arbitrary { const text = textOf(1) const unknownType = fc.oneof(fc.stringMatching(/^[a-z][A-Za-z0-9]{0,7}$/), text).filter((type) => !spelledTypes.has(type)) -const numberValue = fc.oneof({ arbitrary: fc.integer({ max: 10, min: -1 }), weight: 3 }, { arbitrary: fc.double({ noDefaultInfinity: true, noNaN: true }), weight: 1 }) +const numberValue = fc.oneof( + { arbitrary: fc.integer({ max: 10, min: -1 }), weight: 6 }, + { arbitrary: fc.double({ noDefaultInfinity: true, noNaN: true }), weight: 2 }, + { arbitrary: fc.constant(-0), weight: 1 }, +) // V8's JSON.parse returns a wrong key after parsing a key holding an escaped backslash (https://issues.chromium.org/issues/521080746); Bun is unaffected. const keyPiece = fc @@ -73,19 +77,37 @@ function heldAttributes(held: Readonly>): return attrs } +// An empty attrs, content or marks key, and adjacent text a reader would merge, stay occasional: each takes a spelling outside CommonMark. +// The copy gives fast-check's null-prototype records the prototype a parsed node has. +function occasionallyEmpty(arbitrary: Arbitrary): Arbitrary { + return fc.tuple(arbitrary, fc.nat({ max: 9 })).map(([held, roll]) => (roll === 0 ? { ...held } : withoutEmptyKeys(held))) +} + +function withoutEmptyKeys(held: T): T { + const kept = { ...held } + if (kept.attrs !== undefined && Object.keys(kept.attrs).length === 0) delete kept.attrs + if ('content' in kept && kept.content?.length === 0) delete kept.content + if ('marks' in kept && kept.marks?.length === 0) delete kept.marks + return kept +} + +function occasionallyApart(arbitrary: Arbitrary): Arbitrary { + return fc.tuple(arbitrary, fc.nat({ max: 5 })).map(([nodes, roll]) => (roll === 0 ? nodes : mergeAdjacentText(nodes))) +} + function pipeTable({ body, header }: { body: AdfNode[][]; header: AdfNode[] }): AdfNode { const rows = [header, ...body.map((cells) => header.map((_, index) => cells[index] ?? emptyCell))] return { content: rows.map((content): AdfNode => ({ content, type: 'tableRow' })), type: 'table' } } -const mark: Arbitrary = fc.oneof( +const mark: Arbitrary = occasionallyEmpty(fc.oneof( { arbitrary: fc.oneof(...Object.entries(markAttributes).map(([type, vocabulary]) => attributes(vocabulary).map((attrs) => ({ attrs, type })))), weight: 9 }, { arbitrary: attributes({ color: 'string' }).map((attrs) => ({ attrs, type: 'backgroundColor' })), weight: 2 }, { arbitrary: fc.record({ attrs: fc.dictionary(jsonKey, jsonValue, { maxKeys: 2, noNullPrototype: true }), type: unknownType }), weight: 1 }, -) +)) const marks = fc.uniqueArray(mark, { maxLength: 3, selector: (held) => held.type }) -const textNode = fc.record({ marks, text }).map((held): AdfNode => ({ ...held, type: 'text' })) +const textNode = occasionallyEmpty(fc.record({ marks, text }).map((held): AdfNode => ({ ...held, type: 'text' }))) const backtickRunNode = fc .record({ marks: fc.oneof(fc.constant([]), fc.constant([{ type: 'code' }]), marks), text: fc.string({ maxLength: 6, minLength: 1, unit: fc.constantFrom('`', '``', ' ', 'a') }) }) @@ -96,7 +118,7 @@ const autolinkTextNode = fc .map(({ href, marks: held }): AdfNode => ({ marks: [...held.filter((outer) => outer.type !== 'link'), { attrs: { href }, type: 'link' }], text: href, type: 'text' })) const inlineArbitraries = Object.entries(inlineNodes).map(([type, model]) => - fc.record({ attrs: attributes(model.attributes), marks }).map((held): AdfNode => ({ ...held, type })), + occasionallyEmpty(fc.record({ attrs: attributes(model.attributes), marks }).map((held): AdfNode => ({ ...held, type }))), ) function weighted(arbitraries: readonly Arbitrary[], weight: number): { arbitrary: Arbitrary; weight: number }[] { @@ -105,10 +127,10 @@ function weighted(arbitraries: readonly Arbitrary[], weight: number): { const positions = fc.letrec((tie) => { const blockContent = fc.array(tie('block'), { depthIdentifier, maxLength: 3 }) - const inlineContent = fc.array(tie('inline'), { depthIdentifier, maxLength: 4 }) + const inlineContent = occasionallyApart(fc.array(tie('inline'), { depthIdentifier, maxLength: 4 })) const contentByModel = { block: blockContent, - code: fc.array(text.map((held): AdfNode => ({ text: held, type: 'text' })), { maxLength: 2 }), + code: occasionallyApart(fc.array(text.map((held): AdfNode => ({ text: held, type: 'text' })), { maxLength: 2 })), inline: inlineContent, none: fc.constant([]), } @@ -116,35 +138,40 @@ const positions = fc.letrec((tie) => { const blockArbitraries = Object.entries(blockNodes).map(([type, model]) => { const argument = blockArgument(type) const vocabulary: AttributeVocabulary = argument === undefined ? model.attributes : { ...model.attributes, [argument]: 'string' } - const node = fc.record({ attrs: attributes(vocabulary), content: contentByModel[model.contentModel], marks: blockMarks }).map((held): AdfNode => ({ ...held, type })) + const node = occasionallyEmpty(fc.record({ attrs: attributes(vocabulary), content: contentByModel[model.contentModel], marks: blockMarks }).map((held): AdfNode => ({ ...held, type }))) return { leaf: model.contentModel === 'code' || model.contentModel === 'none', node } }) - const unknownNode = fc - .record({ attrs: fc.dictionary(jsonKey, jsonValue, { maxKeys: 2, noNullPrototype: true }), content: fc.array(tie('inline'), { depthIdentifier, maxLength: 2 }), marks, type: unknownType }) - .map((held): AdfNode => held) + const unknownNode = occasionallyEmpty( + fc.record({ attrs: fc.dictionary(jsonKey, jsonValue, { maxKeys: 2, noNullPrototype: true }), content: fc.array(tie('inline'), { depthIdentifier, maxLength: 2 }), marks, type: unknownType }), + ) const leafBlocks = blockArbitraries.filter((entry) => entry.leaf).map((entry) => entry.node) const containerBlocks = blockArbitraries.filter((entry) => !entry.leaf).map((entry) => entry.node) const misplacedWeight = 7 - const paragraph = fc.oneof({ arbitrary: inlineContent, weight: 3 }, { arbitrary: fc.array(backtickRunNode, { maxLength: 4, minLength: 2 }), weight: 1 }).map((content): AdfNode => ({ content, type: 'paragraph' })) + const paragraph = occasionallyEmpty( + fc.oneof({ arbitrary: inlineContent, weight: 3 }, { arbitrary: occasionallyApart(fc.array(backtickRunNode, { maxLength: 4, minLength: 2 })), weight: 1 }).map((content): AdfNode => ({ content, type: 'paragraph' })), + ) const cell = (type: string) => paragraph.map((held): AdfNode => ({ content: [held], type })) const listItems = fc.array( - blockContent.map((content): AdfNode => ({ content, type: 'listItem' })), + occasionallyEmpty(blockContent.map((content): AdfNode => ({ content, type: 'listItem' }))), { depthIdentifier, maxLength: 3, minLength: 1 }, ) const flatCommonMarkShapes = [ - fc.record({ content: inlineContent, level: fc.integer({ max: 6, min: 1 }) }).map(({ content, level }): AdfNode => ({ attrs: { level }, content, type: 'heading' })), + occasionallyEmpty(fc.record({ content: inlineContent, level: fc.integer({ max: 6, min: 1 }) }).map(({ content, level }): AdfNode => ({ attrs: { level }, content, type: 'heading' }))), paragraph, fc.record({ body: fc.array(fc.array(cell('tableCell'), { maxLength: 3 }), { maxLength: 2 }), header: fc.array(cell('tableHeader'), { maxLength: 3, minLength: 1 }) }).map(pipeTable), ] const task = (type: string, content: Arbitrary) => - fc.record({ content, state: fc.constantFrom('DONE', 'TODO') }).map(({ content: held, state }): AdfNode => ({ attrs: { state }, content: held, type })) + occasionallyEmpty(fc.record({ content, state: fc.constantFrom('DONE', 'TODO') }).map(({ content: held, state }): AdfNode => ({ attrs: { state }, content: held, type }))) const taskItem = fc.oneof({ arbitrary: task('taskItem', inlineContent), weight: 3 }, { arbitrary: task('blockTaskItem', blockContent), weight: 1 }) const nestingCommonMarkShapes = [ - blockContent.map((content): AdfNode => ({ content, type: 'blockquote' })), + occasionallyEmpty(blockContent.map((content): AdfNode => ({ content, type: 'blockquote' }))), fc.array(fc.oneof({ arbitrary: taskItem, weight: 3 }, { arbitrary: tie('block'), weight: 1 }), { depthIdentifier, maxLength: 3, minLength: 1 }).map((content): AdfNode => ({ content, type: 'taskList' })), listItems.map((content): AdfNode => ({ content, type: 'bulletList' })), fc - .record({ content: listItems, order: fc.oneof({ arbitrary: fc.integer({ max: 3, min: 0 }), weight: 4 }, { arbitrary: fc.integer({ max: 999999999, min: 0 }), weight: 1 }) }) + .record({ + content: listItems, + order: fc.oneof({ arbitrary: fc.integer({ max: 3, min: 0 }), weight: 8 }, { arbitrary: fc.integer({ max: 999999999, min: 0 }), weight: 2 }, { arbitrary: fc.constant(-0), weight: 1 }), + }) .map(({ content, order }): AdfNode => ({ attrs: { order }, content, type: 'orderedList' })), ] const flatBlocks = [...weighted(leafBlocks, 2), ...weighted(flatCommonMarkShapes, flatCommonMarkShapeWeight)] @@ -167,7 +194,11 @@ const positions = fc.letrec((tie) => { } }) -export const adfDocument = fc.array(positions.block, { depthIdentifier, maxLength: 4, minLength: 1 }).map((content): AdfDocument => toEditorNormal({ content, type: 'doc', version: 1 })) +export const adfDocument = fc.oneof( + { arbitrary: fc.array(positions.block, { depthIdentifier, maxLength: 4, minLength: 1 }).map((content): AdfDocument => ({ content, type: 'doc', version: 1 })), weight: 30 }, + { arbitrary: fc.constant({ content: [], type: 'doc', version: 1 }), weight: 1 }, + { arbitrary: fc.constant({ type: 'doc', version: 1 }), weight: 1 }, +) export function propertyRuns(gateRuns: number): { gate: boolean; numRuns: number; seed?: number } { const deepRuns = env[deepRunsVariable] diff --git a/src/markdown/block-directive.ts b/src/markdown/block-directive.ts index c528e23..dc305bb 100644 --- a/src/markdown/block-directive.ts +++ b/src/markdown/block-directive.ts @@ -2,7 +2,7 @@ import type { AdfMark } from '../adf/document.ts' import type { BlockType } from '../adf/block-nodes.ts' import type { JsonValue } from '../json-value.ts' import { blockNodeModel } from '../adf/block-nodes.ts' -import { isAdfMark, nodeAttrs } from '../adf/document.ts' +import { isAdfMark } from '../adf/document.ts' import { serializeCanonicalJson } from '../canonical-json.ts' import { spellDirectiveOpener } from './directive-syntax.ts' @@ -14,6 +14,11 @@ const argumentByType = new Map( } satisfies Partial>), ) +export const documentName = 'doc' + +// spec/flavour.md, Directives: a document holding no content key, which the empty string cannot spell. +export const documentSpelling = spellDirectiveOpener(documentName, undefined, '{content=none}') + export const listBreakName = 'listBreak' export const listBreakSpelling = spellDirectiveOpener(listBreakName, undefined, '') @@ -25,17 +30,14 @@ export function blockArgument(type: string): string | undefined { } export function blockDirectiveForm(name: string): 'container' | 'leaf' | undefined { - if (name === listBreakName) return 'leaf' + if (name === listBreakName || name === documentName) return 'leaf' const model = blockNodeModel(name) if (model === undefined) return undefined return model.contentModel === 'none' ? 'leaf' : 'container' } export function markValues(marks: readonly AdfMark[]): JsonValue { - return marks.map((mark) => { - const attrs = nodeAttrs(mark) - return Object.keys(attrs).length === 0 ? { type: mark.type } : { attrs, type: mark.type } - }) + return marks.map((mark) => (mark.attrs === undefined ? { type: mark.type } : { attrs: mark.attrs, type: mark.type })) } export function readMarkValues(value: JsonValue): AdfMark[] | undefined { diff --git a/src/markdown/code-language.ts b/src/markdown/code-language.ts index 223e75a..75ddf83 100644 --- a/src/markdown/code-language.ts +++ b/src/markdown/code-language.ts @@ -1,14 +1,12 @@ import type { JsonValue } from '../json-value.ts' -import { carryName } from './opaque-carry.ts' -import { holdsControlCharacter } from './commonmark/grammar.ts' -import { holdsEntityReference } from './commonmark/entity-references.ts' +import { carryFencePrefix } from './opaque-carry.ts' +import { infoStringCarries } from './commonmark/grammar.ts' export type LanguageSlot = { info: string; kind: 'fence' } | { kind: 'attribute' } | { kind: 'none' } // spec/flavour.md, The CommonMark blocks: the one slot a codeBlock's language rides. export function languageSlot(language: JsonValue | undefined): LanguageSlot { if (language === undefined) return { kind: 'none' } - if (typeof language !== 'string' || language === '' || language === carryName) return { kind: 'attribute' } - if (/[`\\]/.test(language) || holdsControlCharacter(language) || language !== language.trim() || holdsEntityReference(language)) return { kind: 'attribute' } + if (typeof language !== 'string' || language.startsWith(carryFencePrefix) || !infoStringCarries(language)) return { kind: 'attribute' } return { info: language, kind: 'fence' } } diff --git a/src/markdown/commonmark/grammar.ts b/src/markdown/commonmark/grammar.ts index 53c9f1a..5d394e2 100644 --- a/src/markdown/commonmark/grammar.ts +++ b/src/markdown/commonmark/grammar.ts @@ -1,4 +1,4 @@ -import { readEntityReference, replacementCharacter } from './entity-references.ts' +import { holdsEntityReference, readEntityReference, replacementCharacter } from './entity-references.ts' export type LinePosition = 'first' | 'later' @@ -133,6 +133,11 @@ export function holdsControlCharacter(text: string): boolean { return controlCharacter.test(text) } +// What a backtick fence's info string reads back verbatim: escapes and entity references decode, and the edges trim. +export function infoStringCarries(text: string): boolean { + return text !== '' && !/[`\\]/.test(text) && !holdsControlCharacter(text) && text === text.trim() && !holdsEntityReference(text) +} + export function holdsNullCharacter(text: string): boolean { return nullCharacter.test(text) } diff --git a/src/markdown/directive-syntax.ts b/src/markdown/directive-syntax.ts index 7c992da..c4f4cfd 100644 --- a/src/markdown/directive-syntax.ts +++ b/src/markdown/directive-syntax.ts @@ -139,7 +139,7 @@ export function spellStringAttribute(text: string): string { export function spellAttributeValue(value: VocabularyValue): string { if (value.kind === 'boolean') return `${value.value}` if (value.kind === 'json') return spellJsonAttribute(value.value) - if (value.kind === 'number') return spellStringAttribute(JSON.stringify(value.value)) + if (value.kind === 'number') return spellStringAttribute(serializeCanonicalJson(value.value, 'compact')) return spellStringAttribute(value.value) } diff --git a/src/markdown/emit/adf-to-markdown.test.ts b/src/markdown/emit/adf-to-markdown.test.ts index daf574f..b92e06c 100644 --- a/src/markdown/emit/adf-to-markdown.test.ts +++ b/src/markdown/emit/adf-to-markdown.test.ts @@ -6,14 +6,13 @@ import type { JsonValue } from '../../json-value.ts' import type { Result } from '../../result.ts' import { adfToMarkdown, markdownToAdf } from '../../index.ts' import { largestNesting } from '../../nesting.ts' -import { toEditorNormal } from '../../adf/editor-normal.ts' function document(...content: AdfNode[]): AdfDocument { return { content, type: 'doc', version: 1 } } function paragraph(...content: AdfNode[]): AdfNode { - return { content, type: 'paragraph' } + return content.length === 0 ? { type: 'paragraph' } : { content, type: 'paragraph' } } function code(result: Result): string { @@ -168,21 +167,24 @@ test('parts two adjacent lists of the same kind, the marker spelling being what const nested: AdfNode = { content: [{ content: [list, list], type: 'listItem' }], type: 'bulletList' } assert.equal(markdown(adfToMarkdown(document(nested))), '- - x\n\n !adf:listBreak\n\n - x\n') const carried: AdfNode = { ...list, attrs: { unknown: 'x' } } - assert.ok(markdown(adfToMarkdown(document(carried, carried))).includes('```\n\n```carry\n')) + assert.ok(markdown(adfToMarkdown(document(carried, carried))).includes('```\n\n```adf:bulletList\n')) assert.ok(markdown(adfToMarkdown(document(carried, list))).endsWith('```\n\n- x\n')) - assert.ok(markdown(adfToMarkdown(document(list, carried))).startsWith('- x\n\n```carry\n')) + assert.ok(markdown(adfToMarkdown(document(list, carried))).startsWith('- x\n\n```adf:bulletList\n')) }) -test('carries a node type no section spells', () => { - assert.equal(markdown(adfToMarkdown(document({ type: 'blockCard' }))), '```carry\n{\n "type": "blockCard"\n}\n```\n') - assert.equal(markdown(adfToMarkdown(document({ type: 'toString' }))), '```carry\n{\n "type": "toString"\n}\n```\n') +test('carries a node type no section spells, the fence naming the type wherever an info string carries it', () => { + assert.equal(markdown(adfToMarkdown(document({ type: 'blockCard' }))), '```adf:blockCard\n{}\n```\n') + assert.equal(markdown(adfToMarkdown(document({ type: 'toString' }))), '```adf:toString\n{}\n```\n') assert.equal(markdown(adfToMarkdown(document(paragraph({ type: 'blockCard' })))), '!adf:carry{json="{\\"type\\":\\"blockCard\\"}"}\n') - assert.equal(markdown(adfToMarkdown(document({ text: 'x', type: 'text' }))), '```carry\n{\n "text": "x",\n "type": "text"\n}\n```\n') - assert.equal(markdown(adfToMarkdown(document({ type: 'hardBreak' }))), '```carry\n{\n "type": "hardBreak"\n}\n```\n') + assert.equal(markdown(adfToMarkdown(document({ text: 'x', type: 'text' }))), '```adf:text\n{\n "text": "x"\n}\n```\n') + assert.equal(markdown(adfToMarkdown(document({ type: 'hardBreak' }))), '```adf:hardBreak\n{}\n```\n') + assert.equal(markdown(adfToMarkdown(document({ type: 'a&b' }))), '```adf:\n{\n "type": "a&b"\n}\n```\n') + assert.equal(markdown(adfToMarkdown(document({ type: 'a\\b' }))), '```adf:\n{\n "type": "a\\\\b"\n}\n```\n') }) -test('spells the code block whose language is the reserved info string', () => { - assert.equal(markdown(adfToMarkdown(document({ attrs: { language: 'carry' }, type: 'codeBlock' }))), '!adf:codeBlock {language=carry}\n```\n```\n!adf:/codeBlock\n') +test('spells the code block whose language opens with the reserved info string prefix', () => { + assert.equal(markdown(adfToMarkdown(document({ attrs: { language: 'adf:x' }, type: 'codeBlock' }))), '!adf:codeBlock {language="adf:x"}\n```\n```\n!adf:/codeBlock\n') + assert.equal(markdown(adfToMarkdown(document({ attrs: { language: 'carry' }, type: 'codeBlock' }))), '```carry\n```\n') assert.equal(markdown(adfToMarkdown(document({ attrs: { language: 'adf' }, type: 'codeBlock' }))), '```adf\n```\n') }) @@ -208,21 +210,17 @@ test('refuses a carried node nested deeper than the levels its position leaves', assert.equal(code(adfToMarkdown(document(quoted))), 'unsupported-nesting-depth') }) -test('refuses a node whose content model the canonical form cannot emit', () => { - assert.equal( - markdown(adfToMarkdown(document({ content: [paragraph()], type: 'codeBlock' }))), - 'unsupported-node-shape: a codeBlock holds plain text nodes only: this paragraph node is not one', - ) - assert.equal( - markdown(adfToMarkdown(document({ content: [{ content: [{ text: 'lost', type: 'text' }], text: 'x', type: 'text' }], type: 'codeBlock' }))), - 'unsupported-node-shape: a codeBlock holds plain text nodes only: this text node is not one', - ) +test('carries a code block holding a node no fence holds', () => { + assert.equal(markdown(adfToMarkdown(document({ content: [paragraph()], type: 'codeBlock' }))), '```adf:codeBlock\n{\n "content": [\n {\n "type": "paragraph"\n }\n ]\n}\n```\n') + for (const child of [{ attrs: {}, text: 'x', type: 'text' }, { content: [], text: 'x', type: 'text' }, { marks: [], text: 'x', type: 'text' }, { content: [{ text: 'lost', type: 'text' }], text: 'x', type: 'text' }]) { + assert.ok(markdown(adfToMarkdown(document({ content: [child], type: 'codeBlock' }))).startsWith('```adf:codeBlock\n'), JSON.stringify(child)) + } }) test('spells a list its own content shape cannot hold as a directive', () => { assert.equal(markdown(adfToMarkdown(document({ content: [paragraph()], type: 'bulletList' }))), '!adf:bulletList\n!adf:paragraph\n!adf:/paragraph\n!adf:/bulletList\n') assert.equal(markdown(adfToMarkdown(document({ type: 'bulletList' }))), '!adf:bulletList\n!adf:/bulletList\n') - assert.equal(markdown(adfToMarkdown(document({ attrs: { order: 2 }, content: [], type: 'orderedList' }))), '!adf:orderedList {order=2}\n!adf:/orderedList\n') + assert.equal(markdown(adfToMarkdown(document({ attrs: { order: 2 }, content: [], type: 'orderedList' }))), '!adf:orderedList {content=empty order=2}\n!adf:/orderedList\n') }) test('spells an ordered list no marker fits as a directive', () => { @@ -334,7 +332,7 @@ test('refuses a node carrying one mark type twice', () => { }) test('parts a nested list the tight spelling would swallow from the block above it', () => { - const item = (...content: AdfNode[]): AdfNode => ({ content, type: 'listItem' }) + const item = (...content: AdfNode[]): AdfNode => (content.length === 0 ? { type: 'listItem' } : { content, type: 'listItem' }) const text = (value: string): AdfNode => ({ content: [{ text: value, type: 'text' }], type: 'paragraph' }) const outer = (...content: AdfNode[]): AdfDocument => document({ content: [item(...content)], type: 'bulletList' }) const ordered: AdfNode = { attrs: { order: 2 }, content: [item(text('b'))], type: 'orderedList' } @@ -345,7 +343,7 @@ test('parts a nested list the tight spelling would swallow from the block above const list: AdfNode = { content: [item(text('b'))], type: 'bulletList' } const panel: AdfNode = { attrs: { panelType: 'info' }, content: [text('p')], type: 'panel' } assert.equal(markdown(adfToMarkdown(outer(panel, list))), '- !adf:panel info\n p\n !adf:/panel\n\n - b\n') - assert.ok(markdown(adfToMarkdown(outer(text('a'), { ...list, attrs: { unknown: 'x' } }))).startsWith('- a\n\n ```carry\n')) + assert.ok(markdown(adfToMarkdown(outer(text('a'), { ...list, attrs: { unknown: 'x' } }))).startsWith('- a\n\n ```adf:bulletList\n')) }) test('refuses marks and attributes nested deeper than the emitter carries', () => { @@ -372,7 +370,7 @@ test('refuses marks and attributes nested deeper than the emitter carries', () = assert.ok(spelled.ok, spelled.ok ? '' : spelled.error.message) const read = markdownToAdf(spelled.value) assert.ok(read.ok, read.ok ? '' : read.error.message) - assert.deepEqual(toEditorNormal(read.value), document(node)) + assert.deepEqual(read.value, document(node)) } assert.equal(markdown(adfToMarkdown(document(paragraph({ marks: [{ attrs, type: 'em' }], text: 'x', type: 'text' })))), deeper('depth', 'em')) @@ -442,7 +440,7 @@ test('escapes a hyphen underline a hard break would expose', () => { }) test('spells a list item whose marker completes a thematic break as a directive', () => { - const item = (...content: AdfNode[]): AdfNode => ({ content, type: 'listItem' }) + const item = (...content: AdfNode[]): AdfNode => (content.length === 0 ? { type: 'listItem' } : { content, type: 'listItem' }) assert.equal(markdown(adfToMarkdown(document({ content: [item({ type: 'rule' })], type: 'bulletList' }))), '!adf:bulletList\n!adf:listItem\n---\n!adf:/listItem\n!adf:/bulletList\n') const nested: AdfNode = { content: [item({ content: [item()], type: 'bulletList' })], type: 'bulletList' } assert.equal(markdown(adfToMarkdown(document(nested))), '- -\n') @@ -469,10 +467,7 @@ test('refuses the characters CommonMark rewrites', () => { ) assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\u0000b', type: 'text' })))), 'unspellable-character') assert.equal(code(adfToMarkdown(document({ content: [{ text: 'a\u0000b', type: 'text' }], type: 'codeBlock' }))), 'unspellable-character') - assert.equal( - markdown(adfToMarkdown(document({ content: [{ text: '', type: 'text' }], type: 'codeBlock' }))), - 'unsupported-node-shape: a codeBlock holds plain text nodes only: this text node is not one', - ) + assert.equal(markdown(adfToMarkdown(document({ content: [{ text: '', type: 'text' }], type: 'codeBlock' }))), 'unsupported-node-shape: a text node holds text: this one has none') }) test('refuses a text node the spelling would empty out', () => { @@ -501,11 +496,11 @@ test('pads a code span whose edges CommonMark would strip', () => { assert.equal(markdown(adfToMarkdown(document(paragraph({ marks: [{ type: 'code' }], text: ' \t ', type: 'text' })))), '` \t `\n') }) -test('spells one code span over a run of code-marked nodes', () => { +test('spells a code span per code-marked node, parted by the text break', () => { const code_ = { type: 'code' } assert.equal( markdown(adfToMarkdown(document(paragraph({ marks: [code_], text: 'a', type: 'text' }, { marks: [code_], text: 'b', type: 'text' })))), - '`ab`\n', + '`a`!adf:textBreak{}`b`\n', ) }) @@ -545,7 +540,7 @@ test('emits an empty list item without trailing whitespace', () => { test('spells a block directive as its node type, arg and attributes', () => { const panel = (attrs: AdfAttributes): AdfDocument => document({ attrs, content: [paragraph({ text: 'x', type: 'text' })], type: 'panel' }) assert.equal(markdown(adfToMarkdown(panel({ panelType: 'warning' }))), '!adf:panel warning\nx\n!adf:/panel\n') - assert.equal(markdown(adfToMarkdown(panel({}))), '!adf:panel\nx\n!adf:/panel\n') + assert.equal(markdown(adfToMarkdown(panel({}))), '!adf:panel {attrs=empty}\nx\n!adf:/panel\n') assert.equal(markdown(adfToMarkdown(document({ content: [{ text: 'x', type: 'text' }], type: 'caption' }))), '!adf:caption\nx\n!adf:/caption\n') assert.equal(markdown(adfToMarkdown(document({ type: 'caption' }))), '!adf:caption\n!adf:/caption\n') assert.equal(markdown(adfToMarkdown(document({ attrs: { localId: 'a' }, type: 'syncBlock' }))), '!adf:syncBlock {localId=a}\n') @@ -553,20 +548,20 @@ test('spells a block directive as its node type, arg and attributes', () => { test('carries a directive attribute no section spells', () => { const carried = (node: AdfNode): string => markdown(adfToMarkdown(document(node))) - assert.equal(carried({ attrs: { rounded: true }, type: 'panel' }), '```carry\n{\n "attrs": {\n "rounded": true\n },\n "type": "panel"\n}\n```\n') - assert.equal(carried({ attrs: { toString: 'x' }, type: 'panel' }), '```carry\n{\n "attrs": {\n "toString": "x"\n },\n "type": "panel"\n}\n```\n') - assert.equal(carried({ attrs: { localId: 4 }, type: 'panel' }), '```carry\n{\n "attrs": {\n "localId": 4\n },\n "type": "panel"\n}\n```\n') + assert.equal(carried({ attrs: { rounded: true }, type: 'panel' }), '```adf:panel\n{\n "attrs": {\n "rounded": true\n }\n}\n```\n') + assert.equal(carried({ attrs: { toString: 'x' }, type: 'panel' }), '```adf:panel\n{\n "attrs": {\n "toString": "x"\n }\n}\n```\n') + assert.equal(carried({ attrs: { localId: 4 }, type: 'panel' }), '```adf:panel\n{\n "attrs": {\n "localId": 4\n }\n}\n```\n') assert.equal( carried({ attrs: { width: '50' }, type: 'layoutColumn' }), - '```carry\n{\n "attrs": {\n "width": "50"\n },\n "type": "layoutColumn"\n}\n```\n', + '```adf:layoutColumn\n{\n "attrs": {\n "width": "50"\n }\n}\n```\n', ) assert.equal( carried({ attrs: { isNumberColumnEnabled: 'true' }, type: 'table' }), - '```carry\n{\n "attrs": {\n "isNumberColumnEnabled": "true"\n },\n "type": "table"\n}\n```\n', + '```adf:table\n{\n "attrs": {\n "isNumberColumnEnabled": "true"\n }\n}\n```\n', ) assert.equal( carried({ content: [{ attrs: { alt: 4 }, type: 'media' }], type: 'mediaGroup' }), - '!adf:mediaGroup\n```carry\n{\n "attrs": {\n "alt": 4\n },\n "type": "media"\n}\n```\n!adf:/mediaGroup\n', + '!adf:mediaGroup\n```adf:media\n{\n "attrs": {\n "alt": 4\n }\n}\n```\n!adf:/mediaGroup\n', ) }) @@ -574,15 +569,16 @@ test('carries an arg slot value no bare token spells', () => { const carried = (node: AdfNode): string => markdown(adfToMarkdown(document(node))) assert.equal( carried({ attrs: { panelType: 'extra info' }, type: 'panel' }), - '```carry\n{\n "attrs": {\n "panelType": "extra info"\n },\n "type": "panel"\n}\n```\n', + '```adf:panel\n{\n "attrs": {\n "panelType": "extra info"\n }\n}\n```\n', ) - assert.equal(carried({ attrs: { state: 2 }, type: 'taskItem' }), '```carry\n{\n "attrs": {\n "state": 2\n },\n "type": "taskItem"\n}\n```\n') + assert.equal(carried({ attrs: { state: 2 }, type: 'taskItem' }), '```adf:taskItem\n{\n "attrs": {\n "state": 2\n }\n}\n```\n') }) test('carries a block node mark in the reserved attribute', () => { const section = (...marks: AdfMark[]): AdfDocument => document({ marks, type: 'layoutSection' }) assert.equal(markdown(adfToMarkdown(section({ type: 'breakout' }))), '!adf:layoutSection {marks="[{\\"type\\":\\"breakout\\"}]"}\n!adf:/layoutSection\n') - assert.equal(markdown(adfToMarkdown(section({ attrs: {}, type: 'breakout' }))), '!adf:layoutSection {marks="[{\\"type\\":\\"breakout\\"}]"}\n!adf:/layoutSection\n') + assert.equal(markdown(adfToMarkdown(section({ attrs: {}, type: 'breakout' }))), '!adf:layoutSection {marks="[{\\"attrs\\":{},\\"type\\":\\"breakout\\"}]"}\n!adf:/layoutSection\n') + assert.equal(markdown(adfToMarkdown(section())), '!adf:layoutSection {marks=empty}\n!adf:/layoutSection\n') }) test('refuses the content a directive body has no room for', () => { @@ -604,7 +600,7 @@ test('separates blocks in a container body by a blank line only where a directiv test('spells the image form for exactly the centered external media shape', () => { const url = 'https://example.com/moon.png' const single = (attrs: AdfAttributes, ...content: AdfNode[]): AdfDocument => - document({ attrs: { layout: 'center' }, content: [{ attrs, content, type: 'media' }], type: 'mediaSingle' }) + document({ attrs: { layout: 'center' }, content: [content.length === 0 ? { attrs, type: 'media' } : { attrs, content, type: 'media' }], type: 'mediaSingle' }) assert.equal(markdown(adfToMarkdown(single({ alt: 'The moon', type: 'external', url }))), `![The moon](${url})\n`) assert.equal(markdown(adfToMarkdown(single({ type: 'external', url }))), `![](${url})\n`) assert.equal(markdown(adfToMarkdown(single({ alt: 'a [b] c', type: 'external', url }))), `![a \\[b\\] c](${url})\n`) @@ -701,7 +697,8 @@ test('spells the directive marks around the longest run they cover', () => { const emitted = (...content: AdfNode[]): string => markdown(adfToMarkdown(document(paragraph(...content)))) const underline: AdfMark = { type: 'underline' } assert.equal(emitted(marked('x', underline)), '!adf:underline[x]\n') - assert.equal(emitted(marked('a', underline), marked('b', underline)), '!adf:underline[ab]\n') + assert.equal(emitted(marked('a', underline), marked('b', underline)), '!adf:underline[a!adf:textBreak{}b]\n') + assert.equal(emitted(marked('a', { attrs: {}, type: 'underline' }), marked('b', underline)), '!adf:underline[a]{attrs=empty}!adf:underline[b]\n') assert.equal(emitted(marked('x', { attrs: { type: 'sub' }, type: 'subsup' })), '!adf:subsup[x]{type=sub}\n') assert.equal(emitted(marked('x', { attrs: { color: '#ae2e24' }, type: 'textColor' })), '!adf:textColor[x]{color="#ae2e24"}\n') assert.equal(emitted(marked('x', { attrs: { color: '#091e42', size: 2 }, type: 'border' })), '!adf:border[x]{color="#091e42" size=2}\n') @@ -746,5 +743,5 @@ test('carries whitespace CommonMark strips in the reserved text directive', () = test("joins a mark run's segments as a walk rather than as one call's arguments", () => { const run = Array.from({ length: 200000 }, (): AdfNode => ({ marks: [{ type: 'strong' }], text: 'a', type: 'text' })) - assert.equal(markdown(adfToMarkdown(document({ content: run, type: 'paragraph' }))), `**${'a'.repeat(200000)}**\n`) + assert.equal(markdown(adfToMarkdown(document({ content: run, type: 'paragraph' }))), `**${Array.from({ length: 200000 }, () => 'a').join('!adf:textBreak{}')}**\n`) }) diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index 0d41fe4..1a63f78 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -1,16 +1,16 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { Flavour } from '../plain-conventions.ts' -import { adfDocumentFault, carriesOnly, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' +import { adfDocumentFault, carriesOnly, nodeAttrs, nodeContent } from '../../adf/document.ts' import { alertMarker, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../plain-conventions.ts' -import { blockDirectiveForm, listBreakSpelling } from '../block-directive.ts' +import { blockDirectiveForm, documentSpelling, listBreakSpelling } from '../block-directive.ts' import { blockNodeModel, blockNodes } from '../../adf/block-nodes.ts' import { carriedBlock } from '../opaque-carry.ts' import { emitInlineLine } from './inline-line.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { fencedCodeBlock } from '../commonmark/backtick-runs.ts' import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts' -import { languageSlot } from '../code-language.ts' +import { languageSlot, type LanguageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' import { spellBlockDirectiveOpener } from './block-directive-spelling.ts' import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts' @@ -40,7 +40,8 @@ export function writeMarkdown(document: AdfDocument, flavour: Flavour): Result 0) return undefined + return success(commonMarkText(fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', only))) } +// spec/flavour.md, The CommonMark blocks: one fence per text node, the language on each; with no fence to carry it, the attribute does. function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number): Result { - const slot = languageSlot(nodeAttrs(node)['language']) + const texts = fencedTexts(node, path) + if (!texts.ok) return texts + if (texts.value === undefined) return commonMarkLine(carriedBlock(node, path, depth)) + const slot: LanguageSlot = texts.value.length === 0 ? { kind: 'attribute' } : languageSlot(nodeAttrs(node)['language']) const opener = spellBlockDirectiveOpener(node, model, path, slot.kind === 'attribute' ? [] : ['language']) if (opener === undefined) return commonMarkLine(carriedBlock(node, path, depth)) if (!opener.ok) return opener - const text = codeBlockText(node, path) - if (!text.ok) return text - return success(directivePair(node, opener.value, fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', text.value))) + const info = slot.kind === 'fence' ? slot.info : '' + return success(directivePair(node, opener.value, texts.value.map((text) => fencedCodeBlock(info, text)).join('\n'))) } -function codeBlockText(node: AdfNode, path: ConvertErrorPath): Result { - let text = '' - for (const [index, child] of nodeContent(node).entries()) { +// The text each fence holds, or `undefined` where a child is no plain text node, which the carry holds instead. +function fencedTexts(node: AdfNode, path: ConvertErrorPath): Result { + if (node.content === undefined) return success(['']) + if (node.content.some((child) => child.type !== 'text' || child.attrs !== undefined || child.content !== undefined || child.marks !== undefined)) return success(undefined) + const texts: string[] = [] + for (const [index, child] of node.content.entries()) { const childPath = [...path, 'content', index] - if ( - child.type !== 'text' || - typeof child.text !== 'string' || - child.text === '' || - nodeContent(child).length > 0 || - nodeMarks(child).length > 0 || - Object.keys(nodeAttrs(child)).length > 0 - ) { - return failure('unsupported-node-shape', `a codeBlock holds plain text nodes only: this ${child.type} node is not one`, childPath) - } + if (typeof child.text !== 'string' || child.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', childPath) if (/\r/.test(child.text)) return failure('unspellable-character', 'a codeBlock holds no carriage return CommonMark keeps: this text holds one', childPath) if (holdsNullCharacter(child.text)) return failure('unspellable-character', 'a codeBlock holds a null character CommonMark replaces', childPath) - text += child.text + texts.push(child.text) } - return success(text) + return success(texts) } function tryHeading(node: AdfNode, path: ConvertErrorPath, flavour: Flavour): Result | undefined { @@ -340,7 +340,7 @@ function directiveItems(items: readonly WalkedItem[]): PlacedBlock[] { function listStart(node: AdfNode, items: number): number | undefined { if (node.type !== 'orderedList') return 0 const start = nodeAttrs(node)['order'] - if (typeof start !== 'number' || !Number.isInteger(start) || start < 0 || start > largestListMarker) return undefined + if (typeof start !== 'number' || !Number.isInteger(start) || start < 0 || Object.is(start, -0) || start > largestListMarker) return undefined return start + items - 1 > largestListMarker ? undefined : start } diff --git a/src/markdown/emit/block-directive-spelling.ts b/src/markdown/emit/block-directive-spelling.ts index 4d4a964..9c42408 100644 --- a/src/markdown/emit/block-directive-spelling.ts +++ b/src/markdown/emit/block-directive-spelling.ts @@ -5,6 +5,7 @@ import { blockArgument, markValues, marksAttribute } from '../block-directive.ts import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { isBareToken, spellAttributes, spellDirectiveOpener, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts' import { overNested } from '../../json-value.ts' +import { spellEmptyKeys } from '../empty-keys.ts' import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts' export function spellBlockDirectiveOpener(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, spelledByBody: readonly string[] = []): Result | undefined { @@ -14,7 +15,7 @@ export function spellBlockDirectiveOpener(node: AdfNode, model: BlockNodeModel, const spelled = argumentAttribute === undefined ? spelledByBody : [argumentAttribute, ...spelledByBody] const pairs = vocabularyPairs(nodeAttrs(node), model.attributes, spelled) if (pairs === undefined) return undefined - const spelledPairs = spellVocabulary(pairs) + const spelledPairs = [...spellVocabulary(pairs), ...spellEmptyKeys(node)] const marks = nodeMarks(node) if (marks.length > 0) { const values = markValues(marks) diff --git a/src/markdown/emit/inline-directive-spelling.ts b/src/markdown/emit/inline-directive-spelling.ts index 0680214..69f8e76 100644 --- a/src/markdown/emit/inline-directive-spelling.ts +++ b/src/markdown/emit/inline-directive-spelling.ts @@ -2,9 +2,10 @@ import type { AdfNode } from '../../adf/document.ts' import type { InlineNodeModel } from '../../adf/inline-nodes.ts' import { nodeAttrs } from '../../adf/document.ts' import { spellAttributes, spellVocabulary } from '../directive-syntax.ts' +import { spellEmptyKeys } from '../empty-keys.ts' import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts' export function spellInlineNodeAttributes(node: AdfNode, model: InlineNodeModel): string | undefined { const pairs = vocabularyPairs(nodeAttrs(node), model.attributes, model.textAttribute === undefined ? [] : [model.textAttribute]) - return pairs === undefined ? undefined : spellAttributes(spellVocabulary(pairs)) + return pairs === undefined ? undefined : spellAttributes([...spellVocabulary(pairs), ...spellEmptyKeys(node)]) } diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index 6e719e0..5d4c3d2 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -6,14 +6,14 @@ import { assembleInlineLine, isSyntax, type InlineEscaping, type InlineSegment, import { carriedInline } from '../opaque-carry.ts' import { claimsLine, holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts' import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts' +import { emptyKeys, identicalMark, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { escapeUnbalanced, spellDestination } from '../commonmark/link-syntax.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { highlightDelimiter } from '../plain-conventions.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { largestNesting } from '../../nesting.ts' import { longestBacktickRun } from '../commonmark/backtick-runs.ts' -import { nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' -import { sameMark } from '../../adf/editor-normal.ts' +import { readsAsOne, textBreakSpelling } from '../text-break.ts' import { slotLineEndingFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts' import { spellInlineNodeAttributes } from './inline-directive-spelling.ts' import { spellTextDirective } from '../text-directive.ts' @@ -186,6 +186,7 @@ function emitRun(nodes: readonly AdfNode[], depth: number, firstIndex: number, c const runs = inlineRuns(nodes, depth, firstIndex, context.carried) const segments: InlineSegment[] = [] for (const [offset, run] of runs.entries()) { + if (partsText(runs[offset - 1], run, context.carried)) segments.push(syntax(textBreakSpelling)) const runContext = { ...context, atBlockEnd: context.atBlockEnd && offset === runs.length - 1 } const emitted = run.kind === 'plain' ? emitLeaf(run.node, runContext, run.index) : emitMarkedRun(run.nodes, run.mark, depth, run.index, runContext) if (!emitted.ok) return emitted @@ -206,12 +207,18 @@ function inlineRuns(nodes: readonly AdfNode[], depth: number, firstIndex: number continue } const previous = runs[runs.length - 1] - if (previous?.kind === 'marked' && sameMark(previous.mark, mark)) previous.nodes.push(node) + if (previous?.kind === 'marked' && identicalMark(previous.mark, mark)) previous.nodes.push(node) else runs.push({ index, kind: 'marked', mark, nodes: [node] }) } return runs } +function partsText(previous: InlineRun | undefined, run: InlineRun, carried: ReadonlySet): boolean { + if (previous?.kind !== 'plain' || run.kind !== 'plain') return false + if (carries(previous.node, carried, previous.index) || carries(run.node, carried, run.index)) return false + return readsAsOne(previous.node, run.node) +} + function nodePath(context: InlineContext, index: number): ConvertErrorPath { return [...context.path, 'content', index] } @@ -261,7 +268,7 @@ function emitInlineDirective(node: AdfNode, model: InlineNodeModel, index: numbe } function emitText(node: AdfNode, context: InlineContext, index: number, path: ConvertErrorPath): Result { - if (Object.keys(nodeAttrs(node)).length > 0) return success({ carry: { first: index, last: index } }) + if (node.attrs !== undefined || emptyKeys(node).length > 0) return success({ carry: { first: index, last: index } }) if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) if (nodeContent(node).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) if (/\r/.test(node.text)) return failure('unspellable-character', 'a text node holds a carriage return CommonMark rewrites', path) @@ -277,7 +284,7 @@ function emitMarkedRun(nodes: readonly AdfNode[], mark: AdfMark, depth: number, if (mark.type === 'backgroundColor' && context.flavour === 'plain') return emitHighlight(nodes, depth, range, context) const spelling = markSpelling(mark.type) if (spelling === undefined) return success({ carry: range }) - const attributes = spellMarkAttributes(mark, spelling.attributes) + const attributes = spellMarkAttributes(mark, spelling) if (attributes === undefined) return success({ carry: range }) if (spelling.kind === 'code') return emitCodeSpan(nodes, depth, range, path) if (spelling.kind === 'emphasis') return emitEmphasis(nodes, spelling.spelling, depth, range, context) @@ -312,19 +319,20 @@ function emitHighlight(nodes: readonly AdfNode[], depth: number, range: NodeRang }) } +// Each node its own span: CommonMark reads two adjacent text nodes in one span back as one. function emitCodeSpan(nodes: readonly AdfNode[], depth: number, range: NodeRange, path: ConvertErrorPath): Result { - let text = '' + const spans: string[] = [] for (const node of nodes) { - if (node.type !== 'text' || nodeMarks(node).length !== depth + 1) return success({ carry: range }) - if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) + if (node.type !== 'text' || nodeMarks(node).length !== depth + 1 || node.attrs !== undefined || emptyKeys(node).length > 0) return success({ carry: range }) + const { text } = node + if (typeof text !== 'string' || text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) if (nodeContent(node).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) - text += node.text + if (/[\n\r]/.test(text)) return success({ carry: range }) + if (holdsNullCharacter(text)) return failure('unspellable-character', 'a code span holds a null character CommonMark replaces', path) + const fence = '`'.repeat(longestBacktickRun(text) + 1) + spans.push(`${fence}${needsPadding(text) ? ` ${text} ` : text}${fence}`) } - if (/[\n\r]/.test(text)) return success({ carry: range }) - if (holdsNullCharacter(text)) return failure('unspellable-character', 'a code span holds a null character CommonMark replaces', path) - const fence = '`'.repeat(longestBacktickRun(text) + 1) - const padded = needsPadding(text) ? ` ${text} ` : text - return success({ segments: [syntax(`${fence}${padded}${fence}`)] }) + return success({ segments: [syntax(spans.join(textBreakSpelling))] }) } function needsPadding(text: string): boolean { diff --git a/src/markdown/emit/plain-reduction.test.ts b/src/markdown/emit/plain-reduction.test.ts index 7f6ac56..8308214 100644 --- a/src/markdown/emit/plain-reduction.test.ts +++ b/src/markdown/emit/plain-reduction.test.ts @@ -177,7 +177,7 @@ test('keeps the CommonMark blocks in their spelling and drops their attributes a assert.equal(plain(node('paragraph', localId, text('x')), node('heading', { level: 2, localId: '01a0d99b-1f57-7fec-94ae-50c2ee25c9de' }, text('h'))), 'x\n\n## h\n') assert.equal(plain({ attrs: localId, content: [said('q')], marks: [{ type: 'breakout' }], type: 'blockquote' }), '> q\n') assert.equal(plain(node('codeBlock', { language: 'ts', wrap: true }, text('a\r\nb\u0000'))), '```ts\na\nb\n```\n') - assert.equal(plain(node('codeBlock', { language: 'carry' }, text('a'), { type: 'hardBreak' }, text('b', strong))), '```\na\nb\n```\n') + assert.equal(plain(node('codeBlock', { language: 'adf:x' }, text('a'), { type: 'hardBreak' }, text('b', strong))), '```\na\nb\n```\n') assert.equal(plain(node('codeBlock', {})), '```\n```\n') assert.equal(plain(node('rule', { color: '#000' })), '---\n') assert.equal(plain(node('orderedList', { localId: '01a0d99b-1f58-7b95-829b-6f9860371d54', order: 3 }, item(said('c')))), '3. c\n') diff --git a/src/markdown/emit/plain-reduction.ts b/src/markdown/emit/plain-reduction.ts index 145ecf4..593c4fe 100644 --- a/src/markdown/emit/plain-reduction.ts +++ b/src/markdown/emit/plain-reduction.ts @@ -8,6 +8,7 @@ import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' import { taskMarker } from '../plain-conventions.ts' +import { toEditorNormal } from '../../adf/editor-normal.ts' // depth: the level the node reduced stands at, counted as the emitter counts it. type Reduction = { depth: number; memo: SpellingMemo; path: ConvertErrorPath } @@ -50,8 +51,9 @@ export function reduceToPlain(document: AdfDocument): Result { const fault = adfDocumentFault(document) if (fault !== undefined) return faulted(fault, []) if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) - const blocks = reduceBlocks(nodeContent(document), { depth: 0, memo: new Map(), path: [] }) - return blocks.ok ? success({ content: blocks.value, type: 'doc', version: 1 }) : blocks + // The plain flavour is lossy: it reads and writes editor-normal ADF, whose shapes CommonMark spells. + const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] }) + return blocks.ok ? success({ content: nodeContent(toEditorNormal({ content: blocks.value, type: 'doc', version: 1 })).slice(), type: 'doc', version: 1 }) : blocks } function reduceBlocks(nodes: readonly AdfNode[], reduction: Reduction): Result { @@ -262,7 +264,8 @@ function listItem(blocks: Result): Result { // A list item's first line reads as no rule and holds no line of spaces alone: the rule and the spaces give way. function itemOf(blocks: readonly AdfNode[]): AdfNode { const rules = blocks.findIndex((block) => block.type !== 'rule') - return { content: blankedCode(blocks.slice(rules === -1 ? blocks.length : rules)), type: 'listItem' } + const content = blankedCode(blocks.slice(rules === -1 ? blocks.length : rules)) + return content.length === 0 ? { type: 'listItem' } : { content, type: 'listItem' } } function blankedCode(blocks: readonly AdfNode[]): AdfNode[] { diff --git a/src/markdown/empty-keys.ts b/src/markdown/empty-keys.ts new file mode 100644 index 0000000..b59008d --- /dev/null +++ b/src/markdown/empty-keys.ts @@ -0,0 +1,30 @@ +import type { DirectiveAttributes, DirectiveValue, Read } from './directive-syntax.ts' +import type { EmptyKey } from '../adf/document.ts' +import { emptyKeys } from '../adf/document.ts' +import { unsupportedNodeShape } from './directive-syntax.ts' + +type EmptyKeysRead = { empty: ReadonlySet; rest: Map } + +const emptyValue = 'empty' + +// spec/flavour.md, Attributes: no attribute value spells an empty object or array. +export function spellEmptyKeys(held: Parameters[0]): [string, string][] { + return emptyKeys(held).map((key) => [key, emptyValue]) +} + +export function spellsEmpty(value: DirectiveValue | undefined): boolean { + return value?.spelling === emptyValue +} + +export function readEmptyKeys(attributes: DirectiveAttributes, keys: readonly EmptyKey[]): Read { + const rest = new Map(attributes) + const empty = new Set() + for (const key of keys) { + const spelled = rest.get(key) + if (spelled === undefined) continue + if (!spellsEmpty(spelled)) return { fault: unsupportedNodeShape(`the reserved key ${key} reads ${key}=${emptyValue} alone: this one spells ${key}=${spelled.spelling}`) } + empty.add(key) + rest.delete(key) + } + return { value: { empty, rest } } +} diff --git a/src/markdown/mark-spellings.ts b/src/markdown/mark-spellings.ts index a885262..d9a814c 100644 --- a/src/markdown/mark-spellings.ts +++ b/src/markdown/mark-spellings.ts @@ -7,6 +7,7 @@ import { holdsEntityReference } from './commonmark/entity-references.ts' import { isAutolink } from './commonmark/grammar.ts' import { isMarkType, markAttributes } from '../adf/mark-attributes.ts' import { nodeAttrs, nodeMarks } from '../adf/document.ts' +import { spellEmptyKeys } from './empty-keys.ts' import { vocabularyPairs } from '../adf/attribute-vocabulary.ts' type Spelling = { kind: 'code' | 'directive' | 'link'; spelling?: undefined } | { kind: 'emphasis'; spelling: string } @@ -55,7 +56,10 @@ function isBareLink(nodes: readonly AdfNode[], href: string, marksInside: number return nodes.length === 1 && node !== undefined && node.type === 'text' && node.text === href && nodeMarks(node).length === marksInside } -export function spellMarkAttributes(mark: AdfMark, vocabulary: AttributeVocabulary): string | undefined { - const pairs = vocabularyPairs(nodeAttrs(mark), vocabulary, []) - return pairs === undefined ? undefined : spellAttributes(spellVocabulary(pairs)) +// `undefined` where the spelling holds no such attrs: only a directive spells attrs=empty. +export function spellMarkAttributes(mark: AdfMark, spelling: MarkSpelling): string | undefined { + const pairs = vocabularyPairs(nodeAttrs(mark), spelling.attributes, []) + const empty = spellEmptyKeys(mark) + if (pairs === undefined || (empty.length > 0 && spelling.kind !== 'directive')) return undefined + return spellAttributes([...spellVocabulary(pairs), ...empty]) } diff --git a/src/markdown/opaque-carry.ts b/src/markdown/opaque-carry.ts index 0e03467..0a113d6 100644 --- a/src/markdown/opaque-carry.ts +++ b/src/markdown/opaque-carry.ts @@ -1,32 +1,41 @@ import type { AdfNode } from '../adf/document.ts' import type { DirectiveSpan, Read } from './directive-syntax.ts' import type { JsonSpelling } from '../canonical-json.ts' +import type { JsonValue } from '../json-value.ts' import { failure, success, type ConvertErrorPath, type Result } from '../result.ts' import { fencedCodeBlock } from './commonmark/backtick-runs.ts' +import { infoStringCarries } from './commonmark/grammar.ts' import { isAdfNode } from '../adf/document.ts' import { isJsonValue, nestingDepth, overNested } from '../json-value.ts' import { largestNesting } from '../nesting.ts' import { malformedDirective, readSoleStringAttribute, spellAttributes, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts' import { serializeCanonicalJson } from '../canonical-json.ts' +export const carryFencePrefix = 'adf:' + export const carryName = 'carry' const jsonAttribute = 'json' +// spec/flavour.md, The opaque carry: the info string names the type wherever it carries the type back. export function carriedBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result<{ headroom: number; text: string }> { - const json = carriedJson(node, 'two-space', path, largestNesting - depth) + const { type, ...untyped } = node + const named = infoStringCarries(type) + const json = carriedJson(node, named ? untyped : node, 'two-space', path, largestNesting - depth) if (!json.ok) return json - return success({ headroom: json.value.headroom, text: fencedCodeBlock(carryName, json.value.json) }) + return success({ headroom: json.value.headroom, text: fencedCodeBlock(`${carryFencePrefix}${named ? type : ''}`, json.value.json) }) } export function carriedInline(node: AdfNode, path: ConvertErrorPath): Result { - const json = carriedJson(node, 'compact', path, largestNesting) + const json = carriedJson(node, node, 'compact', path, largestNesting) if (!json.ok) return json return success(spellInlineLeafDirective(carryName, spellAttributes([[jsonAttribute, spellStringAttribute(json.value.json)]]))) } -export function readCarriedBlock(body: string, depth: number): Read { - return readCarriedJson(body, 'two-space', largestNesting - depth) +export function readCarriedBlock(type: string, body: string, depth: number): Read { + const read = readCarriedJson(body, 'two-space', largestNesting - depth, type) + if (read.fault !== undefined || type !== '' || !infoStringCarries(read.value.type)) return read + return { fault: unsupportedNodeShape(`the carry fence names a type its info string carries: spell it ${carryFencePrefix}${read.value.type}`) } } export function readCarriedInline(span: DirectiveSpan): Read | undefined { @@ -36,28 +45,38 @@ export function readCarriedInline(span: DirectiveSpan): Read | undefine return readCarriedJson(spelled.value, 'compact', largestNesting) } -function carriedJson(node: AdfNode, spelling: JsonSpelling, path: ConvertErrorPath, levels: number): Result<{ headroom: number; json: string }> { +function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, path: ConvertErrorPath, levels: number): Result<{ headroom: number; json: string }> { const headroom = levels - nestingDepth(node) - if (!isJsonValue(node) || headroom < 0) { + if (!isJsonValue(spelled) || headroom < 0) { return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path) } - return success({ headroom, json: serializeCanonicalJson(node, spelling) }) + return success({ headroom, json: serializeCanonicalJson(spelled, spelling) }) } -function readCarriedJson(raw: string, spelling: JsonSpelling, levels: number): Read { +// `type` is the one the fence's info string names, `undefined` for the inline carry. +function readCarriedJson(raw: string, spelling: JsonSpelling, levels: number, type?: string): Read { const parsed = parseJsonText(raw) if (parsed === undefined) return { fault: malformedDirective('the opaque carry holds invalid JSON') } const { value } = parsed if (!isJsonValue(value)) return { fault: unsupportedNodeShape('the opaque carry holds a number JSON cannot spell') } - if (overNested(value, levels)) { + const typed = type === undefined || type === '' ? { value } : typedValue(value, type) + if (typed.fault !== undefined) return typed + const held = typed.value + if (overNested(held, levels)) { return { fault: { code: 'unsupported-nesting-depth', message: `a carried node's JSON nests deeper than the ${levels} levels its position leaves` } } } if (serializeCanonicalJson(value, spelling) !== raw) { const shape = spelling === 'compact' ? 'compact, keys sorted' : 'two-space indent, keys sorted' return { fault: unsupportedNodeShape(`the opaque carry spells its node's JSON canonically: ${shape}`) } } - if (!isAdfNode(value)) return { fault: unsupportedNodeShape("the opaque carry holds one ADF node's JSON: this JSON is no ADF node") } - return { value } + if (!isAdfNode(held)) return { fault: unsupportedNodeShape("the opaque carry holds one ADF node's JSON: this JSON is no ADF node") } + return { value: held } +} + +function typedValue(value: JsonValue, type: string): Read { + if (value === null || typeof value !== 'object' || Array.isArray(value)) return { value } + if ('type' in value) return { fault: unsupportedNodeShape(`the ${carryFencePrefix}${type} fence names its node's type: this JSON holds a type as well`) } + return { value: { ...value, type } } } function parseJsonText(raw: string): { value: unknown } | undefined { diff --git a/src/markdown/parse/directive-marks.ts b/src/markdown/parse/directive-marks.ts index f1b7df6..5d42ecd 100644 --- a/src/markdown/parse/directive-marks.ts +++ b/src/markdown/parse/directive-marks.ts @@ -3,8 +3,9 @@ import type { ConvertFault } from '../../result.ts' import type { DirectiveAttributes } from '../directive-syntax.ts' import type { MarkSpelling } from '../mark-spellings.ts' import { directivePrefix } from '../directive-syntax.ts' -import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' +import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { markSpelling } from '../mark-spellings.ts' +import { readEmptyKeys } from '../empty-keys.ts' import { readVocabulary } from './directive-attributes.ts' export function readDirectiveMark(name: string, attributes: DirectiveAttributes, path: ConvertErrorPath): Result | undefined { @@ -12,8 +13,11 @@ export function readDirectiveMark(name: string, attributes: DirectiveAttributes, if (spelling === undefined) return undefined const markdown = markdownForm(spelling) if (markdown !== undefined) return failure('unsupported-node-shape', `${name} is spelled ${markdown}, never as a directive`, path) - const attrs = readVocabulary(name, attributes, spelling.attributes, undefined, path) + const empty = readEmptyKeys(attributes, ['attrs']) + if (empty.fault !== undefined) return faulted(empty.fault, path) + const attrs = readVocabulary(name, empty.value.rest, spelling.attributes, undefined, path) if (!attrs.ok) return attrs + if (empty.value.empty.has('attrs')) return Object.keys(attrs.value).length === 0 ? success({ attrs: {}, type: name }) : failure('unsupported-node-shape', `${name} spells attrs=empty beside an attribute it holds`, path) return success(Object.keys(attrs.value).length === 0 ? { type: name } : { attrs: attrs.value, type: name }) } diff --git a/src/markdown/parse/directive-nodes.ts b/src/markdown/parse/directive-nodes.ts index 7235b69..c738fb1 100644 --- a/src/markdown/parse/directive-nodes.ts +++ b/src/markdown/parse/directive-nodes.ts @@ -1,4 +1,4 @@ -import type { AdfAttributes, AdfMark, AdfNode } from '../../adf/document.ts' +import type { AdfAttributes, AdfMark, AdfNode, EmptyKey } from '../../adf/document.ts' import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { ConvertFault } from '../../result.ts' import type { DirectiveAttributes, DirectiveValue } from '../directive-syntax.ts' @@ -7,15 +7,18 @@ import { attributeNestingMessage, nodeAttrs, nodeContent, nodeMarks } from '../. import { attributeValue, directivePrefix, spellAttributeValue, unknownDirectiveFault } from '../directive-syntax.ts' import { blockArgument, blockDirectiveForm, marksAttribute, readMarkValues } from '../block-directive.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' -import { carryName } from '../opaque-carry.ts' +import { carryFencePrefix, carryName } from '../opaque-carry.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { inlineMarkSpellingFault } from './directive-marks.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' +import { readEmptyKeys, spellsEmpty } from '../empty-keys.ts' import { readVocabulary } from './directive-attributes.ts' import { slotLineEndingFault } from '../directive-syntax.ts' +import { textBreakName } from '../text-break.ts' import { textDirectiveName } from '../text-directive.ts' -export type BlockDirectiveNode = { contentModel: BlockNodeModel['contentModel']; node: AdfNode } +// `emptyContent` is whether the directive spells content=empty, which holds no body. +export type BlockDirectiveNode = { contentModel: BlockNodeModel['contentModel']; emptyContent: boolean; node: AdfNode } export function readBlockDirectiveNode( name: string, @@ -24,12 +27,15 @@ export function readBlockDirectiveNode( path: ConvertErrorPath, ): Result { if (name === carryName) { - return failure('malformed-directive', `the name ${carryName} is reserved for the opaque carry, whose block form is the ${carryName} fence`, path) + return failure('malformed-directive', `the name ${carryName} is reserved for the opaque carry, whose block form is the ${carryFencePrefix} fence`, path) } const model = blockNodeModel(name) if (model === undefined) return faulted(inlineSpellingFault(name) ?? unknownDirectiveFault(name), path) const argumentKey = blockArgument(name) - const rest = new Map(attributes) + const spelled = attributes.get(marksAttribute) + const empty = readEmptyKeys(attributes, spellsEmpty(spelled) ? ['attrs', 'content', 'marks'] : ['attrs', 'content']) + if (empty.fault !== undefined) return faulted(empty.fault, path) + const { rest } = empty.value rest.delete(marksAttribute) const elsewhere: Elsewhere | undefined = argumentKey === undefined ? undefined : { key: argumentKey, slot: 'argument' } const attrs = readVocabulary(name, rest, model.attributes, elsewhere, path) @@ -38,10 +44,11 @@ export function readBlockDirectiveNode( if (argumentKey === undefined) return failure('unsupported-node-shape', `${name} takes no argument: this one spells one`, path) attrs.value[argumentKey] = argument } - const spelled = attributes.get(marksAttribute) - const marks: Result = spelled === undefined ? success(undefined) : readMarks(name, spelled, path) + const marks: Result = spelled === undefined || spellsEmpty(spelled) ? success(undefined) : readMarks(name, spelled, path) if (!marks.ok) return marks - return success({ contentModel: model.contentModel, node: namedNode(name, attrs.value, marks.value) }) + const node = namedNode(name, attrs.value, marks.value, empty.value.empty, path) + if (!node.ok) return node + return success({ contentModel: model.contentModel, emptyContent: empty.value.empty.has('content'), node: node.value }) } export function readInlineDirectiveNode( @@ -55,7 +62,9 @@ export function readInlineDirectiveNode( const slot = model.textAttribute if (slot === undefined && content !== undefined) return failure('unsupported-node-shape', `${name} takes no content: this one holds some`, path) const elsewhere: Elsewhere | undefined = slot === undefined ? undefined : { key: slot, slot: 'content' } - const attrs = readVocabulary(name, attributes, model.attributes, elsewhere, path) + const empty = readEmptyKeys(attributes, ['attrs', 'content', 'marks']) + if (empty.fault !== undefined) return faulted(empty.fault, path) + const attrs = readVocabulary(name, empty.value.rest, model.attributes, elsewhere, path) if (!attrs.ok) return attrs if (slot !== undefined && content !== undefined) { const text = slotText(content) @@ -66,14 +75,14 @@ export function readInlineDirectiveNode( if (spans !== undefined) return faulted(spans, path) attrs.value[slot] = text } - return success(namedNode(name, attrs.value, undefined)) + return namedNode(name, attrs.value, undefined, empty.value.empty, path) } // A name the other position spells names that spelling, never the code a later MINOR may fill (docs/decisions.md §Which code a cause takes). function inlineSpellingFault(name: string): ConvertFault | undefined { const mark = inlineMarkSpellingFault(name) if (mark !== undefined) return mark - if (inlineNodeModel(name) === undefined && name !== textDirectiveName) return undefined + if (inlineNodeModel(name) === undefined && name !== textDirectiveName && name !== textBreakName) return undefined return { code: 'unsupported-node-shape', message: `${name} takes the inline form, ${directivePrefix}${name}{…}, never the block form` } } @@ -100,7 +109,11 @@ function readMarks(type: string, spelled: DirectiveValue, path: ConvertErrorPath return success(marks) } -function namedNode(type: string, attrs: AdfAttributes, marks: readonly AdfMark[] | undefined): AdfNode { - const named = Object.keys(attrs).length === 0 ? { type } : { attrs, type } - return marks === undefined ? named : { ...named, marks: [...marks] } +function namedNode(type: string, attrs: AdfAttributes, marks: readonly AdfMark[] | undefined, empty: ReadonlySet, path: ConvertErrorPath): Result { + const held = Object.keys(attrs).length > 0 + if (held && empty.has('attrs')) return failure('unsupported-node-shape', `${type} spells attrs=empty beside an attribute it holds`, path) + const node: AdfNode = held || empty.has('attrs') ? { attrs, type } : { type } + if (empty.has('content')) node.content = [] + if (marks !== undefined || empty.has('marks')) node.marks = [...(marks ?? [])] + return success(node) } diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index 192fd1e..7aeabe8 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -10,16 +10,16 @@ import { commonMarkLink, linkHref } from '../mark-spellings.ts' import { delimiterFlags, matchEmphasis, runLength } from '../commonmark/emphasis-matching.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { highlightDelimiter, highlightFlanking } from '../plain-conventions.ts' +import { identicalMarks, nodeAttrs, nodeMarks } from '../../adf/document.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' -import { mergeAdjacentText, sameMarks } from '../../adf/editor-normal.ts' import { noSpans, readInlineDirective } from '../directive-syntax.ts' -import { nodeAttrs, nodeMarks } from '../../adf/document.ts' import { normalizeLabel, readInlineTarget, readLabel } from '../commonmark/link-syntax.ts' import { openingLinkTakesDirective } from '../emit/inline-line.ts' import { readCarriedInline } from '../opaque-carry.ts' import { readDirectiveMark } from './directive-marks.ts' import { readInlineDirectiveNode } from './directive-nodes.ts' import { readTextDirective } from '../text-directive.ts' +import { readsAsOne, textBreakName, textBreakSpelling } from '../text-break.ts' export type InlineContent = { carry?: undefined; image: AdfNode; nodes?: undefined } | { carry: boolean; image?: undefined; nodes: AdfNode[] } @@ -31,8 +31,10 @@ type HighlightDelimiter = { closes: boolean; holder: AdfNode; index: number; lin type Pairing = EmphasisPairing +// A carry piece holds a node whose marks are its own: an opaque carry, or an inline node spelling marks=empty. type Piece = | Bracket + | { kind: 'break' } | { kind: 'carry'; node: AdfNode } | { alt: string; kind: 'image'; node: AdfNode } | { kind: 'nodes'; nodes: AdfNode[] } @@ -43,6 +45,8 @@ type Run = { canClose: boolean; canOpen: boolean; character: string; index: numb // `container` is `undefined` inside a directive's content slot, the emitter's `bracketed`. type Scan = { + // Nodes a carry piece restored: they never merge, and only identity tells a carried textBreak from the leaf. + carried: Set container: LineContainer | undefined // Pieces below this have been walked for openers to deactivate: an image close folds the link-marked piece into alt text, leaving this the only record that the brackets around it are doomed. deactivatedBefore: number @@ -58,7 +62,7 @@ type Scan = { type SlotContent = { carry: boolean; nodes: AdfNode[] } -const carriedInMark = 'no mark spelling wraps an opaque carry: the carried node restores exactly, marks included' +const carriedInMark = 'no mark spelling wraps an opaque carry or an inline node spelling marks=empty: its marks are its own' const editorHighlight: AdfMark = { attrs: { color: '#f8e6a0' }, type: 'backgroundColor' } const hreflessLink = 'the link mark spells its href: this one spells none' const imageAlone = 'an image fits only as a paragraph of its own: this one sits inside other content' @@ -66,11 +70,11 @@ const linkInLink = 'no link wraps a link: the [content] this one marks already h const spellableLink = 'link takes the directive form only where CommonMark cannot spell it: this one it can, as [text](url "title") or ' export function parseInlineContent(source: string, definitions: LinkDefinitions, path: ConvertErrorPath, container: LineContainer, flavour: Flavour): Result { - return parseInline(source, definitions, path, container, noSpans, flavour === 'plain') + return parseInline(source, { carried: new Set(), container, definitions, highlights: flavour === 'plain', path, spans: noSpans }) } -function parseInline(source: string, definitions: LinkDefinitions, path: ConvertErrorPath, container: LineContainer | undefined, spans: NestedSpans, highlights: boolean): Result { - const scan: Scan = { container, deactivatedBefore: 0, definitions, highlights, openingSpellableLink: false, path, pending: '', pieces: [], source, spans } +function parseInline(source: string, context: Pick): Result { + const scan: Scan = { ...context, deactivatedBefore: 0, openingSpellableLink: false, pending: '', pieces: [], source } let index = 0 while (index < source.length) { switch (source.charAt(index)) { @@ -103,7 +107,7 @@ function parseInline(source: string, definitions: LinkDefinitions, path: Convert index = readCharacter(scan, index) } } - flush(scan, container !== undefined) + flush(scan, scan.container !== undefined) return assemble(scan) } @@ -205,8 +209,9 @@ function directivePiece(scan: Scan, span: DirectiveSpan, index: number): Result< const carried = readCarriedInline(span) if (carried !== undefined) { if (carried.fault !== undefined) return faulted(carried.fault, scan.path) - return success({ kind: 'carry', node: carried.value }) + return success(carryPiece(scan, carried.value)) } + if (span.name === textBreakName) return span.content === undefined && span.attributes.size === 0 ? success({ kind: 'break' }) : failure('unsupported-node-shape', `${textBreakName} spells the bare leaf form, ${textBreakSpelling}: this one spells more`, scan.path) const text = readTextDirective(span) if (text?.fault !== undefined) return faulted(text.fault, scan.path) if (text !== undefined) return success({ kind: 'nodes', nodes: [{ text: text.value, type: 'text' }] }) @@ -216,7 +221,12 @@ function directivePiece(scan: Scan, span: DirectiveSpan, index: number): Result< if (mark !== undefined) return mark.ok ? directiveMarkPiece(scan, span.name, mark.value, slot.value, index) : mark const node = readInlineDirectiveNode(span.name, span.attributes, slot.value?.nodes, scan.path) if (!node.ok) return node - return success({ kind: 'nodes', nodes: [node.value] }) + return success(node.value.marks === undefined ? { kind: 'nodes', nodes: [node.value] } : carryPiece(scan, node.value)) +} + +function carryPiece(scan: Scan, node: AdfNode): Piece { + scan.carried.add(node) + return { kind: 'carry', node } } function directiveMarkPiece(scan: Scan, name: string, mark: AdfMark, slot: SlotContent | undefined, index: number): Result { @@ -242,7 +252,7 @@ function refuseLinkDirective(scan: Scan, mark: AdfMark, nodes: readonly AdfNode[ function slotContent(scan: Scan, span: DirectiveSpan): Result { if (span.content === undefined) return success(undefined) - const parsed = parseInline(span.content, scan.definitions, scan.path, undefined, span.spans, false) + const parsed = parseInline(span.content, { ...scan, container: undefined, highlights: false, spans: span.spans }) if (!parsed.ok) return parsed if (parsed.value.image !== undefined) return failure('unmappable-image', imageAlone, scan.path) return success(parsed.value) @@ -262,7 +272,8 @@ function assemble(scan: Scan): Result { const only = scan.pieces[0] if (scan.pieces.length === 1 && only?.kind === 'image') return success({ image: only.node }) if (holdsImage(scan.pieces)) return failure('unmappable-image', imageAlone, scan.path) - const nodes = resolveNodes(scan.pieces, scan.path, true) + const resolved = resolveNodes(scan.pieces, scan, true) + const nodes = resolved.ok && scan.container !== undefined ? partText(resolved.value, scan) : resolved if (!nodes.ok) return nodes if (scan.openingSpellableLink) { const takesDirective = openingLinkTakesDirective(nodes.value, scan.path) @@ -272,6 +283,27 @@ function assemble(scan: Scan): Result { return success({ carry: holdsCarry(scan.pieces), nodes: nodes.value }) } +// spec/flavour.md, Inline nodes: the leaf builds no node, so only the pair it parts spells it. +function partText(nodes: readonly AdfNode[], scan: Scan): Result { + const parted: AdfNode[] = [] + for (const [index, node] of nodes.entries()) { + if (!isTextBreak(node, scan.carried)) { + parted.push(node) + continue + } + const previous = nodes[index - 1] + const next = nodes[index + 1] + if (previous === undefined || next === undefined || scan.carried.has(previous) || scan.carried.has(next) || !readsAsOne(previous, next)) { + return failure('unsupported-node-shape', `${textBreakName} parts two text nodes CommonMark reads back as one: this one parts something else`, scan.path) + } + } + return success(parted) +} + +function isTextBreak(node: AdfNode, carried: ReadonlySet): boolean { + return node.type === textBreakName && !carried.has(node) +} + function holdsCarry(pieces: readonly Piece[]): boolean { return pieces.some((piece) => piece.kind === 'carry') } @@ -383,7 +415,7 @@ function closeLink(scan: Scan, at: number, inner: readonly Piece[], definition: } if (holdsImage(inner)) return failure('unmappable-image', imageAlone, scan.path) if (holdsCarry(inner)) return failure('unsupported-node-shape', carriedInMark, scan.path) - const resolved = resolveNodes(inner, scan.path, true) + const resolved = resolveNodes(inner, scan, true) if (!resolved.ok) return resolved const nodes = resolved.value // An empty link text gives the mark no node to ride, so the brackets stay text. @@ -411,7 +443,7 @@ function truncatePieces(scan: Scan, to: number): void { function closeImage(scan: Scan, at: number, inner: readonly Piece[], definition: LinkDefinition): Result { if (definition.title !== undefined) return failure('unmappable-image', 'no media node carries a link title', scan.path) - const resolved = imageAlt(inner, scan.path) + const resolved = imageAlt(inner, scan) if (!resolved.ok) return resolved const alt = resolved.value const attrs = alt === '' ? { type: 'external', url: definition.destination } : { alt, type: 'external', url: definition.destination } @@ -420,9 +452,10 @@ function closeImage(scan: Scan, at: number, inner: readonly Piece[], definition: return success(null) } -function imageAlt(inner: readonly Piece[], path: ConvertErrorPath): Result { - const nodes = resolveNodes(inner, path, false) +function imageAlt(inner: readonly Piece[], scan: Scan): Result { + const nodes = resolveNodes(inner, scan, false) if (!nodes.ok) return nodes + if (nodes.value.some((node) => isTextBreak(node, scan.carried))) return failure('unsupported-node-shape', `${textBreakName} parts two text nodes: an image description holds plain text`, scan.path) return success(nodes.value.map(altText).join('')) } @@ -435,20 +468,35 @@ function altText(node: AdfNode): string { } // An image's alt text is plain, so `highlights` is off there and every `==` stays text. -function resolveNodes(pieces: readonly Piece[], path: ConvertErrorPath, highlights: boolean): Result { +function resolveNodes(pieces: readonly Piece[], scan: Scan, highlights: boolean): Result { const nodes = pieces.map(pieceNodes) const runs = delimiterRuns(pieces) const pairings = matchEmphasis(runs) writeUnpaired(nodes, runs, pairings) - if (!markPairings(pieces, nodes, pairings)) return failure('unsupported-node-shape', carriedInMark, path) + if (!markPairings(pieces, nodes, pairings)) return failure('unsupported-node-shape', carriedInMark, scan.path) markHighlights(pieces, nodes, highlights ? pairedHighlights(pieces, nodes) : []) - return success(mergeAdjacentText(nodes.flat())) + return success(mergeText(nodes.flat(), scan.carried)) +} + +function mergeText(nodes: readonly AdfNode[], carried: ReadonlySet): AdfNode[] { + const merged: AdfNode[] = [] + for (const node of nodes) { + const previous = merged[merged.length - 1] + if (previous !== undefined && !carried.has(previous) && !carried.has(node) && readsAsOne(previous, node)) { + merged[merged.length - 1] = { ...previous, text: `${previous.text ?? ''}${node.text ?? ''}` } + continue + } + merged.push(node) + } + return merged } // Only `imageAlt` reaches the image arm: everywhere else an image amid other content is refused first. // A highlight delimiter holds an empty text node until it pairs, so the emphasis around it marks it. function pieceNodes(piece: Piece): AdfNode[] { switch (piece.kind) { + case 'break': + return [{ type: textBreakName }] case 'carry': return [piece.node] case 'highlight': @@ -531,7 +579,7 @@ function pairedHighlights(pieces: readonly Piece[], nodes: readonly AdfNode[][]) let candidate = found[closer] while (candidate !== undefined && (!candidate.closes || candidate.position < earliest)) candidate = found[(closer += 1)] if (candidate === undefined) break - if (candidate.line !== opener.line || !sameMarks(opener.holder, candidate.holder)) continue + if (candidate.line !== opener.line || !identicalMarks(opener.holder, candidate.holder)) continue paired.push({ closer: candidate.index, opener: opener.index }) resume = candidate.position + highlightDelimiter.length } diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 283be1e..2773f6d 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -84,9 +84,21 @@ function table(...rows: AdfNode[]): AdfNode { return { content: rows, type: 'table' } } -test('builds an empty document from input holding no block', () => { - assert.deepEqual(markdownToAdf(''), { ok: true, value: { type: 'doc', version: 1 } }) - assert.deepEqual(content(markdownToAdf('\n \n\t\n')), []) +test('builds an empty document from input holding no block, and one holding no content key from the doc directive alone', () => { + assert.deepEqual(markdownToAdf(''), { ok: true, value: { content: [], type: 'doc', version: 1 } }) + assert.deepEqual(markdownToAdf('\n \n\t\n'), { ok: true, value: { content: [], type: 'doc', version: 1 } }) + assert.deepEqual(markdownToAdf('!adf:doc {content=none}\n'), { ok: true, value: { type: 'doc', version: 1 } }) + assert.deepEqual(markdownToAdf('\n!adf:doc {content=none}\n\n'), { ok: true, value: { type: 'doc', version: 1 } }) + const form = 'unsupported-node-shape: doc spells the one form !adf:doc {content=none}: this one spells another' + assert.equal(content(markdownToAdf('!adf:doc\n')), form) + assert.equal(content(markdownToAdf('!adf:doc x {content=none}\n')), form) + assert.equal(content(markdownToAdf('!adf:doc {content=empty}\n')), form) + assert.equal(content(markdownToAdf('!adf:doc {content="none"}\n')), form) + const alone = 'unsupported-node-shape: !adf:doc {content=none} spells a whole document holding no content key, alone: this one stands among other blocks' + assert.equal(content(markdownToAdf('x\n\n!adf:doc {content=none}\n')), alone) + assert.equal(content(markdownToAdf('!adf:panel info\n!adf:doc {content=none}\n!adf:/panel\n')), alone) + assert.equal(content(markdownToAdf('!adf:doc\n!adf:/doc\n')), 'malformed-directive: doc takes no body, so no !adf:/doc closes it; \\!adf: keeps the prefix literal') + assert.equal(content(markdownToAdf('a !adf:doc{content=none}\n')), 'unsupported-node-shape: doc takes the block form, !adf:doc, never the inline form') }) test('builds one paragraph from the lines a blank line does not part', () => { @@ -153,7 +165,7 @@ test('names the slot a codeBlock spells its language outside of', () => { const slot = 'unsupported-node-shape: codeBlock spells its language in the fence info string, or in the attribute where no info string carries it back' assert.equal(content(markdownToAdf('!adf:codeBlock {language=rust wrap=true}\n```\nx\n```\n!adf:/codeBlock\n')), slot) assert.equal(content(markdownToAdf('!adf:codeBlock {language=rust}\n```sql\nx\n```\n!adf:/codeBlock\n')), slot) - assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n```carry\nx\n```\n!adf:/codeBlock\n')), slot) + assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n```adf:x\nx\n```\n!adf:/codeBlock\n')), slot) assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n```a\\b\nx\n```\n!adf:/codeBlock\n')), slot) }) @@ -343,18 +355,26 @@ test('names the position a directive name the other one spells belongs to', () = }) test('names the reserved carry name a block directive spells', () => { - const reserved = 'malformed-directive: the name carry is reserved for the opaque carry, whose block form is the carry fence' + const reserved = 'malformed-directive: the name carry is reserved for the opaque carry, whose block form is the adf: fence' assert.equal(content(markdownToAdf('!adf:carry\n')), reserved) assert.equal(content(markdownToAdf('!adf:carry\nx\n!adf:/carry\n')), reserved) assert.deepEqual(content(markdownToAdf('```adf\nx\n```\n')), [{ attrs: { language: 'adf' }, content: [text('x')], type: 'codeBlock' }]) + assert.deepEqual(content(markdownToAdf('```carry\nx\n```\n')), [{ attrs: { language: 'carry' }, content: [text('x')], type: 'codeBlock' }]) }) const carried = '!adf:carry{json="{\\"type\\":\\"placeholder\\"}"}' -test('reads the carry fence back to the node its JSON holds', () => { - assert.deepEqual(content(markdownToAdf('```carry\n{\n "attrs": {\n "url": "https://example.com/x"\n },\n "type": "blockCard"\n}\n```\n')), [ +test('reads the carry fence back to the node its info string names and its JSON holds', () => { + assert.deepEqual(content(markdownToAdf('```adf:blockCard\n{\n "attrs": {\n "url": "https://example.com/x"\n }\n}\n```\n')), [ { attrs: { url: 'https://example.com/x' }, type: 'blockCard' }, ]) + assert.deepEqual(content(markdownToAdf('```adf:\n{\n "type": "a`b"\n}\n```\n')), [{ type: 'a`b' }]) + assert.equal( + content(markdownToAdf('```adf:blockCard\n{\n "type": "blockCard"\n}\n```\n')), + "unsupported-node-shape: the adf:blockCard fence names its node's type: this JSON holds a type as well", + ) + assert.equal(content(markdownToAdf('```adf:\n{\n "type": "blockCard"\n}\n```\n')), 'unsupported-node-shape: the carry fence names a type its info string carries: spell it adf:blockCard') + assert.equal(content(markdownToAdf('```adf:\n{}\n```\n')), "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node") }) test('reads the inline carry back to the node its json attribute holds', () => { @@ -365,22 +385,22 @@ test('reads the inline carry back to the node its json attribute holds', () => { test('names the invalid JSON no opaque carry holds', () => { const invalid = 'malformed-directive: the opaque carry holds invalid JSON' - assert.equal(content(markdownToAdf('```carry\n{"type":\n```\n')), invalid) - assert.equal(content(markdownToAdf('```carry\n```\n')), invalid) + assert.equal(content(markdownToAdf('```adf:x\n{"attrs":\n```\n')), invalid) + assert.equal(content(markdownToAdf('```adf:x\n```\n')), invalid) assert.equal(content(markdownToAdf('!adf:carry{json="{"}\n')), invalid) assert.equal(content(markdownToAdf('!adf:carry{json=abc}\n')), invalid) }) test('names the canonical spelling a carried JSON reads alone', () => { const canonically = "unsupported-node-shape: the opaque carry spells its node's JSON canonically: " - assert.equal(content(markdownToAdf('```carry\n{"type":"blockCard"}\n```\n')), `${canonically}two-space indent, keys sorted`) + assert.equal(content(markdownToAdf('```adf:blockCard\n{"attrs":{}}\n```\n')), `${canonically}two-space indent, keys sorted`) assert.equal(content(markdownToAdf('!adf:carry{json="{\\"type\\": \\"blockCard\\"}"}\n')), `${canonically}compact, keys sorted`) assert.equal(content(markdownToAdf('!adf:carry{json="{\\"type\\":\\"blockCard\\",\\"attrs\\":{}}"}\n')), `${canonically}compact, keys sorted`) }) test('names the node JSON an opaque carry restores alone', () => { const node = "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node" - assert.equal(content(markdownToAdf('```carry\n[]\n```\n')), node) + assert.equal(content(markdownToAdf('```adf:x\n[]\n```\n')), node) assert.equal(content(markdownToAdf('!adf:carry{json=null}\n')), node) assert.equal(content(markdownToAdf('!adf:carry{json="{\\"kind\\":\\"x\\"}"}\n')), node) }) @@ -394,7 +414,7 @@ test('names the shape the inline carry reads alone', () => { test('holds a carried JSON value to the nesting its position leaves', () => { const nested = (levels: number): string => `${'['.repeat(levels)}${']'.repeat(levels)}` - const fence = (prefix: string, levels: number): string => `${prefix}\`\`\`carry\n${prefix}${nested(levels)}\n${prefix}\`\`\`\n` + const fence = (prefix: string, levels: number): string => `${prefix}\`\`\`adf:x\n${prefix}${nested(levels)}\n${prefix}\`\`\`\n` const deeper = (levels: number): string => `unsupported-nesting-depth: a carried node's JSON nests deeper than the ${levels} levels its position leaves` assert.equal(content(markdownToAdf(`!adf:carry{json="${nested(largestNesting + 2)}"}\n`)), deeper(largestNesting)) assert.equal( @@ -407,11 +427,11 @@ test('holds a carried JSON value to the nesting its position leaves', () => { test('names the number no JSON spelling carries in an opaque carry', () => { const named = 'unsupported-node-shape: the opaque carry holds a number JSON cannot spell' assert.equal(content(markdownToAdf('!adf:carry{json="{\\"attrs\\":{\\"width\\":1e999},\\"type\\":\\"blockCard\\"}"}\n')), named) - assert.equal(content(markdownToAdf('```carry\n1e999\n```\n')), named) + assert.equal(content(markdownToAdf('```adf:x\n1e999\n```\n')), named) }) test('names the mark spelling no opaque carry sits inside', () => { - const named = 'unsupported-node-shape: no mark spelling wraps an opaque carry: the carried node restores exactly, marks included' + const named = 'unsupported-node-shape: no mark spelling wraps an opaque carry or an inline node spelling marks=empty: its marks are its own' assert.equal(content(markdownToAdf(`_a ${carried} b_\n`)), named) assert.equal(content(markdownToAdf(`**${carried}**\n`)), named) assert.equal(content(markdownToAdf(`~~a ${carried}~~\n`)), named) @@ -454,7 +474,9 @@ test('names the marks key no marks array reads back from', () => { assert.equal(content(markdownToAdf('!adf:rule {marks="[1]"}\n')), named) assert.equal(content(markdownToAdf('!adf:rule {marks="{}"}\n')), named) assert.equal(content(markdownToAdf('!adf:rule {marks=x}\n')), named) - assert.equal(content(markdownToAdf('!adf:rule {marks="[{\\"attrs\\":{},\\"type\\":\\"em\\"}]"}\n')), named) + assert.equal(content(markdownToAdf('!adf:rule {marks="[{\\"type\\":\\"em\\",\\"attrs\\":{}}]"}\n')), named) + assert.deepEqual(content(markdownToAdf('!adf:rule {marks="[{\\"attrs\\":{},\\"type\\":\\"em\\"}]"}\n')), [{ marks: [{ attrs: {}, type: 'em' }], type: 'rule' }]) + assert.deepEqual(content(markdownToAdf('!adf:rule {marks=empty}\n')), [{ marks: [], type: 'rule' }]) }) test('names the attribute a node holds no reading for', () => { @@ -487,7 +509,8 @@ test('names the argument and the body a node takes no reading for', () => { assert.equal(content(markdownToAdf('!adf:rule x\n')), 'unsupported-node-shape: rule takes no argument: this one spells one') assert.equal(content(markdownToAdf('!adf:paragraph\nOne.\n\nTwo.\n!adf:/paragraph\n')), 'unsupported-node-shape: paragraph takes one paragraph as its body: this body is not one') assert.equal(content(markdownToAdf('!adf:paragraph\n---\n!adf:/paragraph\n')), 'unsupported-node-shape: paragraph takes one paragraph as its body: this body is not one') - assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\nx\n!adf:/codeBlock\n')), 'unsupported-node-shape: codeBlock takes one code block as its body: this body is not one') + assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\nx\n!adf:/codeBlock\n')), 'unsupported-node-shape: codeBlock takes code blocks as its body: this body holds another block') + assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n!adf:/codeBlock\n')), 'unsupported-node-shape: codeBlock takes code blocks as its body: this body holds none') assert.equal(content(markdownToAdf('!adf:paragraph\n![a](/u)\n!adf:/paragraph\n')), 'unmappable-image: no ADF node carries an image inside a paragraph') assert.equal(content(markdownToAdf('Part !adf:date[now]{timestamp=1}.\n')), 'unsupported-node-shape: date takes no content: this one holds some') }) @@ -545,7 +568,7 @@ test('names the line and the offset in the input a refusal sits at, the innermos assert.deepEqual(position(markdownToAdf('- Part.\n- a b\n')), { line: 2, offset: 8 }) assert.deepEqual(position(markdownToAdf('Part.\n\n!adf:panel info\nMore.\n')), { line: 3, offset: 7 }) assert.deepEqual(position(markdownToAdf('x\n\na b\n===\n')), { line: 3, offset: 3 }) - assert.deepEqual(position(markdownToAdf('x\n\n```carry\n{\n```\n')), { line: 3, offset: 3 }) + assert.deepEqual(position(markdownToAdf('x\n\n```adf:x\n{\n```\n')), { line: 3, offset: 3 }) assert.deepEqual(position(markdownToAdf('x\n\n| a |\n')), { line: 3, offset: 3 }) assert.deepEqual(position(markdownToAdf('a\nb c\n')), { line: 1, offset: 0 }) assert.deepEqual(position(markdownToAdf('Part.\r\n\r\n
\r\n')), { line: 3, offset: 9 }) diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 8ed8a6a..55c42f0 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -5,14 +5,14 @@ import type { ConvertFault } from '../../result.ts' import type { Flavour } from '../plain-conventions.ts' import type { LineContainer } from '../line-container.ts' import type { LinkDefinitions } from './inline-content.ts' -import { carryName, readCarriedBlock } from '../opaque-carry.ts' +import { carryFencePrefix, readCarriedBlock } from '../opaque-carry.ts' import { commonMarkSpelling, type SpellingMemo } from '../emit/adf-to-markdown.ts' +import { documentName, documentSpelling, listBreakName, listBreakSpelling } from '../block-directive.ts' import { failure, faulted, positioned, success, type ConvertErrorPath, type ParseError, type Result, type SourcePosition } from '../../result.ts' import { inlineLeaves } from '../emit/plain-inline.ts' import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' import { leadingMarker, readAlertMarker, readTaskMarker } from '../plain-conventions.ts' -import { listBreakName, listBreakSpelling } from '../block-directive.ts' import { mintTaskIds } from './task-ids.ts' import { nodeAttrs, nodeContent } from '../../adf/document.ts' import { parseBlocks } from './blocks.ts' @@ -39,10 +39,15 @@ export function plainMarkdownToAdf(markdown: string): Result { const parsed = parseBlocks(markdown) + const [only] = parsed.blocks + if (parsed.blocks.length === 1 && only?.kind === 'directive' && only.name === documentName) { + const fault = documentFault(only) + return fault === undefined ? success({ type: 'doc', version: 1 }) : positioned(faulted(fault, []), only.position) + } const reading: Reading = { carried: new Set(), definitions: parsed.definitions, flavour, inExpand: false, memo: new Map() } const content = positioned(readBlocks(parsed.blocks, reading, [], 0), documentStart) if (!content.ok) return content - const document: AdfDocument = content.value.length === 0 ? { type: 'doc', version: 1 } : { content: content.value, type: 'doc', version: 1 } + const document: AdfDocument = { content: content.value, type: 'doc', version: 1 } if (flavour === 'plain') mintTaskIds(document, markdown, reading.carried) return success(document) } @@ -52,6 +57,10 @@ function readBlocks(blocks: readonly Block[], reading: Reading, path: ConvertErr const content: AdfNode[] = [] for (const [index, block] of blocks.entries()) { const nodePath = [...path, 'content', content.length] + if (block.kind === 'directive' && block.name === documentName) { + const fault = documentFault(block) ?? unsupportedNodeShape(`${documentSpelling} spells a whole document holding no content key, alone: this one stands among other blocks`) + return positioned(faulted(fault, nodePath), block.position) + } if (block.kind === 'directive' && block.name === listBreakName) { const fault = listBreakFault(block, blocks[index - 1], blocks[index + 1]) if (fault !== undefined) return positioned(faulted(fault, nodePath), block.position) @@ -73,6 +82,12 @@ function listBreakFault(block: DirectiveBlock, previous: Block | undefined, next return previous.kind === next?.kind ? undefined : partsFault() } +function documentFault(block: DirectiveBlock): ConvertFault | undefined { + const content = block.attributes.get('content') + if (block.argument === undefined && block.attributes.size === 1 && content?.spelling === 'none') return undefined + return unsupportedNodeShape(`${documentName} spells the one form ${documentSpelling}: this one spells another`) +} + function partsFault(): ConvertFault { return unsupportedNodeShape(`${listBreakName} parts two adjacent lists of one type: this one parts something else`) } @@ -214,24 +229,33 @@ function directiveNode(block: DirectiveBlock, reading: Reading, path: ConvertErr } function directiveBody(read: BlockDirectiveNode, blocks: Block[] | undefined, reading: Reading, path: ConvertErrorPath, depth: number): Result { - const { contentModel, node } = read + const { contentModel, emptyContent, node } = read if (blocks === undefined) return success(node) + if (emptyContent) return blocks.length === 0 ? success(node) : failure('unsupported-node-shape', `${node.type} spells content=empty, which holds no body: this one holds one`, path) if (contentModel === 'code') return codeDirectiveNode(node, blocks, path) if (contentModel === 'inline') return inlineBodyNode(node, blocks, reading, path) return containerNode(node, blocks, reading, path, depth) } +// spec/flavour.md, The CommonMark blocks: one fence per text node, every fence carrying the one language. function codeDirectiveNode(node: AdfNode, blocks: readonly Block[], path: ConvertErrorPath): Result { - const only = blocks.length === 1 ? blocks[0] : undefined - if (only?.kind !== 'code') return failure('unsupported-node-shape', `${node.type} takes one code block as its body: this body is not one`, path) + const fences: Extract[] = [] + for (const block of blocks) { + if (block.kind !== 'code') return failure('unsupported-node-shape', `${node.type} takes code blocks as its body: this body holds another block`, path) + fences.push(block) + } + const [first] = fences + if (first === undefined) return failure('unsupported-node-shape', `${node.type} takes code blocks as its body: this body holds none`, path) + if (fences.some((fence) => fence.language !== first.language)) return failure('unsupported-node-shape', `${node.type} holds one language, so its fences carry one info string: these differ`, path) + if (fences.length > 1 && fences.some((fence) => fence.text === '')) return failure('unsupported-node-shape', `a fence beside another spells a text node, which holds text: this one is empty`, path) const attribute = nodeAttrs(node)['language'] - const fromFence = only.language !== '' - const slot = languageSlot(fromFence ? only.language : attribute) + const fromFence = first.language !== '' + const slot = languageSlot(fromFence ? first.language : attribute) if ((slot.kind === 'fence') !== fromFence || (fromFence && attribute !== undefined)) { return failure('unsupported-node-shape', `${node.type} spells its language in the fence info string, or in the attribute where no info string carries it back`, path) } - const spelled = fromFence ? { ...node, attrs: { ...node.attrs, language: only.language } } : node - return success(withContent(spelled, only.text === '' ? [] : [{ text: only.text, type: 'text' }])) + const spelled = fromFence ? { ...node, attrs: { ...node.attrs, language: first.language } } : node + return success(withContent(spelled, first.text === '' ? [] : fences.map((fence): AdfNode => ({ text: fence.text, type: 'text' })))) } function tableNode(rows: readonly string[][], reading: Reading, path: ConvertErrorPath): Result { @@ -278,8 +302,8 @@ function listNode(node: AdfNode, items: readonly Block[][], reading: Reading, pa } function codeBlockNode(language: string, text: string, reading: Reading, path: ConvertErrorPath, depth: number): Result { - if (language === carryName) { - const carried = readCarriedBlock(text, depth) + if (language.startsWith(carryFencePrefix)) { + const carried = readCarriedBlock(language.slice(carryFencePrefix.length), text, depth) if (carried.fault !== undefined) return faulted(carried.fault, path) reading.carried.add(carried.value) return success(carried.value) diff --git a/src/markdown/text-break.ts b/src/markdown/text-break.ts new file mode 100644 index 0000000..de1174c --- /dev/null +++ b/src/markdown/text-break.ts @@ -0,0 +1,16 @@ +import type { AdfNode } from '../adf/document.ts' +import { identicalMarks } from '../adf/document.ts' +import { spellInlineLeafDirective } from './directive-syntax.ts' + +export const textBreakName = 'textBreak' + +export const textBreakSpelling = spellInlineLeafDirective(textBreakName, '') + +// Whether CommonMark reads the pair back as one text node, where neither rides the carry (spec/flavour.md, Inline nodes). +export function readsAsOne(previous: AdfNode, node: AdfNode): boolean { + return spelledAsText(previous) && spelledAsText(node) && identicalMarks(previous, node) +} + +function spelledAsText(node: AdfNode): boolean { + return node.type === 'text' && node.attrs === undefined && node.content === undefined && node.marks?.length !== 0 +} -- 2.52.0 From f180943edfe395800f063035ee0c76cd9c259d82 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:28:20 +0200 Subject: [PATCH 03/67] Record deep equality, the text break, empty keys, -0, the typed carry fence and per-node code fences; file items 52 and 53 --- AGENTS.md | 8 +- CHANGELOG.md | 18 ++++- MIGRATION.md | 6 +- README.md | 8 +- corpus/README.md | 4 +- docs/decisions.md | 106 +++++++++++++++++++++++---- spec/flavour.md | 102 +++++++++++++++++--------- src/markdown/emit/plain-reduction.ts | 2 +- todo.md | 24 +++--- 9 files changed, 203 insertions(+), 75 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index e954f90..23f0546 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,8 +10,14 @@ In `docs/decisions.md`: - Plain markdown is a flavour of the grammar - The round-trip is the product - Markdown in is a canonical fixpoint -- Equality is editor-normal +- Equality is deep +- `!adf:textBreak{}` parts text CommonMark would join +- An empty key spells `empty` +- `-0` is spelled `-0` +- Empty markdown is a document of no blocks - Unknown nodes ride the carry +- The carry fence names the node type +- A code block is a fence per text node - Foreign HTML sorts three ways - Names stay text - Directives under `!adf:` diff --git a/CHANGELOG.md b/CHANGELOG.md index 442e445..a02d444 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,10 +2,20 @@ ## Unreleased -- **Breaking:** directives, the opaque carry among them (now `carry`), are spelled under an `!adf:` - prefix (`!adf:name … !adf:/name`, `!adf:name[content]{attrs}`, `!adf:name arg {attrs}`) in place - of the `:::`/`::`/`:name` forms: text holding an unescaped `!adf:` is claimed, and `adf` is an - ordinary code block language. Convert stored markdown per `MIGRATION.md`. +- **Breaking:** directives, the inline opaque carry among them (now `!adf:carry{json="…"}`), are + spelled under an `!adf:` prefix (`!adf:name … !adf:/name`, `!adf:name[content]{attrs}`, + `!adf:name arg {attrs}`) in place of the `:::`/`::`/`:name` forms, and the block carry is a code + fence whose info string `adf:` names the node's type, its body the node's JSON without + `type`: text holding an unescaped `!adf:` and a code fence whose info string opens `adf:` are + claimed, and `adf` is an ordinary code block language. Convert stored markdown per `MIGRATION.md`. +- **Breaking:** `markdownToAdf` reads markdown holding no block as a document whose `content` is + empty, as Atlassian's schema requires; `!adf:doc {content=none}` spells a document holding no + `content` key. +- `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc`: two adjacent text nodes a reader would + join are parted by `!adf:textBreak{}`, an empty `attrs`, `content` or `marks` is spelled + `{attrs=empty}`, `{content=empty}` or `{marks=empty}`, `-0` is spelled `-0`, and a `codeBlock` + of several text nodes is a fence per node. A `codeBlock` holding other than plain text nodes + rides the block carry, where it was refused. - **Breaking:** `unspellable-link` leaves `ConvertErrorCode`; a link whose `href` or `title` no CommonMark escape spells is written as `!adf:link[text]{attrs}`. - **Breaking:** some directive refusals carry `malformed-directive` where they carried diff --git a/MIGRATION.md b/MIGRATION.md index 2dfb73f..edd56de 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -40,12 +40,12 @@ function migrateMarkdown(stored: string) { | `::media {id=a type=file}` | `!adf:media {id=a type=file}` | | `::taskItem TODO {localId=i}`: an empty `caption`, `decisionItem`, `paragraph` or `taskItem`, or an empty `heading` carrying `localId` | `!adf:taskItem TODO {localId=i}` then `!adf:/taskItem` | | `:mention[@Mikael]{id=5b10a2}` | `!adf:mention[@Mikael]{id=5b10a2}` | -| the `adf` code fence and `:adf{json="…"}` | the `carry` code fence and `!adf:carry{json="…"}` | +| the `adf` code fence and `:adf{json="…"}` | the `adf:` code fence, its JSON without `type`, and `!adf:carry{json="…"}` | | `\:` keeps a directive literal | `\!adf:` keeps a directive literal | | `:adf{json="…"}` carrying a link for its `collection`, `id` or `occurrenceKey` | `!adf:link[text]{attrs}` | A colon run and `:name[` are plain text now, and `adf` an ordinary code block language; text -holding an unescaped `!adf:` and a `carry` fence are claimed instead. +holding an unescaped `!adf:` and a code fence whose info string opens `adf:` are claimed instead. ### Readings @@ -54,6 +54,8 @@ Markdown the spelling table leaves alone, which `0.2.0` reads as a different doc | Input | `0.1.0` | `0.2.0` | | --- | --- | --- | | a link whose text already holds one (`[ab](/v)`) | marks every node the inner link does not, splitting the outer link around it | leaves the outer brackets literal text; write the pieces as separate links to keep them | +| markdown holding no block (`markdownToAdf("")`) | `{ type: 'doc', version: 1 }` | `{ content: [], type: 'doc', version: 1 }`; `!adf:doc {content=none}` reads as the former | +| a code fence whose info string opens `adf:` (```` ```adf:x ````) | a `codeBlock` with that language | the block carry; write `!adf:codeBlock {language="adf:x"}` around a bare fence to keep the code block | ### Error codes diff --git a/README.md b/README.md index f164f6b..e69daa6 100644 --- a/README.md +++ b/README.md @@ -175,7 +175,7 @@ Parsing — `markdownToAdf` and `plainMarkdownToAdf`, and `htmlToAdf` at `0.2.0` | Code | Fires when | What you can do | | --- | --- | --- | -| `malformed-directive` | an `!adf:` the grammar cannot read — a prefix completing no directive, an unclosed container, `[content]` or `{attrs}`, a closer with no container of its name open, a leaf given a body, `{attrs}` out of order or duplicated, invalid JSON in a `carry` | write the spelling the message names, or escape the prefix — `\!adf:`, block and inline alike — to keep it literal text | +| `malformed-directive` | an `!adf:` the grammar cannot read — a prefix completing no directive, an unclosed container, `[content]` or `{attrs}`, a closer with no container of its name open, a leaf given a body, `{attrs}` out of order or duplicated, invalid JSON in an opaque carry | write the spelling the message names, or escape the prefix — `\!adf:`, block and inline alike — to keep it literal text | | `malformed-pipe-table` | a pipe row that is no pipe table — a missing or ragged `---` delimiter row, an alignment colon in it, or a row not opening with a pipe | open every row with a pipe and give the delimiter row the header's cell count; to keep the lines literal text instead, escape the leading pipe of every one — escaping a single row leaves the next to open a fresh table and fail the same way | | `unknown-directive-name` | a directive whose name is no node or mark this version spells | check the name in `spec/flavour.md`, or escape the prefix as `\!adf:`; the spelling itself is well formed, so a later minor may give the name meaning | | `unmappable-html` | the input holds an HTML construct the documented element set does not map, a comment and a processing instruction among them — at this version that is every raw HTML construct in markdown, the element set landing at `0.2.0` | remove the construct, or write what it holds in the lossless flavour | @@ -204,7 +204,9 @@ emit refuses: Serves Goals 1, 3 and 4. -- `markdownToAdf(adfToMarkdown(doc))` equals `doc` — unknown node types included, carried opaquely +- `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc` — every key and value as `doc` holds it, + adjacent text nodes, an empty `attrs`, `content` or `marks` and `-0` included, and unknown node + types carried opaquely ([`docs/decisions.md`](https://gitea.larvit.se/larvit/adf-codec/src/branch/main/docs/decisions.md#unknown-nodes-ride-the-carry)). - Markdown this library reads, and markdown it writes, means what the CommonMark spec says; from `0.2.0`, well-formed HTML means what the HTML standard says, read or written. The bullets below @@ -239,7 +241,7 @@ Serves Goals 1, 3 and 4. makes a call loop forever. - The emitted formats are semver surface ([`docs/decisions.md`](https://gitea.larvit.se/larvit/adf-codec/src/branch/main/docs/decisions.md#the-formats-are-api)). -- **`0.2.0`** — `htmlToAdf(adfToHtml(doc))` equals `doc`; fidelity HTML cannot express rides +- **`0.2.0`** — `htmlToAdf(adfToHtml(doc))` deep-equals `doc`; fidelity HTML cannot express rides `data-*` attributes. Foreign HTML maps a documented element set, which markdown's raw HTML reads through as well, and a construct outside it is an error; well-formed HTML only — no tag-soup recovery. diff --git a/corpus/README.md b/corpus/README.md index 097d0af..6ae3974 100644 --- a/corpus/README.md +++ b/corpus/README.md @@ -18,8 +18,8 @@ One directory per contract kind: `reason`. `kind` is `mark-model` (the permanent count divergence from ADF's mark-per-text-node model), `unspellable` (parses but the flavour has no spelling) or `pending` (a parser gap). -JSON is editor-normal (`docs/decisions.md` §Equality is editor-normal), two-space indent, keys -sorted. `spec.json` is the vendored, upstream machine-readable suite, byte-exact from +JSON is two-space indent, keys sorted, and a document read back must deep-equal the fixture's +(`docs/decisions.md` §Equality is deep). `spec.json` is the vendored, upstream machine-readable suite, byte-exact from [spec.commonmark.org](https://spec.commonmark.org/0.31.2/spec.json) (CommonMark 0.31.2, © John MacFarlane, [CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)), and is not re-serialized by the corpus gate. diff --git a/docs/decisions.md b/docs/decisions.md index b7f65ed..df52047 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -14,7 +14,7 @@ a backslash reach them intact; what the flavour cannot spell reduces ADF→ADF a 2026-08-23, real payloads 2026-09-15, the maintainer. Goal 1. Valid while a consumer saves back through the lossless pair. -`markdownToAdf(adfToMarkdown(doc))` and `htmlToAdf(adfToHtml(doc))` must equal `doc` — anything +`markdownToAdf(adfToMarkdown(doc))` and `htmlToAdf(adfToHtml(doc))` must deep-equal `doc` — anything less silently destroys content an editor could not represent, in a document it did not author. When losslessness and readability conflict, losslessness wins. Round-trip equality is a property tested over a checked-in corpus (`corpus/README.md`), not a claim made in prose. Its real payloads @@ -31,27 +31,100 @@ where there is a way back. CommonMark spells some things the flavour has no esca paragraph opening with a code span whose backticks read back as a fence — so a parse succeeding does not imply a spellable document; `corpus/commonmark-spec/exceptions.json` names those. -## Equality is editor-normal +## Equality is deep -2026-08-24, the maintainer. Goal 1. Valid while markdown cannot tell apart the ADF shapes this -merges. +2026-08-24, deep 2026-10-03, the maintainer. Goal 1. Valid while a pipeline or a bot can build a +shape the editor would not. -"Equals" is structural equality over editor-normal ADF — adjacent text nodes with identical marks -and no attributes merged, JSON number semantics, an empty attrs object, marks array or content -array the absent key — the only domain markdown can restore. -`todo.md` item 40 replaces this with deep equality (2026-09-28, the maintainer). +"Equals" is `assert.deepStrictEqual`: every key and value `doc` holds, two adjacent text nodes, an +empty `attrs`, `content` or `marks` and `-0` included, with no normalization on either side. +CommonMark's spelling stays wherever a document holds none of those shapes. The plain flavour is +lossy and stays editor-normal: its reduction reads and writes ADF with adjacent text nodes of +identical marks and no attributes merged, `-0` read as `0`, and an empty `attrs`, `content` or +`marks` the absent key. + +## `!adf:textBreak{}` parts text CommonMark would join + +2026-10-03, the maintainer. Goal 1. Valid while CommonMark reads adjacent text as one run. + +Two adjacent text nodes a reader would build back as one — neither carried, neither holding +`attrs` or an empty key, their marks identical, attributes included — are parted by the reserved +inline leaf `!adf:textBreak{}`, mirroring `!adf:listBreak`: it builds no node, and anywhere but +between two such nodes, or given `[content]` or `{attrs}`, it is `unsupported-node-shape`. It sits +inside every mark spelling the pair shares, `**Hello, !adf:textBreak{}world**`; a code span holds +no directive, so it closes and reopens, `` `a`!adf:textBreak{}`b` ``. A carried node never joins +its neighbour, so a carry needs no break. A mark run breaks on any difference in the mark, `attrs: +{}` against no `attrs` included. + +## An empty key spells `empty` + +2026-10-03, a writer panel and the maintainer. Goals 1 and 5. Valid while no attribute value +spells an empty object or array. + +An `attrs`, `content` or `marks` key holding an empty object or array is the reserved key with the +bare value `empty` on every directive — block, inline node and mark: `{attrs=empty}`, +`{content=empty}`, `{marks=empty}`. A writer panel chose the spelling, 5 of 7. A container +spelling `content=empty` closes with no body, so an empty pair stays the node holding no content +key; a leaf spells it too, `!adf:rule {content=empty}`. A node CommonMark spells takes its +directive form to hold an empty key, and a text node holding one rides the inline carry, as does a +node under an `em`, `strong`, `strike`, `code` or `link` mark whose `attrs` is empty, since none of +those spellings holds attributes. Any other value of a reserved key is `unsupported-node-shape`, +except on a block's `marks`, which reads it as the marks array in JSON. + +## `-0` is spelled `-0` + +2026-10-03, the maintainer. Goal 1. Valid while JSON's own serialization writes `-0` as `0`. + +Wherever the flavour writes a number or a JSON value — an attribute, a `json` value, the carry — +`-0` is `-0`, which JSON's grammar reads back as `-0`. An `orderedList` whose `order` is `-0` takes +the directive form, since no list marker spells the sign. + +## Empty markdown is a document of no blocks + +2026-10-03, a panel and the maintainer. Goals 1 and 5. Valid while ADF's schema requires +`content` on `doc`. + +Markdown holding no block reads as `{ content: [], type: 'doc', version: 1 }`, the document +`spec/adf-schema/full.json` requires. A document holding no `content` key is +`!adf:doc {content=none}` standing alone as its only block, and a named error anywhere else. The +panel split 4 for `none` and 3 for `absent`, and the maintainer chose `none`; all seven rejected a +bare `!adf:doc`. ## Unknown nodes ride the carry -2026-08-23, extended to misplaced known nodes 2026-08-26, the maintainer. Goal 1. Valid while ADF -holds nodes, or node positions, this library does not spell. +2026-08-23, extended to misplaced known nodes 2026-08-26 and to code block children 2026-10-03, the +maintainer. Goal 1. Valid while ADF holds nodes, or node positions, this library does not spell. An unknown ADF node is carried opaquely — raw JSON rides a dedicated syntax in both formats and restores to a deep-equal node. The round-trip holds for documents newer than the library. So does a known node no section spells where it stands: a markdown serializer spells a node by type without -checking its position, and refusing loses a document ADF itself keeps in an `unsupportedBlock`. -Where a container's own spelling cannot hold the child it has — a `bulletList` holding other than -`listItem`, a `codeBlock` other than text — the error result names that instead. +checking its position, and refusing loses a document ADF itself keeps in an `unsupportedBlock`. A +`codeBlock` holding a child no fence holds — anything but a text node carrying no marks, `attrs` or +`content` — rides the carry whole. + +## The carry fence names the node type + +2026-10-03, the maintainer. Goals 1 and 5. Valid while a code fence's info string reads back +verbatim. + +The block carry is a code fence whose info string `adf:` names the node's type, its body the +node's JSON without `type`: ```` ```adf:blockCard ````. A type no info string carries back — by the +rule a code language follows — leaves the info string `adf:` and keeps `type` in the body. Every +info string opening `adf:` is reserved, so a `codeBlock` whose language opens so takes the +`language` attribute, and `carry` is an ordinary language. A body holding `type` under a named +type, or an `adf:` fence whose type an info string carries, is `unsupported-node-shape`. + +## A code block is a fence per text node + +2026-10-03, the maintainer. Goal 1. Valid while ADF holds a code block's text in more than one +node. + +A `codeBlock` holding several text nodes is the `!adf:codeBlock` container holding one fence per +node, each fence's info string the language; its other attributes sit on the opener, and a +language no info string carries stays the opener's `language` with bare fences. A `codeBlock` +spelling `content=empty` has no fence to carry the language, so the opener does. Fences with +differing info strings are `unsupported-node-shape` — ADF holds one language — and so is an empty +fence beside another, since a text node holds text. ## Foreign HTML sorts three ways @@ -295,10 +368,11 @@ handles one cause alike whichever node, attribute or direction raised it. - A directive whose name reads back to no node is `unknown-directive-name` rather than a claim code — the spelling is well formed, and telling that apart from a typo is what a consumer switches on when a later MINOR gives the name meaning. A reserved name is a known name, so never - that code, and the two the flavour reserves part on form: a form the grammar does not have is a - claim code — `!adf:carry`, whose carry is the fence — and a well-formed form in the wrong place + that code, and the names the flavour reserves part on form: a form the grammar does not have is + a claim code — `!adf:carry`, whose carry is the fence — and a well-formed form in the wrong place is `unsupported-node-shape`, `!adf:listBreak` parting anything but two adjacent lists of one - type (2026-09-01). + type, `!adf:textBreak{}` anything but two text nodes a reader joins, `!adf:doc` standing beside + another block (2026-09-01, the text break and `doc` 2026-10-03). - A well-formed directive the node tables refuse — an attribute a node does not hold or spells elsewhere, a value outside its kind or its canonical spelling, an argument, or a body of a shape its content model does not take — is `unsupported-node-shape`, the emitter's code for the same diff --git a/spec/flavour.md b/spec/flavour.md index fcb97ba..d61d2f0 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -44,7 +44,8 @@ normalizes to it through the round-trip. CommonMark admits no spelling — the end of a block, inside an ATX heading — or where the node carries an attribute, it is the inline directive. - An empty paragraph — real payloads carry them — is an `!adf:paragraph` … `!adf:/paragraph` pair - holding nothing. + holding nothing, and one whose `content` is an empty array the pair + `!adf:paragraph {content=empty}` … `!adf:/paragraph` (Attributes). - Links `[text](url)`; `<…>` around a destination containing spaces, `<>` an empty one beside a title; title in double quotes. A backslash escapes a parenthesis the destination leaves unbalanced, and a quote inside the title; a balanced pair stays bare. `` autolink form only @@ -67,7 +68,10 @@ normalizes to it through the round-trip. matching below, which is what lets the emitter decide its own pairings. - Blocks separated by one blank line at document level, inside a blockquote and between CommonMark blocks; inside a directive container a pair holding a directive block takes none. No trailing - whitespace outside a code block's content, single trailing newline; a document with no blocks is the empty string. + whitespace outside a code block's content, single trailing newline. A document whose `content` is + an empty array is the empty string, and markdown holding no block reads back to it; a document + holding no `content` key is the leaf `!adf:doc {content=none}` as its only block, which is a + named error anywhere else or spelled any other way. ## Directives @@ -79,9 +83,11 @@ result naming it at the opener, whatever follows it — so output an old emitter escaped, and erroring input gaining meaning later is MINOR, never a reparse (`docs/decisions.md` §The formats are API). Each name belongs to one position, and a name the other one spells — a mark or an inline node written as a block directive, a block node written inline — is a different error, -naming the spelling it takes. Two reserved names read back to no node: `carry` for the opaque carry, -as both directive name and fence info string, and `listBreak` for the leaf that parts two adjacent -lists (Canonical form). +naming the spelling it takes. Four reserved names read back to no node: `carry` for the inline +opaque carry, `listBreak` for the leaf that parts two adjacent lists and `doc` for a document +holding no `content` key (Canonical form), and `textBreak` for the leaf that parts two text nodes +(Inline nodes). Every fence info string opening `adf:` is reserved for the block carry (The opaque +carry). **Claiming**: an unescaped `!adf:` claims wherever it stands. What follows picks the form: `/name` closes a container, and a name picks by what follows it in turn — a space or the line's end a block @@ -142,6 +148,13 @@ ends the name (`!adf:hardBreak{}`). Input reads that spelling alone: keys out of quoted where bare carries it, an escape longer than it need be, an empty `{attrs}` on a block line or after a `[content]`, and a number or `json` value outside its canonical JSON spelling are each a named error naming the spelling to write instead. +`attrs`, `content` and `marks` are reserved keys on every directive — block, inline node and mark — +whose bare value `empty` spells the node's or mark's key holding an empty object or array: +`!adf:underline[a]{attrs=empty}`, `!adf:date{content=empty}`, `!adf:hardBreak{marks=empty}`. A +container spelling `content=empty` closes with no body; `attrs=empty` stands beside no other +attribute, argument or content slot; and an inline node spelling `marks=empty` stands inside no +mark spelling. Any other value of a reserved key is a named error, except on a block's `marks` +(Block nodes). **Escaping**: the emitter backslash-escapes whatever literal text would otherwise parse as directive syntax — every literal `!adf:`, `]` inside content, a bracket a link's destination and @@ -163,15 +176,18 @@ and restores to a deep-equal node. A carry may hold a node the emitter spells na unreinterpreted, and the next emit spells it canonically (`docs/decisions.md` §The round-trip is the product). Block and inline positions canonicalize differently, each fitting where it sits: -- **Block position**: a fenced code block with info string `carry`, body = the node's JSON — - two-space indent, object keys sorted. +- **Block position**: a fenced code block with info string `adf:` and the node's type, body = the + node's JSON without its `type` — two-space indent, object keys sorted: ```` ```adf:blockCard ````. + A type no info string carries back, by the rule a `codeBlock`'s language follows, leaves the info + string `adf:` and keeps `type` in the body. A body holding `type` under a named type, or an + `adf:` fence whose type an info string carries, is a named error. - **Inline position**: `!adf:carry{json="…"}` — compact serialization (keys sorted, no whitespace), JSON-string-escaped into the attribute. -The info string `carry` is reserved: a genuine `codeBlock` whose `language` is exactly `carry` takes +Every info string opening `adf:` is reserved: a genuine `codeBlock` whose `language` opens so takes the attribute the section below keeps for a language no info string holds, so the reservation -stays absolute. -In block-directive position `!adf:carry` is a named error — the carry's block form is the fence. +stays absolute. In block-directive position `!adf:carry` is a named error — the carry's block form +is the fence. ## Raw HTML in input @@ -193,13 +209,14 @@ Each section lists attributes as `name (type)`. A parenthesized value set docume payloads hold; the type stays string and any value round-trips verbatim. Values map to attrs by type: strings verbatim, numbers and booleans in canonical JSON spelling — quoted where not bare (`width="33.33"`) — and `json` values as the inline carry's serialization (compact, keys sorted), -quoted. `markdownToAdf` emits `attrs`, `content` and `marks` keys only when non-empty; editor-normal -ADF reads an empty attrs object, marks array or content array as the absent key (`docs/decisions.md` -§Equality is editor-normal) — the grammar's empty-`{attrs}` omission already collapses the two -spellings. +quoted, `-0` spelled `-0`. `markdownToAdf` builds an `attrs`, `content` or `marks` key only where the +markdown spells one, an empty one through its reserved key (Attributes), so a document reads back +deep-equal (`docs/decisions.md` §Equality is deep). A node CommonMark spells takes the directive +form to hold an empty key. Marks on a block node ride the reserved attribute key `marks` — the node's marks array as a -`json` value: `!adf:layoutSection {marks="[{\"attrs\":{\"mode\":\"wide\"},\"type\":\"breakout\"}]"}`. +`json` value, `marks=empty` where it is empty: +`!adf:layoutSection {marks="[{\"attrs\":{\"mode\":\"wide\"},\"type\":\"breakout\"}]"}`. A section saying its body is inline takes at most one paragraph, whose inline content becomes the node's `content`; any other body is a named error, and an empty pair is a node holding none. @@ -215,14 +232,17 @@ cannot — `localId` (string) on any of them, marks, and the values below — ta form. - `blockquote`, `bulletList`, `listItem` — containers, block body. Attributes: `localId` (string). -- `codeBlock` — container, body one fenced code block whose info string is the language and whose - content is the node's. Attributes: `hideLineNumbers` (boolean), `language` (string), `localId` - (string), `uniqueId` (string), `wrap` (boolean). A language no info string carries back — empty, - the reserved `carry`, or holding a backtick, a backslash, a control character, edge whitespace or - an entity reference — rides the `language` attribute instead and the fence carries no info - string; writing it in the slot that rule leaves empty, or in both, is a named error. The body is - one ordinary code block, and a fence's info string decodes escapes and entity references as any - other does. +- `codeBlock` — container, body one fenced code block per text node, each fence's info string the + language and its content the node's text; a node holding no `content` key is one empty fence. + Attributes: `hideLineNumbers` (boolean), `language` (string), `localId` (string), `uniqueId` + (string), `wrap` (boolean). A language no info string carries back — empty, opening the reserved + `adf:`, or holding a backtick, a backslash, a control character, edge whitespace or an entity + reference — rides the `language` attribute instead and the fences carry no info string, and so + does a language beside `content=empty`, which has no fence; writing it in the slot that rule + leaves empty, or in both, is a named error, and so are fences whose info strings differ and an + empty fence beside another. Each fence is an ordinary code block, and its info string decodes + escapes and entity references as any other does. A node holding a child no fence holds — any but + a text node carrying no marks, `attrs` or `content` — rides the block carry. - `heading` — container, inline body. Attributes: `level` (number), `localId` (string). `level` is the `#` count, so a heading carrying none, or one that is no whole number from 1 to 6, has no CommonMark spelling. @@ -424,9 +444,8 @@ Right. Attributes and the carry fallback read as in the block sections, the carry in its inline form. Of the nodes below, `emoji`, `mention` and `status` spell their `text` attribute in the content slot as plain text: `[]` is the empty string, absent content is the absent attribute, non-empty content -parsing to anything but one text node carrying neither marks, attributes nor content — adjacent text -nodes with identical marks and no attributes merged first — is a named error, and so is a `text` key -in `{attrs}`. An enclosing mark spelling does not reach into the slot. The rest take no content, +parsing to anything but one text node carrying neither marks, attributes nor content is a named +error, and so is a `text` key in `{attrs}`. An enclosing mark spelling does not reach into the slot. The rest take no content, `!adf:text` included; content on a node that takes none is a named error. - `date` — Attributes: `localId` (string), `timestamp` (string, epoch milliseconds). @@ -453,16 +472,27 @@ Shipped !adf:emoji[🎉]{shortName=":tada:"} on !adf:date{timestamp=175608000000 CommonMark strips or refuses one — a block's inline content edges, either side of a line break, an em, strong or strike spelling's inner edges, a pipe cell's edges — is spelled `!adf:text{text="…"}`, the reserved key carrying the node's text, escaped by the attribute grammar and never literal: pipe -cells trim and pad. The emitter wraps the whitespace run alone and leaves the rest plain text; -`markdownToAdf` merges adjacent text nodes carrying identical marks and no attributes -(`docs/decisions.md` §Equality is editor-normal). Input reads that spelling alone: the value is one -run of spaces and tabs, or one run of newlines, and anything else — a mixed run, or text CommonMark -carries plainly — is a named error. +cells trim and pad. The emitter wraps the whitespace run alone and leaves the rest plain text, which +the spelled run joins on reading. Input reads that spelling alone: the value is one run of spaces +and tabs, or one run of newlines, and anything else — a mixed run, or text CommonMark carries +plainly — is a named error. ``` !adf:text{text=" "}Two leading spaces held, and one text node split!adf:text{text="\n"}over two lines. ``` +**Adjacent text nodes.** CommonMark reads two adjacent text nodes back as one where neither is +carried, neither holds `attrs` or an empty key, and their marks are identical, attributes included. +The reserved leaf `!adf:textBreak{}` parts such a pair, inside every mark spelling the two share; a +code span holds no directive, so it closes and reopens. It builds no node and reads only between +two such nodes: elsewhere, or with `[content]` or `{attrs}`, it is a named error (`docs/decisions.md` +§`!adf:textBreak{}` parts text CommonMark would join). A text node holding `attrs` or an empty key +rides the inline carry. + +``` +Hello, !adf:textBreak{}world — **Hello, !adf:textBreak{}world** — `a`!adf:textBreak{}`b` +``` + ## Marks An inline node's marks ride the spelling wrapped around them, never the block sections' reserved @@ -490,14 +520,14 @@ the directive form, open to no literal reading, is a named error. A spelling adds its mark to every inline node it wraps, and nesting is the marks array in order, outermost first: `_!adf:underline[x]_` gives marks `[em, underline]`, `!adf:underline[_x_]` the reverse. `adfToMarkdown` nests in the order the array holds rather than sorting it — -`docs/decisions.md` §Equality is editor-normal restores the array, not a set — and opens each -spelling once over the longest run of adjacent inline nodes carrying an identical mark, attributes -included, at that depth. A run breaks at every node the emitter carries, so no emitted carry sits -inside a mark spelling. +`docs/decisions.md` §Equality is deep restores the array, not a set — and opens each spelling once +over the longest run of adjacent inline nodes carrying an identical mark, attributes included, at +that depth: `attrs: {}` differs from no `attrs`, and a directive spells it `{attrs=empty}`. A run +breaks at every node the emitter carries, so no emitted carry sits inside a mark spelling. An inline node whose marks no nesting spells — a mark type not listed here, an attrs key its spelling does not list, a value that is not the spelling's type, an attribute the spelling needs -and the mark lacks, an order putting a code span outside another mark, `code` over anything but a +and the mark lacks, an empty `attrs` on a mark CommonMark spells, an order putting a code span outside another mark, `code` over anything but a text node or over text holding a newline, or a spelling CommonMark's flanking rules cannot open or close where the run sits (`un**-real**istic`), or one CommonMark's matching pairs elsewhere — the intra-word `*` runs together with a neighbouring `**`, and the multiple-of-3 rule can leave the diff --git a/src/markdown/emit/plain-reduction.ts b/src/markdown/emit/plain-reduction.ts index 593c4fe..30d82bb 100644 --- a/src/markdown/emit/plain-reduction.ts +++ b/src/markdown/emit/plain-reduction.ts @@ -51,7 +51,7 @@ export function reduceToPlain(document: AdfDocument): Result { const fault = adfDocumentFault(document) if (fault !== undefined) return faulted(fault, []) if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) - // The plain flavour is lossy: it reads and writes editor-normal ADF, whose shapes CommonMark spells. + // docs/decisions.md §Equality is deep: the plain flavour reads and writes editor-normal ADF. const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] }) return blocks.ok ? success({ content: nodeContent(toEditorNormal({ content: blocks.value, type: 'doc', version: 1 })).slice(), type: 'doc', version: 1 }) : blocks } diff --git a/todo.md b/todo.md index 9fd26b3..cab4975 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 52` +`Next ID = 54` | Goal | W | |---|---| @@ -24,13 +24,14 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| -| 40 | 0.2.0 | decision | **Make `markdownToAdf(adfToMarkdown(doc))` deep-equal `doc` for every document `adfToMarkdown` takes.** | 6 | 7 | 8 | 9 | 1 | 26.2 | | 7 | 0.2.0 | | **Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF.** | 6 | 9 | 9 | 9 | 2, 3 | 25.8 | | 45 | 0.2.0 | | **Replace `isAdfDocument` with a reader returning `Result`.** | 2 | 3 | 6 | 8 | 1 | 25.2 | | 6 | 0.2.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 3 | 24.7 | | 43 | 0.2.0 | | **Give each markdown input its own reader, strict to its own standard.** | 6 | 7 | 8 | 9 | 3, 4 | 22.3 | | 49 | 0.2.0 | | **Read a list whose bullet or ordered delimiter changes as two lists in the CommonMark reader.** | 4 | 5 | 5 | 8 | 3, 4 | 17.2 | +| 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 5 | 13.0 | | 51 | 0.2.0 | | **Match a reference label to its definition under Unicode case folding.** | 2 | 2 | 2 | 7 | 3, 4 | 12.4 | +| 53 | 0.2.0 | | **Bench a block's `marks` spelling with the writer panel and adopt its pick.** | 4 | 5 | 5 | 6 | 5 | 11.5 | | 50 | 0.2.0 | | **Read `[](/url)` and `[]()` as CommonMark's empty link.** | 4 | 4 | 3 | 6 | 3, 4, 6 | 10.4 | | 38 | 0.3.0 | | **Spell a lone surrogate in a text node so it survives a UTF-8 encode.** | 2 | 2 | 4 | 7 | 1 | 19.5 | | 47 | 0.3.0 | | **Open the README with what the package is, what it does and for whom.** | 1 | 4 | 7 | 9 | 9 | 14.0 | @@ -45,14 +46,6 @@ ## Details -### 40. Make `markdownToAdf(adfToMarkdown(doc))` deep-equal `doc` for every document `adfToMarkdown` takes. - -Today it holds for editor-normal documents only: two adjacent text nodes with the same marks merge, -an empty `attrs`, `marks` or `content` drops, and `-0` reads back `0` — shapes pipelines and bots -build. Spell each so it reads back as written; CommonMark's spelling stays wherever the document -holds none of these shapes. The spellings are part of the chunk. `docs/decisions.md` §Equality is -editor-normal, `spec/flavour.md` and `corpus/README.md` follow, and the tests drop `toEditorNormal`. - ### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF. Lands after item 6. The CommonMark spec suite also runs against `markdownToHtml`. The README @@ -90,6 +83,11 @@ Readings table gains its row, and its Spellings table one if `!adf:listBreak` re and 302 lose their `pending` exceptions, and the spelling leaves the README's "Four CommonMark spellings" bullet, which counts one fewer. +### 52. Spell `colwidth` as a comma list, `colwidth="340,420"`. + +A writer panel chose it on 2026-10-03, 5 of 7, over today's `colwidth="[340,420]"`. Breaking: +`MIGRATION.md`'s Spellings table gains its row. + ### 51. Match a reference label to its definition under Unicode case folding. `link-syntax.ts` normalizes a label with `toLowerCase`, so `[ẞ]` misses its `[SS]` definition (spec @@ -97,6 +95,12 @@ example 540); lowercasing and then uppercasing folds it. Breaking, so it ships b `MIGRATION.md`'s Readings table gains its row. Its `pending` exceptions go, and its spelling leaves the README's "Four CommonMark spellings" bullet, which counts one fewer. +### 53. Bench a block's `marks` spelling with the writer panel and adopt its pick. + +Today `marks="[{\"attrs\":{\"mode\":\"wide\"},\"type\":\"breakout\"}]"`, the marks array as +escaped JSON. Breaking where the panel picks another spelling: `MIGRATION.md`'s Spellings table +gains its row. + ### 50. Read `[](/url)` and `[]()` as CommonMark's empty link. Both stay literal text today (spec examples 484 and 487). ADF holds no empty text node to carry a -- 2.52.0 From 1d0ae695da4a9d292e910ec78f76dd3ea26a030c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:31:10 +0200 Subject: [PATCH 04/67] Name the code block the carry now holds in the migration's error codes --- MIGRATION.md | 1 + 1 file changed, 1 insertion(+) diff --git a/MIGRATION.md b/MIGRATION.md index edd56de..6ab5f2e 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -65,6 +65,7 @@ it named converts. | Input | `0.1.0` | `0.2.0` | | --- | --- | --- | | a link whose `href` or `title` no CommonMark escape spells, on emit | `unspellable-link` | spells `!adf:link[text]{attrs}` | +| a `codeBlock` holding other than plain text nodes, on emit | `unsupported-node-shape` | rides the block carry | | a leaf node given a body (`media`, `listBreak`) | `unsupported-node-shape` | `malformed-directive` | | a node with a block body written as a leaf (`panel`) | `unsupported-node-shape` | `malformed-directive` | | an empty node the `::taskItem` spelling row names, written as a leaf | parses | `malformed-directive` | -- 2.52.0 From 0c507f981fae1ba9296908ccbbd2d676a5e02232 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 12:31:49 +0200 Subject: [PATCH 05/67] Name the writer panel behind the doc spelling --- docs/decisions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/decisions.md b/docs/decisions.md index df52047..f8b1581 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -87,7 +87,7 @@ the directive form, since no list marker spells the sign. Markdown holding no block reads as `{ content: [], type: 'doc', version: 1 }`, the document `spec/adf-schema/full.json` requires. A document holding no `content` key is `!adf:doc {content=none}` standing alone as its only block, and a named error anywhere else. The -panel split 4 for `none` and 3 for `absent`, and the maintainer chose `none`; all seven rejected a +writer panel split 4 for `none` and 3 for `absent`, and the maintainer chose `none`; all seven rejected a bare `!adf:doc`. ## Unknown nodes ride the carry -- 2.52.0 From fdbf43398c2243ea74c24532a28ac02e73d4f3d7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:17:22 +0200 Subject: [PATCH 06/67] Keep one merge loop and one mark key: editor-normal compares normalized marks with identicalMark --- src/adf/document.ts | 18 ++++++++++++-- src/adf/editor-normal.ts | 35 ++++++---------------------- src/conformance/property-harness.ts | 5 ++-- src/markdown/emit/plain-inline.ts | 6 ++--- src/markdown/parse/inline-content.ts | 19 +++------------ src/markdown/text-break.ts | 4 ++-- 6 files changed, 34 insertions(+), 53 deletions(-) diff --git a/src/adf/document.ts b/src/adf/document.ts index 370876c..752e517 100644 --- a/src/adf/document.ts +++ b/src/adf/document.ts @@ -68,8 +68,22 @@ export function identicalMark(left: AdfMark, right: AdfMark): boolean { return marksKey([left]) === marksKey([right]) } -export function identicalMarks(left: AdfNode, right: AdfNode): boolean { - return marksKey(nodeMarks(left)) === marksKey(nodeMarks(right)) +export function identicalMarks(left: readonly AdfMark[], right: readonly AdfMark[]): boolean { + return marksKey(left) === marksKey(right) +} + +// `joins` is the caller's: a reader joins what CommonMark reads as one, editor-normal ADF what the editor would. +export function mergeAdjacentText(nodes: readonly AdfNode[], joins: (previous: AdfNode, node: AdfNode) => boolean): AdfNode[] { + const merged: AdfNode[] = [] + for (const node of nodes) { + const previous = merged[merged.length - 1] + if (previous !== undefined && joins(previous, node)) { + merged[merged.length - 1] = { ...previous, text: `${previous.text ?? ''}${node.text ?? ''}` } + continue + } + merged.push(node) + } + return merged } // Depth is the walks' business, not the shape's: the guard waves a deep document through as blocks and marks do. diff --git a/src/adf/editor-normal.ts b/src/adf/editor-normal.ts index cd37b1f..a89c0df 100644 --- a/src/adf/editor-normal.ts +++ b/src/adf/editor-normal.ts @@ -1,34 +1,25 @@ import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from './document.ts' import type { JsonValue } from '../json-value.ts' -import { nodeAttrs, nodeContent, nodeMarks } from './document.ts' -import { serializeCanonicalJson } from '../canonical-json.ts' +import { identicalMark, identicalMarks, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from './document.ts' type JsonContainer = JsonValue[] | { [key: string]: JsonValue } type NodeHolder = { content?: AdfNode[] } -export function sameMark(candidate: AdfMark, mark: AdfMark): boolean { - return markKey(candidate) === markKey(mark) +export function sameMark(left: AdfMark, right: AdfMark): boolean { + return identicalMark(normalMark(left), normalMark(right)) } -export function mergeAdjacentText(nodes: readonly AdfNode[]): AdfNode[] { - const merged: AdfNode[] = [] - for (const node of nodes) { - const previous = merged[merged.length - 1] - if (previous !== undefined && mergesText(previous) && mergesText(node) && sameMarks(previous, node)) { - merged[merged.length - 1] = { ...previous, text: `${previous.text ?? ''}${node.text ?? ''}` } - continue - } - merged.push(node) - } - return merged +export function joinsNormally(previous: AdfNode, node: AdfNode): boolean { + if (!mergesText(previous) || !mergesText(node)) return false + return identicalMarks(nodeMarks(previous).map(normalMark), nodeMarks(node).map(normalMark)) } export function toEditorNormal(document: AdfDocument): AdfDocument { const normal: AdfDocument = { type: document.type, version: Object.is(document.version, -0) ? 0 : document.version } const pending: { holder: NodeHolder; source: NodeHolder }[] = [{ holder: normal, source: document }] for (let entry = pending.pop(); entry !== undefined; entry = pending.pop()) { - const content = mergeAdjacentText(nodeContent(entry.source)) + const content = mergeAdjacentText(nodeContent(entry.source), joinsNormally) if (content.length === 0) continue entry.holder.content = content.map((source) => { const holder = normalNode(source) @@ -76,15 +67,3 @@ function normalValue(value: JsonValue, pending: JsonContainer[]): JsonValue { function mergesText(node: AdfNode): boolean { return node.type === 'text' && Object.keys(nodeAttrs(node)).length === 0 } - -function sameMarks(previous: AdfNode, node: AdfNode): boolean { - return marksKey(nodeMarks(previous)) === marksKey(nodeMarks(node)) -} - -function marksKey(marks: readonly AdfMark[]): string { - return marks.map(markKey).join('\n') -} - -function markKey(mark: AdfMark): string { - return `${mark.type} ${serializeCanonicalJson(normalAttributes(nodeAttrs(mark)) ?? {}, 'compact')}` -} diff --git a/src/conformance/property-harness.ts b/src/conformance/property-harness.ts index cd935f8..6afc116 100644 --- a/src/conformance/property-harness.ts +++ b/src/conformance/property-harness.ts @@ -10,8 +10,9 @@ import { blockArgument } from '../markdown/block-directive.ts' import { blockNodes } from '../adf/block-nodes.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' import { inlineNodes } from '../adf/inline-nodes.ts' +import { joinsNormally } from '../adf/editor-normal.ts' import { markAttributes } from '../adf/mark-attributes.ts' -import { mergeAdjacentText } from '../adf/editor-normal.ts' +import { mergeAdjacentText } from '../adf/document.ts' type Positions = { block: AdfNode; inline: AdfNode } @@ -92,7 +93,7 @@ function withoutEmptyKeys(held: T): T { } function occasionallyApart(arbitrary: Arbitrary): Arbitrary { - return fc.tuple(arbitrary, fc.nat({ max: 5 })).map(([nodes, roll]) => (roll === 0 ? nodes : mergeAdjacentText(nodes))) + return fc.tuple(arbitrary, fc.nat({ max: 5 })).map(([nodes, roll]) => (roll === 0 ? nodes : mergeAdjacentText(nodes, joinsNormally))) } function pipeTable({ body, header }: { body: AdfNode[][]; header: AdfNode[] }): AdfNode { diff --git a/src/markdown/emit/plain-inline.ts b/src/markdown/emit/plain-inline.ts index 7da6adc..f8e89b5 100644 --- a/src/markdown/emit/plain-inline.ts +++ b/src/markdown/emit/plain-inline.ts @@ -3,9 +3,9 @@ import type { LineContainer } from '../line-container.ts' import type { MarkRun } from './line-escaping.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' +import { joinsNormally, sameMark } from '../../adf/editor-normal.ts' import { largestNesting } from '../../nesting.ts' -import { mergeAdjacentText, sameMark } from '../../adf/editor-normal.ts' -import { nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' +import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { plainLineFallback, type PlainLineFallback } from './inline-line.ts' import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts' @@ -193,7 +193,7 @@ function withMarks(leaf: AdfNode, marks: readonly AdfMark[]): AdfNode { function trimmedEdges(leaves: readonly AdfNode[]): AdfNode[] { for (let current = leaves; ; ) { - const merged = withoutEdgeBreaks(mergeAdjacentText(current)) + const merged = withoutEdgeBreaks(mergeAdjacentText(current, joinsNormally)) let changed = false const trimmed: AdfNode[] = [] for (const [index, leaf] of merged.entries()) { diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index 7aeabe8..132d1ee 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -10,7 +10,7 @@ import { commonMarkLink, linkHref } from '../mark-spellings.ts' import { delimiterFlags, matchEmphasis, runLength } from '../commonmark/emphasis-matching.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { highlightDelimiter, highlightFlanking } from '../plain-conventions.ts' -import { identicalMarks, nodeAttrs, nodeMarks } from '../../adf/document.ts' +import { identicalMarks, mergeAdjacentText, nodeAttrs, nodeMarks } from '../../adf/document.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { noSpans, readInlineDirective } from '../directive-syntax.ts' import { normalizeLabel, readInlineTarget, readLabel } from '../commonmark/link-syntax.ts' @@ -475,20 +475,7 @@ function resolveNodes(pieces: readonly Piece[], scan: Scan, highlights: boolean) writeUnpaired(nodes, runs, pairings) if (!markPairings(pieces, nodes, pairings)) return failure('unsupported-node-shape', carriedInMark, scan.path) markHighlights(pieces, nodes, highlights ? pairedHighlights(pieces, nodes) : []) - return success(mergeText(nodes.flat(), scan.carried)) -} - -function mergeText(nodes: readonly AdfNode[], carried: ReadonlySet): AdfNode[] { - const merged: AdfNode[] = [] - for (const node of nodes) { - const previous = merged[merged.length - 1] - if (previous !== undefined && !carried.has(previous) && !carried.has(node) && readsAsOne(previous, node)) { - merged[merged.length - 1] = { ...previous, text: `${previous.text ?? ''}${node.text ?? ''}` } - continue - } - merged.push(node) - } - return merged + return success(mergeAdjacentText(nodes.flat(), (previous, node) => !scan.carried.has(previous) && !scan.carried.has(node) && readsAsOne(previous, node))) } // Only `imageAlt` reaches the image arm: everywhere else an image amid other content is refused first. @@ -579,7 +566,7 @@ function pairedHighlights(pieces: readonly Piece[], nodes: readonly AdfNode[][]) let candidate = found[closer] while (candidate !== undefined && (!candidate.closes || candidate.position < earliest)) candidate = found[(closer += 1)] if (candidate === undefined) break - if (candidate.line !== opener.line || !identicalMarks(opener.holder, candidate.holder)) continue + if (candidate.line !== opener.line || !identicalMarks(nodeMarks(opener.holder), nodeMarks(candidate.holder))) continue paired.push({ closer: candidate.index, opener: opener.index }) resume = candidate.position + highlightDelimiter.length } diff --git a/src/markdown/text-break.ts b/src/markdown/text-break.ts index de1174c..994ae19 100644 --- a/src/markdown/text-break.ts +++ b/src/markdown/text-break.ts @@ -1,5 +1,5 @@ import type { AdfNode } from '../adf/document.ts' -import { identicalMarks } from '../adf/document.ts' +import { identicalMarks, nodeMarks } from '../adf/document.ts' import { spellInlineLeafDirective } from './directive-syntax.ts' export const textBreakName = 'textBreak' @@ -8,7 +8,7 @@ export const textBreakSpelling = spellInlineLeafDirective(textBreakName, '') // Whether CommonMark reads the pair back as one text node, where neither rides the carry (spec/flavour.md, Inline nodes). export function readsAsOne(previous: AdfNode, node: AdfNode): boolean { - return spelledAsText(previous) && spelledAsText(node) && identicalMarks(previous, node) + return spelledAsText(previous) && spelledAsText(node) && identicalMarks(nodeMarks(previous), nodeMarks(node)) } function spelledAsText(node: AdfNode): boolean { -- 2.52.0 From 70e2639982e4a72375b573624b95afcb02b2fb1a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:19:55 +0200 Subject: [PATCH 07/67] Carry the text break as its own item, not a fake node, and check it where the outermost content resolves --- src/markdown/parse/inline-content.ts | 143 ++++++++++++++------- src/markdown/parse/markdown-to-adf.test.ts | 14 ++ 2 files changed, 108 insertions(+), 49 deletions(-) diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index 132d1ee..978b7d9 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -23,6 +23,13 @@ import { readsAsOne, textBreakName, textBreakSpelling } from '../text-break.ts' export type InlineContent = { carry?: undefined; image: AdfNode; nodes?: undefined } | { carry: boolean; image?: undefined; nodes: AdfNode[] } +// The text break leaf: it holds no marks, and stands between the nodes it parts until the outermost content checks and drops it. +type TextBreak = { kind: 'textBreak' } + +type Inline = AdfNode | TextBreak + +type Scanned = { carry?: undefined; image: AdfNode; nodes?: undefined } | { carry: boolean; image?: undefined; nodes: Inline[] } + export type LinkDefinitions = ReadonlyMap type Bracket = { active: boolean; image: boolean; kind: 'open'; start: number } @@ -34,10 +41,10 @@ type Pairing = EmphasisPairing // A carry piece holds a node whose marks are its own: an opaque carry, or an inline node spelling marks=empty. type Piece = | Bracket - | { kind: 'break' } | { kind: 'carry'; node: AdfNode } | { alt: string; kind: 'image'; node: AdfNode } - | { kind: 'nodes'; nodes: AdfNode[] } + | { kind: 'nodes'; nodes: Inline[] } + | TextBreak | { canClose: boolean; canOpen: boolean; character: string; kind: 'run'; length: number } | { closes: boolean; kind: 'highlight'; opens: boolean } @@ -45,14 +52,14 @@ type Run = { canClose: boolean; canOpen: boolean; character: string; index: numb // `container` is `undefined` inside a directive's content slot, the emitter's `bracketed`. type Scan = { - // Nodes a carry piece restored: they never merge, and only identity tells a carried textBreak from the leaf. - carried: Set container: LineContainer | undefined // Pieces below this have been walked for openers to deactivate: an image close folds the link-marked piece into alt text, leaving this the only record that the brackets around it are doomed. deactivatedBefore: number definitions: LinkDefinitions highlights: boolean openingSpellableLink: boolean + // The nodes carry pieces hold: none joins its neighbour. + ownMarks: Set path: ConvertErrorPath pending: string pieces: Piece[] @@ -60,7 +67,7 @@ type Scan = { spans: NestedSpans } -type SlotContent = { carry: boolean; nodes: AdfNode[] } +type SlotContent = { carry: boolean; nodes: Inline[] } const carriedInMark = 'no mark spelling wraps an opaque carry or an inline node spelling marks=empty: its marks are its own' const editorHighlight: AdfMark = { attrs: { color: '#f8e6a0' }, type: 'backgroundColor' } @@ -68,13 +75,26 @@ const hreflessLink = 'the link mark spells its href: this one spells none' const imageAlone = 'an image fits only as a paragraph of its own: this one sits inside other content' const linkInLink = 'no link wraps a link: the [content] this one marks already holds one' const spellableLink = 'link takes the directive form only where CommonMark cannot spell it: this one it can, as [text](url "title") or ' +const textBreak: TextBreak = { kind: 'textBreak' } +// The outermost content: only here does a text break see both its neighbours. export function parseInlineContent(source: string, definitions: LinkDefinitions, path: ConvertErrorPath, container: LineContainer, flavour: Flavour): Result { - return parseInline(source, { carried: new Set(), container, definitions, highlights: flavour === 'plain', path, spans: noSpans }) + const scan: Scan = { container, deactivatedBefore: 0, definitions, highlights: flavour === 'plain', openingSpellableLink: false, ownMarks: new Set(), path, pending: '', pieces: [], source, spans: noSpans } + const scanned = scanInline(scan) + if (!scanned.ok) return scanned + if (scanned.value.image !== undefined) return success({ image: scanned.value.image }) + const nodes = partText(scanned.value.nodes, scan) + if (!nodes.ok) return nodes + if (scan.openingSpellableLink) { + const takesDirective = openingLinkTakesDirective(nodes.value, path) + if (!takesDirective.ok) return takesDirective + if (!takesDirective.value) return failure('unsupported-node-shape', spellableLink, path) + } + return success({ carry: scanned.value.carry, nodes: nodes.value }) } -function parseInline(source: string, context: Pick): Result { - const scan: Scan = { ...context, deactivatedBefore: 0, openingSpellableLink: false, pending: '', pieces: [], source } +function scanInline(scan: Scan): Result { + const { source } = scan let index = 0 while (index < source.length) { switch (source.charAt(index)) { @@ -211,7 +231,7 @@ function directivePiece(scan: Scan, span: DirectiveSpan, index: number): Result< if (carried.fault !== undefined) return faulted(carried.fault, scan.path) return success(carryPiece(scan, carried.value)) } - if (span.name === textBreakName) return span.content === undefined && span.attributes.size === 0 ? success({ kind: 'break' }) : failure('unsupported-node-shape', `${textBreakName} spells the bare leaf form, ${textBreakSpelling}: this one spells more`, scan.path) + if (span.name === textBreakName) return span.content === undefined && span.attributes.size === 0 ? success(textBreak) : failure('unsupported-node-shape', `${textBreakName} spells the bare leaf form, ${textBreakSpelling}: this one spells more`, scan.path) const text = readTextDirective(span) if (text?.fault !== undefined) return faulted(text.fault, scan.path) if (text !== undefined) return success({ kind: 'nodes', nodes: [{ text: text.value, type: 'text' }] }) @@ -219,13 +239,15 @@ function directivePiece(scan: Scan, span: DirectiveSpan, index: number): Result< if (!slot.ok) return slot const mark = readDirectiveMark(span.name, span.attributes, scan.path) if (mark !== undefined) return mark.ok ? directiveMarkPiece(scan, span.name, mark.value, slot.value, index) : mark - const node = readInlineDirectiveNode(span.name, span.attributes, slot.value?.nodes, scan.path) + const content = slot.value?.nodes + if (content?.some(isTextBreak) === true) return failure('unsupported-node-shape', `${textBreakName} parts two text nodes: a content slot holds plain text`, scan.path) + const node = readInlineDirectiveNode(span.name, span.attributes, content === undefined ? undefined : adfNodes(content), scan.path) if (!node.ok) return node return success(node.value.marks === undefined ? { kind: 'nodes', nodes: [node.value] } : carryPiece(scan, node.value)) } function carryPiece(scan: Scan, node: AdfNode): Piece { - scan.carried.add(node) + scan.ownMarks.add(node) return { kind: 'carry', node } } @@ -240,11 +262,11 @@ function directiveMarkPiece(scan: Scan, name: string, mark: AdfMark, slot: SlotC } // spec/flavour.md, Marks. A link opening a paragraph may still need the directive form for the line it opens, which `assemble` asks the emitter. -function refuseLinkDirective(scan: Scan, mark: AdfMark, nodes: readonly AdfNode[], index: number): Result | undefined { +function refuseLinkDirective(scan: Scan, mark: AdfMark, nodes: readonly Inline[], index: number): Result | undefined { const href = linkHref(nodeAttrs(mark)) if (href === undefined) return failure('unsupported-node-shape', hreflessLink, scan.path) if (marksLink(nodes)) return failure('unsupported-node-shape', linkInLink, scan.path) - if (commonMarkLink(nodeAttrs(mark), href, nodes, 0, scan.container === undefined) === undefined) return undefined + if (commonMarkLink(nodeAttrs(mark), href, adfNodes(nodes), 0, scan.container === undefined) === undefined) return undefined if (index !== 0 || scan.container !== 'paragraph') return failure('unsupported-node-shape', spellableLink, scan.path) scan.openingSpellableLink = true return undefined @@ -252,7 +274,7 @@ function refuseLinkDirective(scan: Scan, mark: AdfMark, nodes: readonly AdfNode[ function slotContent(scan: Scan, span: DirectiveSpan): Result { if (span.content === undefined) return success(undefined) - const parsed = parseInline(span.content, { ...scan, container: undefined, highlights: false, spans: span.spans }) + const parsed = scanInline({ ...scan, container: undefined, deactivatedBefore: 0, highlights: false, openingSpellableLink: false, pending: '', pieces: [], source: span.content, spans: span.spans }) if (!parsed.ok) return parsed if (parsed.value.image !== undefined) return failure('unmappable-image', imageAlone, scan.path) return success(parsed.value) @@ -268,40 +290,44 @@ function pushNode(scan: Scan, node: AdfNode): void { scan.pieces.push({ kind: 'nodes', nodes: [node] }) } -function assemble(scan: Scan): Result { +function assemble(scan: Scan): Result { const only = scan.pieces[0] if (scan.pieces.length === 1 && only?.kind === 'image') return success({ image: only.node }) if (holdsImage(scan.pieces)) return failure('unmappable-image', imageAlone, scan.path) - const resolved = resolveNodes(scan.pieces, scan, true) - const nodes = resolved.ok && scan.container !== undefined ? partText(resolved.value, scan) : resolved + const nodes = resolveNodes(scan.pieces, scan, true) if (!nodes.ok) return nodes - if (scan.openingSpellableLink) { - const takesDirective = openingLinkTakesDirective(nodes.value, scan.path) - if (!takesDirective.ok) return takesDirective - if (!takesDirective.value) return failure('unsupported-node-shape', spellableLink, scan.path) - } return success({ carry: holdsCarry(scan.pieces), nodes: nodes.value }) } // spec/flavour.md, Inline nodes: the leaf builds no node, so only the pair it parts spells it. -function partText(nodes: readonly AdfNode[], scan: Scan): Result { +function partText(items: readonly Inline[], scan: Scan): Result { const parted: AdfNode[] = [] - for (const [index, node] of nodes.entries()) { - if (!isTextBreak(node, scan.carried)) { - parted.push(node) + for (const [index, item] of items.entries()) { + if (!isTextBreak(item)) { + parted.push(item) continue } - const previous = nodes[index - 1] - const next = nodes[index + 1] - if (previous === undefined || next === undefined || scan.carried.has(previous) || scan.carried.has(next) || !readsAsOne(previous, next)) { + const previous = items[index - 1] + const next = items[index + 1] + if (previous === undefined || next === undefined || isTextBreak(previous) || isTextBreak(next) || !readJoins(scan)(previous, next)) { return failure('unsupported-node-shape', `${textBreakName} parts two text nodes CommonMark reads back as one: this one parts something else`, scan.path) } } return success(parted) } -function isTextBreak(node: AdfNode, carried: ReadonlySet): boolean { - return node.type === textBreakName && !carried.has(node) +function readJoins(scan: Scan): (previous: AdfNode, node: AdfNode) => boolean { + return (previous, node) => !scan.ownMarks.has(previous) && !scan.ownMarks.has(node) && readsAsOne(previous, node) +} + +function isTextBreak(item: Inline): item is TextBreak { + return 'kind' in item +} + +function adfNodes(items: readonly Inline[]): AdfNode[] { + const nodes: AdfNode[] = [] + for (const item of items) if (!isTextBreak(item)) nodes.push(item) + return nodes } function holdsCarry(pieces: readonly Piece[]): boolean { @@ -317,8 +343,8 @@ function holdsLink(pieces: readonly Piece[]): boolean { return pieces.some((piece) => piece.kind === 'nodes' && marksLink(piece.nodes)) } -function marksLink(nodes: readonly AdfNode[]): boolean { - return nodes.some((node) => nodeMarks(node).some((mark) => mark.type === 'link')) +function marksLink(nodes: readonly Inline[]): boolean { + return adfNodes(nodes).some((node) => nodeMarks(node).some((mark) => mark.type === 'link')) } function readDelimiterRun(scan: Scan, index: number): number { @@ -455,8 +481,8 @@ function closeImage(scan: Scan, at: number, inner: readonly Piece[], definition: function imageAlt(inner: readonly Piece[], scan: Scan): Result { const nodes = resolveNodes(inner, scan, false) if (!nodes.ok) return nodes - if (nodes.value.some((node) => isTextBreak(node, scan.carried))) return failure('unsupported-node-shape', `${textBreakName} parts two text nodes: an image description holds plain text`, scan.path) - return success(nodes.value.map(altText).join('')) + if (nodes.value.some(isTextBreak)) return failure('unsupported-node-shape', `${textBreakName} parts two text nodes: an image description holds plain text`, scan.path) + return success(adfNodes(nodes.value).map(altText).join('')) } // spec/flavour.md, The CommonMark image: the description's plain text, the content slot included. @@ -468,22 +494,37 @@ function altText(node: AdfNode): string { } // An image's alt text is plain, so `highlights` is off there and every `==` stays text. -function resolveNodes(pieces: readonly Piece[], scan: Scan, highlights: boolean): Result { +function resolveNodes(pieces: readonly Piece[], scan: Scan, highlights: boolean): Result { const nodes = pieces.map(pieceNodes) const runs = delimiterRuns(pieces) const pairings = matchEmphasis(runs) writeUnpaired(nodes, runs, pairings) if (!markPairings(pieces, nodes, pairings)) return failure('unsupported-node-shape', carriedInMark, scan.path) markHighlights(pieces, nodes, highlights ? pairedHighlights(pieces, nodes) : []) - return success(mergeAdjacentText(nodes.flat(), (previous, node) => !scan.carried.has(previous) && !scan.carried.has(node) && readsAsOne(previous, node))) + return success(mergeText(nodes.flat(), readJoins(scan))) +} + +// A text break is a wall: the text on either side joins only its own side. +function mergeText(items: readonly Inline[], joins: (previous: AdfNode, node: AdfNode) => boolean): Inline[] { + const merged: Inline[] = [] + let run: AdfNode[] = [] + for (const item of items) { + if (!isTextBreak(item)) { + run.push(item) + continue + } + for (const node of mergeAdjacentText(run, joins)) merged.push(node) + merged.push(item) + run = [] + } + for (const node of mergeAdjacentText(run, joins)) merged.push(node) + return merged } // Only `imageAlt` reaches the image arm: everywhere else an image amid other content is refused first. // A highlight delimiter holds an empty text node until it pairs, so the emphasis around it marks it. -function pieceNodes(piece: Piece): AdfNode[] { +function pieceNodes(piece: Piece): Inline[] { switch (piece.kind) { - case 'break': - return [{ type: textBreakName }] case 'carry': return [piece.node] case 'highlight': @@ -496,6 +537,8 @@ function pieceNodes(piece: Piece): AdfNode[] { return bracketNodes(piece) case 'run': return [] + case 'textBreak': + return [piece] } } @@ -508,7 +551,7 @@ function delimiterRuns(pieces: readonly Piece[]): Run[] { } // A run gives its delimiters up from the head closing and the tail opening; what is left between them is text. -function writeUnpaired(nodes: AdfNode[][], runs: readonly Run[], pairings: readonly Pairing[]): void { +function writeUnpaired(nodes: Inline[][], runs: readonly Run[], pairings: readonly Pairing[]): void { const heads = new Map() const tails = new Map() for (const pairing of pairings) { @@ -523,7 +566,7 @@ function writeUnpaired(nodes: AdfNode[][], runs: readonly Run[], pairings: reado } // Innermost pairing first, so prepending leaves the marks array outermost first (spec/flavour.md, Marks). -function markPairings(pieces: readonly Piece[], nodes: AdfNode[][], pairings: readonly Pairing[]): boolean { +function markPairings(pieces: readonly Piece[], nodes: Inline[][], pairings: readonly Pairing[]): boolean { for (const pairing of pairings) { const mark: AdfMark = { type: markType(pairing.opener.character, pairing.used) } for (let index = pairing.opener.index + 1; index < pairing.closer.index; index += 1) { @@ -534,12 +577,12 @@ function markPairings(pieces: readonly Piece[], nodes: AdfNode[][], pairings: re return true } -function highlightDelimiters(pieces: readonly Piece[], nodes: readonly AdfNode[][]): HighlightDelimiter[] { +function highlightDelimiters(pieces: readonly Piece[], nodes: readonly Inline[][]): HighlightDelimiter[] { const found: HighlightDelimiter[] = [] let line = 0 let position = 0 for (const [index, piece] of pieces.entries()) { - const held = nodes[index] ?? [] + const held = adfNodes(nodes[index] ?? []) const [holder] = held if (piece.kind === 'highlight' && holder !== undefined) { found.push({ closes: piece.closes, holder, index, line, opens: piece.opens, position }) @@ -555,7 +598,7 @@ function highlightDelimiters(pieces: readonly Piece[], nodes: readonly AdfNode[] } // Each opener takes the next closer holding at least one character after it, both in one line and under the same marks. -function pairedHighlights(pieces: readonly Piece[], nodes: readonly AdfNode[][]): { closer: number; opener: number }[] { +function pairedHighlights(pieces: readonly Piece[], nodes: readonly Inline[][]): { closer: number; opener: number }[] { const found = highlightDelimiters(pieces, nodes) const paired: { closer: number; opener: number }[] = [] let closer = 0 @@ -574,9 +617,9 @@ function pairedHighlights(pieces: readonly Piece[], nodes: readonly AdfNode[][]) } // Atlassian's schema refuses a highlight on code, a node holds one highlight, and a rebuilt node would lose its attributes. -function markHighlights(pieces: readonly Piece[], nodes: AdfNode[][], paired: readonly { closer: number; opener: number }[]): void { +function markHighlights(pieces: readonly Piece[], nodes: Inline[][], paired: readonly { closer: number; opener: number }[]): void { for (const [index, piece] of pieces.entries()) { - if (piece.kind === 'highlight') nodes[index] = (nodes[index] ?? []).map((holder) => ({ ...holder, text: highlightDelimiter })) + if (piece.kind === 'highlight') nodes[index] = adfNodes(nodes[index] ?? []).map((holder) => ({ ...holder, text: highlightDelimiter })) } for (const { closer, opener } of paired) { nodes[opener] = [] @@ -585,7 +628,8 @@ function markHighlights(pieces: readonly Piece[], nodes: AdfNode[][], paired: re } } -function highlighted(node: AdfNode): AdfNode { +function highlighted(node: Inline): Inline { + if (isTextBreak(node)) return node const marks = nodeMarks(node) if (node.type !== 'text' || Object.keys(nodeAttrs(node)).length > 0 || marks.some((mark) => mark.type === 'code' || mark.type === 'backgroundColor')) return node return { ...node, marks: [editorHighlight, ...marks] } @@ -597,8 +641,9 @@ function markType(character: string, used: number): string { } // A node cannot carry one mark type twice (docs/decisions.md §No schema validation). -function applyMark(nodes: readonly AdfNode[], mark: AdfMark): AdfNode[] { +function applyMark(nodes: readonly Inline[], mark: AdfMark): Inline[] { return nodes.map((node) => { + if (isTextBreak(node)) return node const marks = nodeMarks(node) return marks.some((carried) => carried.type === mark.type) ? node : { ...node, marks: [mark, ...marks] } }) diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 2773f6d..490975e 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -1076,3 +1076,17 @@ test('names the directive mark left without the content it wraps', () => { assert.equal(content(markdownToAdf('!adf:underline[]\n')), named) assert.equal(content(markdownToAdf('!adf:underline{}\n')), named) }) + +test('reads the text break only between two text nodes CommonMark joins, building no node', () => { + assert.deepEqual(content(markdownToAdf('a!adf:textBreak{}b\n')), [{ content: [text('a'), text('b')], type: 'paragraph' }]) + assert.deepEqual(content(markdownToAdf('==a!adf:textBreak{}b==\n')), [{ content: [text('==a'), text('b==')], type: 'paragraph' }]) + const parts = 'unsupported-node-shape: textBreak parts two text nodes CommonMark reads back as one: this one parts something else' + const carried = '!adf:carry{json="{\\"text\\":\\"b\\",\\"type\\":\\"text\\"}"}' + for (const markdown of ['!adf:textBreak{}a\n', 'a!adf:textBreak{}\n', 'a!adf:textBreak{}!adf:textBreak{}b\n', '**a**!adf:textBreak{}b\n', '[!adf:textBreak{}](/u)\n', `a!adf:textBreak{}${carried}\n`]) { + assert.equal(content(markdownToAdf(markdown)), parts, markdown) + } + assert.equal(content(markdownToAdf('!adf:status[a!adf:textBreak{}b]\n')), 'unsupported-node-shape: textBreak parts two text nodes: a content slot holds plain text') + assert.equal(content(markdownToAdf('![a!adf:textBreak{}b](/i)\n')), 'unsupported-node-shape: textBreak parts two text nodes: an image description holds plain text') + assert.equal(content(markdownToAdf('a!adf:textBreak{x=y}b\n')), 'unsupported-node-shape: textBreak spells the bare leaf form, !adf:textBreak{}: this one spells more') + assert.deepEqual(content(markdownToAdf('!adf:carry{json="{\\"type\\":\\"textBreak\\"}"}\n')), [{ content: [{ type: 'textBreak' }], type: 'paragraph' }]) +}) -- 2.52.0 From afb53ca594d6668823a570a0959f77194e531149 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:20:03 +0200 Subject: [PATCH 08/67] Keep an empty stored document empty in the migration recipe --- MIGRATION.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/MIGRATION.md b/MIGRATION.md index 6ab5f2e..2dc0d2b 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -22,7 +22,8 @@ import { markdownToAdf as markdownToAdf010 } from 'adf-codec-0.1' function migrateMarkdown(stored: string) { const parsed = markdownToAdf010(stored) - return parsed.ok ? adfToMarkdown(parsed.value) : parsed + // 0.1.0's reader was editor-normal, so a document with no content key always meant an empty one. + return parsed.ok ? adfToMarkdown({ ...parsed.value, content: parsed.value.content ?? [] }) : parsed } ``` -- 2.52.0 From eab28ae8ff70543c2e2c300c8e444a4542c923ad Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:20:47 +0200 Subject: [PATCH 09/67] Refuse a carry fence naming a type no info string carries back --- corpus/errors/carry-fence-type-unspellable.error | 1 + corpus/errors/carry-fence-type-unspellable.md | 3 +++ src/markdown/opaque-carry.ts | 3 +++ src/markdown/parse/markdown-to-adf.test.ts | 2 ++ 4 files changed, 9 insertions(+) create mode 100644 corpus/errors/carry-fence-type-unspellable.error create mode 100644 corpus/errors/carry-fence-type-unspellable.md diff --git a/corpus/errors/carry-fence-type-unspellable.error b/corpus/errors/carry-fence-type-unspellable.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/carry-fence-type-unspellable.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/carry-fence-type-unspellable.md b/corpus/errors/carry-fence-type-unspellable.md new file mode 100644 index 0000000..26231ce --- /dev/null +++ b/corpus/errors/carry-fence-type-unspellable.md @@ -0,0 +1,3 @@ +```adf:\\ +{} +``` diff --git a/src/markdown/opaque-carry.ts b/src/markdown/opaque-carry.ts index 0a113d6..d70e116 100644 --- a/src/markdown/opaque-carry.ts +++ b/src/markdown/opaque-carry.ts @@ -33,6 +33,9 @@ export function carriedInline(node: AdfNode, path: ConvertErrorPath): Result { + if (type !== '' && !infoStringCarries(type)) { + return { fault: unsupportedNodeShape(`the carry fence names a type no info string carries back: spell it ${carryFencePrefix} with the type in the JSON`) } + } const read = readCarriedJson(body, 'two-space', largestNesting - depth, type) if (read.fault !== undefined || type !== '' || !infoStringCarries(read.value.type)) return read return { fault: unsupportedNodeShape(`the carry fence names a type its info string carries: spell it ${carryFencePrefix}${read.value.type}`) } diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 490975e..320b1d6 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -375,6 +375,8 @@ test('reads the carry fence back to the node its info string names and its JSON ) assert.equal(content(markdownToAdf('```adf:\n{\n "type": "blockCard"\n}\n```\n')), 'unsupported-node-shape: the carry fence names a type its info string carries: spell it adf:blockCard') assert.equal(content(markdownToAdf('```adf:\n{}\n```\n')), "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node") + const unnamed = 'unsupported-node-shape: the carry fence names a type no info string carries back: spell it adf: with the type in the JSON' + assert.equal(content(markdownToAdf('```adf:\\\\\n{}\n```\n')), unnamed) }) test('reads the inline carry back to the node its json attribute holds', () => { -- 2.52.0 From 1458b09449469ed2171a328a479cb02d3db8bc81 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:22:14 +0200 Subject: [PATCH 10/67] Index a plain refusal's path in the caller's document: normalize only the reduction's output --- src/markdown/emit/plain-reduction.test.ts | 1 + src/markdown/emit/plain-reduction.ts | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/src/markdown/emit/plain-reduction.test.ts b/src/markdown/emit/plain-reduction.test.ts index 8308214..2a3e15a 100644 --- a/src/markdown/emit/plain-reduction.test.ts +++ b/src/markdown/emit/plain-reduction.test.ts @@ -64,6 +64,7 @@ test('refuses what the document guard refuses, and nothing else', () => { let deep: AdfNode = said('x') for (let level = 0; level <= largestNesting; level += 1) deep = { content: [deep], type: 'layoutColumn' } assert.match(plainDocument(document(deep)), /^unsupported-nesting-depth at \/content\/0(\/content\/0)+$/) + assert.match(plainDocument(document(paragraph(text('a'), text('b')), text('c'), text('d'), deep)), /^unsupported-nesting-depth at \/content\/3(\/content\/0)+$/) let deepInline: AdfNode = text('x') for (let level = 0; level <= largestNesting; level += 1) deepInline = { content: [deepInline], type: 'unknownInline' } assert.match(plainDocument(document(paragraph(deepInline))), /^unsupported-nesting-depth at /) diff --git a/src/markdown/emit/plain-reduction.ts b/src/markdown/emit/plain-reduction.ts index 30d82bb..fb8916b 100644 --- a/src/markdown/emit/plain-reduction.ts +++ b/src/markdown/emit/plain-reduction.ts @@ -51,8 +51,8 @@ export function reduceToPlain(document: AdfDocument): Result { const fault = adfDocumentFault(document) if (fault !== undefined) return faulted(fault, []) if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) - // docs/decisions.md §Equality is deep: the plain flavour reads and writes editor-normal ADF. - const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] }) + // A refusal's path indexes the caller's document, so only the reduction's output normalizes (docs/decisions.md §Equality is deep). + const blocks = reduceBlocks(nodeContent(document), { depth: 0, memo: new Map(), path: [] }) return blocks.ok ? success({ content: nodeContent(toEditorNormal({ content: blocks.value, type: 'doc', version: 1 })).slice(), type: 'doc', version: 1 }) : blocks } -- 2.52.0 From 0bc19238b43d2afd8a7423ad6c26fb8ede7a9d18 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:23:04 +0200 Subject: [PATCH 11/67] Say only the plain writer is editor-normal, and read plain markdown in the tests without normalizing --- docs/decisions.md | 8 ++--- .../parse/plain-markdown-to-adf.test.ts | 33 +++++++++---------- 2 files changed, 20 insertions(+), 21 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index f8b1581..60c20f9 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -38,10 +38,10 @@ shape the editor would not. "Equals" is `assert.deepStrictEqual`: every key and value `doc` holds, two adjacent text nodes, an empty `attrs`, `content` or `marks` and `-0` included, with no normalization on either side. -CommonMark's spelling stays wherever a document holds none of those shapes. The plain flavour is -lossy and stays editor-normal: its reduction reads and writes ADF with adjacent text nodes of -identical marks and no attributes merged, `-0` read as `0`, and an empty `attrs`, `content` or -`marks` the absent key. +CommonMark's spelling stays wherever a document holds none of those shapes. The plain reader +builds what is written, as `markdownToAdf` does. Only the plain writer is lossy: its reduction +writes editor-normal ADF — adjacent text nodes of identical marks and no attributes merged, `-0` +as `0`, and an empty `attrs`, `content` or `marks` the absent key. ## `!adf:textBreak{}` parts text CommonMark would join diff --git a/src/markdown/parse/plain-markdown-to-adf.test.ts b/src/markdown/parse/plain-markdown-to-adf.test.ts index d38e883..13dadfb 100644 --- a/src/markdown/parse/plain-markdown-to-adf.test.ts +++ b/src/markdown/parse/plain-markdown-to-adf.test.ts @@ -6,7 +6,6 @@ import { adfToMarkdown } from '../emit/adf-to-markdown.ts' import { adfToPlainMarkdown } from '../emit/plain-reduction.ts' import { largestNesting } from '../../nesting.ts' import { markdownToAdf, plainMarkdownToAdf } from './markdown-to-adf.ts' -import { toEditorNormal } from '../../adf/editor-normal.ts' const code: AdfMark = { type: 'code' } const em: AdfMark = { type: 'em' } @@ -18,7 +17,7 @@ const taskTypes = ['blockTaskItem', 'taskItem', 'taskList'] function read(markdown: string): readonly AdfNode[] | string { const parsed = plainMarkdownToAdf(markdown) if (!parsed.ok) return parsed.error.code - const blocks = toEditorNormal(parsed.value).content ?? [] + const blocks = parsed.value.content ?? [] const pending = [...blocks] for (let block = pending.pop(); block !== undefined; block = pending.pop()) { for (const child of block.content ?? []) pending.push(child) @@ -31,8 +30,8 @@ function read(markdown: string): readonly AdfNode[] | string { return blocks } -function normal(...blocks: AdfNode[]): readonly AdfNode[] { - return toEditorNormal(document(...blocks)).content ?? [] +function blocks(...content: AdfNode[]): readonly AdfNode[] { + return content } function document(...content: AdfNode[]): AdfDocument { @@ -57,7 +56,7 @@ function bare(type: string, ...content: AdfNode[]): AdfNode { } function paragraph(...content: AdfNode[]): AdfNode { - return bare('paragraph', ...content) + return content.length === 0 ? { type: 'paragraph' } : bare('paragraph', ...content) } function said(value: string): AdfNode { @@ -101,7 +100,7 @@ test('reads the rest of an alert marker line as the panel first body paragraph, assert.deepEqual(read('> [!tip] Title\n> body\n'), [panel('tip', said('Title'), said('body'))]) assert.deepEqual(read('> [!tip] Title\\\n> body\n'), [panel('tip', said('Title'), said('body'))]) assert.deepEqual(read('> [!NOTE]\\\n> Broken.\n'), [panel('info', said('Broken.'))]) - assert.deepEqual(read('> [!NOTE]\n'), normal(panel('info', paragraph()))) + assert.deepEqual(read('> [!NOTE]\n'), blocks(panel('info', paragraph()))) assert.deepEqual(read('> > [!WARNING]\n> > Inner.\n'), [bare('blockquote', panel('warning', said('Inner.')))]) }) @@ -120,27 +119,27 @@ test('reads a folded callout to an expand titled by the rest of its marker line, node('expand', { title: 'Why?' }, paragraph(text('See '), text('the docs', link), text(', '), text('now', strong), text('.'))), ]) assert.deepEqual(read('> [!NOTE]- Two\\\n> lines\n'), [node('expand', { title: 'Two' }, said('lines'))]) - assert.deepEqual(read('> [!faq]- See [x](http://y)\n'), normal(node('expand', { title: 'See x (http://y)' }, paragraph()))) - assert.deepEqual(read('> [!faq]- [a **b**](u "t")[c](u) and [d](v)\n'), normal(node('expand', { title: 'a bc (u) and d (v)' }, paragraph()))) - assert.deepEqual(read('> [!faq]- or or \n'), normal(node('expand', { title: 'http://y or a@b.c or http://a\\b' }, paragraph()))) - assert.deepEqual(read('> [!faq]- [a b](u)\n'), normal(node('expand', { title: 'a\nb (u)' }, paragraph()))) - assert.deepEqual(read('> [!faq]- [!adf:mention[@M]{id=5}](u) !adf:inlineCard{url="http://y"}\n'), normal(node('expand', { title: '@M (u) http://y' }, paragraph()))) - assert.deepEqual(read('> [!NOTE]- Set ==x== here\n'), normal(node('expand', { title: 'Set ==x== here' }, paragraph()))) + assert.deepEqual(read('> [!faq]- See [x](http://y)\n'), blocks(node('expand', { title: 'See x (http://y)' }, paragraph()))) + assert.deepEqual(read('> [!faq]- [a **b**](u "t")[c](u) and [d](v)\n'), blocks(node('expand', { title: 'a bc (u) and d (v)' }, paragraph()))) + assert.deepEqual(read('> [!faq]- or or \n'), blocks(node('expand', { title: 'http://y or a@b.c or http://a\\b' }, paragraph()))) + assert.deepEqual(read('> [!faq]- [a b](u)\n'), blocks(node('expand', { title: 'a\nb (u)' }, paragraph()))) + assert.deepEqual(read('> [!faq]- [!adf:mention[@M]{id=5}](u) !adf:inlineCard{url="http://y"}\n'), blocks(node('expand', { title: '@M (u) http://y' }, paragraph()))) + assert.deepEqual(read('> [!NOTE]- Set ==x== here\n'), blocks(node('expand', { title: 'Set ==x== here' }, paragraph()))) assert.deepEqual(read('> [!NOTE]-\n>\n> Line.\n'), [bare('expand', said('Line.'))]) }) test('reads a folded callout inside an expand to a nested expand', () => { const markdown = '> [!NOTE]- Outer\n>\n> > [!NOTE]- Inner\n> >\n> > Deep.\n>\n> > [!TIP]\n> >\n> > > [!NOTE]-\n' - assert.deepEqual(read(markdown), normal(node('expand', { title: 'Outer' }, node('nestedExpand', { title: 'Inner' }, said('Deep.')), panel('tip', bare('nestedExpand', paragraph()))))) - assert.deepEqual(read('- > [!NOTE]-\n'), normal(bare('bulletList', bare('listItem', bare('expand', paragraph()))))) - assert.deepEqual(read('!adf:expand\n> [!NOTE]- Inner\n!adf:/expand\n'), normal(bare('expand', node('nestedExpand', { title: 'Inner' }, paragraph())))) + assert.deepEqual(read(markdown), blocks(node('expand', { title: 'Outer' }, node('nestedExpand', { title: 'Inner' }, said('Deep.')), panel('tip', bare('nestedExpand', paragraph()))))) + assert.deepEqual(read('- > [!NOTE]-\n'), blocks(bare('bulletList', bare('listItem', bare('expand', paragraph()))))) + assert.deepEqual(read('!adf:expand\n> [!NOTE]- Inner\n!adf:/expand\n'), blocks(bare('expand', node('nestedExpand', { title: 'Inner' }, paragraph())))) }) test('reads a bullet list whose every item leads with a task marker to a task list', () => { assert.deepEqual(read('- [x] Write the spec\n- [ ] Ship **it**\n- [X] Tell\n'), [ bare('taskList', task('DONE', text('Write the spec')), task('TODO', text('Ship '), text('it', strong)), task('DONE', text('Tell'))), ]) - assert.deepEqual(read('- [x]\n- [ ]\\\n after\n'), normal(bare('taskList', task('DONE'), task('TODO', text('after'))))) + assert.deepEqual(read('- [x]\n- [ ]\\\n after\n'), blocks(bare('taskList', task('DONE'), task('TODO', text('after'))))) const minted = plainMarkdownToAdf('- [x] Parent\n - [ ] Child\n') assert.deepEqual(minted.ok ? minted.value.content : minted.error.code, [ node( @@ -202,7 +201,7 @@ test('reads what the reduction wrote back to the node it reduced, less the attri assert.deepEqual(roundTripped(node('panel', { localId, panelType }, said('Check.'))), [panel(panelType, said('Check.'))], panelType) } const expand = node('expand', { localId, title: 'Log' }, said('Line.'), node('nestedExpand', { title: 'Inner' }, said('Deep.'))) - assert.deepEqual(roundTripped(node('panel', { panelType: 'tip' }, paragraph()), node('expand', { title: 'Empty' }, paragraph())), normal(panel('tip', paragraph()), node('expand', { title: 'Empty' }, paragraph()))) + assert.deepEqual(roundTripped(node('panel', { panelType: 'tip' }, paragraph()), node('expand', { title: 'Empty' }, paragraph())), blocks(panel('tip', paragraph()), node('expand', { title: 'Empty' }, paragraph()))) assert.deepEqual(roundTripped(expand), [node('expand', { title: 'Log' }, said('Line.'), node('nestedExpand', { title: 'Inner' }, said('Deep.')))]) const tasks = bare( 'taskList', -- 2.52.0 From 3da45a9da7a41833632fcabf9ea02777aac0299d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:23:13 +0200 Subject: [PATCH 12/67] File item 54: the source-parting decision misses an import parse/ takes from emit/ --- todo.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/todo.md b/todo.md index cab4975..75e3bf0 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 54` +`Next ID = 55` | Goal | W | |---|---| @@ -31,6 +31,7 @@ | 49 | 0.2.0 | | **Read a list whose bullet or ordered delimiter changes as two lists in the CommonMark reader.** | 4 | 5 | 5 | 8 | 3, 4 | 17.2 | | 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 5 | 13.0 | | 51 | 0.2.0 | | **Match a reference label to its definition under Unicode case folding.** | 2 | 2 | 2 | 7 | 3, 4 | 12.4 | +| 54 | 0.2.0 | | **Make `docs/decisions.md` §The source parts by ADF and format name every import `parse/` takes from `emit/`.** | 2 | 2 | 2 | 5 | 1 | 11.5 | | 53 | 0.2.0 | | **Bench a block's `marks` spelling with the writer panel and adopt its pick.** | 4 | 5 | 5 | 6 | 5 | 11.5 | | 50 | 0.2.0 | | **Read `[](/url)` and `[]()` as CommonMark's empty link.** | 4 | 4 | 3 | 6 | 3, 4, 6 | 10.4 | | 38 | 0.3.0 | | **Spell a lone surrogate in a text node so it survives a UTF-8 encode.** | 2 | 2 | 4 | 7 | 1 | 19.5 | @@ -95,6 +96,12 @@ example 540); lowercasing and then uppercasing folds it. Breaking, so it ships b `MIGRATION.md`'s Readings table gains its row. Its `pending` exceptions go, and its spelling leaves the README's "Four CommonMark spellings" bullet, which counts one fewer. +### 54. Make `docs/decisions.md` §The source parts by ADF and format name every import `parse/` takes from `emit/`. + +The entry says `parse/` imports only `commonMarkSpelling` and `openingLinkTakesDirective` from +`emit/`; `parse/markdown-to-adf.ts` also imports `inlineLeaves` from `emit/plain-inline.ts`. Either +the entry names it with its reason, or the reader stops importing it. + ### 53. Bench a block's `marks` spelling with the writer panel and adopt its pick. Today `marks="[{\"attrs\":{\"mode\":\"wide\"},\"type\":\"breakout\"}]"`, the marks array as -- 2.52.0 From 20a191e7e995ee0ba32cb9e43338d266ee53df01 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:32:34 +0200 Subject: [PATCH 13/67] Write the same plain markdown for documents the editor holds equal: normalize the reduction's input again, naming a refusal by the caller's document --- docs/decisions.md | 5 +++-- src/conformance/adf-property.test.ts | 10 ++++++++++ src/markdown/emit/plain-reduction.test.ts | 6 ++++++ src/markdown/emit/plain-reduction.ts | 13 +++++++++---- 4 files changed, 28 insertions(+), 6 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 60c20f9..f93d396 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -40,8 +40,9 @@ shape the editor would not. empty `attrs`, `content` or `marks` and `-0` included, with no normalization on either side. CommonMark's spelling stays wherever a document holds none of those shapes. The plain reader builds what is written, as `markdownToAdf` does. Only the plain writer is lossy: its reduction -writes editor-normal ADF — adjacent text nodes of identical marks and no attributes merged, `-0` -as `0`, and an empty `attrs`, `content` or `marks` the absent key. +reads and writes editor-normal ADF — adjacent text nodes of identical marks and no attributes +merged, `-0` as `0`, and an empty `attrs`, `content` or `marks` the absent key — so two documents +the editor holds equal write the same plain markdown. ## `!adf:textBreak{}` parts text CommonMark would join diff --git a/src/conformance/adf-property.test.ts b/src/conformance/adf-property.test.ts index 0061ea9..4b787ac 100644 --- a/src/conformance/adf-property.test.ts +++ b/src/conformance/adf-property.test.ts @@ -8,6 +8,7 @@ import { adfToMarkdown } from '../markdown/emit/adf-to-markdown.ts' import { adfToPlainMarkdown, reduceToPlain } from '../markdown/emit/plain-reduction.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' import { markdownToAdf, plainMarkdownToAdf } from '../markdown/parse/markdown-to-adf.ts' +import { toEditorNormal } from '../adf/editor-normal.ts' const gateRuns = 1600 const renamedPrefix = '!adg:' @@ -61,3 +62,12 @@ test('a generated document writes plain markdown refusing only what the guard re propertyRuns(gateRuns), ) }) + +test('a generated document writes the plain markdown its editor-normal form writes', { timeout: propertyTimeout }, () => { + fc.assert( + fc.property(adfDocument, (document) => { + assert.deepEqual(adfToPlainMarkdown(document), adfToPlainMarkdown(toEditorNormal(document))) + }), + propertyRuns(gateRuns), + ) +}) diff --git a/src/markdown/emit/plain-reduction.test.ts b/src/markdown/emit/plain-reduction.test.ts index 2a3e15a..185ae42 100644 --- a/src/markdown/emit/plain-reduction.test.ts +++ b/src/markdown/emit/plain-reduction.test.ts @@ -338,3 +338,9 @@ test('keeps the nodes the plain flavour spells and degrades only what it cannot' const reduced = reduceToPlain(document(node('panel', { localId: '01a0d99b-1f56-7a50-889a-f4375f09ee05', panelType: 'info' }, said('x')), tasks)) assert.deepEqual(reduced.ok ? reduced.value : undefined, document(node('panel', { panelType: 'info' }, said('x')), { content: [node('taskItem', { state: 'DONE' }, text('t'))], type: 'taskList' })) }) + +test('writes what the editor-normal form of the document writes', () => { + assert.equal(plain({ attrs: { order: -0 }, content: [{ content: [], type: 'listItem' }], type: 'orderedList' }), '0.\n') + assert.equal(plain({ content: [], text: 'hi', type: 'futureInline' }), 'hi\n') + assert.equal(plain(paragraph({ marks: [{ attrs: {}, type: 'code' }], text: 'a', type: 'text' }, { marks: [{ type: 'code' }], text: '|b', type: 'text' })), '`a|b`\n') +}) diff --git a/src/markdown/emit/plain-reduction.ts b/src/markdown/emit/plain-reduction.ts index fb8916b..055e7e1 100644 --- a/src/markdown/emit/plain-reduction.ts +++ b/src/markdown/emit/plain-reduction.ts @@ -51,9 +51,14 @@ export function reduceToPlain(document: AdfDocument): Result { const fault = adfDocumentFault(document) if (fault !== undefined) return faulted(fault, []) if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) - // A refusal's path indexes the caller's document, so only the reduction's output normalizes (docs/decisions.md §Equality is deep). - const blocks = reduceBlocks(nodeContent(document), { depth: 0, memo: new Map(), path: [] }) - return blocks.ok ? success({ content: nodeContent(toEditorNormal({ content: blocks.value, type: 'doc', version: 1 })).slice(), type: 'doc', version: 1 }) : blocks + // docs/decisions.md §Equality is deep: the reduction reads and writes editor-normal ADF. + const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] }) + if (!blocks.ok) { + // Merging text renumbers siblings, so the caller's document names the refusal's path. + const raw = reduceBlocks(nodeContent(document), { depth: 0, memo: new Map(), path: [] }) + return raw.ok ? blocks : raw + } + return success({ content: nodeContent(toEditorNormal({ content: blocks.value, type: 'doc', version: 1 })).slice(), type: 'doc', version: 1 }) } function reduceBlocks(nodes: readonly AdfNode[], reduction: Reduction): Result { @@ -86,7 +91,7 @@ function reduceStanding(node: AdfNode, reduction: Reduction): Result function standsInline(node: AdfNode): boolean { if (node.type === 'text' || inlineNodeModel(node.type) !== undefined) return true - return !isBlockNodeType(node.type) && node.content === undefined + return !isBlockNodeType(node.type) && nodeContent(node).length === 0 } function reduceBody(node: AdfNode, reduction: Reduction): Result { -- 2.52.0 From 45f1bee94b06fb16c755ab6868110e25ac3de1a0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:33:06 +0200 Subject: [PATCH 14/67] Set up every inline scan through one freshScan --- src/markdown/parse/inline-content.ts | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index 978b7d9..821943a 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -67,6 +67,9 @@ type Scan = { spans: NestedSpans } +// What a directive's content slot inherits from the content holding it. +type SharedScan = Pick + type SlotContent = { carry: boolean; nodes: Inline[] } const carriedInMark = 'no mark spelling wraps an opaque carry or an inline node spelling marks=empty: its marks are its own' @@ -79,7 +82,7 @@ const textBreak: TextBreak = { kind: 'textBreak' } // The outermost content: only here does a text break see both its neighbours. export function parseInlineContent(source: string, definitions: LinkDefinitions, path: ConvertErrorPath, container: LineContainer, flavour: Flavour): Result { - const scan: Scan = { container, deactivatedBefore: 0, definitions, highlights: flavour === 'plain', openingSpellableLink: false, ownMarks: new Set(), path, pending: '', pieces: [], source, spans: noSpans } + const scan = freshScan(source, { definitions, ownMarks: new Set(), path }, { container, highlights: flavour === 'plain', spans: noSpans }) const scanned = scanInline(scan) if (!scanned.ok) return scanned if (scanned.value.image !== undefined) return success({ image: scanned.value.image }) @@ -93,6 +96,10 @@ export function parseInlineContent(source: string, definitions: LinkDefinitions, return success({ carry: scanned.value.carry, nodes: nodes.value }) } +function freshScan(source: string, shared: SharedScan, own: Pick): Scan { + return { ...shared, ...own, deactivatedBefore: 0, openingSpellableLink: false, pending: '', pieces: [], source } +} + function scanInline(scan: Scan): Result { const { source } = scan let index = 0 @@ -274,7 +281,8 @@ function refuseLinkDirective(scan: Scan, mark: AdfMark, nodes: readonly Inline[] function slotContent(scan: Scan, span: DirectiveSpan): Result { if (span.content === undefined) return success(undefined) - const parsed = scanInline({ ...scan, container: undefined, deactivatedBefore: 0, highlights: false, openingSpellableLink: false, pending: '', pieces: [], source: span.content, spans: span.spans }) + const { definitions, ownMarks, path } = scan + const parsed = scanInline(freshScan(span.content, { definitions, ownMarks, path }, { container: undefined, highlights: false, spans: span.spans })) if (!parsed.ok) return parsed if (parsed.value.image !== undefined) return failure('unmappable-image', imageAlone, scan.path) return success(parsed.value) -- 2.52.0 From 9f0f520afdf9c02e4ccb58a4e2537bb341ce6dae Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:34:28 +0200 Subject: [PATCH 15/67] Give the comprehension panel its fill-ins --- AGENTS.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 23f0546..368df4d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -190,3 +190,14 @@ and `todo.md` and trusting them over anything remembered from earlier iterations is a thin driver: each chunk's work runs in a fresh-context subagent holding this file as its charter, and the driver only relays maintainer questions, runs the review flow, merges, and cleans up. The loop stops when only maintainer-reserved acts remain. + +## 8. Scoring run + +The comprehension panel's fill-ins: + +- Language: TypeScript. +- Kind: a pure-function document converter with hand-written parsers and emitters. +- Domain: Atlassian Document Format and CommonMark parsing. +- Domain docs: the CommonMark spec, ADF's JSON schema and `spec/flavour.md`. +- 3am question: a viewer/editor app reports that a document it saved comes back with two text + nodes merged and a mark gone after `markdownToAdf(adfToMarkdown(doc))`. -- 2.52.0 From a5238e4fe5245aebeceda8a9dcf4caf796fb4ed3 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:40:31 +0200 Subject: [PATCH 16/67] Cite item 43 as what ends the CommonMark carve-outs and the adf: fence reservation --- docs/decisions.md | 8 ++++++-- todo.md | 3 ++- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index f93d396..38bea5b 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -113,7 +113,9 @@ node's JSON without `type`: ```` ```adf:blockCard ````. A type no info string ca rule a code language follows — leaves the info string `adf:` and keeps `type` in the body. Every info string opening `adf:` is reserved, so a `codeBlock` whose language opens so takes the `language` attribute, and `carry` is an ordinary language. A body holding `type` under a named -type, or an `adf:` fence whose type an info string carries, is `unsupported-node-shape`. +type, or an `adf:` fence whose type an info string carries, is `unsupported-node-shape`. The +reservation claims a fence CommonMark reads as code until `todo.md` item 43 gives CommonMark its own +reader. ## A code block is a fence per text node @@ -171,7 +173,9 @@ opener nests by itself and leaf versus container falls out of the node's content 2026-08-23, the maintainer. Goal 4. Valid while prose rarely writes the shapes the carve-outs claim. Plain CommonMark is a subset, with carve-outs (`spec/flavour.md`): literal text shaped like a -directive, a pipe table or a `~~` pair is claimed — plus one image gap. +directive, a pipe table or a `~~` pair is claimed, and so is a code fence whose info string opens +`adf:` (§The carry fence names the node type) — plus one image gap. `todo.md` item 43 ends the +claims, giving CommonMark its own reader. ## Tables diff --git a/todo.md b/todo.md index 75e3bf0..fba68d1 100644 --- a/todo.md +++ b/todo.md @@ -68,7 +68,8 @@ HTML in input). The set sorts per `docs/decisions.md` §Foreign HTML sorts three ### 43. Give each markdown input its own reader, strict to its own standard. Today `markdownToAdf` reads CommonMark and the lossless flavour as one input: text shaped like a -directive, a pipe table or a `~~` pair becomes a flavour node where CommonMark reads plain text. A +directive, a pipe table or a `~~` pair becomes a flavour node where CommonMark reads plain text, and +a code fence whose info string opens `adf:` becomes the block carry where CommonMark reads code. A caller names the markdown it hands in: CommonMark, read as its spec says, or the lossless flavour, read as `spec/flavour.md` says. Breaking: `MIGRATION.md` says which call a caller takes. -- 2.52.0 From 723c0e31efb42da81da72eb43043379171c632e9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:41:30 +0200 Subject: [PATCH 17/67] Name the adf: fence among the carve-outs, and the way a fence stays code in its errors --- README.md | 7 +++-- spec/flavour.md | 7 +++-- src/markdown/opaque-carry.ts | 14 ++++++--- src/markdown/parse/markdown-to-adf.test.ts | 35 ++++++++++++---------- 4 files changed, 38 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index e69daa6..1b2a592 100644 --- a/README.md +++ b/README.md @@ -175,7 +175,7 @@ Parsing — `markdownToAdf` and `plainMarkdownToAdf`, and `htmlToAdf` at `0.2.0` | Code | Fires when | What you can do | | --- | --- | --- | -| `malformed-directive` | an `!adf:` the grammar cannot read — a prefix completing no directive, an unclosed container, `[content]` or `{attrs}`, a closer with no container of its name open, a leaf given a body, `{attrs}` out of order or duplicated, invalid JSON in an opaque carry | write the spelling the message names, or escape the prefix — `\!adf:`, block and inline alike — to keep it literal text | +| `malformed-directive` | an `!adf:` the grammar cannot read — a prefix completing no directive, an unclosed container, `[content]` or `{attrs}`, a closer with no container of its name open, a leaf given a body, `{attrs}` out of order or duplicated, invalid JSON in an opaque carry | write the spelling the message names, or keep it literal: escape the prefix — `\!adf:`, block and inline alike — or wrap a code fence whose info string opens `adf:` in `!adf:codeBlock {language="adf:…"}` with a bare fence | | `malformed-pipe-table` | a pipe row that is no pipe table — a missing or ragged `---` delimiter row, an alignment colon in it, or a row not opening with a pipe | open every row with a pipe and give the delimiter row the header's cell count; to keep the lines literal text instead, escape the leading pipe of every one — escaping a single row leaves the next to open a fresh table and fail the same way | | `unknown-directive-name` | a directive whose name is no node or mark this version spells | check the name in `spec/flavour.md`, or escape the prefix as `\!adf:`; the spelling itself is well formed, so a later minor may give the name meaning | | `unmappable-html` | the input holds an HTML construct the documented element set does not map, a comment and a processing instruction among them — at this version that is every raw HTML construct in markdown, the element set landing at `0.2.0` | remove the construct, or write what it holds in the lossless flavour | @@ -212,8 +212,9 @@ Serves Goals 1, 3 and 4. `0.2.0`, well-formed HTML means what the HTML standard says, read or written. The bullets below name every exception. - Plain CommonMark is valid input to `markdownToAdf` apart from the raw HTML `unmappable-html` - names, with three carve-outs — literal text matching directive, pipe-table or strikethrough - syntax is claimed (escapable — `spec/flavour.md`) — and one gap: a CommonMark image fits only as + names, with four carve-outs — literal text matching directive, pipe-table or strikethrough + syntax, and a code fence whose info string opens `adf:`, is claimed (escapable — `spec/flavour.md`) + — and one gap: a CommonMark image fits only as its own title-less paragraph; mid-text and titled images are error results, save an image inside another's description, which flattens into the alt text. Converting back yields the library's canonical spelling, which round-trips byte-identically — where it converts back at all: a parse diff --git a/spec/flavour.md b/spec/flavour.md index d61d2f0..e9a35b5 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -1,9 +1,10 @@ # The markdown flavour The grammar of the extended markdown `adfToMarkdown` emits and `markdownToAdf` parses. Plain -CommonMark is a subset apart from raw HTML (below), with three carve-outs: literal text that matches -directive syntax below or reads as a pipe table is claimed by the flavour, and a matched `~~` pair -spells `strike` (escape the `!adf:`, `|` or `~` to keep it literal) — and one gap: a CommonMark +CommonMark is a subset apart from raw HTML (below), with four carve-outs: literal text that matches +directive syntax below or reads as a pipe table is claimed by the flavour, a matched `~~` pair +spells `strike` (escape the `!adf:`, `|` or `~` to keep it literal), and a code fence whose info +string opens `adf:` is the opaque carry (The opaque carry says how to keep it code) — and one gap: a CommonMark image fits only as its own title-less paragraph — mid-text and titled images are named errors. The emitted form is contract (`docs/decisions.md` §The formats are API). Per-node syntaxes build on this grammar in the sections below. diff --git a/src/markdown/opaque-carry.ts b/src/markdown/opaque-carry.ts index d70e116..f15960f 100644 --- a/src/markdown/opaque-carry.ts +++ b/src/markdown/opaque-carry.ts @@ -8,7 +8,7 @@ import { infoStringCarries } from './commonmark/grammar.ts' import { isAdfNode } from '../adf/document.ts' import { isJsonValue, nestingDepth, overNested } from '../json-value.ts' import { largestNesting } from '../nesting.ts' -import { malformedDirective, readSoleStringAttribute, spellAttributes, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts' +import { directiveEscape, malformedDirective, readSoleStringAttribute, spellAttributes, spellDirectiveOpener, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts' import { serializeCanonicalJson } from '../canonical-json.ts' export const carryFencePrefix = 'adf:' @@ -58,8 +58,9 @@ function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, pat // `type` is the one the fence's info string names, `undefined` for the inline carry. function readCarriedJson(raw: string, spelling: JsonSpelling, levels: number, type?: string): Read { + const wayOut = type === undefined ? directiveEscape : fenceEscape(type) const parsed = parseJsonText(raw) - if (parsed === undefined) return { fault: malformedDirective('the opaque carry holds invalid JSON') } + if (parsed === undefined) return { fault: malformedDirective(`the opaque carry holds invalid JSON; ${wayOut}`) } const { value } = parsed if (!isJsonValue(value)) return { fault: unsupportedNodeShape('the opaque carry holds a number JSON cannot spell') } const typed = type === undefined || type === '' ? { value } : typedValue(value, type) @@ -70,12 +71,17 @@ function readCarriedJson(raw: string, spelling: JsonSpelling, levels: number, ty } if (serializeCanonicalJson(value, spelling) !== raw) { const shape = spelling === 'compact' ? 'compact, keys sorted' : 'two-space indent, keys sorted' - return { fault: unsupportedNodeShape(`the opaque carry spells its node's JSON canonically: ${shape}`) } + return { fault: unsupportedNodeShape(`the opaque carry spells its node's JSON canonically: ${shape}; ${wayOut}`) } } - if (!isAdfNode(held)) return { fault: unsupportedNodeShape("the opaque carry holds one ADF node's JSON: this JSON is no ADF node") } + if (!isAdfNode(held)) return { fault: unsupportedNodeShape(`the opaque carry holds one ADF node's JSON: this JSON is no ADF node; ${wayOut}`) } return { value: held } } +// A code fence the carry claims stays code under the directive, whose language rides the attribute. +function fenceEscape(type: string): string { + return `${spellDirectiveOpener('codeBlock', undefined, spellAttributes([['language', spellStringAttribute(`${carryFencePrefix}${type}`)]]))} around a bare fence keeps it a code block` +} + function typedValue(value: JsonValue, type: string): Read { if (value === null || typeof value !== 'object' || Array.isArray(value)) return { value } if ('type' in value) return { fault: unsupportedNodeShape(`the ${carryFencePrefix}${type} fence names its node's type: this JSON holds a type as well`) } diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 320b1d6..f692896 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -374,7 +374,7 @@ test('reads the carry fence back to the node its info string names and its JSON "unsupported-node-shape: the adf:blockCard fence names its node's type: this JSON holds a type as well", ) assert.equal(content(markdownToAdf('```adf:\n{\n "type": "blockCard"\n}\n```\n')), 'unsupported-node-shape: the carry fence names a type its info string carries: spell it adf:blockCard') - assert.equal(content(markdownToAdf('```adf:\n{}\n```\n')), "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node") + assert.equal(content(markdownToAdf('```adf:\n{}\n```\n')), "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node; !adf:codeBlock {language=\"adf:\"} around a bare fence keeps it a code block") const unnamed = 'unsupported-node-shape: the carry fence names a type no info string carries back: spell it adf: with the type in the JSON' assert.equal(content(markdownToAdf('```adf:\\\\\n{}\n```\n')), unnamed) }) @@ -385,26 +385,31 @@ test('reads the inline carry back to the node its json attribute holds', () => { ]) }) -test('names the invalid JSON no opaque carry holds', () => { - const invalid = 'malformed-directive: the opaque carry holds invalid JSON' - assert.equal(content(markdownToAdf('```adf:x\n{"attrs":\n```\n')), invalid) - assert.equal(content(markdownToAdf('```adf:x\n```\n')), invalid) - assert.equal(content(markdownToAdf('!adf:carry{json="{"}\n')), invalid) - assert.equal(content(markdownToAdf('!adf:carry{json=abc}\n')), invalid) +test('names the invalid JSON no opaque carry holds, and the spelling that keeps it literal', () => { + const invalid = 'malformed-directive: the opaque carry holds invalid JSON; ' + const fence = `${invalid}!adf:codeBlock {language="adf:x"} around a bare fence keeps it a code block` + assert.equal(content(markdownToAdf('```adf:x\n{"attrs":\n```\n')), fence) + assert.equal(content(markdownToAdf('```adf:x\n```\n')), fence) + assert.equal(content(markdownToAdf('!adf:carry{json="{"}\n')), `${invalid}\\!adf: keeps the prefix literal`) + assert.equal(content(markdownToAdf('!adf:carry{json=abc}\n')), `${invalid}\\!adf: keeps the prefix literal`) }) test('names the canonical spelling a carried JSON reads alone', () => { const canonically = "unsupported-node-shape: the opaque carry spells its node's JSON canonically: " - assert.equal(content(markdownToAdf('```adf:blockCard\n{"attrs":{}}\n```\n')), `${canonically}two-space indent, keys sorted`) - assert.equal(content(markdownToAdf('!adf:carry{json="{\\"type\\": \\"blockCard\\"}"}\n')), `${canonically}compact, keys sorted`) - assert.equal(content(markdownToAdf('!adf:carry{json="{\\"type\\":\\"blockCard\\",\\"attrs\\":{}}"}\n')), `${canonically}compact, keys sorted`) + const literal = '; \\!adf: keeps the prefix literal' + assert.equal( + content(markdownToAdf('```adf:blockCard\n{"attrs":{}}\n```\n')), + `${canonically}two-space indent, keys sorted; !adf:codeBlock {language="adf:blockCard"} around a bare fence keeps it a code block`, + ) + assert.equal(content(markdownToAdf('!adf:carry{json="{\\"type\\": \\"blockCard\\"}"}\n')), `${canonically}compact, keys sorted${literal}`) + assert.equal(content(markdownToAdf('!adf:carry{json="{\\"type\\":\\"blockCard\\",\\"attrs\\":{}}"}\n')), `${canonically}compact, keys sorted${literal}`) }) test('names the node JSON an opaque carry restores alone', () => { - const node = "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node" - assert.equal(content(markdownToAdf('```adf:x\n[]\n```\n')), node) - assert.equal(content(markdownToAdf('!adf:carry{json=null}\n')), node) - assert.equal(content(markdownToAdf('!adf:carry{json="{\\"kind\\":\\"x\\"}"}\n')), node) + const node = "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node; " + assert.equal(content(markdownToAdf('```adf:x\n[]\n```\n')), `${node}!adf:codeBlock {language="adf:x"} around a bare fence keeps it a code block`) + assert.equal(content(markdownToAdf('!adf:carry{json=null}\n')), `${node}\\!adf: keeps the prefix literal`) + assert.equal(content(markdownToAdf('!adf:carry{json="{\\"kind\\":\\"x\\"}"}\n')), `${node}\\!adf: keeps the prefix literal`) }) test('names the shape the inline carry reads alone', () => { @@ -421,7 +426,7 @@ test('holds a carried JSON value to the nesting its position leaves', () => { assert.equal(content(markdownToAdf(`!adf:carry{json="${nested(largestNesting + 2)}"}\n`)), deeper(largestNesting)) assert.equal( content(markdownToAdf(fence('', largestNesting + 1))), - "unsupported-node-shape: the opaque carry spells its node's JSON canonically: two-space indent, keys sorted", + 'unsupported-node-shape: the opaque carry spells its node\'s JSON canonically: two-space indent, keys sorted; !adf:codeBlock {language="adf:x"} around a bare fence keeps it a code block', ) assert.equal(content(markdownToAdf(fence('> ', largestNesting + 1))), deeper(largestNesting - 1)) }) -- 2.52.0 From 31576bb8249a9eb03fa73ffe3fe14f8a057f802a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:41:42 +0200 Subject: [PATCH 18/67] Take the raw run's refusal path only where its code matches the editor-normal refusal --- src/markdown/emit/plain-reduction.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/markdown/emit/plain-reduction.ts b/src/markdown/emit/plain-reduction.ts index 055e7e1..00883e0 100644 --- a/src/markdown/emit/plain-reduction.ts +++ b/src/markdown/emit/plain-reduction.ts @@ -54,9 +54,9 @@ export function reduceToPlain(document: AdfDocument): Result { // docs/decisions.md §Equality is deep: the reduction reads and writes editor-normal ADF. const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] }) if (!blocks.ok) { - // Merging text renumbers siblings, so the caller's document names the refusal's path. + // Merging text renumbers siblings, so the caller's document names the path of the refusal the editor-normal one meets. const raw = reduceBlocks(nodeContent(document), { depth: 0, memo: new Map(), path: [] }) - return raw.ok ? blocks : raw + return !raw.ok && raw.error.code === blocks.error.code ? raw : blocks } return success({ content: nodeContent(toEditorNormal({ content: blocks.value, type: 'doc', version: 1 })).slice(), type: 'doc', version: 1 }) } -- 2.52.0 From ade3997cfdb67f98817c9a13a3f6ee890433a8fd Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:43:03 +0200 Subject: [PATCH 19/67] Hold a carried node's own marks on the item, not in a side set copies fall out of --- src/markdown/parse/inline-content.ts | 61 +++++++++---------- .../parse/plain-markdown-to-adf.test.ts | 5 ++ 2 files changed, 35 insertions(+), 31 deletions(-) diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index 821943a..a2d6d24 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -26,7 +26,10 @@ export type InlineContent = { carry?: undefined; image: AdfNode; nodes?: undefin // The text break leaf: it holds no marks, and stands between the nodes it parts until the outermost content checks and drops it. type TextBreak = { kind: 'textBreak' } -type Inline = AdfNode | TextBreak +// A node whose marks are its own — an opaque carry, or an inline node spelling marks=empty: no mark spelling wraps it, and it joins no neighbour. +type Carried = { kind: 'carry'; node: AdfNode } + +type Inline = AdfNode | Carried | TextBreak type Scanned = { carry?: undefined; image: AdfNode; nodes?: undefined } | { carry: boolean; image?: undefined; nodes: Inline[] } @@ -38,10 +41,9 @@ type HighlightDelimiter = { closes: boolean; holder: AdfNode; index: number; lin type Pairing = EmphasisPairing -// A carry piece holds a node whose marks are its own: an opaque carry, or an inline node spelling marks=empty. type Piece = | Bracket - | { kind: 'carry'; node: AdfNode } + | Carried | { alt: string; kind: 'image'; node: AdfNode } | { kind: 'nodes'; nodes: Inline[] } | TextBreak @@ -58,8 +60,6 @@ type Scan = { definitions: LinkDefinitions highlights: boolean openingSpellableLink: boolean - // The nodes carry pieces hold: none joins its neighbour. - ownMarks: Set path: ConvertErrorPath pending: string pieces: Piece[] @@ -68,7 +68,7 @@ type Scan = { } // What a directive's content slot inherits from the content holding it. -type SharedScan = Pick +type SharedScan = Pick type SlotContent = { carry: boolean; nodes: Inline[] } @@ -82,7 +82,7 @@ const textBreak: TextBreak = { kind: 'textBreak' } // The outermost content: only here does a text break see both its neighbours. export function parseInlineContent(source: string, definitions: LinkDefinitions, path: ConvertErrorPath, container: LineContainer, flavour: Flavour): Result { - const scan = freshScan(source, { definitions, ownMarks: new Set(), path }, { container, highlights: flavour === 'plain', spans: noSpans }) + const scan = freshScan(source, { definitions, path }, { container, highlights: flavour === 'plain', spans: noSpans }) const scanned = scanInline(scan) if (!scanned.ok) return scanned if (scanned.value.image !== undefined) return success({ image: scanned.value.image }) @@ -236,7 +236,7 @@ function directivePiece(scan: Scan, span: DirectiveSpan, index: number): Result< const carried = readCarriedInline(span) if (carried !== undefined) { if (carried.fault !== undefined) return faulted(carried.fault, scan.path) - return success(carryPiece(scan, carried.value)) + return success({ kind: 'carry', node: carried.value }) } if (span.name === textBreakName) return span.content === undefined && span.attributes.size === 0 ? success(textBreak) : failure('unsupported-node-shape', `${textBreakName} spells the bare leaf form, ${textBreakSpelling}: this one spells more`, scan.path) const text = readTextDirective(span) @@ -250,12 +250,7 @@ function directivePiece(scan: Scan, span: DirectiveSpan, index: number): Result< if (content?.some(isTextBreak) === true) return failure('unsupported-node-shape', `${textBreakName} parts two text nodes: a content slot holds plain text`, scan.path) const node = readInlineDirectiveNode(span.name, span.attributes, content === undefined ? undefined : adfNodes(content), scan.path) if (!node.ok) return node - return success(node.value.marks === undefined ? { kind: 'nodes', nodes: [node.value] } : carryPiece(scan, node.value)) -} - -function carryPiece(scan: Scan, node: AdfNode): Piece { - scan.ownMarks.add(node) - return { kind: 'carry', node } + return success(node.value.marks === undefined ? { kind: 'nodes', nodes: [node.value] } : { kind: 'carry', node: node.value }) } function directiveMarkPiece(scan: Scan, name: string, mark: AdfMark, slot: SlotContent | undefined, index: number): Result { @@ -281,8 +276,8 @@ function refuseLinkDirective(scan: Scan, mark: AdfMark, nodes: readonly Inline[] function slotContent(scan: Scan, span: DirectiveSpan): Result { if (span.content === undefined) return success(undefined) - const { definitions, ownMarks, path } = scan - const parsed = scanInline(freshScan(span.content, { definitions, ownMarks, path }, { container: undefined, highlights: false, spans: span.spans })) + const { definitions, path } = scan + const parsed = scanInline(freshScan(span.content, { definitions, path }, { container: undefined, highlights: false, spans: span.spans })) if (!parsed.ok) return parsed if (parsed.value.image !== undefined) return failure('unmappable-image', imageAlone, scan.path) return success(parsed.value) @@ -312,29 +307,33 @@ function partText(items: readonly Inline[], scan: Scan): Result { const parted: AdfNode[] = [] for (const [index, item] of items.entries()) { if (!isTextBreak(item)) { - parted.push(item) + parted.push(isNode(item) ? item : item.node) continue } const previous = items[index - 1] const next = items[index + 1] - if (previous === undefined || next === undefined || isTextBreak(previous) || isTextBreak(next) || !readJoins(scan)(previous, next)) { + if (previous === undefined || next === undefined || !isNode(previous) || !isNode(next) || !readsAsOne(previous, next)) { return failure('unsupported-node-shape', `${textBreakName} parts two text nodes CommonMark reads back as one: this one parts something else`, scan.path) } } return success(parted) } -function readJoins(scan: Scan): (previous: AdfNode, node: AdfNode) => boolean { - return (previous, node) => !scan.ownMarks.has(previous) && !scan.ownMarks.has(node) && readsAsOne(previous, node) +function isNode(item: Inline): item is AdfNode { + return !('kind' in item) } function isTextBreak(item: Inline): item is TextBreak { - return 'kind' in item + return 'kind' in item && item.kind === 'textBreak' } +// The nodes the items hold, carried ones unwrapped and text breaks dropped. function adfNodes(items: readonly Inline[]): AdfNode[] { const nodes: AdfNode[] = [] - for (const item of items) if (!isTextBreak(item)) nodes.push(item) + for (const item of items) { + if (isNode(item)) nodes.push(item) + else if (!isTextBreak(item)) nodes.push(item.node) + } return nodes } @@ -509,23 +508,23 @@ function resolveNodes(pieces: readonly Piece[], scan: Scan, highlights: boolean) writeUnpaired(nodes, runs, pairings) if (!markPairings(pieces, nodes, pairings)) return failure('unsupported-node-shape', carriedInMark, scan.path) markHighlights(pieces, nodes, highlights ? pairedHighlights(pieces, nodes) : []) - return success(mergeText(nodes.flat(), readJoins(scan))) + return success(mergeText(nodes.flat())) } -// A text break is a wall: the text on either side joins only its own side. -function mergeText(items: readonly Inline[], joins: (previous: AdfNode, node: AdfNode) => boolean): Inline[] { +// A text break and a carried node are walls: the text on either side joins only its own side. +function mergeText(items: readonly Inline[]): Inline[] { const merged: Inline[] = [] let run: AdfNode[] = [] for (const item of items) { - if (!isTextBreak(item)) { + if (isNode(item)) { run.push(item) continue } - for (const node of mergeAdjacentText(run, joins)) merged.push(node) + for (const node of mergeAdjacentText(run, readsAsOne)) merged.push(node) merged.push(item) run = [] } - for (const node of mergeAdjacentText(run, joins)) merged.push(node) + for (const node of mergeAdjacentText(run, readsAsOne)) merged.push(node) return merged } @@ -534,7 +533,7 @@ function mergeText(items: readonly Inline[], joins: (previous: AdfNode, node: Ad function pieceNodes(piece: Piece): Inline[] { switch (piece.kind) { case 'carry': - return [piece.node] + return [piece] case 'highlight': return [{ text: '', type: 'text' }] case 'image': @@ -637,7 +636,7 @@ function markHighlights(pieces: readonly Piece[], nodes: Inline[][], paired: rea } function highlighted(node: Inline): Inline { - if (isTextBreak(node)) return node + if (!isNode(node)) return node const marks = nodeMarks(node) if (node.type !== 'text' || Object.keys(nodeAttrs(node)).length > 0 || marks.some((mark) => mark.type === 'code' || mark.type === 'backgroundColor')) return node return { ...node, marks: [editorHighlight, ...marks] } @@ -651,7 +650,7 @@ function markType(character: string, used: number): string { // A node cannot carry one mark type twice (docs/decisions.md §No schema validation). function applyMark(nodes: readonly Inline[], mark: AdfMark): Inline[] { return nodes.map((node) => { - if (isTextBreak(node)) return node + if (!isNode(node)) return node const marks = nodeMarks(node) return marks.some((carried) => carried.type === mark.type) ? node : { ...node, marks: [mark, ...marks] } }) diff --git a/src/markdown/parse/plain-markdown-to-adf.test.ts b/src/markdown/parse/plain-markdown-to-adf.test.ts index 13dadfb..e6144c6 100644 --- a/src/markdown/parse/plain-markdown-to-adf.test.ts +++ b/src/markdown/parse/plain-markdown-to-adf.test.ts @@ -265,3 +265,8 @@ test('keeps what markdownToAdf reads that no row reads, and refuses only what it for (let level = 0; level < largestNesting; level += 1) deep = `> ${deep}` assert.equal(typeof read(deep), 'object') }) + +test('joins a carried text node to no neighbour, inside a highlight too', () => { + const carried = '!adf:carry{json="{\\"marks\\":[],\\"text\\":\\"b\\",\\"type\\":\\"text\\"}"}' + assert.deepEqual(read(`==a${carried}==\n`), blocks(paragraph(text('a', highlight), { marks: [], text: 'b', type: 'text' }))) +}) -- 2.52.0 From a565d49c24859082e9aca113fb33cd6d269f6510 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:44:00 +0200 Subject: [PATCH 20/67] Spell 'a text node holding its text and marks alone' once, as isPlainText --- corpus/errors/directive-slot-carried-marks.error | 1 + corpus/errors/directive-slot-carried-marks.md | 1 + src/adf/document.ts | 5 +++++ src/markdown/emit/adf-to-markdown.ts | 4 ++-- src/markdown/emit/inline-line.ts | 10 +++++----- src/markdown/parse/directive-nodes.ts | 4 ++-- src/markdown/text-break.ts | 8 ++------ 7 files changed, 18 insertions(+), 15 deletions(-) create mode 100644 corpus/errors/directive-slot-carried-marks.error create mode 100644 corpus/errors/directive-slot-carried-marks.md diff --git a/corpus/errors/directive-slot-carried-marks.error b/corpus/errors/directive-slot-carried-marks.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/directive-slot-carried-marks.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/directive-slot-carried-marks.md b/corpus/errors/directive-slot-carried-marks.md new file mode 100644 index 0000000..c593b3e --- /dev/null +++ b/corpus/errors/directive-slot-carried-marks.md @@ -0,0 +1 @@ +!adf:status[!adf:carry{json="{\"marks\":[],\"text\":\"x\",\"type\":\"text\"}"}] diff --git a/src/adf/document.ts b/src/adf/document.ts index 752e517..1ae004c 100644 --- a/src/adf/document.ts +++ b/src/adf/document.ts @@ -64,6 +64,11 @@ export function emptyKeys(held: { attrs?: AdfAttributes; content?: AdfNode[]; ma return keys } +// A text node holding its text and marks alone: a format spells it as text, where anything more rides a carry. +export function isPlainText(node: AdfNode): boolean { + return node.type === 'text' && node.attrs === undefined && node.content === undefined && node.marks?.length !== 0 +} + export function identicalMark(left: AdfMark, right: AdfMark): boolean { return marksKey([left]) === marksKey([right]) } diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index 1a63f78..867dfc8 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -1,7 +1,7 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { Flavour } from '../plain-conventions.ts' -import { adfDocumentFault, carriesOnly, nodeAttrs, nodeContent } from '../../adf/document.ts' +import { adfDocumentFault, carriesOnly, isPlainText, nodeAttrs, nodeContent } from '../../adf/document.ts' import { alertMarker, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../plain-conventions.ts' import { blockDirectiveForm, documentSpelling, listBreakSpelling } from '../block-directive.ts' import { blockNodeModel, blockNodes } from '../../adf/block-nodes.ts' @@ -278,7 +278,7 @@ function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertEr // The text each fence holds, or `undefined` where a child is no plain text node, which the carry holds instead. function fencedTexts(node: AdfNode, path: ConvertErrorPath): Result { if (node.content === undefined) return success(['']) - if (node.content.some((child) => child.type !== 'text' || child.attrs !== undefined || child.content !== undefined || child.marks !== undefined)) return success(undefined) + if (node.content.some((child) => !isPlainText(child) || child.marks !== undefined)) return success(undefined) const texts: string[] = [] for (const [index, child] of node.content.entries()) { const childPath = [...path, 'content', index] diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index 5d4c3d2..6be52b0 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -6,7 +6,7 @@ import { assembleInlineLine, isSyntax, type InlineEscaping, type InlineSegment, import { carriedInline } from '../opaque-carry.ts' import { claimsLine, holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts' import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts' -import { emptyKeys, identicalMark, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' +import { identicalMark, isPlainText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { escapeUnbalanced, spellDestination } from '../commonmark/link-syntax.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { highlightDelimiter } from '../plain-conventions.ts' @@ -268,9 +268,9 @@ function emitInlineDirective(node: AdfNode, model: InlineNodeModel, index: numbe } function emitText(node: AdfNode, context: InlineContext, index: number, path: ConvertErrorPath): Result { - if (node.attrs !== undefined || emptyKeys(node).length > 0) return success({ carry: { first: index, last: index } }) - if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) if (nodeContent(node).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) + if (!isPlainText(node)) return success({ carry: { first: index, last: index } }) + if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) if (/\r/.test(node.text)) return failure('unspellable-character', 'a text node holds a carriage return CommonMark rewrites', path) if (holdsNullCharacter(node.text)) return failure('unspellable-character', 'a text node holds a null character CommonMark replaces', path) const escaping: InlineEscaping = context.bracketed ? 'bracketed' : 'backslash' @@ -323,10 +323,10 @@ function emitHighlight(nodes: readonly AdfNode[], depth: number, range: NodeRang function emitCodeSpan(nodes: readonly AdfNode[], depth: number, range: NodeRange, path: ConvertErrorPath): Result { const spans: string[] = [] for (const node of nodes) { - if (node.type !== 'text' || nodeMarks(node).length !== depth + 1 || node.attrs !== undefined || emptyKeys(node).length > 0) return success({ carry: range }) + if (node.type === 'text' && nodeContent(node).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) + if (!isPlainText(node) || nodeMarks(node).length !== depth + 1) return success({ carry: range }) const { text } = node if (typeof text !== 'string' || text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) - if (nodeContent(node).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) if (/[\n\r]/.test(text)) return success({ carry: range }) if (holdsNullCharacter(text)) return failure('unspellable-character', 'a code span holds a null character CommonMark replaces', path) const fence = '`'.repeat(longestBacktickRun(text) + 1) diff --git a/src/markdown/parse/directive-nodes.ts b/src/markdown/parse/directive-nodes.ts index c738fb1..3aab5db 100644 --- a/src/markdown/parse/directive-nodes.ts +++ b/src/markdown/parse/directive-nodes.ts @@ -3,7 +3,7 @@ import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { ConvertFault } from '../../result.ts' import type { DirectiveAttributes, DirectiveValue } from '../directive-syntax.ts' import type { Elsewhere } from './directive-attributes.ts' -import { attributeNestingMessage, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' +import { attributeNestingMessage, isPlainText } from '../../adf/document.ts' import { attributeValue, directivePrefix, spellAttributeValue, unknownDirectiveFault } from '../directive-syntax.ts' import { blockArgument, blockDirectiveForm, marksAttribute, readMarkValues } from '../block-directive.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' @@ -95,7 +95,7 @@ function blockSpellingFault(name: string): ConvertFault | undefined { function slotText(content: readonly AdfNode[]): string | undefined { if (content.length === 0) return '' const only = content.length === 1 ? content[0] : undefined - if (only?.type !== 'text' || nodeMarks(only).length > 0 || Object.keys(nodeAttrs(only)).length > 0 || nodeContent(only).length > 0 || typeof only.text !== 'string') return undefined + if (only === undefined || !isPlainText(only) || only.marks !== undefined || typeof only.text !== 'string') return undefined return only.text } diff --git a/src/markdown/text-break.ts b/src/markdown/text-break.ts index 994ae19..2055539 100644 --- a/src/markdown/text-break.ts +++ b/src/markdown/text-break.ts @@ -1,5 +1,5 @@ import type { AdfNode } from '../adf/document.ts' -import { identicalMarks, nodeMarks } from '../adf/document.ts' +import { identicalMarks, isPlainText, nodeMarks } from '../adf/document.ts' import { spellInlineLeafDirective } from './directive-syntax.ts' export const textBreakName = 'textBreak' @@ -8,9 +8,5 @@ export const textBreakSpelling = spellInlineLeafDirective(textBreakName, '') // Whether CommonMark reads the pair back as one text node, where neither rides the carry (spec/flavour.md, Inline nodes). export function readsAsOne(previous: AdfNode, node: AdfNode): boolean { - return spelledAsText(previous) && spelledAsText(node) && identicalMarks(nodeMarks(previous), nodeMarks(node)) -} - -function spelledAsText(node: AdfNode): boolean { - return node.type === 'text' && node.attrs === undefined && node.content === undefined && node.marks?.length !== 0 + return isPlainText(previous) && isPlainText(node) && identicalMarks(nodeMarks(previous), nodeMarks(node)) } -- 2.52.0 From 2d02165702aa7db2b04207a51e67d6e70945221f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:44:14 +0200 Subject: [PATCH 21/67] Keep the browser leg on dist/index.js: tag -0 in the JSON it hands back --- browser-tests/convert-corpus.js | 5 ++--- browser-tests/run.js | 11 ++++++++--- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/browser-tests/convert-corpus.js b/browser-tests/convert-corpus.js index 3b535bc..84bba51 100644 --- a/browser-tests/convert-corpus.js +++ b/browser-tests/convert-corpus.js @@ -1,8 +1,7 @@ try { const { adfToMarkdown, isAdfDocument, markdownToAdf } = await import('/dist/index.js') - const { serializeCanonicalJson } = await import('/dist/canonical-json.js') - // WebDriver's JSON reads -0 back as 0, so a document crosses as its canonical spelling. - const spelled = (result) => (result.ok ? { ok: true, value: `${serializeCanonicalJson(result.value, 'two-space')}\n` } : result) + // WebDriver's JSON reads -0 back as 0, so a document crosses as JSON text with -0 tagged; run.js revives it. + const spelled = (result) => (result.ok ? { ok: true, value: JSON.stringify(result.value, (_, value) => (Object.is(value, -0) ? '\u0000-0' : value)) } : result) window.convertCorpus = (corpus) => ({ errors: corpus.errors.map(({ markdown, name }) => ({ name, parsed: markdownToAdf(markdown) })), normalization: corpus.normalization.map(({ markdown, name }) => ({ name, parsed: spelled(markdownToAdf(markdown)) })), diff --git a/browser-tests/run.js b/browser-tests/run.js index 6739064..b0652df 100644 --- a/browser-tests/run.js +++ b/browser-tests/run.js @@ -49,6 +49,11 @@ function fixtureNames(kind, extension) { .sort() } +// convert-corpus.js tags -0 so it survives WebDriver's JSON. +function revived(text) { + return JSON.parse(text, (_, value) => (value === '\u0000-0' ? -0 : value)) +} + function refusal(result) { return result.ok ? '' : `${result.error.code}: ${result.error.message}` } @@ -106,7 +111,7 @@ for (const [index, { json, markdown, name }] of corpus.roundTrip.entries()) { assert.ok(result.emitted.ok, `it did not emit — ${refusal(result.emitted)}`) assert.equal(result.emitted.value, markdown) assert.ok(result.parsed.ok, `it did not parse — ${refusal(result.parsed)}`) - assert.equal(result.parsed.value, json) + assert.deepEqual(revived(result.parsed.value), JSON.parse(json)) }) } @@ -114,7 +119,7 @@ for (const [index, { name }] of corpus.normalization.entries()) { const result = results.normalization[index] checking(name, result, () => { assert.ok(result.parsed.ok, `it did not parse — ${refusal(result.parsed)}`) - assert.equal(result.parsed.value, fixture(name, '.json')) + assert.deepEqual(revived(result.parsed.value), JSON.parse(fixture(name, '.json'))) }) } @@ -124,7 +129,7 @@ for (const [index, { json, name }] of corpus.realPayloads.entries()) { assert.ok(result.isDocument, `${name}.json is no ADF document`) assert.ok(result.emitted.ok, `it did not emit — ${refusal(result.emitted)}`) assert.ok(result.parsed.ok, `it did not parse back — ${refusal(result.parsed)}`) - assert.equal(result.parsed.value, json) + assert.deepEqual(revived(result.parsed.value), JSON.parse(json)) }) } -- 2.52.0 From d9c3e4b3490eab323de83b677ee67387f49e36b4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:45:12 +0200 Subject: [PATCH 22/67] Derive a block directive's empty content from its node, and let readEmptyKeys own the attrs=empty-stands-alone rule --- src/markdown/empty-keys.ts | 4 +++- src/markdown/parse/directive-marks.ts | 4 ++-- src/markdown/parse/directive-nodes.ts | 21 ++++++++------------- src/markdown/parse/markdown-to-adf.ts | 5 +++-- 4 files changed, 16 insertions(+), 18 deletions(-) diff --git a/src/markdown/empty-keys.ts b/src/markdown/empty-keys.ts index b59008d..4bbbf97 100644 --- a/src/markdown/empty-keys.ts +++ b/src/markdown/empty-keys.ts @@ -16,7 +16,8 @@ export function spellsEmpty(value: DirectiveValue | undefined): boolean { return value?.spelling === emptyValue } -export function readEmptyKeys(attributes: DirectiveAttributes, keys: readonly EmptyKey[]): Read { +// `held` is whether the directive spells an attribute outside {attrs}: an argument or a content slot. +export function readEmptyKeys(type: string, attributes: DirectiveAttributes, keys: readonly EmptyKey[], held: boolean): Read { const rest = new Map(attributes) const empty = new Set() for (const key of keys) { @@ -26,5 +27,6 @@ export function readEmptyKeys(attributes: DirectiveAttributes, keys: readonly Em empty.add(key) rest.delete(key) } + if (empty.has('attrs') && (held || [...rest.keys()].some((key) => key !== 'marks'))) return { fault: unsupportedNodeShape(`${type} spells attrs=empty beside an attribute it holds`) } return { value: { empty, rest } } } diff --git a/src/markdown/parse/directive-marks.ts b/src/markdown/parse/directive-marks.ts index 5d42ecd..22e5c7a 100644 --- a/src/markdown/parse/directive-marks.ts +++ b/src/markdown/parse/directive-marks.ts @@ -13,11 +13,11 @@ export function readDirectiveMark(name: string, attributes: DirectiveAttributes, if (spelling === undefined) return undefined const markdown = markdownForm(spelling) if (markdown !== undefined) return failure('unsupported-node-shape', `${name} is spelled ${markdown}, never as a directive`, path) - const empty = readEmptyKeys(attributes, ['attrs']) + const empty = readEmptyKeys(name, attributes, ['attrs'], false) if (empty.fault !== undefined) return faulted(empty.fault, path) const attrs = readVocabulary(name, empty.value.rest, spelling.attributes, undefined, path) if (!attrs.ok) return attrs - if (empty.value.empty.has('attrs')) return Object.keys(attrs.value).length === 0 ? success({ attrs: {}, type: name }) : failure('unsupported-node-shape', `${name} spells attrs=empty beside an attribute it holds`, path) + if (empty.value.empty.has('attrs')) return success({ attrs: {}, type: name }) return success(Object.keys(attrs.value).length === 0 ? { type: name } : { attrs: attrs.value, type: name }) } diff --git a/src/markdown/parse/directive-nodes.ts b/src/markdown/parse/directive-nodes.ts index 3aab5db..63e1e56 100644 --- a/src/markdown/parse/directive-nodes.ts +++ b/src/markdown/parse/directive-nodes.ts @@ -17,8 +17,7 @@ import { slotLineEndingFault } from '../directive-syntax.ts' import { textBreakName } from '../text-break.ts' import { textDirectiveName } from '../text-directive.ts' -// `emptyContent` is whether the directive spells content=empty, which holds no body. -export type BlockDirectiveNode = { contentModel: BlockNodeModel['contentModel']; emptyContent: boolean; node: AdfNode } +export type BlockDirectiveNode = { contentModel: BlockNodeModel['contentModel']; node: AdfNode } export function readBlockDirectiveNode( name: string, @@ -33,7 +32,7 @@ export function readBlockDirectiveNode( if (model === undefined) return faulted(inlineSpellingFault(name) ?? unknownDirectiveFault(name), path) const argumentKey = blockArgument(name) const spelled = attributes.get(marksAttribute) - const empty = readEmptyKeys(attributes, spellsEmpty(spelled) ? ['attrs', 'content', 'marks'] : ['attrs', 'content']) + const empty = readEmptyKeys(name, attributes, spellsEmpty(spelled) ? ['attrs', 'content', 'marks'] : ['attrs', 'content'], argument !== undefined) if (empty.fault !== undefined) return faulted(empty.fault, path) const { rest } = empty.value rest.delete(marksAttribute) @@ -46,9 +45,7 @@ export function readBlockDirectiveNode( } const marks: Result = spelled === undefined || spellsEmpty(spelled) ? success(undefined) : readMarks(name, spelled, path) if (!marks.ok) return marks - const node = namedNode(name, attrs.value, marks.value, empty.value.empty, path) - if (!node.ok) return node - return success({ contentModel: model.contentModel, emptyContent: empty.value.empty.has('content'), node: node.value }) + return success({ contentModel: model.contentModel, node: namedNode(name, attrs.value, marks.value, empty.value.empty) }) } export function readInlineDirectiveNode( @@ -62,7 +59,7 @@ export function readInlineDirectiveNode( const slot = model.textAttribute if (slot === undefined && content !== undefined) return failure('unsupported-node-shape', `${name} takes no content: this one holds some`, path) const elsewhere: Elsewhere | undefined = slot === undefined ? undefined : { key: slot, slot: 'content' } - const empty = readEmptyKeys(attributes, ['attrs', 'content', 'marks']) + const empty = readEmptyKeys(name, attributes, ['attrs', 'content', 'marks'], slot !== undefined && content !== undefined) if (empty.fault !== undefined) return faulted(empty.fault, path) const attrs = readVocabulary(name, empty.value.rest, model.attributes, elsewhere, path) if (!attrs.ok) return attrs @@ -75,7 +72,7 @@ export function readInlineDirectiveNode( if (spans !== undefined) return faulted(spans, path) attrs.value[slot] = text } - return namedNode(name, attrs.value, undefined, empty.value.empty, path) + return success(namedNode(name, attrs.value, undefined, empty.value.empty)) } // A name the other position spells names that spelling, never the code a later MINOR may fill (docs/decisions.md §Which code a cause takes). @@ -109,11 +106,9 @@ function readMarks(type: string, spelled: DirectiveValue, path: ConvertErrorPath return success(marks) } -function namedNode(type: string, attrs: AdfAttributes, marks: readonly AdfMark[] | undefined, empty: ReadonlySet, path: ConvertErrorPath): Result { - const held = Object.keys(attrs).length > 0 - if (held && empty.has('attrs')) return failure('unsupported-node-shape', `${type} spells attrs=empty beside an attribute it holds`, path) - const node: AdfNode = held || empty.has('attrs') ? { attrs, type } : { type } +function namedNode(type: string, attrs: AdfAttributes, marks: readonly AdfMark[] | undefined, empty: ReadonlySet): AdfNode { + const node: AdfNode = Object.keys(attrs).length > 0 || empty.has('attrs') ? { attrs, type } : { type } if (empty.has('content')) node.content = [] if (marks !== undefined || empty.has('marks')) node.marks = [...(marks ?? [])] - return success(node) + return node } diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 55c42f0..7521fbc 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -229,9 +229,10 @@ function directiveNode(block: DirectiveBlock, reading: Reading, path: ConvertErr } function directiveBody(read: BlockDirectiveNode, blocks: Block[] | undefined, reading: Reading, path: ConvertErrorPath, depth: number): Result { - const { contentModel, emptyContent, node } = read + const { contentModel, node } = read if (blocks === undefined) return success(node) - if (emptyContent) return blocks.length === 0 ? success(node) : failure('unsupported-node-shape', `${node.type} spells content=empty, which holds no body: this one holds one`, path) + // A directive builds a content key only from content=empty, which holds no body. + if (node.content !== undefined) return blocks.length === 0 ? success(node) : failure('unsupported-node-shape', `${node.type} spells content=empty, which holds no body: this one holds one`, path) if (contentModel === 'code') return codeDirectiveNode(node, blocks, path) if (contentModel === 'inline') return inlineBodyNode(node, blocks, reading, path) return containerNode(node, blocks, reading, path, depth) -- 2.52.0 From 2700184e56683b4eca053d907aa5573865b04e8a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:45:28 +0200 Subject: [PATCH 23/67] Spell the doc directive's content=none through spellAttributes from one constant --- src/markdown/block-directive.ts | 6 ++++-- src/markdown/parse/markdown-to-adf.ts | 6 +++--- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/src/markdown/block-directive.ts b/src/markdown/block-directive.ts index dc305bb..00f4643 100644 --- a/src/markdown/block-directive.ts +++ b/src/markdown/block-directive.ts @@ -4,7 +4,7 @@ import type { JsonValue } from '../json-value.ts' import { blockNodeModel } from '../adf/block-nodes.ts' import { isAdfMark } from '../adf/document.ts' import { serializeCanonicalJson } from '../canonical-json.ts' -import { spellDirectiveOpener } from './directive-syntax.ts' +import { spellAttributes, spellDirectiveOpener } from './directive-syntax.ts' const argumentByType = new Map( Object.entries({ @@ -17,7 +17,9 @@ const argumentByType = new Map( export const documentName = 'doc' // spec/flavour.md, Directives: a document holding no content key, which the empty string cannot spell. -export const documentSpelling = spellDirectiveOpener(documentName, undefined, '{content=none}') +export const documentAttribute = { key: 'content', value: 'none' } as const + +export const documentSpelling = spellDirectiveOpener(documentName, undefined, spellAttributes([[documentAttribute.key, documentAttribute.value]])) export const listBreakName = 'listBreak' diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 7521fbc..40a0742 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -7,7 +7,7 @@ import type { LineContainer } from '../line-container.ts' import type { LinkDefinitions } from './inline-content.ts' import { carryFencePrefix, readCarriedBlock } from '../opaque-carry.ts' import { commonMarkSpelling, type SpellingMemo } from '../emit/adf-to-markdown.ts' -import { documentName, documentSpelling, listBreakName, listBreakSpelling } from '../block-directive.ts' +import { documentAttribute, documentName, documentSpelling, listBreakName, listBreakSpelling } from '../block-directive.ts' import { failure, faulted, positioned, success, type ConvertErrorPath, type ParseError, type Result, type SourcePosition } from '../../result.ts' import { inlineLeaves } from '../emit/plain-inline.ts' import { languageSlot } from '../code-language.ts' @@ -83,8 +83,8 @@ function listBreakFault(block: DirectiveBlock, previous: Block | undefined, next } function documentFault(block: DirectiveBlock): ConvertFault | undefined { - const content = block.attributes.get('content') - if (block.argument === undefined && block.attributes.size === 1 && content?.spelling === 'none') return undefined + const spelled = block.attributes.get(documentAttribute.key) + if (block.argument === undefined && block.attributes.size === 1 && spelled?.spelling === documentAttribute.value) return undefined return unsupportedNodeShape(`${documentName} spells the one form ${documentSpelling}: this one spells another`) } -- 2.52.0 From c3bc2eb5728552614651f9c42731d40c2384634f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:45:48 +0200 Subject: [PATCH 24/67] Read a carry fence's type in one function --- src/markdown/code-language.ts | 4 ++-- src/markdown/opaque-carry.ts | 5 +++++ src/markdown/parse/markdown-to-adf.ts | 7 ++++--- 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/src/markdown/code-language.ts b/src/markdown/code-language.ts index 75ddf83..846b64e 100644 --- a/src/markdown/code-language.ts +++ b/src/markdown/code-language.ts @@ -1,5 +1,5 @@ import type { JsonValue } from '../json-value.ts' -import { carryFencePrefix } from './opaque-carry.ts' +import { carryFenceType } from './opaque-carry.ts' import { infoStringCarries } from './commonmark/grammar.ts' export type LanguageSlot = { info: string; kind: 'fence' } | { kind: 'attribute' } | { kind: 'none' } @@ -7,6 +7,6 @@ export type LanguageSlot = { info: string; kind: 'fence' } | { kind: 'attribute' // spec/flavour.md, The CommonMark blocks: the one slot a codeBlock's language rides. export function languageSlot(language: JsonValue | undefined): LanguageSlot { if (language === undefined) return { kind: 'none' } - if (typeof language !== 'string' || language.startsWith(carryFencePrefix) || !infoStringCarries(language)) return { kind: 'attribute' } + if (typeof language !== 'string' || carryFenceType(language) !== undefined || !infoStringCarries(language)) return { kind: 'attribute' } return { info: language, kind: 'fence' } } diff --git a/src/markdown/opaque-carry.ts b/src/markdown/opaque-carry.ts index f15960f..90bd538 100644 --- a/src/markdown/opaque-carry.ts +++ b/src/markdown/opaque-carry.ts @@ -32,6 +32,11 @@ export function carriedInline(node: AdfNode, path: ConvertErrorPath): Result { if (type !== '' && !infoStringCarries(type)) { return { fault: unsupportedNodeShape(`the carry fence names a type no info string carries back: spell it ${carryFencePrefix} with the type in the JSON`) } diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 40a0742..161fdfb 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -5,7 +5,7 @@ import type { ConvertFault } from '../../result.ts' import type { Flavour } from '../plain-conventions.ts' import type { LineContainer } from '../line-container.ts' import type { LinkDefinitions } from './inline-content.ts' -import { carryFencePrefix, readCarriedBlock } from '../opaque-carry.ts' +import { carryFenceType, readCarriedBlock } from '../opaque-carry.ts' import { commonMarkSpelling, type SpellingMemo } from '../emit/adf-to-markdown.ts' import { documentAttribute, documentName, documentSpelling, listBreakName, listBreakSpelling } from '../block-directive.ts' import { failure, faulted, positioned, success, type ConvertErrorPath, type ParseError, type Result, type SourcePosition } from '../../result.ts' @@ -303,8 +303,9 @@ function listNode(node: AdfNode, items: readonly Block[][], reading: Reading, pa } function codeBlockNode(language: string, text: string, reading: Reading, path: ConvertErrorPath, depth: number): Result { - if (language.startsWith(carryFencePrefix)) { - const carried = readCarriedBlock(language.slice(carryFencePrefix.length), text, depth) + const carriedType = carryFenceType(language) + if (carriedType !== undefined) { + const carried = readCarriedBlock(carriedType, text, depth) if (carried.fault !== undefined) return faulted(carried.fault, path) reading.carried.add(carried.value) return success(carried.value) -- 2.52.0 From bba2fa409efa7ec25da162c858060510ccc0f1de Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:46:29 +0200 Subject: [PATCH 25/67] Drop empty keys in the property harness through emptyKeys --- src/conformance/property-harness.ts | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/src/conformance/property-harness.ts b/src/conformance/property-harness.ts index 6afc116..d38579e 100644 --- a/src/conformance/property-harness.ts +++ b/src/conformance/property-harness.ts @@ -2,17 +2,17 @@ import assert from 'node:assert/strict' import { env } from 'node:process' import fc from 'fast-check' -import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../adf/document.ts' +import type { AdfAttributes, AdfDocument, AdfMark, AdfNode, EmptyKey } from '../adf/document.ts' import type { Arbitrary } from 'fast-check' import type { AttributeKind, AttributeVocabulary } from '../adf/attribute-vocabulary.ts' import type { JsonValue } from '../json-value.ts' import { blockArgument } from '../markdown/block-directive.ts' import { blockNodes } from '../adf/block-nodes.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' +import { emptyKeys, mergeAdjacentText } from '../adf/document.ts' import { inlineNodes } from '../adf/inline-nodes.ts' import { joinsNormally } from '../adf/editor-normal.ts' import { markAttributes } from '../adf/mark-attributes.ts' -import { mergeAdjacentText } from '../adf/document.ts' type Positions = { block: AdfNode; inline: AdfNode } @@ -85,10 +85,8 @@ function occasionallyEmpty(arbitrary: Arbitrary) } function withoutEmptyKeys(held: T): T { - const kept = { ...held } - if (kept.attrs !== undefined && Object.keys(kept.attrs).length === 0) delete kept.attrs - if ('content' in kept && kept.content?.length === 0) delete kept.content - if ('marks' in kept && kept.marks?.length === 0) delete kept.marks + const kept: Partial> & T = { ...held } + for (const key of emptyKeys(held)) delete kept[key] return kept } -- 2.52.0 From d7ba8e6aaec2de1513b206278e359137d6009c18 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:46:29 +0200 Subject: [PATCH 26/67] Keep format knowledge out of mergeAdjacentText's comment --- src/adf/document.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/adf/document.ts b/src/adf/document.ts index 1ae004c..61afdc7 100644 --- a/src/adf/document.ts +++ b/src/adf/document.ts @@ -77,7 +77,7 @@ export function identicalMarks(left: readonly AdfMark[], right: readonly AdfMark return marksKey(left) === marksKey(right) } -// `joins` is the caller's: a reader joins what CommonMark reads as one, editor-normal ADF what the editor would. +// Callers say when two nodes join. export function mergeAdjacentText(nodes: readonly AdfNode[], joins: (previous: AdfNode, node: AdfNode) => boolean): AdfNode[] { const merged: AdfNode[] = [] for (const node of nodes) { -- 2.52.0 From b9760d0af2dc0a0ad84e80580f1f7b0c7532517d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:46:29 +0200 Subject: [PATCH 27/67] Name plainMarkdownToAdf in the empty-markdown changelog entry --- CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a02d444..a591365 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,8 +8,8 @@ fence whose info string `adf:` names the node's type, its body the node's JSON without `type`: text holding an unescaped `!adf:` and a code fence whose info string opens `adf:` are claimed, and `adf` is an ordinary code block language. Convert stored markdown per `MIGRATION.md`. -- **Breaking:** `markdownToAdf` reads markdown holding no block as a document whose `content` is - empty, as Atlassian's schema requires; `!adf:doc {content=none}` spells a document holding no +- **Breaking:** `markdownToAdf` and `plainMarkdownToAdf` read markdown holding no block as a + document whose `content` is empty, as Atlassian's schema requires; `!adf:doc {content=none}` spells a document holding no `content` key. - `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc`: two adjacent text nodes a reader would join are parted by `!adf:textBreak{}`, an empty `attrs`, `content` or `marks` is spelled -- 2.52.0 From 423dcbbff5b23a13e22ccb2fc972a2121eae29e6 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 14:47:57 +0200 Subject: [PATCH 28/67] File the audits' pre-existing findings, and mark item 43 cited by decisions --- todo.md | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/todo.md b/todo.md index fba68d1..e4bce9a 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 55` +`Next ID = 59` | Goal | W | |---|---| @@ -25,9 +25,10 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| | 7 | 0.2.0 | | **Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF.** | 6 | 9 | 9 | 9 | 2, 3 | 25.8 | +| 55 | 0.2.0 | defect | **Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start or a newline inside an emoji, mention or status.** | 5 | 5 | 7 | 9 | 1 | 25.8 | | 45 | 0.2.0 | | **Replace `isAdfDocument` with a reader returning `Result`.** | 2 | 3 | 6 | 8 | 1 | 25.2 | | 6 | 0.2.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 3 | 24.7 | -| 43 | 0.2.0 | | **Give each markdown input its own reader, strict to its own standard.** | 6 | 7 | 8 | 9 | 3, 4 | 22.3 | +| 43 | 0.2.0 | decision | **Give each markdown input its own reader, strict to its own standard.** | 6 | 7 | 8 | 9 | 3, 4 | 22.3 | | 49 | 0.2.0 | | **Read a list whose bullet or ordered delimiter changes as two lists in the CommonMark reader.** | 4 | 5 | 5 | 8 | 3, 4 | 17.2 | | 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 5 | 13.0 | | 51 | 0.2.0 | | **Match a reference label to its definition under Unicode case folding.** | 2 | 2 | 2 | 7 | 3, 4 | 12.4 | @@ -43,6 +44,9 @@ | 33 | 0.3.0 | | **Emit a line in time linear in its mark runs, in `adfToMarkdown` and `adfToPlainMarkdown`.** | 4 | 5 | 6 | 9 | 8 | 10.7 | | 46 | 0.3.0 | | **Publish the bundle size in the README, failing the release pipeline when it drifts.** | 2 | 4 | 5 | 5 | 7, 9 | 10.3 | | 9 | 0.3.0 | | **Ship an online sandbox: a web page with two textboxes converting between ADF and markdown on the library's browser build.** | 2 | 6 | 6 | 8 | 9 | 10.3 | +| 56 | 0.3.0 | principle | **Give each piece of `blocks.ts`'s block-walk state one owner that returns what it changes.** | 4 | 5 | 2 | 4 | 1 | 6.8 | +| 57 | 0.3.0 | principle | **Make each `ci.sh` leg build what it reads, so one leg run alone tests the current tree.** | 2 | 3 | 1 | 3 | 7 | 1.2 | +| 58 | 0.3.0 | principle | **Port `ci.sh`, `publish.sh` and `docker-runner.sh` to standalone Python scripts.** | 4 | 6 | 1 | 2 | 7 | -2.2 | | 8 | 0.4.0 | | **Ship a CLI.** | 3 | 7 | 7 | 6 | 9 | 10.6 | ## Details @@ -53,6 +57,16 @@ Lands after item 6. The CommonMark spec suite also runs against `markdownToHtml` documents HTML as it documents markdown, and its tagline and `package.json`'s `description` regain HTML. +### 55. Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start or a newline inside an emoji, mention or status. + +`inline-line.ts` refuses a carriage return or NUL in text (`unspellable-character`) and a paragraph +opening with a code-span run (`unspellable-line-start`); `fencedTexts` refuses the same characters +in a code block; `directive-syntax.ts` refuses a newline in an emoji, mention or status +(`unspellable-whitespace`). The inline carry's JSON escapes all of them, so a lossless spelling +exists, and §The code list says a cause the carry answers gets no code. Fixing it revises §Which +code a cause takes and §Markdown in is a canonical fixpoint, which the maintainer decides. Found by +the README-goals audit, 2026-10-03. + ### 45. Replace `isAdfDocument` with a reader returning `Result`. Goal 1 has every call return a result; the boolean guard is the one export that does not, and it @@ -185,6 +199,23 @@ or report the unminified gzip). The figure lands in README §The package beside dependencies" claim. Measured today, unminified: tarball 60.4 kB, unpacked 221.5 kB, JS gzipped 45.6 kB. +### 56. Give each piece of `blocks.ts`'s block-walk state one owner that returns what it changes. + +Technical principle "One owner per value": the `Walk` record passes through twelve functions that +mutate it and return `void`; `walk.leaf` alone is written in seven places. `ContainerStack` in the +same file shows the shape to follow. + +### 57. Make each `ci.sh` leg build what it reads, so one leg run alone tests the current tree. + +Technical principle "Compose, do not entangle": Deno, Bun and the tests read the install leg's +`node_modules`, the floor and consumer legs the pack leg's, and the browser leg whatever `dist/` the +build leg left, so a leg run alone can pass on stale output. + +### 58. Port `ci.sh`, `publish.sh` and `docker-runner.sh` to standalone Python scripts. + +Technical principle "Prefer a standalone Python script over a shell script": `AGENTS.md` §3 teaches +bash footguns (`&&` chaining under `||`, `tee /dev/stderr`) the port removes. + ### 8. Ship a CLI. The Goals and G cells are provisional: no README goal or persona covers a CLI yet. The chunk -- 2.52.0 From 6fc0d6bba92e23956c4512c58f989a9c14d16ecf Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:03:58 +0200 Subject: [PATCH 29/67] Refuse attrs=empty on a code block whose fences carry its language --- corpus/errors/code-block-empty-attrs-fence-language.error | 1 + corpus/errors/code-block-empty-attrs-fence-language.md | 8 ++++++++ src/markdown/parse/markdown-to-adf.ts | 3 ++- 3 files changed, 11 insertions(+), 1 deletion(-) create mode 100644 corpus/errors/code-block-empty-attrs-fence-language.error create mode 100644 corpus/errors/code-block-empty-attrs-fence-language.md diff --git a/corpus/errors/code-block-empty-attrs-fence-language.error b/corpus/errors/code-block-empty-attrs-fence-language.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/code-block-empty-attrs-fence-language.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/code-block-empty-attrs-fence-language.md b/corpus/errors/code-block-empty-attrs-fence-language.md new file mode 100644 index 0000000..75c0abb --- /dev/null +++ b/corpus/errors/code-block-empty-attrs-fence-language.md @@ -0,0 +1,8 @@ +!adf:codeBlock {attrs=empty} +```js +a +``` +```js +b +``` +!adf:/codeBlock diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 161fdfb..5a6bdc3 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -8,13 +8,13 @@ import type { LinkDefinitions } from './inline-content.ts' import { carryFenceType, readCarriedBlock } from '../opaque-carry.ts' import { commonMarkSpelling, type SpellingMemo } from '../emit/adf-to-markdown.ts' import { documentAttribute, documentName, documentSpelling, listBreakName, listBreakSpelling } from '../block-directive.ts' +import { emptyKeys, nodeAttrs, nodeContent } from '../../adf/document.ts' import { failure, faulted, positioned, success, type ConvertErrorPath, type ParseError, type Result, type SourcePosition } from '../../result.ts' import { inlineLeaves } from '../emit/plain-inline.ts' import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' import { leadingMarker, readAlertMarker, readTaskMarker } from '../plain-conventions.ts' import { mintTaskIds } from './task-ids.ts' -import { nodeAttrs, nodeContent } from '../../adf/document.ts' import { parseBlocks } from './blocks.ts' import { parseInlineContent } from './inline-content.ts' import { readBlockDirectiveNode } from './directive-nodes.ts' @@ -255,6 +255,7 @@ function codeDirectiveNode(node: AdfNode, blocks: readonly Block[], path: Conver if ((slot.kind === 'fence') !== fromFence || (fromFence && attribute !== undefined)) { return failure('unsupported-node-shape', `${node.type} spells its language in the fence info string, or in the attribute where no info string carries it back`, path) } + if (fromFence && emptyKeys(node).includes('attrs')) return failure('unsupported-node-shape', `${node.type} spells attrs=empty beside an attribute it holds`, path) const spelled = fromFence ? { ...node, attrs: { ...node.attrs, language: first.language } } : node return success(withContent(spelled, first.text === '' ? [] : fences.map((fence): AdfNode => ({ text: fence.text, type: 'text' })))) } -- 2.52.0 From 0ac5dccf11742df13794efa51ee2546e061caa83 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:04:04 +0200 Subject: [PATCH 30/67] Pin the empty fence beside another as an error fixture --- corpus/errors/code-block-empty-fence.error | 1 + corpus/errors/code-block-empty-fence.md | 7 +++++++ 2 files changed, 8 insertions(+) create mode 100644 corpus/errors/code-block-empty-fence.error create mode 100644 corpus/errors/code-block-empty-fence.md diff --git a/corpus/errors/code-block-empty-fence.error b/corpus/errors/code-block-empty-fence.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/code-block-empty-fence.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/code-block-empty-fence.md b/corpus/errors/code-block-empty-fence.md new file mode 100644 index 0000000..481ca7a --- /dev/null +++ b/corpus/errors/code-block-empty-fence.md @@ -0,0 +1,7 @@ +!adf:codeBlock +``` +a +``` +``` +``` +!adf:/codeBlock -- 2.52.0 From 5a7279059defa618a2bb9af3af5b0b5ba1a8b8aa Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:04:11 +0200 Subject: [PATCH 31/67] Drop the as const on the doc directive's attribute --- src/markdown/block-directive.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/markdown/block-directive.ts b/src/markdown/block-directive.ts index 00f4643..83ff2a0 100644 --- a/src/markdown/block-directive.ts +++ b/src/markdown/block-directive.ts @@ -17,7 +17,7 @@ const argumentByType = new Map( export const documentName = 'doc' // spec/flavour.md, Directives: a document holding no content key, which the empty string cannot spell. -export const documentAttribute = { key: 'content', value: 'none' } as const +export const documentAttribute = { key: 'content', value: 'none' } export const documentSpelling = spellDirectiveOpener(documentName, undefined, spellAttributes([[documentAttribute.key, documentAttribute.value]])) -- 2.52.0 From ff9bb7fab2f15397e47a90d89ad90f80044ca8e9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:04:15 +0200 Subject: [PATCH 32/67] Inline the identity blocks() helper in the plain reader tests --- .../parse/plain-markdown-to-adf.test.ts | 30 ++++++++----------- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git a/src/markdown/parse/plain-markdown-to-adf.test.ts b/src/markdown/parse/plain-markdown-to-adf.test.ts index e6144c6..ae1f96d 100644 --- a/src/markdown/parse/plain-markdown-to-adf.test.ts +++ b/src/markdown/parse/plain-markdown-to-adf.test.ts @@ -30,10 +30,6 @@ function read(markdown: string): readonly AdfNode[] | string { return blocks } -function blocks(...content: AdfNode[]): readonly AdfNode[] { - return content -} - function document(...content: AdfNode[]): AdfDocument { return { content, type: 'doc', version: 1 } } @@ -100,7 +96,7 @@ test('reads the rest of an alert marker line as the panel first body paragraph, assert.deepEqual(read('> [!tip] Title\n> body\n'), [panel('tip', said('Title'), said('body'))]) assert.deepEqual(read('> [!tip] Title\\\n> body\n'), [panel('tip', said('Title'), said('body'))]) assert.deepEqual(read('> [!NOTE]\\\n> Broken.\n'), [panel('info', said('Broken.'))]) - assert.deepEqual(read('> [!NOTE]\n'), blocks(panel('info', paragraph()))) + assert.deepEqual(read('> [!NOTE]\n'), [panel('info', paragraph())]) assert.deepEqual(read('> > [!WARNING]\n> > Inner.\n'), [bare('blockquote', panel('warning', said('Inner.')))]) }) @@ -119,27 +115,27 @@ test('reads a folded callout to an expand titled by the rest of its marker line, node('expand', { title: 'Why?' }, paragraph(text('See '), text('the docs', link), text(', '), text('now', strong), text('.'))), ]) assert.deepEqual(read('> [!NOTE]- Two\\\n> lines\n'), [node('expand', { title: 'Two' }, said('lines'))]) - assert.deepEqual(read('> [!faq]- See [x](http://y)\n'), blocks(node('expand', { title: 'See x (http://y)' }, paragraph()))) - assert.deepEqual(read('> [!faq]- [a **b**](u "t")[c](u) and [d](v)\n'), blocks(node('expand', { title: 'a bc (u) and d (v)' }, paragraph()))) - assert.deepEqual(read('> [!faq]- or or \n'), blocks(node('expand', { title: 'http://y or a@b.c or http://a\\b' }, paragraph()))) - assert.deepEqual(read('> [!faq]- [a b](u)\n'), blocks(node('expand', { title: 'a\nb (u)' }, paragraph()))) - assert.deepEqual(read('> [!faq]- [!adf:mention[@M]{id=5}](u) !adf:inlineCard{url="http://y"}\n'), blocks(node('expand', { title: '@M (u) http://y' }, paragraph()))) - assert.deepEqual(read('> [!NOTE]- Set ==x== here\n'), blocks(node('expand', { title: 'Set ==x== here' }, paragraph()))) + assert.deepEqual(read('> [!faq]- See [x](http://y)\n'), [node('expand', { title: 'See x (http://y)' }, paragraph())]) + assert.deepEqual(read('> [!faq]- [a **b**](u "t")[c](u) and [d](v)\n'), [node('expand', { title: 'a bc (u) and d (v)' }, paragraph())]) + assert.deepEqual(read('> [!faq]- or or \n'), [node('expand', { title: 'http://y or a@b.c or http://a\\b' }, paragraph())]) + assert.deepEqual(read('> [!faq]- [a b](u)\n'), [node('expand', { title: 'a\nb (u)' }, paragraph())]) + assert.deepEqual(read('> [!faq]- [!adf:mention[@M]{id=5}](u) !adf:inlineCard{url="http://y"}\n'), [node('expand', { title: '@M (u) http://y' }, paragraph())]) + assert.deepEqual(read('> [!NOTE]- Set ==x== here\n'), [node('expand', { title: 'Set ==x== here' }, paragraph())]) assert.deepEqual(read('> [!NOTE]-\n>\n> Line.\n'), [bare('expand', said('Line.'))]) }) test('reads a folded callout inside an expand to a nested expand', () => { const markdown = '> [!NOTE]- Outer\n>\n> > [!NOTE]- Inner\n> >\n> > Deep.\n>\n> > [!TIP]\n> >\n> > > [!NOTE]-\n' - assert.deepEqual(read(markdown), blocks(node('expand', { title: 'Outer' }, node('nestedExpand', { title: 'Inner' }, said('Deep.')), panel('tip', bare('nestedExpand', paragraph()))))) - assert.deepEqual(read('- > [!NOTE]-\n'), blocks(bare('bulletList', bare('listItem', bare('expand', paragraph()))))) - assert.deepEqual(read('!adf:expand\n> [!NOTE]- Inner\n!adf:/expand\n'), blocks(bare('expand', node('nestedExpand', { title: 'Inner' }, paragraph())))) + assert.deepEqual(read(markdown), [node('expand', { title: 'Outer' }, node('nestedExpand', { title: 'Inner' }, said('Deep.')), panel('tip', bare('nestedExpand', paragraph())))]) + assert.deepEqual(read('- > [!NOTE]-\n'), [bare('bulletList', bare('listItem', bare('expand', paragraph())))]) + assert.deepEqual(read('!adf:expand\n> [!NOTE]- Inner\n!adf:/expand\n'), [bare('expand', node('nestedExpand', { title: 'Inner' }, paragraph()))]) }) test('reads a bullet list whose every item leads with a task marker to a task list', () => { assert.deepEqual(read('- [x] Write the spec\n- [ ] Ship **it**\n- [X] Tell\n'), [ bare('taskList', task('DONE', text('Write the spec')), task('TODO', text('Ship '), text('it', strong)), task('DONE', text('Tell'))), ]) - assert.deepEqual(read('- [x]\n- [ ]\\\n after\n'), blocks(bare('taskList', task('DONE'), task('TODO', text('after'))))) + assert.deepEqual(read('- [x]\n- [ ]\\\n after\n'), [bare('taskList', task('DONE'), task('TODO', text('after')))]) const minted = plainMarkdownToAdf('- [x] Parent\n - [ ] Child\n') assert.deepEqual(minted.ok ? minted.value.content : minted.error.code, [ node( @@ -201,7 +197,7 @@ test('reads what the reduction wrote back to the node it reduced, less the attri assert.deepEqual(roundTripped(node('panel', { localId, panelType }, said('Check.'))), [panel(panelType, said('Check.'))], panelType) } const expand = node('expand', { localId, title: 'Log' }, said('Line.'), node('nestedExpand', { title: 'Inner' }, said('Deep.'))) - assert.deepEqual(roundTripped(node('panel', { panelType: 'tip' }, paragraph()), node('expand', { title: 'Empty' }, paragraph())), blocks(panel('tip', paragraph()), node('expand', { title: 'Empty' }, paragraph()))) + assert.deepEqual(roundTripped(node('panel', { panelType: 'tip' }, paragraph()), node('expand', { title: 'Empty' }, paragraph())), [panel('tip', paragraph()), node('expand', { title: 'Empty' }, paragraph())]) assert.deepEqual(roundTripped(expand), [node('expand', { title: 'Log' }, said('Line.'), node('nestedExpand', { title: 'Inner' }, said('Deep.')))]) const tasks = bare( 'taskList', @@ -268,5 +264,5 @@ test('keeps what markdownToAdf reads that no row reads, and refuses only what it test('joins a carried text node to no neighbour, inside a highlight too', () => { const carried = '!adf:carry{json="{\\"marks\\":[],\\"text\\":\\"b\\",\\"type\\":\\"text\\"}"}' - assert.deepEqual(read(`==a${carried}==\n`), blocks(paragraph(text('a', highlight), { marks: [], text: 'b', type: 'text' }))) + assert.deepEqual(read(`==a${carried}==\n`), [paragraph(text('a', highlight), { marks: [], text: 'b', type: 'text' })]) }) -- 2.52.0 From 4d7e8c944c04008a83a59ddf8bc619cf5429c865 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:04:35 +0200 Subject: [PATCH 33/67] Pin a code block holding content=empty beside its language --- corpus/round-trip/block-nodes/empty-keys.json | 7 +++++++ corpus/round-trip/block-nodes/empty-keys.md | 3 +++ 2 files changed, 10 insertions(+) diff --git a/corpus/round-trip/block-nodes/empty-keys.json b/corpus/round-trip/block-nodes/empty-keys.json index 3fcc6d2..725bfee 100644 --- a/corpus/round-trip/block-nodes/empty-keys.json +++ b/corpus/round-trip/block-nodes/empty-keys.json @@ -82,6 +82,13 @@ }, "marks": [], "type": "codeBlock" + }, + { + "attrs": { + "language": "js" + }, + "content": [], + "type": "codeBlock" } ], "type": "doc", diff --git a/corpus/round-trip/block-nodes/empty-keys.md b/corpus/round-trip/block-nodes/empty-keys.md index 04299da..cad1dfd 100644 --- a/corpus/round-trip/block-nodes/empty-keys.md +++ b/corpus/round-trip/block-nodes/empty-keys.md @@ -30,3 +30,6 @@ Next ```js ``` !adf:/codeBlock + +!adf:codeBlock {content=empty language=js} +!adf:/codeBlock -- 2.52.0 From 419bbcb4f2bad3e4c61794643c813d3a726d8540 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:04:44 +0200 Subject: [PATCH 34/67] Say equality is deep over the document's JSON values --- README.md | 6 +++--- docs/decisions.md | 6 ++++-- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 1b2a592..e14b29a 100644 --- a/README.md +++ b/README.md @@ -204,9 +204,9 @@ emit refuses: Serves Goals 1, 3 and 4. -- `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc` — every key and value as `doc` holds it, - adjacent text nodes, an empty `attrs`, `content` or `marks` and `-0` included, and unknown node - types carried opaquely +- `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc` as JSON, for a document of plain objects + as `JSON.parse` builds them — every key and value as `doc` holds it, adjacent text nodes, an + empty `attrs`, `content` or `marks` and `-0` included, and unknown node types carried opaquely ([`docs/decisions.md`](https://gitea.larvit.se/larvit/adf-codec/src/branch/main/docs/decisions.md#unknown-nodes-ride-the-carry)). - Markdown this library reads, and markdown it writes, means what the CommonMark spec says; from `0.2.0`, well-formed HTML means what the HTML standard says, read or written. The bullets below diff --git a/docs/decisions.md b/docs/decisions.md index 38bea5b..0aab2e4 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -36,8 +36,10 @@ does not imply a spellable document; `corpus/commonmark-spec/exceptions.json` na 2026-08-24, deep 2026-10-03, the maintainer. Goal 1. Valid while a pipeline or a bot can build a shape the editor would not. -"Equals" is `assert.deepStrictEqual`: every key and value `doc` holds, two adjacent text nodes, an -empty `attrs`, `content` or `marks` and `-0` included, with no normalization on either side. +"Equals" is deep equality over the document's JSON values: `assert.deepStrictEqual` on plain +objects as `JSON.parse` builds them, since ADF is JSON. Every key and value `doc` holds counts, two +adjacent text nodes, an empty `attrs`, `content` or `marks` and `-0` included, with no +normalization on either side. CommonMark's spelling stays wherever a document holds none of those shapes. The plain reader builds what is written, as `markdownToAdf` does. Only the plain writer is lossy: its reduction reads and writes editor-normal ADF — adjacent text nodes of identical marks and no attributes -- 2.52.0 From d9f2d325cafc4b3dcd9b7c36be72e22defa9164e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:04:50 +0200 Subject: [PATCH 35/67] Tell migrators an empty stored document gets content: [], and what an adf: fence refuses --- MIGRATION.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/MIGRATION.md b/MIGRATION.md index 2dc0d2b..f9b9b49 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -6,7 +6,8 @@ Directives moved under the `!adf:` prefix. `0.2.0` reads `0.1.0`'s spelling with turning each directive into text and each carried node into an `adf` code block. Before `0.2.0` reads any `0.1.0` markdown, convert what is stored or in flight (an open editor, a queue) with the recipe below, and rewrite markdown your code writes or matches (templates, prompts, patterns) by -the tables below. Stored ADF needs no change. +the tables below. Stored ADF needs one change: a document `0.1.0` read from empty markdown holds +no `content` key, and gets `content: []`, the empty document it meant. ### Convert markdown @@ -30,8 +31,9 @@ function migrateMarkdown(stored: string) { - Convert each document once: a second pass can return ok while turning the directives into text. Stop `0.1.0` writing first, and record which documents are converted. - A refusal carrying `position` is `0.1.0`'s parse, which refused that markdown before too. One - without is `0.2.0`'s emit: store the document `markdownToAdf010` read as ADF rather than keeping - the unconverted markdown. + without is `0.2.0`'s emit: store the document `markdownToAdf010` read as ADF, with + `content: parsed.value.content ?? []` as the recipe gives it, rather than keeping the unconverted + markdown. ### Spellings @@ -56,7 +58,7 @@ Markdown the spelling table leaves alone, which `0.2.0` reads as a different doc | --- | --- | --- | | a link whose text already holds one (`[ab](/v)`) | marks every node the inner link does not, splitting the outer link around it | leaves the outer brackets literal text; write the pieces as separate links to keep them | | markdown holding no block (`markdownToAdf("")`) | `{ type: 'doc', version: 1 }` | `{ content: [], type: 'doc', version: 1 }`; `!adf:doc {content=none}` reads as the former | -| a code fence whose info string opens `adf:` (```` ```adf:x ````) | a `codeBlock` with that language | the block carry; write `!adf:codeBlock {language="adf:x"}` around a bare fence to keep the code block | +| a code fence whose info string opens `adf:` (```` ```adf:x ````) | a `codeBlock` with that language | the block carry, refusing a body that is not one node's canonical JSON; write `!adf:codeBlock {language="adf:x"}` around a bare fence to keep the code block | ### Error codes -- 2.52.0 From 5cffe82dd51a4536e51ce667fba50c6c7f59acd4 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:05:36 +0200 Subject: [PATCH 36/67] Lead the refusals ordinary editing meets with the edit that fixes them --- README.md | 2 +- src/markdown/parse/inline-content.ts | 2 +- src/markdown/parse/markdown-to-adf.test.ts | 20 +++++++++++++++++--- src/markdown/parse/markdown-to-adf.ts | 10 +++++++--- 4 files changed, 26 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index e14b29a..2196a61 100644 --- a/README.md +++ b/README.md @@ -198,7 +198,7 @@ emit refuses: | `unspellable-line-start` | a paragraph line begins with a code span whose backticks would read back as a code fence | put any text before the code span | | `unspellable-whitespace` | an `emoji`, `mention` or `status` holds a newline in the text its inline directive spells in the content slot | replace it with a space — an inline directive never spans lines | | `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it | -| `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark | write the shape the message names; `spec/flavour.md` lists every type's attributes and body | +| `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark | write the shape the message names, or remove the reserved directive it names; `spec/flavour.md` lists every type's attributes and body | ## The guarantees diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index a2d6d24..8743d99 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -313,7 +313,7 @@ function partText(items: readonly Inline[], scan: Scan): Result { const previous = items[index - 1] const next = items[index + 1] if (previous === undefined || next === undefined || !isNode(previous) || !isNode(next) || !readsAsOne(previous, next)) { - return failure('unsupported-node-shape', `${textBreakName} parts two text nodes CommonMark reads back as one: this one parts something else`, scan.path) + return failure('unsupported-node-shape', `delete ${textBreakSpelling} here: it stands only between two runs of text with the same formatting, which would otherwise read as one`, scan.path) } } return success(parted) diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index f692896..2c72bfe 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -92,9 +92,12 @@ test('builds an empty document from input holding no block, and one holding no c const form = 'unsupported-node-shape: doc spells the one form !adf:doc {content=none}: this one spells another' assert.equal(content(markdownToAdf('!adf:doc\n')), form) assert.equal(content(markdownToAdf('!adf:doc x {content=none}\n')), form) - assert.equal(content(markdownToAdf('!adf:doc {content=empty}\n')), form) + assert.equal( + content(markdownToAdf('!adf:doc {content=empty}\n')), + 'unsupported-node-shape: an empty document is empty markdown, and !adf:doc {content=none} spells a document holding no content key: this one spells content=empty', + ) assert.equal(content(markdownToAdf('!adf:doc {content="none"}\n')), form) - const alone = 'unsupported-node-shape: !adf:doc {content=none} spells a whole document holding no content key, alone: this one stands among other blocks' + const alone = 'unsupported-node-shape: delete the !adf:doc {content=none} line to give the document content: it stands only as the whole document' assert.equal(content(markdownToAdf('x\n\n!adf:doc {content=none}\n')), alone) assert.equal(content(markdownToAdf('!adf:panel info\n!adf:doc {content=none}\n!adf:/panel\n')), alone) assert.equal(content(markdownToAdf('!adf:doc\n!adf:/doc\n')), 'malformed-directive: doc takes no body, so no !adf:/doc closes it; \\!adf: keeps the prefix literal') @@ -1087,7 +1090,7 @@ test('names the directive mark left without the content it wraps', () => { test('reads the text break only between two text nodes CommonMark joins, building no node', () => { assert.deepEqual(content(markdownToAdf('a!adf:textBreak{}b\n')), [{ content: [text('a'), text('b')], type: 'paragraph' }]) assert.deepEqual(content(markdownToAdf('==a!adf:textBreak{}b==\n')), [{ content: [text('==a'), text('b==')], type: 'paragraph' }]) - const parts = 'unsupported-node-shape: textBreak parts two text nodes CommonMark reads back as one: this one parts something else' + const parts = 'unsupported-node-shape: delete !adf:textBreak{} here: it stands only between two runs of text with the same formatting, which would otherwise read as one' const carried = '!adf:carry{json="{\\"text\\":\\"b\\",\\"type\\":\\"text\\"}"}' for (const markdown of ['!adf:textBreak{}a\n', 'a!adf:textBreak{}\n', 'a!adf:textBreak{}!adf:textBreak{}b\n', '**a**!adf:textBreak{}b\n', '[!adf:textBreak{}](/u)\n', `a!adf:textBreak{}${carried}\n`]) { assert.equal(content(markdownToAdf(markdown)), parts, markdown) @@ -1097,3 +1100,14 @@ test('reads the text break only between two text nodes CommonMark joins, buildin assert.equal(content(markdownToAdf('a!adf:textBreak{x=y}b\n')), 'unsupported-node-shape: textBreak spells the bare leaf form, !adf:textBreak{}: this one spells more') assert.deepEqual(content(markdownToAdf('!adf:carry{json="{\\"type\\":\\"textBreak\\"}"}\n')), [{ content: [{ type: 'textBreak' }], type: 'paragraph' }]) }) + +test('leads a refusal ordinary editing meets with the edit that fixes it', () => { + assert.equal( + content(markdownToAdf('!adf:paragraph {content=empty}\nText.\n!adf:/paragraph\n')), + 'unsupported-node-shape: remove content=empty to give the paragraph a body: content=empty holds none, and this one holds one', + ) + assert.equal( + content(markdownToAdf('!adf:codeBlock\n```\na\n```\n```\n```\n!adf:/codeBlock\n')), + 'unsupported-node-shape: delete the empty fence: a fence beside another holds code, and this one holds none', + ) +}) diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 5a6bdc3..9fda1e0 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -18,6 +18,7 @@ import { mintTaskIds } from './task-ids.ts' import { parseBlocks } from './blocks.ts' import { parseInlineContent } from './inline-content.ts' import { readBlockDirectiveNode } from './directive-nodes.ts' +import { spellsEmpty } from '../empty-keys.ts' import { unsupportedNodeShape } from '../directive-syntax.ts' type Paragraph = Extract @@ -58,7 +59,7 @@ function readBlocks(blocks: readonly Block[], reading: Reading, path: ConvertErr for (const [index, block] of blocks.entries()) { const nodePath = [...path, 'content', content.length] if (block.kind === 'directive' && block.name === documentName) { - const fault = documentFault(block) ?? unsupportedNodeShape(`${documentSpelling} spells a whole document holding no content key, alone: this one stands among other blocks`) + const fault = documentFault(block) ?? unsupportedNodeShape(`delete the ${documentSpelling} line to give the document content: it stands only as the whole document`) return positioned(faulted(fault, nodePath), block.position) } if (block.kind === 'directive' && block.name === listBreakName) { @@ -85,6 +86,9 @@ function listBreakFault(block: DirectiveBlock, previous: Block | undefined, next function documentFault(block: DirectiveBlock): ConvertFault | undefined { const spelled = block.attributes.get(documentAttribute.key) if (block.argument === undefined && block.attributes.size === 1 && spelled?.spelling === documentAttribute.value) return undefined + if (block.argument === undefined && block.attributes.size === 1 && spellsEmpty(spelled)) { + return unsupportedNodeShape(`an empty document is empty markdown, and ${documentSpelling} spells a document holding no content key: this one spells ${documentAttribute.key}=empty`) + } return unsupportedNodeShape(`${documentName} spells the one form ${documentSpelling}: this one spells another`) } @@ -232,7 +236,7 @@ function directiveBody(read: BlockDirectiveNode, blocks: Block[] | undefined, re const { contentModel, node } = read if (blocks === undefined) return success(node) // A directive builds a content key only from content=empty, which holds no body. - if (node.content !== undefined) return blocks.length === 0 ? success(node) : failure('unsupported-node-shape', `${node.type} spells content=empty, which holds no body: this one holds one`, path) + if (node.content !== undefined) return blocks.length === 0 ? success(node) : failure('unsupported-node-shape', `remove content=empty to give the ${node.type} a body: content=empty holds none, and this one holds one`, path) if (contentModel === 'code') return codeDirectiveNode(node, blocks, path) if (contentModel === 'inline') return inlineBodyNode(node, blocks, reading, path) return containerNode(node, blocks, reading, path, depth) @@ -248,7 +252,7 @@ function codeDirectiveNode(node: AdfNode, blocks: readonly Block[], path: Conver const [first] = fences if (first === undefined) return failure('unsupported-node-shape', `${node.type} takes code blocks as its body: this body holds none`, path) if (fences.some((fence) => fence.language !== first.language)) return failure('unsupported-node-shape', `${node.type} holds one language, so its fences carry one info string: these differ`, path) - if (fences.length > 1 && fences.some((fence) => fence.text === '')) return failure('unsupported-node-shape', `a fence beside another spells a text node, which holds text: this one is empty`, path) + if (fences.length > 1 && fences.some((fence) => fence.text === '')) return failure('unsupported-node-shape', `delete the empty fence: a fence beside another holds code, and this one holds none`, path) const attribute = nodeAttrs(node)['language'] const fromFence = first.language !== '' const slot = languageSlot(fromFence ? first.language : attribute) -- 2.52.0 From fb78063f127a5a2300d77d0c49f2783e0bce92c1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:05:51 +0200 Subject: [PATCH 37/67] Name the code block wrapper in the two carry-fence refusals that lacked it --- src/markdown/opaque-carry.ts | 4 ++-- src/markdown/parse/markdown-to-adf.test.ts | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/markdown/opaque-carry.ts b/src/markdown/opaque-carry.ts index 90bd538..1c2a218 100644 --- a/src/markdown/opaque-carry.ts +++ b/src/markdown/opaque-carry.ts @@ -39,7 +39,7 @@ export function carryFenceType(info: string): string | undefined { export function readCarriedBlock(type: string, body: string, depth: number): Read { if (type !== '' && !infoStringCarries(type)) { - return { fault: unsupportedNodeShape(`the carry fence names a type no info string carries back: spell it ${carryFencePrefix} with the type in the JSON`) } + return { fault: unsupportedNodeShape(`the carry fence names a type no info string carries back: spell it ${carryFencePrefix} with the type in the JSON; ${fenceEscape(type)}`) } } const read = readCarriedJson(body, 'two-space', largestNesting - depth, type) if (read.fault !== undefined || type !== '' || !infoStringCarries(read.value.type)) return read @@ -89,7 +89,7 @@ function fenceEscape(type: string): string { function typedValue(value: JsonValue, type: string): Read { if (value === null || typeof value !== 'object' || Array.isArray(value)) return { value } - if ('type' in value) return { fault: unsupportedNodeShape(`the ${carryFencePrefix}${type} fence names its node's type: this JSON holds a type as well`) } + if ('type' in value) return { fault: unsupportedNodeShape(`the ${carryFencePrefix}${type} fence names its node's type: this JSON holds a type as well; ${fenceEscape(type)}`) } return { value: { ...value, type } } } diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 2c72bfe..d121624 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -374,11 +374,11 @@ test('reads the carry fence back to the node its info string names and its JSON assert.deepEqual(content(markdownToAdf('```adf:\n{\n "type": "a`b"\n}\n```\n')), [{ type: 'a`b' }]) assert.equal( content(markdownToAdf('```adf:blockCard\n{\n "type": "blockCard"\n}\n```\n')), - "unsupported-node-shape: the adf:blockCard fence names its node's type: this JSON holds a type as well", + 'unsupported-node-shape: the adf:blockCard fence names its node\'s type: this JSON holds a type as well; !adf:codeBlock {language="adf:blockCard"} around a bare fence keeps it a code block', ) assert.equal(content(markdownToAdf('```adf:\n{\n "type": "blockCard"\n}\n```\n')), 'unsupported-node-shape: the carry fence names a type its info string carries: spell it adf:blockCard') assert.equal(content(markdownToAdf('```adf:\n{}\n```\n')), "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node; !adf:codeBlock {language=\"adf:\"} around a bare fence keeps it a code block") - const unnamed = 'unsupported-node-shape: the carry fence names a type no info string carries back: spell it adf: with the type in the JSON' + const unnamed = 'unsupported-node-shape: the carry fence names a type no info string carries back: spell it adf: with the type in the JSON; !adf:codeBlock {language="adf:\\\\"} around a bare fence keeps it a code block' assert.equal(content(markdownToAdf('```adf:\\\\\n{}\n```\n')), unnamed) }) -- 2.52.0 From 912c5039aa49214603402a97ba603ec3330aec3c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:06:11 +0200 Subject: [PATCH 38/67] Rewrap the broken lines, and say plainly how an adf: fence stays code --- CHANGELOG.md | 4 ++-- README.md | 15 +++++++-------- corpus/README.md | 9 +++++---- spec/flavour.md | 29 +++++++++++++++-------------- 4 files changed, 29 insertions(+), 28 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a591365..6e548ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,8 +9,8 @@ `type`: text holding an unescaped `!adf:` and a code fence whose info string opens `adf:` are claimed, and `adf` is an ordinary code block language. Convert stored markdown per `MIGRATION.md`. - **Breaking:** `markdownToAdf` and `plainMarkdownToAdf` read markdown holding no block as a - document whose `content` is empty, as Atlassian's schema requires; `!adf:doc {content=none}` spells a document holding no - `content` key. + document whose `content` is empty, as Atlassian's schema requires; `!adf:doc {content=none}` + spells a document holding no `content` key. - `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc`: two adjacent text nodes a reader would join are parted by `!adf:textBreak{}`, an empty `attrs`, `content` or `marks` is spelled `{attrs=empty}`, `{content=empty}` or `{marks=empty}`, `-0` is spelled `-0`, and a `codeBlock` diff --git a/README.md b/README.md index 2196a61..bf3c06c 100644 --- a/README.md +++ b/README.md @@ -212,14 +212,13 @@ Serves Goals 1, 3 and 4. `0.2.0`, well-formed HTML means what the HTML standard says, read or written. The bullets below name every exception. - Plain CommonMark is valid input to `markdownToAdf` apart from the raw HTML `unmappable-html` - names, with four carve-outs — literal text matching directive, pipe-table or strikethrough - syntax, and a code fence whose info string opens `adf:`, is claimed (escapable — `spec/flavour.md`) - — and one gap: a CommonMark image fits only as - its own title-less paragraph; mid-text and titled images are error results, save an image inside - another's description, which flattens into the alt text. Converting back yields the library's - canonical spelling, which round-trips byte-identically — where it converts back at all: a parse - succeeding is no promise of that, so keep the source until the way back succeeds. - ``` ` `` ` ``` reads cleanly and then refuses. + names, with four carve-outs — literal text matching directive, pipe-table or strikethrough syntax, + and a code fence whose info string opens `adf:`, are claimed (escapable — `spec/flavour.md`) — and + one gap: a CommonMark image fits only as its own title-less paragraph; mid-text and titled images + are error results, save an image inside another's description, which flattens into the alt text. + Converting back yields the library's canonical spelling, which round-trips byte-identically — + where it converts back at all: a parse succeeding is no promise of that, so keep the source until + the way back succeeds. ``` ` `` ` ``` reads cleanly and then refuses. - Four CommonMark spellings parse without an error and build a document the reference implementation renders differently: `[](/url)` and `[]()` stay literal text against CommonMark's empty link, a list continuing past a marker change stays one list against CommonMark's two, a diff --git a/corpus/README.md b/corpus/README.md index 6ae3974..3629e43 100644 --- a/corpus/README.md +++ b/corpus/README.md @@ -19,7 +19,8 @@ One directory per contract kind: model), `unspellable` (parses but the flavour has no spelling) or `pending` (a parser gap). JSON is two-space indent, keys sorted, and a document read back must deep-equal the fixture's -(`docs/decisions.md` §Equality is deep). `spec.json` is the vendored, upstream machine-readable suite, byte-exact from -[spec.commonmark.org](https://spec.commonmark.org/0.31.2/spec.json) (CommonMark 0.31.2, © John -MacFarlane, [CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)), and is not -re-serialized by the corpus gate. +(`docs/decisions.md` §Equality is deep). `spec.json` is the vendored, upstream machine-readable +suite, byte-exact from [spec.commonmark.org](https://spec.commonmark.org/0.31.2/spec.json) +(CommonMark 0.31.2, © John MacFarlane, +[CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)), and is not re-serialized by the +corpus gate. diff --git a/spec/flavour.md b/spec/flavour.md index e9a35b5..f0f9e57 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -4,10 +4,11 @@ The grammar of the extended markdown `adfToMarkdown` emits and `markdownToAdf` p CommonMark is a subset apart from raw HTML (below), with four carve-outs: literal text that matches directive syntax below or reads as a pipe table is claimed by the flavour, a matched `~~` pair spells `strike` (escape the `!adf:`, `|` or `~` to keep it literal), and a code fence whose info -string opens `adf:` is the opaque carry (The opaque carry says how to keep it code) — and one gap: a CommonMark -image fits only as its own title-less paragraph — mid-text and titled images are named errors. The -emitted form is contract (`docs/decisions.md` §The formats are API). Per-node syntaxes build on this -grammar in the sections below. +string opens `adf:` is the opaque carry (wrap it as a bare fence in `!adf:codeBlock +{language="adf:…"}` to keep it code) — and one gap: a CommonMark image fits only as its own +title-less paragraph — mid-text and titled images are named errors. The emitted form is contract +(`docs/decisions.md` §The formats are API). Per-node syntaxes build on this grammar in the sections +below. ## Canonical form @@ -446,8 +447,8 @@ Attributes and the carry fallback read as in the block sections, the carry in it the nodes below, `emoji`, `mention` and `status` spell their `text` attribute in the content slot as plain text: `[]` is the empty string, absent content is the absent attribute, non-empty content parsing to anything but one text node carrying neither marks, attributes nor content is a named -error, and so is a `text` key in `{attrs}`. An enclosing mark spelling does not reach into the slot. The rest take no content, -`!adf:text` included; content on a node that takes none is a named error. +error, and so is a `text` key in `{attrs}`. An enclosing mark spelling does not reach into the slot. +The rest take no content, `!adf:text` included; content on a node that takes none is a named error. - `date` — Attributes: `localId` (string), `timestamp` (string, epoch milliseconds). - `emoji` — Attributes: `id` (string), `localId` (string), `shortName` (string, `:name:`), `text` @@ -527,14 +528,14 @@ that depth: `attrs: {}` differs from no `attrs`, and a directive spells it `{att breaks at every node the emitter carries, so no emitted carry sits inside a mark spelling. An inline node whose marks no nesting spells — a mark type not listed here, an attrs key its -spelling does not list, a value that is not the spelling's type, an attribute the spelling needs -and the mark lacks, an empty `attrs` on a mark CommonMark spells, an order putting a code span outside another mark, `code` over anything but a -text node or over text holding a newline, or a spelling CommonMark's flanking rules cannot open or -close where the run sits (`un**-real**istic`), or one CommonMark's matching pairs elsewhere — the -intra-word `*` runs together with a neighbouring `**`, and the multiple-of-3 rule can leave the -merged run's pairing to another delimiter — rides the inline carry whole. An opaque carry inside a -mark spelling is a named error in input: the carry restores its node exactly, marks included -(`docs/decisions.md` §Unknown nodes ride the carry). +spelling does not list, a value that is not the spelling's type, an attribute the spelling needs and +the mark lacks, an empty `attrs` on a mark CommonMark spells, an order putting a code span outside +another mark, `code` over anything but a text node or over text holding a newline, or a spelling +CommonMark's flanking rules cannot open or close where the run sits (`un**-real**istic`), or one +CommonMark's matching pairs elsewhere — the intra-word `*` runs together with a neighbouring `**`, +and the multiple-of-3 rule can leave the merged run's pairing to another delimiter — rides the +inline carry whole. An opaque carry inside a mark spelling is a named error in input: the carry +restores its node exactly, marks included (`docs/decisions.md` §Unknown nodes ride the carry). ``` !adf:textColor[**Overdue**]{color="#ae2e24"}, H!adf:subsup[2]{type=sub}O, !adf:underline[signed]. -- 2.52.0 From 79d9b4db53f4d5750aa5a887b396bc3d7f895d1d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:06:22 +0200 Subject: [PATCH 39/67] Keep the fence wrapper's code span on one line --- spec/flavour.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/spec/flavour.md b/spec/flavour.md index f0f9e57..5347a62 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -4,11 +4,11 @@ The grammar of the extended markdown `adfToMarkdown` emits and `markdownToAdf` p CommonMark is a subset apart from raw HTML (below), with four carve-outs: literal text that matches directive syntax below or reads as a pipe table is claimed by the flavour, a matched `~~` pair spells `strike` (escape the `!adf:`, `|` or `~` to keep it literal), and a code fence whose info -string opens `adf:` is the opaque carry (wrap it as a bare fence in `!adf:codeBlock -{language="adf:…"}` to keep it code) — and one gap: a CommonMark image fits only as its own -title-less paragraph — mid-text and titled images are named errors. The emitted form is contract -(`docs/decisions.md` §The formats are API). Per-node syntaxes build on this grammar in the sections -below. +string opens `adf:` is the opaque carry (wrap it as a bare fence in +`!adf:codeBlock {language="adf:…"}` to keep it code) — and one gap: a CommonMark image fits only as +its own title-less paragraph — mid-text and titled images are named errors. The emitted form is +contract (`docs/decisions.md` §The formats are API). Per-node syntaxes build on this grammar in the +sections below. ## Canonical form -- 2.52.0 From 7846502a1a451ad70bc9cfdaab6da31710030542 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:12:51 +0200 Subject: [PATCH 40/67] Lead the code block, empty-key, carry fence, text break and carry-in-mark refusals with their fix --- src/markdown/empty-keys.ts | 4 ++-- src/markdown/opaque-carry.ts | 2 +- src/markdown/parse/inline-content.ts | 6 +++--- src/markdown/parse/markdown-to-adf.test.ts | 18 +++++++++++++----- src/markdown/parse/markdown-to-adf.ts | 7 ++++++- 5 files changed, 25 insertions(+), 12 deletions(-) diff --git a/src/markdown/empty-keys.ts b/src/markdown/empty-keys.ts index 4bbbf97..08050d2 100644 --- a/src/markdown/empty-keys.ts +++ b/src/markdown/empty-keys.ts @@ -23,10 +23,10 @@ export function readEmptyKeys(type: string, attributes: DirectiveAttributes, key for (const key of keys) { const spelled = rest.get(key) if (spelled === undefined) continue - if (!spellsEmpty(spelled)) return { fault: unsupportedNodeShape(`the reserved key ${key} reads ${key}=${emptyValue} alone: this one spells ${key}=${spelled.spelling}`) } + if (!spellsEmpty(spelled)) return { fault: unsupportedNodeShape(`the reserved key ${key} takes only the value ${emptyValue}: this one spells ${key}=${spelled.spelling}`) } empty.add(key) rest.delete(key) } - if (empty.has('attrs') && (held || [...rest.keys()].some((key) => key !== 'marks'))) return { fault: unsupportedNodeShape(`${type} spells attrs=empty beside an attribute it holds`) } + if (empty.has('attrs') && (held || [...rest.keys()].some((key) => key !== 'marks'))) return { fault: unsupportedNodeShape(`${type} spells attrs=empty beside another attribute, an argument or content: drop attrs=empty or the rest`) } return { value: { empty, rest } } } diff --git a/src/markdown/opaque-carry.ts b/src/markdown/opaque-carry.ts index 1c2a218..3941cf3 100644 --- a/src/markdown/opaque-carry.ts +++ b/src/markdown/opaque-carry.ts @@ -43,7 +43,7 @@ export function readCarriedBlock(type: string, body: string, depth: number): Rea } const read = readCarriedJson(body, 'two-space', largestNesting - depth, type) if (read.fault !== undefined || type !== '' || !infoStringCarries(read.value.type)) return read - return { fault: unsupportedNodeShape(`the carry fence names a type its info string carries: spell it ${carryFencePrefix}${read.value.type}`) } + return { fault: unsupportedNodeShape(`the ${carryFencePrefix} fence holds a type its info string can carry: spell the fence ${carryFencePrefix}${read.value.type} and drop type from the JSON`) } } export function readCarriedInline(span: DirectiveSpan): Read | undefined { diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index 8743d99..b2bfd69 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -72,7 +72,7 @@ type SharedScan = Pick type SlotContent = { carry: boolean; nodes: Inline[] } -const carriedInMark = 'no mark spelling wraps an opaque carry or an inline node spelling marks=empty: its marks are its own' +const carriedInMark = 'move the opaque carry, or the node spelling marks=empty, out of the mark spelling: it holds its own marks' const editorHighlight: AdfMark = { attrs: { color: '#f8e6a0' }, type: 'backgroundColor' } const hreflessLink = 'the link mark spells its href: this one spells none' const imageAlone = 'an image fits only as a paragraph of its own: this one sits inside other content' @@ -247,7 +247,7 @@ function directivePiece(scan: Scan, span: DirectiveSpan, index: number): Result< const mark = readDirectiveMark(span.name, span.attributes, scan.path) if (mark !== undefined) return mark.ok ? directiveMarkPiece(scan, span.name, mark.value, slot.value, index) : mark const content = slot.value?.nodes - if (content?.some(isTextBreak) === true) return failure('unsupported-node-shape', `${textBreakName} parts two text nodes: a content slot holds plain text`, scan.path) + if (content?.some(isTextBreak) === true) return failure('unsupported-node-shape', `delete ${textBreakSpelling} from this content slot: a slot holds one text node`, scan.path) const node = readInlineDirectiveNode(span.name, span.attributes, content === undefined ? undefined : adfNodes(content), scan.path) if (!node.ok) return node return success(node.value.marks === undefined ? { kind: 'nodes', nodes: [node.value] } : { kind: 'carry', node: node.value }) @@ -488,7 +488,7 @@ function closeImage(scan: Scan, at: number, inner: readonly Piece[], definition: function imageAlt(inner: readonly Piece[], scan: Scan): Result { const nodes = resolveNodes(inner, scan, false) if (!nodes.ok) return nodes - if (nodes.value.some(isTextBreak)) return failure('unsupported-node-shape', `${textBreakName} parts two text nodes: an image description holds plain text`, scan.path) + if (nodes.value.some(isTextBreak)) return failure('unsupported-node-shape', `delete ${textBreakSpelling} from this image description: the description reads as plain alt text`, scan.path) return success(adfNodes(nodes.value).map(altText).join('')) } diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index d121624..396602f 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -166,7 +166,7 @@ test('reads the codeBlock directive body as the node content, the info string it test('names the slot a codeBlock spells its language outside of', () => { const slot = 'unsupported-node-shape: codeBlock spells its language in the fence info string, or in the attribute where no info string carries it back' - assert.equal(content(markdownToAdf('!adf:codeBlock {language=rust wrap=true}\n```\nx\n```\n!adf:/codeBlock\n')), slot) + assert.equal(content(markdownToAdf('!adf:codeBlock {language=rust wrap=true}\n```\nx\n```\n!adf:/codeBlock\n')), "unsupported-node-shape: move language=rust to the fence's info string: a fence carries the language wherever its info string can") assert.equal(content(markdownToAdf('!adf:codeBlock {language=rust}\n```sql\nx\n```\n!adf:/codeBlock\n')), slot) assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n```adf:x\nx\n```\n!adf:/codeBlock\n')), slot) assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n```a\\b\nx\n```\n!adf:/codeBlock\n')), slot) @@ -376,7 +376,7 @@ test('reads the carry fence back to the node its info string names and its JSON content(markdownToAdf('```adf:blockCard\n{\n "type": "blockCard"\n}\n```\n')), 'unsupported-node-shape: the adf:blockCard fence names its node\'s type: this JSON holds a type as well; !adf:codeBlock {language="adf:blockCard"} around a bare fence keeps it a code block', ) - assert.equal(content(markdownToAdf('```adf:\n{\n "type": "blockCard"\n}\n```\n')), 'unsupported-node-shape: the carry fence names a type its info string carries: spell it adf:blockCard') + assert.equal(content(markdownToAdf('```adf:\n{\n "type": "blockCard"\n}\n```\n')), 'unsupported-node-shape: the adf: fence holds a type its info string can carry: spell the fence adf:blockCard and drop type from the JSON') assert.equal(content(markdownToAdf('```adf:\n{}\n```\n')), "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node; !adf:codeBlock {language=\"adf:\"} around a bare fence keeps it a code block") const unnamed = 'unsupported-node-shape: the carry fence names a type no info string carries back: spell it adf: with the type in the JSON; !adf:codeBlock {language="adf:\\\\"} around a bare fence keeps it a code block' assert.equal(content(markdownToAdf('```adf:\\\\\n{}\n```\n')), unnamed) @@ -441,7 +441,7 @@ test('names the number no JSON spelling carries in an opaque carry', () => { }) test('names the mark spelling no opaque carry sits inside', () => { - const named = 'unsupported-node-shape: no mark spelling wraps an opaque carry or an inline node spelling marks=empty: its marks are its own' + const named = 'unsupported-node-shape: move the opaque carry, or the node spelling marks=empty, out of the mark spelling: it holds its own marks' assert.equal(content(markdownToAdf(`_a ${carried} b_\n`)), named) assert.equal(content(markdownToAdf(`**${carried}**\n`)), named) assert.equal(content(markdownToAdf(`~~a ${carried}~~\n`)), named) @@ -1095,8 +1095,8 @@ test('reads the text break only between two text nodes CommonMark joins, buildin for (const markdown of ['!adf:textBreak{}a\n', 'a!adf:textBreak{}\n', 'a!adf:textBreak{}!adf:textBreak{}b\n', '**a**!adf:textBreak{}b\n', '[!adf:textBreak{}](/u)\n', `a!adf:textBreak{}${carried}\n`]) { assert.equal(content(markdownToAdf(markdown)), parts, markdown) } - assert.equal(content(markdownToAdf('!adf:status[a!adf:textBreak{}b]\n')), 'unsupported-node-shape: textBreak parts two text nodes: a content slot holds plain text') - assert.equal(content(markdownToAdf('![a!adf:textBreak{}b](/i)\n')), 'unsupported-node-shape: textBreak parts two text nodes: an image description holds plain text') + assert.equal(content(markdownToAdf('!adf:status[a!adf:textBreak{}b]\n')), 'unsupported-node-shape: delete !adf:textBreak{} from this content slot: a slot holds one text node') + assert.equal(content(markdownToAdf('![a!adf:textBreak{}b](/i)\n')), 'unsupported-node-shape: delete !adf:textBreak{} from this image description: the description reads as plain alt text') assert.equal(content(markdownToAdf('a!adf:textBreak{x=y}b\n')), 'unsupported-node-shape: textBreak spells the bare leaf form, !adf:textBreak{}: this one spells more') assert.deepEqual(content(markdownToAdf('!adf:carry{json="{\\"type\\":\\"textBreak\\"}"}\n')), [{ content: [{ type: 'textBreak' }], type: 'paragraph' }]) }) @@ -1110,4 +1110,12 @@ test('leads a refusal ordinary editing meets with the edit that fixes it', () => content(markdownToAdf('!adf:codeBlock\n```\na\n```\n```\n```\n!adf:/codeBlock\n')), 'unsupported-node-shape: delete the empty fence: a fence beside another holds code, and this one holds none', ) + assert.equal( + content(markdownToAdf('!adf:codeBlock {attrs=empty}\n```js\na\n```\n```js\nb\n```\n!adf:/codeBlock\n')), + "unsupported-node-shape: remove attrs=empty to give the codeBlock the fence's language: attrs=empty holds no language, and this fence names one", + ) + assert.equal( + content(markdownToAdf('!adf:panel info {attrs=empty}\nText.\n!adf:/panel\n')), + 'unsupported-node-shape: panel spells attrs=empty beside another attribute, an argument or content: drop attrs=empty or the rest', + ) }) diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 9fda1e0..d88db4a 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -256,10 +256,15 @@ function codeDirectiveNode(node: AdfNode, blocks: readonly Block[], path: Conver const attribute = nodeAttrs(node)['language'] const fromFence = first.language !== '' const slot = languageSlot(fromFence ? first.language : attribute) + if (!fromFence && slot.kind === 'fence') { + return failure('unsupported-node-shape', `move language=${slot.info} to the fence's info string: a fence carries the language wherever its info string can`, path) + } if ((slot.kind === 'fence') !== fromFence || (fromFence && attribute !== undefined)) { return failure('unsupported-node-shape', `${node.type} spells its language in the fence info string, or in the attribute where no info string carries it back`, path) } - if (fromFence && emptyKeys(node).includes('attrs')) return failure('unsupported-node-shape', `${node.type} spells attrs=empty beside an attribute it holds`, path) + if (fromFence && emptyKeys(node).includes('attrs')) { + return failure('unsupported-node-shape', `remove attrs=empty to give the ${node.type} the fence's language: attrs=empty holds no language, and this fence names one`, path) + } const spelled = fromFence ? { ...node, attrs: { ...node.attrs, language: first.language } } : node return success(withContent(spelled, first.text === '' ? [] : fences.map((fence): AdfNode => ({ text: fence.text, type: 'text' })))) } -- 2.52.0 From 230d590fb406dda5b1768821a8e13f322023f964 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:12:54 +0200 Subject: [PATCH 41/67] Trim two comments --- src/adf/document.ts | 1 - src/markdown/emit/plain-reduction.ts | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/src/adf/document.ts b/src/adf/document.ts index 61afdc7..375a36f 100644 --- a/src/adf/document.ts +++ b/src/adf/document.ts @@ -77,7 +77,6 @@ export function identicalMarks(left: readonly AdfMark[], right: readonly AdfMark return marksKey(left) === marksKey(right) } -// Callers say when two nodes join. export function mergeAdjacentText(nodes: readonly AdfNode[], joins: (previous: AdfNode, node: AdfNode) => boolean): AdfNode[] { const merged: AdfNode[] = [] for (const node of nodes) { diff --git a/src/markdown/emit/plain-reduction.ts b/src/markdown/emit/plain-reduction.ts index 00883e0..2545002 100644 --- a/src/markdown/emit/plain-reduction.ts +++ b/src/markdown/emit/plain-reduction.ts @@ -54,7 +54,7 @@ export function reduceToPlain(document: AdfDocument): Result { // docs/decisions.md §Equality is deep: the reduction reads and writes editor-normal ADF. const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] }) if (!blocks.ok) { - // Merging text renumbers siblings, so the caller's document names the path of the refusal the editor-normal one meets. + // Merging text renumbers siblings, so a refusal's path comes from the caller's document. const raw = reduceBlocks(nodeContent(document), { depth: 0, memo: new Map(), path: [] }) return !raw.ok && raw.error.code === blocks.error.code ? raw : blocks } -- 2.52.0 From e4eecebeab824b033ebcf54d1090ea1a210562a0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:13:30 +0200 Subject: [PATCH 42/67] Keep the decisions to their reasons and link the grammar; name the bare adf: fence's refusal exactly; call the doc panel a writer panel --- docs/decisions.md | 69 +++++++++++++++++++++-------------------------- spec/flavour.md | 4 +-- 2 files changed, 33 insertions(+), 40 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 0aab2e4..19c83f3 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -36,28 +36,23 @@ does not imply a spellable document; `corpus/commonmark-spec/exceptions.json` na 2026-08-24, deep 2026-10-03, the maintainer. Goal 1. Valid while a pipeline or a bot can build a shape the editor would not. -"Equals" is deep equality over the document's JSON values: `assert.deepStrictEqual` on plain -objects as `JSON.parse` builds them, since ADF is JSON. Every key and value `doc` holds counts, two -adjacent text nodes, an empty `attrs`, `content` or `marks` and `-0` included, with no -normalization on either side. -CommonMark's spelling stays wherever a document holds none of those shapes. The plain reader -builds what is written, as `markdownToAdf` does. Only the plain writer is lossy: its reduction -reads and writes editor-normal ADF — adjacent text nodes of identical marks and no attributes -merged, `-0` as `0`, and an empty `attrs`, `content` or `marks` the absent key — so two documents -the editor holds equal write the same plain markdown. +"Equals" is deep equality over the document's JSON values: `assert.deepStrictEqual` on plain objects +as `JSON.parse` builds them, since ADF is JSON. Every key and value in `doc` counts, including two +adjacent text nodes, an empty `attrs`, `content` or `marks`, and `-0`. Neither side is normalized. +CommonMark's spelling stays wherever a document holds none of those shapes. The plain reader builds +what is written, as `markdownToAdf` does. Only the plain writer is lossy: its reduction reads and +writes editor-normal ADF — adjacent text nodes of identical marks and no attributes merged, `-0` as +`0`, and an empty `attrs`, `content` or `marks` the absent key — so two documents the editor holds +equal write the same plain markdown. ## `!adf:textBreak{}` parts text CommonMark would join 2026-10-03, the maintainer. Goal 1. Valid while CommonMark reads adjacent text as one run. -Two adjacent text nodes a reader would build back as one — neither carried, neither holding -`attrs` or an empty key, their marks identical, attributes included — are parted by the reserved -inline leaf `!adf:textBreak{}`, mirroring `!adf:listBreak`: it builds no node, and anywhere but -between two such nodes, or given `[content]` or `{attrs}`, it is `unsupported-node-shape`. It sits -inside every mark spelling the pair shares, `**Hello, !adf:textBreak{}world**`; a code span holds -no directive, so it closes and reopens, `` `a`!adf:textBreak{}`b` ``. A carried node never joins -its neighbour, so a carry needs no break. A mark run breaks on any difference in the mark, `attrs: -{}` against no `attrs` included. +Two adjacent text nodes CommonMark would read back as one are parted by the reserved inline leaf +`!adf:textBreak{}`, mirroring `!adf:listBreak`: a leaf building no node keeps both nodes and asks +nothing of the text around it. A code span holds no directive, so the spans close and reopen +around the leaf. The grammar: `spec/flavour.md` §Inline nodes, **Adjacent text nodes**. ## An empty key spells `empty` @@ -84,14 +79,14 @@ the directive form, since no list marker spells the sign. ## Empty markdown is a document of no blocks -2026-10-03, a panel and the maintainer. Goals 1 and 5. Valid while ADF's schema requires +2026-10-03, a writer panel and the maintainer. Goals 1 and 5. Valid while ADF's schema requires `content` on `doc`. Markdown holding no block reads as `{ content: [], type: 'doc', version: 1 }`, the document `spec/adf-schema/full.json` requires. A document holding no `content` key is -`!adf:doc {content=none}` standing alone as its only block, and a named error anywhere else. The -writer panel split 4 for `none` and 3 for `absent`, and the maintainer chose `none`; all seven rejected a -bare `!adf:doc`. +`!adf:doc {content=none}` as its only block, and a named error anywhere else. The writer panel split +4 for `none` and 3 for `absent`, and the maintainer chose `none`; all seven rejected a bare +`!adf:doc`. ## Unknown nodes ride the carry @@ -114,10 +109,10 @@ The block carry is a code fence whose info string `adf:` names the node's node's JSON without `type`: ```` ```adf:blockCard ````. A type no info string carries back — by the rule a code language follows — leaves the info string `adf:` and keeps `type` in the body. Every info string opening `adf:` is reserved, so a `codeBlock` whose language opens so takes the -`language` attribute, and `carry` is an ordinary language. A body holding `type` under a named -type, or an `adf:` fence whose type an info string carries, is `unsupported-node-shape`. The -reservation claims a fence CommonMark reads as code until `todo.md` item 43 gives CommonMark its own -reader. +`language` attribute, and `carry` is an ordinary language. A body holding `type` under a named type, +or a bare `adf:` fence whose body's `type` an info string could carry, is `unsupported-node-shape`. +The reservation claims a fence CommonMark reads as code until `todo.md` item 43 gives CommonMark its +own reader. ## A code block is a fence per text node @@ -125,11 +120,9 @@ reader. node. A `codeBlock` holding several text nodes is the `!adf:codeBlock` container holding one fence per -node, each fence's info string the language; its other attributes sit on the opener, and a -language no info string carries stays the opener's `language` with bare fences. A `codeBlock` -spelling `content=empty` has no fence to carry the language, so the opener does. Fences with -differing info strings are `unsupported-node-shape` — ADF holds one language — and so is an empty -fence beside another, since a text node holds text. +node, so each node keeps its own text. The fences carry one info string, since ADF holds one +language, and none is empty beside another, since a text node holds text. The grammar: +`spec/flavour.md` §The CommonMark blocks, the `codeBlock` bullet. ## Foreign HTML sorts three ways @@ -372,14 +365,14 @@ handles one cause alike whichever node, attribute or direction raised it. colon rather than ADF's missing column model doing it. What the grammar itself refuses stays a claim code, key order among it, and a leaf given a body is refused at its opener, as a container missing its closer is (2026-09-16). -- A directive whose name reads back to no node is `unknown-directive-name` rather than a claim - code — the spelling is well formed, and telling that apart from a typo is what a consumer - switches on when a later MINOR gives the name meaning. A reserved name is a known name, so never - that code, and the names the flavour reserves part on form: a form the grammar does not have is - a claim code — `!adf:carry`, whose carry is the fence — and a well-formed form in the wrong place - is `unsupported-node-shape`, `!adf:listBreak` parting anything but two adjacent lists of one - type, `!adf:textBreak{}` anything but two text nodes a reader joins, `!adf:doc` standing beside - another block (2026-09-01, the text break and `doc` 2026-10-03). +- A directive whose name reads back to no node is `unknown-directive-name` rather than a claim code + — the spelling is well formed, and telling that apart from a typo is what a consumer switches on + when a later MINOR gives the name meaning. A reserved name is a known name, so never that code, + and the names the flavour reserves part on form: a form the grammar does not have is a claim code + — `!adf:carry`, whose carry is the fence — and a well-formed form in the wrong place is + `unsupported-node-shape`, `!adf:listBreak` parting anything but two adjacent lists of one type, + `!adf:textBreak{}` anything but two adjacent text nodes CommonMark would read back as one, + `!adf:doc` standing beside another block (2026-09-01, the text break and `doc` 2026-10-03). - A well-formed directive the node tables refuse — an attribute a node does not hold or spells elsewhere, a value outside its kind or its canonical spelling, an argument, or a body of a shape its content model does not take — is `unsupported-node-shape`, the emitter's code for the same diff --git a/spec/flavour.md b/spec/flavour.md index 5347a62..a3170d6 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -181,8 +181,8 @@ product). Block and inline positions canonicalize differently, each fitting wher - **Block position**: a fenced code block with info string `adf:` and the node's type, body = the node's JSON without its `type` — two-space indent, object keys sorted: ```` ```adf:blockCard ````. A type no info string carries back, by the rule a `codeBlock`'s language follows, leaves the info - string `adf:` and keeps `type` in the body. A body holding `type` under a named type, or an - `adf:` fence whose type an info string carries, is a named error. + string `adf:` and keeps `type` in the body. A body holding `type` under a named type, or a bare + `adf:` fence whose body's `type` an info string could carry, is a named error. - **Inline position**: `!adf:carry{json="…"}` — compact serialization (keys sorted, no whitespace), JSON-string-escaped into the attribute. -- 2.52.0 From a2c620da2798f754c275e022cb12cfdb9ad53165 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:13:50 +0200 Subject: [PATCH 43/67] Define the writer panel, split the carry fence into its own changelog entry, and say how an adf: fence stays code --- AGENTS.md | 6 +++--- CHANGELOG.md | 21 ++++++++++++--------- MIGRATION.md | 8 ++++---- README.md | 17 +++++++++-------- spec/flavour.md | 2 +- 5 files changed, 29 insertions(+), 25 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 368df4d..af7002b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -174,9 +174,9 @@ allow, rendered, shuffled, with no rationale and nothing saying what is implemen settle it; otherwise four more read, five of seven settle it, and less is a missing goal, asked. The verdict lands in `docs/decisions.md`. -Every new or changed markdown or HTML spelling goes to such a panel, seated by the people who read -and write that format (README `## Audience`); a question about what an app relies on seats the -developer personas. +A writer panel settles every new or changed markdown or HTML spelling: a reader panel whose readers +are the people who read and write that format (README `## Audience`). A question about what an app +relies on goes to the developer personas instead. ### Stated numbers diff --git a/CHANGELOG.md b/CHANGELOG.md index 6e548ed..a10c5de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,18 +4,21 @@ - **Breaking:** directives, the inline opaque carry among them (now `!adf:carry{json="…"}`), are spelled under an `!adf:` prefix (`!adf:name … !adf:/name`, `!adf:name[content]{attrs}`, - `!adf:name arg {attrs}`) in place of the `:::`/`::`/`:name` forms, and the block carry is a code - fence whose info string `adf:` names the node's type, its body the node's JSON without - `type`: text holding an unescaped `!adf:` and a code fence whose info string opens `adf:` are - claimed, and `adf` is an ordinary code block language. Convert stored markdown per `MIGRATION.md`. + `!adf:name arg {attrs}`) in place of the `:::`/`::`/`:name` forms: text holding an unescaped + `!adf:` is claimed, and `adf` is an ordinary code block language. Convert stored markdown per + `MIGRATION.md`. +- **Breaking:** the block carry is a code fence whose info string `adf:` names the node's + type, its body the node's JSON without `type`, and a code fence whose info string opens `adf:` is + claimed. Convert stored markdown per `MIGRATION.md`. - **Breaking:** `markdownToAdf` and `plainMarkdownToAdf` read markdown holding no block as a document whose `content` is empty, as Atlassian's schema requires; `!adf:doc {content=none}` spells a document holding no `content` key. -- `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc`: two adjacent text nodes a reader would - join are parted by `!adf:textBreak{}`, an empty `attrs`, `content` or `marks` is spelled - `{attrs=empty}`, `{content=empty}` or `{marks=empty}`, `-0` is spelled `-0`, and a `codeBlock` - of several text nodes is a fence per node. A `codeBlock` holding other than plain text nodes - rides the block carry, where it was refused. +- `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc` as JSON, for a document of plain objects as + `JSON.parse` builds them: two adjacent text nodes CommonMark would read back as one are parted by + `!adf:textBreak{}`, an empty `attrs`, `content` or `marks` is spelled `{attrs=empty}`, + `{content=empty}` or `{marks=empty}`, `-0` is spelled `-0`, and a `codeBlock` of several text + nodes is a fence per node. A `codeBlock` holding other than plain text nodes rides the block + carry, where it was refused. - **Breaking:** `unspellable-link` leaves `ConvertErrorCode`; a link whose `href` or `title` no CommonMark escape spells is written as `!adf:link[text]{attrs}`. - **Breaking:** some directive refusals carry `malformed-directive` where they carried diff --git a/MIGRATION.md b/MIGRATION.md index f9b9b49..d213215 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -5,9 +5,9 @@ Directives moved under the `!adf:` prefix. `0.2.0` reads `0.1.0`'s spelling without an error, turning each directive into text and each carried node into an `adf` code block. Before `0.2.0` reads any `0.1.0` markdown, convert what is stored or in flight (an open editor, a queue) with the -recipe below, and rewrite markdown your code writes or matches (templates, prompts, patterns) by -the tables below. Stored ADF needs one change: a document `0.1.0` read from empty markdown holds -no `content` key, and gets `content: []`, the empty document it meant. +recipe below, and rewrite markdown your code writes or matches (templates, prompts, patterns) by the +tables below. Stored ADF needs one change: give a document holding no `content` key `content: []`. +`0.1.0` read empty markdown to that shape, meaning the empty document. ### Convert markdown @@ -23,7 +23,7 @@ import { markdownToAdf as markdownToAdf010 } from 'adf-codec-0.1' function migrateMarkdown(stored: string) { const parsed = markdownToAdf010(stored) - // 0.1.0's reader was editor-normal, so a document with no content key always meant an empty one. + // 0.1.0 dropped an empty content array, so a document with no content key meant an empty one. return parsed.ok ? adfToMarkdown({ ...parsed.value, content: parsed.value.content ?? [] }) : parsed } ``` diff --git a/README.md b/README.md index bf3c06c..0fb2474 100644 --- a/README.md +++ b/README.md @@ -175,7 +175,7 @@ Parsing — `markdownToAdf` and `plainMarkdownToAdf`, and `htmlToAdf` at `0.2.0` | Code | Fires when | What you can do | | --- | --- | --- | -| `malformed-directive` | an `!adf:` the grammar cannot read — a prefix completing no directive, an unclosed container, `[content]` or `{attrs}`, a closer with no container of its name open, a leaf given a body, `{attrs}` out of order or duplicated, invalid JSON in an opaque carry | write the spelling the message names, or keep it literal: escape the prefix — `\!adf:`, block and inline alike — or wrap a code fence whose info string opens `adf:` in `!adf:codeBlock {language="adf:…"}` with a bare fence | +| `malformed-directive` | an `!adf:` the grammar cannot read — a prefix completing no directive, an unclosed container, `[content]` or `{attrs}`, a closer with no container of its name open, a leaf given a body, `{attrs}` out of order or duplicated, invalid JSON in an opaque carry | write the spelling the message names, or keep it literal: escape the prefix — `\!adf:`, block and inline alike — or, for a code fence whose info string opens `adf:`, drop the info string and wrap the fence in `!adf:codeBlock {language="adf:…"}` | | `malformed-pipe-table` | a pipe row that is no pipe table — a missing or ragged `---` delimiter row, an alignment colon in it, or a row not opening with a pipe | open every row with a pipe and give the delimiter row the header's cell count; to keep the lines literal text instead, escape the leading pipe of every one — escaping a single row leaves the next to open a fresh table and fail the same way | | `unknown-directive-name` | a directive whose name is no node or mark this version spells | check the name in `spec/flavour.md`, or escape the prefix as `\!adf:`; the spelling itself is well formed, so a later minor may give the name meaning | | `unmappable-html` | the input holds an HTML construct the documented element set does not map, a comment and a processing instruction among them — at this version that is every raw HTML construct in markdown, the element set landing at `0.2.0` | remove the construct, or write what it holds in the lossless flavour | @@ -198,7 +198,7 @@ emit refuses: | `unspellable-line-start` | a paragraph line begins with a code span whose backticks would read back as a code fence | put any text before the code span | | `unspellable-whitespace` | an `emoji`, `mention` or `status` holds a newline in the text its inline directive spells in the content slot | replace it with a space — an inline directive never spans lines | | `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it | -| `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark | write the shape the message names, or remove the reserved directive it names; `spec/flavour.md` lists every type's attributes and body | +| `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark, or a reserved directive (`!adf:textBreak{}`, `!adf:listBreak`, `!adf:doc`) stands where it parts nothing | make the edit the message opens with; `spec/flavour.md` lists every type's attributes and body | ## The guarantees @@ -213,12 +213,13 @@ Serves Goals 1, 3 and 4. name every exception. - Plain CommonMark is valid input to `markdownToAdf` apart from the raw HTML `unmappable-html` names, with four carve-outs — literal text matching directive, pipe-table or strikethrough syntax, - and a code fence whose info string opens `adf:`, are claimed (escapable — `spec/flavour.md`) — and - one gap: a CommonMark image fits only as its own title-less paragraph; mid-text and titled images - are error results, save an image inside another's description, which flattens into the alt text. - Converting back yields the library's canonical spelling, which round-trips byte-identically — - where it converts back at all: a parse succeeding is no promise of that, so keep the source until - the way back succeeds. ``` ` `` ` ``` reads cleanly and then refuses. + and a code fence whose info string opens `adf:`, are claimed (each can be kept literal — + `spec/flavour.md`) — and one gap: a CommonMark image fits only as its own title-less paragraph; + mid-text and titled images are error results, save an image inside another's description, which + flattens into the alt text. Converting back yields the library's canonical spelling, which + round-trips byte-identically — where it converts back at all: a parse succeeding is no promise of + that, so keep the source until the way back succeeds. ``` ` `` ` ``` reads cleanly and then + refuses. - Four CommonMark spellings parse without an error and build a document the reference implementation renders differently: `[](/url)` and `[]()` stay literal text against CommonMark's empty link, a list continuing past a marker change stays one list against CommonMark's two, a diff --git a/spec/flavour.md b/spec/flavour.md index a3170d6..9e7396b 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -4,7 +4,7 @@ The grammar of the extended markdown `adfToMarkdown` emits and `markdownToAdf` p CommonMark is a subset apart from raw HTML (below), with four carve-outs: literal text that matches directive syntax below or reads as a pipe table is claimed by the flavour, a matched `~~` pair spells `strike` (escape the `!adf:`, `|` or `~` to keep it literal), and a code fence whose info -string opens `adf:` is the opaque carry (wrap it as a bare fence in +string opens `adf:` is the opaque carry (drop the info string and wrap the fence in `!adf:codeBlock {language="adf:…"}` to keep it code) — and one gap: a CommonMark image fits only as its own title-less paragraph — mid-text and titled images are named errors. The emitted form is contract (`docs/decisions.md` §The formats are API). Per-node syntaxes build on this grammar in the -- 2.52.0 From 608ef266ed46f537c5c5c06e5c3e02507ff79dfc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:13:57 +0200 Subject: [PATCH 44/67] Reword item 53's tagline --- todo.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/todo.md b/todo.md index e4bce9a..4e44c5e 100644 --- a/todo.md +++ b/todo.md @@ -33,7 +33,7 @@ | 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 5 | 13.0 | | 51 | 0.2.0 | | **Match a reference label to its definition under Unicode case folding.** | 2 | 2 | 2 | 7 | 3, 4 | 12.4 | | 54 | 0.2.0 | | **Make `docs/decisions.md` §The source parts by ADF and format name every import `parse/` takes from `emit/`.** | 2 | 2 | 2 | 5 | 1 | 11.5 | -| 53 | 0.2.0 | | **Bench a block's `marks` spelling with the writer panel and adopt its pick.** | 4 | 5 | 5 | 6 | 5 | 11.5 | +| 53 | 0.2.0 | | **Put a block's `marks` spelling to the writer panel and adopt its pick.** | 4 | 5 | 5 | 6 | 5 | 11.5 | | 50 | 0.2.0 | | **Read `[](/url)` and `[]()` as CommonMark's empty link.** | 4 | 4 | 3 | 6 | 3, 4, 6 | 10.4 | | 38 | 0.3.0 | | **Spell a lone surrogate in a text node so it survives a UTF-8 encode.** | 2 | 2 | 4 | 7 | 1 | 19.5 | | 47 | 0.3.0 | | **Open the README with what the package is, what it does and for whom.** | 1 | 4 | 7 | 9 | 9 | 14.0 | @@ -117,7 +117,7 @@ The entry says `parse/` imports only `commonMarkSpelling` and `openingLinkTakesD `emit/`; `parse/markdown-to-adf.ts` also imports `inlineLeaves` from `emit/plain-inline.ts`. Either the entry names it with its reason, or the reader stops importing it. -### 53. Bench a block's `marks` spelling with the writer panel and adopt its pick. +### 53. Put a block's `marks` spelling to the writer panel and adopt its pick. Today `marks="[{\"attrs\":{\"mode\":\"wide\"},\"type\":\"breakout\"}]"`, the marks array as escaped JSON. Breaking where the panel picks another spelling: `MIGRATION.md`'s Spellings table -- 2.52.0 From d68f311a147de1542ea1003f7258976ef776d716 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:17:40 +0200 Subject: [PATCH 45/67] Reword the attrs=empty and language-placement refusals, quoting a language that needs it --- src/markdown/empty-keys.ts | 2 +- src/markdown/parse/markdown-to-adf.test.ts | 8 ++++++-- src/markdown/parse/markdown-to-adf.ts | 4 ++-- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/src/markdown/empty-keys.ts b/src/markdown/empty-keys.ts index 08050d2..5aba8e7 100644 --- a/src/markdown/empty-keys.ts +++ b/src/markdown/empty-keys.ts @@ -27,6 +27,6 @@ export function readEmptyKeys(type: string, attributes: DirectiveAttributes, key empty.add(key) rest.delete(key) } - if (empty.has('attrs') && (held || [...rest.keys()].some((key) => key !== 'marks'))) return { fault: unsupportedNodeShape(`${type} spells attrs=empty beside another attribute, an argument or content: drop attrs=empty or the rest`) } + if (empty.has('attrs') && (held || [...rest.keys()].some((key) => key !== 'marks'))) return { fault: unsupportedNodeShape(`${type} spells attrs=empty beside another attribute, an argument or a [content] slot: drop attrs=empty, or the rest`) } return { value: { empty, rest } } } diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 396602f..b542d01 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -166,7 +166,11 @@ test('reads the codeBlock directive body as the node content, the info string it test('names the slot a codeBlock spells its language outside of', () => { const slot = 'unsupported-node-shape: codeBlock spells its language in the fence info string, or in the attribute where no info string carries it back' - assert.equal(content(markdownToAdf('!adf:codeBlock {language=rust wrap=true}\n```\nx\n```\n!adf:/codeBlock\n')), "unsupported-node-shape: move language=rust to the fence's info string: a fence carries the language wherever its info string can") + assert.equal(content(markdownToAdf('!adf:codeBlock {language=rust wrap=true}\n```\nx\n```\n!adf:/codeBlock\n')), "unsupported-node-shape: move language=rust to the fences' info strings: they can spell this language") + assert.equal( + content(markdownToAdf('!adf:codeBlock {language="c sharp"}\n```\nx\n```\n```\ny\n```\n!adf:/codeBlock\n')), + "unsupported-node-shape: move language=\"c sharp\" to the fences' info strings: they can spell this language", + ) assert.equal(content(markdownToAdf('!adf:codeBlock {language=rust}\n```sql\nx\n```\n!adf:/codeBlock\n')), slot) assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n```adf:x\nx\n```\n!adf:/codeBlock\n')), slot) assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n```a\\b\nx\n```\n!adf:/codeBlock\n')), slot) @@ -1116,6 +1120,6 @@ test('leads a refusal ordinary editing meets with the edit that fixes it', () => ) assert.equal( content(markdownToAdf('!adf:panel info {attrs=empty}\nText.\n!adf:/panel\n')), - 'unsupported-node-shape: panel spells attrs=empty beside another attribute, an argument or content: drop attrs=empty or the rest', + 'unsupported-node-shape: panel spells attrs=empty beside another attribute, an argument or a [content] slot: drop attrs=empty, or the rest', ) }) diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index d88db4a..1df55ff 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -18,8 +18,8 @@ import { mintTaskIds } from './task-ids.ts' import { parseBlocks } from './blocks.ts' import { parseInlineContent } from './inline-content.ts' import { readBlockDirectiveNode } from './directive-nodes.ts' +import { spellStringAttribute, unsupportedNodeShape } from '../directive-syntax.ts' import { spellsEmpty } from '../empty-keys.ts' -import { unsupportedNodeShape } from '../directive-syntax.ts' type Paragraph = Extract @@ -257,7 +257,7 @@ function codeDirectiveNode(node: AdfNode, blocks: readonly Block[], path: Conver const fromFence = first.language !== '' const slot = languageSlot(fromFence ? first.language : attribute) if (!fromFence && slot.kind === 'fence') { - return failure('unsupported-node-shape', `move language=${slot.info} to the fence's info string: a fence carries the language wherever its info string can`, path) + return failure('unsupported-node-shape', `move language=${spellStringAttribute(slot.info)} to the fences' info strings: they can spell this language`, path) } if ((slot.kind === 'fence') !== fromFence || (fromFence && attribute !== undefined)) { return failure('unsupported-node-shape', `${node.type} spells its language in the fence info string, or in the attribute where no info string carries it back`, path) -- 2.52.0 From 4349df9017aaa7cc2549f4d872dd45d47120cabc Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:17:53 +0200 Subject: [PATCH 46/67] Final wording: the reserved-directive refusal row, the adf:-alone fence, the changelog's carry and scope, the developer panel, the migration's empty shape --- AGENTS.md | 4 ++-- CHANGELOG.md | 16 +++++++--------- MIGRATION.md | 2 +- README.md | 2 +- docs/decisions.md | 6 +++--- spec/flavour.md | 5 +++-- src/conformance/property-harness.ts | 2 +- 7 files changed, 18 insertions(+), 19 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index af7002b..39b5314 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -175,8 +175,8 @@ settle it; otherwise four more read, five of seven settle it, and less is a miss The verdict lands in `docs/decisions.md`. A writer panel settles every new or changed markdown or HTML spelling: a reader panel whose readers -are the people who read and write that format (README `## Audience`). A question about what an app -relies on goes to the developer personas instead. +are the people who read and write that format (README `## Audience`). A panel of the developer +personas settles a question about what an app relies on. ### Stated numbers diff --git a/CHANGELOG.md b/CHANGELOG.md index a10c5de..3123e48 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,20 +5,18 @@ - **Breaking:** directives, the inline opaque carry among them (now `!adf:carry{json="…"}`), are spelled under an `!adf:` prefix (`!adf:name … !adf:/name`, `!adf:name[content]{attrs}`, `!adf:name arg {attrs}`) in place of the `:::`/`::`/`:name` forms: text holding an unescaped - `!adf:` is claimed, and `adf` is an ordinary code block language. Convert stored markdown per - `MIGRATION.md`. + `!adf:` is claimed. Convert stored markdown per `MIGRATION.md`. - **Breaking:** the block carry is a code fence whose info string `adf:` names the node's type, its body the node's JSON without `type`, and a code fence whose info string opens `adf:` is - claimed. Convert stored markdown per `MIGRATION.md`. + claimed, and `adf` is an ordinary code block language. Convert stored markdown per `MIGRATION.md`. - **Breaking:** `markdownToAdf` and `plainMarkdownToAdf` read markdown holding no block as a document whose `content` is empty, as Atlassian's schema requires; `!adf:doc {content=none}` spells a document holding no `content` key. -- `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc` as JSON, for a document of plain objects as - `JSON.parse` builds them: two adjacent text nodes CommonMark would read back as one are parted by - `!adf:textBreak{}`, an empty `attrs`, `content` or `marks` is spelled `{attrs=empty}`, - `{content=empty}` or `{marks=empty}`, `-0` is spelled `-0`, and a `codeBlock` of several text - nodes is a fence per node. A `codeBlock` holding other than plain text nodes rides the block - carry, where it was refused. +- `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc` as `JSON.parse` builds it: two adjacent text + nodes CommonMark would read back as one are parted by `!adf:textBreak{}`, an empty `attrs`, + `content` or `marks` is spelled `{attrs=empty}`, `{content=empty}` or `{marks=empty}`, `-0` is + spelled `-0`, and a `codeBlock` of several text nodes is a fence per node. A `codeBlock` holding + other than plain text nodes rides the block carry, where it was refused. - **Breaking:** `unspellable-link` leaves `ConvertErrorCode`; a link whose `href` or `title` no CommonMark escape spells is written as `!adf:link[text]{attrs}`. - **Breaking:** some directive refusals carry `malformed-directive` where they carried diff --git a/MIGRATION.md b/MIGRATION.md index d213215..a5fb98d 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -7,7 +7,7 @@ turning each directive into text and each carried node into an `adf` code block. reads any `0.1.0` markdown, convert what is stored or in flight (an open editor, a queue) with the recipe below, and rewrite markdown your code writes or matches (templates, prompts, patterns) by the tables below. Stored ADF needs one change: give a document holding no `content` key `content: []`. -`0.1.0` read empty markdown to that shape, meaning the empty document. +`0.1.0` built that shape from empty markdown, meaning the empty document. ### Convert markdown diff --git a/README.md b/README.md index 0fb2474..37874cf 100644 --- a/README.md +++ b/README.md @@ -198,7 +198,7 @@ emit refuses: | `unspellable-line-start` | a paragraph line begins with a code span whose backticks would read back as a code fence | put any text before the code span | | `unspellable-whitespace` | an `emoji`, `mention` or `status` holds a newline in the text its inline directive spells in the content slot | replace it with a space — an inline directive never spans lines | | `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it | -| `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark, or a reserved directive (`!adf:textBreak{}`, `!adf:listBreak`, `!adf:doc`) stands where it parts nothing | make the edit the message opens with; `spec/flavour.md` lists every type's attributes and body | +| `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark, or a reserved directive stands out of place: `!adf:textBreak{}` or `!adf:listBreak` parting nothing, `!adf:doc` beside another block | fix what the message names, or give the node the shape it names; `spec/flavour.md` lists every type's attributes and body | ## The guarantees diff --git a/docs/decisions.md b/docs/decisions.md index 19c83f3..b8f09d3 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -110,9 +110,9 @@ node's JSON without `type`: ```` ```adf:blockCard ````. A type no info string ca rule a code language follows — leaves the info string `adf:` and keeps `type` in the body. Every info string opening `adf:` is reserved, so a `codeBlock` whose language opens so takes the `language` attribute, and `carry` is an ordinary language. A body holding `type` under a named type, -or a bare `adf:` fence whose body's `type` an info string could carry, is `unsupported-node-shape`. -The reservation claims a fence CommonMark reads as code until `todo.md` item 43 gives CommonMark its -own reader. +or a fence whose info string is `adf:` alone while its body's `type` could be spelled in the info +string, is `unsupported-node-shape`. The reservation claims a fence CommonMark reads as code until +`todo.md` item 43 gives CommonMark its own reader. ## A code block is a fence per text node diff --git a/spec/flavour.md b/spec/flavour.md index 9e7396b..3c4b6fc 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -181,8 +181,9 @@ product). Block and inline positions canonicalize differently, each fitting wher - **Block position**: a fenced code block with info string `adf:` and the node's type, body = the node's JSON without its `type` — two-space indent, object keys sorted: ```` ```adf:blockCard ````. A type no info string carries back, by the rule a `codeBlock`'s language follows, leaves the info - string `adf:` and keeps `type` in the body. A body holding `type` under a named type, or a bare - `adf:` fence whose body's `type` an info string could carry, is a named error. + string `adf:` and keeps `type` in the body. A body holding `type` under a named type, or a fence + whose info string is `adf:` alone while its body's `type` could be spelled in the info string, is + a named error. - **Inline position**: `!adf:carry{json="…"}` — compact serialization (keys sorted, no whitespace), JSON-string-escaped into the attribute. diff --git a/src/conformance/property-harness.ts b/src/conformance/property-harness.ts index d38579e..32ddcf5 100644 --- a/src/conformance/property-harness.ts +++ b/src/conformance/property-harness.ts @@ -78,7 +78,7 @@ function heldAttributes(held: Readonly>): return attrs } -// An empty attrs, content or marks key, and adjacent text a reader would merge, stay occasional: each takes a spelling outside CommonMark. +// An empty attrs, content or marks key, and adjacent text CommonMark would read back as one, stay occasional: each takes a spelling outside CommonMark. // The copy gives fast-check's null-prototype records the prototype a parsed node has. function occasionallyEmpty(arbitrary: Arbitrary): Arbitrary { return fc.tuple(arbitrary, fc.nat({ max: 9 })).map(([held, roll]) => (roll === 0 ? { ...held } : withoutEmptyKeys(held))) -- 2.52.0 From 6df17a7069a1670e29a26a9313d79781b67536bf Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:50:24 +0200 Subject: [PATCH 47/67] Gather the plain flavour's code in markdown/plain/ --- docs/decisions.md | 7 +++++-- src/conformance/adf-property.test.ts | 4 ++-- src/conformance/property-harness.ts | 2 +- src/index.ts | 2 +- src/markdown/emit/adf-to-markdown.ts | 4 ++-- src/markdown/emit/inline-line.ts | 4 ++-- src/markdown/emit/line-escaping.ts | 4 ++-- src/markdown/emit/pipe-table.ts | 2 +- src/markdown/parse/inline-content.ts | 4 ++-- src/markdown/parse/markdown-to-adf.ts | 8 ++++---- src/markdown/parse/plain-markdown-to-adf.test.ts | 2 +- .../adf-to-plain-markdown.test.ts} | 2 +- .../plain-reduction.ts => plain/adf-to-plain-markdown.ts} | 8 ++++---- .../{plain-conventions.ts => plain/conventions.ts} | 2 +- src/{adf => markdown/plain}/editor-normal.test.ts | 4 ++-- src/{adf => markdown/plain}/editor-normal.ts | 6 +++--- .../{emit/plain-inline.ts => plain/inline-reduction.ts} | 6 +++--- src/markdown/{parse => plain}/task-ids.test.ts | 0 src/markdown/{parse => plain}/task-ids.ts | 0 todo.md | 2 +- 20 files changed, 38 insertions(+), 35 deletions(-) rename src/markdown/{emit/plain-reduction.test.ts => plain/adf-to-plain-markdown.test.ts} (99%) rename src/markdown/{emit/plain-reduction.ts => plain/adf-to-plain-markdown.ts} (99%) rename src/markdown/{plain-conventions.ts => plain/conventions.ts} (97%) rename src/{adf => markdown/plain}/editor-normal.test.ts (97%) rename src/{adf => markdown/plain}/editor-normal.ts (95%) rename src/markdown/{emit/plain-inline.ts => plain/inline-reduction.ts} (98%) rename src/markdown/{parse => plain}/task-ids.test.ts (100%) rename src/markdown/{parse => plain}/task-ids.ts (100%) diff --git a/docs/decisions.md b/docs/decisions.md index b8f09d3..74cce04 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -642,8 +642,8 @@ is the markdown flavour's choice, not ADF's. ## The source parts by ADF and format -2026-08-27, placement 2026-09-18, `adf/`'s bar 2026-09-21, the maintainer. Goal 2. Valid while -each format has a reader and a writer through ADF. +2026-08-27, placement 2026-09-18, `adf/`'s bar 2026-09-21, `plain/` 2026-10-03, the maintainer. +Goal 2. Valid while each format has a reader and a writer through ADF. `src/adf/` holds ADF's own knowledge, imports no format, and is where a construct both formats read lives: the question is answered in ADF's vocabulary — a node type, an attribute kind, a content @@ -666,3 +666,6 @@ conservative the copy would be. Where the rule is the emitter's own choice, inpu rather than restating it, and that is the only import `parse/` takes from `emit/` — `commonMarkSpelling` and `openingLinkTakesDirective` — so no fixture the emitter writes can be refused, and a spelling the emitter refuses gives its own error rather than a second name for it. +`markdown/plain/` holds the plain flavour's own code — the writer's reduction to editor-normal ADF +and the conventions and task ids the reader takes — so neither direction reaches into the other +for it. diff --git a/src/conformance/adf-property.test.ts b/src/conformance/adf-property.test.ts index 4b787ac..96a8c83 100644 --- a/src/conformance/adf-property.test.ts +++ b/src/conformance/adf-property.test.ts @@ -5,10 +5,10 @@ import test from 'node:test' import type { AdfNode } from '../adf/document.ts' import { adfDocument, propertyRuns, propertyTimeout } from './property-harness.ts' import { adfToMarkdown } from '../markdown/emit/adf-to-markdown.ts' -import { adfToPlainMarkdown, reduceToPlain } from '../markdown/emit/plain-reduction.ts' +import { adfToPlainMarkdown, reduceToPlain } from '../markdown/plain/adf-to-plain-markdown.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' import { markdownToAdf, plainMarkdownToAdf } from '../markdown/parse/markdown-to-adf.ts' -import { toEditorNormal } from '../adf/editor-normal.ts' +import { toEditorNormal } from '../markdown/plain/editor-normal.ts' const gateRuns = 1600 const renamedPrefix = '!adg:' diff --git a/src/conformance/property-harness.ts b/src/conformance/property-harness.ts index 32ddcf5..0743e5a 100644 --- a/src/conformance/property-harness.ts +++ b/src/conformance/property-harness.ts @@ -11,7 +11,7 @@ import { blockNodes } from '../adf/block-nodes.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' import { emptyKeys, mergeAdjacentText } from '../adf/document.ts' import { inlineNodes } from '../adf/inline-nodes.ts' -import { joinsNormally } from '../adf/editor-normal.ts' +import { joinsNormally } from '../markdown/plain/editor-normal.ts' import { markAttributes } from '../adf/mark-attributes.ts' type Positions = { block: AdfNode; inline: AdfNode } diff --git a/src/index.ts b/src/index.ts index a25f670..e4de874 100644 --- a/src/index.ts +++ b/src/index.ts @@ -3,6 +3,6 @@ export type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from './adf/documen export type { ConvertError, ConvertErrorCode, ConvertErrorPath, ParseError, Result, SourcePosition } from './result.ts' export type { JsonValue } from './json-value.ts' export { adfToMarkdown } from './markdown/emit/adf-to-markdown.ts' -export { adfToPlainMarkdown } from './markdown/emit/plain-reduction.ts' +export { adfToPlainMarkdown } from './markdown/plain/adf-to-plain-markdown.ts' export { isAdfDocument } from './adf/document.ts' export { markdownToAdf, plainMarkdownToAdf } from './markdown/parse/markdown-to-adf.ts' diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index 867dfc8..99331ec 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -1,8 +1,8 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { BlockNodeModel } from '../../adf/block-nodes.ts' -import type { Flavour } from '../plain-conventions.ts' +import type { Flavour } from '../plain/conventions.ts' import { adfDocumentFault, carriesOnly, isPlainText, nodeAttrs, nodeContent } from '../../adf/document.ts' -import { alertMarker, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../plain-conventions.ts' +import { alertMarker, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../plain/conventions.ts' import { blockDirectiveForm, documentSpelling, listBreakSpelling } from '../block-directive.ts' import { blockNodeModel, blockNodes } from '../../adf/block-nodes.ts' import { carriedBlock } from '../opaque-carry.ts' diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index 6be52b0..6513e8d 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -1,5 +1,5 @@ import type { AdfMark, AdfNode } from '../../adf/document.ts' -import type { Flavour } from '../plain-conventions.ts' +import type { Flavour } from '../plain/conventions.ts' import type { InlineNodeModel } from '../../adf/inline-nodes.ts' import type { LineContainer } from '../line-container.ts' import { assembleInlineLine, isSyntax, type InlineEscaping, type InlineSegment, type MarkRun, type NodeRange } from './line-escaping.ts' @@ -9,7 +9,7 @@ import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../ import { identicalMark, isPlainText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { escapeUnbalanced, spellDestination } from '../commonmark/link-syntax.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' -import { highlightDelimiter } from '../plain-conventions.ts' +import { highlightDelimiter } from '../plain/conventions.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { largestNesting } from '../../nesting.ts' import { longestBacktickRun } from '../commonmark/backtick-runs.ts' diff --git a/src/markdown/emit/line-escaping.ts b/src/markdown/emit/line-escaping.ts index 253df18..120faf8 100644 --- a/src/markdown/emit/line-escaping.ts +++ b/src/markdown/emit/line-escaping.ts @@ -1,10 +1,10 @@ -import type { Flavour } from '../plain-conventions.ts' +import type { Flavour } from '../plain/conventions.ts' import type { LineContainer } from '../line-container.ts' import { backslashEscape, escapesLineClaim, inlineHtmlConstruct, opensBracketedAutolink, opensEmailAutolink, type LinePosition } from '../commonmark/grammar.ts' import { backtickRun, closingBacktickRun } from '../commonmark/backtick-runs.ts' import { claimsDirectivePrefix } from '../directive-syntax.ts' import { delimiterFlags, isWordCharacter, matchEmphasis, runLength } from '../commonmark/emphasis-matching.ts' -import { highlightDelimiter, highlightFlanking } from '../plain-conventions.ts' +import { highlightDelimiter, highlightFlanking } from '../plain/conventions.ts' import { isBareDelimiterRow } from '../pipe-table-syntax.ts' import { opensLinkDefinition } from '../commonmark/link-reference-definitions.ts' import { readEntityReference } from '../commonmark/entity-references.ts' diff --git a/src/markdown/emit/pipe-table.ts b/src/markdown/emit/pipe-table.ts index f200e6d..89ab454 100644 --- a/src/markdown/emit/pipe-table.ts +++ b/src/markdown/emit/pipe-table.ts @@ -1,6 +1,6 @@ import type { AdfNode } from '../../adf/document.ts' import type { ConvertErrorPath } from '../../result.ts' -import type { Flavour } from '../plain-conventions.ts' +import type { Flavour } from '../plain/conventions.ts' import { carriesOnly, nodeContent } from '../../adf/document.ts' import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts' import { tryPipeCell } from './inline-line.ts' diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index b2bfd69..22d7114 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -1,7 +1,7 @@ import type { AdfMark, AdfNode } from '../../adf/document.ts' import type { DirectiveSpan, NestedSpans } from '../directive-syntax.ts' import type { EmphasisPairing } from '../commonmark/emphasis-matching.ts' -import type { Flavour } from '../plain-conventions.ts' +import type { Flavour } from '../plain/conventions.ts' import type { LineContainer } from '../line-container.ts' import type { LinkDefinition } from '../commonmark/link-syntax.ts' import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct, readBracketedAutolink, readEmailAutolink, trimTrailingSpace } from '../commonmark/grammar.ts' @@ -9,7 +9,7 @@ import { backtickRun, closingBacktickRun } from '../commonmark/backtick-runs.ts' import { commonMarkLink, linkHref } from '../mark-spellings.ts' import { delimiterFlags, matchEmphasis, runLength } from '../commonmark/emphasis-matching.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' -import { highlightDelimiter, highlightFlanking } from '../plain-conventions.ts' +import { highlightDelimiter, highlightFlanking } from '../plain/conventions.ts' import { identicalMarks, mergeAdjacentText, nodeAttrs, nodeMarks } from '../../adf/document.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { noSpans, readInlineDirective } from '../directive-syntax.ts' diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 1df55ff..26907f8 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -2,7 +2,7 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { Block, DirectiveBlock } from './blocks.ts' import type { BlockDirectiveNode } from './directive-nodes.ts' import type { ConvertFault } from '../../result.ts' -import type { Flavour } from '../plain-conventions.ts' +import type { Flavour } from '../plain/conventions.ts' import type { LineContainer } from '../line-container.ts' import type { LinkDefinitions } from './inline-content.ts' import { carryFenceType, readCarriedBlock } from '../opaque-carry.ts' @@ -10,11 +10,11 @@ import { commonMarkSpelling, type SpellingMemo } from '../emit/adf-to-markdown.t import { documentAttribute, documentName, documentSpelling, listBreakName, listBreakSpelling } from '../block-directive.ts' import { emptyKeys, nodeAttrs, nodeContent } from '../../adf/document.ts' import { failure, faulted, positioned, success, type ConvertErrorPath, type ParseError, type Result, type SourcePosition } from '../../result.ts' -import { inlineLeaves } from '../emit/plain-inline.ts' +import { inlineLeaves } from '../plain/inline-reduction.ts' import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' -import { leadingMarker, readAlertMarker, readTaskMarker } from '../plain-conventions.ts' -import { mintTaskIds } from './task-ids.ts' +import { leadingMarker, readAlertMarker, readTaskMarker } from '../plain/conventions.ts' +import { mintTaskIds } from '../plain/task-ids.ts' import { parseBlocks } from './blocks.ts' import { parseInlineContent } from './inline-content.ts' import { readBlockDirectiveNode } from './directive-nodes.ts' diff --git a/src/markdown/parse/plain-markdown-to-adf.test.ts b/src/markdown/parse/plain-markdown-to-adf.test.ts index ae1f96d..af6aee2 100644 --- a/src/markdown/parse/plain-markdown-to-adf.test.ts +++ b/src/markdown/parse/plain-markdown-to-adf.test.ts @@ -3,7 +3,7 @@ import test from 'node:test' import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' import { adfToMarkdown } from '../emit/adf-to-markdown.ts' -import { adfToPlainMarkdown } from '../emit/plain-reduction.ts' +import { adfToPlainMarkdown } from '../plain/adf-to-plain-markdown.ts' import { largestNesting } from '../../nesting.ts' import { markdownToAdf, plainMarkdownToAdf } from './markdown-to-adf.ts' diff --git a/src/markdown/emit/plain-reduction.test.ts b/src/markdown/plain/adf-to-plain-markdown.test.ts similarity index 99% rename from src/markdown/emit/plain-reduction.test.ts rename to src/markdown/plain/adf-to-plain-markdown.test.ts index 185ae42..28db1b7 100644 --- a/src/markdown/emit/plain-reduction.test.ts +++ b/src/markdown/plain/adf-to-plain-markdown.test.ts @@ -2,7 +2,7 @@ import assert from 'node:assert/strict' import test from 'node:test' import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' -import { adfToPlainMarkdown, reduceToPlain } from './plain-reduction.ts' +import { adfToPlainMarkdown, reduceToPlain } from './adf-to-plain-markdown.ts' import { largestNesting } from '../../nesting.ts' const code: AdfMark = { type: 'code' } diff --git a/src/markdown/emit/plain-reduction.ts b/src/markdown/plain/adf-to-plain-markdown.ts similarity index 99% rename from src/markdown/emit/plain-reduction.ts rename to src/markdown/plain/adf-to-plain-markdown.ts index 2545002..f324ba3 100644 --- a/src/markdown/emit/plain-reduction.ts +++ b/src/markdown/plain/adf-to-plain-markdown.ts @@ -1,14 +1,14 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import { adfDocumentFault, nodeAttrs, nodeContent } from '../../adf/document.ts' -import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from './adf-to-markdown.ts' +import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from '../emit/adf-to-markdown.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' -import { inlineLeaves, isBlockNodeType, oneLine, reduceInline, writableHref } from './plain-inline.ts' +import { inlineLeaves, isBlockNodeType, oneLine, reduceInline, writableHref } from './inline-reduction.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' -import { taskMarker } from '../plain-conventions.ts' -import { toEditorNormal } from '../../adf/editor-normal.ts' +import { taskMarker } from './conventions.ts' +import { toEditorNormal } from './editor-normal.ts' // depth: the level the node reduced stands at, counted as the emitter counts it. type Reduction = { depth: number; memo: SpellingMemo; path: ConvertErrorPath } diff --git a/src/markdown/plain-conventions.ts b/src/markdown/plain/conventions.ts similarity index 97% rename from src/markdown/plain-conventions.ts rename to src/markdown/plain/conventions.ts index c5d6f09..be6c5c7 100644 --- a/src/markdown/plain-conventions.ts +++ b/src/markdown/plain/conventions.ts @@ -1,4 +1,4 @@ -import { isWordCharacter } from './commonmark/emphasis-matching.ts' +import { isWordCharacter } from '../commonmark/emphasis-matching.ts' export type Flavour = 'lossless' | 'plain' diff --git a/src/adf/editor-normal.test.ts b/src/markdown/plain/editor-normal.test.ts similarity index 97% rename from src/adf/editor-normal.test.ts rename to src/markdown/plain/editor-normal.test.ts index bb0dd40..33d538a 100644 --- a/src/adf/editor-normal.test.ts +++ b/src/markdown/plain/editor-normal.test.ts @@ -1,8 +1,8 @@ import assert from 'node:assert/strict' import test from 'node:test' -import type { AdfNode } from './document.ts' -import type { JsonValue } from '../json-value.ts' +import type { AdfNode } from '../../adf/document.ts' +import type { JsonValue } from '../../json-value.ts' import { toEditorNormal } from './editor-normal.ts' test('merges adjacent text nodes carrying identical marks, at every level', () => { diff --git a/src/adf/editor-normal.ts b/src/markdown/plain/editor-normal.ts similarity index 95% rename from src/adf/editor-normal.ts rename to src/markdown/plain/editor-normal.ts index a89c0df..a5ffb7e 100644 --- a/src/adf/editor-normal.ts +++ b/src/markdown/plain/editor-normal.ts @@ -1,6 +1,6 @@ -import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from './document.ts' -import type { JsonValue } from '../json-value.ts' -import { identicalMark, identicalMarks, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from './document.ts' +import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' +import type { JsonValue } from '../../json-value.ts' +import { identicalMark, identicalMarks, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' type JsonContainer = JsonValue[] | { [key: string]: JsonValue } diff --git a/src/markdown/emit/plain-inline.ts b/src/markdown/plain/inline-reduction.ts similarity index 98% rename from src/markdown/emit/plain-inline.ts rename to src/markdown/plain/inline-reduction.ts index f8e89b5..b2d3501 100644 --- a/src/markdown/emit/plain-inline.ts +++ b/src/markdown/plain/inline-reduction.ts @@ -1,12 +1,12 @@ import type { AdfAttributes, AdfMark, AdfNode } from '../../adf/document.ts' import type { LineContainer } from '../line-container.ts' -import type { MarkRun } from './line-escaping.ts' +import type { MarkRun } from '../emit/line-escaping.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' -import { joinsNormally, sameMark } from '../../adf/editor-normal.ts' +import { joinsNormally, sameMark } from './editor-normal.ts' import { largestNesting } from '../../nesting.ts' import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' -import { plainLineFallback, type PlainLineFallback } from './inline-line.ts' +import { plainLineFallback, type PlainLineFallback } from '../emit/inline-line.ts' import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts' const highlight = 'backgroundColor' diff --git a/src/markdown/parse/task-ids.test.ts b/src/markdown/plain/task-ids.test.ts similarity index 100% rename from src/markdown/parse/task-ids.test.ts rename to src/markdown/plain/task-ids.test.ts diff --git a/src/markdown/parse/task-ids.ts b/src/markdown/plain/task-ids.ts similarity index 100% rename from src/markdown/parse/task-ids.ts rename to src/markdown/plain/task-ids.ts diff --git a/todo.md b/todo.md index 4e44c5e..f775b28 100644 --- a/todo.md +++ b/todo.md @@ -154,7 +154,7 @@ punctuation — and, where they do, read the code point, with a fixture per dire ### 42. Trim a text leaf's trailing blanks in linear time. -`plain-inline.ts`'s `leafEdges` finds the trail with an unanchored `/[ \t]*$/`, quadratic in a run +`plain/inline-reduction.ts`'s `leafEdges` finds the trail with an unanchored `/[ \t]*$/`, quadratic in a run of blanks inside one leaf: a paragraph of `a`, 80 000 spaces, `b` takes 6.5 s in `adfToPlainMarkdown`. Scan backward, as the expand title's trim does. -- 2.52.0 From 4579f32fcdec8b2bb0b10e9ce1ea065fa60f4c80 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:51:41 +0200 Subject: [PATCH 48/67] Name the two text-joining rules for whose rule they are, side by side in adjacent-text.ts --- src/conformance/property-harness.ts | 4 +- src/markdown/adjacent-text.ts | 53 ++++++++++++++++++++++++++ src/markdown/emit/inline-line.ts | 8 ++-- src/markdown/parse/directive-nodes.ts | 2 +- src/markdown/parse/inline-content.ts | 12 +++--- src/markdown/plain/editor-normal.ts | 47 ++--------------------- src/markdown/plain/inline-reduction.ts | 10 ++--- src/markdown/text-break.ts | 12 ------ 8 files changed, 75 insertions(+), 73 deletions(-) create mode 100644 src/markdown/adjacent-text.ts delete mode 100644 src/markdown/text-break.ts diff --git a/src/conformance/property-harness.ts b/src/conformance/property-harness.ts index 0743e5a..6951572 100644 --- a/src/conformance/property-harness.ts +++ b/src/conformance/property-harness.ts @@ -11,7 +11,7 @@ import { blockNodes } from '../adf/block-nodes.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' import { emptyKeys, mergeAdjacentText } from '../adf/document.ts' import { inlineNodes } from '../adf/inline-nodes.ts' -import { joinsNormally } from '../markdown/plain/editor-normal.ts' +import { joinsWhenEditorNormal } from '../markdown/adjacent-text.ts' import { markAttributes } from '../adf/mark-attributes.ts' type Positions = { block: AdfNode; inline: AdfNode } @@ -91,7 +91,7 @@ function withoutEmptyKeys(held: T): T { } function occasionallyApart(arbitrary: Arbitrary): Arbitrary { - return fc.tuple(arbitrary, fc.nat({ max: 5 })).map(([nodes, roll]) => (roll === 0 ? nodes : mergeAdjacentText(nodes, joinsNormally))) + return fc.tuple(arbitrary, fc.nat({ max: 5 })).map(([nodes, roll]) => (roll === 0 ? nodes : mergeAdjacentText(nodes, joinsWhenEditorNormal))) } function pipeTable({ body, header }: { body: AdfNode[][]; header: AdfNode[] }): AdfNode { diff --git a/src/markdown/adjacent-text.ts b/src/markdown/adjacent-text.ts new file mode 100644 index 0000000..b063caa --- /dev/null +++ b/src/markdown/adjacent-text.ts @@ -0,0 +1,53 @@ +import type { AdfAttributes, AdfMark, AdfNode } from '../adf/document.ts' +import type { JsonValue } from '../json-value.ts' +import { identicalMark, identicalMarks, isPlainText, nodeAttrs, nodeMarks } from '../adf/document.ts' +import { spellInlineLeafDirective } from './directive-syntax.ts' + +type JsonContainer = JsonValue[] | { [key: string]: JsonValue } + +export const textBreakName = 'textBreak' + +export const textBreakSpelling = spellInlineLeafDirective(textBreakName, '') + +// The lossless reader's rule: CommonMark reads the pair back as one text run (spec/flavour.md, Inline nodes). +export function joinsWhenRead(previous: AdfNode, node: AdfNode): boolean { + return isPlainText(previous) && isPlainText(node) && identicalMarks(nodeMarks(previous), nodeMarks(node)) +} + +// The editor's rule, parting from the reader's only on shapes editor-normal ADF erases: an empty attrs, content or marks, and -0 in a mark. +export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean { + if (!joinsAsEditorText(previous) || !joinsAsEditorText(node)) return false + return identicalMarks(nodeMarks(previous).map(normalMark), nodeMarks(node).map(normalMark)) +} + +export function sameMarkWhenEditorNormal(left: AdfMark, right: AdfMark): boolean { + return identicalMark(normalMark(left), normalMark(right)) +} + +export function normalMark(mark: AdfMark): AdfMark { + const attrs = normalAttributes(nodeAttrs(mark)) + return attrs === undefined ? { type: mark.type } : { attrs, type: mark.type } +} + +export function normalAttributes(attrs: AdfAttributes): AdfAttributes | undefined { + if (Object.keys(attrs).length === 0) return undefined + const normal = { ...attrs } + const pending: JsonContainer[] = [normal] + for (let held = pending.pop(); held !== undefined; held = pending.pop()) { + if (Array.isArray(held)) for (const [index, value] of held.entries()) held[index] = normalValue(value, pending) + else for (const [key, value] of Object.entries(held)) held[key] = normalValue(value, pending) + } + return normal +} + +function normalValue(value: JsonValue, pending: JsonContainer[]): JsonValue { + if (Object.is(value, -0)) return 0 + if (value === null || typeof value !== 'object') return value + const copy = Array.isArray(value) ? [...value] : { ...value } + pending.push(copy) + return copy +} + +function joinsAsEditorText(node: AdfNode): boolean { + return node.type === 'text' && Object.keys(nodeAttrs(node)).length === 0 +} diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index 6513e8d..c4ce116 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -11,9 +11,9 @@ import { escapeUnbalanced, spellDestination } from '../commonmark/link-syntax.ts import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { highlightDelimiter } from '../plain/conventions.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' +import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts' import { largestNesting } from '../../nesting.ts' import { longestBacktickRun } from '../commonmark/backtick-runs.ts' -import { readsAsOne, textBreakSpelling } from '../text-break.ts' import { slotLineEndingFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts' import { spellInlineNodeAttributes } from './inline-directive-spelling.ts' import { spellTextDirective } from '../text-directive.ts' @@ -186,7 +186,7 @@ function emitRun(nodes: readonly AdfNode[], depth: number, firstIndex: number, c const runs = inlineRuns(nodes, depth, firstIndex, context.carried) const segments: InlineSegment[] = [] for (const [offset, run] of runs.entries()) { - if (partsText(runs[offset - 1], run, context.carried)) segments.push(syntax(textBreakSpelling)) + if (takesTextBreak(runs[offset - 1], run, context.carried)) segments.push(syntax(textBreakSpelling)) const runContext = { ...context, atBlockEnd: context.atBlockEnd && offset === runs.length - 1 } const emitted = run.kind === 'plain' ? emitLeaf(run.node, runContext, run.index) : emitMarkedRun(run.nodes, run.mark, depth, run.index, runContext) if (!emitted.ok) return emitted @@ -213,10 +213,10 @@ function inlineRuns(nodes: readonly AdfNode[], depth: number, firstIndex: number return runs } -function partsText(previous: InlineRun | undefined, run: InlineRun, carried: ReadonlySet): boolean { +function takesTextBreak(previous: InlineRun | undefined, run: InlineRun, carried: ReadonlySet): boolean { if (previous?.kind !== 'plain' || run.kind !== 'plain') return false if (carries(previous.node, carried, previous.index) || carries(run.node, carried, run.index)) return false - return readsAsOne(previous.node, run.node) + return joinsWhenRead(previous.node, run.node) } function nodePath(context: InlineContext, index: number): ConvertErrorPath { diff --git a/src/markdown/parse/directive-nodes.ts b/src/markdown/parse/directive-nodes.ts index 63e1e56..61c1471 100644 --- a/src/markdown/parse/directive-nodes.ts +++ b/src/markdown/parse/directive-nodes.ts @@ -14,7 +14,7 @@ import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { readEmptyKeys, spellsEmpty } from '../empty-keys.ts' import { readVocabulary } from './directive-attributes.ts' import { slotLineEndingFault } from '../directive-syntax.ts' -import { textBreakName } from '../text-break.ts' +import { textBreakName } from '../adjacent-text.ts' import { textDirectiveName } from '../text-directive.ts' export type BlockDirectiveNode = { contentModel: BlockNodeModel['contentModel']; node: AdfNode } diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index 22d7114..1d7c0db 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -12,6 +12,7 @@ import { failure, faulted, success, type ConvertErrorPath, type Result } from '. import { highlightDelimiter, highlightFlanking } from '../plain/conventions.ts' import { identicalMarks, mergeAdjacentText, nodeAttrs, nodeMarks } from '../../adf/document.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' +import { joinsWhenRead, textBreakName, textBreakSpelling } from '../adjacent-text.ts' import { noSpans, readInlineDirective } from '../directive-syntax.ts' import { normalizeLabel, readInlineTarget, readLabel } from '../commonmark/link-syntax.ts' import { openingLinkTakesDirective } from '../emit/inline-line.ts' @@ -19,7 +20,6 @@ import { readCarriedInline } from '../opaque-carry.ts' import { readDirectiveMark } from './directive-marks.ts' import { readInlineDirectiveNode } from './directive-nodes.ts' import { readTextDirective } from '../text-directive.ts' -import { readsAsOne, textBreakName, textBreakSpelling } from '../text-break.ts' export type InlineContent = { carry?: undefined; image: AdfNode; nodes?: undefined } | { carry: boolean; image?: undefined; nodes: AdfNode[] } @@ -312,7 +312,7 @@ function partText(items: readonly Inline[], scan: Scan): Result { } const previous = items[index - 1] const next = items[index + 1] - if (previous === undefined || next === undefined || !isNode(previous) || !isNode(next) || !readsAsOne(previous, next)) { + if (previous === undefined || next === undefined || !isNode(previous) || !isNode(next) || !joinsWhenRead(previous, next)) { return failure('unsupported-node-shape', `delete ${textBreakSpelling} here: it stands only between two runs of text with the same formatting, which would otherwise read as one`, scan.path) } } @@ -508,11 +508,11 @@ function resolveNodes(pieces: readonly Piece[], scan: Scan, highlights: boolean) writeUnpaired(nodes, runs, pairings) if (!markPairings(pieces, nodes, pairings)) return failure('unsupported-node-shape', carriedInMark, scan.path) markHighlights(pieces, nodes, highlights ? pairedHighlights(pieces, nodes) : []) - return success(mergeText(nodes.flat())) + return success(mergeReadText(nodes.flat())) } // A text break and a carried node are walls: the text on either side joins only its own side. -function mergeText(items: readonly Inline[]): Inline[] { +function mergeReadText(items: readonly Inline[]): Inline[] { const merged: Inline[] = [] let run: AdfNode[] = [] for (const item of items) { @@ -520,11 +520,11 @@ function mergeText(items: readonly Inline[]): Inline[] { run.push(item) continue } - for (const node of mergeAdjacentText(run, readsAsOne)) merged.push(node) + for (const node of mergeAdjacentText(run, joinsWhenRead)) merged.push(node) merged.push(item) run = [] } - for (const node of mergeAdjacentText(run, readsAsOne)) merged.push(node) + for (const node of mergeAdjacentText(run, joinsWhenRead)) merged.push(node) return merged } diff --git a/src/markdown/plain/editor-normal.ts b/src/markdown/plain/editor-normal.ts index a5ffb7e..e43a0a5 100644 --- a/src/markdown/plain/editor-normal.ts +++ b/src/markdown/plain/editor-normal.ts @@ -1,25 +1,14 @@ -import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' -import type { JsonValue } from '../../json-value.ts' -import { identicalMark, identicalMarks, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' - -type JsonContainer = JsonValue[] | { [key: string]: JsonValue } +import type { AdfDocument, AdfNode } from '../../adf/document.ts' +import { joinsWhenEditorNormal, normalAttributes, normalMark } from '../adjacent-text.ts' +import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' type NodeHolder = { content?: AdfNode[] } -export function sameMark(left: AdfMark, right: AdfMark): boolean { - return identicalMark(normalMark(left), normalMark(right)) -} - -export function joinsNormally(previous: AdfNode, node: AdfNode): boolean { - if (!mergesText(previous) || !mergesText(node)) return false - return identicalMarks(nodeMarks(previous).map(normalMark), nodeMarks(node).map(normalMark)) -} - export function toEditorNormal(document: AdfDocument): AdfDocument { const normal: AdfDocument = { type: document.type, version: Object.is(document.version, -0) ? 0 : document.version } const pending: { holder: NodeHolder; source: NodeHolder }[] = [{ holder: normal, source: document }] for (let entry = pending.pop(); entry !== undefined; entry = pending.pop()) { - const content = mergeAdjacentText(nodeContent(entry.source), joinsNormally) + const content = mergeAdjacentText(nodeContent(entry.source), joinsWhenEditorNormal) if (content.length === 0) continue entry.holder.content = content.map((source) => { const holder = normalNode(source) @@ -39,31 +28,3 @@ function normalNode(node: AdfNode): AdfNode { if (node.text !== undefined) normal.text = node.text return normal } - -function normalMark(mark: AdfMark): AdfMark { - const attrs = normalAttributes(nodeAttrs(mark)) - return attrs === undefined ? { type: mark.type } : { attrs, type: mark.type } -} - -function normalAttributes(attrs: AdfAttributes): AdfAttributes | undefined { - if (Object.keys(attrs).length === 0) return undefined - const normal = { ...attrs } - const pending: JsonContainer[] = [normal] - for (let held = pending.pop(); held !== undefined; held = pending.pop()) { - if (Array.isArray(held)) for (const [index, value] of held.entries()) held[index] = normalValue(value, pending) - else for (const [key, value] of Object.entries(held)) held[key] = normalValue(value, pending) - } - return normal -} - -function normalValue(value: JsonValue, pending: JsonContainer[]): JsonValue { - if (Object.is(value, -0)) return 0 - if (value === null || typeof value !== 'object') return value - const copy = Array.isArray(value) ? [...value] : { ...value } - pending.push(copy) - return copy -} - -function mergesText(node: AdfNode): boolean { - return node.type === 'text' && Object.keys(nodeAttrs(node)).length === 0 -} diff --git a/src/markdown/plain/inline-reduction.ts b/src/markdown/plain/inline-reduction.ts index b2d3501..0df6095 100644 --- a/src/markdown/plain/inline-reduction.ts +++ b/src/markdown/plain/inline-reduction.ts @@ -3,7 +3,7 @@ import type { LineContainer } from '../line-container.ts' import type { MarkRun } from '../emit/line-escaping.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' -import { joinsNormally, sameMark } from './editor-normal.ts' +import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from '../adjacent-text.ts' import { largestNesting } from '../../nesting.ts' import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { plainLineFallback, type PlainLineFallback } from '../emit/inline-line.ts' @@ -175,11 +175,11 @@ function highlighted(leaves: readonly AdfNode[]): AdfNode[] { const marks = nodeMarks(leaf) if (marks[0]?.type === highlight) { const held = marks.slice(1) - shared = run.length === 0 ? held : shared.filter((mark) => held.some((other) => sameMark(other, mark))) + shared = run.length === 0 ? held : shared.filter((mark) => held.some((other) => sameMarkWhenEditorNormal(other, mark))) run.push(leaf) continue } - for (const held of run) spelled.push(withMarks(held, [...shared, highlightMark, ...nodeMarks(held).slice(1).filter((mark) => !shared.some((other) => sameMark(other, mark)))])) + for (const held of run) spelled.push(withMarks(held, [...shared, highlightMark, ...nodeMarks(held).slice(1).filter((mark) => !shared.some((other) => sameMarkWhenEditorNormal(other, mark)))])) run = [] spelled.push(leaf) } @@ -193,7 +193,7 @@ function withMarks(leaf: AdfNode, marks: readonly AdfMark[]): AdfNode { function trimmedEdges(leaves: readonly AdfNode[]): AdfNode[] { for (let current = leaves; ; ) { - const merged = withoutEdgeBreaks(mergeAdjacentText(current, joinsNormally)) + const merged = withoutEdgeBreaks(mergeAdjacentText(current, joinsWhenEditorNormal)) let changed = false const trimmed: AdfNode[] = [] for (const [index, leaf] of merged.entries()) { @@ -251,7 +251,7 @@ function edgeDepth(marks: readonly AdfMark[], neighbour: AdfNode | undefined, wh function sameMarkAt(marks: readonly AdfMark[], others: readonly AdfMark[], index: number): boolean { const mark = marks[index] const other = others[index] - return mark !== undefined && other !== undefined && sameMark(mark, other) + return mark !== undefined && other !== undefined && sameMarkWhenEditorNormal(mark, other) } function spellableLine(leaves: AdfNode[], container: LineContainer, path: ConvertErrorPath): Result { diff --git a/src/markdown/text-break.ts b/src/markdown/text-break.ts deleted file mode 100644 index 2055539..0000000 --- a/src/markdown/text-break.ts +++ /dev/null @@ -1,12 +0,0 @@ -import type { AdfNode } from '../adf/document.ts' -import { identicalMarks, isPlainText, nodeMarks } from '../adf/document.ts' -import { spellInlineLeafDirective } from './directive-syntax.ts' - -export const textBreakName = 'textBreak' - -export const textBreakSpelling = spellInlineLeafDirective(textBreakName, '') - -// Whether CommonMark reads the pair back as one text node, where neither rides the carry (spec/flavour.md, Inline nodes). -export function readsAsOne(previous: AdfNode, node: AdfNode): boolean { - return isPlainText(previous) && isPlainText(node) && identicalMarks(nodeMarks(previous), nodeMarks(node)) -} -- 2.52.0 From 1d3a020926b1e92cf39fe3fe912d610b36f96756 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:52:19 +0200 Subject: [PATCH 49/67] Keep 'carry' for the opaque carry: holdsOnlyAttributes, spellEdgeWhitespace, bothCommonMark --- src/adf/document.ts | 2 +- src/markdown/emit/adf-to-markdown.ts | 22 +++++++++++----------- src/markdown/emit/image.ts | 6 +++--- src/markdown/emit/inline-line.ts | 24 ++++++++++++------------ src/markdown/emit/pipe-table.ts | 10 +++++----- 5 files changed, 32 insertions(+), 32 deletions(-) diff --git a/src/adf/document.ts b/src/adf/document.ts index 375a36f..42da38e 100644 --- a/src/adf/document.ts +++ b/src/adf/document.ts @@ -51,7 +51,7 @@ export function attributeNestingMessage(key: string, type: string): string { return `the ${key} attribute of ${type} nests deeper than the ${largestNesting} levels an attribute carries` } -export function carriesOnly(node: AdfNode, attributes: readonly string[]): boolean { +export function holdsOnlyAttributes(node: AdfNode, attributes: readonly string[]): boolean { if (nodeMarks(node).length > 0 || node.text !== undefined || emptyKeys(node).length > 0) return false return holdsOnly(nodeAttrs(node), attributes) } diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index 99331ec..5ab6977 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -1,7 +1,7 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { Flavour } from '../plain/conventions.ts' -import { adfDocumentFault, carriesOnly, isPlainText, nodeAttrs, nodeContent } from '../../adf/document.ts' +import { adfDocumentFault, holdsOnlyAttributes, isPlainText, nodeAttrs, nodeContent } from '../../adf/document.ts' import { alertMarker, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../plain/conventions.ts' import { blockDirectiveForm, documentSpelling, listBreakSpelling } from '../block-directive.ts' import { blockNodeModel, blockNodes } from '../../adf/block-nodes.ts' @@ -76,15 +76,15 @@ function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): } function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: BlockContainer): string { - const plainPair = previous.spelling !== 'directive' && next.spelling !== 'directive' - if (plainPair && next.spelling === 'list') { + const bothCommonMark = previous.spelling !== 'directive' && next.spelling !== 'directive' + if (bothCommonMark && next.spelling === 'list') { if (previous.spelling === 'list' && previous.node.type === next.node.type) { const gap = container === 'directive' ? '\n' : '\n\n' return `${gap}${listBreakSpelling}${gap}` } if (container === 'list-item') return interruptsParagraph(next.node) ? '\n' : '\n\n' } - return container === 'directive' && !plainPair ? '\n' : '\n\n' + return container === 'directive' && !bothCommonMark ? '\n' : '\n\n' } function interruptsParagraph(node: AdfNode): boolean { @@ -243,7 +243,7 @@ function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string, } function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result | undefined { - if (!carriesOnly(node, [])) return undefined + if (!holdsOnlyAttributes(node, [])) return undefined const inner = walkBlocks(nodeContent(node), path, depth + 1, writing) if (!inner.ok) return inner const text = joinBlocks(inner.value.blocks, 'document') @@ -252,7 +252,7 @@ function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, wri } function tryCodeBlock(node: AdfNode, path: ConvertErrorPath): Result | undefined { - if (!carriesOnly(node, ['language'])) return undefined + if (!holdsOnlyAttributes(node, ['language'])) return undefined const slot = languageSlot(nodeAttrs(node)['language']) if (slot.kind === 'attribute') return undefined const texts = fencedTexts(node, path) @@ -291,7 +291,7 @@ function fencedTexts(node: AdfNode, path: ConvertErrorPath): Result | undefined { - if (!carriesOnly(node, ['level'])) return undefined + if (!holdsOnlyAttributes(node, ['level'])) return undefined const level = nodeAttrs(node)['level'] if (typeof level !== 'number' || !Number.isInteger(level) || level < 1 || level > 6) return undefined const hashes = '#'.repeat(level) @@ -304,11 +304,11 @@ function tryHeading(node: AdfNode, path: ConvertErrorPath, flavour: Flavour): Re function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result | undefined { const ordered = node.type === 'orderedList' - if (!carriesOnly(node, ordered ? ['order'] : [])) return undefined + if (!holdsOnlyAttributes(node, ordered ? ['order'] : [])) return undefined const items = nodeContent(node) const start = listStart(node, items.length) if (start === undefined || items.length === 0) return undefined - if (items.some((item) => item.type !== 'listItem' || !carriesOnly(item, []))) return undefined + if (items.some((item) => item.type !== 'listItem' || !holdsOnlyAttributes(item, []))) return undefined const walked: WalkedItem[] = [] let headroom = Number.POSITIVE_INFINITY for (const [offset, item] of items.entries()) { @@ -356,13 +356,13 @@ function tryListItemLines(inner: string, marker: string): string | undefined { function tryParagraph(node: AdfNode, path: ConvertErrorPath, flavour: Flavour): Result | undefined { const content = nodeContent(node) - if (content.length === 0 || !carriesOnly(node, [])) return undefined + if (content.length === 0 || !holdsOnlyAttributes(node, [])) return undefined const line = emitInlineLine(content, 'paragraph', path, flavour) if (!line.ok) return line return success(commonMarkText(line.value)) } function tryRule(node: AdfNode): string | undefined { - if (!carriesOnly(node, []) || nodeContent(node).length > 0) return undefined + if (!holdsOnlyAttributes(node, []) || nodeContent(node).length > 0) return undefined return '---' } diff --git a/src/markdown/emit/image.ts b/src/markdown/emit/image.ts index be6828e..e9ca734 100644 --- a/src/markdown/emit/image.ts +++ b/src/markdown/emit/image.ts @@ -1,6 +1,6 @@ import type { AdfNode } from '../../adf/document.ts' import type { ConvertErrorPath } from '../../result.ts' -import { carriesOnly, nodeAttrs, nodeContent } from '../../adf/document.ts' +import { holdsOnlyAttributes, nodeAttrs, nodeContent } from '../../adf/document.ts' import { serializeCanonicalJson } from '../../canonical-json.ts' import { tryImageLine } from './inline-line.ts' @@ -16,8 +16,8 @@ export function tryImage(node: AdfNode, path: ConvertErrorPath): string | undefi function imageShape(node: AdfNode): { alt: string | undefined; url: string } | undefined { const content = nodeContent(node) const media = content[0] - if (!carriesOnly(node, ['layout']) || serializeCanonicalJson(nodeAttrs(node), 'compact') !== centeredMediaSingle) return undefined - if (media === undefined || content.length !== 1 || media.type !== 'media' || !carriesOnly(media, imageAttributes) || nodeContent(media).length > 0) return undefined + if (!holdsOnlyAttributes(node, ['layout']) || serializeCanonicalJson(nodeAttrs(node), 'compact') !== centeredMediaSingle) return undefined + if (media === undefined || content.length !== 1 || media.type !== 'media' || !holdsOnlyAttributes(media, imageAttributes) || nodeContent(media).length > 0) return undefined const attrs = nodeAttrs(media) const alt = attrs['alt'] const url = attrs['url'] diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index c4ce116..bcd3be4 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -114,7 +114,7 @@ function lineSegments(nodes: readonly AdfNode[], container: LineContainer, path: const emission = emitRun(nodes, 0, 0, context) if (!emission.ok) return emission if (emission.value.carry !== undefined) return emission - return success({ segments: carryStrippedWhitespace(emission.value.segments) }) + return success({ segments: spellEdgeWhitespace(emission.value.segments) }) } function attemptLine(segments: readonly InlineSegment[], container: LineContainer, path: ConvertErrorPath, flavour: Flavour): Result { @@ -137,32 +137,32 @@ function lineVerdict(segments: readonly InlineSegment[], container: LineContaine } // spec/flavour.md, Inline nodes. -function carryStrippedWhitespace(segments: readonly InlineSegment[]): InlineSegment[] { - const carried: InlineSegment[] = [] +function spellEdgeWhitespace(segments: readonly InlineSegment[]): InlineSegment[] { + const spelled: InlineSegment[] = [] for (const [index, segment] of segments.entries()) { const previous = segments[index - 1] const next = segments[index + 1] const leading = previous === undefined || previous.text.includes('\n') const trailing = next === undefined || next.text.includes('\n') - carried.push(...carryEdges(segment, leading, trailing)) + spelled.push(...edgeWhitespaceSegments(segment, leading, trailing)) } - return carried + return spelled } -function carryEdges(segment: InlineSegment, leading: boolean, trailing: boolean): InlineSegment[] { +function edgeWhitespaceSegments(segment: InlineSegment, leading: boolean, trailing: boolean): InlineSegment[] { if (segment.escaping !== 'backslash' && segment.escaping !== 'bracketed') return [segment] const head = leading ? (/^[ \t]+/.exec(segment.text)?.[0] ?? '') : '' const body = segment.text.slice(head.length) const middle = trailing ? trimTrailingSpace(body) : body const tail = body.slice(middle.length) const edges: InlineSegment[] = [] - if (head !== '') edges.push(carriedText(head)) + if (head !== '') edges.push(textDirectiveSegment(head)) if (middle !== '') edges.push({ escaping: segment.escaping, text: middle }) - if (tail !== '') edges.push(carriedText(tail)) + if (tail !== '') edges.push(textDirectiveSegment(tail)) return edges } -function carriedText(text: string): InlineSegment { +function textDirectiveSegment(text: string): InlineSegment { return syntax(spellTextDirective(text)) } @@ -275,7 +275,7 @@ function emitText(node: AdfNode, context: InlineContext, index: number, path: Co if (holdsNullCharacter(node.text)) return failure('unspellable-character', 'a text node holds a null character CommonMark replaces', path) const escaping: InlineEscaping = context.bracketed ? 'bracketed' : 'backslash' const parts = node.text.split(/(\n+)/).filter((part) => part !== '') - return success({ segments: parts.map((part) => (part.startsWith('\n') ? carriedText(part) : { escaping, text: part })) }) + return success({ segments: parts.map((part) => (part.startsWith('\n') ? textDirectiveSegment(part) : { escaping, text: part })) }) } function emitMarkedRun(nodes: readonly AdfNode[], mark: AdfMark, depth: number, index: number, context: InlineContext): Result { @@ -300,11 +300,11 @@ function emitEmphasis(nodes: readonly AdfNode[], spelling: string, depth: number const inner = emitRun(nodes, depth + 1, range.first, context) if (!inner.ok) return inner if (inner.value.carry !== undefined) return inner - const carried = carryStrippedWhitespace(inner.value.segments) + const spelled = spellEdgeWhitespace(inner.value.segments) return success({ segments: [ { emphasis: 'open', escaping: 'none', nodes: { ...range, depth }, text: spelling }, - ...carried, + ...spelled, { emphasis: 'close', escaping: 'none', nodes: { ...range, depth }, text: spelling }, ], }) diff --git a/src/markdown/emit/pipe-table.ts b/src/markdown/emit/pipe-table.ts index 89ab454..0998cf2 100644 --- a/src/markdown/emit/pipe-table.ts +++ b/src/markdown/emit/pipe-table.ts @@ -1,7 +1,7 @@ import type { AdfNode } from '../../adf/document.ts' import type { ConvertErrorPath } from '../../result.ts' import type { Flavour } from '../plain/conventions.ts' -import { carriesOnly, nodeContent } from '../../adf/document.ts' +import { holdsOnlyAttributes, nodeContent } from '../../adf/document.ts' import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts' import { tryPipeCell } from './inline-line.ts' @@ -26,16 +26,16 @@ export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: Fla function pipeRows(node: AdfNode): AdfNode[][] | undefined { const rows = nodeContent(node) const columns = rows[0] === undefined ? 0 : nodeContent(rows[0]).length - if (!carriesOnly(node, []) || columns === 0) return undefined + if (!holdsOnlyAttributes(node, []) || columns === 0) return undefined const grid: AdfNode[][] = [] for (const [index, row] of rows.entries()) { const cells = nodeContent(row) - if (row.type !== 'tableRow' || !carriesOnly(row, []) || cells.length !== columns) return undefined + if (row.type !== 'tableRow' || !holdsOnlyAttributes(row, []) || cells.length !== columns) return undefined const wanted = index === 0 ? 'tableHeader' : 'tableCell' const paragraphs: AdfNode[] = [] for (const cell of cells) { const paragraph = plainParagraph(cell) - if (paragraph === undefined || cell.type !== wanted || !carriesOnly(cell, [])) return undefined + if (paragraph === undefined || cell.type !== wanted || !holdsOnlyAttributes(cell, [])) return undefined paragraphs.push(paragraph) } grid.push(paragraphs) @@ -46,6 +46,6 @@ function pipeRows(node: AdfNode): AdfNode[][] | undefined { function plainParagraph(cell: AdfNode): AdfNode | undefined { const content = nodeContent(cell) const paragraph = content[0] - if (paragraph === undefined || content.length !== 1 || paragraph.type !== 'paragraph' || !carriesOnly(paragraph, [])) return undefined + if (paragraph === undefined || content.length !== 1 || paragraph.type !== 'paragraph' || !holdsOnlyAttributes(paragraph, [])) return undefined return paragraph } -- 2.52.0 From d0cb63fcf2ddc3050cdcb647cff443124f23e8f1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:52:43 +0200 Subject: [PATCH 50/67] Name the emitter's block walk PlacedBlocks, apart from the parser's Walk --- src/markdown/emit/adf-to-markdown.ts | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index 5ab6977..e134aba 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -22,10 +22,10 @@ type BlockSpelling = 'commonmark' | 'directive' | 'list' type EmittedBlock = { headroom: number; spelling: BlockSpelling; text: string } type KeptSpelling = { block: EmittedBlock | undefined; depth: number } type PlacedBlock = Omit & { node: AdfNode } +type PlacedBlocks = { blocks: readonly PlacedBlock[]; headroom: number } // Keyed by reference: only a caller building one object per position (the parse, the plain reduction) passes one; a consumer's document may share a node. export type SpellingMemo = Map -type Walk = { blocks: readonly PlacedBlock[]; headroom: number } -type WalkedItem = { node: AdfNode; walk: Walk } +type WalkedItem = { node: AdfNode; walk: PlacedBlocks } export type Writing = { flavour: Flavour; memo: SpellingMemo | undefined } export const largestListMarker = 999999999 @@ -48,7 +48,7 @@ export function writeMarkdown(document: AdfDocument, flavour: Flavour): Result { +function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: number, writing: Writing): Result { let headroom = largestNesting - depth if (headroom < 0) return tooDeep(path) const blocks: PlacedBlock[] = [] @@ -181,13 +181,13 @@ function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writi return lines.includes(undefined) ? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path) : success({ headroom, spelling: 'list', text: lines.join('\n') }) } -function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result { +function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result { const block = emitBlock(node, path, depth, writing) return block.ok ? success({ blocks: [{ ...block.value, node }], headroom: block.value.headroom }) : block } // The marker leads the first paragraph, or stands as one where the blocks open with another. -function taskBlocks(task: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result { +function taskBlocks(task: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result { const marker = taskMarker(nodeAttrs(task)['state']) const markerBlock: PlacedBlock = { node: { type: 'paragraph' }, spelling: 'commonmark', text: marker } if (task.type === 'taskItem') { @@ -220,7 +220,7 @@ function directivePair(node: AdfNode, opener: string, body: string, headroom: nu return { headroom, spelling: 'directive', text: `${opener}\n${body === '' ? '' : `${body}\n`}${spellDirectiveCloser(node.type)}` } } -function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number, walkBody: () => Result): Result { +function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number, walkBody: () => Result): Result { if (node.text !== undefined) return failure('unsupported-node-shape', `a ${node.type} carries no text: this one holds text`, path) if (blockDirectiveForm(node.type) === 'leaf' && nodeContent(node).length > 0) return failure('unsupported-node-shape', `a ${node.type} holds no content: this one holds some`, path) if (model.contentModel === 'code') return emitCodeDirective(node, model, path, depth) @@ -230,7 +230,7 @@ function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertE return emitDirectiveBody(node, model, opener.value, path, walkBody) } -function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string, path: ConvertErrorPath, walkBody: () => Result): Result { +function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string, path: ConvertErrorPath, walkBody: () => Result): Result { if (blockDirectiveForm(node.type) === 'leaf') return success({ headroom: Number.POSITIVE_INFINITY, spelling: 'directive', text: opener }) if (model.contentModel === 'inline') { const line = emitInlineLine(nodeContent(node), 'paragraph', path, 'lossless') -- 2.52.0 From b39525cad74ad03a8d2bcccc12b4b9bae95a098e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 15:53:25 +0200 Subject: [PATCH 51/67] Ship under the comprehension floor with the panel's scores as the baseline, and file items 59-62 that end it --- AGENTS.md | 1 + docs/decisions.md | 16 ++++++++++++++++ todo.md | 40 +++++++++++++++++++++++++++++----------- 3 files changed, 46 insertions(+), 11 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 39b5314..85e0c87 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,6 +48,7 @@ In `docs/decisions.md`: - Firefox reads the build - The coverage floors - The size ratchet +- The comprehension floor ships under 7 until items 59, 60, 61 and 62 land - Properties on a fixed seed - The CommonMark suite checks three ways - The flavour spec is read as a source diff --git a/docs/decisions.md b/docs/decisions.md index 74cce04..a28dd4b 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -496,6 +496,22 @@ config gone missing fails the leg instead of falling back to oxlint's own defaul `--deny-warnings`, since a rule from a category this config never names arrives as a warning it exits 0 on. +## The comprehension floor ships under 7 until items 59, 60, 61 and 62 land + +2026-10-03, the maintainer. KISS, a technical principle, and its comprehension floor of 7. Valid +while `todo.md` items 59, 60, 61 and 62 are open. + +A four-seat comprehension panel scored the project 6 against the floor of 7, and the project ships +under it while the restructures it named land: `todo.md` items 59, 60, 61 and 62 end this. The +scores are the baseline — a later panel may not score lower on any dimension: + +| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall | +|---|---|---|---|---|---| +| Junior | 6.5 | 5.5 | 6 | 5.5 | 6 | +| Mid | 7 | 5.5 | 6 | 5 | 5.5 | +| Senior | 7 | 5.5 | 6.5 | 5.5 | 6 | +| Architect | 6 | 5.5 | 5.5 | 6 | 6 | + ## Properties on a fixed seed 2026-09-14, the maintainer. Goal 1. Valid while a red gate must reproduce. diff --git a/todo.md b/todo.md index f775b28..0215807 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 59` +`Next ID = 63` | Goal | W | |---|---| @@ -30,11 +30,14 @@ | 6 | 0.2.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 3 | 24.7 | | 43 | 0.2.0 | decision | **Give each markdown input its own reader, strict to its own standard.** | 6 | 7 | 8 | 9 | 3, 4 | 22.3 | | 49 | 0.2.0 | | **Read a list whose bullet or ordered delimiter changes as two lists in the CommonMark reader.** | 4 | 5 | 5 | 8 | 3, 4 | 17.2 | +| 61 | 0.2.0 | decision | **Have the emitter ask the inline reader how a line reads back, in place of `line-escaping.ts` predicting it.** | 6 | 8 | 3 | 8 | 1 | 14.0 | | 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 5 | 13.0 | | 51 | 0.2.0 | | **Match a reference label to its definition under Unicode case folding.** | 2 | 2 | 2 | 7 | 3, 4 | 12.4 | -| 54 | 0.2.0 | | **Make `docs/decisions.md` §The source parts by ADF and format name every import `parse/` takes from `emit/`.** | 2 | 2 | 2 | 5 | 1 | 11.5 | | 53 | 0.2.0 | | **Put a block's `marks` spelling to the writer panel and adopt its pick.** | 4 | 5 | 5 | 6 | 5 | 11.5 | +| 60 | 0.2.0 | decision | **Give the parser's questions to the emitter one named module, so the parse→emit dependency reads as intended.** | 3 | 4 | 2 | 6 | 2 | 10.7 | +| 59 | 0.2.0 | decision | **Group the directive grammar into `src/markdown/directive/`, and move `Read` to `result.ts` and `Flavour` out of the plain flavour.** | 3 | 5 | 2 | 6 | 2 | 10.4 | | 50 | 0.2.0 | | **Read `[](/url)` and `[]()` as CommonMark's empty link.** | 4 | 4 | 3 | 6 | 3, 4, 6 | 10.4 | +| 62 | 0.2.0 | decision | **Move the plain reader out of `parse/markdown-to-adf.ts` into `markdown/plain/`.** | 3 | 4 | 2 | 5 | 2 | 8.9 | | 38 | 0.3.0 | | **Spell a lone surrogate in a text node so it survives a UTF-8 encode.** | 2 | 2 | 4 | 7 | 1 | 19.5 | | 47 | 0.3.0 | | **Open the README with what the package is, what it does and for whom.** | 1 | 4 | 7 | 9 | 9 | 14.0 | | 34 | 0.3.0 | | **Read emphasis flanking by the whole character beside an astral symbol.** | 2 | 3 | 3 | 6 | 3, 4 | 12.6 | @@ -53,9 +56,9 @@ ### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF. -Lands after item 6. The CommonMark spec suite also runs against `markdownToHtml`. The README -documents HTML as it documents markdown, and its tagline and `package.json`'s `description` regain -HTML. +Lands after items 6, 59, 60, 61 and 62. The CommonMark spec suite also runs against +`markdownToHtml`. The README documents HTML as it documents markdown, and its tagline and +`package.json`'s `description` regain HTML. ### 55. Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start or a newline inside an emoji, mention or status. @@ -99,6 +102,12 @@ Readings table gains its row, and its Spellings table one if `!adf:listBreak` re and 302 lose their `pending` exceptions, and the spelling leaves the README's "Four CommonMark spellings" bullet, which counts one fewer. +### 61. Have the emitter ask the inline reader how a line reads back, in place of `line-escaping.ts` predicting it. + +The comprehension panel's worst place: `escapeClaims` and `escapeClosedRuns` re-implement the +reader's view — flanking, code-span closers, link-definition openings, highlight flanking — and +only the property tests catch drift. It caps Locality. + ### 52. Spell `colwidth` as a comma list, `colwidth="340,420"`. A writer panel chose it on 2026-10-03, 5 of 7, over today's `colwidth="[340,420]"`. Breaking: @@ -111,18 +120,23 @@ example 540); lowercasing and then uppercasing folds it. Breaking, so it ships b `MIGRATION.md`'s Readings table gains its row. Its `pending` exceptions go, and its spelling leaves the README's "Four CommonMark spellings" bullet, which counts one fewer. -### 54. Make `docs/decisions.md` §The source parts by ADF and format name every import `parse/` takes from `emit/`. - -The entry says `parse/` imports only `commonMarkSpelling` and `openingLinkTakesDirective` from -`emit/`; `parse/markdown-to-adf.ts` also imports `inlineLeaves` from `emit/plain-inline.ts`. Either -the entry names it with its reason, or the reader stops importing it. - ### 53. Put a block's `marks` spelling to the writer panel and adopt its pick. Today `marks="[{\"attrs\":{\"mode\":\"wide\"},\"type\":\"breakout\"}]"`, the marks array as escaped JSON. Breaking where the panel picks another spelling: `MIGRATION.md`'s Spellings table gains its row. +### 60. Give the parser's questions to the emitter one named module, so the parse→emit dependency reads as intended. + +`parse/` asks `emit/` through `commonMarkSpelling` and `openingLinkTakesDirective`, each imported +from where it happens to live. One module naming the questions keeps `docs/decisions.md` §The source +parts by ADF and format true as they grow. + +### 59. Group the directive grammar into `src/markdown/directive/`, and move `Read` to `result.ts` and `Flavour` out of the plain flavour. + +Eight directive files sit across three directories, and the `markdown/` root holds 14 entries. HTML +needs `Read`, `Flavour`, the carry and the mark spellings out of `markdown/`. + ### 50. Read `[](/url)` and `[]()` as CommonMark's empty link. Both stay literal text today (spec examples 484 and 487). ADF holds no empty text node to carry a @@ -132,6 +146,10 @@ input. Breaking, so it ships beside item 43: `MIGRATION.md`'s Readings table gai `pending` exceptions go, and its spelling leaves the README's "Four CommonMark spellings" bullet, which counts one fewer. +### 62. Move the plain reader out of `parse/markdown-to-adf.ts` into `markdown/plain/`. + + + ### 38. Spell a lone surrogate in a text node so it survives a UTF-8 encode. `adfToMarkdown` emits it verbatim, so markdown stored as UTF-8 reads back U+FFFD; attribute values -- 2.52.0 From 7f67998c3bf3a584883c1ad1846de22a11693744 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:11:56 +0200 Subject: [PATCH 52/67] Prose round 3: retitle the floor decision, put the plain/ exception first, sharpen items 59-62 --- AGENTS.md | 2 +- README.md | 2 +- docs/decisions.md | 27 ++++++++++++--------------- src/markdown/adjacent-text.ts | 2 +- todo.md | 20 +++++++++++--------- 5 files changed, 26 insertions(+), 27 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 85e0c87..614fcc0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,7 +48,7 @@ In `docs/decisions.md`: - Firefox reads the build - The coverage floors - The size ratchet -- The comprehension floor ships under 7 until items 59, 60, 61 and 62 land +- The project ships under the comprehension floor until items 59, 60, 61 and 62 land - Properties on a fixed seed - The CommonMark suite checks three ways - The flavour spec is read as a source diff --git a/README.md b/README.md index 37874cf..e0e48f1 100644 --- a/README.md +++ b/README.md @@ -198,7 +198,7 @@ emit refuses: | `unspellable-line-start` | a paragraph line begins with a code span whose backticks would read back as a code fence | put any text before the code span | | `unspellable-whitespace` | an `emoji`, `mention` or `status` holds a newline in the text its inline directive spells in the content slot | replace it with a space — an inline directive never spans lines | | `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it | -| `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark, or a reserved directive stands out of place: `!adf:textBreak{}` or `!adf:listBreak` parting nothing, `!adf:doc` beside another block | fix what the message names, or give the node the shape it names; `spec/flavour.md` lists every type's attributes and body | +| `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark, or a reserved directive stands out of place: `!adf:textBreak{}` or `!adf:listBreak` parting nothing, `!adf:doc` anywhere but as the whole document | fix what the message names; `spec/flavour.md` lists every type's attributes and body | ## The guarantees diff --git a/docs/decisions.md b/docs/decisions.md index a28dd4b..5feca6d 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -496,14 +496,13 @@ config gone missing fails the leg instead of falling back to oxlint's own defaul `--deny-warnings`, since a rule from a category this config never names arrives as a warning it exits 0 on. -## The comprehension floor ships under 7 until items 59, 60, 61 and 62 land +## The project ships under the comprehension floor until items 59, 60, 61 and 62 land 2026-10-03, the maintainer. KISS, a technical principle, and its comprehension floor of 7. Valid while `todo.md` items 59, 60, 61 and 62 are open. -A four-seat comprehension panel scored the project 6 against the floor of 7, and the project ships -under it while the restructures it named land: `todo.md` items 59, 60, 61 and 62 end this. The -scores are the baseline — a later panel may not score lower on any dimension: +A four-seat comprehension panel scored the project 6 against the floor of 7. The scores are the +baseline: a later panel may not score lower on any dimension. | Seat | Navigation | Locality | Shape | Self-sufficiency | Overall | |---|---|---|---|---|---| @@ -672,16 +671,14 @@ them. A primitive knowing neither ADF nor a format stays at `src/` root. A const `spec/flavour.md` draws, and a placement nothing here settles goes beside its only reader, or in what both read where there are two. -Each format directory parts into `emit/` (ADF→format) and `parse/` (format→ADF), the rest of it -holding what both directions read. A construct's reader lives there beside the regex the emitter -escapes against, so the two cannot drift; a reader with no emit counterpart goes in `parse/`, -unless it is part of a construct that side already holds — a grammar stays in one file rather than -splitting across the seam. A rule both directions must answer alike — whether a list marker -interrupts a paragraph — is one function there too, never a copy per direction, however -conservative the copy would be. Where the rule is the emitter's own choice, input consults it -rather than restating it, and that is the only import `parse/` takes from `emit/` — +A flavour's own code, both directions included, sits in its own directory: `markdown/plain/`. +Otherwise each format directory parts into `emit/` (ADF→format) and `parse/` (format→ADF), the rest +of it holding what both directions read. A construct's reader lives there beside the regex the +emitter escapes against, so the two cannot drift; a reader with no emit counterpart goes in +`parse/`, unless it is part of a construct that side already holds — a grammar stays in one file +rather than splitting across the seam. A rule both directions must answer alike — whether a list +marker interrupts a paragraph — is one function there too, never a copy per direction, however +conservative the copy would be. Where the rule is the emitter's own choice, input consults it rather +than restating it, and that is the only value import `parse/` takes from `emit/` — `commonMarkSpelling` and `openingLinkTakesDirective` — so no fixture the emitter writes can be refused, and a spelling the emitter refuses gives its own error rather than a second name for it. -`markdown/plain/` holds the plain flavour's own code — the writer's reduction to editor-normal ADF -and the conventions and task ids the reader takes — so neither direction reaches into the other -for it. diff --git a/src/markdown/adjacent-text.ts b/src/markdown/adjacent-text.ts index b063caa..9b1547e 100644 --- a/src/markdown/adjacent-text.ts +++ b/src/markdown/adjacent-text.ts @@ -14,7 +14,7 @@ export function joinsWhenRead(previous: AdfNode, node: AdfNode): boolean { return isPlainText(previous) && isPlainText(node) && identicalMarks(nodeMarks(previous), nodeMarks(node)) } -// The editor's rule, parting from the reader's only on shapes editor-normal ADF erases: an empty attrs, content or marks, and -0 in a mark. +// The editor's rule, differing from the reader's only on shapes editor-normal ADF erases: an empty attrs, content or marks, and -0 in a mark. export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean { if (!joinsAsEditorText(previous) || !joinsAsEditorText(node)) return false return identicalMarks(nodeMarks(previous).map(normalMark), nodeMarks(node).map(normalMark)) diff --git a/todo.md b/todo.md index 0215807..7842034 100644 --- a/todo.md +++ b/todo.md @@ -34,8 +34,8 @@ | 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 5 | 13.0 | | 51 | 0.2.0 | | **Match a reference label to its definition under Unicode case folding.** | 2 | 2 | 2 | 7 | 3, 4 | 12.4 | | 53 | 0.2.0 | | **Put a block's `marks` spelling to the writer panel and adopt its pick.** | 4 | 5 | 5 | 6 | 5 | 11.5 | -| 60 | 0.2.0 | decision | **Give the parser's questions to the emitter one named module, so the parse→emit dependency reads as intended.** | 3 | 4 | 2 | 6 | 2 | 10.7 | -| 59 | 0.2.0 | decision | **Group the directive grammar into `src/markdown/directive/`, and move `Read` to `result.ts` and `Flavour` out of the plain flavour.** | 3 | 5 | 2 | 6 | 2 | 10.4 | +| 60 | 0.2.0 | decision | **Collect the questions `parse/` asks `emit/` into one named module.** | 3 | 4 | 2 | 6 | 2 | 10.7 | +| 59 | 0.2.0 | decision | **Group the directive grammar into `src/markdown/directive/`, move `Read` to `result.ts`, and move `Flavour` to `markdown/flavour.ts`.** | 3 | 5 | 2 | 6 | 2 | 10.4 | | 50 | 0.2.0 | | **Read `[](/url)` and `[]()` as CommonMark's empty link.** | 4 | 4 | 3 | 6 | 3, 4, 6 | 10.4 | | 62 | 0.2.0 | decision | **Move the plain reader out of `parse/markdown-to-adf.ts` into `markdown/plain/`.** | 3 | 4 | 2 | 5 | 2 | 8.9 | | 38 | 0.3.0 | | **Spell a lone surrogate in a text node so it survives a UTF-8 encode.** | 2 | 2 | 4 | 7 | 1 | 19.5 | @@ -106,7 +106,7 @@ spellings" bullet, which counts one fewer. The comprehension panel's worst place: `escapeClaims` and `escapeClosedRuns` re-implement the reader's view — flanking, code-span closers, link-definition openings, highlight flanking — and -only the property tests catch drift. It caps Locality. +only the property tests catch drift. It caps the panel's Locality score. ### 52. Spell `colwidth` as a comma list, `colwidth="340,420"`. @@ -126,16 +126,17 @@ Today `marks="[{\"attrs\":{\"mode\":\"wide\"},\"type\":\"breakout\"}]"`, the mar escaped JSON. Breaking where the panel picks another spelling: `MIGRATION.md`'s Spellings table gains its row. -### 60. Give the parser's questions to the emitter one named module, so the parse→emit dependency reads as intended. +### 60. Collect the questions `parse/` asks `emit/` into one named module. `parse/` asks `emit/` through `commonMarkSpelling` and `openingLinkTakesDirective`, each imported from where it happens to live. One module naming the questions keeps `docs/decisions.md` §The source parts by ADF and format true as they grow. -### 59. Group the directive grammar into `src/markdown/directive/`, and move `Read` to `result.ts` and `Flavour` out of the plain flavour. +### 59. Group the directive grammar into `src/markdown/directive/`, move `Read` to `result.ts`, and move `Flavour` to `markdown/flavour.ts`. Eight directive files sit across three directories, and the `markdown/` root holds 14 entries. HTML -needs `Read`, `Flavour`, the carry and the mark spellings out of `markdown/`. +needs `Read` and `Flavour` out of the plain flavour and `markdown/`; it needs the carry and the +mark spellings too, which item 7 moves where it learns what HTML shares. ### 50. Read `[](/url)` and `[]()` as CommonMark's empty link. @@ -148,7 +149,8 @@ which counts one fewer. ### 62. Move the plain reader out of `parse/markdown-to-adf.ts` into `markdown/plain/`. - +What moves: the plain reader's code in `markdown-to-adf.ts` — the alert and task-marker reads, the +`mintTaskIds` call and the `inlineLeaves` use. ### 38. Spell a lone surrogate in a text node so it survives a UTF-8 encode. @@ -172,8 +174,8 @@ punctuation — and, where they do, read the code point, with a fixture per dire ### 42. Trim a text leaf's trailing blanks in linear time. -`plain/inline-reduction.ts`'s `leafEdges` finds the trail with an unanchored `/[ \t]*$/`, quadratic in a run -of blanks inside one leaf: a paragraph of `a`, 80 000 spaces, `b` takes 6.5 s in +`plain/inline-reduction.ts`'s `leafEdges` finds the trail with an unanchored `/[ \t]*$/`, quadratic +in a run of blanks inside one leaf: a paragraph of `a`, 80 000 spaces, `b` takes 6.5 s in `adfToPlainMarkdown`. Scan backward, as the expand title's trim does. ### 48. Keep the release path publishing past npm's bypass-2FA token retirement. -- 2.52.0 From 79005f07bee75aeea298582355eb01fb9e682670 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:16:46 +0200 Subject: [PATCH 53/67] File the round-4 goals audit's defects and widen items 55 and 62 --- todo.md | 46 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 37 insertions(+), 9 deletions(-) diff --git a/todo.md b/todo.md index 7842034..eca1829 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 63` +`Next ID = 66` | Goal | W | |---|---| @@ -24,11 +24,14 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| +| 63 | 0.2.0 | defect | **Refuse a cyclic input as `not-an-adf-document` in place of looping forever.** | 2 | 2 | 6 | 9 | 1 | 27.5 | | 7 | 0.2.0 | | **Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF.** | 6 | 9 | 9 | 9 | 2, 3 | 25.8 | -| 55 | 0.2.0 | defect | **Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start or a newline inside an emoji, mention or status.** | 5 | 5 | 7 | 9 | 1 | 25.8 | +| 55 | 0.2.0 | defect | **Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start, a newline inside an emoji, mention or status, or a text node with empty, missing or content-holding text.** | 5 | 5 | 7 | 9 | 1 | 25.8 | | 45 | 0.2.0 | | **Replace `isAdfDocument` with a reader returning `Result`.** | 2 | 3 | 6 | 8 | 1 | 25.2 | | 6 | 0.2.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 3 | 24.7 | +| 64 | 0.2.0 | defect | **Read every JSON key the emitter writes back unchanged on V8, with a fixture whose key holds `\`, `"` or a control character.** | 4 | 4 | 6 | 8 | 1, 7 | 23.0 | | 43 | 0.2.0 | decision | **Give each markdown input its own reader, strict to its own standard.** | 6 | 7 | 8 | 9 | 3, 4 | 22.3 | +| 65 | 0.2.0 | defect | **Read a mid-text or titled CommonMark image as Goal 6 allows, in place of refusing it with `unmappable-image`.** | 5 | 5 | 7 | 7 | 3, 4, 6 | 18.7 | | 49 | 0.2.0 | | **Read a list whose bullet or ordered delimiter changes as two lists in the CommonMark reader.** | 4 | 5 | 5 | 8 | 3, 4 | 17.2 | | 61 | 0.2.0 | decision | **Have the emitter ask the inline reader how a line reads back, in place of `line-escaping.ts` predicting it.** | 6 | 8 | 3 | 8 | 1 | 14.0 | | 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 5 | 13.0 | @@ -37,7 +40,7 @@ | 60 | 0.2.0 | decision | **Collect the questions `parse/` asks `emit/` into one named module.** | 3 | 4 | 2 | 6 | 2 | 10.7 | | 59 | 0.2.0 | decision | **Group the directive grammar into `src/markdown/directive/`, move `Read` to `result.ts`, and move `Flavour` to `markdown/flavour.ts`.** | 3 | 5 | 2 | 6 | 2 | 10.4 | | 50 | 0.2.0 | | **Read `[](/url)` and `[]()` as CommonMark's empty link.** | 4 | 4 | 3 | 6 | 3, 4, 6 | 10.4 | -| 62 | 0.2.0 | decision | **Move the plain reader out of `parse/markdown-to-adf.ts` into `markdown/plain/`.** | 3 | 4 | 2 | 5 | 2 | 8.9 | +| 62 | 0.2.0 | decision | **Move the plain flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/plain/`, so `plain/` and `emit/` no longer import each other.** | 4 | 5 | 2 | 6 | 2 | 9.4 | | 38 | 0.3.0 | | **Spell a lone surrogate in a text node so it survives a UTF-8 encode.** | 2 | 2 | 4 | 7 | 1 | 19.5 | | 47 | 0.3.0 | | **Open the README with what the package is, what it does and for whom.** | 1 | 4 | 7 | 9 | 9 | 14.0 | | 34 | 0.3.0 | | **Read emphasis flanking by the whole character beside an astral symbol.** | 2 | 3 | 3 | 6 | 3, 4 | 12.6 | @@ -54,21 +57,29 @@ ## Details +### 63. Refuse a cyclic input as `not-an-adf-document` in place of looping forever. + +`adfDocumentFault` walks with `isNodeArray` (`adf/document.ts`) and `isJsonValue` (`json-value.ts`), +worklists that record no visited object, so a node whose `content` holds itself hangs +`adfToMarkdown`, `adfToPlainMarkdown` and `isAdfDocument`; README §The guarantees promises no input +loops forever. + ### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF. Lands after items 6, 59, 60, 61 and 62. The CommonMark spec suite also runs against `markdownToHtml`. The README documents HTML as it documents markdown, and its tagline and `package.json`'s `description` regain HTML. -### 55. Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start or a newline inside an emoji, mention or status. +### 55. Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start, a newline inside an emoji, mention or status, or a text node with empty, missing or content-holding text. `inline-line.ts` refuses a carriage return or NUL in text (`unspellable-character`) and a paragraph opening with a code-span run (`unspellable-line-start`); `fencedTexts` refuses the same characters in a code block; `directive-syntax.ts` refuses a newline in an emoji, mention or status (`unspellable-whitespace`). The inline carry's JSON escapes all of them, so a lossless spelling exists, and §The code list says a cause the carry answers gets no code. Fixing it revises §Which -code a cause takes and §Markdown in is a canonical fixpoint, which the maintainer decides. Found by -the README-goals audit, 2026-10-03. +code a cause takes and §Markdown in is a canonical fixpoint, which the maintainer decides. Also a +text node whose `text` is empty or missing, or which holds `content`: `inline-line.ts` refuses it +inline and `fencedTexts` in a code block. Found by the README-goals audit, 2026-10-03. ### 45. Replace `isAdfDocument` with a reader returning `Result`. @@ -82,6 +93,15 @@ Element-by-element mapping, the `data-*` fidelity scheme, the opaque-carry form, foreign-element set `htmlToAdf` accepts — the set `markdownToAdf` shares (`spec/flavour.md` §Raw HTML in input). The set sorts per `docs/decisions.md` §Foreign HTML sorts three ways. +### 64. Read every JSON key the emitter writes back unchanged on V8, with a fixture whose key holds `\`, `"` or a control character. + +`property-harness.ts` strips `\`, `"` and control characters from generated JSON keys, citing V8's +`JSON.parse` returning a wrong key for an escaped backslash. The library reads `json` attributes +(`directive-syntax.ts`) and both carries (`opaque-carry.ts`) through that `JSON.parse`, so on Node, +Deno and Chrome the round-trip would refuse its own output. Confirm with a fixture first; then read +JSON with our own parser or record the gap with an ending item. Found by the README-goals audit, +2026-10-03. + ### 43. Give each markdown input its own reader, strict to its own standard. Today `markdownToAdf` reads CommonMark and the lossless flavour as one input: text shaped like a @@ -90,6 +110,12 @@ a code fence whose info string opens `adf:` becomes the block carry where Common caller names the markdown it hands in: CommonMark, read as its spec says, or the lossless flavour, read as `spec/flavour.md` says. Breaking: `MIGRATION.md` says which call a caller takes. +### 65. Read a mid-text or titled CommonMark image as Goal 6 allows, in place of refusing it with `unmappable-image`. + +§CommonMark is a subset keeps the image gap and no item ends it: a bot's `See ![diagram](url) here` +or a titled image is refused, 14 examples in `corpus/commonmark-spec/refusals.json`. Settle what such +an image builds — Goal 6 drops form and keeps the target — and have the decision cite this item. + ### 49. Read a list whose bullet or ordered delimiter changes as two lists in the CommonMark reader. Lands after item 43. Today `- a` then `+ b`, or `1.` then `1)`, reads as one list; CommonMark reads @@ -147,10 +173,12 @@ input. Breaking, so it ships beside item 43: `MIGRATION.md`'s Readings table gai `pending` exceptions go, and its spelling leaves the README's "Four CommonMark spellings" bullet, which counts one fewer. -### 62. Move the plain reader out of `parse/markdown-to-adf.ts` into `markdown/plain/`. +### 62. Move the plain flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/plain/`, so `plain/` and `emit/` no longer import each other. -What moves: the plain reader's code in `markdown-to-adf.ts` — the alert and task-marker reads, the -`mintTaskIds` call and the `inlineLeaves` use. +Reading: the alert and task-marker reads in `parse/markdown-to-adf.ts`, the `mintTaskIds` call and +the `inlineLeaves` use. Writing: `spellPlainBlock`, `quotedUnder`, `tryTaskList` and `taskBlocks` +in `emit/adf-to-markdown.ts`. Every comprehension seat on 2026-10-03 named the plain flavour's +spread across three directories. ### 38. Spell a lone surrogate in a text node so it survives a UTF-8 encode. -- 2.52.0 From f7d2eb6517b0f17baa6021bcf4793f81ee101051 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:18:30 +0200 Subject: [PATCH 54/67] Widen item 56 to the inline scanner's Scan --- todo.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/todo.md b/todo.md index eca1829..be6a04d 100644 --- a/todo.md +++ b/todo.md @@ -50,7 +50,7 @@ | 33 | 0.3.0 | | **Emit a line in time linear in its mark runs, in `adfToMarkdown` and `adfToPlainMarkdown`.** | 4 | 5 | 6 | 9 | 8 | 10.7 | | 46 | 0.3.0 | | **Publish the bundle size in the README, failing the release pipeline when it drifts.** | 2 | 4 | 5 | 5 | 7, 9 | 10.3 | | 9 | 0.3.0 | | **Ship an online sandbox: a web page with two textboxes converting between ADF and markdown on the library's browser build.** | 2 | 6 | 6 | 8 | 9 | 10.3 | -| 56 | 0.3.0 | principle | **Give each piece of `blocks.ts`'s block-walk state one owner that returns what it changes.** | 4 | 5 | 2 | 4 | 1 | 6.8 | +| 56 | 0.3.0 | principle | **Give each piece of `blocks.ts`'s block-walk state and `inline-content.ts`'s `Scan` one owner that returns what it changes.** | 4 | 5 | 2 | 4 | 1 | 6.8 | | 57 | 0.3.0 | principle | **Make each `ci.sh` leg build what it reads, so one leg run alone tests the current tree.** | 2 | 3 | 1 | 3 | 7 | 1.2 | | 58 | 0.3.0 | principle | **Port `ci.sh`, `publish.sh` and `docker-runner.sh` to standalone Python scripts.** | 4 | 6 | 1 | 2 | 7 | -2.2 | | 8 | 0.4.0 | | **Ship a CLI.** | 3 | 7 | 7 | 6 | 9 | 10.6 | @@ -247,11 +247,14 @@ or report the unminified gzip). The figure lands in README §The package beside dependencies" claim. Measured today, unminified: tarball 60.4 kB, unpacked 221.5 kB, JS gzipped 45.6 kB. -### 56. Give each piece of `blocks.ts`'s block-walk state one owner that returns what it changes. +### 56. Give each piece of `blocks.ts`'s block-walk state and `inline-content.ts`'s `Scan` one owner that returns what it changes. Technical principle "One owner per value": the `Walk` record passes through twelve functions that mutate it and return `void`; `walk.leaf` alone is written in seven places. `ContainerStack` in the same file shows the shape to follow. +`inline-content.ts`'s `Scan` has the same shape: about fifteen functions write `pending`, `pieces`, +`deactivatedBefore` and `openingSpellableLink` and return `void`, and `parseInlineContent` reads a +flag `scanInline` leaves on it. ### 57. Make each `ci.sh` leg build what it reads, so one leg run alone tests the current tree. -- 2.52.0 From 1791542b3cc4b885816cad5bfd767717e6b0ffc8 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:19:09 +0200 Subject: [PATCH 55/67] Rename isPlainText to isBareText, apart from the plain flavour, and say truly what it admits --- src/adf/document.ts | 4 ++-- src/markdown/adjacent-text.ts | 4 ++-- src/markdown/emit/adf-to-markdown.ts | 4 ++-- src/markdown/emit/inline-line.ts | 6 +++--- src/markdown/parse/directive-nodes.ts | 4 ++-- 5 files changed, 11 insertions(+), 11 deletions(-) diff --git a/src/adf/document.ts b/src/adf/document.ts index 42da38e..b84f6ff 100644 --- a/src/adf/document.ts +++ b/src/adf/document.ts @@ -64,8 +64,8 @@ export function emptyKeys(held: { attrs?: AdfAttributes; content?: AdfNode[]; ma return keys } -// A text node holding its text and marks alone: a format spells it as text, where anything more rides a carry. -export function isPlainText(node: AdfNode): boolean { +// A text node holding no attrs or content key and no empty marks: a format spells it as text, where anything more rides a carry. +export function isBareText(node: AdfNode): boolean { return node.type === 'text' && node.attrs === undefined && node.content === undefined && node.marks?.length !== 0 } diff --git a/src/markdown/adjacent-text.ts b/src/markdown/adjacent-text.ts index 9b1547e..bea9d7b 100644 --- a/src/markdown/adjacent-text.ts +++ b/src/markdown/adjacent-text.ts @@ -1,6 +1,6 @@ import type { AdfAttributes, AdfMark, AdfNode } from '../adf/document.ts' import type { JsonValue } from '../json-value.ts' -import { identicalMark, identicalMarks, isPlainText, nodeAttrs, nodeMarks } from '../adf/document.ts' +import { identicalMark, identicalMarks, isBareText, nodeAttrs, nodeMarks } from '../adf/document.ts' import { spellInlineLeafDirective } from './directive-syntax.ts' type JsonContainer = JsonValue[] | { [key: string]: JsonValue } @@ -11,7 +11,7 @@ export const textBreakSpelling = spellInlineLeafDirective(textBreakName, '') // The lossless reader's rule: CommonMark reads the pair back as one text run (spec/flavour.md, Inline nodes). export function joinsWhenRead(previous: AdfNode, node: AdfNode): boolean { - return isPlainText(previous) && isPlainText(node) && identicalMarks(nodeMarks(previous), nodeMarks(node)) + return isBareText(previous) && isBareText(node) && identicalMarks(nodeMarks(previous), nodeMarks(node)) } // The editor's rule, differing from the reader's only on shapes editor-normal ADF erases: an empty attrs, content or marks, and -0 in a mark. diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index e134aba..936a646 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -1,7 +1,7 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { Flavour } from '../plain/conventions.ts' -import { adfDocumentFault, holdsOnlyAttributes, isPlainText, nodeAttrs, nodeContent } from '../../adf/document.ts' +import { adfDocumentFault, holdsOnlyAttributes, isBareText, nodeAttrs, nodeContent } from '../../adf/document.ts' import { alertMarker, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../plain/conventions.ts' import { blockDirectiveForm, documentSpelling, listBreakSpelling } from '../block-directive.ts' import { blockNodeModel, blockNodes } from '../../adf/block-nodes.ts' @@ -278,7 +278,7 @@ function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertEr // The text each fence holds, or `undefined` where a child is no plain text node, which the carry holds instead. function fencedTexts(node: AdfNode, path: ConvertErrorPath): Result { if (node.content === undefined) return success(['']) - if (node.content.some((child) => !isPlainText(child) || child.marks !== undefined)) return success(undefined) + if (node.content.some((child) => !isBareText(child) || child.marks !== undefined)) return success(undefined) const texts: string[] = [] for (const [index, child] of node.content.entries()) { const childPath = [...path, 'content', index] diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index bcd3be4..77c4afe 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -6,7 +6,7 @@ import { assembleInlineLine, isSyntax, type InlineEscaping, type InlineSegment, import { carriedInline } from '../opaque-carry.ts' import { claimsLine, holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts' import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts' -import { identicalMark, isPlainText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' +import { identicalMark, isBareText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { escapeUnbalanced, spellDestination } from '../commonmark/link-syntax.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { highlightDelimiter } from '../plain/conventions.ts' @@ -269,7 +269,7 @@ function emitInlineDirective(node: AdfNode, model: InlineNodeModel, index: numbe function emitText(node: AdfNode, context: InlineContext, index: number, path: ConvertErrorPath): Result { if (nodeContent(node).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) - if (!isPlainText(node)) return success({ carry: { first: index, last: index } }) + if (!isBareText(node)) return success({ carry: { first: index, last: index } }) if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) if (/\r/.test(node.text)) return failure('unspellable-character', 'a text node holds a carriage return CommonMark rewrites', path) if (holdsNullCharacter(node.text)) return failure('unspellable-character', 'a text node holds a null character CommonMark replaces', path) @@ -324,7 +324,7 @@ function emitCodeSpan(nodes: readonly AdfNode[], depth: number, range: NodeRange const spans: string[] = [] for (const node of nodes) { if (node.type === 'text' && nodeContent(node).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) - if (!isPlainText(node) || nodeMarks(node).length !== depth + 1) return success({ carry: range }) + if (!isBareText(node) || nodeMarks(node).length !== depth + 1) return success({ carry: range }) const { text } = node if (typeof text !== 'string' || text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) if (/[\n\r]/.test(text)) return success({ carry: range }) diff --git a/src/markdown/parse/directive-nodes.ts b/src/markdown/parse/directive-nodes.ts index 61c1471..3af6098 100644 --- a/src/markdown/parse/directive-nodes.ts +++ b/src/markdown/parse/directive-nodes.ts @@ -3,7 +3,7 @@ import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { ConvertFault } from '../../result.ts' import type { DirectiveAttributes, DirectiveValue } from '../directive-syntax.ts' import type { Elsewhere } from './directive-attributes.ts' -import { attributeNestingMessage, isPlainText } from '../../adf/document.ts' +import { attributeNestingMessage, isBareText } from '../../adf/document.ts' import { attributeValue, directivePrefix, spellAttributeValue, unknownDirectiveFault } from '../directive-syntax.ts' import { blockArgument, blockDirectiveForm, marksAttribute, readMarkValues } from '../block-directive.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' @@ -92,7 +92,7 @@ function blockSpellingFault(name: string): ConvertFault | undefined { function slotText(content: readonly AdfNode[]): string | undefined { if (content.length === 0) return '' const only = content.length === 1 ? content[0] : undefined - if (only === undefined || !isPlainText(only) || only.marks !== undefined || typeof only.text !== 'string') return undefined + if (only === undefined || !isBareText(only) || only.marks !== undefined || typeof only.text !== 'string') return undefined return only.text } -- 2.52.0 From 36604b1e2bf38cbde1d6e04d23c3c05ac16e089b Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:19:55 +0200 Subject: [PATCH 56/67] Hold the editor's join rule in plain/editor-normal.ts as the reader's over editor-normal forms --- src/conformance/property-harness.ts | 2 +- src/markdown/adjacent-text.ts | 45 ++---------------------- src/markdown/plain/editor-normal.test.ts | 5 +++ src/markdown/plain/editor-normal.ts | 42 ++++++++++++++++++++-- src/markdown/plain/inline-reduction.ts | 2 +- 5 files changed, 48 insertions(+), 48 deletions(-) diff --git a/src/conformance/property-harness.ts b/src/conformance/property-harness.ts index 6951572..5f4700e 100644 --- a/src/conformance/property-harness.ts +++ b/src/conformance/property-harness.ts @@ -11,7 +11,7 @@ import { blockNodes } from '../adf/block-nodes.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' import { emptyKeys, mergeAdjacentText } from '../adf/document.ts' import { inlineNodes } from '../adf/inline-nodes.ts' -import { joinsWhenEditorNormal } from '../markdown/adjacent-text.ts' +import { joinsWhenEditorNormal } from '../markdown/plain/editor-normal.ts' import { markAttributes } from '../adf/mark-attributes.ts' type Positions = { block: AdfNode; inline: AdfNode } diff --git a/src/markdown/adjacent-text.ts b/src/markdown/adjacent-text.ts index bea9d7b..851f2a3 100644 --- a/src/markdown/adjacent-text.ts +++ b/src/markdown/adjacent-text.ts @@ -1,10 +1,7 @@ -import type { AdfAttributes, AdfMark, AdfNode } from '../adf/document.ts' -import type { JsonValue } from '../json-value.ts' -import { identicalMark, identicalMarks, isBareText, nodeAttrs, nodeMarks } from '../adf/document.ts' +import type { AdfNode } from '../adf/document.ts' +import { identicalMarks, isBareText, nodeMarks } from '../adf/document.ts' import { spellInlineLeafDirective } from './directive-syntax.ts' -type JsonContainer = JsonValue[] | { [key: string]: JsonValue } - export const textBreakName = 'textBreak' export const textBreakSpelling = spellInlineLeafDirective(textBreakName, '') @@ -13,41 +10,3 @@ export const textBreakSpelling = spellInlineLeafDirective(textBreakName, '') export function joinsWhenRead(previous: AdfNode, node: AdfNode): boolean { return isBareText(previous) && isBareText(node) && identicalMarks(nodeMarks(previous), nodeMarks(node)) } - -// The editor's rule, differing from the reader's only on shapes editor-normal ADF erases: an empty attrs, content or marks, and -0 in a mark. -export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean { - if (!joinsAsEditorText(previous) || !joinsAsEditorText(node)) return false - return identicalMarks(nodeMarks(previous).map(normalMark), nodeMarks(node).map(normalMark)) -} - -export function sameMarkWhenEditorNormal(left: AdfMark, right: AdfMark): boolean { - return identicalMark(normalMark(left), normalMark(right)) -} - -export function normalMark(mark: AdfMark): AdfMark { - const attrs = normalAttributes(nodeAttrs(mark)) - return attrs === undefined ? { type: mark.type } : { attrs, type: mark.type } -} - -export function normalAttributes(attrs: AdfAttributes): AdfAttributes | undefined { - if (Object.keys(attrs).length === 0) return undefined - const normal = { ...attrs } - const pending: JsonContainer[] = [normal] - for (let held = pending.pop(); held !== undefined; held = pending.pop()) { - if (Array.isArray(held)) for (const [index, value] of held.entries()) held[index] = normalValue(value, pending) - else for (const [key, value] of Object.entries(held)) held[key] = normalValue(value, pending) - } - return normal -} - -function normalValue(value: JsonValue, pending: JsonContainer[]): JsonValue { - if (Object.is(value, -0)) return 0 - if (value === null || typeof value !== 'object') return value - const copy = Array.isArray(value) ? [...value] : { ...value } - pending.push(copy) - return copy -} - -function joinsAsEditorText(node: AdfNode): boolean { - return node.type === 'text' && Object.keys(nodeAttrs(node)).length === 0 -} diff --git a/src/markdown/plain/editor-normal.test.ts b/src/markdown/plain/editor-normal.test.ts index 33d538a..65ede45 100644 --- a/src/markdown/plain/editor-normal.test.ts +++ b/src/markdown/plain/editor-normal.test.ts @@ -75,3 +75,8 @@ test('normalizes blocks and mark attributes nesting far past the levels a recurs const merged = toEditorNormal({ content: [{ content: [{ marks, text: 'a', type: 'text' }, { marks, text: 'b', type: 'text' }], type: 'paragraph' }], type: 'doc', version: 1 }) assert.deepEqual(merged.content?.[0]?.content?.map((text) => text.text), ['ab']) }) + +test('joins a text node holding content to its neighbour, as editor-normal forms hold no content to part them', () => { + const paragraph: AdfNode = { content: [{ content: [{ text: 'lost', type: 'text' }], text: 'a', type: 'text' }, { text: 'b', type: 'text' }], type: 'paragraph' } + assert.deepEqual(toEditorNormal({ content: [paragraph], type: 'doc', version: 1 }).content, [{ content: [{ content: [{ text: 'lost', type: 'text' }], text: 'ab', type: 'text' }], type: 'paragraph' }]) +}) diff --git a/src/markdown/plain/editor-normal.ts b/src/markdown/plain/editor-normal.ts index e43a0a5..e50f56c 100644 --- a/src/markdown/plain/editor-normal.ts +++ b/src/markdown/plain/editor-normal.ts @@ -1,9 +1,21 @@ -import type { AdfDocument, AdfNode } from '../../adf/document.ts' -import { joinsWhenEditorNormal, normalAttributes, normalMark } from '../adjacent-text.ts' -import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' +import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' +import type { JsonValue } from '../../json-value.ts' +import { identicalMark, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' +import { joinsWhenRead } from '../adjacent-text.ts' + +type JsonContainer = JsonValue[] | { [key: string]: JsonValue } type NodeHolder = { content?: AdfNode[] } +// The editor's rule is the reader's over the pair's editor-normal forms, which hold no content: a text node's content never parts it. +export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean { + return joinsWhenRead(normalNode(previous), normalNode(node)) +} + +export function sameMarkWhenEditorNormal(left: AdfMark, right: AdfMark): boolean { + return identicalMark(normalMark(left), normalMark(right)) +} + export function toEditorNormal(document: AdfDocument): AdfDocument { const normal: AdfDocument = { type: document.type, version: Object.is(document.version, -0) ? 0 : document.version } const pending: { holder: NodeHolder; source: NodeHolder }[] = [{ holder: normal, source: document }] @@ -28,3 +40,27 @@ function normalNode(node: AdfNode): AdfNode { if (node.text !== undefined) normal.text = node.text return normal } + +function normalMark(mark: AdfMark): AdfMark { + const attrs = normalAttributes(nodeAttrs(mark)) + return attrs === undefined ? { type: mark.type } : { attrs, type: mark.type } +} + +function normalAttributes(attrs: AdfAttributes): AdfAttributes | undefined { + if (Object.keys(attrs).length === 0) return undefined + const normal = { ...attrs } + const pending: JsonContainer[] = [normal] + for (let held = pending.pop(); held !== undefined; held = pending.pop()) { + if (Array.isArray(held)) for (const [index, value] of held.entries()) held[index] = normalValue(value, pending) + else for (const [key, value] of Object.entries(held)) held[key] = normalValue(value, pending) + } + return normal +} + +function normalValue(value: JsonValue, pending: JsonContainer[]): JsonValue { + if (Object.is(value, -0)) return 0 + if (value === null || typeof value !== 'object') return value + const copy = Array.isArray(value) ? [...value] : { ...value } + pending.push(copy) + return copy +} diff --git a/src/markdown/plain/inline-reduction.ts b/src/markdown/plain/inline-reduction.ts index 0df6095..882f732 100644 --- a/src/markdown/plain/inline-reduction.ts +++ b/src/markdown/plain/inline-reduction.ts @@ -3,7 +3,7 @@ import type { LineContainer } from '../line-container.ts' import type { MarkRun } from '../emit/line-escaping.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' -import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from '../adjacent-text.ts' +import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from './editor-normal.ts' import { largestNesting } from '../../nesting.ts' import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { plainLineFallback, type PlainLineFallback } from '../emit/inline-line.ts' -- 2.52.0 From d56e5b4f57a7cceceba7cd833cf31170c6371b7e Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:21:06 +0200 Subject: [PATCH 57/67] Keep near-miss text pairs apart in the property harness: merge only what the reader joins --- src/conformance/property-harness.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/conformance/property-harness.ts b/src/conformance/property-harness.ts index 5f4700e..0f3cc41 100644 --- a/src/conformance/property-harness.ts +++ b/src/conformance/property-harness.ts @@ -11,7 +11,7 @@ import { blockNodes } from '../adf/block-nodes.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' import { emptyKeys, mergeAdjacentText } from '../adf/document.ts' import { inlineNodes } from '../adf/inline-nodes.ts' -import { joinsWhenEditorNormal } from '../markdown/plain/editor-normal.ts' +import { joinsWhenRead } from '../markdown/adjacent-text.ts' import { markAttributes } from '../adf/mark-attributes.ts' type Positions = { block: AdfNode; inline: AdfNode } @@ -91,7 +91,7 @@ function withoutEmptyKeys(held: T): T { } function occasionallyApart(arbitrary: Arbitrary): Arbitrary { - return fc.tuple(arbitrary, fc.nat({ max: 5 })).map(([nodes, roll]) => (roll === 0 ? nodes : mergeAdjacentText(nodes, joinsWhenEditorNormal))) + return fc.tuple(arbitrary, fc.nat({ max: 5 })).map(([nodes, roll]) => (roll === 0 ? nodes : mergeAdjacentText(nodes, joinsWhenRead))) } function pipeTable({ body, header }: { body: AdfNode[][]; header: AdfNode[] }): AdfNode { -- 2.52.0 From fd3452b124aabc332f2c610f9daf748a044d5205 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:21:49 +0200 Subject: [PATCH 58/67] Name the reader's own-marks items for what they hold, and drop the carry flag the items already say --- src/markdown/parse/inline-content.ts | 42 ++++++++++++++-------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index 1d7c0db..d02f699 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -21,17 +21,17 @@ import { readDirectiveMark } from './directive-marks.ts' import { readInlineDirectiveNode } from './directive-nodes.ts' import { readTextDirective } from '../text-directive.ts' -export type InlineContent = { carry?: undefined; image: AdfNode; nodes?: undefined } | { carry: boolean; image?: undefined; nodes: AdfNode[] } +export type InlineContent = { image: AdfNode; nodes?: undefined } | { image?: undefined; nodes: AdfNode[] } // The text break leaf: it holds no marks, and stands between the nodes it parts until the outermost content checks and drops it. type TextBreak = { kind: 'textBreak' } // A node whose marks are its own — an opaque carry, or an inline node spelling marks=empty: no mark spelling wraps it, and it joins no neighbour. -type Carried = { kind: 'carry'; node: AdfNode } +type OwnMarks = { kind: 'ownMarks'; node: AdfNode } -type Inline = AdfNode | Carried | TextBreak +type Inline = AdfNode | OwnMarks | TextBreak -type Scanned = { carry?: undefined; image: AdfNode; nodes?: undefined } | { carry: boolean; image?: undefined; nodes: Inline[] } +type Scanned = { image: AdfNode; nodes?: undefined } | { image?: undefined; nodes: Inline[] } export type LinkDefinitions = ReadonlyMap @@ -43,7 +43,7 @@ type Pairing = EmphasisPairing type Piece = | Bracket - | Carried + | OwnMarks | { alt: string; kind: 'image'; node: AdfNode } | { kind: 'nodes'; nodes: Inline[] } | TextBreak @@ -70,9 +70,9 @@ type Scan = { // What a directive's content slot inherits from the content holding it. type SharedScan = Pick -type SlotContent = { carry: boolean; nodes: Inline[] } +type SlotContent = { nodes: Inline[] } -const carriedInMark = 'move the opaque carry, or the node spelling marks=empty, out of the mark spelling: it holds its own marks' +const ownMarksInMark = 'move the opaque carry, or the node spelling marks=empty, out of the mark spelling: it holds its own marks' const editorHighlight: AdfMark = { attrs: { color: '#f8e6a0' }, type: 'backgroundColor' } const hreflessLink = 'the link mark spells its href: this one spells none' const imageAlone = 'an image fits only as a paragraph of its own: this one sits inside other content' @@ -93,7 +93,7 @@ export function parseInlineContent(source: string, definitions: LinkDefinitions, if (!takesDirective.ok) return takesDirective if (!takesDirective.value) return failure('unsupported-node-shape', spellableLink, path) } - return success({ carry: scanned.value.carry, nodes: nodes.value }) + return success({ nodes: nodes.value }) } function freshScan(source: string, shared: SharedScan, own: Pick): Scan { @@ -236,7 +236,7 @@ function directivePiece(scan: Scan, span: DirectiveSpan, index: number): Result< const carried = readCarriedInline(span) if (carried !== undefined) { if (carried.fault !== undefined) return faulted(carried.fault, scan.path) - return success({ kind: 'carry', node: carried.value }) + return success({ kind: 'ownMarks', node: carried.value }) } if (span.name === textBreakName) return span.content === undefined && span.attributes.size === 0 ? success(textBreak) : failure('unsupported-node-shape', `${textBreakName} spells the bare leaf form, ${textBreakSpelling}: this one spells more`, scan.path) const text = readTextDirective(span) @@ -250,14 +250,14 @@ function directivePiece(scan: Scan, span: DirectiveSpan, index: number): Result< if (content?.some(isTextBreak) === true) return failure('unsupported-node-shape', `delete ${textBreakSpelling} from this content slot: a slot holds one text node`, scan.path) const node = readInlineDirectiveNode(span.name, span.attributes, content === undefined ? undefined : adfNodes(content), scan.path) if (!node.ok) return node - return success(node.value.marks === undefined ? { kind: 'nodes', nodes: [node.value] } : { kind: 'carry', node: node.value }) + return success(node.value.marks === undefined ? { kind: 'nodes', nodes: [node.value] } : { kind: 'ownMarks', node: node.value }) } function directiveMarkPiece(scan: Scan, name: string, mark: AdfMark, slot: SlotContent | undefined, index: number): Result { if (slot === undefined || slot.nodes.length === 0) { return failure('unsupported-node-shape', `the ${name} mark wraps the [content] it marks: this one wraps none`, scan.path) } - if (slot.carry) return failure('unsupported-node-shape', carriedInMark, scan.path) + if (slot.nodes.some(holdsOwnMarks)) return failure('unsupported-node-shape', ownMarksInMark, scan.path) const refused = mark.type === 'link' ? refuseLinkDirective(scan, mark, slot.nodes, index) : undefined if (refused !== undefined) return refused return success({ kind: 'nodes', nodes: applyMark(slot.nodes, mark) }) @@ -299,7 +299,7 @@ function assemble(scan: Scan): Result { if (holdsImage(scan.pieces)) return failure('unmappable-image', imageAlone, scan.path) const nodes = resolveNodes(scan.pieces, scan, true) if (!nodes.ok) return nodes - return success({ carry: holdsCarry(scan.pieces), nodes: nodes.value }) + return success({ nodes: nodes.value }) } // spec/flavour.md, Inline nodes: the leaf builds no node, so only the pair it parts spells it. @@ -327,7 +327,7 @@ function isTextBreak(item: Inline): item is TextBreak { return 'kind' in item && item.kind === 'textBreak' } -// The nodes the items hold, carried ones unwrapped and text breaks dropped. +// The nodes the items hold, own-marks ones unwrapped and text breaks dropped. function adfNodes(items: readonly Inline[]): AdfNode[] { const nodes: AdfNode[] = [] for (const item of items) { @@ -337,15 +337,15 @@ function adfNodes(items: readonly Inline[]): AdfNode[] { return nodes } -function holdsCarry(pieces: readonly Piece[]): boolean { - return pieces.some((piece) => piece.kind === 'carry') +function holdsOwnMarks(item: Inline | Piece): boolean { + return 'kind' in item && item.kind === 'ownMarks' } function holdsImage(pieces: readonly Piece[]): boolean { return pieces.some((piece) => piece.kind === 'image') } -// A carry rides its own piece: the carried node's marks restore with it rather than riding a spelling, so the guard below answers for it. +// A node whose marks are its own rides its own piece, its marks restoring with it rather than riding a spelling, so the guard below answers for it. function holdsLink(pieces: readonly Piece[]): boolean { return pieces.some((piece) => piece.kind === 'nodes' && marksLink(piece.nodes)) } @@ -447,7 +447,7 @@ function closeLink(scan: Scan, at: number, inner: readonly Piece[], definition: return success(false) } if (holdsImage(inner)) return failure('unmappable-image', imageAlone, scan.path) - if (holdsCarry(inner)) return failure('unsupported-node-shape', carriedInMark, scan.path) + if (inner.some(holdsOwnMarks)) return failure('unsupported-node-shape', ownMarksInMark, scan.path) const resolved = resolveNodes(inner, scan, true) if (!resolved.ok) return resolved const nodes = resolved.value @@ -506,12 +506,12 @@ function resolveNodes(pieces: readonly Piece[], scan: Scan, highlights: boolean) const runs = delimiterRuns(pieces) const pairings = matchEmphasis(runs) writeUnpaired(nodes, runs, pairings) - if (!markPairings(pieces, nodes, pairings)) return failure('unsupported-node-shape', carriedInMark, scan.path) + if (!markPairings(pieces, nodes, pairings)) return failure('unsupported-node-shape', ownMarksInMark, scan.path) markHighlights(pieces, nodes, highlights ? pairedHighlights(pieces, nodes) : []) return success(mergeReadText(nodes.flat())) } -// A text break and a carried node are walls: the text on either side joins only its own side. +// A text break and a node whose marks are its own are walls: the text on either side joins only its own side. function mergeReadText(items: readonly Inline[]): Inline[] { const merged: Inline[] = [] let run: AdfNode[] = [] @@ -532,7 +532,7 @@ function mergeReadText(items: readonly Inline[]): Inline[] { // A highlight delimiter holds an empty text node until it pairs, so the emphasis around it marks it. function pieceNodes(piece: Piece): Inline[] { switch (piece.kind) { - case 'carry': + case 'ownMarks': return [piece] case 'highlight': return [{ text: '', type: 'text' }] @@ -577,7 +577,7 @@ function markPairings(pieces: readonly Piece[], nodes: Inline[][], pairings: rea for (const pairing of pairings) { const mark: AdfMark = { type: markType(pairing.opener.character, pairing.used) } for (let index = pairing.opener.index + 1; index < pairing.closer.index; index += 1) { - if (pieces[index]?.kind === 'carry') return false + if (pieces[index]?.kind === 'ownMarks') return false nodes[index] = applyMark(nodes[index] ?? [], mark) } } -- 2.52.0 From 00c89dd63b3de19624632b2b5ea6c0bd696adaca Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:22:19 +0200 Subject: [PATCH 59/67] Give the unmarked bare text check and the text-holding-content refusal one owner each --- src/adf/document.ts | 4 ++++ src/markdown/emit/adf-to-markdown.ts | 4 ++-- src/markdown/emit/inline-line.ts | 10 ++++++++-- src/markdown/parse/directive-nodes.ts | 4 ++-- 4 files changed, 16 insertions(+), 6 deletions(-) diff --git a/src/adf/document.ts b/src/adf/document.ts index b84f6ff..cd5d0e6 100644 --- a/src/adf/document.ts +++ b/src/adf/document.ts @@ -69,6 +69,10 @@ export function isBareText(node: AdfNode): boolean { return node.type === 'text' && node.attrs === undefined && node.content === undefined && node.marks?.length !== 0 } +export function isUnmarkedBareText(node: AdfNode): boolean { + return isBareText(node) && node.marks === undefined +} + export function identicalMark(left: AdfMark, right: AdfMark): boolean { return marksKey([left]) === marksKey([right]) } diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index 936a646..5c17534 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -1,7 +1,7 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { Flavour } from '../plain/conventions.ts' -import { adfDocumentFault, holdsOnlyAttributes, isBareText, nodeAttrs, nodeContent } from '../../adf/document.ts' +import { adfDocumentFault, holdsOnlyAttributes, isUnmarkedBareText, nodeAttrs, nodeContent } from '../../adf/document.ts' import { alertMarker, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../plain/conventions.ts' import { blockDirectiveForm, documentSpelling, listBreakSpelling } from '../block-directive.ts' import { blockNodeModel, blockNodes } from '../../adf/block-nodes.ts' @@ -278,7 +278,7 @@ function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertEr // The text each fence holds, or `undefined` where a child is no plain text node, which the carry holds instead. function fencedTexts(node: AdfNode, path: ConvertErrorPath): Result { if (node.content === undefined) return success(['']) - if (node.content.some((child) => !isBareText(child) || child.marks !== undefined)) return success(undefined) + if (node.content.some((child) => !isUnmarkedBareText(child))) return success(undefined) const texts: string[] = [] for (const [index, child] of node.content.entries()) { const childPath = [...path, 'content', index] diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index 77c4afe..636b120 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -267,8 +267,13 @@ function emitInlineDirective(node: AdfNode, model: InlineNodeModel, index: numbe return success({ segments: [syntax(spellInlineDirectiveOpener(node.type)), ...content, syntax(`]${attributes}`)] }) } +function textHoldingContent(node: AdfNode, path: ConvertErrorPath): Result | undefined { + return node.type === 'text' && nodeContent(node).length > 0 ? failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) : undefined +} + function emitText(node: AdfNode, context: InlineContext, index: number, path: ConvertErrorPath): Result { - if (nodeContent(node).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) + const holding = textHoldingContent(node, path) + if (holding !== undefined) return holding if (!isBareText(node)) return success({ carry: { first: index, last: index } }) if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) if (/\r/.test(node.text)) return failure('unspellable-character', 'a text node holds a carriage return CommonMark rewrites', path) @@ -323,7 +328,8 @@ function emitHighlight(nodes: readonly AdfNode[], depth: number, range: NodeRang function emitCodeSpan(nodes: readonly AdfNode[], depth: number, range: NodeRange, path: ConvertErrorPath): Result { const spans: string[] = [] for (const node of nodes) { - if (node.type === 'text' && nodeContent(node).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) + const holding = textHoldingContent(node, path) + if (holding !== undefined) return holding if (!isBareText(node) || nodeMarks(node).length !== depth + 1) return success({ carry: range }) const { text } = node if (typeof text !== 'string' || text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) diff --git a/src/markdown/parse/directive-nodes.ts b/src/markdown/parse/directive-nodes.ts index 3af6098..19afd52 100644 --- a/src/markdown/parse/directive-nodes.ts +++ b/src/markdown/parse/directive-nodes.ts @@ -3,7 +3,7 @@ import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { ConvertFault } from '../../result.ts' import type { DirectiveAttributes, DirectiveValue } from '../directive-syntax.ts' import type { Elsewhere } from './directive-attributes.ts' -import { attributeNestingMessage, isBareText } from '../../adf/document.ts' +import { attributeNestingMessage, isUnmarkedBareText } from '../../adf/document.ts' import { attributeValue, directivePrefix, spellAttributeValue, unknownDirectiveFault } from '../directive-syntax.ts' import { blockArgument, blockDirectiveForm, marksAttribute, readMarkValues } from '../block-directive.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' @@ -92,7 +92,7 @@ function blockSpellingFault(name: string): ConvertFault | undefined { function slotText(content: readonly AdfNode[]): string | undefined { if (content.length === 0) return '' const only = content.length === 1 ? content[0] : undefined - if (only === undefined || !isBareText(only) || only.marks !== undefined || typeof only.text !== 'string') return undefined + if (only === undefined || !isUnmarkedBareText(only) || typeof only.text !== 'string') return undefined return only.text } -- 2.52.0 From 7bab478729eb3d101ee824e9e54012e50beaa5d7 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:22:54 +0200 Subject: [PATCH 60/67] Keep an empty doc content in editor-normal ADF, as the schema requires --- docs/decisions.md | 4 ++-- src/markdown/plain/adf-to-plain-markdown.ts | 2 +- src/markdown/plain/editor-normal.test.ts | 5 +++-- src/markdown/plain/editor-normal.ts | 2 ++ 4 files changed, 8 insertions(+), 5 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 5feca6d..6005569 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -42,8 +42,8 @@ adjacent text nodes, an empty `attrs`, `content` or `marks`, and `-0`. Neither s CommonMark's spelling stays wherever a document holds none of those shapes. The plain reader builds what is written, as `markdownToAdf` does. Only the plain writer is lossy: its reduction reads and writes editor-normal ADF — adjacent text nodes of identical marks and no attributes merged, `-0` as -`0`, and an empty `attrs`, `content` or `marks` the absent key — so two documents the editor holds -equal write the same plain markdown. +`0`, and an empty `attrs`, `content` or `marks` the absent key, but the doc's `content` — so two +documents the editor holds equal write the same plain markdown. ## `!adf:textBreak{}` parts text CommonMark would join diff --git a/src/markdown/plain/adf-to-plain-markdown.ts b/src/markdown/plain/adf-to-plain-markdown.ts index f324ba3..256f97c 100644 --- a/src/markdown/plain/adf-to-plain-markdown.ts +++ b/src/markdown/plain/adf-to-plain-markdown.ts @@ -58,7 +58,7 @@ export function reduceToPlain(document: AdfDocument): Result { const raw = reduceBlocks(nodeContent(document), { depth: 0, memo: new Map(), path: [] }) return !raw.ok && raw.error.code === blocks.error.code ? raw : blocks } - return success({ content: nodeContent(toEditorNormal({ content: blocks.value, type: 'doc', version: 1 })).slice(), type: 'doc', version: 1 }) + return success(toEditorNormal({ content: blocks.value, type: 'doc', version: 1 })) } function reduceBlocks(nodes: readonly AdfNode[], reduction: Reduction): Result { diff --git a/src/markdown/plain/editor-normal.test.ts b/src/markdown/plain/editor-normal.test.ts index 65ede45..3eb32e7 100644 --- a/src/markdown/plain/editor-normal.test.ts +++ b/src/markdown/plain/editor-normal.test.ts @@ -50,14 +50,15 @@ test('reads negative zero as zero, as JSON does', () => { ) }) -test('reads an empty attrs object, marks array or content array as the absent key', () => { +test('reads an empty attrs object, marks array or content array as the absent key, but on doc', () => { const paragraph: AdfNode = { attrs: {}, content: [{ attrs: {}, marks: [], text: 'a', type: 'text' }, { marks: [{ attrs: {}, type: 'em' }], text: 'b', type: 'text' }], marks: [], type: 'paragraph' } assert.deepEqual(toEditorNormal({ content: [paragraph, { content: [], type: 'rule' }], type: 'doc', version: 1 }), { content: [{ content: [{ text: 'a', type: 'text' }, { marks: [{ type: 'em' }], text: 'b', type: 'text' }], type: 'paragraph' }, { type: 'rule' }], type: 'doc', version: 1, }) - assert.deepEqual(toEditorNormal({ content: [], type: 'doc', version: 1 }), { type: 'doc', version: 1 }) + assert.deepEqual(toEditorNormal({ content: [], type: 'doc', version: 1 }), { content: [], type: 'doc', version: 1 }) + assert.deepEqual(toEditorNormal({ type: 'doc', version: 1 }), { type: 'doc', version: 1 }) }) test('normalizes blocks and mark attributes nesting far past the levels a recursive walk survives', () => { diff --git a/src/markdown/plain/editor-normal.ts b/src/markdown/plain/editor-normal.ts index e50f56c..74b3a74 100644 --- a/src/markdown/plain/editor-normal.ts +++ b/src/markdown/plain/editor-normal.ts @@ -28,6 +28,8 @@ export function toEditorNormal(document: AdfDocument): AdfDocument { return holder }) } + // ADF's schema requires content on doc, so an empty one stays. + if (document.content !== undefined && normal.content === undefined) normal.content = [] return normal } -- 2.52.0 From eccfd12d2d886d5b715a9b69c53e8b33b525358d Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 16:23:23 +0200 Subject: [PATCH 61/67] Cut the empty-key and -0 grammar from the decisions into the spec, and cite item 65 as what ends the image gap --- docs/decisions.md | 18 ++++++------------ spec/flavour.md | 4 ++-- 2 files changed, 8 insertions(+), 14 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 6005569..75d25b2 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -60,22 +60,16 @@ around the leaf. The grammar: `spec/flavour.md` §Inline nodes, **Adjacent text spells an empty object or array. An `attrs`, `content` or `marks` key holding an empty object or array is the reserved key with the -bare value `empty` on every directive — block, inline node and mark: `{attrs=empty}`, -`{content=empty}`, `{marks=empty}`. A writer panel chose the spelling, 5 of 7. A container -spelling `content=empty` closes with no body, so an empty pair stays the node holding no content -key; a leaf spells it too, `!adf:rule {content=empty}`. A node CommonMark spells takes its -directive form to hold an empty key, and a text node holding one rides the inline carry, as does a -node under an `em`, `strong`, `strike`, `code` or `link` mark whose `attrs` is empty, since none of -those spellings holds attributes. Any other value of a reserved key is `unsupported-node-shape`, -except on a block's `marks`, which reads it as the marks array in JSON. +bare value `empty`, so an empty pair stays the node holding no content key. A writer panel chose +the spelling, 5 of 7. The grammar: `spec/flavour.md` §Directives, **Attributes**, and §Marks. ## `-0` is spelled `-0` 2026-10-03, the maintainer. Goal 1. Valid while JSON's own serialization writes `-0` as `0`. -Wherever the flavour writes a number or a JSON value — an attribute, a `json` value, the carry — -`-0` is `-0`, which JSON's grammar reads back as `-0`. An `orderedList` whose `order` is `-0` takes -the directive form, since no list marker spells the sign. +`-0` is spelled `-0` wherever the flavour writes a number or a JSON value, since JSON's grammar +reads it back as `-0` and no list marker spells the sign. The grammar: `spec/flavour.md` §Block +nodes and §The CommonMark blocks. ## Empty markdown is a document of no blocks @@ -170,7 +164,7 @@ opener nests by itself and leaf versus container falls out of the node's content Plain CommonMark is a subset, with carve-outs (`spec/flavour.md`): literal text shaped like a directive, a pipe table or a `~~` pair is claimed, and so is a code fence whose info string opens `adf:` (§The carry fence names the node type) — plus one image gap. `todo.md` item 43 ends the -claims, giving CommonMark its own reader. +claims, giving CommonMark its own reader, and `todo.md` item 65 ends the image gap. ## Tables diff --git a/spec/flavour.md b/spec/flavour.md index 3c4b6fc..c29681d 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -250,8 +250,8 @@ form. the `#` count, so a heading carrying none, or one that is no whole number from 1 to 6, has no CommonMark spelling. - `orderedList` — container of `listItem`, block body. Attributes: `localId` (string), `order` - (number). `order` is the first marker, so a list carrying none, one that is no whole number - from 0, or one whose markers would run past 999999999, has no CommonMark spelling. + (number). `order` is the first marker, so a list carrying none, one that is `-0` or no whole + number from 0, or one whose markers would run past 999999999, has no CommonMark spelling. - `paragraph` — container, inline body. Attributes: `localId` (string). - `rule` — leaf. Attributes: `color` (string, `#rrggbb`), `localId` (string), `style` (`dashed` `dotted` `fade` `sketch` `solid`), `weight` (number, 1–3). -- 2.52.0 From 317262b2409c39f6b7c98d2a4c7a0bb5b36dbda1 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:02:57 +0200 Subject: [PATCH 62/67] Say which emitter refuses which text node in item 55 --- todo.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/todo.md b/todo.md index be6a04d..e2c82b7 100644 --- a/todo.md +++ b/todo.md @@ -78,8 +78,8 @@ in a code block; `directive-syntax.ts` refuses a newline in an emoji, mention or (`unspellable-whitespace`). The inline carry's JSON escapes all of them, so a lossless spelling exists, and §The code list says a cause the carry answers gets no code. Fixing it revises §Which code a cause takes and §Markdown in is a canonical fixpoint, which the maintainer decides. Also a -text node whose `text` is empty or missing, or which holds `content`: `inline-line.ts` refuses it -inline and `fencedTexts` in a code block. Found by the README-goals audit, 2026-10-03. +text node whose `text` is empty or missing, which `inline-line.ts` and `fencedTexts` refuse, or +which holds `content`, which `inline-line.ts` refuses. Found by the README-goals audit, 2026-10-03. ### 45. Replace `isAdfDocument` with a reader returning `Result`. -- 2.52.0 From ee9a7e7562fe21d3db5f239662b95822aec0fece Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:52:48 +0200 Subject: [PATCH 63/67] =?UTF-8?q?Join=20only=20content-free=20text=20nodes?= =?UTF-8?q?=20in=20editor-normal=20ADF,=20type=20checked=20first:=20adfToP?= =?UTF-8?q?lainMarkdown=20on=203,000=20extension=20blocks=20of=20200-key?= =?UTF-8?q?=20parameters=201,752-1,941=20ms=20=E2=86=92=201,034-1,240=20ms?= =?UTF-8?q?=20(1,018-1,283=20ms=20at=20608ef26)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/markdown/plain/adf-to-plain-markdown.test.ts | 6 ++++++ src/markdown/plain/editor-normal.test.ts | 10 ++++++---- src/markdown/plain/editor-normal.ts | 3 ++- 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/src/markdown/plain/adf-to-plain-markdown.test.ts b/src/markdown/plain/adf-to-plain-markdown.test.ts index 28db1b7..d634fe0 100644 --- a/src/markdown/plain/adf-to-plain-markdown.test.ts +++ b/src/markdown/plain/adf-to-plain-markdown.test.ts @@ -344,3 +344,9 @@ test('writes what the editor-normal form of the document writes', () => { assert.equal(plain({ content: [], text: 'hi', type: 'futureInline' }), 'hi\n') assert.equal(plain(paragraph({ marks: [{ attrs: {}, type: 'code' }], text: 'a', type: 'text' }, { marks: [{ type: 'code' }], text: '|b', type: 'text' })), '`a|b`\n') }) + +test('writes a text node holding content the same in either order beside its neighbour', () => { + const holding: AdfNode = { content: [text('inner')], text: 'a', type: 'text' } + assert.equal(plain(paragraph(holding, text('b'))), 'ab\n') + assert.equal(plain(paragraph(text('b'), holding)), 'ba\n') +}) diff --git a/src/markdown/plain/editor-normal.test.ts b/src/markdown/plain/editor-normal.test.ts index 3eb32e7..851c660 100644 --- a/src/markdown/plain/editor-normal.test.ts +++ b/src/markdown/plain/editor-normal.test.ts @@ -50,7 +50,7 @@ test('reads negative zero as zero, as JSON does', () => { ) }) -test('reads an empty attrs object, marks array or content array as the absent key, but on doc', () => { +test('reads an empty attrs object, marks array or content array as the absent key, except on doc', () => { const paragraph: AdfNode = { attrs: {}, content: [{ attrs: {}, marks: [], text: 'a', type: 'text' }, { marks: [{ attrs: {}, type: 'em' }], text: 'b', type: 'text' }], marks: [], type: 'paragraph' } assert.deepEqual(toEditorNormal({ content: [paragraph, { content: [], type: 'rule' }], type: 'doc', version: 1 }), { content: [{ content: [{ text: 'a', type: 'text' }, { marks: [{ type: 'em' }], text: 'b', type: 'text' }], type: 'paragraph' }, { type: 'rule' }], @@ -77,7 +77,9 @@ test('normalizes blocks and mark attributes nesting far past the levels a recurs assert.deepEqual(merged.content?.[0]?.content?.map((text) => text.text), ['ab']) }) -test('joins a text node holding content to its neighbour, as editor-normal forms hold no content to part them', () => { - const paragraph: AdfNode = { content: [{ content: [{ text: 'lost', type: 'text' }], text: 'a', type: 'text' }, { text: 'b', type: 'text' }], type: 'paragraph' } - assert.deepEqual(toEditorNormal({ content: [paragraph], type: 'doc', version: 1 }).content, [{ content: [{ content: [{ text: 'lost', type: 'text' }], text: 'ab', type: 'text' }], type: 'paragraph' }]) +test('joins a text node holding content to no neighbour, so its content is kept', () => { + const holding: AdfNode = { content: [{ text: 'kept', type: 'text' }], text: 'a', type: 'text' } + for (const content of [[holding, { text: 'b', type: 'text' }], [{ text: 'b', type: 'text' }, holding]]) { + assert.deepEqual(toEditorNormal({ content: [{ content, type: 'paragraph' }], type: 'doc', version: 1 }).content, [{ content, type: 'paragraph' }]) + } }) diff --git a/src/markdown/plain/editor-normal.ts b/src/markdown/plain/editor-normal.ts index 74b3a74..3877967 100644 --- a/src/markdown/plain/editor-normal.ts +++ b/src/markdown/plain/editor-normal.ts @@ -7,8 +7,9 @@ type JsonContainer = JsonValue[] | { [key: string]: JsonValue } type NodeHolder = { content?: AdfNode[] } -// The editor's rule is the reader's over the pair's editor-normal forms, which hold no content: a text node's content never parts it. +// The editor's rule is the reader's over the pair's editor-normal forms; a text node holding content never joins, so its content is kept. export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean { + if (previous.type !== 'text' || node.type !== 'text' || nodeContent(previous).length > 0 || nodeContent(node).length > 0) return false return joinsWhenRead(normalNode(previous), normalNode(node)) } -- 2.52.0 From 7b38eba29e108f6758e634a4fc35da2f5d785bc8 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:52:56 +0200 Subject: [PATCH 64/67] Reword the isBareText and adfNodes comments --- src/adf/document.ts | 2 +- src/markdown/parse/inline-content.ts | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/adf/document.ts b/src/adf/document.ts index cd5d0e6..772dd34 100644 --- a/src/adf/document.ts +++ b/src/adf/document.ts @@ -64,7 +64,7 @@ export function emptyKeys(held: { attrs?: AdfAttributes; content?: AdfNode[]; ma return keys } -// A text node holding no attrs or content key and no empty marks: a format spells it as text, where anything more rides a carry. +// A format spells this node as text; anything more rides a carry, save content, which is refused. export function isBareText(node: AdfNode): boolean { return node.type === 'text' && node.attrs === undefined && node.content === undefined && node.marks?.length !== 0 } diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index d02f699..6ca3bba 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -327,7 +327,7 @@ function isTextBreak(item: Inline): item is TextBreak { return 'kind' in item && item.kind === 'textBreak' } -// The nodes the items hold, own-marks ones unwrapped and text breaks dropped. +// The nodes the items hold, with text breaks dropped. function adfNodes(items: readonly Inline[]): AdfNode[] { const nodes: AdfNode[] = [] for (const item of items) { -- 2.52.0 From 8287a483db6fb85c678d618d59beb2d10de5246f Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:53:13 +0200 Subject: [PATCH 65/67] Take round 5 as the comprehension baseline, and tighten four decision sentences --- docs/decisions.md | 50 +++++++++++++++++++++++++---------------------- 1 file changed, 27 insertions(+), 23 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 75d25b2..2c11f61 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -42,8 +42,8 @@ adjacent text nodes, an empty `attrs`, `content` or `marks`, and `-0`. Neither s CommonMark's spelling stays wherever a document holds none of those shapes. The plain reader builds what is written, as `markdownToAdf` does. Only the plain writer is lossy: its reduction reads and writes editor-normal ADF — adjacent text nodes of identical marks and no attributes merged, `-0` as -`0`, and an empty `attrs`, `content` or `marks` the absent key, but the doc's `content` — so two -documents the editor holds equal write the same plain markdown. +`0`, and an empty `attrs`, `content` or `marks` the absent key, except the doc's `content`, which +ADF's schema requires — so two documents the editor holds equal write the same plain markdown. ## `!adf:textBreak{}` parts text CommonMark would join @@ -60,16 +60,16 @@ around the leaf. The grammar: `spec/flavour.md` §Inline nodes, **Adjacent text spells an empty object or array. An `attrs`, `content` or `marks` key holding an empty object or array is the reserved key with the -bare value `empty`, so an empty pair stays the node holding no content key. A writer panel chose -the spelling, 5 of 7. The grammar: `spec/flavour.md` §Directives, **Attributes**, and §Marks. +bare value `empty`, so a container opener and closer with nothing between them stays the node +holding no `content` key. A writer panel chose the spelling, 5 of 7. The grammar: `spec/flavour.md` +§Directives, **Attributes**, and §Marks. ## `-0` is spelled `-0` 2026-10-03, the maintainer. Goal 1. Valid while JSON's own serialization writes `-0` as `0`. `-0` is spelled `-0` wherever the flavour writes a number or a JSON value, since JSON's grammar -reads it back as `-0` and no list marker spells the sign. The grammar: `spec/flavour.md` §Block -nodes and §The CommonMark blocks. +reads it back as `-0`. The grammar: `spec/flavour.md` §Block nodes and §The CommonMark blocks. ## Empty markdown is a document of no blocks @@ -495,15 +495,18 @@ exits 0 on. 2026-10-03, the maintainer. KISS, a technical principle, and its comprehension floor of 7. Valid while `todo.md` items 59, 60, 61 and 62 are open. -A four-seat comprehension panel scored the project 6 against the floor of 7. The scores are the -baseline: a later panel may not score lower on any dimension. +A four-seat comprehension panel scored the project under the floor of 7. Its round-5 scores +(2026-10-03) are the baseline: a later panel may not score lower on any dimension. | Seat | Navigation | Locality | Shape | Self-sufficiency | Overall | |---|---|---|---|---|---| -| Junior | 6.5 | 5.5 | 6 | 5.5 | 6 | -| Mid | 7 | 5.5 | 6 | 5 | 5.5 | -| Senior | 7 | 5.5 | 6.5 | 5.5 | 6 | -| Architect | 6 | 5.5 | 5.5 | 6 | 6 | +| Junior | 6 | 5 | 6 | 5 | 5.5 | +| Mid | 6.5 | 5 | 6 | 4.5 | 5.5 | +| Senior | 6.5 | 5.5 | 6.5 | 5.5 | 6 | +| Architect | 6.5 | 5.5 | 6 | 6 | 6 | + +Three panels on near-identical code scored overall means of 5.88, 5.75 and 5.75, and a seat moves +±0.5 between runs. ## Properties on a fixed seed @@ -665,14 +668,15 @@ them. A primitive knowing neither ADF nor a format stays at `src/` root. A const `spec/flavour.md` draws, and a placement nothing here settles goes beside its only reader, or in what both read where there are two. -A flavour's own code, both directions included, sits in its own directory: `markdown/plain/`. -Otherwise each format directory parts into `emit/` (ADF→format) and `parse/` (format→ADF), the rest -of it holding what both directions read. A construct's reader lives there beside the regex the -emitter escapes against, so the two cannot drift; a reader with no emit counterpart goes in -`parse/`, unless it is part of a construct that side already holds — a grammar stays in one file -rather than splitting across the seam. A rule both directions must answer alike — whether a list -marker interrupts a paragraph — is one function there too, never a copy per direction, however -conservative the copy would be. Where the rule is the emitter's own choice, input consults it rather -than restating it, and that is the only value import `parse/` takes from `emit/` — -`commonMarkSpelling` and `openingLinkTakesDirective` — so no fixture the emitter writes can be -refused, and a spelling the emitter refuses gives its own error rather than a second name for it. +A flavour's own code, both directions included, sits in its own directory: `markdown/plain/`; +`todo.md` item 62 moves what still sits in `parse/` and `emit/`. Otherwise each format directory +parts into `emit/` (ADF→format) and `parse/` (format→ADF), the rest of it holding what both +directions read. A construct's reader lives there beside the regex the emitter escapes against, so +the two cannot drift; a reader with no emit counterpart goes in `parse/`, unless it is part of a +construct that side already holds — a grammar stays in one file rather than splitting across the +seam. A rule both directions must answer alike — whether a list marker interrupts a paragraph — is +one function there too, never a copy per direction, however conservative the copy would be. Where +the rule is the emitter's own choice, input consults it rather than restating it, and that is the +only value import `parse/` takes from `emit/` — `commonMarkSpelling` and `openingLinkTakesDirective` +— so no fixture the emitter writes can be refused, and a spelling the emitter refuses gives its own +error rather than a second name for it. -- 2.52.0 From 80a45b69dae9a835ec819b8481fd746c1265806c Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:53:30 +0200 Subject: [PATCH 66/67] Sharpen items 55, 62, 63, 64 and 65 --- todo.md | 39 ++++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/todo.md b/todo.md index e2c82b7..845aaa4 100644 --- a/todo.md +++ b/todo.md @@ -24,14 +24,14 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| -| 63 | 0.2.0 | defect | **Refuse a cyclic input as `not-an-adf-document` in place of looping forever.** | 2 | 2 | 6 | 9 | 1 | 27.5 | +| 63 | 0.2.0 | defect | **Refuse a cyclic input as `not-an-adf-document`.** | 2 | 2 | 6 | 9 | 1 | 27.5 | | 7 | 0.2.0 | | **Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF.** | 6 | 9 | 9 | 9 | 2, 3 | 25.8 | -| 55 | 0.2.0 | defect | **Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start, a newline inside an emoji, mention or status, or a text node with empty, missing or content-holding text.** | 5 | 5 | 7 | 9 | 1 | 25.8 | +| 55 | 0.2.0 | defect | **Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start, a newline inside an emoji, mention or status, or a text node whose text is empty or missing, or which holds content.** | 5 | 5 | 7 | 9 | 1 | 25.8 | | 45 | 0.2.0 | | **Replace `isAdfDocument` with a reader returning `Result`.** | 2 | 3 | 6 | 8 | 1 | 25.2 | | 6 | 0.2.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 3 | 24.7 | -| 64 | 0.2.0 | defect | **Read every JSON key the emitter writes back unchanged on V8, with a fixture whose key holds `\`, `"` or a control character.** | 4 | 4 | 6 | 8 | 1, 7 | 23.0 | +| 64 | 0.2.0 | defect | **Read back unchanged on V8 every JSON key the emitter writes, with a fixture whose key holds `\`, `"` or a control character.** | 4 | 4 | 6 | 8 | 1, 7 | 23.0 | | 43 | 0.2.0 | decision | **Give each markdown input its own reader, strict to its own standard.** | 6 | 7 | 8 | 9 | 3, 4 | 22.3 | -| 65 | 0.2.0 | defect | **Read a mid-text or titled CommonMark image as Goal 6 allows, in place of refusing it with `unmappable-image`.** | 5 | 5 | 7 | 7 | 3, 4, 6 | 18.7 | +| 65 | 0.2.0 | defect | **Read a mid-text or titled CommonMark image as Goal 6 allows.** | 5 | 5 | 7 | 7 | 3, 4, 6 | 18.7 | | 49 | 0.2.0 | | **Read a list whose bullet or ordered delimiter changes as two lists in the CommonMark reader.** | 4 | 5 | 5 | 8 | 3, 4 | 17.2 | | 61 | 0.2.0 | decision | **Have the emitter ask the inline reader how a line reads back, in place of `line-escaping.ts` predicting it.** | 6 | 8 | 3 | 8 | 1 | 14.0 | | 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 5 | 13.0 | @@ -40,7 +40,7 @@ | 60 | 0.2.0 | decision | **Collect the questions `parse/` asks `emit/` into one named module.** | 3 | 4 | 2 | 6 | 2 | 10.7 | | 59 | 0.2.0 | decision | **Group the directive grammar into `src/markdown/directive/`, move `Read` to `result.ts`, and move `Flavour` to `markdown/flavour.ts`.** | 3 | 5 | 2 | 6 | 2 | 10.4 | | 50 | 0.2.0 | | **Read `[](/url)` and `[]()` as CommonMark's empty link.** | 4 | 4 | 3 | 6 | 3, 4, 6 | 10.4 | -| 62 | 0.2.0 | decision | **Move the plain flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/plain/`, so `plain/` and `emit/` no longer import each other.** | 4 | 5 | 2 | 6 | 2 | 9.4 | +| 62 | 0.2.0 | decision | **Move the plain flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/plain/`, so `emit/` no longer imports `plain/`.** | 4 | 5 | 2 | 6 | 2 | 9.4 | | 38 | 0.3.0 | | **Spell a lone surrogate in a text node so it survives a UTF-8 encode.** | 2 | 2 | 4 | 7 | 1 | 19.5 | | 47 | 0.3.0 | | **Open the README with what the package is, what it does and for whom.** | 1 | 4 | 7 | 9 | 9 | 14.0 | | 34 | 0.3.0 | | **Read emphasis flanking by the whole character beside an astral symbol.** | 2 | 3 | 3 | 6 | 3, 4 | 12.6 | @@ -57,7 +57,7 @@ ## Details -### 63. Refuse a cyclic input as `not-an-adf-document` in place of looping forever. +### 63. Refuse a cyclic input as `not-an-adf-document`. `adfDocumentFault` walks with `isNodeArray` (`adf/document.ts`) and `isJsonValue` (`json-value.ts`), worklists that record no visited object, so a node whose `content` holds itself hangs @@ -70,7 +70,7 @@ Lands after items 6, 59, 60, 61 and 62. The CommonMark spec suite also runs agai `markdownToHtml`. The README documents HTML as it documents markdown, and its tagline and `package.json`'s `description` regain HTML. -### 55. Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start, a newline inside an emoji, mention or status, or a text node with empty, missing or content-holding text. +### 55. Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start, a newline inside an emoji, mention or status, or a text node whose text is empty or missing, or which holds content. `inline-line.ts` refuses a carriage return or NUL in text (`unspellable-character`) and a paragraph opening with a code-span run (`unspellable-line-start`); `fencedTexts` refuses the same characters @@ -93,14 +93,14 @@ Element-by-element mapping, the `data-*` fidelity scheme, the opaque-carry form, foreign-element set `htmlToAdf` accepts — the set `markdownToAdf` shares (`spec/flavour.md` §Raw HTML in input). The set sorts per `docs/decisions.md` §Foreign HTML sorts three ways. -### 64. Read every JSON key the emitter writes back unchanged on V8, with a fixture whose key holds `\`, `"` or a control character. +### 64. Read back unchanged on V8 every JSON key the emitter writes, with a fixture whose key holds `\`, `"` or a control character. `property-harness.ts` strips `\`, `"` and control characters from generated JSON keys, citing V8's `JSON.parse` returning a wrong key for an escaped backslash. The library reads `json` attributes (`directive-syntax.ts`) and both carries (`opaque-carry.ts`) through that `JSON.parse`, so on Node, -Deno and Chrome the round-trip would refuse its own output. Confirm with a fixture first; then read -JSON with our own parser or record the gap with an ending item. Found by the README-goals audit, -2026-10-03. +Deno and Chrome the round-trip would refuse its own output or read back a different key; the fixture +tells which. Confirm with a fixture first; then read JSON with our own parser or record the gap with +an ending item. Found by the README-goals audit, 2026-10-03. ### 43. Give each markdown input its own reader, strict to its own standard. @@ -110,11 +110,11 @@ a code fence whose info string opens `adf:` becomes the block carry where Common caller names the markdown it hands in: CommonMark, read as its spec says, or the lossless flavour, read as `spec/flavour.md` says. Breaking: `MIGRATION.md` says which call a caller takes. -### 65. Read a mid-text or titled CommonMark image as Goal 6 allows, in place of refusing it with `unmappable-image`. +### 65. Read a mid-text or titled CommonMark image as Goal 6 allows. -§CommonMark is a subset keeps the image gap and no item ends it: a bot's `See ![diagram](url) here` -or a titled image is refused, 14 examples in `corpus/commonmark-spec/refusals.json`. Settle what such -an image builds — Goal 6 drops form and keeps the target — and have the decision cite this item. +A mid-text or titled image, such as a bot's `See ![diagram](url) here`, is refused: 14 examples in +`corpus/commonmark-spec/refusals.json`. Settle what such an image builds. Goal 6 drops form and +keeps the target. ### 49. Read a list whose bullet or ordered delimiter changes as two lists in the CommonMark reader. @@ -173,12 +173,13 @@ input. Breaking, so it ships beside item 43: `MIGRATION.md`'s Readings table gai `pending` exceptions go, and its spelling leaves the README's "Four CommonMark spellings" bullet, which counts one fewer. -### 62. Move the plain flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/plain/`, so `plain/` and `emit/` no longer import each other. +### 62. Move the plain flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/plain/`, so `emit/` no longer imports `plain/`. Reading: the alert and task-marker reads in `parse/markdown-to-adf.ts`, the `mintTaskIds` call and -the `inlineLeaves` use. Writing: `spellPlainBlock`, `quotedUnder`, `tryTaskList` and `taskBlocks` -in `emit/adf-to-markdown.ts`. Every comprehension seat on 2026-10-03 named the plain flavour's -spread across three directories. +the `inlineLeaves` use. Writing: `spellPlainBlock`, `quotedUnder`, `tryTaskList` and `taskBlocks` in +`emit/adf-to-markdown.ts`. And `highlightDelimiter`, `highlightFlanking` and `Flavour` move out of +`plain/conventions.ts`, which `emit/` imports them from. Every comprehension reader on 2026-10-03 +named the plain flavour's spread across three directories. ### 38. Spell a lone surrogate in a text node so it survives a UTF-8 encode. -- 2.52.0 From 31d3f53a58e8b765bf606653bdcbb9ef9a16e9e0 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Sat, 3 Oct 2026 17:54:41 +0200 Subject: [PATCH 67/67] Leave Flavour's move to item 59 alone --- todo.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/todo.md b/todo.md index 845aaa4..f4d656c 100644 --- a/todo.md +++ b/todo.md @@ -177,8 +177,8 @@ which counts one fewer. Reading: the alert and task-marker reads in `parse/markdown-to-adf.ts`, the `mintTaskIds` call and the `inlineLeaves` use. Writing: `spellPlainBlock`, `quotedUnder`, `tryTaskList` and `taskBlocks` in -`emit/adf-to-markdown.ts`. And `highlightDelimiter`, `highlightFlanking` and `Flavour` move out of -`plain/conventions.ts`, which `emit/` imports them from. Every comprehension reader on 2026-10-03 +`emit/adf-to-markdown.ts`. And `highlightDelimiter` and `highlightFlanking` move out of +`plain/conventions.ts`, which `emit/` imports them from; item 59 moves `Flavour`. Every comprehension reader on 2026-10-03 named the plain flavour's spread across three directories. ### 38. Spell a lone surrogate in a text node so it survives a UTF-8 encode. -- 2.52.0