diff --git a/AGENTS.md b/AGENTS.md index 25442ed..614fcc0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,8 +10,14 @@ In `docs/decisions.md`: - Plain markdown is a flavour of the grammar - The round-trip is the product - Markdown in is a canonical fixpoint -- Equality is editor-normal +- Equality is deep +- `!adf:textBreak{}` parts text CommonMark would join +- An empty key spells `empty` +- `-0` is spelled `-0` +- Empty markdown is a document of no blocks - Unknown nodes ride the carry +- The carry fence names the node type +- A code block is a fence per text node - Foreign HTML sorts three ways - Names stay text - Directives under `!adf:` @@ -42,6 +48,7 @@ In `docs/decisions.md`: - Firefox reads the build - The coverage floors - The size ratchet +- The project ships under the comprehension floor until items 59, 60, 61 and 62 land - Properties on a fixed seed - The CommonMark suite checks three ways - The flavour spec is read as a source @@ -162,12 +169,16 @@ nearest text, a candidate entry in that file's voice, and the instance it yields keeps collecting instances is wrong: rewrite it. Which output the audience expects — README goal 5 — is settled by a reader panel rather than -asked: three fresh-context readers, one per README persona the conversion serves, each given only +asked: three fresh-context readers, one per README persona the question serves, each given only `## Audience` and the input, writing what they expect before picking among outputs the goals allow, rendered, shuffled, with no rationale and nothing saying what is implemented. Three agreeing settle it; otherwise four more read, five of seven settle it, and less is a missing goal, asked. The verdict lands in `docs/decisions.md`. +A writer panel settles every new or changed markdown or HTML spelling: a reader panel whose readers +are the people who read and write that format (README `## Audience`). A panel of the developer +personas settles a question about what an app relies on. + ### Stated numbers A stated number — 500 levels, the branch floor — is kept; a chunk that cannot keep it asks, naming @@ -180,3 +191,14 @@ and `todo.md` and trusting them over anything remembered from earlier iterations is a thin driver: each chunk's work runs in a fresh-context subagent holding this file as its charter, and the driver only relays maintainer questions, runs the review flow, merges, and cleans up. The loop stops when only maintainer-reserved acts remain. + +## 8. Scoring run + +The comprehension panel's fill-ins: + +- Language: TypeScript. +- Kind: a pure-function document converter with hand-written parsers and emitters. +- Domain: Atlassian Document Format and CommonMark parsing. +- Domain docs: the CommonMark spec, ADF's JSON schema and `spec/flavour.md`. +- 3am question: a viewer/editor app reports that a document it saved comes back with two text + nodes merged and a mark gone after `markdownToAdf(adfToMarkdown(doc))`. diff --git a/CHANGELOG.md b/CHANGELOG.md index 442e445..3123e48 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,10 +2,21 @@ ## Unreleased -- **Breaking:** directives, the opaque carry among them (now `carry`), are spelled under an `!adf:` - prefix (`!adf:name … !adf:/name`, `!adf:name[content]{attrs}`, `!adf:name arg {attrs}`) in place - of the `:::`/`::`/`:name` forms: text holding an unescaped `!adf:` is claimed, and `adf` is an - ordinary code block language. Convert stored markdown per `MIGRATION.md`. +- **Breaking:** directives, the inline opaque carry among them (now `!adf:carry{json="…"}`), are + spelled under an `!adf:` prefix (`!adf:name … !adf:/name`, `!adf:name[content]{attrs}`, + `!adf:name arg {attrs}`) in place of the `:::`/`::`/`:name` forms: text holding an unescaped + `!adf:` is claimed. Convert stored markdown per `MIGRATION.md`. +- **Breaking:** the block carry is a code fence whose info string `adf:` names the node's + type, its body the node's JSON without `type`, and a code fence whose info string opens `adf:` is + claimed, and `adf` is an ordinary code block language. Convert stored markdown per `MIGRATION.md`. +- **Breaking:** `markdownToAdf` and `plainMarkdownToAdf` read markdown holding no block as a + document whose `content` is empty, as Atlassian's schema requires; `!adf:doc {content=none}` + spells a document holding no `content` key. +- `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc` as `JSON.parse` builds it: two adjacent text + nodes CommonMark would read back as one are parted by `!adf:textBreak{}`, an empty `attrs`, + `content` or `marks` is spelled `{attrs=empty}`, `{content=empty}` or `{marks=empty}`, `-0` is + spelled `-0`, and a `codeBlock` of several text nodes is a fence per node. A `codeBlock` holding + other than plain text nodes rides the block carry, where it was refused. - **Breaking:** `unspellable-link` leaves `ConvertErrorCode`; a link whose `href` or `title` no CommonMark escape spells is written as `!adf:link[text]{attrs}`. - **Breaking:** some directive refusals carry `malformed-directive` where they carried diff --git a/MIGRATION.md b/MIGRATION.md index 2dfb73f..a5fb98d 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -5,8 +5,9 @@ Directives moved under the `!adf:` prefix. `0.2.0` reads `0.1.0`'s spelling without an error, turning each directive into text and each carried node into an `adf` code block. Before `0.2.0` reads any `0.1.0` markdown, convert what is stored or in flight (an open editor, a queue) with the -recipe below, and rewrite markdown your code writes or matches (templates, prompts, patterns) by -the tables below. Stored ADF needs no change. +recipe below, and rewrite markdown your code writes or matches (templates, prompts, patterns) by the +tables below. Stored ADF needs one change: give a document holding no `content` key `content: []`. +`0.1.0` built that shape from empty markdown, meaning the empty document. ### Convert markdown @@ -22,15 +23,17 @@ import { markdownToAdf as markdownToAdf010 } from 'adf-codec-0.1' function migrateMarkdown(stored: string) { const parsed = markdownToAdf010(stored) - return parsed.ok ? adfToMarkdown(parsed.value) : parsed + // 0.1.0 dropped an empty content array, so a document with no content key meant an empty one. + return parsed.ok ? adfToMarkdown({ ...parsed.value, content: parsed.value.content ?? [] }) : parsed } ``` - Convert each document once: a second pass can return ok while turning the directives into text. Stop `0.1.0` writing first, and record which documents are converted. - A refusal carrying `position` is `0.1.0`'s parse, which refused that markdown before too. One - without is `0.2.0`'s emit: store the document `markdownToAdf010` read as ADF rather than keeping - the unconverted markdown. + without is `0.2.0`'s emit: store the document `markdownToAdf010` read as ADF, with + `content: parsed.value.content ?? []` as the recipe gives it, rather than keeping the unconverted + markdown. ### Spellings @@ -40,12 +43,12 @@ function migrateMarkdown(stored: string) { | `::media {id=a type=file}` | `!adf:media {id=a type=file}` | | `::taskItem TODO {localId=i}`: an empty `caption`, `decisionItem`, `paragraph` or `taskItem`, or an empty `heading` carrying `localId` | `!adf:taskItem TODO {localId=i}` then `!adf:/taskItem` | | `:mention[@Mikael]{id=5b10a2}` | `!adf:mention[@Mikael]{id=5b10a2}` | -| the `adf` code fence and `:adf{json="…"}` | the `carry` code fence and `!adf:carry{json="…"}` | +| the `adf` code fence and `:adf{json="…"}` | the `adf:` code fence, its JSON without `type`, and `!adf:carry{json="…"}` | | `\:` keeps a directive literal | `\!adf:` keeps a directive literal | | `:adf{json="…"}` carrying a link for its `collection`, `id` or `occurrenceKey` | `!adf:link[text]{attrs}` | A colon run and `:name[` are plain text now, and `adf` an ordinary code block language; text -holding an unescaped `!adf:` and a `carry` fence are claimed instead. +holding an unescaped `!adf:` and a code fence whose info string opens `adf:` are claimed instead. ### Readings @@ -54,6 +57,8 @@ Markdown the spelling table leaves alone, which `0.2.0` reads as a different doc | Input | `0.1.0` | `0.2.0` | | --- | --- | --- | | a link whose text already holds one (`[ab](/v)`) | marks every node the inner link does not, splitting the outer link around it | leaves the outer brackets literal text; write the pieces as separate links to keep them | +| markdown holding no block (`markdownToAdf("")`) | `{ type: 'doc', version: 1 }` | `{ content: [], type: 'doc', version: 1 }`; `!adf:doc {content=none}` reads as the former | +| a code fence whose info string opens `adf:` (```` ```adf:x ````) | a `codeBlock` with that language | the block carry, refusing a body that is not one node's canonical JSON; write `!adf:codeBlock {language="adf:x"}` around a bare fence to keep the code block | ### Error codes @@ -63,6 +68,7 @@ it named converts. | Input | `0.1.0` | `0.2.0` | | --- | --- | --- | | a link whose `href` or `title` no CommonMark escape spells, on emit | `unspellable-link` | spells `!adf:link[text]{attrs}` | +| a `codeBlock` holding other than plain text nodes, on emit | `unsupported-node-shape` | rides the block carry | | a leaf node given a body (`media`, `listBreak`) | `unsupported-node-shape` | `malformed-directive` | | a node with a block body written as a leaf (`panel`) | `unsupported-node-shape` | `malformed-directive` | | an empty node the `::taskItem` spelling row names, written as a leaf | parses | `malformed-directive` | diff --git a/README.md b/README.md index 13cac24..e0e48f1 100644 --- a/README.md +++ b/README.md @@ -52,6 +52,10 @@ which is free text. - **LLM/agent pipeline** — hands documents to a model as markdown and writes the edits back. Relies on the round-trip and on markdown a reader half-knowing the lossless flavour can still edit. +Behind those apps, the people who read and write the markdown, and later the HTML: product +managers, engineers and support agents working in Atlassian products through a plugin or another +UI. They know markdown and not ADF, and rely on every spelling saying what it means to them. + ## The shape ```sh @@ -171,7 +175,7 @@ Parsing — `markdownToAdf` and `plainMarkdownToAdf`, and `htmlToAdf` at `0.2.0` | Code | Fires when | What you can do | | --- | --- | --- | -| `malformed-directive` | an `!adf:` the grammar cannot read — a prefix completing no directive, an unclosed container, `[content]` or `{attrs}`, a closer with no container of its name open, a leaf given a body, `{attrs}` out of order or duplicated, invalid JSON in a `carry` | write the spelling the message names, or escape the prefix — `\!adf:`, block and inline alike — to keep it literal text | +| `malformed-directive` | an `!adf:` the grammar cannot read — a prefix completing no directive, an unclosed container, `[content]` or `{attrs}`, a closer with no container of its name open, a leaf given a body, `{attrs}` out of order or duplicated, invalid JSON in an opaque carry | write the spelling the message names, or keep it literal: escape the prefix — `\!adf:`, block and inline alike — or, for a code fence whose info string opens `adf:`, drop the info string and wrap the fence in `!adf:codeBlock {language="adf:…"}` | | `malformed-pipe-table` | a pipe row that is no pipe table — a missing or ragged `---` delimiter row, an alignment colon in it, or a row not opening with a pipe | open every row with a pipe and give the delimiter row the header's cell count; to keep the lines literal text instead, escape the leading pipe of every one — escaping a single row leaves the next to open a fresh table and fail the same way | | `unknown-directive-name` | a directive whose name is no node or mark this version spells | check the name in `spec/flavour.md`, or escape the prefix as `\!adf:`; the spelling itself is well formed, so a later minor may give the name meaning | | `unmappable-html` | the input holds an HTML construct the documented element set does not map, a comment and a processing instruction among them — at this version that is every raw HTML construct in markdown, the element set landing at `0.2.0` | remove the construct, or write what it holds in the lossless flavour | @@ -194,25 +198,28 @@ emit refuses: | `unspellable-line-start` | a paragraph line begins with a code span whose backticks would read back as a code fence | put any text before the code span | | `unspellable-whitespace` | an `emoji`, `mention` or `status` holds a newline in the text its inline directive spells in the content slot | replace it with a space — an inline directive never spans lines | | `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it | -| `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark | write the shape the message names; `spec/flavour.md` lists every type's attributes and body | +| `unsupported-node-shape` | a node carries an attribute, value, argument or body its type does not take, or lacks one it needs — or markdown writes as a directive a node or mark the lossless flavour spells as CommonMark, or a reserved directive stands out of place: `!adf:textBreak{}` or `!adf:listBreak` parting nothing, `!adf:doc` anywhere but as the whole document | fix what the message names; `spec/flavour.md` lists every type's attributes and body | ## The guarantees Serves Goals 1, 3 and 4. -- `markdownToAdf(adfToMarkdown(doc))` equals `doc` — unknown node types included, carried opaquely +- `markdownToAdf(adfToMarkdown(doc))` deep-equals `doc` as JSON, for a document of plain objects + as `JSON.parse` builds them — every key and value as `doc` holds it, adjacent text nodes, an + empty `attrs`, `content` or `marks` and `-0` included, and unknown node types carried opaquely ([`docs/decisions.md`](https://gitea.larvit.se/larvit/adf-codec/src/branch/main/docs/decisions.md#unknown-nodes-ride-the-carry)). - Markdown this library reads, and markdown it writes, means what the CommonMark spec says; from `0.2.0`, well-formed HTML means what the HTML standard says, read or written. The bullets below name every exception. - Plain CommonMark is valid input to `markdownToAdf` apart from the raw HTML `unmappable-html` - names, with three carve-outs — literal text matching directive, pipe-table or strikethrough - syntax is claimed (escapable — `spec/flavour.md`) — and one gap: a CommonMark image fits only as - its own title-less paragraph; mid-text and titled images are error results, save an image inside - another's description, which flattens into the alt text. Converting back yields the library's - canonical spelling, which round-trips byte-identically — where it converts back at all: a parse - succeeding is no promise of that, so keep the source until the way back succeeds. - ``` ` `` ` ``` reads cleanly and then refuses. + names, with four carve-outs — literal text matching directive, pipe-table or strikethrough syntax, + and a code fence whose info string opens `adf:`, are claimed (each can be kept literal — + `spec/flavour.md`) — and one gap: a CommonMark image fits only as its own title-less paragraph; + mid-text and titled images are error results, save an image inside another's description, which + flattens into the alt text. Converting back yields the library's canonical spelling, which + round-trips byte-identically — where it converts back at all: a parse succeeding is no promise of + that, so keep the source until the way back succeeds. ``` ` `` ` ``` reads cleanly and then + refuses. - Four CommonMark spellings parse without an error and build a document the reference implementation renders differently: `[](/url)` and `[]()` stay literal text against CommonMark's empty link, a list continuing past a marker change stays one list against CommonMark's two, a @@ -235,7 +242,7 @@ Serves Goals 1, 3 and 4. makes a call loop forever. - The emitted formats are semver surface ([`docs/decisions.md`](https://gitea.larvit.se/larvit/adf-codec/src/branch/main/docs/decisions.md#the-formats-are-api)). -- **`0.2.0`** — `htmlToAdf(adfToHtml(doc))` equals `doc`; fidelity HTML cannot express rides +- **`0.2.0`** — `htmlToAdf(adfToHtml(doc))` deep-equals `doc`; fidelity HTML cannot express rides `data-*` attributes. Foreign HTML maps a documented element set, which markdown's raw HTML reads through as well, and a construct outside it is an error; well-formed HTML only — no tag-soup recovery. diff --git a/browser-tests/convert-corpus.js b/browser-tests/convert-corpus.js index 28a6808..84bba51 100644 --- a/browser-tests/convert-corpus.js +++ b/browser-tests/convert-corpus.js @@ -1,16 +1,18 @@ try { const { adfToMarkdown, isAdfDocument, markdownToAdf } = await import('/dist/index.js') + // WebDriver's JSON reads -0 back as 0, so a document crosses as JSON text with -0 tagged; run.js revives it. + const spelled = (result) => (result.ok ? { ok: true, value: JSON.stringify(result.value, (_, value) => (Object.is(value, -0) ? '\u0000-0' : value)) } : result) window.convertCorpus = (corpus) => ({ errors: corpus.errors.map(({ markdown, name }) => ({ name, parsed: markdownToAdf(markdown) })), - normalization: corpus.normalization.map(({ markdown, name }) => ({ name, parsed: markdownToAdf(markdown) })), + normalization: corpus.normalization.map(({ markdown, name }) => ({ name, parsed: spelled(markdownToAdf(markdown)) })), realPayloads: corpus.realPayloads.map(({ json, name }) => { const adf = JSON.parse(json) const emitted = adfToMarkdown(adf) - return { emitted, isDocument: isAdfDocument(adf), name, parsed: emitted.ok ? markdownToAdf(emitted.value) : undefined } + return { emitted, isDocument: isAdfDocument(adf), name, parsed: emitted.ok ? spelled(markdownToAdf(emitted.value)) : undefined } }), roundTrip: corpus.roundTrip.map(({ json, markdown, name }) => { const adf = JSON.parse(json) - return { emitted: adfToMarkdown(adf), isDocument: isAdfDocument(adf), name, parsed: markdownToAdf(markdown) } + return { emitted: adfToMarkdown(adf), isDocument: isAdfDocument(adf), name, parsed: spelled(markdownToAdf(markdown)) } }), }) } catch (cause) { diff --git a/browser-tests/run.js b/browser-tests/run.js index 0369402..b0652df 100644 --- a/browser-tests/run.js +++ b/browser-tests/run.js @@ -2,7 +2,6 @@ import assert from 'node:assert/strict' import { createServer } from 'node:http' import { extname, join } from 'node:path' import { readFileSync, readdirSync } from 'node:fs' -import { toEditorNormal } from '../dist/adf/editor-normal.js' const contentTypes = { '.html': 'text/html; charset=utf-8', '.js': 'text/javascript' } const driver = 'http://127.0.0.1:4444' @@ -50,6 +49,11 @@ function fixtureNames(kind, extension) { .sort() } +// convert-corpus.js tags -0 so it survives WebDriver's JSON. +function revived(text) { + return JSON.parse(text, (_, value) => (value === '\u0000-0' ? -0 : value)) +} + function refusal(result) { return result.ok ? '' : `${result.error.code}: ${result.error.message}` } @@ -107,7 +111,7 @@ for (const [index, { json, markdown, name }] of corpus.roundTrip.entries()) { assert.ok(result.emitted.ok, `it did not emit — ${refusal(result.emitted)}`) assert.equal(result.emitted.value, markdown) assert.ok(result.parsed.ok, `it did not parse — ${refusal(result.parsed)}`) - assert.deepEqual(toEditorNormal(result.parsed.value), JSON.parse(json)) + assert.deepEqual(revived(result.parsed.value), JSON.parse(json)) }) } @@ -115,7 +119,7 @@ for (const [index, { name }] of corpus.normalization.entries()) { const result = results.normalization[index] checking(name, result, () => { assert.ok(result.parsed.ok, `it did not parse — ${refusal(result.parsed)}`) - assert.deepEqual(toEditorNormal(result.parsed.value), JSON.parse(fixture(name, '.json'))) + assert.deepEqual(revived(result.parsed.value), JSON.parse(fixture(name, '.json'))) }) } @@ -125,7 +129,7 @@ for (const [index, { json, name }] of corpus.realPayloads.entries()) { assert.ok(result.isDocument, `${name}.json is no ADF document`) assert.ok(result.emitted.ok, `it did not emit — ${refusal(result.emitted)}`) assert.ok(result.parsed.ok, `it did not parse back — ${refusal(result.parsed)}`) - assert.deepEqual(toEditorNormal(result.parsed.value), JSON.parse(json)) + assert.deepEqual(revived(result.parsed.value), JSON.parse(json)) }) } diff --git a/corpus/README.md b/corpus/README.md index 097d0af..3629e43 100644 --- a/corpus/README.md +++ b/corpus/README.md @@ -18,8 +18,9 @@ One directory per contract kind: `reason`. `kind` is `mark-model` (the permanent count divergence from ADF's mark-per-text-node model), `unspellable` (parses but the flavour has no spelling) or `pending` (a parser gap). -JSON is editor-normal (`docs/decisions.md` §Equality is editor-normal), two-space indent, keys -sorted. `spec.json` is the vendored, upstream machine-readable suite, byte-exact from -[spec.commonmark.org](https://spec.commonmark.org/0.31.2/spec.json) (CommonMark 0.31.2, © John -MacFarlane, [CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)), and is not -re-serialized by the corpus gate. +JSON is two-space indent, keys sorted, and a document read back must deep-equal the fixture's +(`docs/decisions.md` §Equality is deep). `spec.json` is the vendored, upstream machine-readable +suite, byte-exact from [spec.commonmark.org](https://spec.commonmark.org/0.31.2/spec.json) +(CommonMark 0.31.2, © John MacFarlane, +[CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)), and is not re-serialized by the +corpus gate. diff --git a/corpus/errors/carry-fence-type-spellable.error b/corpus/errors/carry-fence-type-spellable.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/carry-fence-type-spellable.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/carry-fence-type-spellable.md b/corpus/errors/carry-fence-type-spellable.md new file mode 100644 index 0000000..1b73fc1 --- /dev/null +++ b/corpus/errors/carry-fence-type-spellable.md @@ -0,0 +1,5 @@ +```adf: +{ + "type": "blockCard" +} +``` diff --git a/corpus/errors/carry-fence-type-unspellable.error b/corpus/errors/carry-fence-type-unspellable.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/carry-fence-type-unspellable.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/carry-fence-type-unspellable.md b/corpus/errors/carry-fence-type-unspellable.md new file mode 100644 index 0000000..26231ce --- /dev/null +++ b/corpus/errors/carry-fence-type-unspellable.md @@ -0,0 +1,3 @@ +```adf:\\ +{} +``` diff --git a/corpus/errors/carry-fence-type.error b/corpus/errors/carry-fence-type.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/carry-fence-type.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/carry-fence-type.md b/corpus/errors/carry-fence-type.md new file mode 100644 index 0000000..7a266ef --- /dev/null +++ b/corpus/errors/carry-fence-type.md @@ -0,0 +1,5 @@ +```adf:blockCard +{ + "type": "blockCard" +} +``` diff --git a/corpus/errors/carry-invalid-json.md b/corpus/errors/carry-invalid-json.md index 1cd3435..febfe45 100644 --- a/corpus/errors/carry-invalid-json.md +++ b/corpus/errors/carry-invalid-json.md @@ -1,3 +1,3 @@ -```carry -{"type": +```adf:blockCard +{"attrs": ``` diff --git a/corpus/errors/code-block-empty-attrs-fence-language.error b/corpus/errors/code-block-empty-attrs-fence-language.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/code-block-empty-attrs-fence-language.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/code-block-empty-attrs-fence-language.md b/corpus/errors/code-block-empty-attrs-fence-language.md new file mode 100644 index 0000000..75c0abb --- /dev/null +++ b/corpus/errors/code-block-empty-attrs-fence-language.md @@ -0,0 +1,8 @@ +!adf:codeBlock {attrs=empty} +```js +a +``` +```js +b +``` +!adf:/codeBlock diff --git a/corpus/errors/code-block-empty-fence.error b/corpus/errors/code-block-empty-fence.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/code-block-empty-fence.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/code-block-empty-fence.md b/corpus/errors/code-block-empty-fence.md new file mode 100644 index 0000000..481ca7a --- /dev/null +++ b/corpus/errors/code-block-empty-fence.md @@ -0,0 +1,7 @@ +!adf:codeBlock +``` +a +``` +``` +``` +!adf:/codeBlock diff --git a/corpus/errors/code-block-fence-languages.error b/corpus/errors/code-block-fence-languages.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/code-block-fence-languages.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/code-block-fence-languages.md b/corpus/errors/code-block-fence-languages.md new file mode 100644 index 0000000..077c903 --- /dev/null +++ b/corpus/errors/code-block-fence-languages.md @@ -0,0 +1,8 @@ +!adf:codeBlock +```js +a +``` +```ts +b +``` +!adf:/codeBlock diff --git a/corpus/errors/directive-slot-carried-marks.error b/corpus/errors/directive-slot-carried-marks.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/directive-slot-carried-marks.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/directive-slot-carried-marks.md b/corpus/errors/directive-slot-carried-marks.md new file mode 100644 index 0000000..c593b3e --- /dev/null +++ b/corpus/errors/directive-slot-carried-marks.md @@ -0,0 +1 @@ +!adf:status[!adf:carry{json="{\"marks\":[],\"text\":\"x\",\"type\":\"text\"}"}] diff --git a/corpus/errors/document-directive-bare.error b/corpus/errors/document-directive-bare.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/document-directive-bare.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/document-directive-bare.md b/corpus/errors/document-directive-bare.md new file mode 100644 index 0000000..82cf095 --- /dev/null +++ b/corpus/errors/document-directive-bare.md @@ -0,0 +1 @@ +!adf:doc diff --git a/corpus/errors/document-directive-misplaced.error b/corpus/errors/document-directive-misplaced.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/document-directive-misplaced.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/document-directive-misplaced.md b/corpus/errors/document-directive-misplaced.md new file mode 100644 index 0000000..f3158b4 --- /dev/null +++ b/corpus/errors/document-directive-misplaced.md @@ -0,0 +1,3 @@ +!adf:doc {content=none} + +Text. diff --git a/corpus/errors/empty-attrs-with-attributes.error b/corpus/errors/empty-attrs-with-attributes.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/empty-attrs-with-attributes.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/empty-attrs-with-attributes.md b/corpus/errors/empty-attrs-with-attributes.md new file mode 100644 index 0000000..560ecc7 --- /dev/null +++ b/corpus/errors/empty-attrs-with-attributes.md @@ -0,0 +1,3 @@ +!adf:panel info {attrs=empty} +Text. +!adf:/panel diff --git a/corpus/errors/empty-content-with-body.error b/corpus/errors/empty-content-with-body.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/empty-content-with-body.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/empty-content-with-body.md b/corpus/errors/empty-content-with-body.md new file mode 100644 index 0000000..648b150 --- /dev/null +++ b/corpus/errors/empty-content-with-body.md @@ -0,0 +1,3 @@ +!adf:paragraph {content=empty} +Text. +!adf:/paragraph diff --git a/corpus/errors/empty-key-value.error b/corpus/errors/empty-key-value.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/empty-key-value.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/empty-key-value.md b/corpus/errors/empty-key-value.md new file mode 100644 index 0000000..eadca54 --- /dev/null +++ b/corpus/errors/empty-key-value.md @@ -0,0 +1,3 @@ +!adf:paragraph {attrs=none} +Text. +!adf:/paragraph diff --git a/corpus/errors/empty-marks-wrapped.error b/corpus/errors/empty-marks-wrapped.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/empty-marks-wrapped.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/empty-marks-wrapped.md b/corpus/errors/empty-marks-wrapped.md new file mode 100644 index 0000000..8824f00 --- /dev/null +++ b/corpus/errors/empty-marks-wrapped.md @@ -0,0 +1 @@ +**!adf:date{marks=empty}** diff --git a/corpus/errors/text-break-attributes.error b/corpus/errors/text-break-attributes.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/text-break-attributes.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/text-break-attributes.md b/corpus/errors/text-break-attributes.md new file mode 100644 index 0000000..d4f7006 --- /dev/null +++ b/corpus/errors/text-break-attributes.md @@ -0,0 +1 @@ +a!adf:textBreak{x=y}b diff --git a/corpus/errors/text-break-content.error b/corpus/errors/text-break-content.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/text-break-content.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/text-break-content.md b/corpus/errors/text-break-content.md new file mode 100644 index 0000000..c5b44ef --- /dev/null +++ b/corpus/errors/text-break-content.md @@ -0,0 +1 @@ +a!adf:textBreak[x]b diff --git a/corpus/errors/text-break-marks-differ.error b/corpus/errors/text-break-marks-differ.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/text-break-marks-differ.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/text-break-marks-differ.md b/corpus/errors/text-break-marks-differ.md new file mode 100644 index 0000000..4a8f869 --- /dev/null +++ b/corpus/errors/text-break-marks-differ.md @@ -0,0 +1 @@ +**a**!adf:textBreak{}b diff --git a/corpus/errors/text-break-misplaced.error b/corpus/errors/text-break-misplaced.error new file mode 100644 index 0000000..a811539 --- /dev/null +++ b/corpus/errors/text-break-misplaced.error @@ -0,0 +1 @@ +unsupported-node-shape diff --git a/corpus/errors/text-break-misplaced.md b/corpus/errors/text-break-misplaced.md new file mode 100644 index 0000000..e84475f --- /dev/null +++ b/corpus/errors/text-break-misplaced.md @@ -0,0 +1 @@ +!adf:textBreak{}Hello diff --git a/corpus/round-trip/block-nodes/code-block-carry.json b/corpus/round-trip/block-nodes/code-block-carry.json index a5bbd7d..9981d4e 100644 --- a/corpus/round-trip/block-nodes/code-block-carry.json +++ b/corpus/round-trip/block-nodes/code-block-carry.json @@ -14,7 +14,19 @@ }, { "attrs": { - "language": "carry" + "language": "adf:blockCard" + }, + "content": [ + { + "text": "{\n \"attrs\": {}\n}", + "type": "text" + } + ], + "type": "codeBlock" + }, + { + "attrs": { + "language": "adf:" }, "content": [ { diff --git a/corpus/round-trip/block-nodes/code-block-carry.md b/corpus/round-trip/block-nodes/code-block-carry.md index 5a032f7..665bd3c 100644 --- a/corpus/round-trip/block-nodes/code-block-carry.md +++ b/corpus/round-trip/block-nodes/code-block-carry.md @@ -1,12 +1,18 @@ -!adf:codeBlock {language=carry} -``` +```carry { "type": "blockCard" } ``` + +!adf:codeBlock {language="adf:blockCard"} +``` +{ + "attrs": {} +} +``` !adf:/codeBlock -!adf:codeBlock {language=carry} +!adf:codeBlock {language="adf:"} ```` ``` ```` diff --git a/corpus/round-trip/block-nodes/code-block-text-nodes.json b/corpus/round-trip/block-nodes/code-block-text-nodes.json new file mode 100644 index 0000000..37554b6 --- /dev/null +++ b/corpus/round-trip/block-nodes/code-block-text-nodes.json @@ -0,0 +1,56 @@ +{ + "content": [ + { + "attrs": { + "language": "js" + }, + "content": [ + { + "text": "const a = 1\n", + "type": "text" + }, + { + "text": "const b = 2", + "type": "text" + } + ], + "type": "codeBlock" + }, + { + "content": [ + { + "text": "a", + "type": "text" + }, + { + "text": "```", + "type": "text" + }, + { + "text": "\n", + "type": "text" + } + ], + "type": "codeBlock" + }, + { + "attrs": { + "language": "has`tick", + "wrap": true + }, + "content": [ + { + "text": "x", + "type": "text" + }, + { + "text": " y ", + "type": "text" + } + ], + "type": "codeBlock" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/block-nodes/code-block-text-nodes.md b/corpus/round-trip/block-nodes/code-block-text-nodes.md new file mode 100644 index 0000000..cb87dbf --- /dev/null +++ b/corpus/round-trip/block-nodes/code-block-text-nodes.md @@ -0,0 +1,31 @@ +!adf:codeBlock +```js +const a = 1 + +``` +```js +const b = 2 +``` +!adf:/codeBlock + +!adf:codeBlock +``` +a +``` +```` +``` +```` +``` + + +``` +!adf:/codeBlock + +!adf:codeBlock {language="has\u0060tick" wrap=true} +``` +x +``` +``` + y +``` +!adf:/codeBlock diff --git a/corpus/round-trip/block-nodes/document-without-content.json b/corpus/round-trip/block-nodes/document-without-content.json new file mode 100644 index 0000000..6591ef5 --- /dev/null +++ b/corpus/round-trip/block-nodes/document-without-content.json @@ -0,0 +1,4 @@ +{ + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/block-nodes/document-without-content.md b/corpus/round-trip/block-nodes/document-without-content.md new file mode 100644 index 0000000..e23047f --- /dev/null +++ b/corpus/round-trip/block-nodes/document-without-content.md @@ -0,0 +1 @@ +!adf:doc {content=none} diff --git a/corpus/round-trip/block-nodes/empty-keys.json b/corpus/round-trip/block-nodes/empty-keys.json new file mode 100644 index 0000000..725bfee --- /dev/null +++ b/corpus/round-trip/block-nodes/empty-keys.json @@ -0,0 +1,96 @@ +{ + "content": [ + { + "content": [], + "type": "paragraph" + }, + { + "attrs": {}, + "content": [ + { + "text": "Plain words.", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [], + "type": "rule" + }, + { + "attrs": { + "level": 2 + }, + "content": [ + { + "text": "Title", + "type": "text" + } + ], + "marks": [], + "type": "heading" + }, + { + "attrs": { + "panelType": "info" + }, + "content": [], + "type": "panel" + }, + { + "content": [ + { + "attrs": {}, + "content": [ + { + "content": [ + { + "text": "Item", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + }, + { + "content": [ + { + "content": [ + { + "text": "Next", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "bulletList" + }, + { + "content": [], + "type": "codeBlock" + }, + { + "attrs": { + "language": "js" + }, + "marks": [], + "type": "codeBlock" + }, + { + "attrs": { + "language": "js" + }, + "content": [], + "type": "codeBlock" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/block-nodes/empty-keys.md b/corpus/round-trip/block-nodes/empty-keys.md new file mode 100644 index 0000000..cad1dfd --- /dev/null +++ b/corpus/round-trip/block-nodes/empty-keys.md @@ -0,0 +1,35 @@ +!adf:paragraph {content=empty} +!adf:/paragraph + +!adf:paragraph {attrs=empty} +Plain words. +!adf:/paragraph + +!adf:rule {content=empty} + +!adf:heading {level=2 marks=empty} +Title +!adf:/heading + +!adf:panel info {content=empty} +!adf:/panel + +!adf:bulletList +!adf:listItem {attrs=empty} +Item +!adf:/listItem +!adf:listItem +Next +!adf:/listItem +!adf:/bulletList + +!adf:codeBlock {content=empty} +!adf:/codeBlock + +!adf:codeBlock {marks=empty} +```js +``` +!adf:/codeBlock + +!adf:codeBlock {content=empty language=js} +!adf:/codeBlock diff --git a/corpus/round-trip/combinations/carry-attributes.md b/corpus/round-trip/combinations/carry-attributes.md index 088dbe9..605ffbd 100644 --- a/corpus/round-trip/combinations/carry-attributes.md +++ b/corpus/round-trip/combinations/carry-attributes.md @@ -1,4 +1,4 @@ -```carry +```adf:panel { "attrs": { "rounded": true @@ -13,12 +13,11 @@ ], "type": "paragraph" } - ], - "type": "panel" + ] } ``` -```carry +```adf:panel { "attrs": { "panelType": "extra info" @@ -33,8 +32,7 @@ ], "type": "paragraph" } - ], - "type": "panel" + ] } ``` diff --git a/corpus/round-trip/combinations/negative-zero.json b/corpus/round-trip/combinations/negative-zero.json new file mode 100644 index 0000000..2ffed45 --- /dev/null +++ b/corpus/round-trip/combinations/negative-zero.json @@ -0,0 +1,83 @@ +{ + "content": [ + { + "attrs": { + "order": -0 + }, + "content": [ + { + "content": [ + { + "content": [ + { + "text": "First", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "orderedList" + }, + { + "attrs": { + "level": -0 + }, + "content": [ + { + "text": "Title", + "type": "text" + } + ], + "type": "heading" + }, + { + "attrs": { + "extensionKey": "toc", + "extensionType": "com.atlassian.confluence.macro.core", + "parameters": { + "maxLevel": -0 + } + }, + "type": "extension" + }, + { + "content": [ + { + "attrs": { + "data": [ + -0 + ], + "url": "https://example.com/" + }, + "type": "inlineCard" + }, + { + "marks": [ + { + "attrs": { + "color": "#000000", + "size": -0 + }, + "type": "border" + } + ], + "text": "x", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "attrs": { + "x": -0 + }, + "type": "blockCard" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/combinations/negative-zero.md b/corpus/round-trip/combinations/negative-zero.md new file mode 100644 index 0000000..766f709 --- /dev/null +++ b/corpus/round-trip/combinations/negative-zero.md @@ -0,0 +1,21 @@ +!adf:orderedList {order=-0} +!adf:listItem +First +!adf:/listItem +!adf:/orderedList + +!adf:heading {level=-0} +Title +!adf:/heading + +!adf:extension {extensionKey=toc extensionType="com.atlassian.confluence.macro.core" parameters="{\"maxLevel\":-0}"} + +!adf:inlineCard{data="[-0]" url="https://example.com/"}!adf:border[x]{color="#000000" size=-0} + +```adf:blockCard +{ + "attrs": { + "x": -0 + } +} +``` diff --git a/corpus/round-trip/commonmark-subset/document-empty.json b/corpus/round-trip/commonmark-subset/document-empty.json index 6591ef5..317b19d 100644 --- a/corpus/round-trip/commonmark-subset/document-empty.json +++ b/corpus/round-trip/commonmark-subset/document-empty.json @@ -1,4 +1,5 @@ { + "content": [], "type": "doc", "version": 1 } diff --git a/corpus/round-trip/inline-nodes/empty-keys.json b/corpus/round-trip/inline-nodes/empty-keys.json new file mode 100644 index 0000000..a5b1623 --- /dev/null +++ b/corpus/round-trip/inline-nodes/empty-keys.json @@ -0,0 +1,98 @@ +{ + "content": [ + { + "content": [ + { + "attrs": {}, + "type": "date" + }, + { + "text": " ", + "type": "text" + }, + { + "content": [], + "type": "date" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "text": "a", + "type": "text" + }, + { + "marks": [], + "type": "hardBreak" + }, + { + "text": "b", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "attrs": { + "id": "x", + "text": "@M" + }, + "content": [], + "type": "mention" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "text": "bare", + "type": "text" + }, + { + "marks": [], + "text": "marked", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "attrs": {}, + "type": "em" + } + ], + "text": "x", + "type": "text" + }, + { + "text": " and ", + "type": "text" + }, + { + "attrs": { + "shortName": ":a:" + }, + "marks": [ + { + "attrs": {}, + "type": "strong" + } + ], + "type": "emoji" + } + ], + "type": "paragraph" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/inline-nodes/empty-keys.md b/corpus/round-trip/inline-nodes/empty-keys.md new file mode 100644 index 0000000..db86fe8 --- /dev/null +++ b/corpus/round-trip/inline-nodes/empty-keys.md @@ -0,0 +1,9 @@ +!adf:date{attrs=empty} !adf:date{content=empty} + +a!adf:hardBreak{marks=empty}b + +!adf:mention[@M]{content=empty id=x} + +bare!adf:carry{json="{\"marks\":[],\"text\":\"marked\",\"type\":\"text\"}"} + +!adf:carry{json="{\"marks\":[{\"attrs\":{},\"type\":\"em\"}],\"text\":\"x\",\"type\":\"text\"}"} and !adf:carry{json="{\"attrs\":{\"shortName\":\":a:\"},\"marks\":[{\"attrs\":{},\"type\":\"strong\"}],\"type\":\"emoji\"}"} diff --git a/corpus/round-trip/inline-nodes/text-break.json b/corpus/round-trip/inline-nodes/text-break.json new file mode 100644 index 0000000..f7e56f4 --- /dev/null +++ b/corpus/round-trip/inline-nodes/text-break.json @@ -0,0 +1,154 @@ +{ + "content": [ + { + "content": [ + { + "text": "Hello, ", + "type": "text" + }, + { + "text": "world", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "type": "strong" + } + ], + "text": "Hello, ", + "type": "text" + }, + { + "marks": [ + { + "type": "strong" + } + ], + "text": "world", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "type": "code" + } + ], + "text": "a", + "type": "text" + }, + { + "marks": [ + { + "type": "code" + } + ], + "text": "b", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "attrs": { + "href": "https://example.com/" + }, + "type": "link" + } + ], + "text": "a", + "type": "text" + }, + { + "marks": [ + { + "attrs": { + "href": "https://example.com/" + }, + "type": "link" + } + ], + "text": "b", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "text": "Line\n", + "type": "text" + }, + { + "text": "next", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "type": "underline" + } + ], + "text": "a", + "type": "text" + }, + { + "marks": [ + { + "type": "underline" + } + ], + "text": "b", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "marks": [ + { + "attrs": {}, + "type": "underline" + } + ], + "text": "a", + "type": "text" + }, + { + "marks": [ + { + "type": "underline" + } + ], + "text": "b", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/inline-nodes/text-break.md b/corpus/round-trip/inline-nodes/text-break.md new file mode 100644 index 0000000..71144ea --- /dev/null +++ b/corpus/round-trip/inline-nodes/text-break.md @@ -0,0 +1,13 @@ +Hello, !adf:textBreak{}world + +**Hello, !adf:textBreak{}world** + +`a`!adf:textBreak{}`b` + +[a!adf:textBreak{}b](https://example.com/) + +Line!adf:text{text="\n"}!adf:textBreak{}next + +!adf:underline[a!adf:textBreak{}b] + +!adf:underline[a]{attrs=empty}!adf:underline[b] diff --git a/corpus/round-trip/opaque-carry/carry-in-wrappers.md b/corpus/round-trip/opaque-carry/carry-in-wrappers.md index df32328..fb4de61 100644 --- a/corpus/round-trip/opaque-carry/carry-in-wrappers.md +++ b/corpus/round-trip/opaque-carry/carry-in-wrappers.md @@ -1,17 +1,15 @@ -> ```carry +> ```adf:blockCard > { > "attrs": { > "url": "https://example.com/quoted" -> }, -> "type": "blockCard" +> } > } > ``` -- ```carry +- ```adf:blockCard { "attrs": { "url": "https://example.com/listed" - }, - "type": "blockCard" + } } ``` diff --git a/corpus/round-trip/opaque-carry/code-block-children.json b/corpus/round-trip/opaque-carry/code-block-children.json new file mode 100644 index 0000000..b849d02 --- /dev/null +++ b/corpus/round-trip/opaque-carry/code-block-children.json @@ -0,0 +1,35 @@ +{ + "content": [ + { + "attrs": { + "language": "js" + }, + "content": [ + { + "marks": [ + { + "type": "strong" + } + ], + "text": "const a = 1", + "type": "text" + } + ], + "type": "codeBlock" + }, + { + "content": [ + { + "text": "a", + "type": "text" + }, + { + "type": "hardBreak" + } + ], + "type": "codeBlock" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/opaque-carry/code-block-children.md b/corpus/round-trip/opaque-carry/code-block-children.md new file mode 100644 index 0000000..4749c97 --- /dev/null +++ b/corpus/round-trip/opaque-carry/code-block-children.md @@ -0,0 +1,32 @@ +```adf:codeBlock +{ + "attrs": { + "language": "js" + }, + "content": [ + { + "marks": [ + { + "type": "strong" + } + ], + "text": "const a = 1", + "type": "text" + } + ] +} +``` + +```adf:codeBlock +{ + "content": [ + { + "text": "a", + "type": "text" + }, + { + "type": "hardBreak" + } + ] +} +``` diff --git a/corpus/round-trip/opaque-carry/unknown-block.md b/corpus/round-trip/opaque-carry/unknown-block.md index a88fe07..5469fb5 100644 --- a/corpus/round-trip/opaque-carry/unknown-block.md +++ b/corpus/round-trip/opaque-carry/unknown-block.md @@ -1,23 +1,21 @@ -```carry +```adf:blockCard { "attrs": { "url": "https://example.com/roadmap" - }, - "type": "blockCard" + } } ``` !adf:panel info The card below has no spelling yet. -```carry +```adf:embedCard { "attrs": { "layout": "wide", "url": "https://example.com/board", "width": 100 - }, - "type": "embedCard" + } } ``` !adf:/panel diff --git a/corpus/round-trip/opaque-carry/unknown-type-spelling.json b/corpus/round-trip/opaque-carry/unknown-type-spelling.json new file mode 100644 index 0000000..45a747a --- /dev/null +++ b/corpus/round-trip/opaque-carry/unknown-type-spelling.json @@ -0,0 +1,21 @@ +{ + "content": [ + { + "attrs": { + "url": "https://example.com/" + }, + "type": "has`tick" + }, + { + "type": "" + }, + { + "type": " padded" + }, + { + "type": "two words" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/opaque-carry/unknown-type-spelling.md b/corpus/round-trip/opaque-carry/unknown-type-spelling.md new file mode 100644 index 0000000..1988b18 --- /dev/null +++ b/corpus/round-trip/opaque-carry/unknown-type-spelling.md @@ -0,0 +1,24 @@ +```adf: +{ + "attrs": { + "url": "https://example.com/" + }, + "type": "has`tick" +} +``` + +```adf: +{ + "type": "" +} +``` + +```adf: +{ + "type": " padded" +} +``` + +```adf:two words +{} +``` diff --git a/docs/decisions.md b/docs/decisions.md index b7f65ed..2c11f61 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -14,7 +14,7 @@ a backslash reach them intact; what the flavour cannot spell reduces ADF→ADF a 2026-08-23, real payloads 2026-09-15, the maintainer. Goal 1. Valid while a consumer saves back through the lossless pair. -`markdownToAdf(adfToMarkdown(doc))` and `htmlToAdf(adfToHtml(doc))` must equal `doc` — anything +`markdownToAdf(adfToMarkdown(doc))` and `htmlToAdf(adfToHtml(doc))` must deep-equal `doc` — anything less silently destroys content an editor could not represent, in a document it did not author. When losslessness and readability conflict, losslessness wins. Round-trip equality is a property tested over a checked-in corpus (`corpus/README.md`), not a claim made in prose. Its real payloads @@ -31,27 +31,92 @@ where there is a way back. CommonMark spells some things the flavour has no esca paragraph opening with a code span whose backticks read back as a fence — so a parse succeeding does not imply a spellable document; `corpus/commonmark-spec/exceptions.json` names those. -## Equality is editor-normal +## Equality is deep -2026-08-24, the maintainer. Goal 1. Valid while markdown cannot tell apart the ADF shapes this -merges. +2026-08-24, deep 2026-10-03, the maintainer. Goal 1. Valid while a pipeline or a bot can build a +shape the editor would not. -"Equals" is structural equality over editor-normal ADF — adjacent text nodes with identical marks -and no attributes merged, JSON number semantics, an empty attrs object, marks array or content -array the absent key — the only domain markdown can restore. -`todo.md` item 40 replaces this with deep equality (2026-09-28, the maintainer). +"Equals" is deep equality over the document's JSON values: `assert.deepStrictEqual` on plain objects +as `JSON.parse` builds them, since ADF is JSON. Every key and value in `doc` counts, including two +adjacent text nodes, an empty `attrs`, `content` or `marks`, and `-0`. Neither side is normalized. +CommonMark's spelling stays wherever a document holds none of those shapes. The plain reader builds +what is written, as `markdownToAdf` does. Only the plain writer is lossy: its reduction reads and +writes editor-normal ADF — adjacent text nodes of identical marks and no attributes merged, `-0` as +`0`, and an empty `attrs`, `content` or `marks` the absent key, except the doc's `content`, which +ADF's schema requires — so two documents the editor holds equal write the same plain markdown. + +## `!adf:textBreak{}` parts text CommonMark would join + +2026-10-03, the maintainer. Goal 1. Valid while CommonMark reads adjacent text as one run. + +Two adjacent text nodes CommonMark would read back as one are parted by the reserved inline leaf +`!adf:textBreak{}`, mirroring `!adf:listBreak`: a leaf building no node keeps both nodes and asks +nothing of the text around it. A code span holds no directive, so the spans close and reopen +around the leaf. The grammar: `spec/flavour.md` §Inline nodes, **Adjacent text nodes**. + +## An empty key spells `empty` + +2026-10-03, a writer panel and the maintainer. Goals 1 and 5. Valid while no attribute value +spells an empty object or array. + +An `attrs`, `content` or `marks` key holding an empty object or array is the reserved key with the +bare value `empty`, so a container opener and closer with nothing between them stays the node +holding no `content` key. A writer panel chose the spelling, 5 of 7. The grammar: `spec/flavour.md` +§Directives, **Attributes**, and §Marks. + +## `-0` is spelled `-0` + +2026-10-03, the maintainer. Goal 1. Valid while JSON's own serialization writes `-0` as `0`. + +`-0` is spelled `-0` wherever the flavour writes a number or a JSON value, since JSON's grammar +reads it back as `-0`. The grammar: `spec/flavour.md` §Block nodes and §The CommonMark blocks. + +## Empty markdown is a document of no blocks + +2026-10-03, a writer panel and the maintainer. Goals 1 and 5. Valid while ADF's schema requires +`content` on `doc`. + +Markdown holding no block reads as `{ content: [], type: 'doc', version: 1 }`, the document +`spec/adf-schema/full.json` requires. A document holding no `content` key is +`!adf:doc {content=none}` as its only block, and a named error anywhere else. The writer panel split +4 for `none` and 3 for `absent`, and the maintainer chose `none`; all seven rejected a bare +`!adf:doc`. ## Unknown nodes ride the carry -2026-08-23, extended to misplaced known nodes 2026-08-26, the maintainer. Goal 1. Valid while ADF -holds nodes, or node positions, this library does not spell. +2026-08-23, extended to misplaced known nodes 2026-08-26 and to code block children 2026-10-03, the +maintainer. Goal 1. Valid while ADF holds nodes, or node positions, this library does not spell. An unknown ADF node is carried opaquely — raw JSON rides a dedicated syntax in both formats and restores to a deep-equal node. The round-trip holds for documents newer than the library. So does a known node no section spells where it stands: a markdown serializer spells a node by type without -checking its position, and refusing loses a document ADF itself keeps in an `unsupportedBlock`. -Where a container's own spelling cannot hold the child it has — a `bulletList` holding other than -`listItem`, a `codeBlock` other than text — the error result names that instead. +checking its position, and refusing loses a document ADF itself keeps in an `unsupportedBlock`. A +`codeBlock` holding a child no fence holds — anything but a text node carrying no marks, `attrs` or +`content` — rides the carry whole. + +## The carry fence names the node type + +2026-10-03, the maintainer. Goals 1 and 5. Valid while a code fence's info string reads back +verbatim. + +The block carry is a code fence whose info string `adf:` names the node's type, its body the +node's JSON without `type`: ```` ```adf:blockCard ````. A type no info string carries back — by the +rule a code language follows — leaves the info string `adf:` and keeps `type` in the body. Every +info string opening `adf:` is reserved, so a `codeBlock` whose language opens so takes the +`language` attribute, and `carry` is an ordinary language. A body holding `type` under a named type, +or a fence whose info string is `adf:` alone while its body's `type` could be spelled in the info +string, is `unsupported-node-shape`. The reservation claims a fence CommonMark reads as code until +`todo.md` item 43 gives CommonMark its own reader. + +## A code block is a fence per text node + +2026-10-03, the maintainer. Goal 1. Valid while ADF holds a code block's text in more than one +node. + +A `codeBlock` holding several text nodes is the `!adf:codeBlock` container holding one fence per +node, so each node keeps its own text. The fences carry one info string, since ADF holds one +language, and none is empty beside another, since a text node holds text. The grammar: +`spec/flavour.md` §The CommonMark blocks, the `codeBlock` bullet. ## Foreign HTML sorts three ways @@ -97,7 +162,9 @@ opener nests by itself and leaf versus container falls out of the node's content 2026-08-23, the maintainer. Goal 4. Valid while prose rarely writes the shapes the carve-outs claim. Plain CommonMark is a subset, with carve-outs (`spec/flavour.md`): literal text shaped like a -directive, a pipe table or a `~~` pair is claimed — plus one image gap. +directive, a pipe table or a `~~` pair is claimed, and so is a code fence whose info string opens +`adf:` (§The carry fence names the node type) — plus one image gap. `todo.md` item 43 ends the +claims, giving CommonMark its own reader, and `todo.md` item 65 ends the image gap. ## Tables @@ -292,13 +359,14 @@ handles one cause alike whichever node, attribute or direction raised it. colon rather than ADF's missing column model doing it. What the grammar itself refuses stays a claim code, key order among it, and a leaf given a body is refused at its opener, as a container missing its closer is (2026-09-16). -- A directive whose name reads back to no node is `unknown-directive-name` rather than a claim - code — the spelling is well formed, and telling that apart from a typo is what a consumer - switches on when a later MINOR gives the name meaning. A reserved name is a known name, so never - that code, and the two the flavour reserves part on form: a form the grammar does not have is a - claim code — `!adf:carry`, whose carry is the fence — and a well-formed form in the wrong place - is `unsupported-node-shape`, `!adf:listBreak` parting anything but two adjacent lists of one - type (2026-09-01). +- A directive whose name reads back to no node is `unknown-directive-name` rather than a claim code + — the spelling is well formed, and telling that apart from a typo is what a consumer switches on + when a later MINOR gives the name meaning. A reserved name is a known name, so never that code, + and the names the flavour reserves part on form: a form the grammar does not have is a claim code + — `!adf:carry`, whose carry is the fence — and a well-formed form in the wrong place is + `unsupported-node-shape`, `!adf:listBreak` parting anything but two adjacent lists of one type, + `!adf:textBreak{}` anything but two adjacent text nodes CommonMark would read back as one, + `!adf:doc` standing beside another block (2026-09-01, the text break and `doc` 2026-10-03). - A well-formed directive the node tables refuse — an attribute a node does not hold or spells elsewhere, a value outside its kind or its canonical spelling, an argument, or a body of a shape its content model does not take — is `unsupported-node-shape`, the emitter's code for the same @@ -422,6 +490,24 @@ config gone missing fails the leg instead of falling back to oxlint's own defaul `--deny-warnings`, since a rule from a category this config never names arrives as a warning it exits 0 on. +## The project ships under the comprehension floor until items 59, 60, 61 and 62 land + +2026-10-03, the maintainer. KISS, a technical principle, and its comprehension floor of 7. Valid +while `todo.md` items 59, 60, 61 and 62 are open. + +A four-seat comprehension panel scored the project under the floor of 7. Its round-5 scores +(2026-10-03) are the baseline: a later panel may not score lower on any dimension. + +| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall | +|---|---|---|---|---|---| +| Junior | 6 | 5 | 6 | 5 | 5.5 | +| Mid | 6.5 | 5 | 6 | 4.5 | 5.5 | +| Senior | 6.5 | 5.5 | 6.5 | 5.5 | 6 | +| Architect | 6.5 | 5.5 | 6 | 6 | 6 | + +Three panels on near-identical code scored overall means of 5.88, 5.75 and 5.75, and a seat moves +±0.5 between runs. + ## Properties on a fixed seed 2026-09-14, the maintainer. Goal 1. Valid while a red gate must reproduce. @@ -568,8 +654,8 @@ is the markdown flavour's choice, not ADF's. ## The source parts by ADF and format -2026-08-27, placement 2026-09-18, `adf/`'s bar 2026-09-21, the maintainer. Goal 2. Valid while -each format has a reader and a writer through ADF. +2026-08-27, placement 2026-09-18, `adf/`'s bar 2026-09-21, `plain/` 2026-10-03, the maintainer. +Goal 2. Valid while each format has a reader and a writer through ADF. `src/adf/` holds ADF's own knowledge, imports no format, and is where a construct both formats read lives: the question is answered in ADF's vocabulary — a node type, an attribute kind, a content @@ -582,13 +668,15 @@ them. A primitive knowing neither ADF nor a format stays at `src/` root. A const `spec/flavour.md` draws, and a placement nothing here settles goes beside its only reader, or in what both read where there are two. -Each format directory parts into `emit/` (ADF→format) and `parse/` (format→ADF), the rest of it -holding what both directions read. A construct's reader lives there beside the regex the emitter -escapes against, so the two cannot drift; a reader with no emit counterpart goes in `parse/`, -unless it is part of a construct that side already holds — a grammar stays in one file rather than -splitting across the seam. A rule both directions must answer alike — whether a list marker -interrupts a paragraph — is one function there too, never a copy per direction, however -conservative the copy would be. Where the rule is the emitter's own choice, input consults it -rather than restating it, and that is the only import `parse/` takes from `emit/` — -`commonMarkSpelling` and `openingLinkTakesDirective` — so no fixture the emitter writes can be -refused, and a spelling the emitter refuses gives its own error rather than a second name for it. +A flavour's own code, both directions included, sits in its own directory: `markdown/plain/`; +`todo.md` item 62 moves what still sits in `parse/` and `emit/`. Otherwise each format directory +parts into `emit/` (ADF→format) and `parse/` (format→ADF), the rest of it holding what both +directions read. A construct's reader lives there beside the regex the emitter escapes against, so +the two cannot drift; a reader with no emit counterpart goes in `parse/`, unless it is part of a +construct that side already holds — a grammar stays in one file rather than splitting across the +seam. A rule both directions must answer alike — whether a list marker interrupts a paragraph — is +one function there too, never a copy per direction, however conservative the copy would be. Where +the rule is the emitter's own choice, input consults it rather than restating it, and that is the +only value import `parse/` takes from `emit/` — `commonMarkSpelling` and `openingLinkTakesDirective` +— so no fixture the emitter writes can be refused, and a spelling the emitter refuses gives its own +error rather than a second name for it. diff --git a/spec/flavour.md b/spec/flavour.md index fcb97ba..c29681d 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -1,12 +1,14 @@ # The markdown flavour The grammar of the extended markdown `adfToMarkdown` emits and `markdownToAdf` parses. Plain -CommonMark is a subset apart from raw HTML (below), with three carve-outs: literal text that matches -directive syntax below or reads as a pipe table is claimed by the flavour, and a matched `~~` pair -spells `strike` (escape the `!adf:`, `|` or `~` to keep it literal) — and one gap: a CommonMark -image fits only as its own title-less paragraph — mid-text and titled images are named errors. The -emitted form is contract (`docs/decisions.md` §The formats are API). Per-node syntaxes build on this -grammar in the sections below. +CommonMark is a subset apart from raw HTML (below), with four carve-outs: literal text that matches +directive syntax below or reads as a pipe table is claimed by the flavour, a matched `~~` pair +spells `strike` (escape the `!adf:`, `|` or `~` to keep it literal), and a code fence whose info +string opens `adf:` is the opaque carry (drop the info string and wrap the fence in +`!adf:codeBlock {language="adf:…"}` to keep it code) — and one gap: a CommonMark image fits only as +its own title-less paragraph — mid-text and titled images are named errors. The emitted form is +contract (`docs/decisions.md` §The formats are API). Per-node syntaxes build on this grammar in the +sections below. ## Canonical form @@ -44,7 +46,8 @@ normalizes to it through the round-trip. CommonMark admits no spelling — the end of a block, inside an ATX heading — or where the node carries an attribute, it is the inline directive. - An empty paragraph — real payloads carry them — is an `!adf:paragraph` … `!adf:/paragraph` pair - holding nothing. + holding nothing, and one whose `content` is an empty array the pair + `!adf:paragraph {content=empty}` … `!adf:/paragraph` (Attributes). - Links `[text](url)`; `<…>` around a destination containing spaces, `<>` an empty one beside a title; title in double quotes. A backslash escapes a parenthesis the destination leaves unbalanced, and a quote inside the title; a balanced pair stays bare. `` autolink form only @@ -67,7 +70,10 @@ normalizes to it through the round-trip. matching below, which is what lets the emitter decide its own pairings. - Blocks separated by one blank line at document level, inside a blockquote and between CommonMark blocks; inside a directive container a pair holding a directive block takes none. No trailing - whitespace outside a code block's content, single trailing newline; a document with no blocks is the empty string. + whitespace outside a code block's content, single trailing newline. A document whose `content` is + an empty array is the empty string, and markdown holding no block reads back to it; a document + holding no `content` key is the leaf `!adf:doc {content=none}` as its only block, which is a + named error anywhere else or spelled any other way. ## Directives @@ -79,9 +85,11 @@ result naming it at the opener, whatever follows it — so output an old emitter escaped, and erroring input gaining meaning later is MINOR, never a reparse (`docs/decisions.md` §The formats are API). Each name belongs to one position, and a name the other one spells — a mark or an inline node written as a block directive, a block node written inline — is a different error, -naming the spelling it takes. Two reserved names read back to no node: `carry` for the opaque carry, -as both directive name and fence info string, and `listBreak` for the leaf that parts two adjacent -lists (Canonical form). +naming the spelling it takes. Four reserved names read back to no node: `carry` for the inline +opaque carry, `listBreak` for the leaf that parts two adjacent lists and `doc` for a document +holding no `content` key (Canonical form), and `textBreak` for the leaf that parts two text nodes +(Inline nodes). Every fence info string opening `adf:` is reserved for the block carry (The opaque +carry). **Claiming**: an unescaped `!adf:` claims wherever it stands. What follows picks the form: `/name` closes a container, and a name picks by what follows it in turn — a space or the line's end a block @@ -142,6 +150,13 @@ ends the name (`!adf:hardBreak{}`). Input reads that spelling alone: keys out of quoted where bare carries it, an escape longer than it need be, an empty `{attrs}` on a block line or after a `[content]`, and a number or `json` value outside its canonical JSON spelling are each a named error naming the spelling to write instead. +`attrs`, `content` and `marks` are reserved keys on every directive — block, inline node and mark — +whose bare value `empty` spells the node's or mark's key holding an empty object or array: +`!adf:underline[a]{attrs=empty}`, `!adf:date{content=empty}`, `!adf:hardBreak{marks=empty}`. A +container spelling `content=empty` closes with no body; `attrs=empty` stands beside no other +attribute, argument or content slot; and an inline node spelling `marks=empty` stands inside no +mark spelling. Any other value of a reserved key is a named error, except on a block's `marks` +(Block nodes). **Escaping**: the emitter backslash-escapes whatever literal text would otherwise parse as directive syntax — every literal `!adf:`, `]` inside content, a bracket a link's destination and @@ -163,15 +178,19 @@ and restores to a deep-equal node. A carry may hold a node the emitter spells na unreinterpreted, and the next emit spells it canonically (`docs/decisions.md` §The round-trip is the product). Block and inline positions canonicalize differently, each fitting where it sits: -- **Block position**: a fenced code block with info string `carry`, body = the node's JSON — - two-space indent, object keys sorted. +- **Block position**: a fenced code block with info string `adf:` and the node's type, body = the + node's JSON without its `type` — two-space indent, object keys sorted: ```` ```adf:blockCard ````. + A type no info string carries back, by the rule a `codeBlock`'s language follows, leaves the info + string `adf:` and keeps `type` in the body. A body holding `type` under a named type, or a fence + whose info string is `adf:` alone while its body's `type` could be spelled in the info string, is + a named error. - **Inline position**: `!adf:carry{json="…"}` — compact serialization (keys sorted, no whitespace), JSON-string-escaped into the attribute. -The info string `carry` is reserved: a genuine `codeBlock` whose `language` is exactly `carry` takes +Every info string opening `adf:` is reserved: a genuine `codeBlock` whose `language` opens so takes the attribute the section below keeps for a language no info string holds, so the reservation -stays absolute. -In block-directive position `!adf:carry` is a named error — the carry's block form is the fence. +stays absolute. In block-directive position `!adf:carry` is a named error — the carry's block form +is the fence. ## Raw HTML in input @@ -193,13 +212,14 @@ Each section lists attributes as `name (type)`. A parenthesized value set docume payloads hold; the type stays string and any value round-trips verbatim. Values map to attrs by type: strings verbatim, numbers and booleans in canonical JSON spelling — quoted where not bare (`width="33.33"`) — and `json` values as the inline carry's serialization (compact, keys sorted), -quoted. `markdownToAdf` emits `attrs`, `content` and `marks` keys only when non-empty; editor-normal -ADF reads an empty attrs object, marks array or content array as the absent key (`docs/decisions.md` -§Equality is editor-normal) — the grammar's empty-`{attrs}` omission already collapses the two -spellings. +quoted, `-0` spelled `-0`. `markdownToAdf` builds an `attrs`, `content` or `marks` key only where the +markdown spells one, an empty one through its reserved key (Attributes), so a document reads back +deep-equal (`docs/decisions.md` §Equality is deep). A node CommonMark spells takes the directive +form to hold an empty key. Marks on a block node ride the reserved attribute key `marks` — the node's marks array as a -`json` value: `!adf:layoutSection {marks="[{\"attrs\":{\"mode\":\"wide\"},\"type\":\"breakout\"}]"}`. +`json` value, `marks=empty` where it is empty: +`!adf:layoutSection {marks="[{\"attrs\":{\"mode\":\"wide\"},\"type\":\"breakout\"}]"}`. A section saying its body is inline takes at most one paragraph, whose inline content becomes the node's `content`; any other body is a named error, and an empty pair is a node holding none. @@ -215,20 +235,23 @@ cannot — `localId` (string) on any of them, marks, and the values below — ta form. - `blockquote`, `bulletList`, `listItem` — containers, block body. Attributes: `localId` (string). -- `codeBlock` — container, body one fenced code block whose info string is the language and whose - content is the node's. Attributes: `hideLineNumbers` (boolean), `language` (string), `localId` - (string), `uniqueId` (string), `wrap` (boolean). A language no info string carries back — empty, - the reserved `carry`, or holding a backtick, a backslash, a control character, edge whitespace or - an entity reference — rides the `language` attribute instead and the fence carries no info - string; writing it in the slot that rule leaves empty, or in both, is a named error. The body is - one ordinary code block, and a fence's info string decodes escapes and entity references as any - other does. +- `codeBlock` — container, body one fenced code block per text node, each fence's info string the + language and its content the node's text; a node holding no `content` key is one empty fence. + Attributes: `hideLineNumbers` (boolean), `language` (string), `localId` (string), `uniqueId` + (string), `wrap` (boolean). A language no info string carries back — empty, opening the reserved + `adf:`, or holding a backtick, a backslash, a control character, edge whitespace or an entity + reference — rides the `language` attribute instead and the fences carry no info string, and so + does a language beside `content=empty`, which has no fence; writing it in the slot that rule + leaves empty, or in both, is a named error, and so are fences whose info strings differ and an + empty fence beside another. Each fence is an ordinary code block, and its info string decodes + escapes and entity references as any other does. A node holding a child no fence holds — any but + a text node carrying no marks, `attrs` or `content` — rides the block carry. - `heading` — container, inline body. Attributes: `level` (number), `localId` (string). `level` is the `#` count, so a heading carrying none, or one that is no whole number from 1 to 6, has no CommonMark spelling. - `orderedList` — container of `listItem`, block body. Attributes: `localId` (string), `order` - (number). `order` is the first marker, so a list carrying none, one that is no whole number - from 0, or one whose markers would run past 999999999, has no CommonMark spelling. + (number). `order` is the first marker, so a list carrying none, one that is `-0` or no whole + number from 0, or one whose markers would run past 999999999, has no CommonMark spelling. - `paragraph` — container, inline body. Attributes: `localId` (string). - `rule` — leaf. Attributes: `color` (string, `#rrggbb`), `localId` (string), `style` (`dashed` `dotted` `fade` `sketch` `solid`), `weight` (number, 1–3). @@ -424,10 +447,9 @@ Right. Attributes and the carry fallback read as in the block sections, the carry in its inline form. Of the nodes below, `emoji`, `mention` and `status` spell their `text` attribute in the content slot as plain text: `[]` is the empty string, absent content is the absent attribute, non-empty content -parsing to anything but one text node carrying neither marks, attributes nor content — adjacent text -nodes with identical marks and no attributes merged first — is a named error, and so is a `text` key -in `{attrs}`. An enclosing mark spelling does not reach into the slot. The rest take no content, -`!adf:text` included; content on a node that takes none is a named error. +parsing to anything but one text node carrying neither marks, attributes nor content is a named +error, and so is a `text` key in `{attrs}`. An enclosing mark spelling does not reach into the slot. +The rest take no content, `!adf:text` included; content on a node that takes none is a named error. - `date` — Attributes: `localId` (string), `timestamp` (string, epoch milliseconds). - `emoji` — Attributes: `id` (string), `localId` (string), `shortName` (string, `:name:`), `text` @@ -453,16 +475,27 @@ Shipped !adf:emoji[🎉]{shortName=":tada:"} on !adf:date{timestamp=175608000000 CommonMark strips or refuses one — a block's inline content edges, either side of a line break, an em, strong or strike spelling's inner edges, a pipe cell's edges — is spelled `!adf:text{text="…"}`, the reserved key carrying the node's text, escaped by the attribute grammar and never literal: pipe -cells trim and pad. The emitter wraps the whitespace run alone and leaves the rest plain text; -`markdownToAdf` merges adjacent text nodes carrying identical marks and no attributes -(`docs/decisions.md` §Equality is editor-normal). Input reads that spelling alone: the value is one -run of spaces and tabs, or one run of newlines, and anything else — a mixed run, or text CommonMark -carries plainly — is a named error. +cells trim and pad. The emitter wraps the whitespace run alone and leaves the rest plain text, which +the spelled run joins on reading. Input reads that spelling alone: the value is one run of spaces +and tabs, or one run of newlines, and anything else — a mixed run, or text CommonMark carries +plainly — is a named error. ``` !adf:text{text=" "}Two leading spaces held, and one text node split!adf:text{text="\n"}over two lines. ``` +**Adjacent text nodes.** CommonMark reads two adjacent text nodes back as one where neither is +carried, neither holds `attrs` or an empty key, and their marks are identical, attributes included. +The reserved leaf `!adf:textBreak{}` parts such a pair, inside every mark spelling the two share; a +code span holds no directive, so it closes and reopens. It builds no node and reads only between +two such nodes: elsewhere, or with `[content]` or `{attrs}`, it is a named error (`docs/decisions.md` +§`!adf:textBreak{}` parts text CommonMark would join). A text node holding `attrs` or an empty key +rides the inline carry. + +``` +Hello, !adf:textBreak{}world — **Hello, !adf:textBreak{}world** — `a`!adf:textBreak{}`b` +``` + ## Marks An inline node's marks ride the spelling wrapped around them, never the block sections' reserved @@ -490,20 +523,20 @@ the directive form, open to no literal reading, is a named error. A spelling adds its mark to every inline node it wraps, and nesting is the marks array in order, outermost first: `_!adf:underline[x]_` gives marks `[em, underline]`, `!adf:underline[_x_]` the reverse. `adfToMarkdown` nests in the order the array holds rather than sorting it — -`docs/decisions.md` §Equality is editor-normal restores the array, not a set — and opens each -spelling once over the longest run of adjacent inline nodes carrying an identical mark, attributes -included, at that depth. A run breaks at every node the emitter carries, so no emitted carry sits -inside a mark spelling. +`docs/decisions.md` §Equality is deep restores the array, not a set — and opens each spelling once +over the longest run of adjacent inline nodes carrying an identical mark, attributes included, at +that depth: `attrs: {}` differs from no `attrs`, and a directive spells it `{attrs=empty}`. A run +breaks at every node the emitter carries, so no emitted carry sits inside a mark spelling. An inline node whose marks no nesting spells — a mark type not listed here, an attrs key its -spelling does not list, a value that is not the spelling's type, an attribute the spelling needs -and the mark lacks, an order putting a code span outside another mark, `code` over anything but a -text node or over text holding a newline, or a spelling CommonMark's flanking rules cannot open or -close where the run sits (`un**-real**istic`), or one CommonMark's matching pairs elsewhere — the -intra-word `*` runs together with a neighbouring `**`, and the multiple-of-3 rule can leave the -merged run's pairing to another delimiter — rides the inline carry whole. An opaque carry inside a -mark spelling is a named error in input: the carry restores its node exactly, marks included -(`docs/decisions.md` §Unknown nodes ride the carry). +spelling does not list, a value that is not the spelling's type, an attribute the spelling needs and +the mark lacks, an empty `attrs` on a mark CommonMark spells, an order putting a code span outside +another mark, `code` over anything but a text node or over text holding a newline, or a spelling +CommonMark's flanking rules cannot open or close where the run sits (`un**-real**istic`), or one +CommonMark's matching pairs elsewhere — the intra-word `*` runs together with a neighbouring `**`, +and the multiple-of-3 rule can leave the merged run's pairing to another delimiter — rides the +inline carry whole. An opaque carry inside a mark spelling is a named error in input: the carry +restores its node exactly, marks included (`docs/decisions.md` §Unknown nodes ride the carry). ``` !adf:textColor[**Overdue**]{color="#ae2e24"}, H!adf:subsup[2]{type=sub}O, !adf:underline[signed]. diff --git a/src/adf/document.ts b/src/adf/document.ts index 490c5ac..772dd34 100644 --- a/src/adf/document.ts +++ b/src/adf/document.ts @@ -1,6 +1,7 @@ import type { ConvertFault } from '../result.ts' import { isJsonValue, overNested, type JsonValue } from '../json-value.ts' import { largestNesting } from '../nesting.ts' +import { serializeCanonicalJson } from '../canonical-json.ts' export type AdfAttributes = { [key: string]: JsonValue } @@ -17,6 +18,8 @@ export type AdfNode = { type: string } +export type EmptyKey = 'attrs' | 'content' | 'marks' + export type AdfDocument = { content?: AdfNode[] type: 'doc' @@ -48,11 +51,49 @@ export function attributeNestingMessage(key: string, type: string): string { return `the ${key} attribute of ${type} nests deeper than the ${largestNesting} levels an attribute carries` } -export function carriesOnly(node: AdfNode, attributes: readonly string[]): boolean { - if (nodeMarks(node).length > 0 || node.text !== undefined) return false +export function holdsOnlyAttributes(node: AdfNode, attributes: readonly string[]): boolean { + if (nodeMarks(node).length > 0 || node.text !== undefined || emptyKeys(node).length > 0) return false return holdsOnly(nodeAttrs(node), attributes) } +export function emptyKeys(held: { attrs?: AdfAttributes; content?: AdfNode[]; marks?: AdfMark[] }): EmptyKey[] { + const keys: EmptyKey[] = [] + if (held.attrs !== undefined && Object.keys(held.attrs).length === 0) keys.push('attrs') + if (held.content?.length === 0) keys.push('content') + if (held.marks?.length === 0) keys.push('marks') + return keys +} + +// A format spells this node as text; anything more rides a carry, save content, which is refused. +export function isBareText(node: AdfNode): boolean { + return node.type === 'text' && node.attrs === undefined && node.content === undefined && node.marks?.length !== 0 +} + +export function isUnmarkedBareText(node: AdfNode): boolean { + return isBareText(node) && node.marks === undefined +} + +export function identicalMark(left: AdfMark, right: AdfMark): boolean { + return marksKey([left]) === marksKey([right]) +} + +export function identicalMarks(left: readonly AdfMark[], right: readonly AdfMark[]): boolean { + return marksKey(left) === marksKey(right) +} + +export function mergeAdjacentText(nodes: readonly AdfNode[], joins: (previous: AdfNode, node: AdfNode) => boolean): AdfNode[] { + const merged: AdfNode[] = [] + for (const node of nodes) { + const previous = merged[merged.length - 1] + if (previous !== undefined && joins(previous, node)) { + merged[merged.length - 1] = { ...previous, text: `${previous.text ?? ''}${node.text ?? ''}` } + continue + } + merged.push(node) + } + return merged +} + // Depth is the walks' business, not the shape's: the guard waves a deep document through as blocks and marks do. export function isAdfDocument(value: unknown): value is AdfDocument { const fault = adfDocumentFault(value) @@ -99,6 +140,13 @@ function isNodeArray(value: readonly unknown[]): value is readonly AdfNode[] { return true } +function marksKey(marks: readonly AdfMark[]): string { + return serializeCanonicalJson( + marks.map((mark) => (mark.attrs === undefined ? [mark.type] : [mark.type, mark.attrs])), + 'compact', + ) +} + function nestingFault(nodes: readonly AdfNode[]): ConvertFault | undefined { const pending: AdfNode[] = [...nodes] while (pending.length > 0) { diff --git a/src/canonical-json.ts b/src/canonical-json.ts index 487dd99..a1fd6ec 100644 --- a/src/canonical-json.ts +++ b/src/canonical-json.ts @@ -18,7 +18,7 @@ export function serializeCanonicalJson(value: JsonValue, spelling: JsonSpelling) const { depth, value: held } = next if (Array.isArray(held)) schedule(pending, '[', held.map((item) => ({ label: '', value: item })), ']', indent, depth) else if (held !== null && typeof held === 'object') schedule(pending, '{', objectMembers(held, indent), '}', indent, depth) - else text.push(JSON.stringify(held)) + else text.push(Object.is(held, -0) ? '-0' : JSON.stringify(held)) } return text.join('') } diff --git a/src/conformance/adf-property.test.ts b/src/conformance/adf-property.test.ts index 16de9d7..96a8c83 100644 --- a/src/conformance/adf-property.test.ts +++ b/src/conformance/adf-property.test.ts @@ -5,10 +5,10 @@ import test from 'node:test' import type { AdfNode } from '../adf/document.ts' import { adfDocument, propertyRuns, propertyTimeout } from './property-harness.ts' import { adfToMarkdown } from '../markdown/emit/adf-to-markdown.ts' -import { adfToPlainMarkdown, reduceToPlain } from '../markdown/emit/plain-reduction.ts' +import { adfToPlainMarkdown, reduceToPlain } from '../markdown/plain/adf-to-plain-markdown.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' import { markdownToAdf, plainMarkdownToAdf } from '../markdown/parse/markdown-to-adf.ts' -import { toEditorNormal } from '../adf/editor-normal.ts' +import { toEditorNormal } from '../markdown/plain/editor-normal.ts' const gateRuns = 1600 const renamedPrefix = '!adg:' @@ -41,7 +41,7 @@ test('a generated document refuses to emit, or its markdown reads back to it', { if (!emitted.ok) return const read = markdownToAdf(emitted.value) assert.ok(read.ok, read.ok ? '' : `${read.error.code}: ${read.error.message} — reading ${JSON.stringify(emitted.value)}`) - assert.deepEqual(toEditorNormal(read.value), document, `reading ${JSON.stringify(emitted.value)}`) + assert.deepEqual(read.value, document, `reading ${JSON.stringify(emitted.value)}`) }), propertyRuns(gateRuns), ) @@ -62,3 +62,12 @@ test('a generated document writes plain markdown refusing only what the guard re propertyRuns(gateRuns), ) }) + +test('a generated document writes the plain markdown its editor-normal form writes', { timeout: propertyTimeout }, () => { + fc.assert( + fc.property(adfDocument, (document) => { + assert.deepEqual(adfToPlainMarkdown(document), adfToPlainMarkdown(toEditorNormal(document))) + }), + propertyRuns(gateRuns), + ) +}) diff --git a/src/conformance/corpus.test.ts b/src/conformance/corpus.test.ts index 22ff052..86fe0f6 100644 --- a/src/conformance/corpus.test.ts +++ b/src/conformance/corpus.test.ts @@ -9,7 +9,6 @@ import { isAdfDocument } from '../adf/document.ts' import { isJsonValue } from '../json-value.ts' import { markdownToAdf } from '../markdown/parse/markdown-to-adf.ts' import { serializeCanonicalJson } from '../canonical-json.ts' -import { toEditorNormal } from '../adf/editor-normal.ts' const corpusRoot = join(dirname(fileURLToPath(import.meta.url)), '..', '..', 'corpus') const errorsRoot = join(corpusRoot, 'errors') @@ -97,7 +96,7 @@ for (const directory of roundTripDirectories) { assert.ok(isAdfDocument(expected), `${name}.json is not an ADF document`) const result = markdownToAdf(readFileSync(join(roundTripRoot, directory, `${name}.md`), 'utf8')) assert.ok(result.ok, result.ok ? '' : `${result.error.code}: ${result.error.message}`) - assert.deepEqual(toEditorNormal(result.value), expected) + assert.deepEqual(result.value, expected) }) } } @@ -125,12 +124,12 @@ for (const name of pairedNames(normalizationRoot, '.md', '.json')) { assert.ok(isAdfDocument(expected), `${name}.json is not an ADF document`) const result = markdownToAdf(readFileSync(join(normalizationRoot, `${name}.md`), 'utf8')) assert.ok(result.ok, result.ok ? '' : `${result.error.code}: ${result.error.message}`) - assert.deepEqual(toEditorNormal(result.value), expected) + assert.deepEqual(result.value, expected) const emitted = adfToMarkdown(result.value) assert.ok(emitted.ok, emitted.ok ? '' : `${emitted.error.code}: ${emitted.error.message}`) const again = markdownToAdf(emitted.value) assert.ok(again.ok, again.ok ? '' : `${again.error.code}: ${again.error.message}`) - assert.deepEqual(toEditorNormal(again.value), expected) + assert.deepEqual(again.value, expected) }) } @@ -146,7 +145,7 @@ for (const name of names(realPayloadsRoot, '.json')) { assert.ok(emitted.ok, emitted.ok ? '' : `${emitted.error.code}: ${emitted.error.message}`) const parsed = markdownToAdf(emitted.value) assert.ok(parsed.ok, parsed.ok ? '' : `${parsed.error.code}: ${parsed.error.message}`) - assert.deepEqual(toEditorNormal(parsed.value), payload) + assert.deepEqual(parsed.value, payload) }) } diff --git a/src/conformance/markdown-property.test.ts b/src/conformance/markdown-property.test.ts index 580dce9..64a435f 100644 --- a/src/conformance/markdown-property.test.ts +++ b/src/conformance/markdown-property.test.ts @@ -31,7 +31,6 @@ import { markdownToAdf } from '../markdown/parse/markdown-to-adf.ts' import { nodeContent, nodeMarks } from '../adf/document.ts' import { serializeCanonicalJson } from '../canonical-json.ts' import { textDirectiveName } from '../markdown/text-directive.ts' -import { toEditorNormal } from '../adf/editor-normal.ts' import { vocabularyPairs } from '../adf/attribute-vocabulary.ts' type Choice = { arbitrary: Arbitrary; hostile?: true; weight: number } @@ -433,7 +432,7 @@ test('generated markdown refuses, or what it parses to refuses to emit, or its s if (holdsDirectiveShape(parsed.value)) directiveShaped += 1 const read = markdownToAdf(emitted.value) assert.ok(read.ok, read.ok ? '' : `${read.error.code}: ${read.error.message} — reading ${JSON.stringify(emitted.value)}`) - assert.deepEqual(toEditorNormal(read.value), toEditorNormal(parsed.value), `reading ${JSON.stringify(emitted.value)}`) + assert.deepEqual(read.value, parsed.value, `reading ${JSON.stringify(emitted.value)}`) const respelled = adfToMarkdown(read.value) assert.ok(respelled.ok, respelled.ok ? '' : `${respelled.error.code}: ${respelled.error.message} — spelling ${JSON.stringify(emitted.value)} again`) assert.equal(respelled.value, emitted.value) diff --git a/src/conformance/property-harness.ts b/src/conformance/property-harness.ts index 5013bfa..0f3cc41 100644 --- a/src/conformance/property-harness.ts +++ b/src/conformance/property-harness.ts @@ -2,16 +2,17 @@ import assert from 'node:assert/strict' import { env } from 'node:process' import fc from 'fast-check' -import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../adf/document.ts' +import type { AdfAttributes, AdfDocument, AdfMark, AdfNode, EmptyKey } from '../adf/document.ts' import type { Arbitrary } from 'fast-check' import type { AttributeKind, AttributeVocabulary } from '../adf/attribute-vocabulary.ts' import type { JsonValue } from '../json-value.ts' import { blockArgument } from '../markdown/block-directive.ts' import { blockNodes } from '../adf/block-nodes.ts' import { directivePrefix } from '../markdown/directive-syntax.ts' +import { emptyKeys, mergeAdjacentText } from '../adf/document.ts' import { inlineNodes } from '../adf/inline-nodes.ts' +import { joinsWhenRead } from '../markdown/adjacent-text.ts' import { markAttributes } from '../adf/mark-attributes.ts' -import { toEditorNormal } from '../adf/editor-normal.ts' type Positions = { block: AdfNode; inline: AdfNode } @@ -36,7 +37,11 @@ export function textOf(minLength: number): Arbitrary { const text = textOf(1) const unknownType = fc.oneof(fc.stringMatching(/^[a-z][A-Za-z0-9]{0,7}$/), text).filter((type) => !spelledTypes.has(type)) -const numberValue = fc.oneof({ arbitrary: fc.integer({ max: 10, min: -1 }), weight: 3 }, { arbitrary: fc.double({ noDefaultInfinity: true, noNaN: true }), weight: 1 }) +const numberValue = fc.oneof( + { arbitrary: fc.integer({ max: 10, min: -1 }), weight: 6 }, + { arbitrary: fc.double({ noDefaultInfinity: true, noNaN: true }), weight: 2 }, + { arbitrary: fc.constant(-0), weight: 1 }, +) // V8's JSON.parse returns a wrong key after parsing a key holding an escaped backslash (https://issues.chromium.org/issues/521080746); Bun is unaffected. const keyPiece = fc @@ -73,19 +78,35 @@ function heldAttributes(held: Readonly>): return attrs } +// An empty attrs, content or marks key, and adjacent text CommonMark would read back as one, stay occasional: each takes a spelling outside CommonMark. +// The copy gives fast-check's null-prototype records the prototype a parsed node has. +function occasionallyEmpty(arbitrary: Arbitrary): Arbitrary { + return fc.tuple(arbitrary, fc.nat({ max: 9 })).map(([held, roll]) => (roll === 0 ? { ...held } : withoutEmptyKeys(held))) +} + +function withoutEmptyKeys(held: T): T { + const kept: Partial> & T = { ...held } + for (const key of emptyKeys(held)) delete kept[key] + return kept +} + +function occasionallyApart(arbitrary: Arbitrary): Arbitrary { + return fc.tuple(arbitrary, fc.nat({ max: 5 })).map(([nodes, roll]) => (roll === 0 ? nodes : mergeAdjacentText(nodes, joinsWhenRead))) +} + function pipeTable({ body, header }: { body: AdfNode[][]; header: AdfNode[] }): AdfNode { const rows = [header, ...body.map((cells) => header.map((_, index) => cells[index] ?? emptyCell))] return { content: rows.map((content): AdfNode => ({ content, type: 'tableRow' })), type: 'table' } } -const mark: Arbitrary = fc.oneof( +const mark: Arbitrary = occasionallyEmpty(fc.oneof( { arbitrary: fc.oneof(...Object.entries(markAttributes).map(([type, vocabulary]) => attributes(vocabulary).map((attrs) => ({ attrs, type })))), weight: 9 }, { arbitrary: attributes({ color: 'string' }).map((attrs) => ({ attrs, type: 'backgroundColor' })), weight: 2 }, { arbitrary: fc.record({ attrs: fc.dictionary(jsonKey, jsonValue, { maxKeys: 2, noNullPrototype: true }), type: unknownType }), weight: 1 }, -) +)) const marks = fc.uniqueArray(mark, { maxLength: 3, selector: (held) => held.type }) -const textNode = fc.record({ marks, text }).map((held): AdfNode => ({ ...held, type: 'text' })) +const textNode = occasionallyEmpty(fc.record({ marks, text }).map((held): AdfNode => ({ ...held, type: 'text' }))) const backtickRunNode = fc .record({ marks: fc.oneof(fc.constant([]), fc.constant([{ type: 'code' }]), marks), text: fc.string({ maxLength: 6, minLength: 1, unit: fc.constantFrom('`', '``', ' ', 'a') }) }) @@ -96,7 +117,7 @@ const autolinkTextNode = fc .map(({ href, marks: held }): AdfNode => ({ marks: [...held.filter((outer) => outer.type !== 'link'), { attrs: { href }, type: 'link' }], text: href, type: 'text' })) const inlineArbitraries = Object.entries(inlineNodes).map(([type, model]) => - fc.record({ attrs: attributes(model.attributes), marks }).map((held): AdfNode => ({ ...held, type })), + occasionallyEmpty(fc.record({ attrs: attributes(model.attributes), marks }).map((held): AdfNode => ({ ...held, type }))), ) function weighted(arbitraries: readonly Arbitrary[], weight: number): { arbitrary: Arbitrary; weight: number }[] { @@ -105,10 +126,10 @@ function weighted(arbitraries: readonly Arbitrary[], weight: number): { const positions = fc.letrec((tie) => { const blockContent = fc.array(tie('block'), { depthIdentifier, maxLength: 3 }) - const inlineContent = fc.array(tie('inline'), { depthIdentifier, maxLength: 4 }) + const inlineContent = occasionallyApart(fc.array(tie('inline'), { depthIdentifier, maxLength: 4 })) const contentByModel = { block: blockContent, - code: fc.array(text.map((held): AdfNode => ({ text: held, type: 'text' })), { maxLength: 2 }), + code: occasionallyApart(fc.array(text.map((held): AdfNode => ({ text: held, type: 'text' })), { maxLength: 2 })), inline: inlineContent, none: fc.constant([]), } @@ -116,35 +137,40 @@ const positions = fc.letrec((tie) => { const blockArbitraries = Object.entries(blockNodes).map(([type, model]) => { const argument = blockArgument(type) const vocabulary: AttributeVocabulary = argument === undefined ? model.attributes : { ...model.attributes, [argument]: 'string' } - const node = fc.record({ attrs: attributes(vocabulary), content: contentByModel[model.contentModel], marks: blockMarks }).map((held): AdfNode => ({ ...held, type })) + const node = occasionallyEmpty(fc.record({ attrs: attributes(vocabulary), content: contentByModel[model.contentModel], marks: blockMarks }).map((held): AdfNode => ({ ...held, type }))) return { leaf: model.contentModel === 'code' || model.contentModel === 'none', node } }) - const unknownNode = fc - .record({ attrs: fc.dictionary(jsonKey, jsonValue, { maxKeys: 2, noNullPrototype: true }), content: fc.array(tie('inline'), { depthIdentifier, maxLength: 2 }), marks, type: unknownType }) - .map((held): AdfNode => held) + const unknownNode = occasionallyEmpty( + fc.record({ attrs: fc.dictionary(jsonKey, jsonValue, { maxKeys: 2, noNullPrototype: true }), content: fc.array(tie('inline'), { depthIdentifier, maxLength: 2 }), marks, type: unknownType }), + ) const leafBlocks = blockArbitraries.filter((entry) => entry.leaf).map((entry) => entry.node) const containerBlocks = blockArbitraries.filter((entry) => !entry.leaf).map((entry) => entry.node) const misplacedWeight = 7 - const paragraph = fc.oneof({ arbitrary: inlineContent, weight: 3 }, { arbitrary: fc.array(backtickRunNode, { maxLength: 4, minLength: 2 }), weight: 1 }).map((content): AdfNode => ({ content, type: 'paragraph' })) + const paragraph = occasionallyEmpty( + fc.oneof({ arbitrary: inlineContent, weight: 3 }, { arbitrary: occasionallyApart(fc.array(backtickRunNode, { maxLength: 4, minLength: 2 })), weight: 1 }).map((content): AdfNode => ({ content, type: 'paragraph' })), + ) const cell = (type: string) => paragraph.map((held): AdfNode => ({ content: [held], type })) const listItems = fc.array( - blockContent.map((content): AdfNode => ({ content, type: 'listItem' })), + occasionallyEmpty(blockContent.map((content): AdfNode => ({ content, type: 'listItem' }))), { depthIdentifier, maxLength: 3, minLength: 1 }, ) const flatCommonMarkShapes = [ - fc.record({ content: inlineContent, level: fc.integer({ max: 6, min: 1 }) }).map(({ content, level }): AdfNode => ({ attrs: { level }, content, type: 'heading' })), + occasionallyEmpty(fc.record({ content: inlineContent, level: fc.integer({ max: 6, min: 1 }) }).map(({ content, level }): AdfNode => ({ attrs: { level }, content, type: 'heading' }))), paragraph, fc.record({ body: fc.array(fc.array(cell('tableCell'), { maxLength: 3 }), { maxLength: 2 }), header: fc.array(cell('tableHeader'), { maxLength: 3, minLength: 1 }) }).map(pipeTable), ] const task = (type: string, content: Arbitrary) => - fc.record({ content, state: fc.constantFrom('DONE', 'TODO') }).map(({ content: held, state }): AdfNode => ({ attrs: { state }, content: held, type })) + occasionallyEmpty(fc.record({ content, state: fc.constantFrom('DONE', 'TODO') }).map(({ content: held, state }): AdfNode => ({ attrs: { state }, content: held, type }))) const taskItem = fc.oneof({ arbitrary: task('taskItem', inlineContent), weight: 3 }, { arbitrary: task('blockTaskItem', blockContent), weight: 1 }) const nestingCommonMarkShapes = [ - blockContent.map((content): AdfNode => ({ content, type: 'blockquote' })), + occasionallyEmpty(blockContent.map((content): AdfNode => ({ content, type: 'blockquote' }))), fc.array(fc.oneof({ arbitrary: taskItem, weight: 3 }, { arbitrary: tie('block'), weight: 1 }), { depthIdentifier, maxLength: 3, minLength: 1 }).map((content): AdfNode => ({ content, type: 'taskList' })), listItems.map((content): AdfNode => ({ content, type: 'bulletList' })), fc - .record({ content: listItems, order: fc.oneof({ arbitrary: fc.integer({ max: 3, min: 0 }), weight: 4 }, { arbitrary: fc.integer({ max: 999999999, min: 0 }), weight: 1 }) }) + .record({ + content: listItems, + order: fc.oneof({ arbitrary: fc.integer({ max: 3, min: 0 }), weight: 8 }, { arbitrary: fc.integer({ max: 999999999, min: 0 }), weight: 2 }, { arbitrary: fc.constant(-0), weight: 1 }), + }) .map(({ content, order }): AdfNode => ({ attrs: { order }, content, type: 'orderedList' })), ] const flatBlocks = [...weighted(leafBlocks, 2), ...weighted(flatCommonMarkShapes, flatCommonMarkShapeWeight)] @@ -167,7 +193,11 @@ const positions = fc.letrec((tie) => { } }) -export const adfDocument = fc.array(positions.block, { depthIdentifier, maxLength: 4, minLength: 1 }).map((content): AdfDocument => toEditorNormal({ content, type: 'doc', version: 1 })) +export const adfDocument = fc.oneof( + { arbitrary: fc.array(positions.block, { depthIdentifier, maxLength: 4, minLength: 1 }).map((content): AdfDocument => ({ content, type: 'doc', version: 1 })), weight: 30 }, + { arbitrary: fc.constant({ content: [], type: 'doc', version: 1 }), weight: 1 }, + { arbitrary: fc.constant({ type: 'doc', version: 1 }), weight: 1 }, +) export function propertyRuns(gateRuns: number): { gate: boolean; numRuns: number; seed?: number } { const deepRuns = env[deepRunsVariable] diff --git a/src/index.ts b/src/index.ts index a25f670..e4de874 100644 --- a/src/index.ts +++ b/src/index.ts @@ -3,6 +3,6 @@ export type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from './adf/documen export type { ConvertError, ConvertErrorCode, ConvertErrorPath, ParseError, Result, SourcePosition } from './result.ts' export type { JsonValue } from './json-value.ts' export { adfToMarkdown } from './markdown/emit/adf-to-markdown.ts' -export { adfToPlainMarkdown } from './markdown/emit/plain-reduction.ts' +export { adfToPlainMarkdown } from './markdown/plain/adf-to-plain-markdown.ts' export { isAdfDocument } from './adf/document.ts' export { markdownToAdf, plainMarkdownToAdf } from './markdown/parse/markdown-to-adf.ts' diff --git a/src/markdown/adjacent-text.ts b/src/markdown/adjacent-text.ts new file mode 100644 index 0000000..851f2a3 --- /dev/null +++ b/src/markdown/adjacent-text.ts @@ -0,0 +1,12 @@ +import type { AdfNode } from '../adf/document.ts' +import { identicalMarks, isBareText, nodeMarks } from '../adf/document.ts' +import { spellInlineLeafDirective } from './directive-syntax.ts' + +export const textBreakName = 'textBreak' + +export const textBreakSpelling = spellInlineLeafDirective(textBreakName, '') + +// The lossless reader's rule: CommonMark reads the pair back as one text run (spec/flavour.md, Inline nodes). +export function joinsWhenRead(previous: AdfNode, node: AdfNode): boolean { + return isBareText(previous) && isBareText(node) && identicalMarks(nodeMarks(previous), nodeMarks(node)) +} diff --git a/src/markdown/block-directive.ts b/src/markdown/block-directive.ts index c528e23..83ff2a0 100644 --- a/src/markdown/block-directive.ts +++ b/src/markdown/block-directive.ts @@ -2,9 +2,9 @@ import type { AdfMark } from '../adf/document.ts' import type { BlockType } from '../adf/block-nodes.ts' import type { JsonValue } from '../json-value.ts' import { blockNodeModel } from '../adf/block-nodes.ts' -import { isAdfMark, nodeAttrs } from '../adf/document.ts' +import { isAdfMark } from '../adf/document.ts' import { serializeCanonicalJson } from '../canonical-json.ts' -import { spellDirectiveOpener } from './directive-syntax.ts' +import { spellAttributes, spellDirectiveOpener } from './directive-syntax.ts' const argumentByType = new Map( Object.entries({ @@ -14,6 +14,13 @@ const argumentByType = new Map( } satisfies Partial>), ) +export const documentName = 'doc' + +// spec/flavour.md, Directives: a document holding no content key, which the empty string cannot spell. +export const documentAttribute = { key: 'content', value: 'none' } + +export const documentSpelling = spellDirectiveOpener(documentName, undefined, spellAttributes([[documentAttribute.key, documentAttribute.value]])) + export const listBreakName = 'listBreak' export const listBreakSpelling = spellDirectiveOpener(listBreakName, undefined, '') @@ -25,17 +32,14 @@ export function blockArgument(type: string): string | undefined { } export function blockDirectiveForm(name: string): 'container' | 'leaf' | undefined { - if (name === listBreakName) return 'leaf' + if (name === listBreakName || name === documentName) return 'leaf' const model = blockNodeModel(name) if (model === undefined) return undefined return model.contentModel === 'none' ? 'leaf' : 'container' } export function markValues(marks: readonly AdfMark[]): JsonValue { - return marks.map((mark) => { - const attrs = nodeAttrs(mark) - return Object.keys(attrs).length === 0 ? { type: mark.type } : { attrs, type: mark.type } - }) + return marks.map((mark) => (mark.attrs === undefined ? { type: mark.type } : { attrs: mark.attrs, type: mark.type })) } export function readMarkValues(value: JsonValue): AdfMark[] | undefined { diff --git a/src/markdown/code-language.ts b/src/markdown/code-language.ts index 223e75a..846b64e 100644 --- a/src/markdown/code-language.ts +++ b/src/markdown/code-language.ts @@ -1,14 +1,12 @@ import type { JsonValue } from '../json-value.ts' -import { carryName } from './opaque-carry.ts' -import { holdsControlCharacter } from './commonmark/grammar.ts' -import { holdsEntityReference } from './commonmark/entity-references.ts' +import { carryFenceType } from './opaque-carry.ts' +import { infoStringCarries } from './commonmark/grammar.ts' export type LanguageSlot = { info: string; kind: 'fence' } | { kind: 'attribute' } | { kind: 'none' } // spec/flavour.md, The CommonMark blocks: the one slot a codeBlock's language rides. export function languageSlot(language: JsonValue | undefined): LanguageSlot { if (language === undefined) return { kind: 'none' } - if (typeof language !== 'string' || language === '' || language === carryName) return { kind: 'attribute' } - if (/[`\\]/.test(language) || holdsControlCharacter(language) || language !== language.trim() || holdsEntityReference(language)) return { kind: 'attribute' } + if (typeof language !== 'string' || carryFenceType(language) !== undefined || !infoStringCarries(language)) return { kind: 'attribute' } return { info: language, kind: 'fence' } } diff --git a/src/markdown/commonmark/grammar.ts b/src/markdown/commonmark/grammar.ts index 53c9f1a..5d394e2 100644 --- a/src/markdown/commonmark/grammar.ts +++ b/src/markdown/commonmark/grammar.ts @@ -1,4 +1,4 @@ -import { readEntityReference, replacementCharacter } from './entity-references.ts' +import { holdsEntityReference, readEntityReference, replacementCharacter } from './entity-references.ts' export type LinePosition = 'first' | 'later' @@ -133,6 +133,11 @@ export function holdsControlCharacter(text: string): boolean { return controlCharacter.test(text) } +// What a backtick fence's info string reads back verbatim: escapes and entity references decode, and the edges trim. +export function infoStringCarries(text: string): boolean { + return text !== '' && !/[`\\]/.test(text) && !holdsControlCharacter(text) && text === text.trim() && !holdsEntityReference(text) +} + export function holdsNullCharacter(text: string): boolean { return nullCharacter.test(text) } diff --git a/src/markdown/directive-syntax.ts b/src/markdown/directive-syntax.ts index 7c992da..c4f4cfd 100644 --- a/src/markdown/directive-syntax.ts +++ b/src/markdown/directive-syntax.ts @@ -139,7 +139,7 @@ export function spellStringAttribute(text: string): string { export function spellAttributeValue(value: VocabularyValue): string { if (value.kind === 'boolean') return `${value.value}` if (value.kind === 'json') return spellJsonAttribute(value.value) - if (value.kind === 'number') return spellStringAttribute(JSON.stringify(value.value)) + if (value.kind === 'number') return spellStringAttribute(serializeCanonicalJson(value.value, 'compact')) return spellStringAttribute(value.value) } diff --git a/src/markdown/emit/adf-to-markdown.test.ts b/src/markdown/emit/adf-to-markdown.test.ts index daf574f..b92e06c 100644 --- a/src/markdown/emit/adf-to-markdown.test.ts +++ b/src/markdown/emit/adf-to-markdown.test.ts @@ -6,14 +6,13 @@ import type { JsonValue } from '../../json-value.ts' import type { Result } from '../../result.ts' import { adfToMarkdown, markdownToAdf } from '../../index.ts' import { largestNesting } from '../../nesting.ts' -import { toEditorNormal } from '../../adf/editor-normal.ts' function document(...content: AdfNode[]): AdfDocument { return { content, type: 'doc', version: 1 } } function paragraph(...content: AdfNode[]): AdfNode { - return { content, type: 'paragraph' } + return content.length === 0 ? { type: 'paragraph' } : { content, type: 'paragraph' } } function code(result: Result): string { @@ -168,21 +167,24 @@ test('parts two adjacent lists of the same kind, the marker spelling being what const nested: AdfNode = { content: [{ content: [list, list], type: 'listItem' }], type: 'bulletList' } assert.equal(markdown(adfToMarkdown(document(nested))), '- - x\n\n !adf:listBreak\n\n - x\n') const carried: AdfNode = { ...list, attrs: { unknown: 'x' } } - assert.ok(markdown(adfToMarkdown(document(carried, carried))).includes('```\n\n```carry\n')) + assert.ok(markdown(adfToMarkdown(document(carried, carried))).includes('```\n\n```adf:bulletList\n')) assert.ok(markdown(adfToMarkdown(document(carried, list))).endsWith('```\n\n- x\n')) - assert.ok(markdown(adfToMarkdown(document(list, carried))).startsWith('- x\n\n```carry\n')) + assert.ok(markdown(adfToMarkdown(document(list, carried))).startsWith('- x\n\n```adf:bulletList\n')) }) -test('carries a node type no section spells', () => { - assert.equal(markdown(adfToMarkdown(document({ type: 'blockCard' }))), '```carry\n{\n "type": "blockCard"\n}\n```\n') - assert.equal(markdown(adfToMarkdown(document({ type: 'toString' }))), '```carry\n{\n "type": "toString"\n}\n```\n') +test('carries a node type no section spells, the fence naming the type wherever an info string carries it', () => { + assert.equal(markdown(adfToMarkdown(document({ type: 'blockCard' }))), '```adf:blockCard\n{}\n```\n') + assert.equal(markdown(adfToMarkdown(document({ type: 'toString' }))), '```adf:toString\n{}\n```\n') assert.equal(markdown(adfToMarkdown(document(paragraph({ type: 'blockCard' })))), '!adf:carry{json="{\\"type\\":\\"blockCard\\"}"}\n') - assert.equal(markdown(adfToMarkdown(document({ text: 'x', type: 'text' }))), '```carry\n{\n "text": "x",\n "type": "text"\n}\n```\n') - assert.equal(markdown(adfToMarkdown(document({ type: 'hardBreak' }))), '```carry\n{\n "type": "hardBreak"\n}\n```\n') + assert.equal(markdown(adfToMarkdown(document({ text: 'x', type: 'text' }))), '```adf:text\n{\n "text": "x"\n}\n```\n') + assert.equal(markdown(adfToMarkdown(document({ type: 'hardBreak' }))), '```adf:hardBreak\n{}\n```\n') + assert.equal(markdown(adfToMarkdown(document({ type: 'a&b' }))), '```adf:\n{\n "type": "a&b"\n}\n```\n') + assert.equal(markdown(adfToMarkdown(document({ type: 'a\\b' }))), '```adf:\n{\n "type": "a\\\\b"\n}\n```\n') }) -test('spells the code block whose language is the reserved info string', () => { - assert.equal(markdown(adfToMarkdown(document({ attrs: { language: 'carry' }, type: 'codeBlock' }))), '!adf:codeBlock {language=carry}\n```\n```\n!adf:/codeBlock\n') +test('spells the code block whose language opens with the reserved info string prefix', () => { + assert.equal(markdown(adfToMarkdown(document({ attrs: { language: 'adf:x' }, type: 'codeBlock' }))), '!adf:codeBlock {language="adf:x"}\n```\n```\n!adf:/codeBlock\n') + assert.equal(markdown(adfToMarkdown(document({ attrs: { language: 'carry' }, type: 'codeBlock' }))), '```carry\n```\n') assert.equal(markdown(adfToMarkdown(document({ attrs: { language: 'adf' }, type: 'codeBlock' }))), '```adf\n```\n') }) @@ -208,21 +210,17 @@ test('refuses a carried node nested deeper than the levels its position leaves', assert.equal(code(adfToMarkdown(document(quoted))), 'unsupported-nesting-depth') }) -test('refuses a node whose content model the canonical form cannot emit', () => { - assert.equal( - markdown(adfToMarkdown(document({ content: [paragraph()], type: 'codeBlock' }))), - 'unsupported-node-shape: a codeBlock holds plain text nodes only: this paragraph node is not one', - ) - assert.equal( - markdown(adfToMarkdown(document({ content: [{ content: [{ text: 'lost', type: 'text' }], text: 'x', type: 'text' }], type: 'codeBlock' }))), - 'unsupported-node-shape: a codeBlock holds plain text nodes only: this text node is not one', - ) +test('carries a code block holding a node no fence holds', () => { + assert.equal(markdown(adfToMarkdown(document({ content: [paragraph()], type: 'codeBlock' }))), '```adf:codeBlock\n{\n "content": [\n {\n "type": "paragraph"\n }\n ]\n}\n```\n') + for (const child of [{ attrs: {}, text: 'x', type: 'text' }, { content: [], text: 'x', type: 'text' }, { marks: [], text: 'x', type: 'text' }, { content: [{ text: 'lost', type: 'text' }], text: 'x', type: 'text' }]) { + assert.ok(markdown(adfToMarkdown(document({ content: [child], type: 'codeBlock' }))).startsWith('```adf:codeBlock\n'), JSON.stringify(child)) + } }) test('spells a list its own content shape cannot hold as a directive', () => { assert.equal(markdown(adfToMarkdown(document({ content: [paragraph()], type: 'bulletList' }))), '!adf:bulletList\n!adf:paragraph\n!adf:/paragraph\n!adf:/bulletList\n') assert.equal(markdown(adfToMarkdown(document({ type: 'bulletList' }))), '!adf:bulletList\n!adf:/bulletList\n') - assert.equal(markdown(adfToMarkdown(document({ attrs: { order: 2 }, content: [], type: 'orderedList' }))), '!adf:orderedList {order=2}\n!adf:/orderedList\n') + assert.equal(markdown(adfToMarkdown(document({ attrs: { order: 2 }, content: [], type: 'orderedList' }))), '!adf:orderedList {content=empty order=2}\n!adf:/orderedList\n') }) test('spells an ordered list no marker fits as a directive', () => { @@ -334,7 +332,7 @@ test('refuses a node carrying one mark type twice', () => { }) test('parts a nested list the tight spelling would swallow from the block above it', () => { - const item = (...content: AdfNode[]): AdfNode => ({ content, type: 'listItem' }) + const item = (...content: AdfNode[]): AdfNode => (content.length === 0 ? { type: 'listItem' } : { content, type: 'listItem' }) const text = (value: string): AdfNode => ({ content: [{ text: value, type: 'text' }], type: 'paragraph' }) const outer = (...content: AdfNode[]): AdfDocument => document({ content: [item(...content)], type: 'bulletList' }) const ordered: AdfNode = { attrs: { order: 2 }, content: [item(text('b'))], type: 'orderedList' } @@ -345,7 +343,7 @@ test('parts a nested list the tight spelling would swallow from the block above const list: AdfNode = { content: [item(text('b'))], type: 'bulletList' } const panel: AdfNode = { attrs: { panelType: 'info' }, content: [text('p')], type: 'panel' } assert.equal(markdown(adfToMarkdown(outer(panel, list))), '- !adf:panel info\n p\n !adf:/panel\n\n - b\n') - assert.ok(markdown(adfToMarkdown(outer(text('a'), { ...list, attrs: { unknown: 'x' } }))).startsWith('- a\n\n ```carry\n')) + assert.ok(markdown(adfToMarkdown(outer(text('a'), { ...list, attrs: { unknown: 'x' } }))).startsWith('- a\n\n ```adf:bulletList\n')) }) test('refuses marks and attributes nested deeper than the emitter carries', () => { @@ -372,7 +370,7 @@ test('refuses marks and attributes nested deeper than the emitter carries', () = assert.ok(spelled.ok, spelled.ok ? '' : spelled.error.message) const read = markdownToAdf(spelled.value) assert.ok(read.ok, read.ok ? '' : read.error.message) - assert.deepEqual(toEditorNormal(read.value), document(node)) + assert.deepEqual(read.value, document(node)) } assert.equal(markdown(adfToMarkdown(document(paragraph({ marks: [{ attrs, type: 'em' }], text: 'x', type: 'text' })))), deeper('depth', 'em')) @@ -442,7 +440,7 @@ test('escapes a hyphen underline a hard break would expose', () => { }) test('spells a list item whose marker completes a thematic break as a directive', () => { - const item = (...content: AdfNode[]): AdfNode => ({ content, type: 'listItem' }) + const item = (...content: AdfNode[]): AdfNode => (content.length === 0 ? { type: 'listItem' } : { content, type: 'listItem' }) assert.equal(markdown(adfToMarkdown(document({ content: [item({ type: 'rule' })], type: 'bulletList' }))), '!adf:bulletList\n!adf:listItem\n---\n!adf:/listItem\n!adf:/bulletList\n') const nested: AdfNode = { content: [item({ content: [item()], type: 'bulletList' })], type: 'bulletList' } assert.equal(markdown(adfToMarkdown(document(nested))), '- -\n') @@ -469,10 +467,7 @@ test('refuses the characters CommonMark rewrites', () => { ) assert.equal(code(adfToMarkdown(document(paragraph({ text: 'a\u0000b', type: 'text' })))), 'unspellable-character') assert.equal(code(adfToMarkdown(document({ content: [{ text: 'a\u0000b', type: 'text' }], type: 'codeBlock' }))), 'unspellable-character') - assert.equal( - markdown(adfToMarkdown(document({ content: [{ text: '', type: 'text' }], type: 'codeBlock' }))), - 'unsupported-node-shape: a codeBlock holds plain text nodes only: this text node is not one', - ) + assert.equal(markdown(adfToMarkdown(document({ content: [{ text: '', type: 'text' }], type: 'codeBlock' }))), 'unsupported-node-shape: a text node holds text: this one has none') }) test('refuses a text node the spelling would empty out', () => { @@ -501,11 +496,11 @@ test('pads a code span whose edges CommonMark would strip', () => { assert.equal(markdown(adfToMarkdown(document(paragraph({ marks: [{ type: 'code' }], text: ' \t ', type: 'text' })))), '` \t `\n') }) -test('spells one code span over a run of code-marked nodes', () => { +test('spells a code span per code-marked node, parted by the text break', () => { const code_ = { type: 'code' } assert.equal( markdown(adfToMarkdown(document(paragraph({ marks: [code_], text: 'a', type: 'text' }, { marks: [code_], text: 'b', type: 'text' })))), - '`ab`\n', + '`a`!adf:textBreak{}`b`\n', ) }) @@ -545,7 +540,7 @@ test('emits an empty list item without trailing whitespace', () => { test('spells a block directive as its node type, arg and attributes', () => { const panel = (attrs: AdfAttributes): AdfDocument => document({ attrs, content: [paragraph({ text: 'x', type: 'text' })], type: 'panel' }) assert.equal(markdown(adfToMarkdown(panel({ panelType: 'warning' }))), '!adf:panel warning\nx\n!adf:/panel\n') - assert.equal(markdown(adfToMarkdown(panel({}))), '!adf:panel\nx\n!adf:/panel\n') + assert.equal(markdown(adfToMarkdown(panel({}))), '!adf:panel {attrs=empty}\nx\n!adf:/panel\n') assert.equal(markdown(adfToMarkdown(document({ content: [{ text: 'x', type: 'text' }], type: 'caption' }))), '!adf:caption\nx\n!adf:/caption\n') assert.equal(markdown(adfToMarkdown(document({ type: 'caption' }))), '!adf:caption\n!adf:/caption\n') assert.equal(markdown(adfToMarkdown(document({ attrs: { localId: 'a' }, type: 'syncBlock' }))), '!adf:syncBlock {localId=a}\n') @@ -553,20 +548,20 @@ test('spells a block directive as its node type, arg and attributes', () => { test('carries a directive attribute no section spells', () => { const carried = (node: AdfNode): string => markdown(adfToMarkdown(document(node))) - assert.equal(carried({ attrs: { rounded: true }, type: 'panel' }), '```carry\n{\n "attrs": {\n "rounded": true\n },\n "type": "panel"\n}\n```\n') - assert.equal(carried({ attrs: { toString: 'x' }, type: 'panel' }), '```carry\n{\n "attrs": {\n "toString": "x"\n },\n "type": "panel"\n}\n```\n') - assert.equal(carried({ attrs: { localId: 4 }, type: 'panel' }), '```carry\n{\n "attrs": {\n "localId": 4\n },\n "type": "panel"\n}\n```\n') + assert.equal(carried({ attrs: { rounded: true }, type: 'panel' }), '```adf:panel\n{\n "attrs": {\n "rounded": true\n }\n}\n```\n') + assert.equal(carried({ attrs: { toString: 'x' }, type: 'panel' }), '```adf:panel\n{\n "attrs": {\n "toString": "x"\n }\n}\n```\n') + assert.equal(carried({ attrs: { localId: 4 }, type: 'panel' }), '```adf:panel\n{\n "attrs": {\n "localId": 4\n }\n}\n```\n') assert.equal( carried({ attrs: { width: '50' }, type: 'layoutColumn' }), - '```carry\n{\n "attrs": {\n "width": "50"\n },\n "type": "layoutColumn"\n}\n```\n', + '```adf:layoutColumn\n{\n "attrs": {\n "width": "50"\n }\n}\n```\n', ) assert.equal( carried({ attrs: { isNumberColumnEnabled: 'true' }, type: 'table' }), - '```carry\n{\n "attrs": {\n "isNumberColumnEnabled": "true"\n },\n "type": "table"\n}\n```\n', + '```adf:table\n{\n "attrs": {\n "isNumberColumnEnabled": "true"\n }\n}\n```\n', ) assert.equal( carried({ content: [{ attrs: { alt: 4 }, type: 'media' }], type: 'mediaGroup' }), - '!adf:mediaGroup\n```carry\n{\n "attrs": {\n "alt": 4\n },\n "type": "media"\n}\n```\n!adf:/mediaGroup\n', + '!adf:mediaGroup\n```adf:media\n{\n "attrs": {\n "alt": 4\n }\n}\n```\n!adf:/mediaGroup\n', ) }) @@ -574,15 +569,16 @@ test('carries an arg slot value no bare token spells', () => { const carried = (node: AdfNode): string => markdown(adfToMarkdown(document(node))) assert.equal( carried({ attrs: { panelType: 'extra info' }, type: 'panel' }), - '```carry\n{\n "attrs": {\n "panelType": "extra info"\n },\n "type": "panel"\n}\n```\n', + '```adf:panel\n{\n "attrs": {\n "panelType": "extra info"\n }\n}\n```\n', ) - assert.equal(carried({ attrs: { state: 2 }, type: 'taskItem' }), '```carry\n{\n "attrs": {\n "state": 2\n },\n "type": "taskItem"\n}\n```\n') + assert.equal(carried({ attrs: { state: 2 }, type: 'taskItem' }), '```adf:taskItem\n{\n "attrs": {\n "state": 2\n }\n}\n```\n') }) test('carries a block node mark in the reserved attribute', () => { const section = (...marks: AdfMark[]): AdfDocument => document({ marks, type: 'layoutSection' }) assert.equal(markdown(adfToMarkdown(section({ type: 'breakout' }))), '!adf:layoutSection {marks="[{\\"type\\":\\"breakout\\"}]"}\n!adf:/layoutSection\n') - assert.equal(markdown(adfToMarkdown(section({ attrs: {}, type: 'breakout' }))), '!adf:layoutSection {marks="[{\\"type\\":\\"breakout\\"}]"}\n!adf:/layoutSection\n') + assert.equal(markdown(adfToMarkdown(section({ attrs: {}, type: 'breakout' }))), '!adf:layoutSection {marks="[{\\"attrs\\":{},\\"type\\":\\"breakout\\"}]"}\n!adf:/layoutSection\n') + assert.equal(markdown(adfToMarkdown(section())), '!adf:layoutSection {marks=empty}\n!adf:/layoutSection\n') }) test('refuses the content a directive body has no room for', () => { @@ -604,7 +600,7 @@ test('separates blocks in a container body by a blank line only where a directiv test('spells the image form for exactly the centered external media shape', () => { const url = 'https://example.com/moon.png' const single = (attrs: AdfAttributes, ...content: AdfNode[]): AdfDocument => - document({ attrs: { layout: 'center' }, content: [{ attrs, content, type: 'media' }], type: 'mediaSingle' }) + document({ attrs: { layout: 'center' }, content: [content.length === 0 ? { attrs, type: 'media' } : { attrs, content, type: 'media' }], type: 'mediaSingle' }) assert.equal(markdown(adfToMarkdown(single({ alt: 'The moon', type: 'external', url }))), `![The moon](${url})\n`) assert.equal(markdown(adfToMarkdown(single({ type: 'external', url }))), `![](${url})\n`) assert.equal(markdown(adfToMarkdown(single({ alt: 'a [b] c', type: 'external', url }))), `![a \\[b\\] c](${url})\n`) @@ -701,7 +697,8 @@ test('spells the directive marks around the longest run they cover', () => { const emitted = (...content: AdfNode[]): string => markdown(adfToMarkdown(document(paragraph(...content)))) const underline: AdfMark = { type: 'underline' } assert.equal(emitted(marked('x', underline)), '!adf:underline[x]\n') - assert.equal(emitted(marked('a', underline), marked('b', underline)), '!adf:underline[ab]\n') + assert.equal(emitted(marked('a', underline), marked('b', underline)), '!adf:underline[a!adf:textBreak{}b]\n') + assert.equal(emitted(marked('a', { attrs: {}, type: 'underline' }), marked('b', underline)), '!adf:underline[a]{attrs=empty}!adf:underline[b]\n') assert.equal(emitted(marked('x', { attrs: { type: 'sub' }, type: 'subsup' })), '!adf:subsup[x]{type=sub}\n') assert.equal(emitted(marked('x', { attrs: { color: '#ae2e24' }, type: 'textColor' })), '!adf:textColor[x]{color="#ae2e24"}\n') assert.equal(emitted(marked('x', { attrs: { color: '#091e42', size: 2 }, type: 'border' })), '!adf:border[x]{color="#091e42" size=2}\n') @@ -746,5 +743,5 @@ test('carries whitespace CommonMark strips in the reserved text directive', () = test("joins a mark run's segments as a walk rather than as one call's arguments", () => { const run = Array.from({ length: 200000 }, (): AdfNode => ({ marks: [{ type: 'strong' }], text: 'a', type: 'text' })) - assert.equal(markdown(adfToMarkdown(document({ content: run, type: 'paragraph' }))), `**${'a'.repeat(200000)}**\n`) + assert.equal(markdown(adfToMarkdown(document({ content: run, type: 'paragraph' }))), `**${Array.from({ length: 200000 }, () => 'a').join('!adf:textBreak{}')}**\n`) }) diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index 0d41fe4..5c17534 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -1,16 +1,16 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { BlockNodeModel } from '../../adf/block-nodes.ts' -import type { Flavour } from '../plain-conventions.ts' -import { adfDocumentFault, carriesOnly, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' -import { alertMarker, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../plain-conventions.ts' -import { blockDirectiveForm, listBreakSpelling } from '../block-directive.ts' +import type { Flavour } from '../plain/conventions.ts' +import { adfDocumentFault, holdsOnlyAttributes, isUnmarkedBareText, nodeAttrs, nodeContent } from '../../adf/document.ts' +import { alertMarker, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../plain/conventions.ts' +import { blockDirectiveForm, documentSpelling, listBreakSpelling } from '../block-directive.ts' import { blockNodeModel, blockNodes } from '../../adf/block-nodes.ts' import { carriedBlock } from '../opaque-carry.ts' import { emitInlineLine } from './inline-line.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { fencedCodeBlock } from '../commonmark/backtick-runs.ts' import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts' -import { languageSlot } from '../code-language.ts' +import { languageSlot, type LanguageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' import { spellBlockDirectiveOpener } from './block-directive-spelling.ts' import { spellDirectiveCloser, spellDirectiveOpener } from '../directive-syntax.ts' @@ -22,10 +22,10 @@ type BlockSpelling = 'commonmark' | 'directive' | 'list' type EmittedBlock = { headroom: number; spelling: BlockSpelling; text: string } type KeptSpelling = { block: EmittedBlock | undefined; depth: number } type PlacedBlock = Omit & { node: AdfNode } +type PlacedBlocks = { blocks: readonly PlacedBlock[]; headroom: number } // Keyed by reference: only a caller building one object per position (the parse, the plain reduction) passes one; a consumer's document may share a node. export type SpellingMemo = Map -type Walk = { blocks: readonly PlacedBlock[]; headroom: number } -type WalkedItem = { node: AdfNode; walk: Walk } +type WalkedItem = { node: AdfNode; walk: PlacedBlocks } export type Writing = { flavour: Flavour; memo: SpellingMemo | undefined } export const largestListMarker = 999999999 @@ -40,14 +40,15 @@ export function writeMarkdown(document: AdfDocument, flavour: Flavour): Result { +function walkBlocks(nodes: readonly AdfNode[], path: ConvertErrorPath, depth: number, writing: Writing): Result { let headroom = largestNesting - depth if (headroom < 0) return tooDeep(path) const blocks: PlacedBlock[] = [] @@ -75,15 +76,15 @@ function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): } function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: BlockContainer): string { - const plainPair = previous.spelling !== 'directive' && next.spelling !== 'directive' - if (plainPair && next.spelling === 'list') { + const bothCommonMark = previous.spelling !== 'directive' && next.spelling !== 'directive' + if (bothCommonMark && next.spelling === 'list') { if (previous.spelling === 'list' && previous.node.type === next.node.type) { const gap = container === 'directive' ? '\n' : '\n\n' return `${gap}${listBreakSpelling}${gap}` } if (container === 'list-item') return interruptsParagraph(next.node) ? '\n' : '\n\n' } - return container === 'directive' && !plainPair ? '\n' : '\n\n' + return container === 'directive' && !bothCommonMark ? '\n' : '\n\n' } function interruptsParagraph(node: AdfNode): boolean { @@ -180,13 +181,13 @@ function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writi return lines.includes(undefined) ? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path) : success({ headroom, spelling: 'list', text: lines.join('\n') }) } -function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result { +function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result { const block = emitBlock(node, path, depth, writing) return block.ok ? success({ blocks: [{ ...block.value, node }], headroom: block.value.headroom }) : block } // The marker leads the first paragraph, or stands as one where the blocks open with another. -function taskBlocks(task: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result { +function taskBlocks(task: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result { const marker = taskMarker(nodeAttrs(task)['state']) const markerBlock: PlacedBlock = { node: { type: 'paragraph' }, spelling: 'commonmark', text: marker } if (task.type === 'taskItem') { @@ -219,7 +220,7 @@ function directivePair(node: AdfNode, opener: string, body: string, headroom: nu return { headroom, spelling: 'directive', text: `${opener}\n${body === '' ? '' : `${body}\n`}${spellDirectiveCloser(node.type)}` } } -function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number, walkBody: () => Result): Result { +function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number, walkBody: () => Result): Result { if (node.text !== undefined) return failure('unsupported-node-shape', `a ${node.type} carries no text: this one holds text`, path) if (blockDirectiveForm(node.type) === 'leaf' && nodeContent(node).length > 0) return failure('unsupported-node-shape', `a ${node.type} holds no content: this one holds some`, path) if (model.contentModel === 'code') return emitCodeDirective(node, model, path, depth) @@ -229,7 +230,7 @@ function emitDirectiveBlock(node: AdfNode, model: BlockNodeModel, path: ConvertE return emitDirectiveBody(node, model, opener.value, path, walkBody) } -function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string, path: ConvertErrorPath, walkBody: () => Result): Result { +function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string, path: ConvertErrorPath, walkBody: () => Result): Result { if (blockDirectiveForm(node.type) === 'leaf') return success({ headroom: Number.POSITIVE_INFINITY, spelling: 'directive', text: opener }) if (model.contentModel === 'inline') { const line = emitInlineLine(nodeContent(node), 'paragraph', path, 'lossless') @@ -242,7 +243,7 @@ function emitDirectiveBody(node: AdfNode, model: BlockNodeModel, opener: string, } function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result | undefined { - if (!carriesOnly(node, [])) return undefined + if (!holdsOnlyAttributes(node, [])) return undefined const inner = walkBlocks(nodeContent(node), path, depth + 1, writing) if (!inner.ok) return inner const text = joinBlocks(inner.value.blocks, 'document') @@ -251,47 +252,46 @@ function tryBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number, wri } function tryCodeBlock(node: AdfNode, path: ConvertErrorPath): Result | undefined { - if (!carriesOnly(node, ['language'])) return undefined + if (!holdsOnlyAttributes(node, ['language'])) return undefined const slot = languageSlot(nodeAttrs(node)['language']) if (slot.kind === 'attribute') return undefined - const text = codeBlockText(node, path) - if (!text.ok) return text - return success(commonMarkText(fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', text.value))) + const texts = fencedTexts(node, path) + if (!texts.ok) return texts + const [only, ...others] = texts.value ?? [] + if (only === undefined || others.length > 0) return undefined + return success(commonMarkText(fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', only))) } +// spec/flavour.md, The CommonMark blocks: one fence per text node, the language on each; with no fence to carry it, the attribute does. function emitCodeDirective(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, depth: number): Result { - const slot = languageSlot(nodeAttrs(node)['language']) + const texts = fencedTexts(node, path) + if (!texts.ok) return texts + if (texts.value === undefined) return commonMarkLine(carriedBlock(node, path, depth)) + const slot: LanguageSlot = texts.value.length === 0 ? { kind: 'attribute' } : languageSlot(nodeAttrs(node)['language']) const opener = spellBlockDirectiveOpener(node, model, path, slot.kind === 'attribute' ? [] : ['language']) if (opener === undefined) return commonMarkLine(carriedBlock(node, path, depth)) if (!opener.ok) return opener - const text = codeBlockText(node, path) - if (!text.ok) return text - return success(directivePair(node, opener.value, fencedCodeBlock(slot.kind === 'fence' ? slot.info : '', text.value))) + const info = slot.kind === 'fence' ? slot.info : '' + return success(directivePair(node, opener.value, texts.value.map((text) => fencedCodeBlock(info, text)).join('\n'))) } -function codeBlockText(node: AdfNode, path: ConvertErrorPath): Result { - let text = '' - for (const [index, child] of nodeContent(node).entries()) { +// The text each fence holds, or `undefined` where a child is no plain text node, which the carry holds instead. +function fencedTexts(node: AdfNode, path: ConvertErrorPath): Result { + if (node.content === undefined) return success(['']) + if (node.content.some((child) => !isUnmarkedBareText(child))) return success(undefined) + const texts: string[] = [] + for (const [index, child] of node.content.entries()) { const childPath = [...path, 'content', index] - if ( - child.type !== 'text' || - typeof child.text !== 'string' || - child.text === '' || - nodeContent(child).length > 0 || - nodeMarks(child).length > 0 || - Object.keys(nodeAttrs(child)).length > 0 - ) { - return failure('unsupported-node-shape', `a codeBlock holds plain text nodes only: this ${child.type} node is not one`, childPath) - } + if (typeof child.text !== 'string' || child.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', childPath) if (/\r/.test(child.text)) return failure('unspellable-character', 'a codeBlock holds no carriage return CommonMark keeps: this text holds one', childPath) if (holdsNullCharacter(child.text)) return failure('unspellable-character', 'a codeBlock holds a null character CommonMark replaces', childPath) - text += child.text + texts.push(child.text) } - return success(text) + return success(texts) } function tryHeading(node: AdfNode, path: ConvertErrorPath, flavour: Flavour): Result | undefined { - if (!carriesOnly(node, ['level'])) return undefined + if (!holdsOnlyAttributes(node, ['level'])) return undefined const level = nodeAttrs(node)['level'] if (typeof level !== 'number' || !Number.isInteger(level) || level < 1 || level > 6) return undefined const hashes = '#'.repeat(level) @@ -304,11 +304,11 @@ function tryHeading(node: AdfNode, path: ConvertErrorPath, flavour: Flavour): Re function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result | undefined { const ordered = node.type === 'orderedList' - if (!carriesOnly(node, ordered ? ['order'] : [])) return undefined + if (!holdsOnlyAttributes(node, ordered ? ['order'] : [])) return undefined const items = nodeContent(node) const start = listStart(node, items.length) if (start === undefined || items.length === 0) return undefined - if (items.some((item) => item.type !== 'listItem' || !carriesOnly(item, []))) return undefined + if (items.some((item) => item.type !== 'listItem' || !holdsOnlyAttributes(item, []))) return undefined const walked: WalkedItem[] = [] let headroom = Number.POSITIVE_INFINITY for (const [offset, item] of items.entries()) { @@ -340,7 +340,7 @@ function directiveItems(items: readonly WalkedItem[]): PlacedBlock[] { function listStart(node: AdfNode, items: number): number | undefined { if (node.type !== 'orderedList') return 0 const start = nodeAttrs(node)['order'] - if (typeof start !== 'number' || !Number.isInteger(start) || start < 0 || start > largestListMarker) return undefined + if (typeof start !== 'number' || !Number.isInteger(start) || start < 0 || Object.is(start, -0) || start > largestListMarker) return undefined return start + items - 1 > largestListMarker ? undefined : start } @@ -356,13 +356,13 @@ function tryListItemLines(inner: string, marker: string): string | undefined { function tryParagraph(node: AdfNode, path: ConvertErrorPath, flavour: Flavour): Result | undefined { const content = nodeContent(node) - if (content.length === 0 || !carriesOnly(node, [])) return undefined + if (content.length === 0 || !holdsOnlyAttributes(node, [])) return undefined const line = emitInlineLine(content, 'paragraph', path, flavour) if (!line.ok) return line return success(commonMarkText(line.value)) } function tryRule(node: AdfNode): string | undefined { - if (!carriesOnly(node, []) || nodeContent(node).length > 0) return undefined + if (!holdsOnlyAttributes(node, []) || nodeContent(node).length > 0) return undefined return '---' } diff --git a/src/markdown/emit/block-directive-spelling.ts b/src/markdown/emit/block-directive-spelling.ts index 4d4a964..9c42408 100644 --- a/src/markdown/emit/block-directive-spelling.ts +++ b/src/markdown/emit/block-directive-spelling.ts @@ -5,6 +5,7 @@ import { blockArgument, markValues, marksAttribute } from '../block-directive.ts import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { isBareToken, spellAttributes, spellDirectiveOpener, spellJsonAttribute, spellVocabulary } from '../directive-syntax.ts' import { overNested } from '../../json-value.ts' +import { spellEmptyKeys } from '../empty-keys.ts' import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts' export function spellBlockDirectiveOpener(node: AdfNode, model: BlockNodeModel, path: ConvertErrorPath, spelledByBody: readonly string[] = []): Result | undefined { @@ -14,7 +15,7 @@ export function spellBlockDirectiveOpener(node: AdfNode, model: BlockNodeModel, const spelled = argumentAttribute === undefined ? spelledByBody : [argumentAttribute, ...spelledByBody] const pairs = vocabularyPairs(nodeAttrs(node), model.attributes, spelled) if (pairs === undefined) return undefined - const spelledPairs = spellVocabulary(pairs) + const spelledPairs = [...spellVocabulary(pairs), ...spellEmptyKeys(node)] const marks = nodeMarks(node) if (marks.length > 0) { const values = markValues(marks) diff --git a/src/markdown/emit/image.ts b/src/markdown/emit/image.ts index be6828e..e9ca734 100644 --- a/src/markdown/emit/image.ts +++ b/src/markdown/emit/image.ts @@ -1,6 +1,6 @@ import type { AdfNode } from '../../adf/document.ts' import type { ConvertErrorPath } from '../../result.ts' -import { carriesOnly, nodeAttrs, nodeContent } from '../../adf/document.ts' +import { holdsOnlyAttributes, nodeAttrs, nodeContent } from '../../adf/document.ts' import { serializeCanonicalJson } from '../../canonical-json.ts' import { tryImageLine } from './inline-line.ts' @@ -16,8 +16,8 @@ export function tryImage(node: AdfNode, path: ConvertErrorPath): string | undefi function imageShape(node: AdfNode): { alt: string | undefined; url: string } | undefined { const content = nodeContent(node) const media = content[0] - if (!carriesOnly(node, ['layout']) || serializeCanonicalJson(nodeAttrs(node), 'compact') !== centeredMediaSingle) return undefined - if (media === undefined || content.length !== 1 || media.type !== 'media' || !carriesOnly(media, imageAttributes) || nodeContent(media).length > 0) return undefined + if (!holdsOnlyAttributes(node, ['layout']) || serializeCanonicalJson(nodeAttrs(node), 'compact') !== centeredMediaSingle) return undefined + if (media === undefined || content.length !== 1 || media.type !== 'media' || !holdsOnlyAttributes(media, imageAttributes) || nodeContent(media).length > 0) return undefined const attrs = nodeAttrs(media) const alt = attrs['alt'] const url = attrs['url'] diff --git a/src/markdown/emit/inline-directive-spelling.ts b/src/markdown/emit/inline-directive-spelling.ts index 0680214..69f8e76 100644 --- a/src/markdown/emit/inline-directive-spelling.ts +++ b/src/markdown/emit/inline-directive-spelling.ts @@ -2,9 +2,10 @@ import type { AdfNode } from '../../adf/document.ts' import type { InlineNodeModel } from '../../adf/inline-nodes.ts' import { nodeAttrs } from '../../adf/document.ts' import { spellAttributes, spellVocabulary } from '../directive-syntax.ts' +import { spellEmptyKeys } from '../empty-keys.ts' import { vocabularyPairs } from '../../adf/attribute-vocabulary.ts' export function spellInlineNodeAttributes(node: AdfNode, model: InlineNodeModel): string | undefined { const pairs = vocabularyPairs(nodeAttrs(node), model.attributes, model.textAttribute === undefined ? [] : [model.textAttribute]) - return pairs === undefined ? undefined : spellAttributes(spellVocabulary(pairs)) + return pairs === undefined ? undefined : spellAttributes([...spellVocabulary(pairs), ...spellEmptyKeys(node)]) } diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index 6e719e0..636b120 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -1,19 +1,19 @@ import type { AdfMark, AdfNode } from '../../adf/document.ts' -import type { Flavour } from '../plain-conventions.ts' +import type { Flavour } from '../plain/conventions.ts' import type { InlineNodeModel } from '../../adf/inline-nodes.ts' import type { LineContainer } from '../line-container.ts' import { assembleInlineLine, isSyntax, type InlineEscaping, type InlineSegment, type MarkRun, type NodeRange } from './line-escaping.ts' import { carriedInline } from '../opaque-carry.ts' import { claimsLine, holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts' import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts' +import { identicalMark, isBareText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { escapeUnbalanced, spellDestination } from '../commonmark/link-syntax.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' -import { highlightDelimiter } from '../plain-conventions.ts' +import { highlightDelimiter } from '../plain/conventions.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' +import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts' import { largestNesting } from '../../nesting.ts' import { longestBacktickRun } from '../commonmark/backtick-runs.ts' -import { nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' -import { sameMark } from '../../adf/editor-normal.ts' import { slotLineEndingFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts' import { spellInlineNodeAttributes } from './inline-directive-spelling.ts' import { spellTextDirective } from '../text-directive.ts' @@ -114,7 +114,7 @@ function lineSegments(nodes: readonly AdfNode[], container: LineContainer, path: const emission = emitRun(nodes, 0, 0, context) if (!emission.ok) return emission if (emission.value.carry !== undefined) return emission - return success({ segments: carryStrippedWhitespace(emission.value.segments) }) + return success({ segments: spellEdgeWhitespace(emission.value.segments) }) } function attemptLine(segments: readonly InlineSegment[], container: LineContainer, path: ConvertErrorPath, flavour: Flavour): Result { @@ -137,32 +137,32 @@ function lineVerdict(segments: readonly InlineSegment[], container: LineContaine } // spec/flavour.md, Inline nodes. -function carryStrippedWhitespace(segments: readonly InlineSegment[]): InlineSegment[] { - const carried: InlineSegment[] = [] +function spellEdgeWhitespace(segments: readonly InlineSegment[]): InlineSegment[] { + const spelled: InlineSegment[] = [] for (const [index, segment] of segments.entries()) { const previous = segments[index - 1] const next = segments[index + 1] const leading = previous === undefined || previous.text.includes('\n') const trailing = next === undefined || next.text.includes('\n') - carried.push(...carryEdges(segment, leading, trailing)) + spelled.push(...edgeWhitespaceSegments(segment, leading, trailing)) } - return carried + return spelled } -function carryEdges(segment: InlineSegment, leading: boolean, trailing: boolean): InlineSegment[] { +function edgeWhitespaceSegments(segment: InlineSegment, leading: boolean, trailing: boolean): InlineSegment[] { if (segment.escaping !== 'backslash' && segment.escaping !== 'bracketed') return [segment] const head = leading ? (/^[ \t]+/.exec(segment.text)?.[0] ?? '') : '' const body = segment.text.slice(head.length) const middle = trailing ? trimTrailingSpace(body) : body const tail = body.slice(middle.length) const edges: InlineSegment[] = [] - if (head !== '') edges.push(carriedText(head)) + if (head !== '') edges.push(textDirectiveSegment(head)) if (middle !== '') edges.push({ escaping: segment.escaping, text: middle }) - if (tail !== '') edges.push(carriedText(tail)) + if (tail !== '') edges.push(textDirectiveSegment(tail)) return edges } -function carriedText(text: string): InlineSegment { +function textDirectiveSegment(text: string): InlineSegment { return syntax(spellTextDirective(text)) } @@ -186,6 +186,7 @@ function emitRun(nodes: readonly AdfNode[], depth: number, firstIndex: number, c const runs = inlineRuns(nodes, depth, firstIndex, context.carried) const segments: InlineSegment[] = [] for (const [offset, run] of runs.entries()) { + if (takesTextBreak(runs[offset - 1], run, context.carried)) segments.push(syntax(textBreakSpelling)) const runContext = { ...context, atBlockEnd: context.atBlockEnd && offset === runs.length - 1 } const emitted = run.kind === 'plain' ? emitLeaf(run.node, runContext, run.index) : emitMarkedRun(run.nodes, run.mark, depth, run.index, runContext) if (!emitted.ok) return emitted @@ -206,12 +207,18 @@ function inlineRuns(nodes: readonly AdfNode[], depth: number, firstIndex: number continue } const previous = runs[runs.length - 1] - if (previous?.kind === 'marked' && sameMark(previous.mark, mark)) previous.nodes.push(node) + if (previous?.kind === 'marked' && identicalMark(previous.mark, mark)) previous.nodes.push(node) else runs.push({ index, kind: 'marked', mark, nodes: [node] }) } return runs } +function takesTextBreak(previous: InlineRun | undefined, run: InlineRun, carried: ReadonlySet): boolean { + if (previous?.kind !== 'plain' || run.kind !== 'plain') return false + if (carries(previous.node, carried, previous.index) || carries(run.node, carried, run.index)) return false + return joinsWhenRead(previous.node, run.node) +} + function nodePath(context: InlineContext, index: number): ConvertErrorPath { return [...context.path, 'content', index] } @@ -260,15 +267,20 @@ function emitInlineDirective(node: AdfNode, model: InlineNodeModel, index: numbe return success({ segments: [syntax(spellInlineDirectiveOpener(node.type)), ...content, syntax(`]${attributes}`)] }) } +function textHoldingContent(node: AdfNode, path: ConvertErrorPath): Result | undefined { + return node.type === 'text' && nodeContent(node).length > 0 ? failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) : undefined +} + function emitText(node: AdfNode, context: InlineContext, index: number, path: ConvertErrorPath): Result { - if (Object.keys(nodeAttrs(node)).length > 0) return success({ carry: { first: index, last: index } }) + const holding = textHoldingContent(node, path) + if (holding !== undefined) return holding + if (!isBareText(node)) return success({ carry: { first: index, last: index } }) if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) - if (nodeContent(node).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) if (/\r/.test(node.text)) return failure('unspellable-character', 'a text node holds a carriage return CommonMark rewrites', path) if (holdsNullCharacter(node.text)) return failure('unspellable-character', 'a text node holds a null character CommonMark replaces', path) const escaping: InlineEscaping = context.bracketed ? 'bracketed' : 'backslash' const parts = node.text.split(/(\n+)/).filter((part) => part !== '') - return success({ segments: parts.map((part) => (part.startsWith('\n') ? carriedText(part) : { escaping, text: part })) }) + return success({ segments: parts.map((part) => (part.startsWith('\n') ? textDirectiveSegment(part) : { escaping, text: part })) }) } function emitMarkedRun(nodes: readonly AdfNode[], mark: AdfMark, depth: number, index: number, context: InlineContext): Result { @@ -277,7 +289,7 @@ function emitMarkedRun(nodes: readonly AdfNode[], mark: AdfMark, depth: number, if (mark.type === 'backgroundColor' && context.flavour === 'plain') return emitHighlight(nodes, depth, range, context) const spelling = markSpelling(mark.type) if (spelling === undefined) return success({ carry: range }) - const attributes = spellMarkAttributes(mark, spelling.attributes) + const attributes = spellMarkAttributes(mark, spelling) if (attributes === undefined) return success({ carry: range }) if (spelling.kind === 'code') return emitCodeSpan(nodes, depth, range, path) if (spelling.kind === 'emphasis') return emitEmphasis(nodes, spelling.spelling, depth, range, context) @@ -293,11 +305,11 @@ function emitEmphasis(nodes: readonly AdfNode[], spelling: string, depth: number const inner = emitRun(nodes, depth + 1, range.first, context) if (!inner.ok) return inner if (inner.value.carry !== undefined) return inner - const carried = carryStrippedWhitespace(inner.value.segments) + const spelled = spellEdgeWhitespace(inner.value.segments) return success({ segments: [ { emphasis: 'open', escaping: 'none', nodes: { ...range, depth }, text: spelling }, - ...carried, + ...spelled, { emphasis: 'close', escaping: 'none', nodes: { ...range, depth }, text: spelling }, ], }) @@ -312,19 +324,21 @@ function emitHighlight(nodes: readonly AdfNode[], depth: number, range: NodeRang }) } +// Each node its own span: CommonMark reads two adjacent text nodes in one span back as one. function emitCodeSpan(nodes: readonly AdfNode[], depth: number, range: NodeRange, path: ConvertErrorPath): Result { - let text = '' + const spans: string[] = [] for (const node of nodes) { - if (node.type !== 'text' || nodeMarks(node).length !== depth + 1) return success({ carry: range }) - if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) - if (nodeContent(node).length > 0) return failure('unsupported-node-shape', 'a text node holds no content: this one holds some', path) - text += node.text + const holding = textHoldingContent(node, path) + if (holding !== undefined) return holding + if (!isBareText(node) || nodeMarks(node).length !== depth + 1) return success({ carry: range }) + const { text } = node + if (typeof text !== 'string' || text === '') return failure('unsupported-node-shape', 'a text node holds text: this one has none', path) + if (/[\n\r]/.test(text)) return success({ carry: range }) + if (holdsNullCharacter(text)) return failure('unspellable-character', 'a code span holds a null character CommonMark replaces', path) + const fence = '`'.repeat(longestBacktickRun(text) + 1) + spans.push(`${fence}${needsPadding(text) ? ` ${text} ` : text}${fence}`) } - if (/[\n\r]/.test(text)) return success({ carry: range }) - if (holdsNullCharacter(text)) return failure('unspellable-character', 'a code span holds a null character CommonMark replaces', path) - const fence = '`'.repeat(longestBacktickRun(text) + 1) - const padded = needsPadding(text) ? ` ${text} ` : text - return success({ segments: [syntax(`${fence}${padded}${fence}`)] }) + return success({ segments: [syntax(spans.join(textBreakSpelling))] }) } function needsPadding(text: string): boolean { diff --git a/src/markdown/emit/line-escaping.ts b/src/markdown/emit/line-escaping.ts index 253df18..120faf8 100644 --- a/src/markdown/emit/line-escaping.ts +++ b/src/markdown/emit/line-escaping.ts @@ -1,10 +1,10 @@ -import type { Flavour } from '../plain-conventions.ts' +import type { Flavour } from '../plain/conventions.ts' import type { LineContainer } from '../line-container.ts' import { backslashEscape, escapesLineClaim, inlineHtmlConstruct, opensBracketedAutolink, opensEmailAutolink, type LinePosition } from '../commonmark/grammar.ts' import { backtickRun, closingBacktickRun } from '../commonmark/backtick-runs.ts' import { claimsDirectivePrefix } from '../directive-syntax.ts' import { delimiterFlags, isWordCharacter, matchEmphasis, runLength } from '../commonmark/emphasis-matching.ts' -import { highlightDelimiter, highlightFlanking } from '../plain-conventions.ts' +import { highlightDelimiter, highlightFlanking } from '../plain/conventions.ts' import { isBareDelimiterRow } from '../pipe-table-syntax.ts' import { opensLinkDefinition } from '../commonmark/link-reference-definitions.ts' import { readEntityReference } from '../commonmark/entity-references.ts' diff --git a/src/markdown/emit/pipe-table.ts b/src/markdown/emit/pipe-table.ts index f200e6d..0998cf2 100644 --- a/src/markdown/emit/pipe-table.ts +++ b/src/markdown/emit/pipe-table.ts @@ -1,7 +1,7 @@ import type { AdfNode } from '../../adf/document.ts' import type { ConvertErrorPath } from '../../result.ts' -import type { Flavour } from '../plain-conventions.ts' -import { carriesOnly, nodeContent } from '../../adf/document.ts' +import type { Flavour } from '../plain/conventions.ts' +import { holdsOnlyAttributes, nodeContent } from '../../adf/document.ts' import { spellPipeDelimiter, spellPipeRow } from '../pipe-table-syntax.ts' import { tryPipeCell } from './inline-line.ts' @@ -26,16 +26,16 @@ export function tryPipeTable(node: AdfNode, path: ConvertErrorPath, flavour: Fla function pipeRows(node: AdfNode): AdfNode[][] | undefined { const rows = nodeContent(node) const columns = rows[0] === undefined ? 0 : nodeContent(rows[0]).length - if (!carriesOnly(node, []) || columns === 0) return undefined + if (!holdsOnlyAttributes(node, []) || columns === 0) return undefined const grid: AdfNode[][] = [] for (const [index, row] of rows.entries()) { const cells = nodeContent(row) - if (row.type !== 'tableRow' || !carriesOnly(row, []) || cells.length !== columns) return undefined + if (row.type !== 'tableRow' || !holdsOnlyAttributes(row, []) || cells.length !== columns) return undefined const wanted = index === 0 ? 'tableHeader' : 'tableCell' const paragraphs: AdfNode[] = [] for (const cell of cells) { const paragraph = plainParagraph(cell) - if (paragraph === undefined || cell.type !== wanted || !carriesOnly(cell, [])) return undefined + if (paragraph === undefined || cell.type !== wanted || !holdsOnlyAttributes(cell, [])) return undefined paragraphs.push(paragraph) } grid.push(paragraphs) @@ -46,6 +46,6 @@ function pipeRows(node: AdfNode): AdfNode[][] | undefined { function plainParagraph(cell: AdfNode): AdfNode | undefined { const content = nodeContent(cell) const paragraph = content[0] - if (paragraph === undefined || content.length !== 1 || paragraph.type !== 'paragraph' || !carriesOnly(paragraph, [])) return undefined + if (paragraph === undefined || content.length !== 1 || paragraph.type !== 'paragraph' || !holdsOnlyAttributes(paragraph, [])) return undefined return paragraph } diff --git a/src/markdown/empty-keys.ts b/src/markdown/empty-keys.ts new file mode 100644 index 0000000..5aba8e7 --- /dev/null +++ b/src/markdown/empty-keys.ts @@ -0,0 +1,32 @@ +import type { DirectiveAttributes, DirectiveValue, Read } from './directive-syntax.ts' +import type { EmptyKey } from '../adf/document.ts' +import { emptyKeys } from '../adf/document.ts' +import { unsupportedNodeShape } from './directive-syntax.ts' + +type EmptyKeysRead = { empty: ReadonlySet; rest: Map } + +const emptyValue = 'empty' + +// spec/flavour.md, Attributes: no attribute value spells an empty object or array. +export function spellEmptyKeys(held: Parameters[0]): [string, string][] { + return emptyKeys(held).map((key) => [key, emptyValue]) +} + +export function spellsEmpty(value: DirectiveValue | undefined): boolean { + return value?.spelling === emptyValue +} + +// `held` is whether the directive spells an attribute outside {attrs}: an argument or a content slot. +export function readEmptyKeys(type: string, attributes: DirectiveAttributes, keys: readonly EmptyKey[], held: boolean): Read { + const rest = new Map(attributes) + const empty = new Set() + for (const key of keys) { + const spelled = rest.get(key) + if (spelled === undefined) continue + if (!spellsEmpty(spelled)) return { fault: unsupportedNodeShape(`the reserved key ${key} takes only the value ${emptyValue}: this one spells ${key}=${spelled.spelling}`) } + empty.add(key) + rest.delete(key) + } + if (empty.has('attrs') && (held || [...rest.keys()].some((key) => key !== 'marks'))) return { fault: unsupportedNodeShape(`${type} spells attrs=empty beside another attribute, an argument or a [content] slot: drop attrs=empty, or the rest`) } + return { value: { empty, rest } } +} diff --git a/src/markdown/mark-spellings.ts b/src/markdown/mark-spellings.ts index a885262..d9a814c 100644 --- a/src/markdown/mark-spellings.ts +++ b/src/markdown/mark-spellings.ts @@ -7,6 +7,7 @@ import { holdsEntityReference } from './commonmark/entity-references.ts' import { isAutolink } from './commonmark/grammar.ts' import { isMarkType, markAttributes } from '../adf/mark-attributes.ts' import { nodeAttrs, nodeMarks } from '../adf/document.ts' +import { spellEmptyKeys } from './empty-keys.ts' import { vocabularyPairs } from '../adf/attribute-vocabulary.ts' type Spelling = { kind: 'code' | 'directive' | 'link'; spelling?: undefined } | { kind: 'emphasis'; spelling: string } @@ -55,7 +56,10 @@ function isBareLink(nodes: readonly AdfNode[], href: string, marksInside: number return nodes.length === 1 && node !== undefined && node.type === 'text' && node.text === href && nodeMarks(node).length === marksInside } -export function spellMarkAttributes(mark: AdfMark, vocabulary: AttributeVocabulary): string | undefined { - const pairs = vocabularyPairs(nodeAttrs(mark), vocabulary, []) - return pairs === undefined ? undefined : spellAttributes(spellVocabulary(pairs)) +// `undefined` where the spelling holds no such attrs: only a directive spells attrs=empty. +export function spellMarkAttributes(mark: AdfMark, spelling: MarkSpelling): string | undefined { + const pairs = vocabularyPairs(nodeAttrs(mark), spelling.attributes, []) + const empty = spellEmptyKeys(mark) + if (pairs === undefined || (empty.length > 0 && spelling.kind !== 'directive')) return undefined + return spellAttributes([...spellVocabulary(pairs), ...empty]) } diff --git a/src/markdown/opaque-carry.ts b/src/markdown/opaque-carry.ts index 0e03467..3941cf3 100644 --- a/src/markdown/opaque-carry.ts +++ b/src/markdown/opaque-carry.ts @@ -1,32 +1,49 @@ import type { AdfNode } from '../adf/document.ts' import type { DirectiveSpan, Read } from './directive-syntax.ts' import type { JsonSpelling } from '../canonical-json.ts' +import type { JsonValue } from '../json-value.ts' import { failure, success, type ConvertErrorPath, type Result } from '../result.ts' import { fencedCodeBlock } from './commonmark/backtick-runs.ts' +import { infoStringCarries } from './commonmark/grammar.ts' import { isAdfNode } from '../adf/document.ts' import { isJsonValue, nestingDepth, overNested } from '../json-value.ts' import { largestNesting } from '../nesting.ts' -import { malformedDirective, readSoleStringAttribute, spellAttributes, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts' +import { directiveEscape, malformedDirective, readSoleStringAttribute, spellAttributes, spellDirectiveOpener, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive-syntax.ts' import { serializeCanonicalJson } from '../canonical-json.ts' +export const carryFencePrefix = 'adf:' + export const carryName = 'carry' const jsonAttribute = 'json' +// spec/flavour.md, The opaque carry: the info string names the type wherever it carries the type back. export function carriedBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result<{ headroom: number; text: string }> { - const json = carriedJson(node, 'two-space', path, largestNesting - depth) + const { type, ...untyped } = node + const named = infoStringCarries(type) + const json = carriedJson(node, named ? untyped : node, 'two-space', path, largestNesting - depth) if (!json.ok) return json - return success({ headroom: json.value.headroom, text: fencedCodeBlock(carryName, json.value.json) }) + return success({ headroom: json.value.headroom, text: fencedCodeBlock(`${carryFencePrefix}${named ? type : ''}`, json.value.json) }) } export function carriedInline(node: AdfNode, path: ConvertErrorPath): Result { - const json = carriedJson(node, 'compact', path, largestNesting) + const json = carriedJson(node, node, 'compact', path, largestNesting) if (!json.ok) return json return success(spellInlineLeafDirective(carryName, spellAttributes([[jsonAttribute, spellStringAttribute(json.value.json)]]))) } -export function readCarriedBlock(body: string, depth: number): Read { - return readCarriedJson(body, 'two-space', largestNesting - depth) +// The type a fence's info string names, `undefined` where the info string opens no carry. +export function carryFenceType(info: string): string | undefined { + return info.startsWith(carryFencePrefix) ? info.slice(carryFencePrefix.length) : undefined +} + +export function readCarriedBlock(type: string, body: string, depth: number): Read { + if (type !== '' && !infoStringCarries(type)) { + return { fault: unsupportedNodeShape(`the carry fence names a type no info string carries back: spell it ${carryFencePrefix} with the type in the JSON; ${fenceEscape(type)}`) } + } + const read = readCarriedJson(body, 'two-space', largestNesting - depth, type) + if (read.fault !== undefined || type !== '' || !infoStringCarries(read.value.type)) return read + return { fault: unsupportedNodeShape(`the ${carryFencePrefix} fence holds a type its info string can carry: spell the fence ${carryFencePrefix}${read.value.type} and drop type from the JSON`) } } export function readCarriedInline(span: DirectiveSpan): Read | undefined { @@ -36,28 +53,44 @@ export function readCarriedInline(span: DirectiveSpan): Read | undefine return readCarriedJson(spelled.value, 'compact', largestNesting) } -function carriedJson(node: AdfNode, spelling: JsonSpelling, path: ConvertErrorPath, levels: number): Result<{ headroom: number; json: string }> { +function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, path: ConvertErrorPath, levels: number): Result<{ headroom: number; json: string }> { const headroom = levels - nestingDepth(node) - if (!isJsonValue(node) || headroom < 0) { + if (!isJsonValue(spelled) || headroom < 0) { return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path) } - return success({ headroom, json: serializeCanonicalJson(node, spelling) }) + return success({ headroom, json: serializeCanonicalJson(spelled, spelling) }) } -function readCarriedJson(raw: string, spelling: JsonSpelling, levels: number): Read { +// `type` is the one the fence's info string names, `undefined` for the inline carry. +function readCarriedJson(raw: string, spelling: JsonSpelling, levels: number, type?: string): Read { + const wayOut = type === undefined ? directiveEscape : fenceEscape(type) const parsed = parseJsonText(raw) - if (parsed === undefined) return { fault: malformedDirective('the opaque carry holds invalid JSON') } + if (parsed === undefined) return { fault: malformedDirective(`the opaque carry holds invalid JSON; ${wayOut}`) } const { value } = parsed if (!isJsonValue(value)) return { fault: unsupportedNodeShape('the opaque carry holds a number JSON cannot spell') } - if (overNested(value, levels)) { + const typed = type === undefined || type === '' ? { value } : typedValue(value, type) + if (typed.fault !== undefined) return typed + const held = typed.value + if (overNested(held, levels)) { return { fault: { code: 'unsupported-nesting-depth', message: `a carried node's JSON nests deeper than the ${levels} levels its position leaves` } } } if (serializeCanonicalJson(value, spelling) !== raw) { const shape = spelling === 'compact' ? 'compact, keys sorted' : 'two-space indent, keys sorted' - return { fault: unsupportedNodeShape(`the opaque carry spells its node's JSON canonically: ${shape}`) } + return { fault: unsupportedNodeShape(`the opaque carry spells its node's JSON canonically: ${shape}; ${wayOut}`) } } - if (!isAdfNode(value)) return { fault: unsupportedNodeShape("the opaque carry holds one ADF node's JSON: this JSON is no ADF node") } - return { value } + if (!isAdfNode(held)) return { fault: unsupportedNodeShape(`the opaque carry holds one ADF node's JSON: this JSON is no ADF node; ${wayOut}`) } + return { value: held } +} + +// A code fence the carry claims stays code under the directive, whose language rides the attribute. +function fenceEscape(type: string): string { + return `${spellDirectiveOpener('codeBlock', undefined, spellAttributes([['language', spellStringAttribute(`${carryFencePrefix}${type}`)]]))} around a bare fence keeps it a code block` +} + +function typedValue(value: JsonValue, type: string): Read { + if (value === null || typeof value !== 'object' || Array.isArray(value)) return { value } + if ('type' in value) return { fault: unsupportedNodeShape(`the ${carryFencePrefix}${type} fence names its node's type: this JSON holds a type as well; ${fenceEscape(type)}`) } + return { value: { ...value, type } } } function parseJsonText(raw: string): { value: unknown } | undefined { diff --git a/src/markdown/parse/directive-marks.ts b/src/markdown/parse/directive-marks.ts index f1b7df6..22e5c7a 100644 --- a/src/markdown/parse/directive-marks.ts +++ b/src/markdown/parse/directive-marks.ts @@ -3,8 +3,9 @@ import type { ConvertFault } from '../../result.ts' import type { DirectiveAttributes } from '../directive-syntax.ts' import type { MarkSpelling } from '../mark-spellings.ts' import { directivePrefix } from '../directive-syntax.ts' -import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' +import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { markSpelling } from '../mark-spellings.ts' +import { readEmptyKeys } from '../empty-keys.ts' import { readVocabulary } from './directive-attributes.ts' export function readDirectiveMark(name: string, attributes: DirectiveAttributes, path: ConvertErrorPath): Result | undefined { @@ -12,8 +13,11 @@ export function readDirectiveMark(name: string, attributes: DirectiveAttributes, if (spelling === undefined) return undefined const markdown = markdownForm(spelling) if (markdown !== undefined) return failure('unsupported-node-shape', `${name} is spelled ${markdown}, never as a directive`, path) - const attrs = readVocabulary(name, attributes, spelling.attributes, undefined, path) + const empty = readEmptyKeys(name, attributes, ['attrs'], false) + if (empty.fault !== undefined) return faulted(empty.fault, path) + const attrs = readVocabulary(name, empty.value.rest, spelling.attributes, undefined, path) if (!attrs.ok) return attrs + if (empty.value.empty.has('attrs')) return success({ attrs: {}, type: name }) return success(Object.keys(attrs.value).length === 0 ? { type: name } : { attrs: attrs.value, type: name }) } diff --git a/src/markdown/parse/directive-nodes.ts b/src/markdown/parse/directive-nodes.ts index 7235b69..19afd52 100644 --- a/src/markdown/parse/directive-nodes.ts +++ b/src/markdown/parse/directive-nodes.ts @@ -1,18 +1,20 @@ -import type { AdfAttributes, AdfMark, AdfNode } from '../../adf/document.ts' +import type { AdfAttributes, AdfMark, AdfNode, EmptyKey } from '../../adf/document.ts' import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { ConvertFault } from '../../result.ts' import type { DirectiveAttributes, DirectiveValue } from '../directive-syntax.ts' import type { Elsewhere } from './directive-attributes.ts' -import { attributeNestingMessage, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' +import { attributeNestingMessage, isUnmarkedBareText } from '../../adf/document.ts' import { attributeValue, directivePrefix, spellAttributeValue, unknownDirectiveFault } from '../directive-syntax.ts' import { blockArgument, blockDirectiveForm, marksAttribute, readMarkValues } from '../block-directive.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' -import { carryName } from '../opaque-carry.ts' +import { carryFencePrefix, carryName } from '../opaque-carry.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { inlineMarkSpellingFault } from './directive-marks.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' +import { readEmptyKeys, spellsEmpty } from '../empty-keys.ts' import { readVocabulary } from './directive-attributes.ts' import { slotLineEndingFault } from '../directive-syntax.ts' +import { textBreakName } from '../adjacent-text.ts' import { textDirectiveName } from '../text-directive.ts' export type BlockDirectiveNode = { contentModel: BlockNodeModel['contentModel']; node: AdfNode } @@ -24,12 +26,15 @@ export function readBlockDirectiveNode( path: ConvertErrorPath, ): Result { if (name === carryName) { - return failure('malformed-directive', `the name ${carryName} is reserved for the opaque carry, whose block form is the ${carryName} fence`, path) + return failure('malformed-directive', `the name ${carryName} is reserved for the opaque carry, whose block form is the ${carryFencePrefix} fence`, path) } const model = blockNodeModel(name) if (model === undefined) return faulted(inlineSpellingFault(name) ?? unknownDirectiveFault(name), path) const argumentKey = blockArgument(name) - const rest = new Map(attributes) + const spelled = attributes.get(marksAttribute) + const empty = readEmptyKeys(name, attributes, spellsEmpty(spelled) ? ['attrs', 'content', 'marks'] : ['attrs', 'content'], argument !== undefined) + if (empty.fault !== undefined) return faulted(empty.fault, path) + const { rest } = empty.value rest.delete(marksAttribute) const elsewhere: Elsewhere | undefined = argumentKey === undefined ? undefined : { key: argumentKey, slot: 'argument' } const attrs = readVocabulary(name, rest, model.attributes, elsewhere, path) @@ -38,10 +43,9 @@ export function readBlockDirectiveNode( if (argumentKey === undefined) return failure('unsupported-node-shape', `${name} takes no argument: this one spells one`, path) attrs.value[argumentKey] = argument } - const spelled = attributes.get(marksAttribute) - const marks: Result = spelled === undefined ? success(undefined) : readMarks(name, spelled, path) + const marks: Result = spelled === undefined || spellsEmpty(spelled) ? success(undefined) : readMarks(name, spelled, path) if (!marks.ok) return marks - return success({ contentModel: model.contentModel, node: namedNode(name, attrs.value, marks.value) }) + return success({ contentModel: model.contentModel, node: namedNode(name, attrs.value, marks.value, empty.value.empty) }) } export function readInlineDirectiveNode( @@ -55,7 +59,9 @@ export function readInlineDirectiveNode( const slot = model.textAttribute if (slot === undefined && content !== undefined) return failure('unsupported-node-shape', `${name} takes no content: this one holds some`, path) const elsewhere: Elsewhere | undefined = slot === undefined ? undefined : { key: slot, slot: 'content' } - const attrs = readVocabulary(name, attributes, model.attributes, elsewhere, path) + const empty = readEmptyKeys(name, attributes, ['attrs', 'content', 'marks'], slot !== undefined && content !== undefined) + if (empty.fault !== undefined) return faulted(empty.fault, path) + const attrs = readVocabulary(name, empty.value.rest, model.attributes, elsewhere, path) if (!attrs.ok) return attrs if (slot !== undefined && content !== undefined) { const text = slotText(content) @@ -66,14 +72,14 @@ export function readInlineDirectiveNode( if (spans !== undefined) return faulted(spans, path) attrs.value[slot] = text } - return success(namedNode(name, attrs.value, undefined)) + return success(namedNode(name, attrs.value, undefined, empty.value.empty)) } // A name the other position spells names that spelling, never the code a later MINOR may fill (docs/decisions.md §Which code a cause takes). function inlineSpellingFault(name: string): ConvertFault | undefined { const mark = inlineMarkSpellingFault(name) if (mark !== undefined) return mark - if (inlineNodeModel(name) === undefined && name !== textDirectiveName) return undefined + if (inlineNodeModel(name) === undefined && name !== textDirectiveName && name !== textBreakName) return undefined return { code: 'unsupported-node-shape', message: `${name} takes the inline form, ${directivePrefix}${name}{…}, never the block form` } } @@ -86,7 +92,7 @@ function blockSpellingFault(name: string): ConvertFault | undefined { function slotText(content: readonly AdfNode[]): string | undefined { if (content.length === 0) return '' const only = content.length === 1 ? content[0] : undefined - if (only?.type !== 'text' || nodeMarks(only).length > 0 || Object.keys(nodeAttrs(only)).length > 0 || nodeContent(only).length > 0 || typeof only.text !== 'string') return undefined + if (only === undefined || !isUnmarkedBareText(only) || typeof only.text !== 'string') return undefined return only.text } @@ -100,7 +106,9 @@ function readMarks(type: string, spelled: DirectiveValue, path: ConvertErrorPath return success(marks) } -function namedNode(type: string, attrs: AdfAttributes, marks: readonly AdfMark[] | undefined): AdfNode { - const named = Object.keys(attrs).length === 0 ? { type } : { attrs, type } - return marks === undefined ? named : { ...named, marks: [...marks] } +function namedNode(type: string, attrs: AdfAttributes, marks: readonly AdfMark[] | undefined, empty: ReadonlySet): AdfNode { + const node: AdfNode = Object.keys(attrs).length > 0 || empty.has('attrs') ? { attrs, type } : { type } + if (empty.has('content')) node.content = [] + if (marks !== undefined || empty.has('marks')) node.marks = [...(marks ?? [])] + return node } diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index 192fd1e..6ca3bba 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -1,7 +1,7 @@ import type { AdfMark, AdfNode } from '../../adf/document.ts' import type { DirectiveSpan, NestedSpans } from '../directive-syntax.ts' import type { EmphasisPairing } from '../commonmark/emphasis-matching.ts' -import type { Flavour } from '../plain-conventions.ts' +import type { Flavour } from '../plain/conventions.ts' import type { LineContainer } from '../line-container.ts' import type { LinkDefinition } from '../commonmark/link-syntax.ts' import { backslashEscape, decodeTextEscapes, inlineHtmlConstruct, readBracketedAutolink, readEmailAutolink, trimTrailingSpace } from '../commonmark/grammar.ts' @@ -9,11 +9,11 @@ import { backtickRun, closingBacktickRun } from '../commonmark/backtick-runs.ts' import { commonMarkLink, linkHref } from '../mark-spellings.ts' import { delimiterFlags, matchEmphasis, runLength } from '../commonmark/emphasis-matching.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' -import { highlightDelimiter, highlightFlanking } from '../plain-conventions.ts' +import { highlightDelimiter, highlightFlanking } from '../plain/conventions.ts' +import { identicalMarks, mergeAdjacentText, nodeAttrs, nodeMarks } from '../../adf/document.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' -import { mergeAdjacentText, sameMarks } from '../../adf/editor-normal.ts' +import { joinsWhenRead, textBreakName, textBreakSpelling } from '../adjacent-text.ts' import { noSpans, readInlineDirective } from '../directive-syntax.ts' -import { nodeAttrs, nodeMarks } from '../../adf/document.ts' import { normalizeLabel, readInlineTarget, readLabel } from '../commonmark/link-syntax.ts' import { openingLinkTakesDirective } from '../emit/inline-line.ts' import { readCarriedInline } from '../opaque-carry.ts' @@ -21,7 +21,17 @@ import { readDirectiveMark } from './directive-marks.ts' import { readInlineDirectiveNode } from './directive-nodes.ts' import { readTextDirective } from '../text-directive.ts' -export type InlineContent = { carry?: undefined; image: AdfNode; nodes?: undefined } | { carry: boolean; image?: undefined; nodes: AdfNode[] } +export type InlineContent = { image: AdfNode; nodes?: undefined } | { image?: undefined; nodes: AdfNode[] } + +// The text break leaf: it holds no marks, and stands between the nodes it parts until the outermost content checks and drops it. +type TextBreak = { kind: 'textBreak' } + +// A node whose marks are its own — an opaque carry, or an inline node spelling marks=empty: no mark spelling wraps it, and it joins no neighbour. +type OwnMarks = { kind: 'ownMarks'; node: AdfNode } + +type Inline = AdfNode | OwnMarks | TextBreak + +type Scanned = { image: AdfNode; nodes?: undefined } | { image?: undefined; nodes: Inline[] } export type LinkDefinitions = ReadonlyMap @@ -33,9 +43,10 @@ type Pairing = EmphasisPairing type Piece = | Bracket - | { kind: 'carry'; node: AdfNode } + | OwnMarks | { alt: string; kind: 'image'; node: AdfNode } - | { kind: 'nodes'; nodes: AdfNode[] } + | { kind: 'nodes'; nodes: Inline[] } + | TextBreak | { canClose: boolean; canOpen: boolean; character: string; kind: 'run'; length: number } | { closes: boolean; kind: 'highlight'; opens: boolean } @@ -56,21 +67,41 @@ type Scan = { spans: NestedSpans } -type SlotContent = { carry: boolean; nodes: AdfNode[] } +// What a directive's content slot inherits from the content holding it. +type SharedScan = Pick -const carriedInMark = 'no mark spelling wraps an opaque carry: the carried node restores exactly, marks included' +type SlotContent = { nodes: Inline[] } + +const ownMarksInMark = 'move the opaque carry, or the node spelling marks=empty, out of the mark spelling: it holds its own marks' const editorHighlight: AdfMark = { attrs: { color: '#f8e6a0' }, type: 'backgroundColor' } const hreflessLink = 'the link mark spells its href: this one spells none' const imageAlone = 'an image fits only as a paragraph of its own: this one sits inside other content' const linkInLink = 'no link wraps a link: the [content] this one marks already holds one' const spellableLink = 'link takes the directive form only where CommonMark cannot spell it: this one it can, as [text](url "title") or ' +const textBreak: TextBreak = { kind: 'textBreak' } +// The outermost content: only here does a text break see both its neighbours. export function parseInlineContent(source: string, definitions: LinkDefinitions, path: ConvertErrorPath, container: LineContainer, flavour: Flavour): Result { - return parseInline(source, definitions, path, container, noSpans, flavour === 'plain') + const scan = freshScan(source, { definitions, path }, { container, highlights: flavour === 'plain', spans: noSpans }) + const scanned = scanInline(scan) + if (!scanned.ok) return scanned + if (scanned.value.image !== undefined) return success({ image: scanned.value.image }) + const nodes = partText(scanned.value.nodes, scan) + if (!nodes.ok) return nodes + if (scan.openingSpellableLink) { + const takesDirective = openingLinkTakesDirective(nodes.value, path) + if (!takesDirective.ok) return takesDirective + if (!takesDirective.value) return failure('unsupported-node-shape', spellableLink, path) + } + return success({ nodes: nodes.value }) } -function parseInline(source: string, definitions: LinkDefinitions, path: ConvertErrorPath, container: LineContainer | undefined, spans: NestedSpans, highlights: boolean): Result { - const scan: Scan = { container, deactivatedBefore: 0, definitions, highlights, openingSpellableLink: false, path, pending: '', pieces: [], source, spans } +function freshScan(source: string, shared: SharedScan, own: Pick): Scan { + return { ...shared, ...own, deactivatedBefore: 0, openingSpellableLink: false, pending: '', pieces: [], source } +} + +function scanInline(scan: Scan): Result { + const { source } = scan let index = 0 while (index < source.length) { switch (source.charAt(index)) { @@ -103,7 +134,7 @@ function parseInline(source: string, definitions: LinkDefinitions, path: Convert index = readCharacter(scan, index) } } - flush(scan, container !== undefined) + flush(scan, scan.container !== undefined) return assemble(scan) } @@ -205,8 +236,9 @@ function directivePiece(scan: Scan, span: DirectiveSpan, index: number): Result< const carried = readCarriedInline(span) if (carried !== undefined) { if (carried.fault !== undefined) return faulted(carried.fault, scan.path) - return success({ kind: 'carry', node: carried.value }) + return success({ kind: 'ownMarks', node: carried.value }) } + if (span.name === textBreakName) return span.content === undefined && span.attributes.size === 0 ? success(textBreak) : failure('unsupported-node-shape', `${textBreakName} spells the bare leaf form, ${textBreakSpelling}: this one spells more`, scan.path) const text = readTextDirective(span) if (text?.fault !== undefined) return faulted(text.fault, scan.path) if (text !== undefined) return success({ kind: 'nodes', nodes: [{ text: text.value, type: 'text' }] }) @@ -214,27 +246,29 @@ function directivePiece(scan: Scan, span: DirectiveSpan, index: number): Result< if (!slot.ok) return slot const mark = readDirectiveMark(span.name, span.attributes, scan.path) if (mark !== undefined) return mark.ok ? directiveMarkPiece(scan, span.name, mark.value, slot.value, index) : mark - const node = readInlineDirectiveNode(span.name, span.attributes, slot.value?.nodes, scan.path) + const content = slot.value?.nodes + if (content?.some(isTextBreak) === true) return failure('unsupported-node-shape', `delete ${textBreakSpelling} from this content slot: a slot holds one text node`, scan.path) + const node = readInlineDirectiveNode(span.name, span.attributes, content === undefined ? undefined : adfNodes(content), scan.path) if (!node.ok) return node - return success({ kind: 'nodes', nodes: [node.value] }) + return success(node.value.marks === undefined ? { kind: 'nodes', nodes: [node.value] } : { kind: 'ownMarks', node: node.value }) } function directiveMarkPiece(scan: Scan, name: string, mark: AdfMark, slot: SlotContent | undefined, index: number): Result { if (slot === undefined || slot.nodes.length === 0) { return failure('unsupported-node-shape', `the ${name} mark wraps the [content] it marks: this one wraps none`, scan.path) } - if (slot.carry) return failure('unsupported-node-shape', carriedInMark, scan.path) + if (slot.nodes.some(holdsOwnMarks)) return failure('unsupported-node-shape', ownMarksInMark, scan.path) const refused = mark.type === 'link' ? refuseLinkDirective(scan, mark, slot.nodes, index) : undefined if (refused !== undefined) return refused return success({ kind: 'nodes', nodes: applyMark(slot.nodes, mark) }) } // spec/flavour.md, Marks. A link opening a paragraph may still need the directive form for the line it opens, which `assemble` asks the emitter. -function refuseLinkDirective(scan: Scan, mark: AdfMark, nodes: readonly AdfNode[], index: number): Result | undefined { +function refuseLinkDirective(scan: Scan, mark: AdfMark, nodes: readonly Inline[], index: number): Result | undefined { const href = linkHref(nodeAttrs(mark)) if (href === undefined) return failure('unsupported-node-shape', hreflessLink, scan.path) if (marksLink(nodes)) return failure('unsupported-node-shape', linkInLink, scan.path) - if (commonMarkLink(nodeAttrs(mark), href, nodes, 0, scan.container === undefined) === undefined) return undefined + if (commonMarkLink(nodeAttrs(mark), href, adfNodes(nodes), 0, scan.container === undefined) === undefined) return undefined if (index !== 0 || scan.container !== 'paragraph') return failure('unsupported-node-shape', spellableLink, scan.path) scan.openingSpellableLink = true return undefined @@ -242,7 +276,8 @@ function refuseLinkDirective(scan: Scan, mark: AdfMark, nodes: readonly AdfNode[ function slotContent(scan: Scan, span: DirectiveSpan): Result { if (span.content === undefined) return success(undefined) - const parsed = parseInline(span.content, scan.definitions, scan.path, undefined, span.spans, false) + const { definitions, path } = scan + const parsed = scanInline(freshScan(span.content, { definitions, path }, { container: undefined, highlights: false, spans: span.spans })) if (!parsed.ok) return parsed if (parsed.value.image !== undefined) return failure('unmappable-image', imageAlone, scan.path) return success(parsed.value) @@ -258,35 +293,65 @@ function pushNode(scan: Scan, node: AdfNode): void { scan.pieces.push({ kind: 'nodes', nodes: [node] }) } -function assemble(scan: Scan): Result { +function assemble(scan: Scan): Result { const only = scan.pieces[0] if (scan.pieces.length === 1 && only?.kind === 'image') return success({ image: only.node }) if (holdsImage(scan.pieces)) return failure('unmappable-image', imageAlone, scan.path) - const nodes = resolveNodes(scan.pieces, scan.path, true) + const nodes = resolveNodes(scan.pieces, scan, true) if (!nodes.ok) return nodes - if (scan.openingSpellableLink) { - const takesDirective = openingLinkTakesDirective(nodes.value, scan.path) - if (!takesDirective.ok) return takesDirective - if (!takesDirective.value) return failure('unsupported-node-shape', spellableLink, scan.path) - } - return success({ carry: holdsCarry(scan.pieces), nodes: nodes.value }) + return success({ nodes: nodes.value }) } -function holdsCarry(pieces: readonly Piece[]): boolean { - return pieces.some((piece) => piece.kind === 'carry') +// spec/flavour.md, Inline nodes: the leaf builds no node, so only the pair it parts spells it. +function partText(items: readonly Inline[], scan: Scan): Result { + const parted: AdfNode[] = [] + for (const [index, item] of items.entries()) { + if (!isTextBreak(item)) { + parted.push(isNode(item) ? item : item.node) + continue + } + const previous = items[index - 1] + const next = items[index + 1] + if (previous === undefined || next === undefined || !isNode(previous) || !isNode(next) || !joinsWhenRead(previous, next)) { + return failure('unsupported-node-shape', `delete ${textBreakSpelling} here: it stands only between two runs of text with the same formatting, which would otherwise read as one`, scan.path) + } + } + return success(parted) +} + +function isNode(item: Inline): item is AdfNode { + return !('kind' in item) +} + +function isTextBreak(item: Inline): item is TextBreak { + return 'kind' in item && item.kind === 'textBreak' +} + +// The nodes the items hold, with text breaks dropped. +function adfNodes(items: readonly Inline[]): AdfNode[] { + const nodes: AdfNode[] = [] + for (const item of items) { + if (isNode(item)) nodes.push(item) + else if (!isTextBreak(item)) nodes.push(item.node) + } + return nodes +} + +function holdsOwnMarks(item: Inline | Piece): boolean { + return 'kind' in item && item.kind === 'ownMarks' } function holdsImage(pieces: readonly Piece[]): boolean { return pieces.some((piece) => piece.kind === 'image') } -// A carry rides its own piece: the carried node's marks restore with it rather than riding a spelling, so the guard below answers for it. +// A node whose marks are its own rides its own piece, its marks restoring with it rather than riding a spelling, so the guard below answers for it. function holdsLink(pieces: readonly Piece[]): boolean { return pieces.some((piece) => piece.kind === 'nodes' && marksLink(piece.nodes)) } -function marksLink(nodes: readonly AdfNode[]): boolean { - return nodes.some((node) => nodeMarks(node).some((mark) => mark.type === 'link')) +function marksLink(nodes: readonly Inline[]): boolean { + return adfNodes(nodes).some((node) => nodeMarks(node).some((mark) => mark.type === 'link')) } function readDelimiterRun(scan: Scan, index: number): number { @@ -382,8 +447,8 @@ function closeLink(scan: Scan, at: number, inner: readonly Piece[], definition: return success(false) } if (holdsImage(inner)) return failure('unmappable-image', imageAlone, scan.path) - if (holdsCarry(inner)) return failure('unsupported-node-shape', carriedInMark, scan.path) - const resolved = resolveNodes(inner, scan.path, true) + if (inner.some(holdsOwnMarks)) return failure('unsupported-node-shape', ownMarksInMark, scan.path) + const resolved = resolveNodes(inner, scan, true) if (!resolved.ok) return resolved const nodes = resolved.value // An empty link text gives the mark no node to ride, so the brackets stay text. @@ -411,7 +476,7 @@ function truncatePieces(scan: Scan, to: number): void { function closeImage(scan: Scan, at: number, inner: readonly Piece[], definition: LinkDefinition): Result { if (definition.title !== undefined) return failure('unmappable-image', 'no media node carries a link title', scan.path) - const resolved = imageAlt(inner, scan.path) + const resolved = imageAlt(inner, scan) if (!resolved.ok) return resolved const alt = resolved.value const attrs = alt === '' ? { type: 'external', url: definition.destination } : { alt, type: 'external', url: definition.destination } @@ -420,10 +485,11 @@ function closeImage(scan: Scan, at: number, inner: readonly Piece[], definition: return success(null) } -function imageAlt(inner: readonly Piece[], path: ConvertErrorPath): Result { - const nodes = resolveNodes(inner, path, false) +function imageAlt(inner: readonly Piece[], scan: Scan): Result { + const nodes = resolveNodes(inner, scan, false) if (!nodes.ok) return nodes - return success(nodes.value.map(altText).join('')) + if (nodes.value.some(isTextBreak)) return failure('unsupported-node-shape', `delete ${textBreakSpelling} from this image description: the description reads as plain alt text`, scan.path) + return success(adfNodes(nodes.value).map(altText).join('')) } // spec/flavour.md, The CommonMark image: the description's plain text, the content slot included. @@ -435,22 +501,39 @@ function altText(node: AdfNode): string { } // An image's alt text is plain, so `highlights` is off there and every `==` stays text. -function resolveNodes(pieces: readonly Piece[], path: ConvertErrorPath, highlights: boolean): Result { +function resolveNodes(pieces: readonly Piece[], scan: Scan, highlights: boolean): Result { const nodes = pieces.map(pieceNodes) const runs = delimiterRuns(pieces) const pairings = matchEmphasis(runs) writeUnpaired(nodes, runs, pairings) - if (!markPairings(pieces, nodes, pairings)) return failure('unsupported-node-shape', carriedInMark, path) + if (!markPairings(pieces, nodes, pairings)) return failure('unsupported-node-shape', ownMarksInMark, scan.path) markHighlights(pieces, nodes, highlights ? pairedHighlights(pieces, nodes) : []) - return success(mergeAdjacentText(nodes.flat())) + return success(mergeReadText(nodes.flat())) +} + +// A text break and a node whose marks are its own are walls: the text on either side joins only its own side. +function mergeReadText(items: readonly Inline[]): Inline[] { + const merged: Inline[] = [] + let run: AdfNode[] = [] + for (const item of items) { + if (isNode(item)) { + run.push(item) + continue + } + for (const node of mergeAdjacentText(run, joinsWhenRead)) merged.push(node) + merged.push(item) + run = [] + } + for (const node of mergeAdjacentText(run, joinsWhenRead)) merged.push(node) + return merged } // Only `imageAlt` reaches the image arm: everywhere else an image amid other content is refused first. // A highlight delimiter holds an empty text node until it pairs, so the emphasis around it marks it. -function pieceNodes(piece: Piece): AdfNode[] { +function pieceNodes(piece: Piece): Inline[] { switch (piece.kind) { - case 'carry': - return [piece.node] + case 'ownMarks': + return [piece] case 'highlight': return [{ text: '', type: 'text' }] case 'image': @@ -461,6 +544,8 @@ function pieceNodes(piece: Piece): AdfNode[] { return bracketNodes(piece) case 'run': return [] + case 'textBreak': + return [piece] } } @@ -473,7 +558,7 @@ function delimiterRuns(pieces: readonly Piece[]): Run[] { } // A run gives its delimiters up from the head closing and the tail opening; what is left between them is text. -function writeUnpaired(nodes: AdfNode[][], runs: readonly Run[], pairings: readonly Pairing[]): void { +function writeUnpaired(nodes: Inline[][], runs: readonly Run[], pairings: readonly Pairing[]): void { const heads = new Map() const tails = new Map() for (const pairing of pairings) { @@ -488,23 +573,23 @@ function writeUnpaired(nodes: AdfNode[][], runs: readonly Run[], pairings: reado } // Innermost pairing first, so prepending leaves the marks array outermost first (spec/flavour.md, Marks). -function markPairings(pieces: readonly Piece[], nodes: AdfNode[][], pairings: readonly Pairing[]): boolean { +function markPairings(pieces: readonly Piece[], nodes: Inline[][], pairings: readonly Pairing[]): boolean { for (const pairing of pairings) { const mark: AdfMark = { type: markType(pairing.opener.character, pairing.used) } for (let index = pairing.opener.index + 1; index < pairing.closer.index; index += 1) { - if (pieces[index]?.kind === 'carry') return false + if (pieces[index]?.kind === 'ownMarks') return false nodes[index] = applyMark(nodes[index] ?? [], mark) } } return true } -function highlightDelimiters(pieces: readonly Piece[], nodes: readonly AdfNode[][]): HighlightDelimiter[] { +function highlightDelimiters(pieces: readonly Piece[], nodes: readonly Inline[][]): HighlightDelimiter[] { const found: HighlightDelimiter[] = [] let line = 0 let position = 0 for (const [index, piece] of pieces.entries()) { - const held = nodes[index] ?? [] + const held = adfNodes(nodes[index] ?? []) const [holder] = held if (piece.kind === 'highlight' && holder !== undefined) { found.push({ closes: piece.closes, holder, index, line, opens: piece.opens, position }) @@ -520,7 +605,7 @@ function highlightDelimiters(pieces: readonly Piece[], nodes: readonly AdfNode[] } // Each opener takes the next closer holding at least one character after it, both in one line and under the same marks. -function pairedHighlights(pieces: readonly Piece[], nodes: readonly AdfNode[][]): { closer: number; opener: number }[] { +function pairedHighlights(pieces: readonly Piece[], nodes: readonly Inline[][]): { closer: number; opener: number }[] { const found = highlightDelimiters(pieces, nodes) const paired: { closer: number; opener: number }[] = [] let closer = 0 @@ -531,7 +616,7 @@ function pairedHighlights(pieces: readonly Piece[], nodes: readonly AdfNode[][]) let candidate = found[closer] while (candidate !== undefined && (!candidate.closes || candidate.position < earliest)) candidate = found[(closer += 1)] if (candidate === undefined) break - if (candidate.line !== opener.line || !sameMarks(opener.holder, candidate.holder)) continue + if (candidate.line !== opener.line || !identicalMarks(nodeMarks(opener.holder), nodeMarks(candidate.holder))) continue paired.push({ closer: candidate.index, opener: opener.index }) resume = candidate.position + highlightDelimiter.length } @@ -539,9 +624,9 @@ function pairedHighlights(pieces: readonly Piece[], nodes: readonly AdfNode[][]) } // Atlassian's schema refuses a highlight on code, a node holds one highlight, and a rebuilt node would lose its attributes. -function markHighlights(pieces: readonly Piece[], nodes: AdfNode[][], paired: readonly { closer: number; opener: number }[]): void { +function markHighlights(pieces: readonly Piece[], nodes: Inline[][], paired: readonly { closer: number; opener: number }[]): void { for (const [index, piece] of pieces.entries()) { - if (piece.kind === 'highlight') nodes[index] = (nodes[index] ?? []).map((holder) => ({ ...holder, text: highlightDelimiter })) + if (piece.kind === 'highlight') nodes[index] = adfNodes(nodes[index] ?? []).map((holder) => ({ ...holder, text: highlightDelimiter })) } for (const { closer, opener } of paired) { nodes[opener] = [] @@ -550,7 +635,8 @@ function markHighlights(pieces: readonly Piece[], nodes: AdfNode[][], paired: re } } -function highlighted(node: AdfNode): AdfNode { +function highlighted(node: Inline): Inline { + if (!isNode(node)) return node const marks = nodeMarks(node) if (node.type !== 'text' || Object.keys(nodeAttrs(node)).length > 0 || marks.some((mark) => mark.type === 'code' || mark.type === 'backgroundColor')) return node return { ...node, marks: [editorHighlight, ...marks] } @@ -562,8 +648,9 @@ function markType(character: string, used: number): string { } // A node cannot carry one mark type twice (docs/decisions.md §No schema validation). -function applyMark(nodes: readonly AdfNode[], mark: AdfMark): AdfNode[] { +function applyMark(nodes: readonly Inline[], mark: AdfMark): Inline[] { return nodes.map((node) => { + if (!isNode(node)) return node const marks = nodeMarks(node) return marks.some((carried) => carried.type === mark.type) ? node : { ...node, marks: [mark, ...marks] } }) diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 283be1e..b542d01 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -84,9 +84,24 @@ function table(...rows: AdfNode[]): AdfNode { return { content: rows, type: 'table' } } -test('builds an empty document from input holding no block', () => { - assert.deepEqual(markdownToAdf(''), { ok: true, value: { type: 'doc', version: 1 } }) - assert.deepEqual(content(markdownToAdf('\n \n\t\n')), []) +test('builds an empty document from input holding no block, and one holding no content key from the doc directive alone', () => { + assert.deepEqual(markdownToAdf(''), { ok: true, value: { content: [], type: 'doc', version: 1 } }) + assert.deepEqual(markdownToAdf('\n \n\t\n'), { ok: true, value: { content: [], type: 'doc', version: 1 } }) + assert.deepEqual(markdownToAdf('!adf:doc {content=none}\n'), { ok: true, value: { type: 'doc', version: 1 } }) + assert.deepEqual(markdownToAdf('\n!adf:doc {content=none}\n\n'), { ok: true, value: { type: 'doc', version: 1 } }) + const form = 'unsupported-node-shape: doc spells the one form !adf:doc {content=none}: this one spells another' + assert.equal(content(markdownToAdf('!adf:doc\n')), form) + assert.equal(content(markdownToAdf('!adf:doc x {content=none}\n')), form) + assert.equal( + content(markdownToAdf('!adf:doc {content=empty}\n')), + 'unsupported-node-shape: an empty document is empty markdown, and !adf:doc {content=none} spells a document holding no content key: this one spells content=empty', + ) + assert.equal(content(markdownToAdf('!adf:doc {content="none"}\n')), form) + const alone = 'unsupported-node-shape: delete the !adf:doc {content=none} line to give the document content: it stands only as the whole document' + assert.equal(content(markdownToAdf('x\n\n!adf:doc {content=none}\n')), alone) + assert.equal(content(markdownToAdf('!adf:panel info\n!adf:doc {content=none}\n!adf:/panel\n')), alone) + assert.equal(content(markdownToAdf('!adf:doc\n!adf:/doc\n')), 'malformed-directive: doc takes no body, so no !adf:/doc closes it; \\!adf: keeps the prefix literal') + assert.equal(content(markdownToAdf('a !adf:doc{content=none}\n')), 'unsupported-node-shape: doc takes the block form, !adf:doc, never the inline form') }) test('builds one paragraph from the lines a blank line does not part', () => { @@ -151,9 +166,13 @@ test('reads the codeBlock directive body as the node content, the info string it test('names the slot a codeBlock spells its language outside of', () => { const slot = 'unsupported-node-shape: codeBlock spells its language in the fence info string, or in the attribute where no info string carries it back' - assert.equal(content(markdownToAdf('!adf:codeBlock {language=rust wrap=true}\n```\nx\n```\n!adf:/codeBlock\n')), slot) + assert.equal(content(markdownToAdf('!adf:codeBlock {language=rust wrap=true}\n```\nx\n```\n!adf:/codeBlock\n')), "unsupported-node-shape: move language=rust to the fences' info strings: they can spell this language") + assert.equal( + content(markdownToAdf('!adf:codeBlock {language="c sharp"}\n```\nx\n```\n```\ny\n```\n!adf:/codeBlock\n')), + "unsupported-node-shape: move language=\"c sharp\" to the fences' info strings: they can spell this language", + ) assert.equal(content(markdownToAdf('!adf:codeBlock {language=rust}\n```sql\nx\n```\n!adf:/codeBlock\n')), slot) - assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n```carry\nx\n```\n!adf:/codeBlock\n')), slot) + assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n```adf:x\nx\n```\n!adf:/codeBlock\n')), slot) assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n```a\\b\nx\n```\n!adf:/codeBlock\n')), slot) }) @@ -343,18 +362,28 @@ test('names the position a directive name the other one spells belongs to', () = }) test('names the reserved carry name a block directive spells', () => { - const reserved = 'malformed-directive: the name carry is reserved for the opaque carry, whose block form is the carry fence' + const reserved = 'malformed-directive: the name carry is reserved for the opaque carry, whose block form is the adf: fence' assert.equal(content(markdownToAdf('!adf:carry\n')), reserved) assert.equal(content(markdownToAdf('!adf:carry\nx\n!adf:/carry\n')), reserved) assert.deepEqual(content(markdownToAdf('```adf\nx\n```\n')), [{ attrs: { language: 'adf' }, content: [text('x')], type: 'codeBlock' }]) + assert.deepEqual(content(markdownToAdf('```carry\nx\n```\n')), [{ attrs: { language: 'carry' }, content: [text('x')], type: 'codeBlock' }]) }) const carried = '!adf:carry{json="{\\"type\\":\\"placeholder\\"}"}' -test('reads the carry fence back to the node its JSON holds', () => { - assert.deepEqual(content(markdownToAdf('```carry\n{\n "attrs": {\n "url": "https://example.com/x"\n },\n "type": "blockCard"\n}\n```\n')), [ +test('reads the carry fence back to the node its info string names and its JSON holds', () => { + assert.deepEqual(content(markdownToAdf('```adf:blockCard\n{\n "attrs": {\n "url": "https://example.com/x"\n }\n}\n```\n')), [ { attrs: { url: 'https://example.com/x' }, type: 'blockCard' }, ]) + assert.deepEqual(content(markdownToAdf('```adf:\n{\n "type": "a`b"\n}\n```\n')), [{ type: 'a`b' }]) + assert.equal( + content(markdownToAdf('```adf:blockCard\n{\n "type": "blockCard"\n}\n```\n')), + 'unsupported-node-shape: the adf:blockCard fence names its node\'s type: this JSON holds a type as well; !adf:codeBlock {language="adf:blockCard"} around a bare fence keeps it a code block', + ) + assert.equal(content(markdownToAdf('```adf:\n{\n "type": "blockCard"\n}\n```\n')), 'unsupported-node-shape: the adf: fence holds a type its info string can carry: spell the fence adf:blockCard and drop type from the JSON') + assert.equal(content(markdownToAdf('```adf:\n{}\n```\n')), "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node; !adf:codeBlock {language=\"adf:\"} around a bare fence keeps it a code block") + const unnamed = 'unsupported-node-shape: the carry fence names a type no info string carries back: spell it adf: with the type in the JSON; !adf:codeBlock {language="adf:\\\\"} around a bare fence keeps it a code block' + assert.equal(content(markdownToAdf('```adf:\\\\\n{}\n```\n')), unnamed) }) test('reads the inline carry back to the node its json attribute holds', () => { @@ -363,26 +392,31 @@ test('reads the inline carry back to the node its json attribute holds', () => { ]) }) -test('names the invalid JSON no opaque carry holds', () => { - const invalid = 'malformed-directive: the opaque carry holds invalid JSON' - assert.equal(content(markdownToAdf('```carry\n{"type":\n```\n')), invalid) - assert.equal(content(markdownToAdf('```carry\n```\n')), invalid) - assert.equal(content(markdownToAdf('!adf:carry{json="{"}\n')), invalid) - assert.equal(content(markdownToAdf('!adf:carry{json=abc}\n')), invalid) +test('names the invalid JSON no opaque carry holds, and the spelling that keeps it literal', () => { + const invalid = 'malformed-directive: the opaque carry holds invalid JSON; ' + const fence = `${invalid}!adf:codeBlock {language="adf:x"} around a bare fence keeps it a code block` + assert.equal(content(markdownToAdf('```adf:x\n{"attrs":\n```\n')), fence) + assert.equal(content(markdownToAdf('```adf:x\n```\n')), fence) + assert.equal(content(markdownToAdf('!adf:carry{json="{"}\n')), `${invalid}\\!adf: keeps the prefix literal`) + assert.equal(content(markdownToAdf('!adf:carry{json=abc}\n')), `${invalid}\\!adf: keeps the prefix literal`) }) test('names the canonical spelling a carried JSON reads alone', () => { const canonically = "unsupported-node-shape: the opaque carry spells its node's JSON canonically: " - assert.equal(content(markdownToAdf('```carry\n{"type":"blockCard"}\n```\n')), `${canonically}two-space indent, keys sorted`) - assert.equal(content(markdownToAdf('!adf:carry{json="{\\"type\\": \\"blockCard\\"}"}\n')), `${canonically}compact, keys sorted`) - assert.equal(content(markdownToAdf('!adf:carry{json="{\\"type\\":\\"blockCard\\",\\"attrs\\":{}}"}\n')), `${canonically}compact, keys sorted`) + const literal = '; \\!adf: keeps the prefix literal' + assert.equal( + content(markdownToAdf('```adf:blockCard\n{"attrs":{}}\n```\n')), + `${canonically}two-space indent, keys sorted; !adf:codeBlock {language="adf:blockCard"} around a bare fence keeps it a code block`, + ) + assert.equal(content(markdownToAdf('!adf:carry{json="{\\"type\\": \\"blockCard\\"}"}\n')), `${canonically}compact, keys sorted${literal}`) + assert.equal(content(markdownToAdf('!adf:carry{json="{\\"type\\":\\"blockCard\\",\\"attrs\\":{}}"}\n')), `${canonically}compact, keys sorted${literal}`) }) test('names the node JSON an opaque carry restores alone', () => { - const node = "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node" - assert.equal(content(markdownToAdf('```carry\n[]\n```\n')), node) - assert.equal(content(markdownToAdf('!adf:carry{json=null}\n')), node) - assert.equal(content(markdownToAdf('!adf:carry{json="{\\"kind\\":\\"x\\"}"}\n')), node) + const node = "unsupported-node-shape: the opaque carry holds one ADF node's JSON: this JSON is no ADF node; " + assert.equal(content(markdownToAdf('```adf:x\n[]\n```\n')), `${node}!adf:codeBlock {language="adf:x"} around a bare fence keeps it a code block`) + assert.equal(content(markdownToAdf('!adf:carry{json=null}\n')), `${node}\\!adf: keeps the prefix literal`) + assert.equal(content(markdownToAdf('!adf:carry{json="{\\"kind\\":\\"x\\"}"}\n')), `${node}\\!adf: keeps the prefix literal`) }) test('names the shape the inline carry reads alone', () => { @@ -394,12 +428,12 @@ test('names the shape the inline carry reads alone', () => { test('holds a carried JSON value to the nesting its position leaves', () => { const nested = (levels: number): string => `${'['.repeat(levels)}${']'.repeat(levels)}` - const fence = (prefix: string, levels: number): string => `${prefix}\`\`\`carry\n${prefix}${nested(levels)}\n${prefix}\`\`\`\n` + const fence = (prefix: string, levels: number): string => `${prefix}\`\`\`adf:x\n${prefix}${nested(levels)}\n${prefix}\`\`\`\n` const deeper = (levels: number): string => `unsupported-nesting-depth: a carried node's JSON nests deeper than the ${levels} levels its position leaves` assert.equal(content(markdownToAdf(`!adf:carry{json="${nested(largestNesting + 2)}"}\n`)), deeper(largestNesting)) assert.equal( content(markdownToAdf(fence('', largestNesting + 1))), - "unsupported-node-shape: the opaque carry spells its node's JSON canonically: two-space indent, keys sorted", + 'unsupported-node-shape: the opaque carry spells its node\'s JSON canonically: two-space indent, keys sorted; !adf:codeBlock {language="adf:x"} around a bare fence keeps it a code block', ) assert.equal(content(markdownToAdf(fence('> ', largestNesting + 1))), deeper(largestNesting - 1)) }) @@ -407,11 +441,11 @@ test('holds a carried JSON value to the nesting its position leaves', () => { test('names the number no JSON spelling carries in an opaque carry', () => { const named = 'unsupported-node-shape: the opaque carry holds a number JSON cannot spell' assert.equal(content(markdownToAdf('!adf:carry{json="{\\"attrs\\":{\\"width\\":1e999},\\"type\\":\\"blockCard\\"}"}\n')), named) - assert.equal(content(markdownToAdf('```carry\n1e999\n```\n')), named) + assert.equal(content(markdownToAdf('```adf:x\n1e999\n```\n')), named) }) test('names the mark spelling no opaque carry sits inside', () => { - const named = 'unsupported-node-shape: no mark spelling wraps an opaque carry: the carried node restores exactly, marks included' + const named = 'unsupported-node-shape: move the opaque carry, or the node spelling marks=empty, out of the mark spelling: it holds its own marks' assert.equal(content(markdownToAdf(`_a ${carried} b_\n`)), named) assert.equal(content(markdownToAdf(`**${carried}**\n`)), named) assert.equal(content(markdownToAdf(`~~a ${carried}~~\n`)), named) @@ -454,7 +488,9 @@ test('names the marks key no marks array reads back from', () => { assert.equal(content(markdownToAdf('!adf:rule {marks="[1]"}\n')), named) assert.equal(content(markdownToAdf('!adf:rule {marks="{}"}\n')), named) assert.equal(content(markdownToAdf('!adf:rule {marks=x}\n')), named) - assert.equal(content(markdownToAdf('!adf:rule {marks="[{\\"attrs\\":{},\\"type\\":\\"em\\"}]"}\n')), named) + assert.equal(content(markdownToAdf('!adf:rule {marks="[{\\"type\\":\\"em\\",\\"attrs\\":{}}]"}\n')), named) + assert.deepEqual(content(markdownToAdf('!adf:rule {marks="[{\\"attrs\\":{},\\"type\\":\\"em\\"}]"}\n')), [{ marks: [{ attrs: {}, type: 'em' }], type: 'rule' }]) + assert.deepEqual(content(markdownToAdf('!adf:rule {marks=empty}\n')), [{ marks: [], type: 'rule' }]) }) test('names the attribute a node holds no reading for', () => { @@ -487,7 +523,8 @@ test('names the argument and the body a node takes no reading for', () => { assert.equal(content(markdownToAdf('!adf:rule x\n')), 'unsupported-node-shape: rule takes no argument: this one spells one') assert.equal(content(markdownToAdf('!adf:paragraph\nOne.\n\nTwo.\n!adf:/paragraph\n')), 'unsupported-node-shape: paragraph takes one paragraph as its body: this body is not one') assert.equal(content(markdownToAdf('!adf:paragraph\n---\n!adf:/paragraph\n')), 'unsupported-node-shape: paragraph takes one paragraph as its body: this body is not one') - assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\nx\n!adf:/codeBlock\n')), 'unsupported-node-shape: codeBlock takes one code block as its body: this body is not one') + assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\nx\n!adf:/codeBlock\n')), 'unsupported-node-shape: codeBlock takes code blocks as its body: this body holds another block') + assert.equal(content(markdownToAdf('!adf:codeBlock {wrap=true}\n!adf:/codeBlock\n')), 'unsupported-node-shape: codeBlock takes code blocks as its body: this body holds none') assert.equal(content(markdownToAdf('!adf:paragraph\n![a](/u)\n!adf:/paragraph\n')), 'unmappable-image: no ADF node carries an image inside a paragraph') assert.equal(content(markdownToAdf('Part !adf:date[now]{timestamp=1}.\n')), 'unsupported-node-shape: date takes no content: this one holds some') }) @@ -545,7 +582,7 @@ test('names the line and the offset in the input a refusal sits at, the innermos assert.deepEqual(position(markdownToAdf('- Part.\n- a b\n')), { line: 2, offset: 8 }) assert.deepEqual(position(markdownToAdf('Part.\n\n!adf:panel info\nMore.\n')), { line: 3, offset: 7 }) assert.deepEqual(position(markdownToAdf('x\n\na b\n===\n')), { line: 3, offset: 3 }) - assert.deepEqual(position(markdownToAdf('x\n\n```carry\n{\n```\n')), { line: 3, offset: 3 }) + assert.deepEqual(position(markdownToAdf('x\n\n```adf:x\n{\n```\n')), { line: 3, offset: 3 }) assert.deepEqual(position(markdownToAdf('x\n\n| a |\n')), { line: 3, offset: 3 }) assert.deepEqual(position(markdownToAdf('a\nb c\n')), { line: 1, offset: 0 }) assert.deepEqual(position(markdownToAdf('Part.\r\n\r\n
\r\n')), { line: 3, offset: 9 }) @@ -1053,3 +1090,36 @@ test('names the directive mark left without the content it wraps', () => { assert.equal(content(markdownToAdf('!adf:underline[]\n')), named) assert.equal(content(markdownToAdf('!adf:underline{}\n')), named) }) + +test('reads the text break only between two text nodes CommonMark joins, building no node', () => { + assert.deepEqual(content(markdownToAdf('a!adf:textBreak{}b\n')), [{ content: [text('a'), text('b')], type: 'paragraph' }]) + assert.deepEqual(content(markdownToAdf('==a!adf:textBreak{}b==\n')), [{ content: [text('==a'), text('b==')], type: 'paragraph' }]) + const parts = 'unsupported-node-shape: delete !adf:textBreak{} here: it stands only between two runs of text with the same formatting, which would otherwise read as one' + const carried = '!adf:carry{json="{\\"text\\":\\"b\\",\\"type\\":\\"text\\"}"}' + for (const markdown of ['!adf:textBreak{}a\n', 'a!adf:textBreak{}\n', 'a!adf:textBreak{}!adf:textBreak{}b\n', '**a**!adf:textBreak{}b\n', '[!adf:textBreak{}](/u)\n', `a!adf:textBreak{}${carried}\n`]) { + assert.equal(content(markdownToAdf(markdown)), parts, markdown) + } + assert.equal(content(markdownToAdf('!adf:status[a!adf:textBreak{}b]\n')), 'unsupported-node-shape: delete !adf:textBreak{} from this content slot: a slot holds one text node') + assert.equal(content(markdownToAdf('![a!adf:textBreak{}b](/i)\n')), 'unsupported-node-shape: delete !adf:textBreak{} from this image description: the description reads as plain alt text') + assert.equal(content(markdownToAdf('a!adf:textBreak{x=y}b\n')), 'unsupported-node-shape: textBreak spells the bare leaf form, !adf:textBreak{}: this one spells more') + assert.deepEqual(content(markdownToAdf('!adf:carry{json="{\\"type\\":\\"textBreak\\"}"}\n')), [{ content: [{ type: 'textBreak' }], type: 'paragraph' }]) +}) + +test('leads a refusal ordinary editing meets with the edit that fixes it', () => { + assert.equal( + content(markdownToAdf('!adf:paragraph {content=empty}\nText.\n!adf:/paragraph\n')), + 'unsupported-node-shape: remove content=empty to give the paragraph a body: content=empty holds none, and this one holds one', + ) + assert.equal( + content(markdownToAdf('!adf:codeBlock\n```\na\n```\n```\n```\n!adf:/codeBlock\n')), + 'unsupported-node-shape: delete the empty fence: a fence beside another holds code, and this one holds none', + ) + assert.equal( + content(markdownToAdf('!adf:codeBlock {attrs=empty}\n```js\na\n```\n```js\nb\n```\n!adf:/codeBlock\n')), + "unsupported-node-shape: remove attrs=empty to give the codeBlock the fence's language: attrs=empty holds no language, and this fence names one", + ) + assert.equal( + content(markdownToAdf('!adf:panel info {attrs=empty}\nText.\n!adf:/panel\n')), + 'unsupported-node-shape: panel spells attrs=empty beside another attribute, an argument or a [content] slot: drop attrs=empty, or the rest', + ) +}) diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 8ed8a6a..26907f8 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -2,23 +2,24 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import type { Block, DirectiveBlock } from './blocks.ts' import type { BlockDirectiveNode } from './directive-nodes.ts' import type { ConvertFault } from '../../result.ts' -import type { Flavour } from '../plain-conventions.ts' +import type { Flavour } from '../plain/conventions.ts' import type { LineContainer } from '../line-container.ts' import type { LinkDefinitions } from './inline-content.ts' -import { carryName, readCarriedBlock } from '../opaque-carry.ts' +import { carryFenceType, readCarriedBlock } from '../opaque-carry.ts' import { commonMarkSpelling, type SpellingMemo } from '../emit/adf-to-markdown.ts' +import { documentAttribute, documentName, documentSpelling, listBreakName, listBreakSpelling } from '../block-directive.ts' +import { emptyKeys, nodeAttrs, nodeContent } from '../../adf/document.ts' import { failure, faulted, positioned, success, type ConvertErrorPath, type ParseError, type Result, type SourcePosition } from '../../result.ts' -import { inlineLeaves } from '../emit/plain-inline.ts' +import { inlineLeaves } from '../plain/inline-reduction.ts' import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' -import { leadingMarker, readAlertMarker, readTaskMarker } from '../plain-conventions.ts' -import { listBreakName, listBreakSpelling } from '../block-directive.ts' -import { mintTaskIds } from './task-ids.ts' -import { nodeAttrs, nodeContent } from '../../adf/document.ts' +import { leadingMarker, readAlertMarker, readTaskMarker } from '../plain/conventions.ts' +import { mintTaskIds } from '../plain/task-ids.ts' import { parseBlocks } from './blocks.ts' import { parseInlineContent } from './inline-content.ts' import { readBlockDirectiveNode } from './directive-nodes.ts' -import { unsupportedNodeShape } from '../directive-syntax.ts' +import { spellStringAttribute, unsupportedNodeShape } from '../directive-syntax.ts' +import { spellsEmpty } from '../empty-keys.ts' type Paragraph = Extract @@ -39,10 +40,15 @@ export function plainMarkdownToAdf(markdown: string): Result { const parsed = parseBlocks(markdown) + const [only] = parsed.blocks + if (parsed.blocks.length === 1 && only?.kind === 'directive' && only.name === documentName) { + const fault = documentFault(only) + return fault === undefined ? success({ type: 'doc', version: 1 }) : positioned(faulted(fault, []), only.position) + } const reading: Reading = { carried: new Set(), definitions: parsed.definitions, flavour, inExpand: false, memo: new Map() } const content = positioned(readBlocks(parsed.blocks, reading, [], 0), documentStart) if (!content.ok) return content - const document: AdfDocument = content.value.length === 0 ? { type: 'doc', version: 1 } : { content: content.value, type: 'doc', version: 1 } + const document: AdfDocument = { content: content.value, type: 'doc', version: 1 } if (flavour === 'plain') mintTaskIds(document, markdown, reading.carried) return success(document) } @@ -52,6 +58,10 @@ function readBlocks(blocks: readonly Block[], reading: Reading, path: ConvertErr const content: AdfNode[] = [] for (const [index, block] of blocks.entries()) { const nodePath = [...path, 'content', content.length] + if (block.kind === 'directive' && block.name === documentName) { + const fault = documentFault(block) ?? unsupportedNodeShape(`delete the ${documentSpelling} line to give the document content: it stands only as the whole document`) + return positioned(faulted(fault, nodePath), block.position) + } if (block.kind === 'directive' && block.name === listBreakName) { const fault = listBreakFault(block, blocks[index - 1], blocks[index + 1]) if (fault !== undefined) return positioned(faulted(fault, nodePath), block.position) @@ -73,6 +83,15 @@ function listBreakFault(block: DirectiveBlock, previous: Block | undefined, next return previous.kind === next?.kind ? undefined : partsFault() } +function documentFault(block: DirectiveBlock): ConvertFault | undefined { + const spelled = block.attributes.get(documentAttribute.key) + if (block.argument === undefined && block.attributes.size === 1 && spelled?.spelling === documentAttribute.value) return undefined + if (block.argument === undefined && block.attributes.size === 1 && spellsEmpty(spelled)) { + return unsupportedNodeShape(`an empty document is empty markdown, and ${documentSpelling} spells a document holding no content key: this one spells ${documentAttribute.key}=empty`) + } + return unsupportedNodeShape(`${documentName} spells the one form ${documentSpelling}: this one spells another`) +} + function partsFault(): ConvertFault { return unsupportedNodeShape(`${listBreakName} parts two adjacent lists of one type: this one parts something else`) } @@ -216,22 +235,38 @@ function directiveNode(block: DirectiveBlock, reading: Reading, path: ConvertErr function directiveBody(read: BlockDirectiveNode, blocks: Block[] | undefined, reading: Reading, path: ConvertErrorPath, depth: number): Result { const { contentModel, node } = read if (blocks === undefined) return success(node) + // A directive builds a content key only from content=empty, which holds no body. + if (node.content !== undefined) return blocks.length === 0 ? success(node) : failure('unsupported-node-shape', `remove content=empty to give the ${node.type} a body: content=empty holds none, and this one holds one`, path) if (contentModel === 'code') return codeDirectiveNode(node, blocks, path) if (contentModel === 'inline') return inlineBodyNode(node, blocks, reading, path) return containerNode(node, blocks, reading, path, depth) } +// spec/flavour.md, The CommonMark blocks: one fence per text node, every fence carrying the one language. function codeDirectiveNode(node: AdfNode, blocks: readonly Block[], path: ConvertErrorPath): Result { - const only = blocks.length === 1 ? blocks[0] : undefined - if (only?.kind !== 'code') return failure('unsupported-node-shape', `${node.type} takes one code block as its body: this body is not one`, path) + const fences: Extract[] = [] + for (const block of blocks) { + if (block.kind !== 'code') return failure('unsupported-node-shape', `${node.type} takes code blocks as its body: this body holds another block`, path) + fences.push(block) + } + const [first] = fences + if (first === undefined) return failure('unsupported-node-shape', `${node.type} takes code blocks as its body: this body holds none`, path) + if (fences.some((fence) => fence.language !== first.language)) return failure('unsupported-node-shape', `${node.type} holds one language, so its fences carry one info string: these differ`, path) + if (fences.length > 1 && fences.some((fence) => fence.text === '')) return failure('unsupported-node-shape', `delete the empty fence: a fence beside another holds code, and this one holds none`, path) const attribute = nodeAttrs(node)['language'] - const fromFence = only.language !== '' - const slot = languageSlot(fromFence ? only.language : attribute) + const fromFence = first.language !== '' + const slot = languageSlot(fromFence ? first.language : attribute) + if (!fromFence && slot.kind === 'fence') { + return failure('unsupported-node-shape', `move language=${spellStringAttribute(slot.info)} to the fences' info strings: they can spell this language`, path) + } if ((slot.kind === 'fence') !== fromFence || (fromFence && attribute !== undefined)) { return failure('unsupported-node-shape', `${node.type} spells its language in the fence info string, or in the attribute where no info string carries it back`, path) } - const spelled = fromFence ? { ...node, attrs: { ...node.attrs, language: only.language } } : node - return success(withContent(spelled, only.text === '' ? [] : [{ text: only.text, type: 'text' }])) + if (fromFence && emptyKeys(node).includes('attrs')) { + return failure('unsupported-node-shape', `remove attrs=empty to give the ${node.type} the fence's language: attrs=empty holds no language, and this fence names one`, path) + } + const spelled = fromFence ? { ...node, attrs: { ...node.attrs, language: first.language } } : node + return success(withContent(spelled, first.text === '' ? [] : fences.map((fence): AdfNode => ({ text: fence.text, type: 'text' })))) } function tableNode(rows: readonly string[][], reading: Reading, path: ConvertErrorPath): Result { @@ -278,8 +313,9 @@ function listNode(node: AdfNode, items: readonly Block[][], reading: Reading, pa } function codeBlockNode(language: string, text: string, reading: Reading, path: ConvertErrorPath, depth: number): Result { - if (language === carryName) { - const carried = readCarriedBlock(text, depth) + const carriedType = carryFenceType(language) + if (carriedType !== undefined) { + const carried = readCarriedBlock(carriedType, text, depth) if (carried.fault !== undefined) return faulted(carried.fault, path) reading.carried.add(carried.value) return success(carried.value) diff --git a/src/markdown/parse/plain-markdown-to-adf.test.ts b/src/markdown/parse/plain-markdown-to-adf.test.ts index d38e883..af6aee2 100644 --- a/src/markdown/parse/plain-markdown-to-adf.test.ts +++ b/src/markdown/parse/plain-markdown-to-adf.test.ts @@ -3,10 +3,9 @@ import test from 'node:test' import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' import { adfToMarkdown } from '../emit/adf-to-markdown.ts' -import { adfToPlainMarkdown } from '../emit/plain-reduction.ts' +import { adfToPlainMarkdown } from '../plain/adf-to-plain-markdown.ts' import { largestNesting } from '../../nesting.ts' import { markdownToAdf, plainMarkdownToAdf } from './markdown-to-adf.ts' -import { toEditorNormal } from '../../adf/editor-normal.ts' const code: AdfMark = { type: 'code' } const em: AdfMark = { type: 'em' } @@ -18,7 +17,7 @@ const taskTypes = ['blockTaskItem', 'taskItem', 'taskList'] function read(markdown: string): readonly AdfNode[] | string { const parsed = plainMarkdownToAdf(markdown) if (!parsed.ok) return parsed.error.code - const blocks = toEditorNormal(parsed.value).content ?? [] + const blocks = parsed.value.content ?? [] const pending = [...blocks] for (let block = pending.pop(); block !== undefined; block = pending.pop()) { for (const child of block.content ?? []) pending.push(child) @@ -31,10 +30,6 @@ function read(markdown: string): readonly AdfNode[] | string { return blocks } -function normal(...blocks: AdfNode[]): readonly AdfNode[] { - return toEditorNormal(document(...blocks)).content ?? [] -} - function document(...content: AdfNode[]): AdfDocument { return { content, type: 'doc', version: 1 } } @@ -57,7 +52,7 @@ function bare(type: string, ...content: AdfNode[]): AdfNode { } function paragraph(...content: AdfNode[]): AdfNode { - return bare('paragraph', ...content) + return content.length === 0 ? { type: 'paragraph' } : bare('paragraph', ...content) } function said(value: string): AdfNode { @@ -101,7 +96,7 @@ test('reads the rest of an alert marker line as the panel first body paragraph, assert.deepEqual(read('> [!tip] Title\n> body\n'), [panel('tip', said('Title'), said('body'))]) assert.deepEqual(read('> [!tip] Title\\\n> body\n'), [panel('tip', said('Title'), said('body'))]) assert.deepEqual(read('> [!NOTE]\\\n> Broken.\n'), [panel('info', said('Broken.'))]) - assert.deepEqual(read('> [!NOTE]\n'), normal(panel('info', paragraph()))) + assert.deepEqual(read('> [!NOTE]\n'), [panel('info', paragraph())]) assert.deepEqual(read('> > [!WARNING]\n> > Inner.\n'), [bare('blockquote', panel('warning', said('Inner.')))]) }) @@ -120,27 +115,27 @@ test('reads a folded callout to an expand titled by the rest of its marker line, node('expand', { title: 'Why?' }, paragraph(text('See '), text('the docs', link), text(', '), text('now', strong), text('.'))), ]) assert.deepEqual(read('> [!NOTE]- Two\\\n> lines\n'), [node('expand', { title: 'Two' }, said('lines'))]) - assert.deepEqual(read('> [!faq]- See [x](http://y)\n'), normal(node('expand', { title: 'See x (http://y)' }, paragraph()))) - assert.deepEqual(read('> [!faq]- [a **b**](u "t")[c](u) and [d](v)\n'), normal(node('expand', { title: 'a bc (u) and d (v)' }, paragraph()))) - assert.deepEqual(read('> [!faq]- or or \n'), normal(node('expand', { title: 'http://y or a@b.c or http://a\\b' }, paragraph()))) - assert.deepEqual(read('> [!faq]- [a b](u)\n'), normal(node('expand', { title: 'a\nb (u)' }, paragraph()))) - assert.deepEqual(read('> [!faq]- [!adf:mention[@M]{id=5}](u) !adf:inlineCard{url="http://y"}\n'), normal(node('expand', { title: '@M (u) http://y' }, paragraph()))) - assert.deepEqual(read('> [!NOTE]- Set ==x== here\n'), normal(node('expand', { title: 'Set ==x== here' }, paragraph()))) + assert.deepEqual(read('> [!faq]- See [x](http://y)\n'), [node('expand', { title: 'See x (http://y)' }, paragraph())]) + assert.deepEqual(read('> [!faq]- [a **b**](u "t")[c](u) and [d](v)\n'), [node('expand', { title: 'a bc (u) and d (v)' }, paragraph())]) + assert.deepEqual(read('> [!faq]- or or \n'), [node('expand', { title: 'http://y or a@b.c or http://a\\b' }, paragraph())]) + assert.deepEqual(read('> [!faq]- [a b](u)\n'), [node('expand', { title: 'a\nb (u)' }, paragraph())]) + assert.deepEqual(read('> [!faq]- [!adf:mention[@M]{id=5}](u) !adf:inlineCard{url="http://y"}\n'), [node('expand', { title: '@M (u) http://y' }, paragraph())]) + assert.deepEqual(read('> [!NOTE]- Set ==x== here\n'), [node('expand', { title: 'Set ==x== here' }, paragraph())]) assert.deepEqual(read('> [!NOTE]-\n>\n> Line.\n'), [bare('expand', said('Line.'))]) }) test('reads a folded callout inside an expand to a nested expand', () => { const markdown = '> [!NOTE]- Outer\n>\n> > [!NOTE]- Inner\n> >\n> > Deep.\n>\n> > [!TIP]\n> >\n> > > [!NOTE]-\n' - assert.deepEqual(read(markdown), normal(node('expand', { title: 'Outer' }, node('nestedExpand', { title: 'Inner' }, said('Deep.')), panel('tip', bare('nestedExpand', paragraph()))))) - assert.deepEqual(read('- > [!NOTE]-\n'), normal(bare('bulletList', bare('listItem', bare('expand', paragraph()))))) - assert.deepEqual(read('!adf:expand\n> [!NOTE]- Inner\n!adf:/expand\n'), normal(bare('expand', node('nestedExpand', { title: 'Inner' }, paragraph())))) + assert.deepEqual(read(markdown), [node('expand', { title: 'Outer' }, node('nestedExpand', { title: 'Inner' }, said('Deep.')), panel('tip', bare('nestedExpand', paragraph())))]) + assert.deepEqual(read('- > [!NOTE]-\n'), [bare('bulletList', bare('listItem', bare('expand', paragraph())))]) + assert.deepEqual(read('!adf:expand\n> [!NOTE]- Inner\n!adf:/expand\n'), [bare('expand', node('nestedExpand', { title: 'Inner' }, paragraph()))]) }) test('reads a bullet list whose every item leads with a task marker to a task list', () => { assert.deepEqual(read('- [x] Write the spec\n- [ ] Ship **it**\n- [X] Tell\n'), [ bare('taskList', task('DONE', text('Write the spec')), task('TODO', text('Ship '), text('it', strong)), task('DONE', text('Tell'))), ]) - assert.deepEqual(read('- [x]\n- [ ]\\\n after\n'), normal(bare('taskList', task('DONE'), task('TODO', text('after'))))) + assert.deepEqual(read('- [x]\n- [ ]\\\n after\n'), [bare('taskList', task('DONE'), task('TODO', text('after')))]) const minted = plainMarkdownToAdf('- [x] Parent\n - [ ] Child\n') assert.deepEqual(minted.ok ? minted.value.content : minted.error.code, [ node( @@ -202,7 +197,7 @@ test('reads what the reduction wrote back to the node it reduced, less the attri assert.deepEqual(roundTripped(node('panel', { localId, panelType }, said('Check.'))), [panel(panelType, said('Check.'))], panelType) } const expand = node('expand', { localId, title: 'Log' }, said('Line.'), node('nestedExpand', { title: 'Inner' }, said('Deep.'))) - assert.deepEqual(roundTripped(node('panel', { panelType: 'tip' }, paragraph()), node('expand', { title: 'Empty' }, paragraph())), normal(panel('tip', paragraph()), node('expand', { title: 'Empty' }, paragraph()))) + assert.deepEqual(roundTripped(node('panel', { panelType: 'tip' }, paragraph()), node('expand', { title: 'Empty' }, paragraph())), [panel('tip', paragraph()), node('expand', { title: 'Empty' }, paragraph())]) assert.deepEqual(roundTripped(expand), [node('expand', { title: 'Log' }, said('Line.'), node('nestedExpand', { title: 'Inner' }, said('Deep.')))]) const tasks = bare( 'taskList', @@ -266,3 +261,8 @@ test('keeps what markdownToAdf reads that no row reads, and refuses only what it for (let level = 0; level < largestNesting; level += 1) deep = `> ${deep}` assert.equal(typeof read(deep), 'object') }) + +test('joins a carried text node to no neighbour, inside a highlight too', () => { + const carried = '!adf:carry{json="{\\"marks\\":[],\\"text\\":\\"b\\",\\"type\\":\\"text\\"}"}' + assert.deepEqual(read(`==a${carried}==\n`), [paragraph(text('a', highlight), { marks: [], text: 'b', type: 'text' })]) +}) diff --git a/src/markdown/emit/plain-reduction.test.ts b/src/markdown/plain/adf-to-plain-markdown.test.ts similarity index 96% rename from src/markdown/emit/plain-reduction.test.ts rename to src/markdown/plain/adf-to-plain-markdown.test.ts index 7f6ac56..d634fe0 100644 --- a/src/markdown/emit/plain-reduction.test.ts +++ b/src/markdown/plain/adf-to-plain-markdown.test.ts @@ -2,7 +2,7 @@ import assert from 'node:assert/strict' import test from 'node:test' import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' -import { adfToPlainMarkdown, reduceToPlain } from './plain-reduction.ts' +import { adfToPlainMarkdown, reduceToPlain } from './adf-to-plain-markdown.ts' import { largestNesting } from '../../nesting.ts' const code: AdfMark = { type: 'code' } @@ -64,6 +64,7 @@ test('refuses what the document guard refuses, and nothing else', () => { let deep: AdfNode = said('x') for (let level = 0; level <= largestNesting; level += 1) deep = { content: [deep], type: 'layoutColumn' } assert.match(plainDocument(document(deep)), /^unsupported-nesting-depth at \/content\/0(\/content\/0)+$/) + assert.match(plainDocument(document(paragraph(text('a'), text('b')), text('c'), text('d'), deep)), /^unsupported-nesting-depth at \/content\/3(\/content\/0)+$/) let deepInline: AdfNode = text('x') for (let level = 0; level <= largestNesting; level += 1) deepInline = { content: [deepInline], type: 'unknownInline' } assert.match(plainDocument(document(paragraph(deepInline))), /^unsupported-nesting-depth at /) @@ -177,7 +178,7 @@ test('keeps the CommonMark blocks in their spelling and drops their attributes a assert.equal(plain(node('paragraph', localId, text('x')), node('heading', { level: 2, localId: '01a0d99b-1f57-7fec-94ae-50c2ee25c9de' }, text('h'))), 'x\n\n## h\n') assert.equal(plain({ attrs: localId, content: [said('q')], marks: [{ type: 'breakout' }], type: 'blockquote' }), '> q\n') assert.equal(plain(node('codeBlock', { language: 'ts', wrap: true }, text('a\r\nb\u0000'))), '```ts\na\nb\n```\n') - assert.equal(plain(node('codeBlock', { language: 'carry' }, text('a'), { type: 'hardBreak' }, text('b', strong))), '```\na\nb\n```\n') + assert.equal(plain(node('codeBlock', { language: 'adf:x' }, text('a'), { type: 'hardBreak' }, text('b', strong))), '```\na\nb\n```\n') assert.equal(plain(node('codeBlock', {})), '```\n```\n') assert.equal(plain(node('rule', { color: '#000' })), '---\n') assert.equal(plain(node('orderedList', { localId: '01a0d99b-1f58-7b95-829b-6f9860371d54', order: 3 }, item(said('c')))), '3. c\n') @@ -337,3 +338,15 @@ test('keeps the nodes the plain flavour spells and degrades only what it cannot' const reduced = reduceToPlain(document(node('panel', { localId: '01a0d99b-1f56-7a50-889a-f4375f09ee05', panelType: 'info' }, said('x')), tasks)) assert.deepEqual(reduced.ok ? reduced.value : undefined, document(node('panel', { panelType: 'info' }, said('x')), { content: [node('taskItem', { state: 'DONE' }, text('t'))], type: 'taskList' })) }) + +test('writes what the editor-normal form of the document writes', () => { + assert.equal(plain({ attrs: { order: -0 }, content: [{ content: [], type: 'listItem' }], type: 'orderedList' }), '0.\n') + assert.equal(plain({ content: [], text: 'hi', type: 'futureInline' }), 'hi\n') + assert.equal(plain(paragraph({ marks: [{ attrs: {}, type: 'code' }], text: 'a', type: 'text' }, { marks: [{ type: 'code' }], text: '|b', type: 'text' })), '`a|b`\n') +}) + +test('writes a text node holding content the same in either order beside its neighbour', () => { + const holding: AdfNode = { content: [text('inner')], text: 'a', type: 'text' } + assert.equal(plain(paragraph(holding, text('b'))), 'ab\n') + assert.equal(plain(paragraph(text('b'), holding)), 'ba\n') +}) diff --git a/src/markdown/emit/plain-reduction.ts b/src/markdown/plain/adf-to-plain-markdown.ts similarity index 95% rename from src/markdown/emit/plain-reduction.ts rename to src/markdown/plain/adf-to-plain-markdown.ts index 145ecf4..256f97c 100644 --- a/src/markdown/emit/plain-reduction.ts +++ b/src/markdown/plain/adf-to-plain-markdown.ts @@ -1,13 +1,14 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import { adfDocumentFault, nodeAttrs, nodeContent } from '../../adf/document.ts' -import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from './adf-to-markdown.ts' +import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from '../emit/adf-to-markdown.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' -import { inlineLeaves, isBlockNodeType, oneLine, reduceInline, writableHref } from './plain-inline.ts' +import { inlineLeaves, isBlockNodeType, oneLine, reduceInline, writableHref } from './inline-reduction.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' -import { taskMarker } from '../plain-conventions.ts' +import { taskMarker } from './conventions.ts' +import { toEditorNormal } from './editor-normal.ts' // depth: the level the node reduced stands at, counted as the emitter counts it. type Reduction = { depth: number; memo: SpellingMemo; path: ConvertErrorPath } @@ -50,8 +51,14 @@ export function reduceToPlain(document: AdfDocument): Result { const fault = adfDocumentFault(document) if (fault !== undefined) return faulted(fault, []) if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) - const blocks = reduceBlocks(nodeContent(document), { depth: 0, memo: new Map(), path: [] }) - return blocks.ok ? success({ content: blocks.value, type: 'doc', version: 1 }) : blocks + // docs/decisions.md §Equality is deep: the reduction reads and writes editor-normal ADF. + const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] }) + if (!blocks.ok) { + // Merging text renumbers siblings, so a refusal's path comes from the caller's document. + const raw = reduceBlocks(nodeContent(document), { depth: 0, memo: new Map(), path: [] }) + return !raw.ok && raw.error.code === blocks.error.code ? raw : blocks + } + return success(toEditorNormal({ content: blocks.value, type: 'doc', version: 1 })) } function reduceBlocks(nodes: readonly AdfNode[], reduction: Reduction): Result { @@ -84,7 +91,7 @@ function reduceStanding(node: AdfNode, reduction: Reduction): Result function standsInline(node: AdfNode): boolean { if (node.type === 'text' || inlineNodeModel(node.type) !== undefined) return true - return !isBlockNodeType(node.type) && node.content === undefined + return !isBlockNodeType(node.type) && nodeContent(node).length === 0 } function reduceBody(node: AdfNode, reduction: Reduction): Result { @@ -262,7 +269,8 @@ function listItem(blocks: Result): Result { // A list item's first line reads as no rule and holds no line of spaces alone: the rule and the spaces give way. function itemOf(blocks: readonly AdfNode[]): AdfNode { const rules = blocks.findIndex((block) => block.type !== 'rule') - return { content: blankedCode(blocks.slice(rules === -1 ? blocks.length : rules)), type: 'listItem' } + const content = blankedCode(blocks.slice(rules === -1 ? blocks.length : rules)) + return content.length === 0 ? { type: 'listItem' } : { content, type: 'listItem' } } function blankedCode(blocks: readonly AdfNode[]): AdfNode[] { diff --git a/src/markdown/plain-conventions.ts b/src/markdown/plain/conventions.ts similarity index 97% rename from src/markdown/plain-conventions.ts rename to src/markdown/plain/conventions.ts index c5d6f09..be6c5c7 100644 --- a/src/markdown/plain-conventions.ts +++ b/src/markdown/plain/conventions.ts @@ -1,4 +1,4 @@ -import { isWordCharacter } from './commonmark/emphasis-matching.ts' +import { isWordCharacter } from '../commonmark/emphasis-matching.ts' export type Flavour = 'lossless' | 'plain' diff --git a/src/adf/editor-normal.test.ts b/src/markdown/plain/editor-normal.test.ts similarity index 82% rename from src/adf/editor-normal.test.ts rename to src/markdown/plain/editor-normal.test.ts index bb0dd40..851c660 100644 --- a/src/adf/editor-normal.test.ts +++ b/src/markdown/plain/editor-normal.test.ts @@ -1,8 +1,8 @@ import assert from 'node:assert/strict' import test from 'node:test' -import type { AdfNode } from './document.ts' -import type { JsonValue } from '../json-value.ts' +import type { AdfNode } from '../../adf/document.ts' +import type { JsonValue } from '../../json-value.ts' import { toEditorNormal } from './editor-normal.ts' test('merges adjacent text nodes carrying identical marks, at every level', () => { @@ -50,14 +50,15 @@ test('reads negative zero as zero, as JSON does', () => { ) }) -test('reads an empty attrs object, marks array or content array as the absent key', () => { +test('reads an empty attrs object, marks array or content array as the absent key, except on doc', () => { const paragraph: AdfNode = { attrs: {}, content: [{ attrs: {}, marks: [], text: 'a', type: 'text' }, { marks: [{ attrs: {}, type: 'em' }], text: 'b', type: 'text' }], marks: [], type: 'paragraph' } assert.deepEqual(toEditorNormal({ content: [paragraph, { content: [], type: 'rule' }], type: 'doc', version: 1 }), { content: [{ content: [{ text: 'a', type: 'text' }, { marks: [{ type: 'em' }], text: 'b', type: 'text' }], type: 'paragraph' }, { type: 'rule' }], type: 'doc', version: 1, }) - assert.deepEqual(toEditorNormal({ content: [], type: 'doc', version: 1 }), { type: 'doc', version: 1 }) + assert.deepEqual(toEditorNormal({ content: [], type: 'doc', version: 1 }), { content: [], type: 'doc', version: 1 }) + assert.deepEqual(toEditorNormal({ type: 'doc', version: 1 }), { type: 'doc', version: 1 }) }) test('normalizes blocks and mark attributes nesting far past the levels a recursive walk survives', () => { @@ -75,3 +76,10 @@ test('normalizes blocks and mark attributes nesting far past the levels a recurs const merged = toEditorNormal({ content: [{ content: [{ marks, text: 'a', type: 'text' }, { marks, text: 'b', type: 'text' }], type: 'paragraph' }], type: 'doc', version: 1 }) assert.deepEqual(merged.content?.[0]?.content?.map((text) => text.text), ['ab']) }) + +test('joins a text node holding content to no neighbour, so its content is kept', () => { + const holding: AdfNode = { content: [{ text: 'kept', type: 'text' }], text: 'a', type: 'text' } + for (const content of [[holding, { text: 'b', type: 'text' }], [{ text: 'b', type: 'text' }, holding]]) { + assert.deepEqual(toEditorNormal({ content: [{ content, type: 'paragraph' }], type: 'doc', version: 1 }).content, [{ content, type: 'paragraph' }]) + } +}) diff --git a/src/adf/editor-normal.ts b/src/markdown/plain/editor-normal.ts similarity index 61% rename from src/adf/editor-normal.ts rename to src/markdown/plain/editor-normal.ts index b8a299b..3877967 100644 --- a/src/adf/editor-normal.ts +++ b/src/markdown/plain/editor-normal.ts @@ -1,34 +1,27 @@ -import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from './document.ts' -import type { JsonValue } from '../json-value.ts' -import { nodeAttrs, nodeContent, nodeMarks } from './document.ts' -import { serializeCanonicalJson } from '../canonical-json.ts' +import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from '../../adf/document.ts' +import type { JsonValue } from '../../json-value.ts' +import { identicalMark, mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' +import { joinsWhenRead } from '../adjacent-text.ts' type JsonContainer = JsonValue[] | { [key: string]: JsonValue } type NodeHolder = { content?: AdfNode[] } -export function sameMark(candidate: AdfMark, mark: AdfMark): boolean { - return markKey(candidate) === markKey(mark) +// The editor's rule is the reader's over the pair's editor-normal forms; a text node holding content never joins, so its content is kept. +export function joinsWhenEditorNormal(previous: AdfNode, node: AdfNode): boolean { + if (previous.type !== 'text' || node.type !== 'text' || nodeContent(previous).length > 0 || nodeContent(node).length > 0) return false + return joinsWhenRead(normalNode(previous), normalNode(node)) } -export function mergeAdjacentText(nodes: readonly AdfNode[]): AdfNode[] { - const merged: AdfNode[] = [] - for (const node of nodes) { - const previous = merged[merged.length - 1] - if (previous !== undefined && mergesText(previous) && mergesText(node) && sameMarks(previous, node)) { - merged[merged.length - 1] = { ...previous, text: `${previous.text ?? ''}${node.text ?? ''}` } - continue - } - merged.push(node) - } - return merged +export function sameMarkWhenEditorNormal(left: AdfMark, right: AdfMark): boolean { + return identicalMark(normalMark(left), normalMark(right)) } export function toEditorNormal(document: AdfDocument): AdfDocument { const normal: AdfDocument = { type: document.type, version: Object.is(document.version, -0) ? 0 : document.version } const pending: { holder: NodeHolder; source: NodeHolder }[] = [{ holder: normal, source: document }] for (let entry = pending.pop(); entry !== undefined; entry = pending.pop()) { - const content = mergeAdjacentText(nodeContent(entry.source)) + const content = mergeAdjacentText(nodeContent(entry.source), joinsWhenEditorNormal) if (content.length === 0) continue entry.holder.content = content.map((source) => { const holder = normalNode(source) @@ -36,6 +29,8 @@ export function toEditorNormal(document: AdfDocument): AdfDocument { return holder }) } + // ADF's schema requires content on doc, so an empty one stays. + if (document.content !== undefined && normal.content === undefined) normal.content = [] return normal } @@ -72,19 +67,3 @@ function normalValue(value: JsonValue, pending: JsonContainer[]): JsonValue { pending.push(copy) return copy } - -function mergesText(node: AdfNode): boolean { - return node.type === 'text' && Object.keys(nodeAttrs(node)).length === 0 -} - -export function sameMarks(previous: AdfNode, node: AdfNode): boolean { - return marksKey(nodeMarks(previous)) === marksKey(nodeMarks(node)) -} - -function marksKey(marks: readonly AdfMark[]): string { - return marks.map(markKey).join('\n') -} - -function markKey(mark: AdfMark): string { - return `${mark.type} ${serializeCanonicalJson(nodeAttrs(mark), 'compact')}` -} diff --git a/src/markdown/emit/plain-inline.ts b/src/markdown/plain/inline-reduction.ts similarity index 95% rename from src/markdown/emit/plain-inline.ts rename to src/markdown/plain/inline-reduction.ts index 7da6adc..882f732 100644 --- a/src/markdown/emit/plain-inline.ts +++ b/src/markdown/plain/inline-reduction.ts @@ -1,12 +1,12 @@ import type { AdfAttributes, AdfMark, AdfNode } from '../../adf/document.ts' import type { LineContainer } from '../line-container.ts' -import type { MarkRun } from './line-escaping.ts' +import type { MarkRun } from '../emit/line-escaping.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' +import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from './editor-normal.ts' import { largestNesting } from '../../nesting.ts' -import { mergeAdjacentText, sameMark } from '../../adf/editor-normal.ts' -import { nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' -import { plainLineFallback, type PlainLineFallback } from './inline-line.ts' +import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' +import { plainLineFallback, type PlainLineFallback } from '../emit/inline-line.ts' import { spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts' const highlight = 'backgroundColor' @@ -175,11 +175,11 @@ function highlighted(leaves: readonly AdfNode[]): AdfNode[] { const marks = nodeMarks(leaf) if (marks[0]?.type === highlight) { const held = marks.slice(1) - shared = run.length === 0 ? held : shared.filter((mark) => held.some((other) => sameMark(other, mark))) + shared = run.length === 0 ? held : shared.filter((mark) => held.some((other) => sameMarkWhenEditorNormal(other, mark))) run.push(leaf) continue } - for (const held of run) spelled.push(withMarks(held, [...shared, highlightMark, ...nodeMarks(held).slice(1).filter((mark) => !shared.some((other) => sameMark(other, mark)))])) + for (const held of run) spelled.push(withMarks(held, [...shared, highlightMark, ...nodeMarks(held).slice(1).filter((mark) => !shared.some((other) => sameMarkWhenEditorNormal(other, mark)))])) run = [] spelled.push(leaf) } @@ -193,7 +193,7 @@ function withMarks(leaf: AdfNode, marks: readonly AdfMark[]): AdfNode { function trimmedEdges(leaves: readonly AdfNode[]): AdfNode[] { for (let current = leaves; ; ) { - const merged = withoutEdgeBreaks(mergeAdjacentText(current)) + const merged = withoutEdgeBreaks(mergeAdjacentText(current, joinsWhenEditorNormal)) let changed = false const trimmed: AdfNode[] = [] for (const [index, leaf] of merged.entries()) { @@ -251,7 +251,7 @@ function edgeDepth(marks: readonly AdfMark[], neighbour: AdfNode | undefined, wh function sameMarkAt(marks: readonly AdfMark[], others: readonly AdfMark[], index: number): boolean { const mark = marks[index] const other = others[index] - return mark !== undefined && other !== undefined && sameMark(mark, other) + return mark !== undefined && other !== undefined && sameMarkWhenEditorNormal(mark, other) } function spellableLine(leaves: AdfNode[], container: LineContainer, path: ConvertErrorPath): Result { diff --git a/src/markdown/parse/task-ids.test.ts b/src/markdown/plain/task-ids.test.ts similarity index 100% rename from src/markdown/parse/task-ids.test.ts rename to src/markdown/plain/task-ids.test.ts diff --git a/src/markdown/parse/task-ids.ts b/src/markdown/plain/task-ids.ts similarity index 100% rename from src/markdown/parse/task-ids.ts rename to src/markdown/plain/task-ids.ts diff --git a/todo.md b/todo.md index 9fd26b3..f4d656c 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 52` +`Next ID = 66` | Goal | W | |---|---| @@ -24,14 +24,23 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| -| 40 | 0.2.0 | decision | **Make `markdownToAdf(adfToMarkdown(doc))` deep-equal `doc` for every document `adfToMarkdown` takes.** | 6 | 7 | 8 | 9 | 1 | 26.2 | +| 63 | 0.2.0 | defect | **Refuse a cyclic input as `not-an-adf-document`.** | 2 | 2 | 6 | 9 | 1 | 27.5 | | 7 | 0.2.0 | | **Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF.** | 6 | 9 | 9 | 9 | 2, 3 | 25.8 | +| 55 | 0.2.0 | defect | **Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start, a newline inside an emoji, mention or status, or a text node whose text is empty or missing, or which holds content.** | 5 | 5 | 7 | 9 | 1 | 25.8 | | 45 | 0.2.0 | | **Replace `isAdfDocument` with a reader returning `Result`.** | 2 | 3 | 6 | 8 | 1 | 25.2 | | 6 | 0.2.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 3 | 24.7 | -| 43 | 0.2.0 | | **Give each markdown input its own reader, strict to its own standard.** | 6 | 7 | 8 | 9 | 3, 4 | 22.3 | +| 64 | 0.2.0 | defect | **Read back unchanged on V8 every JSON key the emitter writes, with a fixture whose key holds `\`, `"` or a control character.** | 4 | 4 | 6 | 8 | 1, 7 | 23.0 | +| 43 | 0.2.0 | decision | **Give each markdown input its own reader, strict to its own standard.** | 6 | 7 | 8 | 9 | 3, 4 | 22.3 | +| 65 | 0.2.0 | defect | **Read a mid-text or titled CommonMark image as Goal 6 allows.** | 5 | 5 | 7 | 7 | 3, 4, 6 | 18.7 | | 49 | 0.2.0 | | **Read a list whose bullet or ordered delimiter changes as two lists in the CommonMark reader.** | 4 | 5 | 5 | 8 | 3, 4 | 17.2 | +| 61 | 0.2.0 | decision | **Have the emitter ask the inline reader how a line reads back, in place of `line-escaping.ts` predicting it.** | 6 | 8 | 3 | 8 | 1 | 14.0 | +| 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 5 | 13.0 | | 51 | 0.2.0 | | **Match a reference label to its definition under Unicode case folding.** | 2 | 2 | 2 | 7 | 3, 4 | 12.4 | +| 53 | 0.2.0 | | **Put a block's `marks` spelling to the writer panel and adopt its pick.** | 4 | 5 | 5 | 6 | 5 | 11.5 | +| 60 | 0.2.0 | decision | **Collect the questions `parse/` asks `emit/` into one named module.** | 3 | 4 | 2 | 6 | 2 | 10.7 | +| 59 | 0.2.0 | decision | **Group the directive grammar into `src/markdown/directive/`, move `Read` to `result.ts`, and move `Flavour` to `markdown/flavour.ts`.** | 3 | 5 | 2 | 6 | 2 | 10.4 | | 50 | 0.2.0 | | **Read `[](/url)` and `[]()` as CommonMark's empty link.** | 4 | 4 | 3 | 6 | 3, 4, 6 | 10.4 | +| 62 | 0.2.0 | decision | **Move the plain flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/plain/`, so `emit/` no longer imports `plain/`.** | 4 | 5 | 2 | 6 | 2 | 9.4 | | 38 | 0.3.0 | | **Spell a lone surrogate in a text node so it survives a UTF-8 encode.** | 2 | 2 | 4 | 7 | 1 | 19.5 | | 47 | 0.3.0 | | **Open the README with what the package is, what it does and for whom.** | 1 | 4 | 7 | 9 | 9 | 14.0 | | 34 | 0.3.0 | | **Read emphasis flanking by the whole character beside an astral symbol.** | 2 | 3 | 3 | 6 | 3, 4 | 12.6 | @@ -41,23 +50,36 @@ | 33 | 0.3.0 | | **Emit a line in time linear in its mark runs, in `adfToMarkdown` and `adfToPlainMarkdown`.** | 4 | 5 | 6 | 9 | 8 | 10.7 | | 46 | 0.3.0 | | **Publish the bundle size in the README, failing the release pipeline when it drifts.** | 2 | 4 | 5 | 5 | 7, 9 | 10.3 | | 9 | 0.3.0 | | **Ship an online sandbox: a web page with two textboxes converting between ADF and markdown on the library's browser build.** | 2 | 6 | 6 | 8 | 9 | 10.3 | +| 56 | 0.3.0 | principle | **Give each piece of `blocks.ts`'s block-walk state and `inline-content.ts`'s `Scan` one owner that returns what it changes.** | 4 | 5 | 2 | 4 | 1 | 6.8 | +| 57 | 0.3.0 | principle | **Make each `ci.sh` leg build what it reads, so one leg run alone tests the current tree.** | 2 | 3 | 1 | 3 | 7 | 1.2 | +| 58 | 0.3.0 | principle | **Port `ci.sh`, `publish.sh` and `docker-runner.sh` to standalone Python scripts.** | 4 | 6 | 1 | 2 | 7 | -2.2 | | 8 | 0.4.0 | | **Ship a CLI.** | 3 | 7 | 7 | 6 | 9 | 10.6 | ## Details -### 40. Make `markdownToAdf(adfToMarkdown(doc))` deep-equal `doc` for every document `adfToMarkdown` takes. +### 63. Refuse a cyclic input as `not-an-adf-document`. -Today it holds for editor-normal documents only: two adjacent text nodes with the same marks merge, -an empty `attrs`, `marks` or `content` drops, and `-0` reads back `0` — shapes pipelines and bots -build. Spell each so it reads back as written; CommonMark's spelling stays wherever the document -holds none of these shapes. The spellings are part of the chunk. `docs/decisions.md` §Equality is -editor-normal, `spec/flavour.md` and `corpus/README.md` follow, and the tests drop `toEditorNormal`. +`adfDocumentFault` walks with `isNodeArray` (`adf/document.ts`) and `isJsonValue` (`json-value.ts`), +worklists that record no visited object, so a node whose `content` holds itself hangs +`adfToMarkdown`, `adfToPlainMarkdown` and `isAdfDocument`; README §The guarantees promises no input +loops forever. ### 7. Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF. -Lands after item 6. The CommonMark spec suite also runs against `markdownToHtml`. The README -documents HTML as it documents markdown, and its tagline and `package.json`'s `description` regain -HTML. +Lands after items 6, 59, 60, 61 and 62. The CommonMark spec suite also runs against +`markdownToHtml`. The README documents HTML as it documents markdown, and its tagline and +`package.json`'s `description` regain HTML. + +### 55. Spell through the carry every document the emitter refuses today for a carriage return, a NUL, a code-span line start, a newline inside an emoji, mention or status, or a text node whose text is empty or missing, or which holds content. + +`inline-line.ts` refuses a carriage return or NUL in text (`unspellable-character`) and a paragraph +opening with a code-span run (`unspellable-line-start`); `fencedTexts` refuses the same characters +in a code block; `directive-syntax.ts` refuses a newline in an emoji, mention or status +(`unspellable-whitespace`). The inline carry's JSON escapes all of them, so a lossless spelling +exists, and §The code list says a cause the carry answers gets no code. Fixing it revises §Which +code a cause takes and §Markdown in is a canonical fixpoint, which the maintainer decides. Also a +text node whose `text` is empty or missing, which `inline-line.ts` and `fencedTexts` refuse, or +which holds `content`, which `inline-line.ts` refuses. Found by the README-goals audit, 2026-10-03. ### 45. Replace `isAdfDocument` with a reader returning `Result`. @@ -71,13 +93,29 @@ Element-by-element mapping, the `data-*` fidelity scheme, the opaque-carry form, foreign-element set `htmlToAdf` accepts — the set `markdownToAdf` shares (`spec/flavour.md` §Raw HTML in input). The set sorts per `docs/decisions.md` §Foreign HTML sorts three ways. +### 64. Read back unchanged on V8 every JSON key the emitter writes, with a fixture whose key holds `\`, `"` or a control character. + +`property-harness.ts` strips `\`, `"` and control characters from generated JSON keys, citing V8's +`JSON.parse` returning a wrong key for an escaped backslash. The library reads `json` attributes +(`directive-syntax.ts`) and both carries (`opaque-carry.ts`) through that `JSON.parse`, so on Node, +Deno and Chrome the round-trip would refuse its own output or read back a different key; the fixture +tells which. Confirm with a fixture first; then read JSON with our own parser or record the gap with +an ending item. Found by the README-goals audit, 2026-10-03. + ### 43. Give each markdown input its own reader, strict to its own standard. Today `markdownToAdf` reads CommonMark and the lossless flavour as one input: text shaped like a -directive, a pipe table or a `~~` pair becomes a flavour node where CommonMark reads plain text. A +directive, a pipe table or a `~~` pair becomes a flavour node where CommonMark reads plain text, and +a code fence whose info string opens `adf:` becomes the block carry where CommonMark reads code. A caller names the markdown it hands in: CommonMark, read as its spec says, or the lossless flavour, read as `spec/flavour.md` says. Breaking: `MIGRATION.md` says which call a caller takes. +### 65. Read a mid-text or titled CommonMark image as Goal 6 allows. + +A mid-text or titled image, such as a bot's `See ![diagram](url) here`, is refused: 14 examples in +`corpus/commonmark-spec/refusals.json`. Settle what such an image builds. Goal 6 drops form and +keeps the target. + ### 49. Read a list whose bullet or ordered delimiter changes as two lists in the CommonMark reader. Lands after item 43. Today `- a` then `+ b`, or `1.` then `1)`, reads as one list; CommonMark reads @@ -90,6 +128,17 @@ Readings table gains its row, and its Spellings table one if `!adf:listBreak` re and 302 lose their `pending` exceptions, and the spelling leaves the README's "Four CommonMark spellings" bullet, which counts one fewer. +### 61. Have the emitter ask the inline reader how a line reads back, in place of `line-escaping.ts` predicting it. + +The comprehension panel's worst place: `escapeClaims` and `escapeClosedRuns` re-implement the +reader's view — flanking, code-span closers, link-definition openings, highlight flanking — and +only the property tests catch drift. It caps the panel's Locality score. + +### 52. Spell `colwidth` as a comma list, `colwidth="340,420"`. + +A writer panel chose it on 2026-10-03, 5 of 7, over today's `colwidth="[340,420]"`. Breaking: +`MIGRATION.md`'s Spellings table gains its row. + ### 51. Match a reference label to its definition under Unicode case folding. `link-syntax.ts` normalizes a label with `toLowerCase`, so `[ẞ]` misses its `[SS]` definition (spec @@ -97,6 +146,24 @@ example 540); lowercasing and then uppercasing folds it. Breaking, so it ships b `MIGRATION.md`'s Readings table gains its row. Its `pending` exceptions go, and its spelling leaves the README's "Four CommonMark spellings" bullet, which counts one fewer. +### 53. Put a block's `marks` spelling to the writer panel and adopt its pick. + +Today `marks="[{\"attrs\":{\"mode\":\"wide\"},\"type\":\"breakout\"}]"`, the marks array as +escaped JSON. Breaking where the panel picks another spelling: `MIGRATION.md`'s Spellings table +gains its row. + +### 60. Collect the questions `parse/` asks `emit/` into one named module. + +`parse/` asks `emit/` through `commonMarkSpelling` and `openingLinkTakesDirective`, each imported +from where it happens to live. One module naming the questions keeps `docs/decisions.md` §The source +parts by ADF and format true as they grow. + +### 59. Group the directive grammar into `src/markdown/directive/`, move `Read` to `result.ts`, and move `Flavour` to `markdown/flavour.ts`. + +Eight directive files sit across three directories, and the `markdown/` root holds 14 entries. HTML +needs `Read` and `Flavour` out of the plain flavour and `markdown/`; it needs the carry and the +mark spellings too, which item 7 moves where it learns what HTML shares. + ### 50. Read `[](/url)` and `[]()` as CommonMark's empty link. Both stay literal text today (spec examples 484 and 487). ADF holds no empty text node to carry a @@ -106,6 +173,14 @@ input. Breaking, so it ships beside item 43: `MIGRATION.md`'s Readings table gai `pending` exceptions go, and its spelling leaves the README's "Four CommonMark spellings" bullet, which counts one fewer. +### 62. Move the plain flavour's reading out of `parse/` and its writing out of `emit/` into `markdown/plain/`, so `emit/` no longer imports `plain/`. + +Reading: the alert and task-marker reads in `parse/markdown-to-adf.ts`, the `mintTaskIds` call and +the `inlineLeaves` use. Writing: `spellPlainBlock`, `quotedUnder`, `tryTaskList` and `taskBlocks` in +`emit/adf-to-markdown.ts`. And `highlightDelimiter` and `highlightFlanking` move out of +`plain/conventions.ts`, which `emit/` imports them from; item 59 moves `Flavour`. Every comprehension reader on 2026-10-03 +named the plain flavour's spread across three directories. + ### 38. Spell a lone surrogate in a text node so it survives a UTF-8 encode. `adfToMarkdown` emits it verbatim, so markdown stored as UTF-8 reads back U+FFFD; attribute values @@ -128,8 +203,8 @@ punctuation — and, where they do, read the code point, with a fixture per dire ### 42. Trim a text leaf's trailing blanks in linear time. -`plain-inline.ts`'s `leafEdges` finds the trail with an unanchored `/[ \t]*$/`, quadratic in a run -of blanks inside one leaf: a paragraph of `a`, 80 000 spaces, `b` takes 6.5 s in +`plain/inline-reduction.ts`'s `leafEdges` finds the trail with an unanchored `/[ \t]*$/`, quadratic +in a run of blanks inside one leaf: a paragraph of `a`, 80 000 spaces, `b` takes 6.5 s in `adfToPlainMarkdown`. Scan backward, as the expand title's trim does. ### 48. Keep the release path publishing past npm's bypass-2FA token retirement. @@ -173,6 +248,26 @@ or report the unminified gzip). The figure lands in README §The package beside dependencies" claim. Measured today, unminified: tarball 60.4 kB, unpacked 221.5 kB, JS gzipped 45.6 kB. +### 56. Give each piece of `blocks.ts`'s block-walk state and `inline-content.ts`'s `Scan` one owner that returns what it changes. + +Technical principle "One owner per value": the `Walk` record passes through twelve functions that +mutate it and return `void`; `walk.leaf` alone is written in seven places. `ContainerStack` in the +same file shows the shape to follow. +`inline-content.ts`'s `Scan` has the same shape: about fifteen functions write `pending`, `pieces`, +`deactivatedBefore` and `openingSpellableLink` and return `void`, and `parseInlineContent` reads a +flag `scanInline` leaves on it. + +### 57. Make each `ci.sh` leg build what it reads, so one leg run alone tests the current tree. + +Technical principle "Compose, do not entangle": Deno, Bun and the tests read the install leg's +`node_modules`, the floor and consumer legs the pack leg's, and the browser leg whatever `dist/` the +build leg left, so a leg run alone can pass on stale output. + +### 58. Port `ci.sh`, `publish.sh` and `docker-runner.sh` to standalone Python scripts. + +Technical principle "Prefer a standalone Python script over a shell script": `AGENTS.md` §3 teaches +bash footguns (`&&` chaining under `||`, `tee /dev/stderr`) the port removes. + ### 8. Ship a CLI. The Goals and G cells are provisional: no README goal or persona covers a CLI yet. The chunk