From 40580e6948cf733e197eb82d4bd55be8162edf12 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 9 Oct 2026 14:50:50 +0200 Subject: [PATCH] Read a list whose bullet or ordered delimiter changes as two lists, and alternate the marker between adjacent lists in place of !adf:listBreak --- CHANGELOG.md | 4 ++ MIGRATION.md | 4 +- README.md | 22 +++--- corpus/commonmark-spec/exceptions.json | 14 ---- corpus/errors/list-break-body.error | 1 - corpus/errors/list-break-body.md | 7 -- corpus/errors/list-break-misplaced.error | 1 - corpus/errors/list-break-misplaced.md | 5 -- corpus/normalization/list-markers.json | 19 +++++ corpus/normalization/list-markers.md | 9 +-- .../round-trip/block-nodes/adjacent-lists.md | 17 ++--- docs/decisions.md | 11 ++- spec/flavour.md | 26 +++---- src/conformance/markdown-property.test.ts | 4 +- src/markdown/block-directive.ts | 6 +- src/markdown/commonmark/grammar.ts | 8 +-- src/markdown/emit/adf-to-markdown.test.ts | 15 ++-- src/markdown/emit/adf-to-markdown.ts | 70 ++++++++++++------- src/markdown/parse/blocks.test.ts | 1 - src/markdown/parse/blocks.ts | 22 +++--- src/markdown/parse/markdown-to-adf.test.ts | 37 ++++------ src/markdown/parse/markdown-to-adf.ts | 22 +----- .../parse/portable-markdown-to-adf.test.ts | 6 ++ .../portable/adf-to-portable-markdown.test.ts | 30 ++++---- .../portable/adf-to-portable-markdown.ts | 49 ++----------- todo.md | 13 ---- 26 files changed, 179 insertions(+), 244 deletions(-) delete mode 100644 corpus/errors/list-break-body.error delete mode 100644 corpus/errors/list-break-body.md delete mode 100644 corpus/errors/list-break-misplaced.error delete mode 100644 corpus/errors/list-break-misplaced.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 665906a..8078452 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ ## Unreleased +- **Breaking:** a list whose bullet character or ordered delimiter changes reads as two lists, as + CommonMark reads it: `- a` then `* b`, `1. a` then `1) b`. Both writers give a list right after + one of its kind the other marker, `* ` or `1)`, in place of the `::listBreak` separator. Convert + stored markdown per `MIGRATION.md`. - **Breaking:** each call names the markdown it reads or writes. `adfToMarkdown` and `markdownToAdf` are now `adfToLosslessMarkdown` and `losslessMarkdownToAdf`. A bot's or a model's markdown reads in `portableMarkdownToAdf`, and markdown known to be strict CommonMark in the new diff --git a/MIGRATION.md b/MIGRATION.md index 3e9c24b..5c9d2bb 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -65,6 +65,7 @@ function migrateMarkdown(stored: string) { | the `adf` code fence and `:adf{json="…"}` | the `adf:` code fence, its JSON without `type`, and `!adf:carry{json="…"}` | | `\:` keeps a directive literal | `\!adf:` keeps a directive literal | | `:adf{json="…"}` carrying a link for its `collection`, `id` or `occurrenceKey` | `!adf:link[text]{attrs}` | +| `::listBreak` between two lists of a kind | no separator, the second list's marker changed: `* ` for `- `, `1)` for `1.` | A colon run and `:name[` are plain text now, and `adf` an ordinary code block language; text holding an unescaped `!adf:` and a code fence whose info string opens `adf:` are claimed instead. @@ -75,6 +76,7 @@ Markdown the spelling table leaves alone, which `0.2.0` reads as a different doc | Input | `0.1.0` | `0.2.0` | | --- | --- | --- | +| a list whose bullet character or ordered delimiter changes (`- a` then `* b`, `1. a` then `1) b`) | one list | two lists, as CommonMark reads them | | a link whose text already holds one (`[ab](/v)`) | marks every node the inner link does not, splitting the outer link around it | leaves the outer brackets literal text; write the pieces as separate links to keep them | | markdown holding no block (`""`) | `{ type: 'doc', version: 1 }` | `{ content: [], type: 'doc', version: 1 }`; `!adf:doc {content=none}` reads as the former | | a hard break inside a mark's spelling (`*a\` then `b*` on the next line) | the hard break holds the mark | the hard break holds no marks, as Atlassian's schema requires, and the recipe above drops the marks `0.1.0` gave it wherever text beside it holds them. A link whose text is only hard breaks reads as its destination linked, followed by those breaks | @@ -103,7 +105,7 @@ with one of the four now reads, or refuses with the code the table gives. | an `emoji`, `mention` or `status` whose `text` holds a line ending or a null character, on emit | `unspellable-whitespace`, `unspellable-character` | rides the inline carry | | a text node holding no text, or holding `content`, on emit | `unsupported-node-shape` | rides the carry | | a `codeBlock` holding other than plain text nodes, on emit | `unsupported-node-shape` | rides the block carry | -| a leaf node given a body (`media`, `listBreak`) | `unsupported-node-shape` | `malformed-directive` | +| a leaf node given a body (`media`) | `unsupported-node-shape` | `malformed-directive` | | a node with a block body written as a leaf (`panel`) | `unsupported-node-shape` | `malformed-directive` | | an empty node the `::taskItem` spelling row names, written as a leaf | parses | `malformed-directive` | | an empty node the `::taskItem` spelling row names, written with a closer | `unsupported-node-shape` | parses | diff --git a/README.md b/README.md index e769318..d416c92 100644 --- a/README.md +++ b/README.md @@ -149,10 +149,9 @@ saving what this pair read replaces mentions, attachments and macros with text. link or code span is trimmed, carriage returns and null characters are removed, and an empty paragraph drops. - A text node's `content` drops: Atlassian's schema forbids it, so no rendered document shows it. -- An ordered list numbered past `999999999`, or adjacent ordered lists whose numbering does not - continue, is one bullet list keeping its numbers as text. -- A task list beside a bullet or decision list, or holding a block other than a task, joins one - bullet list keeping its states as text: `- \[x] Done`. +- An ordered list numbered past `999999999` is a bullet list keeping its numbers as text. +- A task list holding a block other than a task is a bullet list keeping its states as text: + `- \[x] Done`. - Text that would read as a marker takes a backslash: `==` wherever it could open or close a highlight, `[!…]` opening a quote, and `[x]` or `[ ]` opening any list item, since GitHub reads that marker per item. @@ -215,7 +214,7 @@ emit refuses: | Code | Fires when | What you can do | | --- | --- | --- | | `unsupported-nesting-depth` | blocks, marks, an attribute's JSON or a carried node's JSON nest past 500 levels | keep the ADF and pass the document over, or show it read-only; flatten the input where you are the one who wrote it | -| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` or `!adf:listBreak` parting nothing, `!adf:doc` anywhere but as the whole document. Emitting: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content` | fix what the message names; `spec/flavour.md` lists every type's attributes and body | +| `unsupported-node-shape` | parsing, in `losslessMarkdownToAdf`: markdown spells a node with an attribute, value, argument or body its type does not take, or without one it needs, wraps a directive mark around nothing or only hard breaks, writes as a directive a node or mark the lossless flavour spells as CommonMark, or puts a reserved directive out of place: `!adf:textBreak{}` parting nothing, `!adf:doc` anywhere but as the whole document. Emitting: a node not riding the carry that carries one mark type twice, a block or a non-text inline node holding `text`, or a leaf block or a non-text inline node holding `content` | fix what the message names; `spec/flavour.md` lists every type's attributes and body | ## The guarantees @@ -238,7 +237,7 @@ Serves Goals 1, 4 and 5. without a delimiter row, is an error result; each can be kept literal (`spec/flavour.md`). - A document any reader built, written back with `adfToLosslessMarkdown`, takes the library's canonical spelling, which round-trips byte-identically. -- Seven CommonMark spellings parse without an error and build a document the reference +- Six CommonMark spellings parse without an error and build a document the reference implementation renders differently. Four because ADF holds no node for what it renders: - An empty link text reads as its destination, `[](/url)` as `/url` linked to `/url`, and `[]()` as nothing. A link text holding only hard breaks reads the same, followed by its hard breaks. @@ -252,12 +251,11 @@ Serves Goals 1, 4 and 5. - An image's title becomes its caption when the image stays an image, and its link's title when it becomes linked alt text. Inside another link, the title is dropped. - And three more: a list continuing past a marker change stays one list against CommonMark's two, - a shortcut reference matching its definition only under Unicode case folding stays unresolved, - and a link whose text holds an autolink keeps the inner link and leaves the outer brackets - literal text, which the spec requires and the reference itself breaks, nesting one `` in the - other. `corpus/commonmark-spec/exceptions.json` pins the first four `node-model` where the suite - reaches them, and the next two `pending`; the suite holds no example of the last. + And two more: a shortcut reference matching its definition only under Unicode case folding stays + unresolved, and a link whose text holds an autolink keeps the inner link and leaves the outer + brackets literal text, which the spec requires and the reference itself breaks, nesting one `` + in the other. `corpus/commonmark-spec/exceptions.json` pins the first four `node-model` where the + suite reaches them, and the next `pending`; the suite holds no example of the last. - Not every document converts back: `adfToLosslessMarkdown` refuses a document nesting past 500 levels, and the node shapes the `unsupported-node-shape` row lists. Show the refusal and keep the document read-only; saving markdown you could not produce is the loss the round-trip exists to diff --git a/corpus/commonmark-spec/exceptions.json b/corpus/commonmark-spec/exceptions.json index 10d6435..2f0e1bd 100644 --- a/corpus/commonmark-spec/exceptions.json +++ b/corpus/commonmark-spec/exceptions.json @@ -1,18 +1,4 @@ [ - { - "check": "count", - "divergence": "ul 2/1", - "example": 301, - "kind": "pending", - "reason": "A list continuing past a marker change renders as two lists, the parser opens one." - }, - { - "check": "count", - "divergence": "ol 2/1", - "example": 302, - "kind": "pending", - "reason": "A list continuing past a marker change renders as two lists, the parser opens one." - }, { "check": "count", "divergence": "em 2/1", diff --git a/corpus/errors/list-break-body.error b/corpus/errors/list-break-body.error deleted file mode 100644 index 4f35411..0000000 --- a/corpus/errors/list-break-body.error +++ /dev/null @@ -1 +0,0 @@ -malformed-directive diff --git a/corpus/errors/list-break-body.md b/corpus/errors/list-break-body.md deleted file mode 100644 index 8011a94..0000000 --- a/corpus/errors/list-break-body.md +++ /dev/null @@ -1,7 +0,0 @@ -- Bolt M8 - -!adf:listBreak -Packed. -!adf:/listBreak - -- Nut M8 diff --git a/corpus/errors/list-break-misplaced.error b/corpus/errors/list-break-misplaced.error deleted file mode 100644 index a811539..0000000 --- a/corpus/errors/list-break-misplaced.error +++ /dev/null @@ -1 +0,0 @@ -unsupported-node-shape diff --git a/corpus/errors/list-break-misplaced.md b/corpus/errors/list-break-misplaced.md deleted file mode 100644 index e938db7..0000000 --- a/corpus/errors/list-break-misplaced.md +++ /dev/null @@ -1,5 +0,0 @@ -- Bolt M8 - -!adf:listBreak - -Packed. diff --git a/corpus/normalization/list-markers.json b/corpus/normalization/list-markers.json index df8ae34..fe7f1da 100644 --- a/corpus/normalization/list-markers.json +++ b/corpus/normalization/list-markers.json @@ -85,6 +85,25 @@ } ], "type": "orderedList" + }, + { + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Spanner", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "listItem" + } + ], + "type": "bulletList" } ], "type": "doc", diff --git a/corpus/normalization/list-markers.md b/corpus/normalization/list-markers.md index ae8bc61..be0ac54 100644 --- a/corpus/normalization/list-markers.md +++ b/corpus/normalization/list-markers.md @@ -1,8 +1,9 @@ -- Bolt M8 +* Bolt M8 * Nut M8 -- +* -+ Washer M8 +* Washer M8 -1. Loosen the clamp +1) Loosen the clamp 1) Lift the cover ++ Spanner diff --git a/corpus/round-trip/block-nodes/adjacent-lists.md b/corpus/round-trip/block-nodes/adjacent-lists.md index 1c6b0d1..a04f222 100644 --- a/corpus/round-trip/block-nodes/adjacent-lists.md +++ b/corpus/round-trip/block-nodes/adjacent-lists.md @@ -1,23 +1,16 @@ - Bolt -!adf:listBreak - -- Nut +* Nut 1. Cut -!adf:listBreak - -1. Weld +1) Weld !adf:panel info - Left -!adf:listBreak -- Right + +* Right !adf:/panel - - Head - - !adf:listBreak - - - Tail + * Tail diff --git a/docs/decisions.md b/docs/decisions.md index 9550489..0693f92 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -68,9 +68,8 @@ same portable markdown. 2026-10-03, the maintainer. Goal 1. Valid while CommonMark reads adjacent text as one run. Two adjacent text nodes CommonMark would read back as one are parted by the reserved inline leaf -`!adf:textBreak{}`, mirroring `!adf:listBreak`: a leaf building no node keeps both nodes and asks -nothing of the text around it. A code span holds no directive, so the spans close and reopen -around the leaf. The grammar: `spec/flavour.md` §Inline nodes, **Adjacent text nodes**. +`!adf:textBreak{}`: a leaf building no node keeps both nodes and asks nothing of the text around +it. A code span holds no directive, so the spans close and reopen around the leaf. The grammar: `spec/flavour.md` §Inline nodes, **Adjacent text nodes**. ## An empty key spells `empty` @@ -430,9 +429,9 @@ handles one cause alike whichever node, attribute or direction raised it. when a later MINOR gives the name meaning. A reserved name is a known name, so never that code, and the names the flavour reserves part on form: a form the grammar does not have is a claim code — `!adf:carry`, whose carry is the fence — and a well-formed form in the wrong place is - `unsupported-node-shape`, `!adf:listBreak` parting anything but two adjacent lists of one type, - `!adf:textBreak{}` anything but two adjacent text nodes CommonMark would read back as one, - `!adf:doc` standing beside another block (2026-09-01, the text break and `doc` 2026-10-03). + `unsupported-node-shape`, `!adf:textBreak{}` parting anything but two adjacent text nodes + CommonMark would read back as one, `!adf:doc` standing beside another block (2026-09-01, the + text break and `doc` 2026-10-03). - A well-formed directive the node tables refuse — an attribute a node does not hold or spells elsewhere, a value outside its kind or its canonical spelling, an argument, or a body of a shape its content model does not take — is `unsupported-node-shape`, the emitter's code for the same diff --git a/spec/flavour.md b/spec/flavour.md index 02722c2..8d8a212 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -19,17 +19,14 @@ normalizes to it through the round-trip. single tilde or a run of three or more is literal text — and block structure resolves before inline, so a `~~~` line opens a CommonMark tilde code fence. - Bullet lists `- `; ordered lists incrementing `1.` `2.` `3.`, the first number the node's - `order` attribute. Continuation lines align with the first character after the marker - (two spaces for `- `, three for `1. `); blank lines inside an item are empty lines, none - between a nested list and a CommonMark block above it — one wherever the nested list's own - marker cannot interrupt a paragraph (an ordered list whose first number is not 1, or a list - whose first item is empty), whatever block sits above it. Blank lines between items normalize - away, and no list opens beside one of its own kind — the marker change CommonMark starts a - second list on merges instead: ADF records no tightness, so one `- ` spelling reads two - adjacent lists of a kind back as one. The leaf `!adf:listBreak` parts them, taking the separation - any directive block takes where it sits. It builds no node, and it reads only between two - adjacent lists of one type: elsewhere, or carrying an argument, `{attrs}` or a body, it is a - named error. A list whose item holds a line of spaces or tabs alone, which a list item reads + `order` attribute. A list right after a list of its kind takes the other marker, `* ` or `1)`, + since CommonMark starts a new list where the bullet character or the delimiter changes; the + list after that takes `- ` or `1.` again. Continuation lines align with the first character + after the marker (two spaces for `- `, three for `1. `); blank lines inside an item are empty + lines, none between a nested list and a CommonMark block above it — one wherever the nested + list's own marker cannot interrupt a paragraph (an ordered list whose first number is not 1, or + a list whose first item is empty), whatever block sits above it. Blank lines between items + normalize away. A list whose item holds a line of spaces or tabs alone, which a list item reads back empty, takes the directive form. - Blockquotes prefix lines with `> `; a blank line inside a blockquote is a bare `>`. - ATX headings (`#` … `######`); setext input normalizes to ATX. @@ -84,10 +81,9 @@ result naming it at the opener, whatever follows it — so output an old emitter escaped, and erroring input gaining meaning later is MINOR, never a reparse (`docs/decisions.md` §The formats are API). Each name belongs to one position, and a name the other one spells — a mark or an inline node written as a block directive, a block node written inline — is a different error, -naming the spelling it takes. Four reserved names read back to no node: `carry` for the inline -opaque carry, `listBreak` for the leaf that parts two adjacent lists and `doc` for a document -holding no `content` key (Canonical form), and `textBreak` for the leaf that parts two text nodes -(Inline nodes). Every fence info string opening `adf:` is reserved for the block carry (The opaque +naming the spelling it takes. Three reserved names read back to no node: `carry` for the inline +opaque carry, `doc` for a document holding no `content` key (Canonical form), and `textBreak` for +the leaf that parts two text nodes (Inline nodes). Every fence info string opening `adf:` is reserved for the block carry (The opaque carry). **Claiming**: an unescaped `!adf:` claims wherever it stands. What follows picks the form: `/name` diff --git a/src/conformance/markdown-property.test.ts b/src/conformance/markdown-property.test.ts index 2b01b2b..126d4f6 100644 --- a/src/conformance/markdown-property.test.ts +++ b/src/conformance/markdown-property.test.ts @@ -10,7 +10,7 @@ import type { Result } from '../result.ts' import { adfDocument, attributes, blockTaskItemChildren, jsonKey, jsonValue, markdownPieces, propertyRuns, propertyTimeout, textOf } from './property-harness.ts' import { adfToLosslessMarkdown } from '../markdown/emit/adf-to-markdown.ts' import { adfToPortableMarkdown } from '../markdown/portable/adf-to-portable-markdown.ts' -import { blockArgument, listBreakName, marksAttribute } from '../markdown/block-directive.ts' +import { blockArgument, marksAttribute } from '../markdown/block-directive.ts' import { blockNodes } from '../adf/block-nodes.ts' import { carryName } from '../markdown/opaque-carry.ts' import { commonMarkToAdf, losslessMarkdownToAdf, portableMarkdownToAdf } from '../markdown/parse/markdown-to-adf.ts' @@ -52,7 +52,7 @@ const vocabularies = [...Object.values(blockNodes).map((model) => model.attribut const attributeKeys = [ ...new Set([...vocabularies.flatMap((vocabulary) => Object.keys(vocabulary)), ...Object.keys(blockNodes).flatMap((type) => blockArgument(type) ?? []), marksAttribute, 'json', textDirectiveName]), ] -const directiveNames = [...Object.keys(blockNodes), ...Object.keys(inlineNodes), ...Object.keys(markAttributes), carryName, listBreakName, textDirectiveName] +const directiveNames = [...Object.keys(blockNodes), ...Object.keys(inlineNodes), ...Object.keys(markAttributes), carryName, textDirectiveName] // Hostile generation reaches refusals; clean generation holds none a single piece would trip, so a whole document reaches the emitter. function choose(hostile: boolean, choices: readonly Choice[], depth?: { depthIdentifier: DepthIdentifier; maxDepth: number }): Arbitrary { diff --git a/src/markdown/block-directive.ts b/src/markdown/block-directive.ts index 83ff2a0..6681d5c 100644 --- a/src/markdown/block-directive.ts +++ b/src/markdown/block-directive.ts @@ -21,10 +21,6 @@ export const documentAttribute = { key: 'content', value: 'none' } export const documentSpelling = spellDirectiveOpener(documentName, undefined, spellAttributes([[documentAttribute.key, documentAttribute.value]])) -export const listBreakName = 'listBreak' - -export const listBreakSpelling = spellDirectiveOpener(listBreakName, undefined, '') - export const marksAttribute = 'marks' export function blockArgument(type: string): string | undefined { @@ -32,7 +28,7 @@ export function blockArgument(type: string): string | undefined { } export function blockDirectiveForm(name: string): 'container' | 'leaf' | undefined { - if (name === listBreakName || name === documentName) return 'leaf' + if (name === documentName) return 'leaf' const model = blockNodeModel(name) if (model === undefined) return undefined return model.contentModel === 'none' ? 'leaf' : 'container' diff --git a/src/markdown/commonmark/grammar.ts b/src/markdown/commonmark/grammar.ts index 6ceb77c..2aab6e3 100644 --- a/src/markdown/commonmark/grammar.ts +++ b/src/markdown/commonmark/grammar.ts @@ -65,7 +65,7 @@ const firstCharacterOpeners = [atxHeadingOpener, /^>/, bulletListOpener, codeFen const emailNameSource = "[A-Za-z0-9.!#$%&'*+/=?^_`{|}~-]+" const emailLabelSource = '[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?' const emailAutolink = new RegExp(`<${emailNameSource}@${emailLabelSource}(?:\\.${emailLabelSource})*>`, 'y') -const orderedListOpener = /^(\d{1,9})(?:[.)])(?:[ \t]|$)/ +const orderedListOpener = /^(\d{1,9})([.)])(?:[ \t]|$)/ const setextUnderline = /^(=+|-+)[ \t]*$/ const unicodeWhitespace = /[\t\n\f\r \p{Zs}]/u @@ -207,13 +207,13 @@ export function isUnicodeWhitespace(character: string): boolean { } // `start` is the list's first number, `undefined` for a bullet. -export function listMarker(line: string): { start: number | undefined; width: number } | undefined { +export function listMarker(line: string): { delimiter: string; start: number | undefined; width: number } | undefined { const ordered = orderedListOpener.exec(line) if (ordered !== null) { const digits = ordered[1] ?? '' - return { start: Number(digits), width: digits.length + 1 } + return { delimiter: ordered[2] ?? '', start: Number(digits), width: digits.length + 1 } } - return bulletListOpener.test(line) ? { start: undefined, width: 1 } : undefined + return bulletListOpener.test(line) ? { delimiter: line.charAt(0), start: undefined, width: 1 } : undefined } export function markerInterruptsParagraph(start: number | undefined, empty: boolean): boolean { diff --git a/src/markdown/emit/adf-to-markdown.test.ts b/src/markdown/emit/adf-to-markdown.test.ts index 2b85b86..13e56a1 100644 --- a/src/markdown/emit/adf-to-markdown.test.ts +++ b/src/markdown/emit/adf-to-markdown.test.ts @@ -166,14 +166,17 @@ test('escapes the delimiter row a hard break leaves opening a pipe table with no assert.equal(broken('c | d'), 'a | b\\\nc | d\n') }) -test('parts two adjacent lists of the same kind, the marker spelling being what merges', () => { +test('alternates the marker between adjacent lists of a kind, the marker change being what parts them', () => { const list: AdfNode = { content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }], type: 'bulletList' } - assert.equal(markdown(adfToLosslessMarkdown(document(list, list))), '- x\n\n!adf:listBreak\n\n- x\n') - const ordered: AdfNode = { attrs: { order: 1 }, content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }], type: 'orderedList' } - assert.equal(markdown(adfToLosslessMarkdown(document(ordered, ordered))), '1. x\n\n!adf:listBreak\n\n1. x\n') - assert.equal(markdown(adfToLosslessMarkdown(document({ attrs: { panelType: 'info' }, content: [list, list], type: 'panel' }))), '!adf:panel info\n- x\n!adf:listBreak\n- x\n!adf:/panel\n') + assert.equal(markdown(adfToLosslessMarkdown(document(list, list, list))), '- x\n\n* x\n\n- x\n') + const ordered: AdfNode = { attrs: { order: 9 }, content: [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }, { content: [paragraph({ text: 'y', type: 'text' }), paragraph({ text: 'z', type: 'text' })], type: 'listItem' }], type: 'orderedList' } + assert.equal(markdown(adfToLosslessMarkdown(document(ordered, ordered))), '9. x\n10. y\n\n z\n\n9) x\n10) y\n\n z\n') + assert.equal(markdown(adfToLosslessMarkdown(document(list, ordered, list))), '- x\n\n9. x\n10. y\n\n z\n\n- x\n') + assert.equal(markdown(adfToLosslessMarkdown(document({ attrs: { panelType: 'info' }, content: [list, list], type: 'panel' }))), '!adf:panel info\n- x\n\n* x\n!adf:/panel\n') const nested: AdfNode = { content: [{ content: [list, list], type: 'listItem' }], type: 'bulletList' } - assert.equal(markdown(adfToLosslessMarkdown(document(nested))), '- - x\n\n !adf:listBreak\n\n - x\n') + assert.equal(markdown(adfToLosslessMarkdown(document(nested))), '- - x\n * x\n') + const stars: AdfNode = { content: [{ content: [paragraph({ text: '* *', type: 'text' })], type: 'listItem' }], type: 'bulletList' } + assert.equal(markdown(adfToLosslessMarkdown(document(list, stars))), '- x\n\n* \\* *\n') const carried: AdfNode = { ...list, attrs: { unknown: 'x' } } assert.ok(markdown(adfToLosslessMarkdown(document(carried, carried))).includes('```\n\n```adf:bulletList\n')) assert.ok(markdown(adfToLosslessMarkdown(document(carried, list))).endsWith('```\n\n- x\n')) diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index 00f4008..a71a65a 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -3,7 +3,7 @@ import type { BlockNodeModel } from '../../adf/block-nodes.ts' import type { WrittenFlavour } from '../portable/conventions.ts' import { adfDocumentFault, holdsOnlyAttributes, isUnmarkedSpellableText, nodeAttrs, nodeContent } from '../../adf/document.ts' import { alertMarker, flavourClaims, foldedAlertMarker, leadingMarker, readAlertMarker, readTaskMarker, taskMarker } from '../portable/conventions.ts' -import { blockDirectiveForm, documentSpelling, listBreakSpelling } from '../block-directive.ts' +import { blockDirectiveForm, documentSpelling } from '../block-directive.ts' import { blockNodeModel, blockNodes, isTaskItem } from '../../adf/block-nodes.ts' import { carriedBlock } from '../opaque-carry.ts' import { emitInlineLine } from './inline-line.ts' @@ -18,18 +18,23 @@ import { tryImage } from './image.ts' import { tryPipeTable } from './pipe-table.ts' type BlockContainer = 'directive' | 'document' | 'list-item' -type BlockSpelling = 'commonmark' | 'directive' | 'list' +// The usual delimiter, then the one a list beside another of its kind takes. +type Delimiters = readonly [string, string] +// CommonMark starts a new list where the marker changes, so a list beside one of its kind takes `alternateText`. +type SpelledBlock = { spelling: 'commonmark' | 'directive'; text: string } | { alternateText: string; spelling: 'list'; text: string } // headroom: the nesting levels the deepest node below may still spend before `largestNesting` refuses it. -type EmittedBlock = { headroom: number; spelling: BlockSpelling; text: string } +type EmittedBlock = SpelledBlock & { headroom: number } type KeptSpelling = { block: EmittedBlock | undefined; depth: number } -type PlacedBlock = Omit & { node: AdfNode } +type PlacedBlock = SpelledBlock & { node: AdfNode } type PlacedBlocks = { blocks: readonly PlacedBlock[]; headroom: number } // Keyed by reference: only a caller building one object per position (the parse, the portable reduction) passes one; a consumer's document may share a node. export type SpellingMemo = Map type WalkedItem = { node: AdfNode; walk: PlacedBlocks } export type Writing = { flavour: WrittenFlavour; memo: SpellingMemo | undefined } +const bulletDelimiters: Delimiters = ['-', '*'] export const largestListMarker = 999999999 +const orderedDelimiters: Delimiters = ['.', ')'] // Bare because tryList admits no item carrying attributes, marks or text. const listItemOpener = spellDirectiveOpener('listItem', undefined, '') @@ -67,23 +72,23 @@ function tooDeep(path: ConvertErrorPath): Result { function joinBlocks(blocks: readonly PlacedBlock[], container: BlockContainer): string { let text = '' + let alternated = false for (const [index, block] of blocks.entries()) { const previous = blocks[index - 1] if (previous !== undefined) text += separationBetween(previous, block, container) - text += block.text + alternated = !alternated && previous?.spelling === 'list' && block.spelling === 'list' && isOrdered(previous.node) === isOrdered(block.node) + text += alternated && block.spelling === 'list' ? block.alternateText : block.text } return text } +function isOrdered(node: AdfNode): boolean { + return node.type === 'orderedList' +} + function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: BlockContainer): string { const bothCommonMark = previous.spelling !== 'directive' && next.spelling !== 'directive' - if (bothCommonMark && next.spelling === 'list') { - if (previous.spelling === 'list' && previous.node.type === next.node.type) { - const gap = container === 'directive' ? '\n' : '\n\n' - return `${gap}${listBreakSpelling}${gap}` - } - if (container === 'list-item') return interruptsParagraph(next.node) ? '\n' : '\n\n' - } + if (bothCommonMark && next.spelling === 'list' && container === 'list-item') return interruptsParagraph(next.node) ? '\n' : '\n\n' return container === 'directive' && !bothCommonMark ? '\n' : '\n\n' } @@ -175,9 +180,14 @@ function tryTaskList(node: AdfNode, path: ConvertErrorPath, depth: number, writi if (task || previous === undefined) items.push([...walk.value.blocks]) else for (const block of walk.value.blocks) previous.push(block) } - const lines = items.map((blocks) => tryListItemLines(joinBlocks(blocks, 'list-item'), '- ')) + const list = spelledList( + items.map((blocks) => joinBlocks(blocks, 'list-item')), + () => '', + bulletDelimiters, + headroom, + ) // The portable reduction leaves no task a list item cannot hold: a directive here would break the flavour. - return lines.includes(undefined) ? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path) : success({ headroom, spelling: 'list', text: lines.join('\n') }) + return list === undefined ? failure('unsupported-node-shape', 'a task holds blocks no list item spells', path) : success(list) } function placedBlock(node: AdfNode, path: ConvertErrorPath, depth: number, writing: Writing): Result { @@ -309,20 +319,32 @@ function tryList(node: AdfNode, path: ConvertErrorPath, depth: number, writing: headroom = Math.min(headroom, walk.value.headroom) walked.push({ node: item, walk: walk.value }) } - const lines: string[] = [] - for (const [offset, item] of walked.entries()) { + const inners = walked.map((item) => { const inner = joinBlocks(item.walk.blocks, 'list-item') // GitHub reads a task marker opening any item's first paragraph as a checkbox, whatever its siblings hold. - const escaped = flavourClaims[writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined ? `\\${inner}` : inner - const line = tryListItemLines(escaped, ordered ? `${start + offset}. ` : '- ') - if (line === undefined) { - // The directive form spends a level the walk did not count. - if (headroom < 1) return tooDeep(path) - return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, path, depth, () => success({ blocks: directiveItems(walked), headroom: headroom - 1 })) - } + return flavourClaims[writing.flavour].taskMarkers && leadingMarker(inner, readTaskMarker) !== undefined ? `\\${inner}` : inner + }) + const list = ordered ? spelledList(inners, (offset) => `${start + offset}`, orderedDelimiters, headroom) : spelledList(inners, () => '', bulletDelimiters, headroom) + if (list !== undefined) return success(list) + // The directive form spends a level the walk did not count. + if (headroom < 1) return tooDeep(path) + return emitDirectiveBlock(node, ordered ? blockNodes.orderedList : blockNodes.bulletList, path, depth, () => success({ blocks: directiveItems(walked), headroom: headroom - 1 })) +} + +function spelledList(inners: readonly string[], number: (offset: number) => string, [usual, alternate]: Delimiters, headroom: number): EmittedBlock | undefined { + const text = listLines(inners, number, usual) + const alternateText = listLines(inners, number, alternate) + return text === undefined || alternateText === undefined ? undefined : { alternateText, headroom, spelling: 'list', text } +} + +function listLines(inners: readonly string[], number: (offset: number) => string, delimiter: string): string | undefined { + const lines: string[] = [] + for (const [offset, inner] of inners.entries()) { + const line = tryListItemLines(inner, `${number(offset)}${delimiter} `) + if (line === undefined) return undefined lines.push(line) } - return success({ headroom, spelling: 'list', text: lines.join('\n') }) + return lines.join('\n') } function directiveItems(items: readonly WalkedItem[]): PlacedBlock[] { diff --git a/src/markdown/parse/blocks.test.ts b/src/markdown/parse/blocks.test.ts index 2152213..b9ccd62 100644 --- a/src/markdown/parse/blocks.test.ts +++ b/src/markdown/parse/blocks.test.ts @@ -84,7 +84,6 @@ test('carries a claimed line as the block it opens, the refusal the node layer b test('opens a container where the content model takes content, and holds it open until the closer naming it', () => { assert.deepEqual(kinds('!adf:panel info\nPart.\n!adf:/panel\nMore.\n'), ['directive', 'paragraph']) assert.deepEqual(kinds('!adf:rule\nPart.\n'), ['directive', 'paragraph']) - assert.deepEqual(kinds('!adf:listBreak\nPart.\n'), ['directive', 'paragraph']) assert.deepEqual(kinds('!adf:widget\nPart.\n'), ['directive', 'paragraph']) assert.deepEqual(kinds('!adf:mention\nPart.\n'), ['directive', 'paragraph']) assert.deepEqual(faults('!adf:panel info\n\nPart.\n\n!adf:/panel\n'), []) diff --git a/src/markdown/parse/blocks.ts b/src/markdown/parse/blocks.ts index e3bdee6..9198748 100644 --- a/src/markdown/parse/blocks.ts +++ b/src/markdown/parse/blocks.ts @@ -29,8 +29,8 @@ export type Block = { position: SourcePosition } & ( | { blocks: Block[]; kind: 'blockquote' } | { construct: string; kind: 'html' } | { fault: ConvertFault; kind: 'fault' } - | { items: Block[][]; kind: 'bulletList' } - | { items: Block[][]; kind: 'orderedList'; start: number } + | { delimiter: string; items: Block[][]; kind: 'bulletList' } + | { delimiter: string; items: Block[][]; kind: 'orderedList'; start: number } | { kind: 'code'; language: string; text: string } | { kind: 'heading'; level: number; text: string } | { kind: 'paragraph'; text: string } @@ -214,19 +214,23 @@ function itemStart(line: Line, opener: Line, paragraphOpen: boolean, enclosing: if (paragraphOpen && !markerInterruptsParagraph(marker.start, blank)) return undefined const spaces = leadingColumns(after) const padding = blank || spaces > indentedCodeColumns ? 1 : spaces - const kind = marker.start === undefined ? 'bulletList' : 'orderedList' - const continued = enclosing?.kind === 'item' && enclosing.list.kind === kind + const list = openList(marker.delimiter, marker.start, position) + const continued = enclosing?.kind === 'item' && continuesList(enclosing.list, list) return { fresh: !continued, indentation: leadingColumns(line) + marker.width + padding, kind: 'item', - list: continued ? enclosing.list : openList(marker.start, position), + list: continued ? enclosing.list : list, rest: blank ? after : removeColumns(after, padding), } } -function openList(start: number | undefined, position: SourcePosition): ListBlock { - return start === undefined ? { items: [], kind: 'bulletList', position } : { items: [], kind: 'orderedList', position, start } +function openList(delimiter: string, start: number | undefined, position: SourcePosition): ListBlock { + return start === undefined ? { delimiter, items: [], kind: 'bulletList', position } : { delimiter, items: [], kind: 'orderedList', position, start } +} + +function continuesList(list: ListBlock, next: ListBlock): boolean { + return list.kind === next.kind && list.delimiter === next.delimiter } function openContainer(walk: Walk, start: ContainerStart): void { @@ -242,12 +246,12 @@ function openContainer(walk: Walk, start: ContainerStart): void { walk.stack.push({ blocks, indentation: start.indentation, kind: 'item', list }) } -// Two lists of a kind never sit adjacent: one `- ` spelling reads them back as one (spec/flavour.md). +// An item after a closed one, such as an empty item a blank line ends, continues the list beside it. function openedList(walk: Walk, start: Extract): ListBlock { if (!start.fresh) return start.list const blocks = currentBlocks(walk) const previous = blocks.at(-1) - if ((previous?.kind === 'bulletList' || previous?.kind === 'orderedList') && previous.kind === start.list.kind) return previous + if ((previous?.kind === 'bulletList' || previous?.kind === 'orderedList') && continuesList(previous, start.list)) return previous blocks.push(start.list) return start.list } diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index b284a07..9a2ea39 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -111,7 +111,6 @@ test('reads each shape a flavour claims as the flavour of the reader it reaches' ['!adf:panel info\nx\n!adf:/panel\n', [paragraph('!adf:panel info x !adf:/panel')], [{ attrs: { panelType: 'info' }, content: [paragraph('x')], type: 'panel' }], [paragraph('!adf:panel info x !adf:/panel')]], ['a!adf:textBreak{}a\n', [paragraph('a!adf:textBreak{}a')], [{ content: [text('a'), text('a')], type: 'paragraph' }], [paragraph('a!adf:textBreak{}a')]], ['!adf:doc {content=none}\n', [paragraph('!adf:doc {content=none}')], [], [paragraph('!adf:doc {content=none}')]], - ['- a\n\n!adf:listBreak\n\n- b\n', [bulletList(item(paragraph('a'))), paragraph('!adf:listBreak'), bulletList(item(paragraph('b')))], [bulletList(item(paragraph('a'))), bulletList(item(paragraph('b')))], [bulletList(item(paragraph('a'))), paragraph('!adf:listBreak'), bulletList(item(paragraph('b')))]], ['| a |\n| --- |\n| b |\n', [paragraph('| a | | --- | | b |')], [table(row(cell('tableHeader', text('a'))), row(cell('tableCell', text('b'))))], [table(row(cell('tableHeader', text('a'))), row(cell('tableCell', text('b'))))]], ['~~x~~\n', [paragraph('~~x~~')], [{ content: [marked('x', strike)], type: 'paragraph' }], [{ content: [marked('x', strike)], type: 'paragraph' }]], ['==x==\n', [paragraph('==x==')], [paragraph('==x==')], [{ content: [marked('x', highlight)], type: 'paragraph' }]], @@ -271,27 +270,22 @@ test('gives back the refusal an inline body holds, never the shape check above i assert.equal(content(losslessMarkdownToAdf('!adf:caption\n- a\n!adf:/caption\n')), 'unsupported-node-shape: caption takes one paragraph as its body: this body is not one') }) -test('reads the separator that parts two adjacent lists of one kind', () => { +test('reads a change of bullet character or ordered delimiter as a second list, in every reader', () => { const parted = [bulletList(item(paragraph('a'))), bulletList(item(paragraph('b')))] - assert.deepEqual(content(losslessMarkdownToAdf('- a\n\n!adf:listBreak\n\n- b\n')), parted) - assert.deepEqual(content(losslessMarkdownToAdf('- a\n!adf:listBreak\n- b\n')), parted) - assert.deepEqual(content(losslessMarkdownToAdf('1. a\n\n!adf:listBreak\n\n1. b\n')), [orderedList(1, item(paragraph('a'))), orderedList(1, item(paragraph('b')))]) - assert.deepEqual(content(losslessMarkdownToAdf('> - a\n> !adf:listBreak\n> - b\n')), [quote(...parted)]) - assert.deepEqual(path(losslessMarkdownToAdf('- a\n\n!adf:listBreak\n\n- b\n\n| x |\n')), ['content', 2]) + for (const read of [commonMarkToAdf, losslessMarkdownToAdf, portableMarkdownToAdf]) { + assert.deepEqual(content(read('- a\n* b\n')), parted) + assert.deepEqual(content(read('- a\n\n+ b\n')), parted) + assert.deepEqual(content(read('- a\n-\n\n* b\n')), [bulletList(item(paragraph('a')), item()), bulletList(item(paragraph('b')))]) + assert.deepEqual(content(read('- a\n* b\n- c\n')), [...parted, bulletList(item(paragraph('c')))]) + assert.deepEqual(content(read('1. a\n1) b\n')), [orderedList(1, item(paragraph('a'))), orderedList(1, item(paragraph('b')))]) + assert.deepEqual(content(read('> - a\n> * b\n')), [quote(...parted)]) + assert.deepEqual(content(read('- - a\n * b\n')), [bulletList(item(...parted))]) + } + assert.deepEqual(path(losslessMarkdownToAdf('- a\n\n* b\n\n| x |\n')), ['content', 2]) }) -test('refuses the list separator that parts anything else', () => { - const parts = 'unsupported-node-shape: listBreak parts two adjacent lists of one type: this one parts something else' - assert.equal(content(losslessMarkdownToAdf('!adf:listBreak\n')), parts) - assert.equal(content(losslessMarkdownToAdf('- a\n\n!adf:listBreak\n')), parts) - assert.equal(content(losslessMarkdownToAdf('- a\n\n!adf:listBreak\n\n1. b\n')), parts) - assert.equal(content(losslessMarkdownToAdf('Part.\n\n!adf:listBreak\n\n- b\n')), parts) - const bare = 'unsupported-node-shape: listBreak spells the bare leaf form, !adf:listBreak: this one spells more' - assert.equal(content(losslessMarkdownToAdf('- a\n\n!adf:listBreak x\n\n- b\n')), bare) - assert.equal(content(losslessMarkdownToAdf('- a\n\n!adf:listBreak {id=x}\n\n- b\n')), bare) - assert.equal(content(losslessMarkdownToAdf('- a\n\n!adf:listBreak\n- b\n!adf:/listBreak\n')), 'malformed-directive: listBreak takes no body, so no !adf:/listBreak closes it; \\!adf: keeps the prefix literal') - assert.equal(content(losslessMarkdownToAdf('!adf:listBreak{}\n')), 'unsupported-node-shape: listBreak takes the block form, !adf:listBreak, never the inline form') - assert.deepEqual(path(losslessMarkdownToAdf('Part.\n\n!adf:listBreak\n')), ['content', 1]) +test('reads the retired list separator as a directive no node spells', () => { + assert.equal(content(losslessMarkdownToAdf('- a\n\n!adf:listBreak\n\n- b\n')), 'unknown-directive-name: the directive name listBreak reads back to no node; \\!adf: keeps the prefix literal') }) test('reads the image a pipe cell holds as its linked alt text', () => { @@ -382,8 +376,6 @@ test('names the leaf given a body at its opener, ahead of any refusal the leaf h const body = (name: string): string => `malformed-directive: ${name} takes no body, so no !adf:/${name} closes it; \\!adf: keeps the prefix literal` assert.equal(content(losslessMarkdownToAdf('!adf:rule\nPart.\n!adf:/rule\n')), body('rule')) assert.deepEqual(position(losslessMarkdownToAdf('Part.\n\n!adf:rule\nPart.\n!adf:/rule\n')), { line: 3, offset: 7 }) - assert.equal(content(losslessMarkdownToAdf('- a\n\n!adf:listBreak\n!adf:/listBreak\n\n- b\n')), body('listBreak')) - assert.equal(content(losslessMarkdownToAdf('- a\n\n!adf:listBreak\nPart.\n!adf:/listBreak\n\n- b\n')), body('listBreak')) assert.deepEqual(path(losslessMarkdownToAdf('!adf:rule {localId=a-1}\n!adf:rule {localId=a-2}\nPart.\n!adf:/rule\n')), ['content', 1]) assert.deepEqual(path(losslessMarkdownToAdf('!adf:rule {localId=a-1}\n!adf:/rule\n!adf:/rule\n')), ['content', 0]) assert.deepEqual(path(losslessMarkdownToAdf('Part.\n\n!adf:/rule\n')), ['content', 1]) @@ -683,8 +675,6 @@ test('reads a bullet list, the marker width setting the continuation', () => { assert.deepEqual(content(losslessMarkdownToAdf('-\n')), [bulletList(item())]) assert.deepEqual(content(losslessMarkdownToAdf('- One\n\n Two.\n')), [bulletList(item(paragraph('One'), paragraph('Two.')))]) assert.deepEqual(content(losslessMarkdownToAdf('- Code.\n')), [bulletList(item({ content: [text('Code.')], type: 'codeBlock' }))]) - assert.deepEqual(content(losslessMarkdownToAdf('- a\n* b\n')), [bulletList(item(paragraph('a')), item(paragraph('b')))]) - assert.deepEqual(content(losslessMarkdownToAdf('- a\n\n+ b\n')), [bulletList(item(paragraph('a')), item(paragraph('b')))]) assert.deepEqual(content(losslessMarkdownToAdf('- a\n-\n\n- c\n')), [bulletList(item(paragraph('a')), item(), item(paragraph('c')))]) assert.deepEqual(content(losslessMarkdownToAdf('- a\n1. b\n')), [bulletList(item(paragraph('a'))), orderedList(1, item(paragraph('b')))]) assert.deepEqual(content(losslessMarkdownToAdf('- a\n\n[r]: /u\n\n- b\n')), [bulletList(item(paragraph('a')), item(paragraph('b')))]) @@ -694,7 +684,6 @@ test('reads a bullet list, the marker width setting the continuation', () => { test('reads an ordered list, its first marker the order attribute', () => { assert.deepEqual(content(losslessMarkdownToAdf('9. Bolt M8\n10. Nut M8\n')), [orderedList(9, item(paragraph('Bolt M8')), item(paragraph('Nut M8')))]) assert.deepEqual(content(losslessMarkdownToAdf('1) Loosen the clamp\n')), [orderedList(1, item(paragraph('Loosen the clamp')))]) - assert.deepEqual(content(losslessMarkdownToAdf('1. a\n1) b\n')), [orderedList(1, item(paragraph('a')), item(paragraph('b')))]) assert.deepEqual(content(losslessMarkdownToAdf('0. Zero\n')), [orderedList(0, item(paragraph('Zero')))]) }) diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 59ec8c9..c8f56f8 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -7,7 +7,7 @@ import type { LineContainer } from '../line-container.ts' import type { LinkDefinitions } from './inline-content.ts' import { carryFenceType, readCarriedBlock } from '../opaque-carry.ts' import { commonMarkSpelling, type SpellingMemo } from '../emit/adf-to-markdown.ts' -import { documentAttribute, documentName, documentSpelling, listBreakName, listBreakSpelling } from '../block-directive.ts' +import { documentAttribute, documentName, documentSpelling } from '../block-directive.ts' import { emptyKeys, nodeAttrs } from '../../adf/document.ts' import { failure, faulted, positioned, success, type ConvertErrorPath, type ParseError, type Result, type SourcePosition } from '../../result.ts' import { flavourClaims, leadingMarker, readAlertMarker, readTaskMarker } from '../portable/conventions.ts' @@ -70,17 +70,12 @@ function typeName(value: unknown): string { function readBlocks(blocks: readonly ReadBlock[], reading: Reading, path: ConvertErrorPath, depth: number): Result { if (depth > largestNesting) return failure('unsupported-nesting-depth', `the input nests deeper than the ${largestNesting} levels the parser carries`, path) const content: AdfNode[] = [] - for (const [index, block] of blocks.entries()) { + for (const block of blocks) { const nodePath = [...path, 'content', content.length] if (block.kind === 'directive' && block.name === documentName) { const fault = documentFault(block) ?? unsupportedNodeShape(`delete the ${documentSpelling} line to give the document content: it stands only as the whole document`) return positioned(faulted(fault, nodePath), block.position) } - if (block.kind === 'directive' && block.name === listBreakName) { - const fault = listBreakFault(block, blocks[index - 1], blocks[index + 1]) - if (fault !== undefined) return positioned(faulted(fault, nodePath), block.position) - continue - } if (block.kind === 'bulletList' && reading.claims.taskMarkers) { const listed = positioned(bulletBlocks(block.items, reading, nodePath, depth), block.position) if (!listed.ok) return listed @@ -94,15 +89,6 @@ function readBlocks(blocks: readonly ReadBlock[], reading: Reading, path: Conver return success(content) } -// spec/flavour.md, Directives: the separator builds no node, so only the pair it parts spells it. -function listBreakFault(block: DirectiveBlock, previous: ReadBlock | undefined, next: ReadBlock | undefined): ConvertFault | undefined { - if (block.argument !== undefined || block.attributes.size > 0) { - return unsupportedNodeShape(`${listBreakName} spells the bare leaf form, ${listBreakSpelling}: this one spells more`) - } - if (previous?.kind !== 'bulletList' && previous?.kind !== 'orderedList') return partsFault() - return previous.kind === next?.kind ? undefined : partsFault() -} - function documentFault(block: DirectiveBlock): ConvertFault | undefined { const spelled = block.attributes.get(documentAttribute.key) if (block.argument === undefined && block.attributes.size === 1 && spelled?.spelling === documentAttribute.value) return undefined @@ -112,10 +98,6 @@ function documentFault(block: DirectiveBlock): ConvertFault | undefined { return unsupportedNodeShape(`${documentName} spells the one form ${documentSpelling}: this one spells another`) } -function partsFault(): ConvertFault { - return unsupportedNodeShape(`${listBreakName} parts two adjacent lists of one type: this one parts something else`) -} - function readBlock(block: ReadBlock, reading: Reading, path: ConvertErrorPath, depth: number): Result { switch (block.kind) { case 'blockquote': diff --git a/src/markdown/parse/portable-markdown-to-adf.test.ts b/src/markdown/parse/portable-markdown-to-adf.test.ts index f1fd1e9..5f8ca02 100644 --- a/src/markdown/parse/portable-markdown-to-adf.test.ts +++ b/src/markdown/parse/portable-markdown-to-adf.test.ts @@ -180,6 +180,12 @@ test('stands a block a task item cannot hold, and what follows it in the item, a ]) }) +test('reads a task list beside a list of another bullet character as two lists', () => { + const tasks = (state: string, value: string): AdfNode => bare('taskList', node('taskItem', { state }, text(value))) + assert.deepEqual(read('- [x] a\n* b\n'), [tasks('DONE', 'a'), bare('bulletList', bare('listItem', bare('paragraph', text('b'))))]) + assert.deepEqual(read('- [x] a\n* [ ] b\n'), [tasks('DONE', 'a'), tasks('TODO', 'b')]) +}) + test('leaves mixed, ordered and unmarked lists plain', () => { for (const markdown of ['- \\[x] a\n', '- [x] a\n- \\[ ] b\n', '- [x] a\n- b\n', '1. [x] a\n', '- [x]a\n', '- **[x]** a\n', '- [x]**a**\n', '- [-] a\n', '- > [x] a\n']) { const blocks = read(markdown) diff --git a/src/markdown/portable/adf-to-portable-markdown.test.ts b/src/markdown/portable/adf-to-portable-markdown.test.ts index 1ab0539..c43025d 100644 --- a/src/markdown/portable/adf-to-portable-markdown.test.ts +++ b/src/markdown/portable/adf-to-portable-markdown.test.ts @@ -127,7 +127,7 @@ test('spells a task list as a bullet list whose items lead with their state', () assert.equal(portable(node('taskList', {}, nested, task('DONE', ''), said('Stray'))), '- - [ ] Review\n- \\[x]\n- Stray\n') assert.equal(portable(node('taskList', {}, node('taskList', {}), task('DONE', 'a'), said('b'), node('taskList', {}, task('TODO', 'c')))), '- \\[x] a\n- b\n - [ ] c\n') assert.equal(portable(node('taskList', {}, paragraph(), task('DONE', 'a'))), '- [x] a\n') - assert.equal(portable(bulletList(item(said('x'))), node('taskList', {}, task('DONE', 'a'), node('taskList', {}, task('TODO', 'c')))), '- x\n- \\[x] a\n - [ ] c\n') + assert.equal(portable(bulletList(item(said('x'))), node('taskList', {}, task('DONE', 'a'), node('taskList', {}, task('TODO', 'c')))), '- x\n\n* [x] a\n - [ ] c\n') assert.equal(portable(node('taskList', {}), task('TODO', 'Loose')), 'Loose\n') const blockTask = node('blockTaskItem', { state: 'DONE' }, said('First.'), said('Second.')) const codeTask = node('blockTaskItem', { state: 'TODO' }, { content: [text('x')], type: 'codeBlock' }) @@ -135,9 +135,9 @@ test('spells a task list as a bullet list whose items lead with their state', () const parent = node('blockTaskItem', { state: 'TODO' }, said('q'), said('q2')) assert.equal(portable(node('taskList', {}, parent, node('taskList', {}, task('DONE', 'n')), task('DONE', 'B'))), '- [ ] q\n\n q2\n - [x] n\n- [x] B\n') const holding = node('blockTaskItem', { state: 'TODO' }, said('A'), node('taskList', {}, task('TODO', 'c'))) - assert.equal(portable(node('taskList', {}, holding, node('taskList', {}, task('TODO', 'd')), task('TODO', 'B'))), '- [ ] A\n - [ ] c\n - [ ] d\n- [ ] B\n') + assert.equal(portable(node('taskList', {}, holding, node('taskList', {}, task('TODO', 'd')), task('TODO', 'B'))), '- [ ] A\n - [ ] c\n * [ ] d\n- [ ] B\n') const listTask = node('blockTaskItem', { state: 'DONE' }, bulletList(item(said('a')))) - assert.equal(portable(node('taskList', {}, task('TODO', ''), listTask, node('taskList', {}, task('TODO', 'b')))), '- \\[ ]\n- \\[x]\n- a\n- \\[ ] b\n') + assert.equal(portable(node('taskList', {}, task('TODO', ''), listTask, node('taskList', {}, task('TODO', 'b')))), '- [ ]\n- [x]\n\n* a\n\n- [ ] b\n') assert.equal(portable(node('taskList', {}, task('DONE', 'a'), node('taskList', {}, task('TODO', '')))), '- [x] a\n - [ ]\n') }) @@ -192,12 +192,12 @@ test('keeps the CommonMark blocks in their spelling and drops their attributes a const code: AdfNode = { content: [text('x')], type: 'codeBlock' } assert.equal(portable(node('orderedList', { order: 1e10 }, item(said('Alpha')), item(code), item())), '- 10000000000. Alpha\n- 10000000001.\n\n ```\n x\n ```\n- 10000000002.\n') const long = node('orderedList', { order: 1e10 }, item(said('y'))) - assert.equal(portable(bulletList(item(said('x'))), long, bulletList(item(said('z')))), '- x\n- 10000000000. y\n- z\n') - assert.equal(portable(node('taskList', {}, node('taskItem', { state: 'DONE' }, text('t'))), long), '- \\[x] t\n- 10000000000. y\n') - assert.equal(portable(bulletList(item(said('a'), bulletList(item(said('x'))), long))), '- a\n - x\n - 10000000000. y\n') + assert.equal(portable(bulletList(item(said('x'))), long, bulletList(item(said('z')))), '- x\n\n* 10000000000. y\n\n- z\n') + assert.equal(portable(node('taskList', {}, node('taskItem', { state: 'DONE' }, text('t'))), long), '- [x] t\n\n* 10000000000. y\n') + assert.equal(portable(bulletList(item(said('a'), bulletList(item(said('x'))), long))), '- a\n - x\n * 10000000000. y\n') const givesWay = node('orderedList', { order: 1e10 }, item(node('rule', {})), item(bulletList(item(bulletList(item()))))) - assert.equal(portable(givesWay, bulletList(item(said('z')))), '- 10000000000.\n- 10000000001.\n - -\n- z\n') - assert.equal(portable(node('orderedList', { order: 999999999 }, item(said('a'))), node('orderedList', { order: 5 }, item(said('b')))), '- 999999999\\. a\n- 5\\. b\n') + assert.equal(portable(givesWay, bulletList(item(said('z')))), '- 10000000000.\n- 10000000001.\n - -\n\n* z\n') + assert.equal(portable(node('orderedList', { order: 999999999 }, item(said('a'))), node('orderedList', { order: 5 }, item(said('b')))), '999999999. a\n\n5) b\n') assert.equal(portable(node('heading', { level: 7 }, text('h'))), 'h\n') }) @@ -328,16 +328,16 @@ test('keeps a code span opening a line, whose backticks open no fence', () => { assert.equal(portable(paragraph(text('a\n'), text('``` x', code))), 'a\\\n```` ``` x ````\n') }) -test('drops an empty paragraph and merges adjacent lists of one type', () => { +test('drops an empty paragraph and alternates the marker between adjacent lists of a kind', () => { const ordered = (order: number, value: string): AdfNode => node('orderedList', { order }, item(said(value))) assert.equal(portable(said('a'), paragraph(), paragraph(text(' ')), said('b')), 'a\n\nb\n') - assert.equal(portable(bulletList(item(said('a'))), paragraph(), node('decisionList', {}, node('decisionItem', {}, text('b')))), '- a\n- b\n') - assert.equal(portable(ordered(2, 'a'), ordered(3, 'b'), bulletList(item(said('c')))), '2. a\n3. b\n\n- c\n') - assert.equal(portable(bulletList(item(said('x'))), ordered(1, 'a'), ordered(5, 'b'), ordered(6, 'c')), '- x\n- 1\\. a\n- 5\\. b\n\n6. c\n') - assert.equal(portable(ordered(1, 'a'), ordered(1e10, 'y')), '- 1\\. a\n- 10000000000. y\n') - assert.equal(portable(node('taskList', {}, ordered(1e10, 'y')), bulletList(ordered(1e10, 'z'))), '- - 10000000000. y\n- - 10000000000. z\n') + assert.equal(portable(bulletList(item(said('a'))), paragraph(), node('decisionList', {}, node('decisionItem', {}, text('b')))), '- a\n\n* b\n') + assert.equal(portable(ordered(2, 'a'), ordered(3, 'b'), bulletList(item(said('c')))), '2. a\n\n3) b\n\n- c\n') + assert.equal(portable(bulletList(item(said('x'))), ordered(1, 'a'), ordered(5, 'b'), ordered(6, 'c')), '- x\n\n1. a\n\n5) b\n\n6. c\n') + assert.equal(portable(ordered(1, 'a'), ordered(1e10, 'y')), '1. a\n\n- 10000000000. y\n') + assert.equal(portable(node('taskList', {}, ordered(1e10, 'y')), bulletList(ordered(1e10, 'z'))), '- - 10000000000. y\n\n* - 10000000000. z\n') const column = (list: AdfNode): AdfNode => node('layoutColumn', {}, list) - assert.equal(portable(node('layoutSection', {}, column(bulletList(item(said('a')))), column(bulletList(item(said('b')))))), '- a\n- b\n') + assert.equal(portable(node('layoutSection', {}, column(bulletList(item(said('a')))), column(bulletList(item(said('b')))))), '- a\n\n* b\n') }) test('keeps the nodes the portable flavour spells and degrades only what it cannot', () => { diff --git a/src/markdown/portable/adf-to-portable-markdown.ts b/src/markdown/portable/adf-to-portable-markdown.ts index 5cce06b..50ccf8c 100644 --- a/src/markdown/portable/adf-to-portable-markdown.ts +++ b/src/markdown/portable/adf-to-portable-markdown.ts @@ -116,26 +116,20 @@ function concatenated(results: readonly Result[]): Result // A list still taking the directive form gives way to its items' blocks. function portableSequence(blocks: readonly AdfNode[], reduction: Reduction): Result { - let sequence = mergedLists(blocks.filter((block) => block.type !== 'paragraph' || nodeContent(block).length > 0)) + let sequence = blocks.filter((block) => block.type !== 'paragraph' || nodeContent(block).length > 0) for (let index = 0; index < sequence.length; index += 1) { const listed = sequence[index] if (listed === undefined || (listed.type !== 'bulletList' && listed.type !== 'orderedList')) continue const block = numberedPastMarkers(listed) const spelled = block === listed && commonMarkSpelling(block, reduction.path, reduction.depth, { flavour: 'portable', memo: reduction.memo })?.ok === true if (spelled) continue - sequence = spliced(sequence, index, block === listed ? nodeContent(block).flatMap(nodeContent) : [block]) - // Step back to the merged neighbour, which may now take the directive form itself. - index = Math.max(0, index - 1) - 1 + sequence = [...sequence.slice(0, index), ...(block === listed ? nodeContent(block).flatMap(nodeContent) : [block]), ...sequence.slice(index + 1)] + // The replacement's first block takes this index, and may itself take the directive form. + index -= 1 } return success(sequence) } -// The replacement merges with the lists beside it, so no two lists of one type stand adjacent. -function spliced(sequence: readonly AdfNode[], index: number, replacement: readonly AdfNode[]): AdfNode[] { - const from = Math.max(0, index - 1) - return [...sequence.slice(0, from), ...mergedLists([...sequence.slice(from, index), ...replacement, ...sequence.slice(index + 1, index + 2)]), ...sequence.slice(index + 2)] -} - // A numbered list whose markers run past CommonMark's keeps its numbers as text in a bullet list. function numberedPastMarkers(list: AdfNode): AdfNode { const order = nodeAttrs(list)['order'] @@ -148,39 +142,13 @@ function numberedAsText(list: AdfNode): AdfNode { return { content: nodeContent(list).map((item, offset) => itemOf(marked(nodeContent(item), `${order + offset}.`))), type: 'bulletList' } } -// Adjacent lists of one marker read back as one list. -function mergedLists(blocks: readonly AdfNode[]): AdfNode[] { - const merged: AdfNode[] = [] - for (const block of blocks) { - let next = block - for (let previous = merged.at(-1); previous !== undefined && listMarker(next) !== undefined && listMarker(previous) === listMarker(next); previous = merged.at(-1)) { - merged.pop() - next = joinedLists(previous, next) - } - merged.push(next) - } - return merged -} - -function listMarker(block: AdfNode): string | undefined { - if (block.type === 'orderedList') return '.' - return block.type === 'bulletList' || block.type === 'taskList' ? '-' : undefined -} - -// Two numbered lists whose numbering breaks between them keep their numbers as text in one bullet list, and a task list joining a bullet list its markers. -function joinedLists(first: AdfNode, second: AdfNode): AdfNode { - const breaks = first.type === 'orderedList' && nodeAttrs(second)['order'] !== Number(nodeAttrs(first)['order']) + nodeContent(first).length - const [head, tail] = breaks ? [numberedAsText(first), numberedAsText(second)] : first.type === second.type ? [first, second] : [tasksAsText(first), tasksAsText(second)] - return { ...head, content: [...nodeContent(head), ...nodeContent(tail)] } -} - // A task keeps its marker as text; a list item stands as one, and anything else nests in the item before it. function tasksAsText(list: AdfNode): AdfNode { if (list.type !== 'taskList') return list const items: AdfNode[] = [] for (const child of nodeContent(list)) { const previous = isTaskItem(child) || child.type === 'listItem' ? undefined : items.pop() - items.push(itemOf(previous === undefined ? taskAsText(child) : mergedLists([...nodeContent(previous), child]))) + items.push(itemOf(previous === undefined ? taskAsText(child) : [...nodeContent(previous), child])) } return { content: items, type: 'bulletList' } } @@ -300,15 +268,10 @@ function reduceTaskList(node: AdfNode, reduction: Reduction): Result const standsAlone = !regular && !isTaskItem(child) && child.type !== 'taskList' for (const block of standsAlone ? [{ content: blocks, type: 'listItem' }] : blocks) tasks.push(block) } - if (regular) return success(splitTasks({ content: tasks, type: 'taskList' })) + if (regular) return success(splitTaskList({ content: tasks, type: 'taskList' })) return success(listOf(nodeContent(tasksAsText({ content: tasks, type: 'taskList' })), 'bulletList')) } -// The split sets a task's nested lists beside the list after it, and adjacent lists read back as one. -function splitTasks(list: AdfNode): AdfNode[] { - return splitTaskList(list).map((block) => (block.type === 'taskList' ? { ...block, content: mergedLists(nodeContent(block)) } : block)) -} - function reduceTask(task: AdfNode, at: Reduction): Result { const attrs = { state: nodeAttrs(task)['state'] === 'DONE' ? 'DONE' : 'TODO' } if (task.type === 'taskItem') { diff --git a/todo.md b/todo.md index 904d302..168880a 100644 --- a/todo.md +++ b/todo.md @@ -25,7 +25,6 @@ | ID | Release | Exempt | Item | R | S | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---| -| 49 | 0.2.0 | | **Read a list whose bullet or ordered delimiter changes as two lists, in every reader, and retire `!adf:listBreak`.** | 4 | 5 | 5 | 8 | 4, 5 | 15.5 | | 61 | 0.2.0 | decision | **Have the emitter ask the inline reader how a line reads back, in place of `line-escaping.ts` predicting it.** | 6 | 8 | 3 | 8 | 1 | 14.0 | | 93 | 0.2.0 | | **Report a refusal's `path` where the node would stand after `portableMarkdownToAdf` moves it out of a task item.** | 2 | 3 | 3 | 5 | 1 | 13.2 | | 86 | 0.2.0 | | **Write an image whose `media` only a link marks, and one captioned with plain text, in the CommonMark spelling a writer panel picks, in `adfToLosslessMarkdown`.** | 4 | 4 | 5 | 6 | 4, 6 | 13.0 | @@ -69,18 +68,6 @@ ## Details -### 49. Read a list whose bullet or ordered delimiter changes as two lists, in every reader, and retire `!adf:listBreak`. - -Today `- a` then `+ b`, or `1.` then `1)`, reads as one list; CommonMark reads two (spec examples -301 and 302). Goal 5 settles it for the lossless flavour too: CommonMark spells adjacent lists by -changing the marker, so `adfToLosslessMarkdown` and `adfToPortableMarkdown` alternate `-` and `*` -between adjacent bullet lists, a task list among them, and `.` and `)` between adjacent ordered -lists, and `!adf:listBreak` retires. `*` is the maintainer's pick (2026-10-04). Breaking: -`MIGRATION.md`'s Readings and Spellings tables gain their rows. Examples 301 and 302 lose their -`pending` exceptions, and the spelling leaves the README's "Seven CommonMark spellings" bullet, -which counts one fewer. Until this lands, a bullet list written right after a task list would read -back as part of it, so `adfToPortableMarkdown` writes that task list's markers as text. - ### 61. Have the emitter ask the inline reader how a line reads back, in place of `line-escaping.ts` predicting it. The comprehension panel's worst place: `escapeClaims` and `escapeClosedRuns` re-implement the