From ec9492635a966cd1bf66d21410de1ec14909862a Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Wed, 14 Oct 2026 00:33:59 +0200 Subject: [PATCH] Read GFM's pipe table without leading pipes and with alignment colons in portableMarkdownToAdf --- CHANGELOG.md | 3 +- README.md | 19 ++++---- docs/decisions.md | 7 +-- spec/flavour.md | 3 +- src/markdown/parse/blocks.ts | 48 +++++++++++++++---- .../parse/portable-markdown-to-adf.test.ts | 17 +++++++ src/markdown/pipe-table-syntax.ts | 2 +- src/markdown/portable/conventions.ts | 9 ++-- todo.md | 9 ---- 9 files changed, 78 insertions(+), 39 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b037203..574d303 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -105,7 +105,8 @@ - Add `adfToPortableMarkdown` and `portableMarkdownToAdf`, a lossy pair converting ADF to and from markdown GitHub, GitLab and Obsidian render: alerts, callouts, task lists, `==highlights==` and pipe tables. A task item keeps its paragraphs, then its nested task lists; the rest of the item, - such as an image or a code block, stands after the task list. + such as an image or a code block, stands after the task list. `portableMarkdownToAdf` reads GFM's + pipe table without its leading pipes and with alignment colons, dropping the alignment. - Spell `rule`'s `color`, `style` and `weight`, `layoutSection`'s `columnRuleStyle` and a link's `collection`, `id` and `occurrenceKey` directly where they rode the opaque carry. - The README opens with what the package does and for whom, and `package.json` carries search diff --git a/README.md b/README.md index 0708dc1..cb2bb90 100644 --- a/README.md +++ b/README.md @@ -12,9 +12,8 @@ products store as ADF. document they started from — panels, mentions, statuses and node types this version does not know included. - **Markdown in:** `portableMarkdownToAdf` reads a bot's or a model's markdown, including `~~` and - pipe tables whose every row opens with a pipe and whose delimiter row holds no alignment colon. - `commonMarkToAdf` reads strict CommonMark as its - spec says: nothing becomes a table, task or panel. + GFM's pipe tables, their column alignment dropped. `commonMarkToAdf` reads strict CommonMark as + its spec says: nothing becomes a table, task or panel. - **Portable markdown out, lossy:** `adfToPortableMarkdown` writes the markdown that GitHub, GitLab and Obsidian render — GitHub Flavored Markdown's alerts, task lists and pipe tables, Obsidian Flavored Markdown's callouts and highlights. It keeps the text, images by URL and link targets and @@ -334,8 +333,8 @@ keeping the content and dropping the rest: attributes, colours, layout, identity reader of the rendered document sees or follows: its text, images and link targets. It writes no directive. -`portableMarkdownToAdf` reads CommonMark, pipe tables and `~~` as `losslessMarkdownToAdf` reads -them, and the conventions below as nodes, taking other tools' spellings too; a directive and an +`portableMarkdownToAdf` reads CommonMark and `~~` as `losslessMarkdownToAdf` reads them, GFM's +pipe tables with or without leading pipes, their column alignment dropped, and the conventions below as nodes, taking other tools' spellings too; a directive and an `adf:` fence read as CommonMark reads them, as text and code. A backslash keeps a marker as text: `\==x==`, `> \[!NOTE]`, `- \[x]`. Markdown `adfToPortableMarkdown` wrote reads back and writes again byte for byte; the document it came from does not come back. @@ -425,7 +424,7 @@ Which reader raises which code: [The calls](#the-calls). Codes every direction r | Code | Fires when | What you can do | | --- | --- | --- | | `malformed-directive` | an `!adf:` the grammar cannot read — a prefix completing no directive, an unclosed container, `[content]` or `{attrs}`, a closer with no container of its name open, a leaf given a body, `{attrs}` out of order or duplicated, invalid JSON in an opaque carry | write the spelling the message names, or keep it literal: escape the prefix — `\!adf:`, block and inline alike — or, for a code fence whose info string opens `adf:`, drop the info string and wrap the fence in `!adf:codeBlock {language="adf:…"}` | -| `malformed-pipe-table` | a pipe row that is no pipe table — a missing or ragged `---` delimiter row, an alignment colon in it, or a row not opening with a pipe | open every row with a pipe and give the delimiter row the header's cell count; to keep the lines literal text, escape the leading pipe of every one — escaping a single row leaves the next to open a fresh table and fail the same way | +| `malformed-pipe-table` | a pipe row that is no pipe table — a missing or ragged `---` delimiter row, or, in `losslessMarkdownToAdf`, an alignment colon in it or a row not opening with a pipe | open every row with a pipe and give the delimiter row the header's cell count; to keep the lines literal text, escape the leading pipe of every one — escaping a single row leaves the next to open a fresh table and fail the same way | | `not-a-string` | the value handed in is not a string, such as `null`, a number or a `String` object | pass the input as a string, `String(value)` for a `String` object; the message names the value's type | | `unknown-directive-name` | a directive whose name is no node or mark this version spells | check the name in [`spec/flavour.md`](https://github.com/larvit/adf-codec/blob/main/spec/flavour.md), or escape the prefix as `\!adf:`; the spelling itself is well formed, so a later minor may give the name meaning | | `unmappable-html` | the input holds raw HTML, a comment and a processing instruction among them | remove the construct, or write what it holds as markdown | @@ -493,10 +492,10 @@ Serves [Goals](#goals) 1, 4 and 5. levels, the node shapes the `unsupported-node-shape` row lists, and a conversion `engine-limit` stops. Show the refusal and keep the document read-only; saving markdown you could not produce is the loss the round-trip exists to stop. -- The pipe table that `losslessMarkdownToAdf` and `portableMarkdownToAdf` read narrows GFM's twice: - every row opens with a pipe, so GFM's bare form is an error result, and an alignment colon in the - delimiter row is an error too — ADF holds no column alignment. The trailing pipe is canonical - output, optional in input. +- The pipe table that `losslessMarkdownToAdf` reads narrows GFM's twice: every row opens with a + pipe, so GFM's bare form is an error result, and an alignment colon in the delimiter row is an + error too — ADF holds no column alignment. `portableMarkdownToAdf` reads both and drops the + alignment. The trailing pipe is canonical output, optional in input. - Past that and `~~`, `losslessMarkdownToAdf` reads no GFM: an autolink literal and a `- [ ]` marker stay text, and a checklist is the `taskList` directive — `portableMarkdownToAdf` turns the marker into a `taskList`. diff --git a/docs/decisions.md b/docs/decisions.md index 28a4309..a9aece4 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -438,9 +438,10 @@ handles one cause alike whichever node, attribute or direction raised it. or across directions, one code covers them all and `path` and `message` say which — `unsupported-nesting-depth` is the 500-level guard whichever direction hits it. - A claim code names the spelling claimed, never the node that spelling would have built: a - malformed `!adf:table` is a `malformed-directive`, and an alignment colon a - `malformed-pipe-table` — the flavour's own delimiter row is `-` runs, so the grammar refuses the - colon rather than ADF's missing column model doing it. What the grammar itself refuses stays a + malformed `!adf:table` is a `malformed-directive`, and an alignment colon in + `losslessMarkdownToAdf` a `malformed-pipe-table` — the flavour's own delimiter row is `-` runs, so + the grammar refuses the colon rather than ADF's missing column model doing it; + `portableMarkdownToAdf` reads GFM's colon and drops the alignment (2026-10-14). What the grammar itself refuses stays a claim code, key order among it, and a leaf given a body is refused at its opener, as a container missing its closer is (2026-09-16). - A directive whose name reads back to no node is `unknown-directive-name` rather than a claim code diff --git a/spec/flavour.md b/spec/flavour.md index 30298e6..c0cfe21 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -347,7 +347,8 @@ claimed and must parse as part of a pipe table, else it is a named error — esc (`\|`) to keep it literal text. The shape a missing leading pipe leaves is claimed too: a line holding an unescaped `|`, followed inside one paragraph by a line whose cells are all `-` runs, an alignment colon among them, and match it in count — GFM's table without the outer pipes — is -a named error rather than the prose it reads as. A pipe table is a header row, a delimiter row whose cells are +a named error rather than the prose it reads as; `portableMarkdownToAdf` reads it, and an +alignment colon, as GFM does, dropping the alignment. A pipe table is a header row, a delimiter row whose cells are runs of one or more `-` (canonical `---`), and body rows; rows follow code-fence indentation. Cells split on unescaped `|` before inline parsing — `\|` stays in the cell text, and the inline layer's ordinary CommonMark escaping yields the pipe; each cell is the inline content of diff --git a/src/markdown/parse/blocks.ts b/src/markdown/parse/blocks.ts index f9adcea..3889f9e 100644 --- a/src/markdown/parse/blocks.ts +++ b/src/markdown/parse/blocks.ts @@ -19,7 +19,7 @@ import { thematicBreakTail, type ThematicBreakTail, } from '../commonmark/grammar.ts' -import { barePipeCells, isDelimiterRow, isPipeAlignment, isPipeDelimiter, malformedPipeTable, pipeCells } from '../pipe-table-syntax.ts' +import { barePipeCells, isDelimiterRow, isPipeAlignment, malformedPipeTable, pipeCells } from '../pipe-table-syntax.ts' import { blockDirectiveForm } from '../block-directive.ts' import { directiveEscape, malformedDirective, readDirectiveLine, spellDirectiveCloser } from '../directive-syntax.ts' import { readLinkDefinitions } from '../commonmark/link-reference-definitions.ts' @@ -355,8 +355,13 @@ function continuesLazily(walk: Walk, line: Line): boolean { if (walk.leaf?.kind !== 'paragraph' || isBlankLine(line.text)) return false if (leadingColumns(line) >= indentedCodeColumns) return true const opener = removeColumns(line, largestOpenerIndentation).text - if ((walk.claims.directives && readDirectiveLine(opener) !== undefined) || (walk.claims.pipeTables && claimsPipeLine(opener)) || isThematicBreak(opener)) return false - return atxHeading(opener) === undefined && openingCodeFence(opener) === undefined && openingHtmlBlock(opener, true) === undefined + return !opensBlock(walk.claims, opener) +} + +// Every leaf a line can open but a paragraph, a setext underline and indented code. +function opensBlock(claims: Claims, opener: string): boolean { + if ((claims.directives && readDirectiveLine(opener) !== undefined) || (claims.pipeTables !== false && claimsPipeLine(opener)) || isThematicBreak(opener)) return true + return atxHeading(opener) !== undefined || openingCodeFence(opener) !== undefined || openingHtmlBlock(opener, true) !== undefined } function readBlockLine(walk: Walk, line: Line): void { @@ -371,7 +376,7 @@ function readBlockLine(walk: Walk, line: Line): void { return } if (leaf?.kind === 'pipe-table') { - const cells = pipeCells(removeColumns(line, largestOpenerIndentation).text) + const cells = pipeTableRow(walk.claims, removeColumns(line, largestOpenerIndentation).text) if (cells !== undefined) { leaf.rows.push(cells) return @@ -414,7 +419,8 @@ function openLeaf(walk: Walk, line: Line): void { else pushFault(walk, directive.fault) return } - const cells = walk.claims.pipeTables ? pipeCells(opener) : undefined + if (openGfmTable(walk, opener)) return + const cells = walk.claims.pipeTables === false ? undefined : pipeCells(opener) if (cells !== undefined) { closeLeaf(walk) walk.leaf = { kind: 'pipe-table', position: walk.position, rows: [cells] } @@ -482,16 +488,38 @@ function closeLeaf(walk: Walk): void { walk.leaf = undefined const { position } = leaf if (leaf.kind === 'html') currentBlocks(walk).push({ construct: leaf.construct, kind: 'html', position }) - else if (leaf.kind === 'pipe-table') currentBlocks(walk).push(pipeTableBlock(leaf.rows, position)) + else if (leaf.kind === 'pipe-table') currentBlocks(walk).push(pipeTableBlock(leaf.rows, position, walk.claims)) else currentBlocks(walk).push({ kind: 'code', language: leaf.kind === 'fenced-code' ? decodeTextEscapes(leaf.info) : '', position, text: leaf.lines.join('\n') }) } -function pipeTableBlock(rows: readonly [string[], ...string[][]], position: SourcePosition): Block { +// GFM, Tables: a paragraph's last line heads a table where a delimiter row of its cell count follows. +function openGfmTable(walk: Walk, opener: string): boolean { + const paragraph = walk.leaf + if (walk.claims.pipeTables !== 'gfm' || paragraph?.kind !== 'paragraph') return false + const delimiter = pipeCells(opener) ?? barePipeCells(opener) + const header = barePipeCells(paragraph.lines.at(-1) ?? '') + if (delimiter === undefined || header?.length !== delimiter.length || !isDelimiterRow(delimiter)) return false + paragraph.lines.pop() + const position = paragraph.positions.pop() ?? walk.position + if (paragraph.lines.length === 0) walk.leaf = undefined + else closeLeaf(walk) + walk.leaf = { kind: 'pipe-table', position, rows: [header, delimiter] } + return true +} + +// GFM, Tables: a row without its leading pipe goes on until a line opens another block. +function pipeTableRow(claims: Claims, line: string): string[] | undefined { + const cells = pipeCells(line) + if (cells !== undefined || claims.pipeTables !== 'gfm' || opensBlock(claims, line)) return cells + return barePipeCells(line) +} + +function pipeTableBlock(rows: readonly [string[], ...string[][]], position: SourcePosition, claims: Claims): Block { const [header, delimiter, ...body] = rows - if (delimiter !== undefined && delimiter.some(isPipeAlignment)) { + if (claims.pipeTables !== 'gfm' && delimiter !== undefined && delimiter.some(isPipeAlignment)) { return faultedBlock('a pipe table carries no column alignment ADF could hold: this delimiter row holds an alignment colon', position) } - if (delimiter === undefined || !delimiter.every(isPipeDelimiter)) { + if (delimiter === undefined || !isDelimiterRow(delimiter)) { return faultedBlock('a pipe table underlines its header with a row of `-` runs: this one has none; \\| at the start of every row keeps them literal text', position) } const ragged = [delimiter, ...body].find((row) => row.length !== header.length) @@ -501,7 +529,7 @@ function pipeTableBlock(rows: readonly [string[], ...string[][]], position: Sour // spec/flavour.md, Tables: GFM's table without the leading pipes, which no line of it claims. function bareTableFault(paragraph: Extract, claims: Claims): Block | undefined { - if (!claims.pipeTables) return undefined + if (claims.pipeTables !== 'flavour') return undefined let header: string[] | undefined for (const line of paragraph.text.split('\n')) { const cells = barePipeCells(line) diff --git a/src/markdown/parse/portable-markdown-to-adf.test.ts b/src/markdown/parse/portable-markdown-to-adf.test.ts index 8242efc..de5917a 100644 --- a/src/markdown/parse/portable-markdown-to-adf.test.ts +++ b/src/markdown/parse/portable-markdown-to-adf.test.ts @@ -278,6 +278,23 @@ test('reads text the writer kept from reading as a marker back as text', () => { assert.deepEqual(roundTripped(bare('taskList', task('DONE', text('[x] ==a==')))), [bare('taskList', task('DONE', text('[x] ==a==')))]) }) +test('reads GFM\'s pipe table without its leading pipes or with alignment colons, dropping the alignment', () => { + const cell = (type: string, value: string): AdfNode => bare(type, said(value)) + const table = bare('table', bare('tableRow', cell('tableHeader', 'a'), cell('tableHeader', 'b')), bare('tableRow', cell('tableCell', 'c'), cell('tableCell', 'd'))) + assert.deepEqual(read('a | b\n--- | ---\nc | d\n'), [table]) + assert.deepEqual(read('| a | b |\n| :--- | ---: |\n| c | d |\n'), [table]) + assert.deepEqual(read('a | b\n:---: | ---:\n| c | d |\n'), [table]) + assert.deepEqual(read('| a | b |\n--- | ---\nc | d |\n'), [table]) + assert.deepEqual(read('a | b\n| --- | --- |\nc | d\n'), [table]) + assert.deepEqual(read('Intro.\na | b\n--- | ---\nc | d\n\nAfter.\n'), [said('Intro.'), table, said('After.')]) + assert.deepEqual(read('> a | b\n> --- | ---\n> c | d\n'), [bare('blockquote', table)]) + assert.deepEqual(read('a | b\n--- | ---\nc | d\nAfter.\n'), [table, said('After.')]) + assert.deepEqual(read('a | b\n--- | ---\nc | d\n# h | i\n'), [table, { attrs: { level: 1 }, content: [text('h | i')], type: 'heading' }]) + assert.deepEqual(read('a | b\n--- | --- | ---\n'), [said('a | b --- | --- | ---')]) + assert.deepEqual(read('- a | b\n--- | ---\n'), [bare('bulletList', bare('listItem', said('a | b --- | ---')))]) + assert.deepEqual(read('a | b\n--- | ---\nc\n'), [bare('table', table.content?.[0] ?? said('')), said('c')]) +}) + test('reads what CommonMark reads where no row reads, a directive and a carry fence among it, an image sharing its paragraph with a marker among it', () => { assert.deepEqual(read('!adf:panel\n'), [said('!adf:panel')]) assert.deepEqual(read('a !adf:carry{json="{}"} \\!adf:x\n'), [said('a !adf:carry{json="{}"} !adf:x')]) diff --git a/src/markdown/pipe-table-syntax.ts b/src/markdown/pipe-table-syntax.ts index 8d5fc58..23f6492 100644 --- a/src/markdown/pipe-table-syntax.ts +++ b/src/markdown/pipe-table-syntax.ts @@ -23,7 +23,7 @@ export function isPipeAlignment(cell: string): boolean { return alignmentCell.test(cell) } -export function isPipeDelimiter(cell: string): boolean { +function isPipeDelimiter(cell: string): boolean { return delimiterCell.test(cell) } diff --git a/src/markdown/portable/conventions.ts b/src/markdown/portable/conventions.ts index 971556d..11f7e56 100644 --- a/src/markdown/portable/conventions.ts +++ b/src/markdown/portable/conventions.ts @@ -10,20 +10,21 @@ export type Claims = { carryFence: boolean directives: boolean highlights: boolean - pipeTables: boolean + // `gfm` reads GFM's whole pipe table, its alignment dropped; `flavour` reads rows opening with `|` over a `-` delimiter row. + pipeTables: 'flavour' | 'gfm' | false strikethrough: boolean taskMarkers: boolean } -type WrittenClaims = Claims & { pipeTables: true; strikethrough: true } +type WrittenClaims = Claims & { pipeTables: 'flavour' | 'gfm'; strikethrough: true } // Directives exclude alerts and highlights: an expand directive leaves `inExpand` unset and a directive's slot reads `==`. type ReadClaims = Claims & ({ directives: false } | { alerts: false; directives: true; highlights: false }) export const flavourClaims: Readonly<{ commonmark: ReadClaims } & Record> = { commonmark: { alerts: false, carryFence: false, directives: false, highlights: false, pipeTables: false, strikethrough: false, taskMarkers: false }, - lossless: { alerts: false, carryFence: true, directives: true, highlights: false, pipeTables: true, strikethrough: true, taskMarkers: false }, - portable: { alerts: true, carryFence: false, directives: false, highlights: true, pipeTables: true, strikethrough: true, taskMarkers: true }, + lossless: { alerts: false, carryFence: true, directives: true, highlights: false, pipeTables: 'flavour', strikethrough: true, taskMarkers: false }, + portable: { alerts: true, carryFence: false, directives: false, highlights: true, pipeTables: 'gfm', strikethrough: true, taskMarkers: true }, } type AlertMarker = { folded: boolean; length: number; panelType: string } diff --git a/todo.md b/todo.md index c0b74db..3917467 100644 --- a/todo.md +++ b/todo.md @@ -25,7 +25,6 @@ | ID | Release | Exempt | Item | R | S | C | A | G | Goals | Score | |---|---|---|---|---|---|---|---|---|---|---| -| 84 | 0.2.0 | | **Read GFM's pipe table without leading pipes, and with alignment colons, in `portableMarkdownToAdf`, dropping the alignment.** | 3 | 4 | 5 | 7 | 7 | 4, 7 | 19.4 | | 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 3 | 5 | 6 | 6 | 11.5 | | 53 | 0.2.0 | | **Put a block's `marks` spelling to the writer panel and adopt its pick.** | 4 | 5 | 4 | 5 | 6 | 6 | 10.0 | | 111 | 0.2.1 | | **Split README's `unsupported-node-shape` error-table row into one bullet per cause group, each carrying its remedy.** | 2 | 2 | 1 | 5 | 5 | 3 | 15.3 | @@ -73,14 +72,6 @@ ## Details -### 84. Read GFM's pipe table without leading pipes, and with alignment colons, in `portableMarkdownToAdf`, dropping the alignment. - -Models write `|:---|` delimiter rows and tables without a leading pipe; both are -`malformed-pipe-table` today, and `portableMarkdownToAdf` is the bot's reader. ADF holds no column -alignment, so the portable reader drops it (Goal 7: form, never content). `losslessMarkdownToAdf` keeps -both refusals. Revise §Which code a cause takes so the alignment colon is `malformed-pipe-table` in -that reader alone, and the README's pipe-table guarantee and `malformed-pipe-table` row to match. - ### 52. Spell `colwidth` as a comma list, `colwidth="340,420"`. A writer panel chose it on 2026-10-03, 5 of 7, over today's `colwidth="[340,420]"`. Breaking: