Emitter 2b: the block-node directives, the pipe table and the refusal corpus #11

Merged
lilleman merged 6 commits from block-nodes into main 2026-08-25 16:20:31 +02:00
36 changed files with 1020 additions and 103 deletions
+3 -1
View File
@@ -75,7 +75,9 @@ Pre-1.0, normal 0.x rules.
The error surface is a contract too. `ConvertError` is `{ code, message, path }` — the code from a
closed list a consumer may switch exhaustively, the message free text, the path the node's position
from the document root. Adding, removing or renaming a code is breaking, so a milestone meeting a
new failure cause reuses a code where one fits; the list is complete at `0.1.0`.
new failure cause reuses a code where one fits; the list is complete at `0.1.0`. A code names the
cause; where one cause recurs across node types, one code covers them all and `path` and `message`
say which.
## 9. Release automation
+2 -1
View File
@@ -3,7 +3,8 @@
Lossless conversion between **Atlassian Document Format** (ADF), an extended markdown flavour, and
an HTML dialect.
**Status: pre-release — `adfToMarkdown` emits the CommonMark subset, nothing else is built.**
**Status: pre-release — `adfToMarkdown` emits the CommonMark subset and the block nodes, nothing
else is built.**
Plan: `todo.md`. Decisions: `AGENTS.md`. The flavour's grammar:
[`spec/flavour.md`](spec/flavour.md).
+4 -1
View File
@@ -8,7 +8,10 @@ in_node() { docker run --rm -u "$(id -u):$(id -g)" -e HOME=/tmp -v "$PWD:/app" -
in_node npm ci
in_node npm run typecheck
test_output=$(in_node npm test)
if ! test_output=$(in_node npm test 2>&1); then
printf '%s\n' "$test_output"
exit 1
fi
printf '%s\n' "$test_output"
if printf '%s' "$test_output" | grep -q ' tests 0'; then
echo 'the gate ran zero tests — failing instead of a vacuous green'
+4 -1
View File
@@ -1,6 +1,6 @@
# The corpus
One directory per contract kind:
One directory per contract kind, each landing with its milestone:
- `round-trip/``<name>.json` + `<name>.md`: the markdown `adfToMarkdown` must emit for that
document, byte for byte, and that `markdownToAdf` must read back to it (AGENTS.md §2). Grouped
@@ -9,6 +9,9 @@ One directory per contract kind:
`markdownToAdf` must build from it. One-way; the markdown is not canonical.
- `errors/``<name>.md`: markdown input that must not convert. A `<name>.error` beside it
pins which error.
- `unspellable/``<name>.json`: ADF `adfToMarkdown` must refuse, the `ConvertErrorCode` in the
`<name>.error` beside it. Two populations live here: refusals that stay contract, and documents
a maintainer decision (`todo.md`) moves to `round-trip/`.
- `real-payloads/``<name>.json`: sanitized live ADF, round-tripped ADF→markdown→ADF. No
expected markdown.
@@ -0,0 +1,112 @@
{
"content": [
{
"attrs": {
"panelType": "info"
},
"content": [
{
"content": [
{
"content": [
{
"attrs": {
"panelType": "warning"
},
"content": [
{
"content": [
{
"text": "Check the torque before signing off.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "panel"
}
],
"type": "listItem"
},
{
"content": [
{
"content": [
{
"content": [
{
"attrs": {
"title": "Deeper still"
},
"content": [
{
"content": [
{
"text": "Two list levels down.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "expand"
}
],
"type": "listItem"
}
],
"type": "bulletList"
}
],
"type": "listItem"
}
],
"type": "bulletList"
},
{
"content": [
{
"attrs": {
"title": "Full build log"
},
"content": [
{
"content": [
{
"text": "The build ran for 11 minutes.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "expand"
},
{
"attrs": {
"panelType": "note"
},
"content": [
{
"content": [
{
"text": "Superseded by the next run.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "panel"
}
],
"type": "blockquote"
}
],
"type": "panel"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1,16 @@
::::panel info
- :::panel warning
Check the torque before signing off.
:::
- - :::expand {title="Deeper still"}
Two list levels down.
:::
> :::expand {title="Full build log"}
> The build ran for 11 minutes.
> :::
>
> :::panel note
> Superseded by the next run.
> :::
::::
@@ -82,6 +82,37 @@
}
],
"type": "mediaSingle"
},
{
"attrs": {
"layout": "center"
},
"content": [
{
"attrs": {
"alt": "",
"type": "external",
"url": "https://example.com/moon.png"
},
"type": "media"
}
],
"type": "mediaSingle"
},
{
"attrs": {
"layout": "center"
},
"content": [
{
"attrs": {
"type": "external",
"url": "https://example.com/plan.png?a=1&amp;b=2"
},
"type": "media"
}
],
"type": "mediaSingle"
}
],
"type": "doc",
@@ -16,3 +16,11 @@ Taken from Kiruna.
:::mediaSingle {layout=center}
::media {alt="The assembly drawing" marks="[{\"attrs\":{\"href\":\"https://example.com/assembly\"},\"type\":\"link\"}]" type=external url="https://example.com/assembly.png"}
:::
:::mediaSingle {layout=center}
::media {alt="" type=external url="https://example.com/moon.png"}
:::
:::mediaSingle {layout=center}
::media {type=external url="https://example.com/plan.png?a=1&amp;b=2"}
:::
+1
View File
@@ -0,0 +1 @@
unspelled-node-attribute
+18
View File
@@ -0,0 +1,18 @@
{
"content": [
{
"attrs": {
"localId": "01a03d5c-9b21-73f4-8e6a-0c47b1d9e2f8"
},
"content": [
{
"text": "Every block node in the schema carries one.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1 @@
ambiguous-attribute-spelling
@@ -0,0 +1,18 @@
{
"content": [
{
"attrs": {
"language": ""
},
"content": [
{
"text": "cargo build --release",
"type": "text"
}
],
"type": "codeBlock"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1 @@
unspelled-block-separation
@@ -0,0 +1,40 @@
{
"content": [
{
"attrs": {
"title": "Full build log"
},
"content": [
{
"content": [
{
"text": "The build ran for 11 minutes.",
"type": "text"
}
],
"type": "paragraph"
},
{
"attrs": {
"panelType": "warning"
},
"content": [
{
"content": [
{
"text": "Three warnings went unread.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "panel"
}
],
"type": "expand"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1 @@
unspellable-link-destination
@@ -0,0 +1,23 @@
{
"content": [
{
"content": [
{
"marks": [
{
"attrs": {
"href": "https://example.com/a)b"
},
"type": "link"
}
],
"text": "The torque table",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1 @@
unspellable-link-title
+24
View File
@@ -0,0 +1,24 @@
{
"content": [
{
"content": [
{
"marks": [
{
"attrs": {
"href": "https://example.com/torque",
"title": "He said \"hi\""
},
"type": "link"
}
],
"text": "The torque table",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1 @@
unspellable-line-start
+47
View File
@@ -0,0 +1,47 @@
{
"content": [
{
"content": [
{
"content": [
{
"content": [
{
"text": "Torque the bolts",
"type": "text"
}
],
"type": "paragraph"
},
{
"attrs": {
"order": 2
},
"content": [
{
"content": [
{
"content": [
{
"text": "Then sign off",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "orderedList"
}
],
"type": "listItem"
}
],
"type": "bulletList"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1 @@
ambiguous-attribute-spelling
@@ -0,0 +1,28 @@
{
"content": [
{
"attrs": {
"order": 1
},
"content": [
{
"content": [
{
"content": [
{
"text": "Loosen the clamp",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "orderedList"
}
],
"type": "doc",
"version": 1
}
+1 -1
View File
@@ -13,7 +13,7 @@
"node": ">=24"
},
"scripts": {
"test": "node --test --experimental-test-coverage --test-coverage-exclude=\"src/**/*.test.ts\" --test-coverage-branches=93 --test-coverage-functions=100 --test-coverage-lines=100 \"src/**/*.test.ts\"",
"test": "node --test --experimental-test-coverage --test-coverage-exclude=\"src/**/*.test.ts\" --test-coverage-branches=94 --test-coverage-functions=100 --test-coverage-lines=100 \"src/**/*.test.ts\"",
"typecheck": "tsc --noEmit"
},
"devDependencies": {
+15 -10
View File
@@ -42,8 +42,9 @@ normalizes to it through the round-trip.
the leading delimiter of a construct that would otherwise open, re-scan from there, and
repeat — with the opener literal the closer parses as text, so `*not emphasis*` is
`\*not emphasis*`, one backslash.
- Blocks separated by one blank line at document level and between CommonMark blocks; two
directive blocks inside a container take none. No trailing whitespace outside a code block's
- Blocks separated by one blank line at document level, inside a blockquote and between CommonMark
blocks; two directive blocks inside a container take none. No trailing whitespace outside a code
block's
content, single trailing newline; a document with no blocks is the empty string.
## Directives
@@ -74,9 +75,9 @@ block content
The fence is three or more colons. `arg` is one optional bare token whose meaning each node
defines (e.g. the panel type). The body is block markdown. The closing fence is a line of at
least the opening's length, and a container's fence is longer than every directive fence line in
its body — counting only lines that parse as directive fences in the body's block structure; a
colon run inside a code fence or opaque carry is content. Canonical form uses minimal lengths.
least the opening's length, and a container's fence is longer than every directive fence line
anywhere in its body, however deeply a list item or blockquote nests it; a colon run inside a code
fence or opaque carry is content. Canonical form uses minimal lengths.
Directive fence lines follow code-fence indentation (up to three spaces relative to their
container); trailing whitespace on a fence line is tolerated in input, never emitted.
@@ -90,7 +91,8 @@ container is open, a named error otherwise.
**Attributes**: `{key=value key2="two words"}`. `{attrs}` is optional in every form, and `{}` is
valid — no attributes. A bare value matches `[A-Za-z0-9_-]+`; any other value is double-quoted
with JSON string escaping (`\"` `\\` `\n` `\t` `\uXXXX`, …) — total over
Unicode, and raw newlines never appear inside quotes. All values are strings at the grammar
Unicode, and raw newlines never appear inside quotes. The closing `}` is the first one outside
quotes, since a quoted value holds `}` unescaped. All values are strings at the grammar
level; each node's section assigns types. Canonical form orders keys alphabetically, spells
values bare wherever allowed, inside quotes escapes only what it must using the shortest escape
form, and omits empty `{attrs}` except where the `{` itself claims the directive
@@ -200,7 +202,9 @@ The moon, at night.
a `mediaSingle` with attrs exactly `{"layout":"center"}` holding an `external` `media``url`
from the destination, `alt` the description's plain-text content when non-empty. `adfToMarkdown`
emits the image form for exactly that shape — those attrs and no others, no marks on either
node, no caption, and a `media` carrying nothing beyond `alt`, `type` and `url`. An image amid
node, no caption, and a `media` carrying nothing beyond `alt`, `type` and `url` — and only where
CommonMark spells the pair: a destination or a description the image form cannot hold, an empty
`alt` included, takes the directive form instead. An image amid
other text, or one carrying a title, is a named error: `mediaInline` carries a media
`collection` + `id`, never a URL, and no media node carries a title.
@@ -210,9 +214,10 @@ One header row plus plain inline cells is a pipe table; anything richer is the d
(AGENTS.md §4). Precisely: a table emits as a pipe table exactly when the `table`, every row
and every cell carry no attrs and no marks, the first row is all `tableHeader` and the rest all
`tableCell`, every row has the header's cell count, and every cell holds exactly one attr-less,
mark-less paragraph — an empty cell holds one empty paragraph — with no `|` in a code span:
backslash escapes are inert there, so pipe form cannot spell that pipe and the table takes the
directive form. A pipe table parses back to exactly that shape.
mark-less paragraph — an empty cell holds one empty paragraph — with no `|` in a code span,
link destination or link title: backslash escapes are inert in everything the inline layer spells
as syntax, so pipe form cannot spell that pipe and the table takes the directive form. A pipe table
parses back to exactly that shape.
```
| Part | Qty |
+2
View File
@@ -2,6 +2,8 @@ import { isJsonValue, type JsonValue } from './json-value.ts'
export type AdfAttributes = { [key: string]: JsonValue }
export type AttributeKind = 'boolean' | 'json' | 'number' | 'string'
export type AdfMark = {
attrs?: AdfAttributes
type: string
+101 -5
View File
@@ -1,7 +1,7 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import type { AdfDocument, AdfMark, AdfNode } from './adf-document.ts'
import type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from './adf-document.ts'
import type { Result } from './result.ts'
import { adfToMarkdown } from './index.ts'
@@ -53,13 +53,13 @@ test('refuses marks on a block node', () => {
test('refuses an ordered list whose markdown start is ambiguous', () => {
const items: AdfNode[] = [{ content: [paragraph({ text: 'x', type: 'text' })], type: 'listItem' }]
assert.equal(code(adfToMarkdown(document({ content: items, type: 'orderedList' }))), 'ambiguous-ordered-list-start')
assert.equal(code(adfToMarkdown(document({ attrs: { order: 1 }, content: items, type: 'orderedList' }))), 'ambiguous-ordered-list-start')
assert.equal(code(adfToMarkdown(document({ content: items, type: 'orderedList' }))), 'ambiguous-attribute-spelling')
assert.equal(code(adfToMarkdown(document({ attrs: { order: 1 }, content: items, type: 'orderedList' }))), 'ambiguous-attribute-spelling')
assert.equal(markdown(adfToMarkdown(document({ attrs: { order: 2 }, content: items, type: 'orderedList' }))), '2. x\n')
})
test('refuses the code block info strings the fence cannot hold', () => {
assert.equal(code(adfToMarkdown(document({ attrs: { language: '' }, type: 'codeBlock' }))), 'ambiguous-empty-code-block-language')
assert.equal(code(adfToMarkdown(document({ attrs: { language: '' }, type: 'codeBlock' }))), 'ambiguous-attribute-spelling')
assert.equal(code(adfToMarkdown(document({ attrs: { language: 'adf' }, type: 'codeBlock' }))), 'reserved-adf-language')
assert.equal(code(adfToMarkdown(document({ attrs: { language: 'a`b' }, type: 'codeBlock' }))), 'unspellable-code-block-language')
assert.equal(code(adfToMarkdown(document({ attrs: { language: ' sql' }, type: 'codeBlock' }))), 'unspellable-code-block-language')
@@ -116,7 +116,8 @@ test('refuses two adjacent lists of the same kind', () => {
})
test('refuses a node type the canonical form does not cover', () => {
assert.equal(code(adfToMarkdown(document({ type: 'panel' }))), 'unsupported-node-type')
assert.equal(code(adfToMarkdown(document({ type: 'blockCard' }))), 'unsupported-node-type')
assert.equal(code(adfToMarkdown(document({ type: 'toString' }))), 'unsupported-node-type')
assert.equal(code(adfToMarkdown(document(paragraph({ type: 'mention' })))), 'unsupported-node-type')
})
@@ -283,3 +284,98 @@ test('refuses a document nested deeper than the emitter carries', () => {
test('emits an empty list item without trailing whitespace', () => {
assert.equal(markdown(adfToMarkdown(document({ content: [{ type: 'listItem' }], type: 'bulletList' }))), '-\n')
})
test('spells a block directive as its node type, arg and attributes', () => {
const panel = (attrs: AdfAttributes): AdfDocument => document({ attrs, content: [paragraph({ text: 'x', type: 'text' })], type: 'panel' })
assert.equal(markdown(adfToMarkdown(panel({ panelType: 'warning' }))), ':::panel warning\nx\n:::\n')
assert.equal(markdown(adfToMarkdown(panel({}))), ':::panel\nx\n:::\n')
assert.equal(markdown(adfToMarkdown(document({ type: 'caption' }))), ':::caption\n:::\n')
assert.equal(markdown(adfToMarkdown(document({ attrs: { localId: 'a' }, type: 'syncBlock' }))), '::syncBlock {localId=a}\n')
})
test('refuses a directive attribute no section spells', () => {
assert.equal(code(adfToMarkdown(document({ attrs: { rounded: true }, type: 'panel' }))), 'unspelled-node-attribute')
assert.equal(code(adfToMarkdown(document({ attrs: { toString: 'x' }, type: 'panel' }))), 'unspelled-node-attribute')
assert.equal(code(adfToMarkdown(document({ attrs: { localId: 4 }, type: 'panel' }))), 'unsupported-node-shape')
assert.equal(code(adfToMarkdown(document({ attrs: { width: '50' }, type: 'layoutColumn' }))), 'unsupported-node-shape')
assert.equal(code(adfToMarkdown(document({ attrs: { isNumberColumnEnabled: 'true' }, type: 'table' }))), 'unsupported-node-shape')
})
test('refuses an arg slot value no bare token spells', () => {
assert.equal(code(adfToMarkdown(document({ attrs: { panelType: 'extra info' }, type: 'panel' }))), 'unspelled-node-attribute')
assert.equal(code(adfToMarkdown(document({ attrs: { state: 2 }, type: 'taskItem' }))), 'unspelled-node-attribute')
})
test('carries a block node mark in the reserved attribute', () => {
const section = (...marks: AdfMark[]): AdfDocument => document({ marks, type: 'layoutSection' })
assert.equal(markdown(adfToMarkdown(section({ type: 'breakout' }))), ':::layoutSection {marks="[{\\"type\\":\\"breakout\\"}]"}\n:::\n')
assert.equal(markdown(adfToMarkdown(section({ attrs: {}, type: 'breakout' }))), ':::layoutSection {marks="[{\\"type\\":\\"breakout\\"}]"}\n:::\n')
})
test('refuses the content a directive body has no room for', () => {
assert.equal(code(adfToMarkdown(document({ content: [paragraph()], type: 'media' }))), 'unsupported-node-shape')
assert.equal(code(adfToMarkdown(document({ text: 'x', type: 'panel' }))), 'unsupported-node-shape')
})
test('separates two directive blocks in a container body by one line, two CommonMark blocks by a blank one', () => {
const text = (value: string): AdfNode => ({ content: [{ text: value, type: 'text' }], type: 'paragraph' })
const panel = (...content: AdfNode[]): AdfDocument => document({ attrs: { panelType: 'info' }, content, type: 'panel' })
assert.equal(markdown(adfToMarkdown(panel(text('a'), text('b')))), ':::panel info\na\n\nb\n:::\n')
assert.equal(markdown(adfToMarkdown(panel({ type: 'caption' }, { type: 'caption' }))), '::::panel info\n:::caption\n:::\n:::caption\n:::\n::::\n')
assert.equal(code(adfToMarkdown(panel(text('a'), { type: 'caption' }))), 'unspelled-block-separation')
assert.equal(code(adfToMarkdown(panel({ type: 'caption' }, text('a')))), 'unspelled-block-separation')
assert.equal(code(adfToMarkdown(panel(paragraph(), text('a')))), 'unspelled-block-separation')
})
test('spells the image form for exactly the centered external media shape', () => {
const url = 'https://example.com/moon.png'
const single = (attrs: AdfAttributes, ...content: AdfNode[]): AdfDocument =>
document({ attrs: { layout: 'center' }, content: [{ attrs, content, type: 'media' }], type: 'mediaSingle' })
assert.equal(markdown(adfToMarkdown(single({ alt: 'The moon', type: 'external', url }))), `![The moon](${url})\n`)
assert.equal(markdown(adfToMarkdown(single({ type: 'external', url }))), `![](${url})\n`)
assert.equal(markdown(adfToMarkdown(single({ alt: 'a [b] c', type: 'external', url }))), `![a \\[b\\] c](${url})\n`)
const fallback = (attrs: AdfAttributes): boolean => markdown(adfToMarkdown(single(attrs))).startsWith(':::mediaSingle {layout=center}')
assert.ok(fallback({ alt: '', type: 'external', url }))
assert.ok(fallback({ alt: 'a\nb', type: 'external', url }))
assert.ok(fallback({ alt: 'a\u0000b', type: 'external', url }))
assert.ok(fallback({ type: 'external', url: 'https://example.com/a b>c' }))
assert.ok(fallback({ alt: ' moon ', type: 'external', url }))
assert.equal(code(adfToMarkdown(single({ alt: 4, type: 'external', url }))), 'unsupported-node-shape')
assert.equal(code(adfToMarkdown(single({ type: 'external', url: 4 }))), 'unsupported-node-shape')
assert.equal(code(adfToMarkdown(single({ type: 'external', url }, paragraph()))), 'unsupported-node-shape')
})
test('spells a mediaSingle the image form does not fit as a directive', () => {
const media: AdfNode = { attrs: { type: 'external', url: 'https://example.com/moon.png' }, type: 'media' }
const single: AdfNode = { attrs: { layout: 'center' }, content: [media], marks: [{ type: 'border' }], type: 'mediaSingle' }
assert.equal(
markdown(adfToMarkdown(document(single))),
':::mediaSingle {layout=center marks="[{\\"type\\":\\"border\\"}]"}\n::media {type=external url="https://example.com/moon.png"}\n:::\n',
)
})
test('spells a table as a pipe table only where every row and cell is plain', () => {
const text = (value: string): AdfNode => ({ content: [{ text: value, type: 'text' }], type: 'paragraph' })
const cell = (type: string, ...content: AdfNode[]): AdfNode => ({ content, type })
const row = (...cells: AdfNode[]): AdfNode => ({ content: cells, type: 'tableRow' })
const table = (...rows: AdfNode[]): AdfDocument => document({ content: rows, type: 'table' })
const directive = (result: Result<string>): boolean => markdown(result).startsWith(':')
const header = row(cell('tableHeader', text('Part')))
assert.equal(markdown(adfToMarkdown(table(header, row(cell('tableCell', text('Bolt M8')))))), '| Part |\n| --- |\n| Bolt M8 |\n')
assert.equal(markdown(adfToMarkdown(table(row(cell('tableHeader', text('a|b')))))), '| a\\|b |\n| --- |\n')
assert.ok(directive(adfToMarkdown(table())))
assert.ok(directive(adfToMarkdown(table(row()))))
assert.ok(directive(adfToMarkdown(table({ type: 'tableRow' }))))
assert.ok(directive(adfToMarkdown(table(header, row()))))
assert.ok(directive(adfToMarkdown(table(row(cell('tableCell', text('Bolt M8')))))))
assert.ok(directive(adfToMarkdown(table(cell('tableHeader', text('Part'))))))
assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader'))))))
assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', text('a'), text('b')))))))
assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { attrs: { localId: 'a' }, type: 'paragraph' }))))), 'unspelled-node-attribute')
assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', { attrs: { level: 1 }, type: 'heading' }))))))
assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: ' a', type: 'text' }], type: 'paragraph' }))))), 'unspellable-whitespace')
const marked = (mark: AdfMark): AdfDocument => table(row(cell('tableHeader', { content: [{ marks: [mark], text: 'l', type: 'text' }], type: 'paragraph' })))
assert.ok(directive(adfToMarkdown(marked({ attrs: { href: 'https://example.com/?x|y' }, type: 'link' }))))
assert.ok(directive(adfToMarkdown(marked({ attrs: { href: 'https://example.com/', title: 'a|b' }, type: 'link' }))))
assert.equal(markdown(adfToMarkdown(marked({ attrs: { href: 'https://example.com/x' }, type: 'link' }))), '| [l](https://example.com/x) |\n| --- |\n')
})
+144 -55
View File
@@ -1,46 +1,75 @@
import type { AdfDocument, AdfNode } from './adf-document.ts'
import type { BlockDirective } from './block-directives.ts'
import type { JsonValue } from './json-value.ts'
import { blockDirective, spellDirectiveHeader } from './block-directives.ts'
import { emitImage } from './markdown-image.ts'
import { emitInlineLine } from './markdown-inline.ts'
import { emitPipeTable } from './markdown-pipe-table.ts'
import { failure, success, type ConvertErrorPath, type Result } from './result.ts'
import { holdsNullCharacter, isThematicBreak } from './commonmark-grammar.ts'
import { isAdfDocument } from './adf-document.ts'
import { largestNesting } from './nesting.ts'
import { longestBacktickRun } from './backtick-runs.ts'
type BlockContainer = 'directive' | 'document' | 'list-item'
type BlockSpelling = 'commonmark' | 'directive'
type EmittedBody = { fenceColons: number; text: string }
type EmittedBlock = EmittedBody & { spelling: BlockSpelling }
type PlacedBlock = EmittedBlock & { node: AdfNode; path: ConvertErrorPath }
const largestListMarker = 999999999
const listTypes = ['bulletList', 'orderedList']
export function adfToMarkdown(document: AdfDocument): Result<string> {
if (!isAdfDocument(document)) return failure('not-an-adf-document', 'the value is not an ADF document', [])
if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, [])
const blocks = emitBlocks(document.content ?? [], false, [], 0)
const blocks = emitBlocks(document.content ?? [], 'document', [], 0)
if (!blocks.ok) return blocks
return success(blocks.value === '' ? '' : `${blocks.value}\n`)
return success(blocks.value.text === '' ? '' : `${blocks.value.text}\n`)
}
function emitBlocks(nodes: readonly AdfNode[], inListItem: boolean, path: ConvertErrorPath, depth: number): Result<string> {
function emitBlocks(nodes: readonly AdfNode[], container: BlockContainer, path: ConvertErrorPath, depth: number): Result<EmittedBody> {
if (depth > largestNesting) return failure('unsupported-node-shape', `the document nests deeper than the ${largestNesting} levels the emitter carries`, path)
let output = ''
let previous: AdfNode | undefined
const blocks: PlacedBlock[] = []
for (const [index, node] of nodes.entries()) {
const nodePath = [...path, 'content', index]
if (previous !== undefined) {
if (listTypes.includes(node.type) && previous.type === node.type) {
return failure('unspellable-adjacent-lists', `two adjacent ${node.type} nodes read back as one list`, nodePath)
}
if (inListItem && listTypes.includes(node.type)) {
if (!interruptsParagraph(node)) {
return failure('unspellable-line-start', `a ${node.type} that cannot interrupt the block above it has no tight spelling`, nodePath)
}
output += '\n'
} else output += '\n\n'
}
const block = emitBlock(node, nodePath, depth)
if (!block.ok) return block
output += block.value
previous = node
blocks.push({ ...block.value, node, path: nodePath })
}
return success(output)
let fenceColons = 0
let text = ''
for (const [index, block] of blocks.entries()) {
const previous = blocks[index - 1]
if (previous !== undefined) {
const separation = separationBetween(previous, block, container)
if (!separation.ok) return separation
text += separation.value
}
fenceColons = Math.max(fenceColons, block.fenceColons)
text += block.text
}
return success({ fenceColons, text })
}
function separationBetween(previous: PlacedBlock, next: PlacedBlock, container: BlockContainer): Result<string> {
if (listTypes.includes(next.node.type) && previous.node.type === next.node.type) {
return failure('unspellable-adjacent-lists', `two adjacent ${next.node.type} nodes read back as one list`, next.path)
}
if (container === 'list-item' && listTypes.includes(next.node.type)) {
if (!interruptsParagraph(next.node)) {
return failure('unspellable-line-start', `a ${next.node.type} that cannot interrupt the block above it has no tight spelling`, next.path)
}
return success('\n')
}
if (container !== 'directive') return success('\n\n')
if (previous.spelling === 'commonmark' && next.spelling === 'commonmark') return success('\n\n')
if (previous.spelling === 'directive' && next.spelling === 'directive') return success('\n')
return failure(
'unspelled-block-separation',
`the canonical form leaves the separation between a ${previous.spelling} and a ${next.spelling} block in a container body unspelled`,
next.path,
)
}
function interruptsParagraph(node: AdfNode): boolean {
@@ -48,30 +77,81 @@ function interruptsParagraph(node: AdfNode): boolean {
return ((node.content ?? [])[0]?.content ?? []).length > 0
}
function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result<string> {
if (node.type === 'blockquote') return emitBlockquote(node, path, depth)
if (node.type === 'bulletList' || node.type === 'orderedList') return emitList(node, path, depth)
if (node.type === 'codeBlock') return emitCodeBlock(node, path)
if (node.type === 'heading') return emitHeading(node, path)
function emitBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result<EmittedBlock> {
if (node.type === 'blockquote') return commonMarkContainer(emitBlockquote(node, path, depth))
if (node.type === 'bulletList' || node.type === 'orderedList') return commonMarkContainer(emitList(node, path, depth))
if (node.type === 'codeBlock') return commonMarkLine(emitCodeBlock(node, path))
if (node.type === 'heading') return commonMarkLine(emitHeading(node, path))
if (node.type === 'paragraph') return emitParagraph(node, path)
if (node.type === 'rule') return emitRule(node, path)
if (node.type === 'rule') return commonMarkLine(emitRule(node, path))
const directive = blockDirective(node.type)
if (directive !== undefined) {
if (node.type === 'mediaSingle') return emitMediaSingle(node, directive, path, depth)
if (node.type === 'table') return emitTable(node, directive, path, depth)
return emitDirectiveBlock(node, directive, path, depth)
}
if (node.type === 'hardBreak' || node.type === 'listItem' || node.type === 'text') {
return failure('unsupported-node-shape', `a ${node.type} node cannot stand where a block belongs`, path)
}
return failure('unsupported-node-type', `the canonical form spells no block node of type ${node.type}`, path)
}
function emitBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number): Result<string> {
function commonMarkLine(text: Result<string>): Result<EmittedBlock> {
if (!text.ok) return text
return success({ fenceColons: 0, spelling: 'commonmark', text: text.value })
}
function commonMarkContainer(body: Result<EmittedBody>): Result<EmittedBlock> {
if (!body.ok) return body
return success({ ...body.value, spelling: 'commonmark' })
}
function emitDirectiveBlock(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result<EmittedBlock> {
if (node.text !== undefined) return failure('unsupported-node-shape', `a ${node.type} carries no text`, path)
const header = spellDirectiveHeader(node, directive, path)
if (!header.ok) return header
const content = node.content ?? []
if (directive.body === 'none') {
if (content.length > 0) return failure('unsupported-node-shape', `a ${node.type} holds no content`, path)
return success({ fenceColons: 2, spelling: 'directive', text: `::${header.value}` })
}
const body = directive.body === 'inline' ? emitInlineBody(content, path) : emitBlocks(content, 'directive', path, depth + 1)
if (!body.ok) return body
const fenceColons = Math.max(3, body.value.fenceColons + 1)
const fence = ':'.repeat(fenceColons)
const lines = body.value.text === '' ? '' : `${body.value.text}\n`
return success({ fenceColons, spelling: 'directive', text: `${fence}${header.value}\n${lines}${fence}` })
}
function emitInlineBody(content: readonly AdfNode[], path: ConvertErrorPath): Result<EmittedBody> {
if (content.length === 0) return success({ fenceColons: 0, text: '' })
const line = emitInlineLine(content, 'paragraph', path)
if (!line.ok) return line
return success({ fenceColons: 0, text: line.value })
}
function emitMediaSingle(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result<EmittedBlock> {
const image = emitImage(node, path)
if (image === undefined) return emitDirectiveBlock(node, directive, path, depth)
return commonMarkLine(image)
}
function emitTable(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath, depth: number): Result<EmittedBlock> {
const pipe = emitPipeTable(node, path)
if (pipe === undefined) return emitDirectiveBlock(node, directive, path, depth)
return commonMarkLine(pipe)
}
function emitBlockquote(node: AdfNode, path: ConvertErrorPath, depth: number): Result<EmittedBody> {
const validation = validateBlockNode(node, [], path)
if (!validation.ok) return validation
const inner = emitBlocks(node.content ?? [], false, path, depth + 1)
const inner = emitBlocks(node.content ?? [], 'document', path, depth + 1)
if (!inner.ok) return inner
return success(
inner.value
const text = inner.value.text
.split('\n')
.map((line) => (line === '' ? '>' : `> ${line}`))
.join('\n'),
)
.join('\n')
return success({ fenceColons: inner.value.fenceColons, text })
}
function emitCodeBlock(node: AdfNode, path: ConvertErrorPath): Result<string> {
@@ -98,7 +178,7 @@ function spellCodeFenceInfo(language: JsonValue | undefined, path: ConvertErrorP
if (language === undefined) return success('')
if (typeof language !== 'string') return failure('unsupported-node-shape', 'a codeBlock language is no string', path)
if (language === '') {
return failure('ambiguous-empty-code-block-language', 'an empty codeBlock language and an absent one share one markdown spelling', path)
return failure('ambiguous-attribute-spelling', 'an empty codeBlock language and an absent one share one markdown spelling', path)
}
if (language === 'adf') return failure('reserved-adf-language', 'the adf info string is reserved for the opaque carry', path)
if (/[`\n\r]/.test(language) || language !== language.trim()) {
@@ -122,54 +202,63 @@ function emitHeading(node: AdfNode, path: ConvertErrorPath): Result<string> {
return success(`${hashes} ${line.value}`)
}
function emitList(node: AdfNode, path: ConvertErrorPath, depth: number): Result<string> {
function emitList(node: AdfNode, path: ConvertErrorPath, depth: number): Result<EmittedBody> {
const ordered = node.type === 'orderedList'
const validation = validateBlockNode(node, ordered ? ['order'] : [], path)
if (!validation.ok) return validation
const items = node.content ?? []
if (items.length === 0) return failure('unsupported-node-shape', `a ${node.type} holds at least one listItem`, path)
const start = ordered ? node.attrs?.['order'] : 0
if (ordered && (start === undefined || start === 1)) {
return failure('ambiguous-ordered-list-start', 'an orderedList starting at 1 and one with no order share one markdown spelling', path)
}
if (typeof start !== 'number' || !Number.isInteger(start) || start < 0 || start > largestListMarker) {
return failure('unsupported-node-shape', `no list marker spells the order ${JSON.stringify(start ?? null)}`, path)
}
if (start + items.length - 1 > largestListMarker) {
return failure('unspellable-list-marker', `no list marker spells the ${items.length} items a list starting at ${start} needs`, path)
}
const start = listStart(node, items.length, path)
if (!start.ok) return start
const lines: string[] = []
let fenceColons = 0
for (const [offset, item] of items.entries()) {
const itemPath = [...path, 'content', offset]
if (item.type !== 'listItem') return failure('unsupported-node-shape', `a ${node.type} holds listItem nodes only`, itemPath)
const emitted = emitListItem(item, ordered ? `${start + offset}. ` : '- ', itemPath, depth)
const emitted = emitListItem(item, ordered ? `${start.value + offset}. ` : '- ', itemPath, depth)
if (!emitted.ok) return emitted
lines.push(emitted.value)
fenceColons = Math.max(fenceColons, emitted.value.fenceColons)
lines.push(emitted.value.text)
}
return success(lines.join('\n'))
return success({ fenceColons, text: lines.join('\n') })
}
function emitListItem(item: AdfNode, marker: string, path: ConvertErrorPath, depth: number): Result<string> {
function listStart(node: AdfNode, items: number, path: ConvertErrorPath): Result<number> {
if (items === 0) return failure('unsupported-node-shape', `a ${node.type} holds at least one listItem`, path)
if (node.type !== 'orderedList') return success(0)
const start = node.attrs?.['order']
if (start === undefined || start === 1) {
return failure('ambiguous-attribute-spelling', 'an orderedList starting at 1 and one with no order share one markdown spelling', path)
}
if (typeof start !== 'number' || !Number.isInteger(start) || start < 0 || start > largestListMarker) {
return failure('unsupported-node-shape', `no list marker spells the order ${JSON.stringify(start)}`, path)
}
if (start + items - 1 > largestListMarker) {
return failure('unspellable-list-marker', `no list marker spells the ${items} items a list starting at ${start} needs`, path)
}
return success(start)
}
function emitListItem(item: AdfNode, marker: string, path: ConvertErrorPath, depth: number): Result<EmittedBody> {
const validation = validateBlockNode(item, [], path)
if (!validation.ok) return validation
const inner = emitBlocks(item.content ?? [], true, path, depth + 1)
const inner = emitBlocks(item.content ?? [], 'list-item', path, depth + 1)
if (!inner.ok) return inner
if (inner.value === '') return success(marker.trimEnd())
if (inner.value.text === '') return success({ fenceColons: 0, text: marker.trimEnd() })
const indent = ' '.repeat(marker.length)
const lines = inner.value.split('\n').map((line, index) => (index === 0 ? `${marker}${line}` : line === '' ? '' : `${indent}${line}`))
const lines = inner.value.text.split('\n').map((line, index) => (index === 0 ? `${marker}${line}` : line === '' ? '' : `${indent}${line}`))
const first = lines[0] ?? ''
if (isThematicBreak(first)) {
return failure('unspellable-line-start', `block parsing would claim the emitted line ${JSON.stringify(first)}`, path)
}
return success(lines.join('\n'))
return success({ fenceColons: inner.value.fenceColons, text: lines.join('\n') })
}
function emitParagraph(node: AdfNode, path: ConvertErrorPath): Result<string> {
function emitParagraph(node: AdfNode, path: ConvertErrorPath): Result<EmittedBlock> {
const validation = validateBlockNode(node, [], path)
if (!validation.ok) return validation
const content = node.content ?? []
if (content.length === 0) return success('::paragraph')
return emitInlineLine(content, 'paragraph', path)
if (content.length === 0) return success({ fenceColons: 2, spelling: 'directive', text: '::paragraph' })
return commonMarkLine(emitInlineLine(content, 'paragraph', path))
}
function emitRule(node: AdfNode, path: ConvertErrorPath): Result<string> {
+115
View File
@@ -0,0 +1,115 @@
import type { AdfMark, AdfNode, AttributeKind } from './adf-document.ts'
import type { JsonValue } from './json-value.ts'
import { failure, success, type ConvertErrorPath, type Result } from './result.ts'
import { isBareToken, spellAttributeValue, spellAttributes, spellJsonAttribute } from './directive-attributes.ts'
export type BlockDirective = {
argument?: string
attributes: Readonly<Record<string, AttributeKind>>
body: 'block' | 'inline' | 'none'
}
const cellAttributes: Readonly<Record<string, AttributeKind>> = {
background: 'string',
colspan: 'number',
colwidth: 'json',
localId: 'string',
rowspan: 'number',
valign: 'string',
}
const expandAttributes: Readonly<Record<string, AttributeKind>> = { localId: 'string', title: 'string' }
const extensionAttributes: Readonly<Record<string, AttributeKind>> = {
extensionKey: 'string',
extensionType: 'string',
layout: 'string',
localId: 'string',
parameters: 'json',
text: 'string',
}
const itemAttributes: Readonly<Record<string, AttributeKind>> = { localId: 'string' }
const mediaAttributes: Readonly<Record<string, AttributeKind>> = {
alt: 'string',
collection: 'string',
height: 'number',
id: 'string',
localId: 'string',
occurrenceKey: 'string',
type: 'string',
url: 'string',
width: 'number',
}
const syncBlockAttributes: Readonly<Record<string, AttributeKind>> = { localId: 'string', resourceId: 'string' }
const blockDirectives: Readonly<Record<string, BlockDirective>> = {
blockTaskItem: { argument: 'state', attributes: itemAttributes, body: 'block' },
bodiedExtension: { attributes: extensionAttributes, body: 'block' },
bodiedSyncBlock: { attributes: syncBlockAttributes, body: 'block' },
caption: { attributes: itemAttributes, body: 'inline' },
decisionItem: { attributes: { localId: 'string', state: 'string' }, body: 'inline' },
decisionList: { attributes: itemAttributes, body: 'block' },
expand: { attributes: expandAttributes, body: 'block' },
extension: { attributes: extensionAttributes, body: 'none' },
extensionFrame: { attributes: {}, body: 'block' },
layoutColumn: { attributes: { localId: 'string', valign: 'string', width: 'number' }, body: 'block' },
layoutSection: { attributes: itemAttributes, body: 'block' },
media: { attributes: mediaAttributes, body: 'none' },
mediaGroup: { attributes: {}, body: 'block' },
mediaSingle: { attributes: { layout: 'string', localId: 'string', width: 'number', widthType: 'string' }, body: 'block' },
multiBodiedExtension: { attributes: extensionAttributes, body: 'block' },
nestedExpand: { attributes: expandAttributes, body: 'block' },
panel: {
argument: 'panelType',
attributes: { localId: 'string', panelColor: 'string', panelIcon: 'string', panelIconId: 'string', panelIconText: 'string' },
body: 'block',
},
syncBlock: { attributes: syncBlockAttributes, body: 'none' },
table: { attributes: { displayMode: 'string', isNumberColumnEnabled: 'boolean', layout: 'string', localId: 'string', width: 'number' }, body: 'block' },
tableCell: { attributes: cellAttributes, body: 'block' },
tableHeader: { attributes: cellAttributes, body: 'block' },
tableRow: { attributes: itemAttributes, body: 'block' },
taskItem: { argument: 'state', attributes: itemAttributes, body: 'inline' },
taskList: { attributes: itemAttributes, body: 'block' },
}
export function blockDirective(type: string): BlockDirective | undefined {
return Object.hasOwn(blockDirectives, type) ? blockDirectives[type] : undefined
}
export function spellDirectiveHeader(node: AdfNode, directive: BlockDirective, path: ConvertErrorPath): Result<string> {
const pairs: [string, string][] = []
let argument = ''
for (const [key, value] of Object.entries(node.attrs ?? {})) {
if (key === directive.argument) {
if (typeof value !== 'string' || !isBareToken(value)) {
return failure('unspelled-node-attribute', `the ${node.type} attribute ${key} holds no bare token the arg slot spells`, path)
}
argument = ` ${value}`
continue
}
const kind = attributeKind(directive, key)
if (kind === undefined) return failure('unspelled-node-attribute', `the ${node.type} attribute ${key} has no canonical markdown spelling`, path)
const spelled = spellAttributeValue(value, kind)
if (spelled === undefined) return failure('unsupported-node-shape', `the ${node.type} attribute ${key} holds no ${kind}`, path)
pairs.push([key, spelled])
}
const marks = node.marks ?? []
if (marks.length > 0) pairs.push(['marks', spellJsonAttribute(markValues(marks))])
const attributes = spellAttributes(pairs)
return success(`${node.type}${argument}${attributes === '' ? '' : ` ${attributes}`}`)
}
function attributeKind(directive: BlockDirective, key: string): AttributeKind | undefined {
return Object.hasOwn(directive.attributes, key) ? directive.attributes[key] : undefined
}
function markValues(marks: readonly AdfMark[]): JsonValue {
return marks.map((mark) => {
const attrs = mark.attrs ?? {}
return Object.keys(attrs).length === 0 ? { type: mark.type } : { attrs, type: mark.type }
})
}
+71 -3
View File
@@ -11,9 +11,10 @@ import { serializeCanonicalJson } from './canonical-json.ts'
const corpusRoot = join(dirname(fileURLToPath(import.meta.url)), '..', 'corpus')
const roundTripRoot = join(corpusRoot, 'round-trip')
const unspellableRoot = join(corpusRoot, 'unspellable')
const emittingDirectories = ['commonmark-subset']
const pendingDirectories = ['block-nodes', 'inline-nodes']
const emittingDirectories = ['block-nodes', 'commonmark-subset']
const pendingDirectories = ['inline-nodes']
function directoryNames(root: string): string[] {
return readdirSync(root, { withFileTypes: true })
@@ -23,7 +24,11 @@ function directoryNames(root: string): string[] {
}
function fixtureNames(directory: string, extension: string): string[] {
return readdirSync(join(roundTripRoot, directory))
return names(join(roundTripRoot, directory), extension)
}
function names(root: string, extension: string): string[] {
return readdirSync(root)
.filter((name) => name.endsWith(extension))
.map((name) => name.slice(0, -extension.length))
.sort()
@@ -67,6 +72,69 @@ for (const directory of emittingDirectories) {
}
}
// spec/flavour.md, Directives: the container fence rule, checked against the emitted bytes.
function fenceNestingFault(markdown: string): string | undefined {
const open: number[] = []
let codeFence: string | undefined
for (const line of markdown.split('\n')) {
const content = line.replace(/^[ \t]*(?:(?:> ?|[-*+] |\d{1,9}[.)] )[ \t]*)*/, '')
const backticks = /^(`{3,}|~{3,})/.exec(content)?.[1]
if (codeFence !== undefined) {
if (backticks !== undefined && backticks[0] === codeFence[0] && backticks.length >= codeFence.length) codeFence = undefined
continue
}
if (backticks !== undefined) {
codeFence = backticks
continue
}
const colons = /^(:{2,})(.*)$/.exec(content)
if (colons === null) continue
const run = colons[1]?.length ?? 0
const enclosing = open[open.length - 1]
if (colons[2] === '') {
open.pop()
continue
}
if (enclosing !== undefined && run >= enclosing) return `${JSON.stringify(line)} sits in a container fenced with ${enclosing} colons`
if (run > 2) open.push(run)
}
return undefined
}
test('the fence nesting check catches a fence a container cannot hold', () => {
assert.equal(fenceNestingFault(':::panel info\n- :::panel warning\n B\n :::\n:::'), '"- :::panel warning" sits in a container fenced with 3 colons')
assert.equal(fenceNestingFault(':::panel info\n- - :::panel warning\n B\n :::\n:::'), '"- - :::panel warning" sits in a container fenced with 3 colons')
assert.equal(fenceNestingFault(':::panel info\n10. :::panel warning\n B\n :::\n:::'), '"10. :::panel warning" sits in a container fenced with 3 colons')
assert.equal(fenceNestingFault('::::panel info\n- - :::panel warning\n B\n :::\n::::'), undefined)
assert.equal(fenceNestingFault(':::tableCell\n```text\n:::::::panel warning\n:::\n```\n:::'), undefined)
})
for (const directory of emittingDirectories) {
for (const name of fixtureNames(directory, '.md')) {
test(`${directory}/${name} fences every container longer than its body`, () => {
assert.equal(fenceNestingFault(readFileSync(join(roundTripRoot, directory, `${name}.md`), 'utf8')), undefined)
})
}
}
test('unspellable pairs every .json with an .error', () => {
assert.deepEqual(names(unspellableRoot, '.json'), names(unspellableRoot, '.error'))
})
test('unspellable holds fixtures', () => {
assert.ok(names(unspellableRoot, '.json').length > 0)
})
for (const name of names(unspellableRoot, '.json')) {
test(`unspellable/${name} is refused with the error it names`, () => {
const parsed: unknown = JSON.parse(readFileSync(join(unspellableRoot, `${name}.json`), 'utf8'))
assert.ok(isAdfDocument(parsed), `${name}.json is not an ADF document`)
const result = adfToMarkdown(parsed)
assert.ok(!result.ok, result.ok ? `emitted ${JSON.stringify(result.value)}` : '')
assert.equal(result.error.code, readFileSync(join(unspellableRoot, `${name}.error`), 'utf8').trimEnd())
})
}
test('the corpus holds JSON to gate', () => {
assert.ok(corpusJsonPaths().length > 0)
})
+34
View File
@@ -0,0 +1,34 @@
import type { AttributeKind } from './adf-document.ts'
import type { JsonValue } from './json-value.ts'
import { serializeCanonicalJson } from './canonical-json.ts'
const bareToken = /^[A-Za-z0-9_-]+$/
export function isBareToken(text: string): boolean {
return bareToken.test(text)
}
export function spellAttributes(pairs: readonly (readonly [string, string])[]): string {
if (pairs.length === 0) return ''
const spelled = [...pairs].sort(([left], [right]) => (left < right ? -1 : 1)).map(([key, value]) => `${key}=${value}`)
return `{${spelled.join(' ')}}`
}
export function spellAttributeValue(value: JsonValue, kind: AttributeKind): string | undefined {
if (kind === 'json') return spellJsonAttribute(value)
if (kind === 'boolean') return typeof value === 'boolean' ? `${value}` : undefined
if (kind === 'number') return typeof value === 'number' ? spell(JSON.stringify(value)) : undefined
return typeof value === 'string' ? spell(value) : undefined
}
export function spellJsonAttribute(value: JsonValue): string {
return quote(serializeCanonicalJson(value, 'compact'))
}
function spell(text: string): string {
return isBareToken(text) ? text : quote(text)
}
function quote(text: string): string {
return JSON.stringify(text)
}
+6 -4
View File
@@ -6,7 +6,7 @@ export type InlineSegment =
export type AssembledLine = { line: string; unspellableMark: string | undefined }
export type LineContainer = 'heading' | 'paragraph'
export type LineContainer = 'heading' | 'paragraph' | 'table-cell'
type DelimiterRun = { character: string; closeMark: string | undefined; end: number; openMark: string | undefined; start: number }
@@ -127,8 +127,9 @@ function isSyntax(kind: InlineSegment['kind'] | undefined): boolean {
}
function opensConstruct(scan: string, index: number, inLinkText: boolean, container: LineContainer, escaped: ReadonlySet<number>): boolean {
const claimsLine = container === 'heading' ? closesHeading(scan, index) : claimsLineStart(scan, index)
return claimsLine || claimsCharacter(scan, index, inLinkText, escaped)
if (container === 'heading' && closesHeading(scan, index)) return true
if (container === 'paragraph' && claimsLineStart(scan, index)) return true
return claimsCharacter(scan, index, inLinkText, container, escaped)
}
function claimsLineStart(scan: string, index: number): boolean {
@@ -144,10 +145,11 @@ function closesHeading(scan: string, index: number): boolean {
return index === 0 || /[ \t]/.test(scan.charAt(index - 1))
}
function claimsCharacter(scan: string, index: number, inLinkText: boolean, escaped: ReadonlySet<number>): boolean {
function claimsCharacter(scan: string, index: number, inLinkText: boolean, container: LineContainer, escaped: ReadonlySet<number>): boolean {
const character = scan.charAt(index)
const rest = scan.slice(index)
if (inLinkText && (character === '[' || character === ']')) return true
if (character === '|') return container === 'table-cell'
if (character === '\\') return asciiPunctuation.test(scan.charAt(index + 1))
if (character === '&') return startsEntityReference(rest)
if (character === '<') return opensBracketedAutolink(rest) || htmlConstructs.some((construct) => construct.test(rest))
+27
View File
@@ -0,0 +1,27 @@
import type { AdfNode } from './adf-document.ts'
import type { ConvertErrorPath, Result } from './result.ts'
import { emitImageLine } from './markdown-inline.ts'
import { serializeCanonicalJson } from './canonical-json.ts'
const centeredMediaSingle = '{"layout":"center"}'
const imageAttributes = ['alt', 'type', 'url']
export function emitImage(node: AdfNode, path: ConvertErrorPath): Result<string> | undefined {
const image = imageShape(node)
if (image === undefined) return undefined
const line = emitImageLine(image.alt, image.url, [...path, 'content', 0])
return line.ok ? line : undefined
}
function imageShape(node: AdfNode): { alt: string | undefined; url: string } | undefined {
const content = node.content ?? []
const media = content[0]
if (serializeCanonicalJson(node.attrs ?? {}, 'compact') !== centeredMediaSingle || (node.marks ?? []).length > 0) return undefined
if (media === undefined || content.length !== 1 || media.type !== 'media' || (media.marks ?? []).length > 0 || (media.content ?? []).length > 0) return undefined
const attrs = media.attrs ?? {}
const alt = attrs['alt']
const url = attrs['url']
if (Object.keys(attrs).some((key) => !imageAttributes.includes(key)) || attrs['type'] !== 'external' || typeof url !== 'string') return undefined
if (alt !== undefined && (typeof alt !== 'string' || alt === '')) return undefined
return { alt, url }
}
+18 -3
View File
@@ -20,7 +20,22 @@ const linkAttributes = ['href', 'title']
export function emitInlineLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath): Result<string> {
const segments = emitRun(nodes, 0, 0, { atBlockEnd: true, container, inLinkText: false, path })
if (!segments.ok) return segments
const assembled = assembleInlineLine(segments.value, container)
return finishLine(segments.value, container, path)
}
export function emitImageLine(alt: string | undefined, href: string, path: ConvertErrorPath): Result<string> {
if (alt !== undefined && /^[ \t]|[ \t]$|[\n\r]/.test(alt)) {
return failure('unspellable-whitespace', 'a media alt holds whitespace no image description spells', path)
}
if (alt !== undefined && holdsNullCharacter(alt)) return failure('unspellable-character', 'a media alt holds a null character CommonMark replaces', path)
const destination = spellDestination(href, path)
if (!destination.ok) return destination
const description: InlineSegment[] = alt === undefined ? [] : [{ kind: 'link-text', text: alt }]
return finishLine([{ kind: 'syntax', text: '![' }, ...description, { kind: 'syntax', text: `](${destination.value})` }], 'paragraph', path)
}
function finishLine(segments: readonly InlineSegment[], container: LineContainer, path: ConvertErrorPath): Result<string> {
const assembled = assembleInlineLine(segments, container)
if (assembled.unspellableMark !== undefined) {
return failure('unspellable-mark', `the ${assembled.unspellableMark} spelling cannot open or close where it sits`, path)
}
@@ -83,8 +98,8 @@ function emitLeaf(node: AdfNode, context: InlineContext, index: number): Result<
const types = (node.marks ?? []).map((mark) => mark.type)
if (new Set(types).size !== types.length) return failure('unsupported-node-shape', `a ${node.type} node carries one mark type twice`, path)
if (node.type === 'hardBreak') {
if (context.container === 'heading' || context.atBlockEnd) return success([{ kind: 'syntax', text: ':hardBreak{}' }])
return success([{ kind: 'syntax', text: '\\\n' }])
if (context.container === 'paragraph' && !context.atBlockEnd) return success([{ kind: 'syntax', text: '\\\n' }])
return success([{ kind: 'syntax', text: ':hardBreak{}' }])
}
if (typeof node.text !== 'string' || node.text === '') return failure('unsupported-node-shape', 'a text node carries no text', path)
if ((node.content ?? []).length > 0) return failure('unsupported-node-shape', 'a text node carries content', path)
+66
View File
@@ -0,0 +1,66 @@
import type { AdfNode } from './adf-document.ts'
import type { JsonValue } from './json-value.ts'
import { emitInlineLine } from './markdown-inline.ts'
import { success, type ConvertErrorPath, type Result } from './result.ts'
export function emitPipeTable(node: AdfNode, path: ConvertErrorPath): Result<string> | undefined {
const rows = pipeRows(node)
if (rows === undefined) return undefined
const lines: string[] = []
for (const [rowIndex, row] of rows.entries()) {
const cells: string[] = []
for (const [cellIndex, paragraph] of row.entries()) {
const content = paragraph.content ?? []
const line = content.length === 0 ? success('') : emitInlineLine(content, 'table-cell', [...path, 'content', rowIndex, 'content', cellIndex, 'content', 0])
if (!line.ok) return line
cells.push(line.value)
}
lines.push(`| ${cells.join(' | ')} |`)
if (rowIndex === 0) lines.push(`| ${cells.map(() => '---').join(' | ')} |`)
}
return success(lines.join('\n'))
}
function pipeRows(node: AdfNode): AdfNode[][] | undefined {
const rows = node.content ?? []
const columns = (rows[0]?.content ?? []).length
if (!isPlain(node) || columns === 0) return undefined
const grid: AdfNode[][] = []
for (const [index, row] of rows.entries()) {
const cells = row.content ?? []
if (row.type !== 'tableRow' || !isPlain(row) || cells.length !== columns) return undefined
const wanted = index === 0 ? 'tableHeader' : 'tableCell'
const paragraphs: AdfNode[] = []
for (const cell of cells) {
const paragraph = plainParagraph(cell)
if (paragraph === undefined || cell.type !== wanted || !isPlain(cell)) return undefined
paragraphs.push(paragraph)
}
grid.push(paragraphs)
}
return grid
}
function isPlain(node: AdfNode): boolean {
return Object.keys(node.attrs ?? {}).length === 0 && (node.marks ?? []).length === 0 && node.text === undefined
}
function plainParagraph(cell: AdfNode): AdfNode | undefined {
const content = cell.content ?? []
const paragraph = content[0]
if (paragraph === undefined || content.length !== 1 || paragraph.type !== 'paragraph' || !isPlain(paragraph)) return undefined
return (paragraph.content ?? []).some(spellsPipeAsSyntax) ? undefined : paragraph
}
// A pipe the inline layer emits as syntax takes no backslash, so the cell has no pipe spelling.
function spellsPipeAsSyntax(child: AdfNode): boolean {
return (child.marks ?? []).some((mark) => {
if (mark.type === 'code') return (child.text ?? '').includes('|')
if (mark.type !== 'link') return false
return holdsPipe(mark.attrs?.['href']) || holdsPipe(mark.attrs?.['title'])
})
}
function holdsPipe(value: JsonValue | undefined): boolean {
return typeof value === 'string' && value.includes('|')
}
+2 -2
View File
@@ -1,6 +1,5 @@
export type ConvertErrorCode =
| 'ambiguous-empty-code-block-language'
| 'ambiguous-ordered-list-start'
| 'ambiguous-attribute-spelling'
| 'not-an-adf-document'
| 'reserved-adf-language'
| 'unspellable-adjacent-lists'
@@ -13,6 +12,7 @@ export type ConvertErrorCode =
| 'unspellable-mark'
| 'unspellable-whitespace'
| 'unspelled-block-marks'
| 'unspelled-block-separation'
| 'unspelled-node-attribute'
| 'unsupported-document-version'
| 'unsupported-heading-level'
+31 -14
View File
@@ -28,21 +28,24 @@ detail is settled at its own milestone.
carries a `localId` with no spelling, `codeBlock` also `hideLineNumbers`, `uniqueId` and
`wrap`, `blockquote` also marks, and `hardBreak` `text` and `localId` with no section for
the carry fallback to reach. Picking one (directive sections for those nodes, or the opaque
carry) is a permanent format decision (§8). Three collision sites are held out of the corpus
meanwhile, each a choice between the absent attribute and the empty value: a `codeBlock`
whose info string is empty and `media` with an empty `alt`, which one "exactly that shape"
rule — as the CommonMark image already uses — could settle together, and an `orderedList`
starting at 1, independent of the totality answer since `order: 9` keeps the markdown form
either way. **Also blocked**: the link rule covers destination spaces only, so two shapes
break §2 silently — href `https://example.com/a)b` emits `[t](https://example.com/a)b)`,
read back as href `…/a` plus literal `b)`; title `He said "hi"` emits
`[t](u "He said "hi"")`, which holds no title. Two defensible spellings each — angle
carry) is a permanent format decision (§8). Two collision sites wait in
`corpus/unspellable/` meanwhile, each a choice between the absent attribute and the empty
value: a `codeBlock` whose info string is empty, and an `orderedList` starting at 1,
independent of the totality answer since `order: 9` keeps the markdown form either way.
Neither has a second spelling to fall back to, which is what settled the third — a `media`
with an empty `alt` takes the directive form (`spec/flavour.md`, the CommonMark image).
**Also blocked**: the link rule covers destination spaces only, so two shapes
have no spelling and are refused meanwhile — href `https://example.com/a)b` and title
`He said "hi"`, both in `corpus/unspellable/`. Two defensible spellings each — angle
brackets or a backslash escape, and for titles `'…'` or `(…)` besides — so §8 leaves
the pick here. **Also blocked**: block separation is unstated for a CommonMark block beside a
directive block in a container body — an `expand` whose content is `paragraph` "A" then a
`panel` (`panelType` `warning`) holding "B" spells `A` and `:::panel warning` either on
consecutive lines or with a blank line between. Two defensible spellings, so §8 leaves the
pick here.
pick here; `unspelled-block-separation` refuses the pair meanwhile, an empty paragraph's
`::paragraph` beside a CommonMark block included — and, since a `mediaSingle`'s spelling now
follows whether CommonMark can spell its URL, two sibling images differing only by an
`&amp;` land in the same refusal.
- [x] **1d1 — The CommonMark subset**: blockquote, bulletList, codeBlock, heading, orderedList,
paragraph, rule, listItem, hardBreak, text, code spans, and the `code`, `em`, `link`,
`strike` and `strong` marks — one mark per text node; nesting is 1d3's.
@@ -70,7 +73,11 @@ detail is settled at its own milestone.
a `.json` beside the `ConvertErrorCode` it must return, the emitter half of `corpus/errors/`.
- [ ] **2c — Inline nodes and marks.** `inline-nodes/` green. `InlineSegment.kind` splits into
its two axes here — escapability (`attribute` for `:text{text="…"}`, `backslash`, `none`)
and the emphasis role — rather than gaining a third value that means one of each.
and the emphasis role — rather than gaining a third value that means one of each. A lone
surrogate in a text node emits verbatim and becomes U+FFFD on any UTF-8 encode, a §2 break
plain text still holds open — attribute values already escape it. The inline directives
arriving here emit their attribute values as syntax, so `spellsPipeAsSyntax` grows with
them alongside the `|` escaping `spec/flavour.md` already mandates in a pipe cell.
- [ ] **2d — The opaque carry** (§3). Fixtures and emitter together, into
`corpus/round-trip/opaque-carry/`: an unknown node in both positions, the reserved `adf`
info string, and the `codeBlock` whose language is `adf`.
@@ -81,6 +88,8 @@ detail is settled at its own milestone.
attribute canonicalization, a pipe cell's whitespace edges and `\u007c` for a `|` inside a
quoted attribute value, documents combining nodes rather than isolating one, and a
paragraph line inside a container body shaped like a closing fence (`:::`, `::: x`).
Guard `fenceNestingFault`'s bare-run pop here too — a run shorter than the open fence is a
fault, not a close — which today's emitter cannot reach.
The gate gains the collision property here: no two corpus documents may emit the same
bytes — one spelling for two documents is a round-trip break no parser can undo, and it is
provable without one. It also settles the emitter's one known approximation: delimiter
@@ -99,16 +108,24 @@ detail is settled at its own milestone.
assumes — CommonMark flanking, as for `*` — which `spec/flavour.md` does not yet pin.
`src/` gets its hierarchy at the same split — `adf/`,
`markdown/`, `html/`, the grammar module shared inside `markdown/` — while the rename is
still mechanical.
still mechanical. `block-directives.ts` is the one file that does not move whole: the node
table is ADF knowledge milestones 6-7 need too and belongs in `adf/`, `spellDirectiveHeader`
in `markdown/`. `AttributeKind` stays above both — it is the vocabulary a string-typed
attribute grammar needs, which is why HTML will want it too, not a markdown spelling. The table is a second copy of `spec/flavour.md`'s prose with no drift guard,
and a mistyped attribute name degrades into a false refusal no test catches.
- [ ] **4 — Round-trip property tests** over the corpus, both ways — the thing that proves 2 and
3. Editor-normal (§2) gets its implementation here — `toEditorNormal(doc)` and the equality
the round-trip asserts, which over normalized input is the canonical serializer's compact
spelling — rather than staying spelled inline as `?? []` at every reader.
spelling — rather than staying spelled inline as `?? []` at every reader. The reading half is
`nodeContent`/`nodeAttrs`/`nodeMarks` over the ~28 sites spelling it inline today, which also
lifts the branch floor §10 keeps below 100 for exactly those halves.
Generators emit editor-normal ADF (§2). Real sanitized ADF from live Atlassian APIs lands
here too (§10), in `corpus/real-payloads/`: an ADF→markdown→ADF check with no expected
markdown, the payloads supplied by the maintainer.
- [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret,
the repo made public first (§6). `0.1.0` is the markdown round-trip: both markdown
the repo made public first (§6). The `ConvertErrorCode` freeze (§8) is checkable here: every
`corpus/unspellable/` document is one of 1d's decisions, so the directory empties as they land
and whatever survives is permanent. `0.1.0` is the markdown round-trip: both markdown
directions, the types, `isAdfDocument`. The build lands here: a build tsconfig emitting JS
and `.d.ts` to `dist/` (the dev config's `allowImportingTsExtensions` forces `noEmit`, so
the build config needs `rewriteRelativeImportExtensions`), plus `exports`/`files` in