Escape the quoted attribute value, and pin the canonical form and the pipe cell's edges
CI / gate (push) Successful in 5s

This commit is contained in:
2026-08-26 23:27:29 +02:00
parent e4704990d7
commit ec0ae1f5db
13 changed files with 273 additions and 44 deletions
@@ -22,5 +22,5 @@ Taken from Kiruna.
::: :::
:::mediaSingle {layout=center} :::mediaSingle {layout=center}
::media {type=external url="https://example.com/plan.png?a=1&b=2"} ::media {type=external url="https://example.com/plan.png?a=1\u0026amp;b=2"}
::: :::
@@ -0,0 +1,60 @@
{
"content": [
{
"attrs": {
"isNumberColumnEnabled": true,
"layout": "wide",
"width": 760.5
},
"content": [
{
"content": [
{
"attrs": {
"colspan": 2,
"colwidth": [
340,
420
]
},
"content": [
{
"content": [
{
"text": "Assembly",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "tableHeader"
}
],
"type": "tableRow"
}
],
"type": "table"
},
{
"attrs": {
"localId": "01a0-33_z",
"title": "a\nb\tc\"d\\e"
},
"content": [
{
"content": [
{
"text": "Notes.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "expand"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1,11 @@
:::::table {isNumberColumnEnabled=true layout=wide width="760.5"}
::::tableRow
:::tableHeader {colspan=2 colwidth="[340,420]"}
Assembly
:::
::::
:::::
:::expand {localId=01a0-33_z title="a\nb\tc\"d\\e"}
Notes.
:::
@@ -0,0 +1,41 @@
{
"content": [
{
"content": [
{
"text": "a `b ",
"type": "text"
},
{
"attrs": {
"url": "c`d"
},
"type": "inlineCard"
}
],
"type": "paragraph"
},
{
"content": [
{
"attrs": {
"url": "https://example.com/s?q=a&amp;b<c"
},
"type": "inlineCard"
}
],
"type": "paragraph"
},
{
"attrs": {
"extensionKey": "jira",
"parameters": {
"jql": "a|b `c` & d < e"
}
},
"type": "extension"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1,5 @@
a `b :inlineCard{url="c\u0060d"}
:inlineCard{url="https://example.com/s?q=a\u0026amp;b\u003cc"}
::extension {extensionKey=jira parameters="{\"jql\":\"a\u007cb \u0060c\u0060 \u0026 d \u003c e\"}"}
@@ -0,0 +1,107 @@
{
"content": [
{
"content": [
{
"content": [
{
"content": [
{
"content": [
{
"text": "Cell",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "tableHeader"
},
{
"content": [
{
"content": [
{
"text": "Note",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "tableHeader"
}
],
"type": "tableRow"
},
{
"content": [
{
"content": [
{
"content": [
{
"text": " lead",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "tableCell"
},
{
"content": [
{
"type": "paragraph"
}
],
"type": "tableCell"
}
],
"type": "tableRow"
},
{
"content": [
{
"content": [
{
"content": [
{
"attrs": {
"style": "a|b",
"text": "ok"
},
"type": "status"
}
],
"type": "paragraph"
}
],
"type": "tableCell"
},
{
"content": [
{
"content": [
{
"text": "trail ",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "tableCell"
}
],
"type": "tableRow"
}
],
"type": "table"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1,4 @@
| Cell | Note |
| --- | --- |
| :text{text=" "}lead | |
| :status[ok]{style="a\u007cb"} | trail:text{text=" "} |
+13 -10
View File
@@ -93,11 +93,14 @@ container is open, a named error otherwise.
**Attributes**: `{key=value key2="two words"}`. `{attrs}` is optional in every form, and `{}` is **Attributes**: `{key=value key2="two words"}`. `{attrs}` is optional in every form, and `{}` is
valid — no attributes. A bare value matches `[A-Za-z0-9_-]+`; any other value is double-quoted valid — no attributes. A bare value matches `[A-Za-z0-9_-]+`; any other value is double-quoted
with JSON string escaping (`\"` `\\` `\n` `\t` `\uXXXX`, …) — total over with JSON string escaping (`\"` `\\` `\n` `\t` `\uXXXX`, …) — total over
Unicode, and raw newlines never appear inside quotes. The closing `}` is the first one outside Unicode, and raw newlines never appear inside quotes. A quoted value also escapes `` ` ``, `&`,
quotes, since a quoted value holds `}` unescaped. All values are strings at the grammar `<` and `|` as `\u0060`, `\u0026`, `\u003c` and `\u007c` — the code span, entity reference,
level; each node's section assigns types. Canonical form orders keys alphabetically, spells autolink, raw HTML and pipe cell's split all bind at or before a directive does, so a raw one
values bare wherever allowed, inside quotes escapes only what it must using the shortest escape would reach back out of the value; `*`, `_`, `~`, `[` and `(` resolve after and stay raw. The
form, and omits empty `{attrs}` except where the `{` itself claims the directive closing `}` is the first one outside quotes, since a quoted value holds `}` unescaped. All values
are strings at the grammar level; each node's section assigns types. Canonical form orders keys
alphabetically, spells values bare wherever allowed, escapes inside quotes in the shortest form
each escape has, and omits empty `{attrs}` except where the `{` itself claims the directive
(`:hardBreak{}`). (`:hardBreak{}`).
**Escaping**: the emitter backslash-escapes whatever literal text would otherwise parse as **Escaping**: the emitter backslash-escapes whatever literal text would otherwise parse as
@@ -219,9 +222,9 @@ One header row plus plain inline cells is a pipe table; anything richer is the d
and every cell carry no attrs and no marks, the first row is all `tableHeader` and the rest all and every cell carry no attrs and no marks, the first row is all `tableHeader` and the rest all
`tableCell`, every row has the header's cell count, and every cell holds exactly one attr-less, `tableCell`, every row has the header's cell count, and every cell holds exactly one attr-less,
mark-less paragraph — an empty cell holds one empty paragraph — with no `|` anywhere the mark-less paragraph — an empty cell holds one empty paragraph — with no `|` anywhere the
inline layer spells as syntax — a code span, a link destination or title, an inline directive's inline layer spells as syntax — a code span, a link destination or title: backslash escapes are
attributes: backslash escapes are inert there, so pipe form cannot spell that pipe and the table inert there, so pipe form cannot spell that pipe and the table takes the directive form. A pipe
takes the directive form. A pipe table parses back to exactly that shape. table parses back to exactly that shape.
``` ```
| Part | Qty | | Part | Qty |
@@ -238,8 +241,8 @@ inline layer's ordinary CommonMark escaping yields the pipe; each cell is the in
one paragraph, trimmed; canonical form pads cells with single spaces and ends rows with `|` one paragraph, trimmed; canonical form pads cells with single spaces and ends rows with `|`
(optional in input). Named errors: a delimiter or body row whose cell count differs from the (optional in input). Named errors: a delimiter or body row whose cell count differs from the
header's, and an alignment colon in the delimiter row — ADF holds no column alignment. In a header's, and an alignment colon in the delimiter row — ADF holds no column alignment. In a
pipe cell a hard break is `:hardBreak{}`, a literal `|` is `\|`, and `|` inside a quoted pipe cell a hard break is `:hardBreak{}` and a literal `|` is `\|`; a `|` inside a quoted
attribute value is `\u007c` (JSON has no `\|`). attribute value is already `\u007c`, so the split never reaches it.
The directive form nests cells as containers of block content inside `tableRow` containers: The directive form nests cells as containers of block content inside `tableRow` containers:
+5 -6
View File
@@ -402,8 +402,8 @@ test('spells a table as a pipe table only where every row and cell is plain', ()
assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', text('a'), text('b'))))))) assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', text('a'), text('b')))))))
assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { attrs: { localId: 'a' }, type: 'paragraph' }))))), 'unspelled-node-attribute') assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { attrs: { localId: 'a' }, type: 'paragraph' }))))), 'unspelled-node-attribute')
assert.equal( assert.equal(
markdown(adfToMarkdown(table(row(cell('tableHeader', { content: [{ type: 'blockCard' }], type: 'paragraph' }))))), markdown(adfToMarkdown(table(row(cell('tableHeader', { content: [{ attrs: { url: 'a|b' }, type: 'blockCard' }], type: 'paragraph' }))))),
'| :adf{json="{\\"type\\":\\"blockCard\\"}"} |\n| --- |\n', '| :adf{json="{\\"attrs\\":{\\"url\\":\\"a\\u007cb\\"},\\"type\\":\\"blockCard\\"}"} |\n| --- |\n',
) )
assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: '\fa', type: 'text' }], type: 'paragraph' }))))), 'unspellable-whitespace') assert.equal(code(adfToMarkdown(table(row(cell('tableHeader', { content: [{ text: '\fa', type: 'text' }], type: 'paragraph' }))))), 'unspellable-whitespace')
assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', { attrs: { level: 1 }, type: 'heading' })))))) assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', { attrs: { level: 1 }, type: 'heading' }))))))
@@ -411,10 +411,8 @@ test('spells a table as a pipe table only where every row and cell is plain', ()
const marked = (mark: AdfMark): AdfDocument => table(row(cell('tableHeader', { content: [{ marks: [mark], text: 'l', type: 'text' }], type: 'paragraph' }))) const marked = (mark: AdfMark): AdfDocument => table(row(cell('tableHeader', { content: [{ marks: [mark], text: 'l', type: 'text' }], type: 'paragraph' })))
assert.ok(directive(adfToMarkdown(marked({ attrs: { href: 'https://example.com/?x|y' }, type: 'link' })))) assert.ok(directive(adfToMarkdown(marked({ attrs: { href: 'https://example.com/?x|y' }, type: 'link' }))))
assert.ok(directive(adfToMarkdown(marked({ attrs: { href: 'https://example.com/', title: 'a|b' }, type: 'link' })))) assert.ok(directive(adfToMarkdown(marked({ attrs: { href: 'https://example.com/', title: 'a|b' }, type: 'link' }))))
const fallsBack = (node: AdfNode): boolean => directive(adfToMarkdown(table(row(cell('tableHeader', { content: [node], type: 'paragraph' }))))) const codeSpan: AdfNode = { marks: [{ type: 'code' }], text: 'a|b', type: 'text' }
assert.ok(fallsBack({ marks: [{ type: 'code' }], text: 'a|b', type: 'text' })) assert.ok(directive(adfToMarkdown(table(row(cell('tableHeader', { content: [codeSpan], type: 'paragraph' }))))))
assert.ok(fallsBack({ attrs: { style: 'a|b' }, type: 'status' }))
assert.ok(fallsBack({ attrs: { url: 'https://example.com/?a|b' }, type: 'blockCard' }))
assert.equal(markdown(adfToMarkdown(marked({ attrs: { href: 'https://example.com/x' }, type: 'link' }))), '| [l](https://example.com/x) |\n| --- |\n') assert.equal(markdown(adfToMarkdown(marked({ attrs: { href: 'https://example.com/x' }, type: 'link' }))), '| [l](https://example.com/x) |\n| --- |\n')
}) })
@@ -427,6 +425,7 @@ test('spells an inline node as a directive with its content slot and attributes'
assert.equal(emitted({ attrs: { id: '1f389', text: 'a]b' }, type: 'emoji' }), ':emoji[a\\]b]{id=1f389}\n') assert.equal(emitted({ attrs: { id: '1f389', text: 'a]b' }, type: 'emoji' }), ':emoji[a\\]b]{id=1f389}\n')
assert.equal(emitted({ attrs: { data: { url: 'https://example.com/' } }, type: 'inlineCard' }), ':inlineCard{data="{\\"url\\":\\"https://example.com/\\"}"}\n') assert.equal(emitted({ attrs: { data: { url: 'https://example.com/' } }, type: 'inlineCard' }), ':inlineCard{data="{\\"url\\":\\"https://example.com/\\"}"}\n')
assert.equal(emitted({ attrs: { height: 24 }, type: 'mediaInline' }), ':mediaInline{height=24}\n') assert.equal(emitted({ attrs: { height: 24 }, type: 'mediaInline' }), ':mediaInline{height=24}\n')
assert.equal(emitted({ attrs: { url: 'a`b&c<d|e' }, type: 'inlineCard' }), ':inlineCard{url="a\\u0060b\\u0026c\\u003cd\\u007ce"}\n')
}) })
test('refuses an inline node attribute no section spells', () => { test('refuses an inline node attribute no section spells', () => {
+4 -1
View File
@@ -9,6 +9,9 @@ export type SpelledPairs = { fault: AttributeFault; pairs?: undefined } | { faul
const bareToken = /^[A-Za-z0-9_-]+$/ const bareToken = /^[A-Za-z0-9_-]+$/
// spec/flavour.md, Attributes.
const quotedEscapes = /[&<`|]/g
export function isBareToken(text: string): boolean { export function isBareToken(text: string): boolean {
return bareToken.test(text) return bareToken.test(text)
} }
@@ -53,5 +56,5 @@ export function spellStringAttribute(text: string): string {
} }
function quote(text: string): string { function quote(text: string): string {
return JSON.stringify(text) return JSON.stringify(text).replace(quotedEscapes, (character) => `\\u${character.charCodeAt(0).toString(16).padStart(4, '0')}`)
} }
+2 -2
View File
@@ -2,7 +2,7 @@ import { escapesLineClaim, isUnicodeWhitespace, opensBracketedAutolink, startsEn
export type EmphasisRole = 'close' | 'open' export type EmphasisRole = 'close' | 'open'
export type InlineEscaping = 'attribute' | 'backslash' | 'bracketed' | 'none' export type InlineEscaping = 'backslash' | 'bracketed' | 'none'
export type InlineSegment = export type InlineSegment =
| { emphasis: EmphasisRole; escaping: 'none'; mark: string; text: string } | { emphasis: EmphasisRole; escaping: 'none'; mark: string; text: string }
@@ -128,7 +128,7 @@ function touchesSyntax(scan: string, escapings: readonly (InlineEscaping | undef
} }
function isSyntax(escaping: InlineEscaping | undefined): boolean { function isSyntax(escaping: InlineEscaping | undefined): boolean {
return escaping === 'attribute' || escaping === 'none' return escaping === 'none'
} }
function opensConstruct(scan: string, index: number, inBrackets: boolean, container: LineContainer, escaped: ReadonlySet<number>): boolean { function opensConstruct(scan: string, index: number, inBrackets: boolean, container: LineContainer, escaped: ReadonlySet<number>): boolean {
+1 -1
View File
@@ -92,7 +92,7 @@ function carryEdges(segment: InlineSegment, leading: boolean, trailing: boolean)
} }
function carriedText(text: string): InlineSegment { function carriedText(text: string): InlineSegment {
return { escaping: 'attribute', text: spellLeafDirective('text', spellAttributes([['text', spellStringAttribute(text)]])) } return syntax(spellLeafDirective('text', spellAttributes([['text', spellStringAttribute(text)]])))
} }
function spellLeafDirective(name: string, attributes: string): string { function spellLeafDirective(name: string, attributes: string): string {
+19 -23
View File
@@ -96,24 +96,20 @@ detail is settled at its own milestone.
`inline-directives.ts`, which holds four of the nine marks while the rest are branch `inline-directives.ts`, which holds four of the nine marks while the rest are branch
literals in the emitter — and the parser (3) needs every name to make `:em[x]` the named literals in the emitter — and the parser (3) needs every name to make `:em[x]` the named
error `spec/flavour.md` promises. error `spec/flavour.md` promises.
- [ ] **2e3 — Attribute canonicalization and the quoted value's escape.** Attribute - [x] **2e3 — Attribute canonicalization and the quoted value's escape.**
canonicalization, a pipe cell's whitespace edges, and `\u007c` for a `|` inside a quoted **Settled** (the maintainer, 2026-08-26): a quoted attribute value escapes `` ` ``, `&`,
attribute value. The rule is wider than the pipe: a quoted value spells `` ` ``, `&` and `<` and `|` as `\u0060`, `\u0026`, `\u003c` and `\u007c`, in every directive, block and
`<` raw today, so a backtick pair in an `inlineCard`'s `data`, a `status`'s `style` or the inline alike — the constructs those four open all bind at or before a directive does, and
inline carry's `json` reads back as a code span, an entity or raw HTML inside the value. nothing else reaches into `{attrs}`. Emitted attributes being inert leaves 3 free to keep
One escape settles all four, and the pick decides whether a directive or a code span wins CommonMark's own precedence between a directive and a code span, and collapsed
where they overlap (3). `escaping: 'attribute'` earns its keep at that rule or collapses `escaping: 'attribute'` into `none`.
into `none`: nothing the escaper does tells the two apart today, since a carried segment
holds only spaces, tabs and newlines.
- [ ] **2e4 — The carry's fallback triggers.** `spec/flavour.md` carries a node its section - [ ] **2e4 — The carry's fallback triggers.** `spec/flavour.md` carries a node its section
cannot spell — an attrs key no section lists, a value that is not the section's type, an cannot spell — an attrs key no section lists, a value that is not the section's type, an
arg slot holding no bare token, marks no nesting spells — where the emitter still refuses, arg slot holding no bare token, marks no nesting spells — where the emitter still refuses,
which leaves the refusals a container's own spelling owns. The carry spells which leaves the refusals a container's own spelling owns. The flanking trigger 2e2
`escaping: 'none'`, which is what makes `tryPipeCell` refuse the pipe form for a carry added to that list is the odd one out: `unspellableMark` finds it after assembly and
whose JSON holds a pipe. The flanking trigger 2e2 added to that list is the odd one out: names a mark type against the line's path, so the failing run needs identifying before
`unspellableMark` finds it after assembly and names a mark type against the line's path, the carry can replace the refusal `corpus/unspellable/mark-inside-word` pins.
so the failing run needs identifying before the carry can replace the refusal
`corpus/unspellable/mark-inside-word` pins.
- [ ] **2e5 — Combined documents and the collision property.** Documents combining nodes rather - [ ] **2e5 — Combined documents and the collision property.** Documents combining nodes rather
than isolating one, and the gate's collision property: no two corpus documents may emit than isolating one, and the gate's collision property: no two corpus documents may emit
the same bytes — one spelling for two documents is a round-trip break no parser can undo, the same bytes — one spelling for two documents is a round-trip break no parser can undo,
@@ -142,14 +138,14 @@ detail is settled at its own milestone.
every raw-HTML construct in input is an error result. The CommonMark spec suite runs every raw-HTML construct in input is an error result. The CommonMark spec suite runs
against it from here (§10). The parser owes `~` the same `can_open`/`can_close` the emitter against it from here (§10). The parser owes `~` the same `can_open`/`can_close` the emitter
assumes — CommonMark flanking, as for `*` — which `spec/flavour.md` does not yet pin, and assumes — CommonMark flanking, as for `*` — which `spec/flavour.md` does not yet pin, and
the precedence between a directive and the constructs a raw attribute value opens inside it — the precedence in input between a directive and a code span, an entity or raw HTML written
a code span, an entity, raw HTML — which one directive alone already reaches until 2e's raw inside its attributes — 2e3's escape keeps emitted output clear of that collision, so the
escape lands. Whether a claimed line interrupts a paragraph is pinned for the plain case and pick binds input alone. Whether a claimed line interrupts a paragraph is pinned for the plain
unstated for the lazy one: a closing fence on the line after a blockquote's open paragraph is case and unstated for the lazy one: a closing fence on the line after a blockquote's open
lazy continuation in CommonMark, which would swallow the fence and leave the container paragraph is lazy continuation in CommonMark, which would swallow the fence and leave the
unclosed. 2e1's `closing-fence-line` orders its blockquote away from that edge meanwhile. container unclosed. 2e1's `closing-fence-line` orders its blockquote away from that edge
`src/` gets its hierarchy at the same split — `adf/`, meanwhile. `src/` gets its hierarchy at the same split — `adf/`, `markdown/`, `html/`, the
`markdown/`, `html/`, the grammar module shared inside `markdown/` — while the rename is grammar module shared inside `markdown/` — while the rename is
still mechanical. Three files do not move whole: `block-directives.ts` and still mechanical. Three files do not move whole: `block-directives.ts` and
`inline-directives.ts` each hold a node table milestones 6-7 need in `adf/` beside a `inline-directives.ts` each hold a node table milestones 6-7 need in `adf/` beside a
markdown spelling that belongs in `markdown/`, and `directive-attributes.ts` fuses the markdown spelling that belongs in `markdown/`, and `directive-attributes.ts` fuses the