3
0
mirror of https://github.com/larvit/adf-codec.git synced 2026-11-19 06:35:44 +01:00

Respell the wiki writer's backslashes and inline carries as Jira reads them

This commit is contained in:
2026-11-17 02:58:01 +01:00
committed by Lillem4n
parent 6776a3c1d3
commit f36c0378fe
8 changed files with 120 additions and 57 deletions
+11 -1
View File
@@ -1453,7 +1453,7 @@ alt text.
Wiki markup has no spelling for some nodes and marks, a layout among them. The lossless wiki
writer wraps each such node's ADF JSON in Jira's own `{adf}` macro: `{adf:display=block}` for a
block node, `{adf:display=inline}` for an inline node or a marked run. The round-trip holds, and
block node, `{adf:display=inline}` for an inline node or each node of a marked run. The round-trip holds, and
Jira still renders the document. The lossy wiki writer ships beside it for consumers who want wiki
markup alone: it drops the form of what wiki markup cannot spell and keeps its content.
@@ -1507,6 +1507,16 @@ deep-equal and Jira's reading doesn't: `[~name]` reads as a `mention` with `id`
has no `language`, an unknown `{code:<language>}` reads `java`, a `bgColor` outside the five reads as
the nearest-looking panel type, not `info`, and a backslash before `:` stays text.
Checked again 2026-11-17 for the lossless writer's backslashes and inline carries. Jira drops the
backslash before `(`, `)`, `?`, `!`, `#`, `[`, `]`, `{`, `}`, `*`, `+`, `-`, `^`, `_` and `~`, and
keeps it as text before `:`, `;`, `.`, `,`, a letter, a digit and another backslash. So a backslash
before a smiley's `(`, or its closing `)`, stops it (`:\)` reads as the text `:)`), while `:D`, `:P`,
a `:` or `;` ending a node inside a line, and the `h` of `h1.` and the `b` of `bq.` at a line's start
ride the carry. An escaped `|` still ends a table cell, so a text holding `|` rides the carry there;
`|`, `\\`, `\"` and `\n` inside a carry's JSON survive in a cell and in a paragraph. A
`{adf:display=inline}` holding an array shows "Invalid ADF Macro", where one holding a node renders
it, so a run of nodes is one macro per node.
## A plain wiki spelling stands for the attributes Jira's editor writes, and any other attribute rides the carry
2026-11-16, the implementer of the wiki spec; the maintainer confirmed on 2026-11-17 that `localId` and a
+17
View File
@@ -1258,3 +1258,20 @@ Read most (opens and searches):
- `AGENTS.md` 3
- `todo.md` 2
- `src/index.ts` 1
## Reading cost, chunk 1fb1b988e9fa, finished 2026-11-17T01:46:15Z
266246 new input tokens before the first commit.
Read most (opens and searches):
- `spec/wiki.md` 8
- `src/adf/document.ts` 7
- `src/wiki/emit/inline-line.ts` 5
- `ci.sh` 4
- `docs/decisions.md` 4
- `src/adf/carried-json.ts` 4
- `src/html/emit/adf-to-html.test.ts` 4
- `README.md` 3
- `package.json` 3
- `src/wiki/emit/text-escaping.ts` 3
+18 -14
View File
@@ -17,8 +17,8 @@ markup carries what it cannot spell in Jira's `{adf}` macro, and a lossy writer
content:
- A block node: `{adf:display=block}`, a line feed, the JSON, a line feed, `{adf}`.
- An inline node, or the run of inline nodes under a mark with no spelling, as an array:
`{adf:display=inline}`, the JSON, `{adf}`.
- An inline node: `{adf:display=inline}`, the node's JSON, `{adf}`. The run of inline nodes under a
mark with no spelling is one such macro per node, since Jira reads a node and never an array there.
The JSON is compact with its keys sorted, as `spec/flavour.md` writes the markdown carry's, and a
`{` inside a string is `\u007b`, so the closing `{adf}` is the only one the text holds. A document
@@ -67,24 +67,28 @@ a `width`, a merged cell and a colour.
| `hardBreak` | `\\` | the same |
| `mention` holding `id` and `accessLevel` `""`, and no other attribute, the `id` not empty and holding no whitespace or markup character | `[~accountid:<id>]` | the same, reading `accessLevel` `""`; `[~name]` reads as text, since Jira Cloud names a user by account id |
| `text` | its text, escaped as below | `\\` reads as a `hardBreak` before `\X` reads `X`; a bare `http://` or `https://` address reads as a `link` |
| any other inline node | the carry | `{adf:display=inline}` reads its nodes |
| any other inline node | the carry | `{adf:display=inline}` reads its node |
A character takes a backslash where wiki markup would read it as markup. Here a line's edge counts
as whitespace, and a node's edge inside the line as punctuation:
- `[`, `]`, `{`, `}` and `|`, always.
- `[`, `]`, `{`, `}` and `|`, always, except that a `|` in a table cell rides the carry, since an
escaped one still ends the cell.
- `*`, `+`, `-`, `^`, `_` and `~`, unless a letter or digit stands on both sides, or whitespace
does.
- `?` before another `?`, and `!` before anything but whitespace.
- The first character of a smiley listed under What only the reader reads, where no letter or digit
follows the smiley, or where the node's edge cuts it inside the line.
- At a line's start, `#`, `*` and `-`, and the `h` of `h1.` to `h6.` and the `b` of `bq.`, in any
case.
- The `(` of a smiley listed under What only the reader reads, or the closing `)` of one that starts
with `:` or `;`, where no letter or digit follows the smiley, or where the node's edge cuts it
inside the line.
- At a line's start, `#`, `*` and `-`.
No backslash protects a backslash, a line feed, a carriage return, or `http://` or `https://` in any
case, which Jira would link, so a text holding one rides the carry. So does a text that starts or
ends its line with whitespace, and a text whose marks are those of the text before it, which the
reader would join. A `hardBreak` or `mention` holding a mark rides too.
Jira keeps a backslash before `:`, `;`, a letter or a digit as text. So no backslash protects a
backslash, a line feed or a carriage return; `http://` or `https://` in any case, which Jira would
link; the `h` of `h1.` to `h6.` or the `b` of `bq.` at a line's start, in any case; `:D` and `:P`;
or a `:` or `;` ending a node inside the line, where the next node's spelling may close a smiley. A
text holding one rides the carry. So does a text that starts or ends its line with whitespace, and
a text whose marks are those of the text before it, which the reader would join. A `hardBreak` or
`mention` holding a mark rides too.
## Marks
@@ -94,7 +98,7 @@ reader would join. A `hardBreak` or `mention` holding a mark rides too.
| `link` holding `href` | `[<text>\|<href>]`, with `\|<title>` after the `href` where it holds a `title`; `[<href>]` where the link is the text's last mark, the text is the `href`, the link marks neither neighbour and holds no `title`, and the `href` doesn't start with `~`; an `href` that is empty or holds whitespace or a markup character rides the carry, as does a `title` that is empty, holds a markup character, a line feed or a carriage return, or starts or ends with whitespace | the same; `[#anchor]` and `[^name.ext]` read as `href` `#anchor` and `^name.ext` |
| `subsup` | `^x^` for `sup`, `~x~` for `sub` | the same |
| `textColor` | `{color:<hex>}x{color}` | the same; a CSS colour name reads as its hex code, so `red` as `#ff0000` |
| any other mark | the carry around the marked run | `{adf:display=inline}` reads its run |
| any other mark | the carry of each node of the marked run | `{adf:display=inline}` reads its node |
Marks nest in the order the node's array holds them, outermost first, and a run of nodes sharing
their outermost mark shares its delimiters. The delimiters of `code`, `em`, `strike`, `strong`,
@@ -128,7 +132,7 @@ The writers never write these, and the reader reads them as follows:
- `---` and `--` read as `—` and `–`.
- A smiley reads as the `emoji` Jira reads it as, wherever the next character is not a letter or a
digit and whatever comes before it: `x:)` and `f(x)` read, `a:Pb` and `:)x` don't, and a backslash
before it stops it. The `emoji`'s `text` is the character where its `id` is a code point, else its
before its `(` or its closing `)` stops it: `\(y)` and `:\)` read as text. The `emoji`'s `text` is the character where its `id` is a code point, else its
`shortName`.
| Smiley | `shortName` | `id` |
+9 -4
View File
@@ -9,10 +9,15 @@ export function blockCarry(node: AdfNode, path: ConvertErrorPath): Result<string
return json.ok ? success(`{adf:display=block}\n${closingMacroEscaped(json.value.json)}\n{adf}`) : json
}
export function inlineCarry(nodes: readonly AdfNode[], path: ConvertErrorPath): Result<string> {
const carried = nodes.length === 1 && nodes[0] !== undefined ? nodes[0] : nodes
const json = carriedJson(carried, carried, 'compact', path, largestNesting)
return json.ok ? success(`{adf:display=inline}${closingMacroEscaped(json.value.json)}{adf}`) : json
// One macro per node, since Jira reads a node and never an array inside `{adf:display=inline}`; `first` is `nodes[0]`'s index in the content of the node at `path`.
export function inlineCarry(nodes: readonly AdfNode[], path: ConvertErrorPath, first: number): Result<string> {
let macros = ''
for (const [index, node] of nodes.entries()) {
const json = carriedJson(node, node, 'compact', [...path, 'content', first + index], largestNesting)
if (!json.ok) return json
macros += `{adf:display=inline}${closingMacroEscaped(json.value.json)}{adf}`
}
return success(macros)
}
// A `{` inside a JSON string is `\u007b`, so the closing `{adf}` is the only one the macro holds.
+22 -5
View File
@@ -200,12 +200,21 @@ test('escapes a character wiki markup would read as markup where it stands', ()
assert.equal(line('a [b] {c} | d'), 'a \\[b\\] \\{c\\} \\| d')
assert.equal(line('# not a list'), '\\# not a list')
assert.equal(line('- not a list'), '\\- not a list')
assert.equal(line('h1. not a heading'), '\\h1. not a heading')
assert.equal(line('bq. not a quote'), '\\bq. not a quote')
assert.equal(line('a -- b'), 'a \\-\\- b')
assert.equal(line('what?? ok'), 'what\\?? ok')
assert.equal(line('Wow!!'), 'Wow\\!!')
assert.equal(line('Wow! (y) f(x) x:)'), 'Wow! \\(y) f\\(x) x\\:)')
assert.equal(line('Wow! (y) f(x) x:)'), 'Wow! \\(y) f\\(x) x:\\)')
assert.equal(line('a :( b ;) c'), 'a :\\( b ;\\) c')
})
test('carries a text no backslash stops a smiley or a line-start word in', () => {
const carried = (value: string): string => inline(`{"text":${JSON.stringify(value)},"type":"text"}`)
assert.equal(line('h1. not a heading'), carried('h1. not a heading'))
assert.equal(line('Bq. not a quote'), carried('Bq. not a quote'))
assert.equal(line('a :D b'), carried('a :D b'))
assert.equal(line('a :P'), carried('a :P'))
assert.equal(wiki(paragraph(text('a :'), { attrs: { accessLevel: '', id: '5b10' }, type: 'mention' })), `${inline('{"text":"a :","type":"text"}')}[~accountid:5b10]`)
assert.equal(line('a ;'), 'a ;')
})
test('leaves a character that reads as itself where it stands', () => {
@@ -252,9 +261,11 @@ test('carries a run whose delimiters would not read', () => {
assert.equal(wiki(paragraph(text('a '), text(' b', { type: 'em' }))), `a ${inline('{"marks":[{"type":"em"}],"text":" b","type":"text"}')}`)
})
test('carries the run of nodes under a mark with no spelling as an array', () => {
test('carries each node under a mark with no spelling in a macro of its own', () => {
const background = { attrs: { color: '#ffff00' }, type: 'backgroundColor' }
assert.equal(wiki(paragraph(text('a', background), text('b', background, { type: 'strong' }))), inline('[{"marks":[{"attrs":{"color":"#ffff00"},"type":"backgroundColor"}],"text":"a","type":"text"},{"marks":[{"attrs":{"color":"#ffff00"},"type":"backgroundColor"},{"type":"strong"}],"text":"b","type":"text"}]'))
const first = inline('{"marks":[{"attrs":{"color":"#ffff00"},"type":"backgroundColor"}],"text":"a","type":"text"}')
const second = inline('{"marks":[{"attrs":{"color":"#ffff00"},"type":"backgroundColor"},{"type":"strong"}],"text":"b","type":"text"}')
assert.equal(wiki(paragraph(text('a', background), text('b', background, { type: 'strong' }))), first + second)
})
test('carries a mark no spelling stands for, and two marks of one type', () => {
@@ -319,6 +330,12 @@ test('escapes a smiley the node edge cuts, unless the line ends there', () => {
assert.equal(line('a ('), 'a (')
})
test('carries a text holding a pipe in a table cell, where a backslash would end the cell', () => {
assert.equal(wiki(table(plainTable, [cell('tableCell', text('a|b')), cell('tableCell', text('c'))])), `|${inline('{"text":"a|b","type":"text"}')}|c|`)
assert.equal(wiki(table(plainTable, [cell('tableCell', text('x', { attrs: { href: 'https://example.com' }, type: 'link' }), text('|'))])), `|[x|https://example.com]${inline('{"text":"|","type":"text"}')}|`)
assert.equal(line('a|b'), 'a\\|b')
})
test('refuses a heading holding a carry past 500 levels', () => {
let value: AdfAttributes[string] = 1
for (let level = 0; level < largestNesting - 1; level += 1) value = [value]
+5 -5
View File
@@ -22,8 +22,8 @@ const spellableUrl = /^[^\s[\]{}|\\]+$/
const spellableColour = /^#[0-9a-f]{6}$/i
// spec/wiki.md, Inline and Marks.
export function inlineLine(nodes: readonly AdfNode[], path: ConvertErrorPath): Result<Line> {
const atoms = nodes.map((node, index) => atom(node, nodes, index))
export function inlineLine(nodes: readonly AdfNode[], path: ConvertErrorPath, inCell: boolean): Result<Line> {
const atoms = nodes.map((node, index) => atom(node, nodes, index, inCell))
const groups = markedGroups(atoms)
let line = ''
let carried = false
@@ -34,7 +34,7 @@ export function inlineLine(nodes: readonly AdfNode[], path: ConvertErrorPath): R
const after = following?.marks.length === 0 ? following.spelled?.text : undefined
const written = groupSpelling(atoms.slice(first, last + 1), lastCharacter(line), after === undefined ? undefined : firstCharacter(after))
if (written === undefined) {
const carry = inlineCarry(nodes.slice(first, last + 1), [...path, 'content', first])
const carry = inlineCarry(nodes.slice(first, last + 1), path, first)
if (!carry.ok) return carry
line += carry.value
carried = true
@@ -43,7 +43,7 @@ export function inlineLine(nodes: readonly AdfNode[], path: ConvertErrorPath): R
return success({ carried, line })
}
function atom(node: AdfNode, nodes: readonly AdfNode[], index: number): Atom {
function atom(node: AdfNode, nodes: readonly AdfNode[], index: number, inCell: boolean): Atom {
const { text } = node
if (node.type !== 'text' || text === undefined) {
const leaf = leafSpelling(node)
@@ -56,7 +56,7 @@ function atom(node: AdfNode, nodes: readonly AdfNode[], index: number): Atom {
if (!isSpellableText(node) || joinsPrevious || spelled === undefined) return { marks, spelled: undefined }
const href = hrefSpelling(node, nodes, index)
if (href !== undefined) return { marks: marks.slice(0, -1), spelled: { marks: spelled.slice(0, -1), text: href } }
const escaped = escapedText(text, index === 0, index === nodes.length - 1)
const escaped = escapedText(text, index === 0, index === nodes.length - 1, inCell)
return { marks, spelled: escaped === undefined ? undefined : { marks: spelled, text: escaped } }
}
+11 -11
View File
@@ -9,7 +9,7 @@ import { success, type ConvertErrorPath, type Result } from '../../result.ts'
// `blank`: a paragraph holding no content or only whitespace, which only a table cell holds; `macros`: the `{quote}` and `{panel}` the block writes, which can't nest in their own kind; `text`: the block's spelling, where no container composes it.
export type Spelling = { blank: boolean; macros: number; text: string | undefined }
type Step = { exit: boolean; node: AdfNode; path: ConvertErrorPath }
type Step = { exit: boolean; inCell: boolean; node: AdfNode; path: ConvertErrorPath }
const quoteMacro = 1
const panelMacro = 2
@@ -21,18 +21,18 @@ const spellableImageUrl = /^[^\s!|\\[\]{}]+$/
// spec/wiki.md, Blocks: each block wiki markup spells, found after its children, since a block rides where one it holds does. A stack rather than recursion (docs/decisions.md §Nothing recurses unbounded).
export function spelledBlocks(content: readonly AdfNode[]): Result<Map<AdfNode, Spelling>> {
const spelled = new Map<AdfNode, Spelling>()
const pending: Step[] = content.map((node, index) => ({ exit: false, node, path: ['content', index] })).reverse()
const pending: Step[] = content.map((node, index) => ({ exit: false, inCell: false, node, path: ['content', index] })).reverse()
for (let step = pending.pop(); step !== undefined; step = pending.pop()) {
const { exit, node, path } = step
const { exit, inCell, node, path } = step
if (exit) {
const spelling = containerSpelling(node, spelled)
if (spelling !== undefined) spelled.set(node, spelling)
} else if (containerHolds(node)) {
pending.push({ exit: true, node, path })
const children = nodeContent(node).map((child, index): Step => ({ exit: false, node: child, path: [...path, 'content', index] }))
pending.push({ exit: true, inCell, node, path })
const children = nodeContent(node).map((child, index): Step => ({ exit: false, inCell: node.type === 'tableCell' || node.type === 'tableHeader', node: child, path: [...path, 'content', index] }))
for (const child of children.reverse()) pending.push(child)
} else {
const spelling = leafSpelling(node, path)
const spelling = leafSpelling(node, path, inCell)
if (!spelling.ok) return spelling
if (spelling.value !== undefined) spelled.set(node, spelling.value)
}
@@ -84,16 +84,16 @@ function containerSpelling(node: AdfNode, spelled: ReadonlyMap<AdfNode, Spelling
return { blank: false, macros, text: undefined }
}
function leafSpelling(node: AdfNode, path: ConvertErrorPath): Result<Spelling | undefined> {
if (node.type === 'paragraph') return paragraphSpelling(node, path)
function leafSpelling(node: AdfNode, path: ConvertErrorPath, inCell: boolean): Result<Spelling | undefined> {
if (node.type === 'paragraph') return paragraphSpelling(node, path, inCell)
if (node.type === 'heading') return headingSpelling(node, path)
return success(spelledText(simpleLeafText(node)))
}
function paragraphSpelling(node: AdfNode, path: ConvertErrorPath): Result<Spelling | undefined> {
function paragraphSpelling(node: AdfNode, path: ConvertErrorPath, inCell: boolean): Result<Spelling | undefined> {
if (!holdsExactly(node, {})) return success(undefined)
if (node.content === undefined) return success({ blank: true, macros: 0, text: ' ' })
const line = inlineLine(node.content, path)
const line = inlineLine(node.content, path, inCell)
if (!line.ok) return line
return success({ blank: node.content.every((child) => child.type === 'text') && /^\s*$/u.test(node.content.map(({ text }) => text).join('')), macros: 0, text: line.value.line })
}
@@ -101,7 +101,7 @@ function paragraphSpelling(node: AdfNode, path: ConvertErrorPath): Result<Spelli
function headingSpelling(node: AdfNode, path: ConvertErrorPath): Result<Spelling | undefined> {
const { level } = nodeAttrs(node)
if (!holdsOnlyAttributes(node, ['level']) || !isHeadingLevel(level) || node.content === undefined) return success(undefined)
const line = inlineLine(node.content, path)
const line = inlineLine(node.content, path, false)
if (!line.ok) return line
return success(line.value.carried ? undefined : spelledText(`h${level}. ${line.value.line}`))
}
+27 -17
View File
@@ -8,16 +8,19 @@ const delimiters = '*+-^_~'
const lineStartMarkers = '#*-'
const lineStartWords = /^(?:bq|h[1-6])\./i
const smileyOpeners = '(:;'
const smileyCharacters = smileys.map((smiley) => Array.from(smiley))
const smileyCharacters = smileys.map((smiley) => ({ characters: Array.from(smiley), parenthesised: /[()]$/.test(smiley) }))
const unprotectable = /[\\\r\n]|https?:\/\//i
// spec/wiki.md, Inline: `undefined` where no backslash protects the text, so it rides the carry.
export function escapedText(text: string, lineStart: boolean, lineEnd: boolean): string | undefined {
if (unprotectable.test(text)) return undefined
// spec/wiki.md, Inline: `undefined` where no backslash protects the text, so it rides the carry. `inCell`: the text stands in a table cell, where an escaped `|` still ends the cell.
export function escapedText(text: string, lineStart: boolean, lineEnd: boolean, inCell: boolean): string | undefined {
if (unprotectable.test(text) || (inCell && text.includes('|'))) return undefined
if ((lineStart && /^\s/u.test(text)) || (lineEnd && /\s$/u.test(text))) return undefined
const characters = Array.from(text)
if (lineStart && lineStartWords.test(text)) return undefined
const edges: { end: Side; start: Side } = { end: lineEnd ? 'space' : 'punctuation', start: lineStart ? 'space' : 'punctuation' }
return characters.map((character, index) => (escapes(character, characters, index, edges, lineStart) ? `\\${character}` : character)).join('')
const stops = smileyStops(characters, edges.end)
if (stops === undefined) return undefined
return characters.map((character, index) => (stops.has(index) || escapes(character, characters, index, edges, lineStart) ? `\\${character}` : character)).join('')
}
function escapes(character: string, characters: readonly string[], index: number, edges: { end: Side; start: Side }, lineStart: boolean): boolean {
@@ -26,23 +29,30 @@ function escapes(character: string, characters: readonly string[], index: number
const before = previous === undefined ? edges.start : side(previous)
const after = next === undefined ? edges.end : side(next)
if (alwaysEscaped.includes(character)) return true
if (lineStart && index === 0 && (lineStartMarkers.includes(character) || lineStartWords.test(characters.join('')))) return true
if (lineStart && index === 0 && lineStartMarkers.includes(character)) return true
// A delimiter both of whose neighbours are words, or both spaces, neither opens nor closes a run.
if (delimiters.includes(character) && !(before === after && before !== 'punctuation')) return true
if (character === '?' && next === '?') return true
if (character === '!' && after !== 'space') return true
return smileyOpeners.includes(character) && opensSmiley(characters, index, edges.end)
return character === '!' && after !== 'space'
}
// A smiley reads wherever no letter or digit follows it; one the node's edge cuts may close in the next node's spelling.
function opensSmiley(characters: readonly string[], index: number, end: Side): boolean {
return smileyCharacters.some((spelled) => {
const held = characters.slice(index, index + spelled.length)
if (held.some((character, offset) => character !== spelled[offset])) return false
if (held.length < spelled.length) return end !== 'space'
const next = characters[index + spelled.length]
return (next === undefined ? end : side(next)) !== 'word'
})
// A smiley reads wherever no letter or digit follows it; one the node's edge cuts may close in the next node's spelling. `undefined` where a smiley reads that no backslash stops: Jira keeps one before `:`, `;` and a letter, so only a `(` or a closing `)` takes it.
function smileyStops(characters: readonly string[], end: Side): Set<number> | undefined {
const stops = new Set<number>()
for (const [index, character] of characters.entries()) {
if (!smileyOpeners.includes(character)) continue
for (const { characters: spelled, parenthesised } of smileyCharacters) {
const held = characters.slice(index, index + spelled.length)
if (held.some((heldCharacter, offset) => heldCharacter !== spelled[offset])) continue
const cut = held.length < spelled.length
const next = characters[index + spelled.length]
if (cut ? end === 'space' : (next === undefined ? end : side(next)) === 'word') continue
if (character === '(') stops.add(index)
else if (!cut && parenthesised) stops.add(index + spelled.length - 1)
else return undefined
}
}
return stops
}
function side(character: string): Side {