3
0
mirror of https://github.com/larvit/adf-codec.git synced 2026-11-21 23:46:21 +01:00

Read adfToHtml's HTML in htmlToAdf as a browser serializes it too

This commit is contained in:
2026-10-31 01:48:10 +01:00
committed by Lillem4n
parent 088494511d
commit 6cc59f8574
13 changed files with 199 additions and 57 deletions
+4 -2
View File
@@ -15,8 +15,10 @@
content, links included. Every URL is written as the ADF holds it, content, links included. Every URL is written as the ADF holds it,
`javascript:` included: sanitize HTML from an author you do not trust before showing it. `javascript:` included: sanitize HTML from an author you do not trust before showing it.
- `htmlToAdf` reads `adfToHtml`'s HTML back to the document it came from, a carry element from its - `htmlToAdf` reads `adfToHtml`'s HTML back to the document it came from, a carry element from its
`data-adf`. It refuses any other HTML, including an edit to what a carry element shows, with the `data-adf`, also as a browser's `innerHTML` writes it (a raw `"` in text, `&nbsp;`, a raw `<` or
codes README §The calls lists. `>` in an attribute value) and as a form post sends it (`\r\n` for each line feed). It refuses
any other HTML, including an edit to what a carry element shows, with the codes README §The calls
lists.
- **Breaking:** `losslessMarkdownToAdf` gives a `date`, `emoji`, `inlineCard`, `mention`, `status` - **Breaking:** `losslessMarkdownToAdf` gives a `date`, `emoji`, `inlineCard`, `mention`, `status`
or `mediaInline` only the marks Atlassian's schema lets it hold: `**!adf:mention[@A]{id=…}**` now or `mediaInline` only the marks Atlassian's schema lets it hold: `**!adf:mention[@A]{id=…}**` now
reads as a plain mention. A link or directive mark around only nodes that cannot take it is reads as a plain mention. A link or directive mark around only nodes that cannot take it is
+4 -2
View File
@@ -540,8 +540,10 @@ content. No stylesheet ships. The complete dialect:
`htmlToAdf` reads exactly that HTML back. It reads a carry element's node from its `data-adf`, and `htmlToAdf` reads exactly that HTML back. It reads a carry element's node from its `data-adf`, and
what the element shows must be what `adfToHtml` shows for that node: an edit there would be lost, so what the element shows must be what `adfToHtml` shows for that node: an edit there would be lost, so
it is refused. Other HTML is refused as `unmappable-html`: another element, attribute or escape, it is refused. It also reads the HTML as a browser's `innerHTML` writes it back and as a form post
such as `<b>`, `<br/>` or `&#39;`, and HTML missing its final line feed or ending a line in `\r\n`. sends it: a raw `"` in text, `&nbsp;` for a no-break space, a raw `<` or `>` in an attribute value,
and `\r\n` for a line feed. Other HTML is refused as `unmappable-html`: another element, attribute or
escape, such as `<b>`, `<br/>` or `&#39;`, a lone `\r`, and HTML missing its final line feed.
A few causes, such as a `data-*` attribute its element does not take, are `unsupported-node-shape` A few causes, such as a `data-*` attribute its element does not take, are `unsupported-node-shape`
instead, and nesting past 500 levels is `unsupported-nesting-depth`. [Either instead, and nesting past 500 levels is `unsupported-nesting-depth`. [Either
direction](#either-direction) lists every cause of both. direction](#either-direction) lists every cause of both.
+10 -1
View File
@@ -2,14 +2,23 @@ try {
const { adfToHtml, adfToLosslessMarkdown, htmlToAdf, isAdfDocument, losslessMarkdownToAdf } = await import('/dist/index.js') const { adfToHtml, adfToLosslessMarkdown, htmlToAdf, isAdfDocument, losslessMarkdownToAdf } = await import('/dist/index.js')
// WebDriver's JSON reads -0 back as 0, so a document crosses as JSON text with -0 tagged; run.js revives it. // WebDriver's JSON reads -0 back as 0, so a document crosses as JSON text with -0 tagged; run.js revives it.
const spelled = (result) => (result.ok ? { ok: true, value: JSON.stringify(result.value, (_, value) => (Object.is(value, -0) ? '\u0000-0' : value)) } : result) const spelled = (result) => (result.ok ? { ok: true, value: JSON.stringify(result.value, (_, value) => (Object.is(value, -0) ? '\u0000-0' : value)) } : result)
// The page's own innerHTML is what a viewer/editor app posts back, so htmlToAdf reads what the browser writes.
const throughInnerHtml = (adf) => {
const html = adfToHtml(adf)
if (!html.ok) return { parsed: html }
const page = document.createElement('div')
page.innerHTML = html.value
return { parsed: spelled(htmlToAdf(page.innerHTML)) }
}
window.convertCorpus = (corpus) => ({ window.convertCorpus = (corpus) => ({
browserDocuments: corpus.browserDocuments.map(({ json, name }) => ({ name, ...throughInnerHtml(JSON.parse(json)) })),
errors: corpus.errors.map(({ markdown, name }) => ({ name, parsed: losslessMarkdownToAdf(markdown) })), errors: corpus.errors.map(({ markdown, name }) => ({ name, parsed: losslessMarkdownToAdf(markdown) })),
normalization: corpus.normalization.map(({ markdown, name }) => ({ name, parsed: spelled(losslessMarkdownToAdf(markdown)) })), normalization: corpus.normalization.map(({ markdown, name }) => ({ name, parsed: spelled(losslessMarkdownToAdf(markdown)) })),
realPayloads: corpus.realPayloads.map(({ json, name }) => { realPayloads: corpus.realPayloads.map(({ json, name }) => {
const adf = JSON.parse(json) const adf = JSON.parse(json)
const emitted = adfToLosslessMarkdown(adf) const emitted = adfToLosslessMarkdown(adf)
const html = adfToHtml(adf) const html = adfToHtml(adf)
return { emitted, html, htmlParsed: html.ok ? spelled(htmlToAdf(html.value)) : undefined, isDocument: isAdfDocument(adf), name, parsed: emitted.ok ? spelled(losslessMarkdownToAdf(emitted.value)) : undefined } return { emitted, html, htmlParsed: html.ok ? spelled(htmlToAdf(html.value)) : undefined, innerHtml: throughInnerHtml(adf), isDocument: isAdfDocument(adf), name, parsed: emitted.ok ? spelled(losslessMarkdownToAdf(emitted.value)) : undefined }
}), }),
roundTrip: corpus.roundTrip.map(({ json, markdown, name }) => { roundTrip: corpus.roundTrip.map(({ json, markdown, name }) => {
const adf = JSON.parse(json) const adf = JSON.parse(json)
+29 -1
View File
@@ -70,7 +70,25 @@ async function whenDriverAnswers() {
} }
} }
// Text the browser's innerHTML spells other than adfToHtml does (WHATWG HTML §13.3): a raw ", &nbsp;, and in an attribute < and > escaped or not by version.
const text = (value, ...marks) => (marks.length === 0 ? { text: value, type: 'text' } : { marks, text: value, type: 'text' })
const browserDocuments = [
{
json: JSON.stringify({
content: [
{ content: [text('Say "hi"\u00a0now & <then>'), text(' link', { attrs: { href: 'https://example.com/?a<b>c&d="e"', title: 'A "title"\u00a0<here>' }, type: 'link' })], type: 'paragraph' },
{ attrs: { language: 'js' }, content: [text('if (a < b) say("x\u00a0y")\n done()')], type: 'codeBlock' },
{ content: [{ attrs: { id: '0199f4c2-8b1e-7d03-9a6f-2c5e8d71b4a0', text: 'Say "hi"\u00a0now' }, type: 'mention' }], type: 'paragraph' },
{ attrs: { panelType: 'info' }, content: [{ content: [text('A "quote"\u00a0here', { type: 'underline' }, { attrs: { href: 'https://example.com/?a<b>', title: 'T"\u00a0' }, type: 'link' })], type: 'paragraph' }], type: 'panel' },
],
type: 'doc',
version: 1,
}),
name: 'quotes, no-break spaces and angle brackets',
},
]
const corpus = { const corpus = {
browserDocuments,
errors: fixtureNames('errors', '.md').map((name) => ({ markdown: fixture(name, '.md'), name })), errors: fixtureNames('errors', '.md').map((name) => ({ markdown: fixture(name, '.md'), name })),
normalization: fixtureNames('normalization', '.md').map((name) => ({ markdown: fixture(name, '.md'), name })), normalization: fixtureNames('normalization', '.md').map((name) => ({ markdown: fixture(name, '.md'), name })),
realPayloads: fixtureNames('real-payloads', '.json').map((name) => ({ json: fixture(name, '.json'), name })), realPayloads: fixtureNames('real-payloads', '.json').map((name) => ({ json: fixture(name, '.json'), name })),
@@ -135,6 +153,16 @@ for (const [index, { json, name }] of corpus.realPayloads.entries()) {
assert.deepEqual(result.html, adfToHtml(JSON.parse(json)), 'it wrote other HTML than Node wrote') assert.deepEqual(result.html, adfToHtml(JSON.parse(json)), 'it wrote other HTML than Node wrote')
assert.ok(result.htmlParsed.ok, `its HTML did not read back — ${refusal(result.htmlParsed)}`) assert.ok(result.htmlParsed.ok, `its HTML did not read back — ${refusal(result.htmlParsed)}`)
assert.deepEqual(revived(result.htmlParsed.value), JSON.parse(json)) assert.deepEqual(revived(result.htmlParsed.value), JSON.parse(json))
assert.ok(result.innerHtml.parsed.ok, `its innerHTML did not read back — ${refusal(result.innerHtml.parsed)}`)
assert.deepEqual(revived(result.innerHtml.parsed.value), JSON.parse(json))
})
}
for (const [index, { json, name }] of corpus.browserDocuments.entries()) {
const result = results.browserDocuments[index]
checking(name, result, () => {
assert.ok(result.parsed.ok, `its innerHTML did not read back — ${refusal(result.parsed)}`)
assert.deepEqual(revived(result.parsed.value), JSON.parse(json))
}) })
} }
@@ -147,5 +175,5 @@ for (const [index, { name }] of corpus.errors.entries()) {
} }
console.log( console.log(
`Firefox ${session.capabilities.browserVersion} converted ${corpus.roundTrip.length} round-trip, ${corpus.normalization.length} normalization and ${corpus.errors.length} error fixtures and ${corpus.realPayloads.length} real payloads`, `Firefox ${session.capabilities.browserVersion} converted ${corpus.roundTrip.length} round-trip, ${corpus.normalization.length} normalization and ${corpus.errors.length} error fixtures, ${corpus.realPayloads.length} real payloads and ${corpus.browserDocuments.length} documents through innerHTML`,
) )
+9 -4
View File
@@ -446,13 +446,17 @@ own: it puts the second panel's `<a>` around the text a card shows.
## `htmlToAdf` also reads the dialect as a browser serializes it ## `htmlToAdf` also reads the dialect as a browser serializes it
2026-10-28, the maintainer. Goal 6. Valid while a viewer/editor app may show `adfToHtml`'s HTML in a 2026-10-28, the maintainer. Goal 6. Valid while a viewer/editor app may show `adfToHtml`'s HTML in a
browser and post the page back. Lands with `todo.md` item 150. browser and post the page back.
A browser does not write the dialect back as `adfToHtml` wrote it. `htmlToAdf` reads both spellings: A browser does not write the dialect back as `adfToHtml` wrote it. `htmlToAdf` reads both spellings:
a page's `innerHTML`, as WHATWG HTML §Serializing HTML fragments spells it, and a form post, which a page's `innerHTML`, as WHATWG HTML §Serializing HTML fragments spells it, and a form post, which
ends each line in `\r\n`. So text's raw `"` reads as `&quot;` does, `&nbsp;` as a no-break space and ends each line in `\r\n`. So text's raw `"` reads as `&quot;` does, `&nbsp;` as a no-break space and
`\r\n` as a line feed, in what a carry element shows too. Every element the dialect writes reads this `\r\n` as a line feed, in what a carry element shows too. An attribute value's raw `<` and `>` read as
way, those later items add included. HTML no browser writes from the dialect stays refused. `&lt;` and `&gt;` do: the serialization algorithm escapes them now, and a browser from before that
writes them raw. These are every difference §Serializing HTML fragments leaves between a browser's
text and `adfToHtml`'s, checked against Firefox's own `innerHTML` in the browser leg. Every element
the dialect writes reads this way, those later items add included. HTML no browser writes from the
dialect stays refused, a lone `\r` among it. A refusal's position counts the HTML as handed in.
## No runtime dependencies ## No runtime dependencies
@@ -740,7 +744,8 @@ resolver maps them, under `NodeNext` alone; a `.d.ts` reader that is not `tsc` s
SpiderMonkey. SpiderMonkey.
A headless Firefox loads `dist/index.js` over HTTP and converts the round-trip, normalization and A headless Firefox loads `dist/index.js` over HTTP and converts the round-trip, normalization and
error fixtures and the real payloads — the `commonmark-spec` sort is the Node suite's to check — error fixtures and the real payloads, and reads the HTML its own `innerHTML` writes back from
`adfToHtml`'s — the `commonmark-spec` sort is the Node suite's to check —
which is the browser half of §Any ES2022 engine and the only SpiderMonkey there is — the gate's which is the browser half of §Any ES2022 engine and the only SpiderMonkey there is — the gate's
other engines are two V8s and a JavaScriptCore that is not Safari's. A WebDriver session is what other engines are two V8s and a JavaScriptCore that is not Safari's. A WebDriver session is what
carries a verdict back out, the driver and the page's server sharing one network namespace so each carries a verdict back out, the driver and the page's server sharing one network namespace so each
+15 -5
View File
@@ -119,14 +119,24 @@ later release writes it as its own element.
## Reading ## Reading
`htmlToAdf` reads exactly the HTML above, so `htmlToAdf(adfToHtml(doc))` deep-equals `doc`. It `htmlToAdf` reads exactly the HTML above, so `htmlToAdf(adfToHtml(doc))` deep-equals `doc`. It also
reads a carry element's node from its `data-adf`, and what the element shows, through its closing reads the HTML above as a browser serializes it (WHATWG HTML §13.3) and as a form post sends it,
tag, must be what `adfToHtml` shows for that node: an edit there would be lost, so it is anywhere the HTML above has the character, a carry element's `data-adf` and what it shows included:
| The HTML above writes | Also read |
| --- | --- |
| `&quot;` in text | a raw `"` |
| a no-break space | `&nbsp;` |
| `&lt;` and `&gt;` in an attribute value | a raw `<` and `>`, as older browsers write them |
| a line feed | `\r\n`; a refusal's position counts the HTML as handed in |
It reads a carry element's node from its `data-adf`, and what the element shows, through its
closing tag, must be what `adfToHtml` shows for that node: an edit there would be lost, so it is
`unmappable-html`. `unmappable-html`.
- Any other HTML is `unmappable-html`: another element, attribute or escape, attributes in another - Any other HTML is `unmappable-html`: another element, attribute or escape, attributes in another
order, a block on the line of another, a carriage return anywhere, or non-empty HTML not ending order, a block on the line of another, a carriage return not followed by a line feed, or
in a line feed. non-empty HTML not ending in a line feed.
- A `data-*` attribute that its element does not take, such as `data-level` on `<h1>`, or that holds - A `data-*` attribute that its element does not take, such as `data-level` on `<h1>`, or that holds
a value outside its attribute's kind or not written as compact JSON, is `unsupported-node-shape`. a value outside its attribute's kind or not written as compact JSON, is `unsupported-node-shape`.
- A carry element's `data-adf` holding invalid JSON is `unmappable-html`. JSON that is no ADF node, - A carry element's `data-adf` holding invalid JSON is `unmappable-html`. JSON that is no ADF node,
+48 -12
View File
@@ -1,17 +1,35 @@
const entities: ReadonlyMap<string, string> = new Map([ export type Spelling = 'attribute' | 'text'
const escapable = /[&<>"]/g
const escapes: ReadonlyMap<string, string> = new Map([
['"', '&quot;'],
['&', '&amp;'],
['<', '&lt;'],
['>', '&gt;'],
])
const writtenEntities = [...escapes.values()]
const entity = /&(?:amp|gt|lt|nbsp|quot);/g
const entityValues: ReadonlyMap<string, string> = new Map([
['&amp;', '&'], ['&amp;', '&'],
['&gt;', '>'], ['&gt;', '>'],
['&lt;', '<'], ['&lt;', '<'],
['&nbsp;', '\u00a0'],
['&quot;', '"'], ['&quot;', '"'],
]) ])
const entity = new RegExp([...entities.keys()].join('|'), 'g')
const escapable = new RegExp(`[${[...entities.values()].join('')}]`, 'g')
const escapes: ReadonlyMap<string, string> = new Map([...entities].map(([spelled, character]) => [character, spelled]))
// What HTML text and attribute values cannot hold (WHATWG HTML §13.1.3): any control but a tab, line feed or form feed, a carriage return among them, since a parser reads it as a line feed; a noncharacter; and a lone surrogate, which UTF-8 cannot encode. // What HTML text and attribute values cannot hold (WHATWG HTML §13.1.3): any control but a tab, line feed or form feed, a carriage return among them, since a parser reads it as a line feed; a noncharacter; and a lone surrogate, which UTF-8 cannot encode.
const unwritable = /(?![\t\n\f ])[\p{Cc}\p{Cs}\p{Noncharacter_Code_Point}]/u const unwritable = /(?![\t\n\f ])[\p{Cc}\p{Cs}\p{Noncharacter_Code_Point}]/u
const unwritables = new RegExp(unwritable.source, 'gu') const unwritables = new RegExp(unwritable.source, 'gu')
// A character `escapeHtml` escapes, other than the `&` opening an entity, or one HTML cannot hold. const strayAmpersand = '&(?!(?:amp|gt|lt|nbsp|quot);)'
const misspelling = new RegExp(`(?!${entity.source})${escapable.source}|${unwritable.source}`, 'u') // What reads wrongly in a value: `&` outside an entity, a character HTML cannot hold, and in text a `<` or `>`. An attribute value may hold `<` and `>` raw, as older browsers write it, and text may hold `"`.
const misspellings: Readonly<Record<Spelling, RegExp>> = {
attribute: new RegExp(`${strayAmpersand}|${unwritable.source}`, 'u'),
text: new RegExp(`${strayAmpersand}|[<>]|${unwritable.source}`, 'u'),
}
// What `adfToHtml` spells with an entity, a browser (WHATWG HTML §13.3) spells raw in text, or in an attribute; a no-break space it spells `&nbsp;` in both.
const browserSpellings: Readonly<Record<Spelling, ReadonlyMap<string, string>>> = {
attribute: new Map([['&gt;', '>'], ['&lt;', '<'], ['\u00a0', '&nbsp;']]),
text: new Map([['&quot;', '"'], ['\u00a0', '&nbsp;']]),
}
// The carry's JSON, which spells each character HTML cannot hold as a JSON escape reading back to the same string. // The carry's JSON, which spells each character HTML cannot hold as a JSON escape reading back to the same string.
export function holdableJson(json: string): string { export function holdableJson(json: string): string {
@@ -22,9 +40,9 @@ export function jsonValue(json: string): string {
return escapeHtml(holdableJson(json)) return escapeHtml(holdableJson(json))
} }
// The index of the first character `writableValue` would not write so, -1 where none stands in `html`. // The index of the first character that reads wrongly in `html`, -1 where none stands in it.
export function misspelledAt(html: string): number { export function misspelledAt(html: string, spelling: Spelling): number {
return html.search(misspelling) return html.search(misspellings[spelling])
} }
// ` name="value"`, `''` where the value is undefined, `undefined` where HTML cannot hold it. // ` name="value"`, `''` where the value is undefined, `undefined` where HTML cannot hold it.
@@ -38,9 +56,27 @@ export function shownValue(value: string, lineFeed: string): string {
return escapeHtml(value.replace(unwritables, '')).replaceAll('\n', lineFeed) return escapeHtml(value.replace(unwritables, '')).replaceAll('\n', lineFeed)
} }
// The value an escaped HTML text or attribute stands for, `undefined` where `writableValue` would not write it so. // Where `html` reads as `shown` from `index`: `whole`, and the index after it, or where it first reads otherwise. `shown` is spelled as `adfToHtml` writes it, `html` as a browser may too.
export function unescapedValue(html: string): string | undefined { export function readsAsShown(html: string, index: number, shown: string): { at: number; whole: boolean } {
return misspelledAt(html) < 0 ? html.replace(entity, (spelled) => entities.get(spelled) ?? spelled) : undefined if (html.startsWith(shown, index)) return { at: index + shown.length, whole: true }
let at = index
let spelling: Spelling = 'text'
for (let next = 0; next < shown.length; ) {
const token = writtenEntities.find((spelled) => shown.startsWith(spelled, next)) ?? shown.charAt(next)
// Only a tag holds a raw `<` or `>`, and only its attribute values an escape.
if (token === '<') spelling = 'attribute'
if (token === '>') spelling = 'text'
const read = [token, browserSpellings[spelling].get(token)].find((spelled) => spelled !== undefined && html.startsWith(spelled, at))
if (read === undefined) return { at, whole: false }
at += read.length
next += token.length
}
return { at, whole: true }
}
// The value an escaped HTML text or attribute stands for, `undefined` where `misspelledAt` finds a character that reads wrongly.
export function unescapedValue(html: string, spelling: Spelling): string | undefined {
return misspelledAt(html, spelling) < 0 ? html.replace(entity, (spelled) => entityValues.get(spelled) ?? spelled) : undefined
} }
// `undefined` where HTML cannot hold the value, so its node rides the carry element. // `undefined` where HTML cannot hold the value, so its node rides the carry element.
+4 -5
View File
@@ -4,7 +4,7 @@ import { carriedNesting, carriedNode, carriedValue } from '../../adf/carried-jso
import { carryContent } from '../carry-content.ts' import { carryContent } from '../carry-content.ts'
import { carryJsonOffset, readCarryAttributes } from '../carry-attributes.ts' import { carryJsonOffset, readCarryAttributes } from '../carry-attributes.ts'
import { foundAt, refused, type Place, type ReadNode, type StartTag } from './source.ts' import { foundAt, refused, type Place, type ReadNode, type StartTag } from './source.ts'
import { holdableJson } from '../escaping.ts' import { holdableJson, readsAsShown } from '../escaping.ts'
import { spellsCanonicalJson } from '../../canonical-json.ts' import { spellsCanonicalJson } from '../../canonical-json.ts'
import { success } from '../../result.ts' import { success } from '../../result.ts'
@@ -18,10 +18,9 @@ export function readCarry(html: string, tag: StartTag, element: 'div' | 'span',
const node = heldNode(json, levels, html, jsonAt) const node = heldNode(json, levels, html, jsonAt)
if (node.fault !== undefined) return refused(node.fault.code, node.fault.message, html, jsonAt, place) if (node.fault !== undefined) return refused(node.fault.code, node.fault.message, html, jsonAt, place)
const shown = `${carryContent(node.value, element)}</${element}>` const shown = `${carryContent(node.value, element)}</${element}>`
if (html.startsWith(shown, tag.end)) return success({ end: tag.end + shown.length, node: node.value }) const read = readsAsShown(html, tag.end, shown)
let at = tag.end if (read.whole) return success({ end: read.at, node: node.value })
while (html[at] === shown[at - tag.end]) at += 1 return refused('unmappable-html', `a carry element shows exactly what adfToHtml writes for its node, then </${element}>; to change the node, edit its ADF and write the HTML again: ${foundAt(html, read.at)}`, html, read.at, place)
return refused('unmappable-html', `a carry element shows exactly what adfToHtml writes for its node, then </${element}>; to change the node, edit its ADF and write the HTML again: ${foundAt(html, at)}`, html, at, place)
} }
function heldNode(json: string, levels: number, html: string, jsonAt: number): Read<AdfNode> { function heldNode(json: string, levels: number, html: string, jsonAt: number): Read<AdfNode> {
+50 -3
View File
@@ -52,7 +52,7 @@ const rule = JSON.stringify({ type: 'unknownBlock' }).replaceAll('"', '&quot;')
// The carry element of `node`, showing `shown`. // The carry element of `node`, showing `shown`.
function carry(tag: 'div' | 'span', node: unknown, shown: string): string { function carry(tag: 'div' | 'span', node: unknown, shown: string): string {
return `<${tag} class="adf-carry" data-adf="${JSON.stringify(node).replaceAll('"', '&quot;')}">${shown}</${tag}>` return `<${tag} class="adf-carry" data-adf="${JSON.stringify(node).replaceAll('&', '&amp;').replaceAll('<', '&lt;').replaceAll('>', '&gt;').replaceAll('"', '&quot;')}">${shown}</${tag}>`
} }
function nested(value: unknown, levels: number, wrap: (inner: unknown) => unknown): unknown { function nested(value: unknown, levels: number, wrap: (inner: unknown) => unknown): unknown {
@@ -83,6 +83,53 @@ test('reads escaped text, and keeps tabs, line feeds and form feeds', () => {
assert.deepEqual(read(htmlToAdf('<p>a &lt; b &amp;&amp; c &gt; &quot;d&quot; \'e\'\tf\ng\fh &amp;lt;</p>\n')), document(paragraph(text('a < b && c > "d" \'e\'\tf\ng\fh &lt;')))) assert.deepEqual(read(htmlToAdf('<p>a &lt; b &amp;&amp; c &gt; &quot;d&quot; \'e\'\tf\ng\fh &amp;lt;</p>\n')), document(paragraph(text('a < b && c > "d" \'e\'\tf\ng\fh &lt;'))))
}) })
test('reads text as a browser\'s innerHTML spells it: a raw ", &nbsp; for a no-break space, and in an attribute a raw < or >', () => {
assert.deepEqual(read(htmlToAdf('<p>a "b" &quot;c&quot;&nbsp;d e</p>\n')), document(paragraph(text('a "b" "c" d e'))))
assert.deepEqual(read(htmlToAdf('<pre><code>say "a"&nbsp;b</code></pre>\n')), document({ content: [text('say "a" b')], type: 'codeBlock' }))
const link: AdfMark = { attrs: { href: 'https://x.test/?a<b>c&d', title: 'a b' }, type: 'link' }
assert.deepEqual(read(htmlToAdf('<p><a href="https://x.test/?a<b>c&amp;d" title="a&nbsp;b">x</a></p>\n')), document(paragraph(text('x', link))))
assert.deepEqual(read(htmlToAdf('<p><a href="https://x.test/?a&lt;b&gt;c&amp;d" title="a&nbsp;b">x</a></p>\n')), document(paragraph(text('x', link))))
assert.deepEqual(read(htmlToAdf('<hr data-style="a&nbsp;&quot;b">\n')), document({ attrs: { style: 'a "b' }, type: 'rule' }))
})
test('reads a line ending of CRLF as a line feed wherever the dialect has one, and keeps a lone carriage return refused', () => {
const crlf = (html: string): string => html.replaceAll('\n', '\r\n')
assert.deepEqual(read(htmlToAdf(crlf('<p>a\nb</p>\n<hr>\n<pre><code>c\n d</code></pre>\n'))), document(paragraph(text('a\nb')), { type: 'rule' }, { content: [text('c\n d')], type: 'codeBlock' }))
assert.deepEqual(read(htmlToAdf(crlf('<blockquote>\n<ul>\n<li>Bolt\n<p>b</p>\n</li>\n</ul>\n</blockquote>\n'))), document({ content: [{ content: [item(paragraph(text('Bolt')), paragraph(text('b')))], type: 'bulletList' }], type: 'blockquote' }))
assert.deepEqual(read(htmlToAdf(crlf(`${carry('div', panel, '<p>Check the collation.</p>')}\n`))), document(panel))
assert.deepEqual(read(htmlToAdf(crlf(`${documentCarry}\n`))), { type: 'doc', version: 1 })
assert.deepEqual(read(htmlToAdf('<hr data-style="a\r\nb">\n')), document({ attrs: { style: 'a\nb' }, type: 'rule' }))
for (const html of ['<p>a\r</p>\n', '<p>a</p>\r', '<p>a</p>\r\r\n', '<p>a\r\r\nb</p>\n']) assert.equal(refusal(htmlToAdf(html)).startsWith('unmappable-html'), true, JSON.stringify(html))
})
test('places a refusal in HTML ending its lines in CRLF at the start of the line it stands on, counted in the HTML handed in', () => {
assert.equal(refusal(htmlToAdf('<p>a</p>\r\n<b>x</b>\r\n')), 'unmappable-html at /content/1 line 2 offset 10')
assert.equal(refusal(htmlToAdf('<p>a</p>\r\n<p>b</p>\r\n<p>c<b>d</b></p>\r\n')), 'unmappable-html at /content/2 line 3 offset 20')
assert.equal(refusal(htmlToAdf('<p>a</p>\r\n<p>b</p>')), 'unmappable-html at / line 2 offset 10')
})
test('reads a carry element showing what a browser spells it as, in any mix with the spelling adfToHtml writes', () => {
const said: AdfNode = { attrs: { id: ids.mention, text: 'Say "hi" now' }, type: 'mention' }
for (const shown of ['Say &quot;hi&quot; now', 'Say "hi"&nbsp;now', 'Say "hi" now', 'Say &quot;hi&quot;&nbsp;now']) {
assert.deepEqual(read(htmlToAdf(`<p>${carry('span', said, shown)}</p>\n`)), document(paragraph(said)), shown)
}
const underlined: AdfNode = text('a<b', { type: 'underline' }, { attrs: { href: 'https://x.test/?a<b&c>d', title: 'T" ' }, type: 'link' })
for (const opener of ['<a href="https://x.test/?a&lt;b&amp;c&gt;d" title="T&quot; ">', '<a href="https://x.test/?a<b&amp;c>d" title="T&quot;&nbsp;">']) {
assert.deepEqual(read(htmlToAdf(`<p>${carry('span', underlined, `${opener}a&lt;b</a>`)}</p>\n`)), document(paragraph(underlined)), opener)
}
const block: AdfNode = { attrs: { panelType: 'info' }, content: [paragraph(text('A "quote" here'))], type: 'panel' }
assert.deepEqual(read(htmlToAdf(`${carry('div', block, '<p>A "quote"&nbsp;here</p>')}\r\n`)), document(block))
})
test('refuses an edit to what a carry element shows even where a browser would spell the same text so', () => {
const said: AdfNode = { attrs: { id: ids.mention, text: 'Say "hi"' }, type: 'mention' }
const underlined: AdfNode = text('a', { type: 'underline' }, { attrs: { href: 'x', title: 'T"' }, type: 'link' })
assert.equal(refusal(htmlToAdf(`<p>${carry('span', said, 'Say "hi')}</p>\n`)), 'unmappable-html at /content/0/content/0 line 1 offset 0')
assert.equal(refusal(htmlToAdf(`<p>${carry('span', said, 'Say &amp;quot;hi&amp;quot;')}</p>\n`)), 'unmappable-html at /content/0/content/0 line 1 offset 0')
assert.equal(refusal(htmlToAdf(`<p>${carry('span', underlined, '<a href="x" title="T"">a</a>')}</p>\n`)), 'unmappable-html at /content/0/content/0 line 1 offset 0')
assert.equal(refusal(htmlToAdf(`<p>${carry('span', underlined, '<a href="x" title="T&quot;">a</a>')}</p>\n`)), 'read')
})
test('reads lists, an item\'s first paragraph on its line and its other blocks a line each', () => { test('reads lists, an item\'s first paragraph on its line and its other blocks a line each', () => {
assert.deepEqual(read(htmlToAdf('<ul>\n<li>Bolt</li>\n<li>Nut</li>\n</ul>\n')), document({ content: [item(paragraph(text('Bolt'))), item(paragraph(text('Nut')))], type: 'bulletList' })) assert.deepEqual(read(htmlToAdf('<ul>\n<li>Bolt</li>\n<li>Nut</li>\n</ul>\n')), document({ content: [item(paragraph(text('Bolt'))), item(paragraph(text('Nut')))], type: 'bulletList' }))
const listed: AdfNode = { attrs: { order: 3 }, content: [item(paragraph(text('Bolt')), { content: [item(paragraph(text('M8')))], type: 'bulletList' }), item(paragraph(text('a')), paragraph(text('b')))], type: 'orderedList' } const listed: AdfNode = { attrs: { order: 3 }, content: [item(paragraph(text('Bolt')), { content: [item(paragraph(text('M8')))], type: 'bulletList' }), item(paragraph(text('a')), paragraph(text('b')))], type: 'orderedList' }
@@ -279,9 +326,9 @@ test('refuses a tag of three million attributes, past what one pattern over them
const messages: readonly (readonly [string, string])[] = [ const messages: readonly (readonly [string, string])[] = [
['<p>a<b>b</b></p>\n', '<b> as written here is no inline element of the dialect: found "<b>b</b></p>\\n"'], ['<p>a<b>b</b></p>\n', '<b> as written here is no inline element of the dialect: found "<b>b</b></p>\\n"'],
['<p>a > b</p>\n', 'text escapes &, <, > and " as &amp;, &lt;, &gt; and &quot;, and holds no character HTML cannot hold: found "> b</p>\\n"'], ['<p>a > b</p>\n', 'text escapes & as &amp;, < as &lt; and > as &gt;, and holds no character HTML cannot hold: found "> b</p>\\n"'],
['<p><a href="a"b">a</a></p>\n', '<a> holds its attributes as name="value", each value escaped as text is, then >: found "b\\">a</a></p>\\n"'], ['<p><a href="a"b">a</a></p>\n', '<a> holds its attributes as name="value", each value escaped as text is, then >: found "b\\">a</a></p>\\n"'],
['<p>a&nbsp;\u00a0</p>\n', 'text escapes &, <, > and " as &amp;, &lt;, &gt; and &quot;, and holds no character HTML cannot hold: found "&nbsp;\\u00a0</p>\\n"'], ['<p>a&nbsp</p>\n', 'text escapes & as &amp;, < as &lt; and > as &gt;, and holds no character HTML cannot hold: found "&nbsp</p>\\n"'],
['<pre><code>a</code></pre\n', 'a code block\'s text ends at </code></pre>: found "</code></pre\\n"'], ['<pre><code>a</code></pre\n', 'a code block\'s text ends at </code></pre>: found "</code></pre\\n"'],
['<pre><code class="hljs language-js">x</code></pre>\n', '<code> as written here is no code element of the dialect, which writes <code> or <code class="language-…">: found "<code class=\\"hljs langua"…'], ['<pre><code class="hljs language-js">x</code></pre>\n', '<code> as written here is no code element of the dialect, which writes <code> or <code class="language-…">: found "<code class=\\"hljs langua"…'],
['<ul>\n<li>a</ul>\n', 'an item\'s line ends at </li>, or at a line feed before the item\'s next block, and its text spells < as &lt;: found "</ul>\\n"'], ['<ul>\n<li>a</ul>\n', 'an item\'s line ends at </li>, or at a line feed before the item\'s next block, and its text spells < as &lt;: found "</ul>\\n"'],
+10 -3
View File
@@ -7,7 +7,7 @@ import { carryContent } from '../carry-content.ts'
import { centeredImage, externalMedia } from '../../adf/plain-image.ts' import { centeredImage, externalMedia } from '../../adf/plain-image.ts'
import { containerElements, headingElements, headingLevel, readListStart, type ContainerType } from '../block-elements.ts' import { containerElements, headingElements, headingLevel, readListStart, type ContainerType } from '../block-elements.ts'
import { firstParagraphForm } from '../first-paragraph.ts' import { firstParagraphForm } from '../first-paragraph.ts'
import { foundAt, misread, readStartTag, refused, startTagName, type Place, type ReadNode, type StartTag } from './source.ts' import { foundAt, misread, positionIn, readStartTag, refused, startTagName, type Place, type ReadNode, type StartTag } from './source.ts'
import { largestNesting } from '../../nesting.ts' import { largestNesting } from '../../nesting.ts'
import { misspelledAt, unescapedValue } from '../escaping.ts' import { misspelledAt, unescapedValue } from '../escaping.ts'
import { misspelledText, readInline } from './inline.ts' import { misspelledText, readInline } from './inline.ts'
@@ -59,8 +59,15 @@ export function htmlToAdf(html: string): Result<AdfDocument, ParseError> {
) )
} }
// A form post ends each line in `\r\n`, which reads as the line feed `adfToHtml` writes; a refusal's position still counts the HTML as handed in.
function readHtml(html: string): Result<AdfDocument> { function readHtml(html: string): Result<AdfDocument> {
if (typeof html !== 'string') return notAString(html, 'HTML') if (typeof html !== 'string') return notAString(html, 'HTML')
const read = readLineFeeds(html.replaceAll('\r\n', '\n'))
if (read.ok || read.error.position === undefined) return read
return { error: { ...read.error, position: positionIn(html, read.error.position) }, ok: false }
}
function readLineFeeds(html: string): Result<AdfDocument> {
if (html === '') return success({ content: [], type: 'doc', version: 1 }) if (html === '') return success({ content: [], type: 'doc', version: 1 })
return readDocumentCarry(html) ?? readBlocks(html) return readDocumentCarry(html) ?? readBlocks(html)
} }
@@ -191,8 +198,8 @@ function readCodeBlock(html: string, tag: StartTag, place: Place): Result<BlockS
const next = html.indexOf('<', code.end) const next = html.indexOf('<', code.end)
const end = next < 0 ? html.length : next const end = next < 0 ? html.length : next
const spelled = html.slice(code.end, end) const spelled = html.slice(code.end, end)
const text = unescapedValue(spelled) const text = unescapedValue(spelled, 'text')
if (text === undefined) return misspelledText(html, code.end + misspelledAt(spelled), place) if (text === undefined) return misspelledText(html, code.end + misspelledAt(spelled, 'text'), place)
if (!html.startsWith(codeCloser, end)) return refused('unmappable-html', `a code block's text ends at ${codeCloser}: ${foundAt(html, end)}`, html, end, place) if (!html.startsWith(codeCloser, end)) return refused('unmappable-html', `a code block's text ends at ${codeCloser}: ${foundAt(html, end)}`, html, end, place)
const { language } = named const { language } = named
return success({ end: end + codeCloser.length, kind: 'node', node: built('codeBlock', language === undefined ? data.value : { ...data.value, language }, text === '' ? [] : [{ text, type: 'text' }]) }) return success({ end: end + codeCloser.length, kind: 'node', node: built('codeBlock', language === undefined ? data.value : { ...data.value, language }, text === '' ? [] : [{ text, type: 'text' }]) })
+4 -3
View File
@@ -63,13 +63,14 @@ function readMarked(html: string, index: number, place: Place): ReadNode {
function readText(html: string, index: number, marks: AdfMark[], place: Place): ReadNode { function readText(html: string, index: number, marks: AdfMark[], place: Place): ReadNode {
const next = html.indexOf('<', index) const next = html.indexOf('<', index)
const end = next < 0 ? html.length : next const end = next < 0 ? html.length : next
const text = unescapedValue(html.slice(index, end)) const spelled = html.slice(index, end)
if (text === undefined) return misspelledText(html, index + misspelledAt(html.slice(index, end)), place) const text = unescapedValue(spelled, 'text')
if (text === undefined) return misspelledText(html, index + misspelledAt(spelled, 'text'), place)
return success({ end, node: marks.length === 0 ? { text, type: 'text' } : { marks, text, type: 'text' } }) return success({ end, node: marks.length === 0 ? { text, type: 'text' } : { marks, text, type: 'text' } })
} }
export function misspelledText<T>(html: string, index: number, place: Place): Result<T, ParseError> { export function misspelledText<T>(html: string, index: number, place: Place): Result<T, ParseError> {
return refused('unmappable-html', `text escapes &, <, > and " as &amp;, &lt;, &gt; and &quot;, and holds no character HTML cannot hold: ${foundAt(html, index)}`, html, index, place) return refused('unmappable-html', `text escapes & as &amp;, < as &lt; and > as &gt;, and holds no character HTML cannot hold: ${foundAt(html, index)}`, html, index, place)
} }
function markOf(tag: StartTag): AdfMark | undefined { function markOf(tag: StartTag): AdfMark | undefined {
+10 -3
View File
@@ -45,6 +45,13 @@ export function foundAt(html: string, index: number): string {
return `found ${spelled}${index + found.length < html.length ? '…' : ''}` return `found ${spelled}${index + found.length < html.length ? '…' : ''}`
} }
// The start of the same line in `html`, where the position was counted after its `\r\n` became line feeds.
export function positionIn(html: string, { line }: SourcePosition): SourcePosition {
let offset = 0
for (let passed = 1; passed < line; passed += 1) offset = html.indexOf('\n', offset) + 1
return { line, offset }
}
// At the start of the line `index` stands on. // At the start of the line `index` stands on.
export function refused<T>(code: ConvertErrorCode, message: string, html: string, index: number, place: Place | undefined): Result<T, ParseError> { export function refused<T>(code: ConvertErrorCode, message: string, html: string, index: number, place: Place | undefined): Result<T, ParseError> {
return { error: { code, message, path: pathOf(place), position: lineStart(html, index) }, ok: false } return { error: { code, message, path: pathOf(place), position: lineStart(html, index) }, ok: false }
@@ -61,14 +68,14 @@ function scanStartTag(html: string, index: number): CutTag | StartTag | undefine
const match = attribute.exec(html) const match = attribute.exec(html)
if (match === null) return html.startsWith('>', at) ? { attributes, end: at + 1, name, start: index } : { name, stopped: at } if (match === null) return html.startsWith('>', at) ? { attributes, end: at + 1, name, start: index } : { name, stopped: at }
const [whole, key = '', spelled = ''] = match const [whole, key = '', spelled = ''] = match
const value = unescapedValue(spelled) const value = unescapedValue(spelled, 'attribute')
if (value === undefined) return { name, stopped: at + whole.length - 1 - spelled.length + misspelledAt(spelled) } if (value === undefined) return { name, stopped: at + whole.length - 1 - spelled.length + misspelledAt(spelled, 'attribute') }
attributes.push([key, value]) attributes.push([key, value])
at += whole.length at += whole.length
} }
} }
// Only a line feed ends a line: the reader refuses a carriage return wherever it stands. // Only a line feed ends a line: readHtml has made each \r\n one, and a lone carriage return is refused.
function lineStart(html: string, index: number): SourcePosition { function lineStart(html: string, index: number): SourcePosition {
let line = 1 let line = 1
let offset = 0 let offset = 0
+2 -13
View File
@@ -12,13 +12,12 @@
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score | | ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|---|---|---|---|---|---|---|---|---|---|---|---| |---|---|---|---|---|---|---|---|---|---|---|---|
| 150 | 0.3.0 | | decision | **Read `adfToHtml`'s HTML in `htmlToAdf` as a browser serializes it too — a raw `"` in text, `&nbsp;`, `\r\n` line endings — so a page's `innerHTML` or a form post reads back.** | 2 | 2 | 2 | 4 | 4 | 6 by 5 | 5.0 |
| 115 | 0.3.0 | | defect | **Build text holding no `border` mark in `losslessMarkdownToAdf`, and have `adfToLosslessMarkdown` carry text holding one.** | 2 | 1 | 2 | 3 | 3 | 4 by 3 | 4.6 | | 115 | 0.3.0 | | defect | **Build text holding no `border` mark in `losslessMarkdownToAdf`, and have `adfToLosslessMarkdown` carry text holding one.** | 2 | 1 | 2 | 3 | 3 | 4 by 3 | 4.6 |
| 91 | 0.3.0 | | | **Spell an emphasis once across the code spans, an emphasis or link once across the hard breaks, dates, emoji, inline cards, mentions and statuses between two nodes it marks, and an emphasis across inline media, in both writers.** | 3 | 2 | 2 | 5 | 5 | 6 by 6 | 3.6 | | 91 | 0.3.0 | | | **Spell an emphasis once across the code spans, an emphasis or link once across the hard breaks, dates, emoji, inline cards, mentions and statuses between two nodes it marks, and an emphasis across inline media, in both writers.** | 3 | 2 | 2 | 5 | 5 | 6 by 6 | 3.6 |
| 133 | 0.3.0 | | decision | **Write mentions, emoji, dates, statuses and inline cards as semantic HTML in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 4 | 6 by 3 | 3.0 | | 133 | 0.3.0 | | decision | **Write mentions, emoji, dates, statuses and inline cards as semantic HTML in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 4 | 6 by 3 | 3.0 |
| 132 | 0.3.0 | | | **Write tables as `<table>` in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 5 | 6 by 4 | 2.4 | | 132 | 0.3.0 | | | **Write tables as `<table>` in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 5 | 6 by 4 | 2.4 |
| 73 | 0.3.0 | 91, 115, 132, 133, 150 | | **Announce the package where someone needing an ADF converter already reads: JRACLOUD-77436, the Atlassian developer community and Stack Overflow's ADF-to-markdown questions.** | 1 | 2 | 2 | 3 | 2 | 3 by 8 | 16.9 | | 73 | 0.3.0 | 91, 115, 132, 133 | | **Announce the package where someone needing an ADF converter already reads: JRACLOUD-77436, the Atlassian developer community and Stack Overflow's ADF-to-markdown questions.** | 1 | 2 | 2 | 3 | 2 | 3 by 8 | 16.9 |
| 72 | 0.3.0 | 91, 115, 132, 133, 150 | decision | **Re-create `gitea.larvit.se/larvit/adf-codec` as a read-only pull mirror of GitHub, its issues and Actions off and its description pointing at GitHub.** | 1 | 2 | 1 | 2 | 1 | 3 by 1 | 2.5 | | 72 | 0.3.0 | 91, 115, 132, 133 | decision | **Re-create `gitea.larvit.se/larvit/adf-codec` as a read-only pull mirror of GitHub, its issues and Actions off and its description pointing at GitHub.** | 1 | 2 | 1 | 2 | 1 | 3 by 1 | 2.5 |
| 153 | 0.4.0 | | question | **Decide whether `src/markdown/` root may import `emit/`, `parse/` or `portable/`, and hold the answer in `.oxlintrc.json` and §The source parts by ADF and format.** | 1 | 2 | 2 | 3 | 2 | code health by 3 | 26.2 | | 153 | 0.4.0 | | question | **Decide whether `src/markdown/` root may import `emit/`, `parse/` or `portable/`, and hold the answer in `.oxlintrc.json` and §The source parts by ADF and format.** | 1 | 2 | 2 | 3 | 2 | code health by 3 | 26.2 |
| 156 | 0.4.0 | | | **Stop a leg's container when its script is interrupted, so Ctrl-C ends `ci.sh` or `mutation-tests/run.sh` at once.** | 1 | 2 | 1 | 2 | 1 | code health by 2 | 20.9 | | 156 | 0.4.0 | | | **Stop a leg's container when its script is interrupted, so Ctrl-C ends `ci.sh` or `mutation-tests/run.sh` at once.** | 1 | 2 | 1 | 2 | 1 | code health by 2 | 20.9 |
| 128 | 0.4.0 | | | **Pin the portable flavour's block reading and writing with tests: callout titles past one line, a link beside other marks in a title, the nesting limit of callouts and task lists, refusals inside tasks and their paths, and empty or non-paragraph tasks.** | 2 | 1 | 1 | 2 | 3 | code health by 3 | 19.6 | | 128 | 0.4.0 | | | **Pin the portable flavour's block reading and writing with tests: callout titles past one line, a link beside other marks in a title, the nesting limit of callouts and task lists, refusals inside tasks and their paths, and empty or non-paragraph tasks.** | 2 | 1 | 1 | 2 | 3 | code health by 3 | 19.6 |
@@ -69,16 +68,6 @@
## Details ## Details
### 150. Read `adfToHtml`'s HTML in `htmlToAdf` as a browser serializes it too — a raw `"` in text, `&nbsp;`, `\r\n` line endings — so a page's `innerHTML` or a form post reads back.
`htmlToAdf` refuses all three today. A browser's `innerHTML` writes text's `"` raw and a no-break
space as `&nbsp;`, and a form post turns every line feed into `\r\n`, so a viewer/editor app
sending its page back gets `unmappable-html`. `docs/decisions.md` §`htmlToAdf` also reads the
dialect as a browser serializes it records the maintainer's yes. Take every difference from WHATWG
HTML §Serializing HTML fragments, not only these three, and check them against a browser's own
`innerHTML`. README §HTML and `spec/html.md` say `\r\n` is refused; update both. The comprehension
panel found `misspelling` in `src/html/escaping.ts` hard to read: make it easier.
### 115. Build text holding no `border` mark in `losslessMarkdownToAdf`, and have `adfToLosslessMarkdown` carry text holding one. ### 115. Build text holding no `border` mark in `losslessMarkdownToAdf`, and have `adfToLosslessMarkdown` carry text holding one.
`!adf:border[x]{color="#091e42" size=2}` builds text holding `border`, which `!adf:border[x]{color="#091e42" size=2}` builds text holding `border`, which