diff --git a/CHANGELOG.md b/CHANGELOG.md index d5cc88f..b484099 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,8 +15,10 @@ content, links included. Every URL is written as the ADF holds it, `javascript:` included: sanitize HTML from an author you do not trust before showing it. - `htmlToAdf` reads `adfToHtml`'s HTML back to the document it came from, a carry element from its - `data-adf`. It refuses any other HTML, including an edit to what a carry element shows, with the - codes README §The calls lists. + `data-adf`, also as a browser's `innerHTML` writes it (a raw `"` in text, ` `, a raw `<` or + `>` in an attribute value) and as a form post sends it (`\r\n` for each line feed). It refuses + any other HTML, including an edit to what a carry element shows, with the codes README §The calls + lists. - **Breaking:** `losslessMarkdownToAdf` gives a `date`, `emoji`, `inlineCard`, `mention`, `status` or `mediaInline` only the marks Atlassian's schema lets it hold: `**!adf:mention[@A]{id=…}**` now reads as a plain mention. A link or directive mark around only nodes that cannot take it is diff --git a/README.md b/README.md index 2158364..09b91ae 100644 --- a/README.md +++ b/README.md @@ -540,8 +540,10 @@ content. No stylesheet ships. The complete dialect: `htmlToAdf` reads exactly that HTML back. It reads a carry element's node from its `data-adf`, and what the element shows must be what `adfToHtml` shows for that node: an edit there would be lost, so -it is refused. Other HTML is refused as `unmappable-html`: another element, attribute or escape, -such as ``, `
` or `'`, and HTML missing its final line feed or ending a line in `\r\n`. +it is refused. It also reads the HTML as a browser's `innerHTML` writes it back and as a form post +sends it: a raw `"` in text, ` ` for a no-break space, a raw `<` or `>` in an attribute value, +and `\r\n` for a line feed. Other HTML is refused as `unmappable-html`: another element, attribute or +escape, such as ``, `
` or `'`, a lone `\r`, and HTML missing its final line feed. A few causes, such as a `data-*` attribute its element does not take, are `unsupported-node-shape` instead, and nesting past 500 levels is `unsupported-nesting-depth`. [Either direction](#either-direction) lists every cause of both. diff --git a/browser-tests/convert-corpus.js b/browser-tests/convert-corpus.js index 6ca2490..7b9886d 100644 --- a/browser-tests/convert-corpus.js +++ b/browser-tests/convert-corpus.js @@ -2,14 +2,23 @@ try { const { adfToHtml, adfToLosslessMarkdown, htmlToAdf, isAdfDocument, losslessMarkdownToAdf } = await import('/dist/index.js') // WebDriver's JSON reads -0 back as 0, so a document crosses as JSON text with -0 tagged; run.js revives it. const spelled = (result) => (result.ok ? { ok: true, value: JSON.stringify(result.value, (_, value) => (Object.is(value, -0) ? '\u0000-0' : value)) } : result) + // The page's own innerHTML is what a viewer/editor app posts back, so htmlToAdf reads what the browser writes. + const throughInnerHtml = (adf) => { + const html = adfToHtml(adf) + if (!html.ok) return { parsed: html } + const page = document.createElement('div') + page.innerHTML = html.value + return { parsed: spelled(htmlToAdf(page.innerHTML)) } + } window.convertCorpus = (corpus) => ({ + browserDocuments: corpus.browserDocuments.map(({ json, name }) => ({ name, ...throughInnerHtml(JSON.parse(json)) })), errors: corpus.errors.map(({ markdown, name }) => ({ name, parsed: losslessMarkdownToAdf(markdown) })), normalization: corpus.normalization.map(({ markdown, name }) => ({ name, parsed: spelled(losslessMarkdownToAdf(markdown)) })), realPayloads: corpus.realPayloads.map(({ json, name }) => { const adf = JSON.parse(json) const emitted = adfToLosslessMarkdown(adf) const html = adfToHtml(adf) - return { emitted, html, htmlParsed: html.ok ? spelled(htmlToAdf(html.value)) : undefined, isDocument: isAdfDocument(adf), name, parsed: emitted.ok ? spelled(losslessMarkdownToAdf(emitted.value)) : undefined } + return { emitted, html, htmlParsed: html.ok ? spelled(htmlToAdf(html.value)) : undefined, innerHtml: throughInnerHtml(adf), isDocument: isAdfDocument(adf), name, parsed: emitted.ok ? spelled(losslessMarkdownToAdf(emitted.value)) : undefined } }), roundTrip: corpus.roundTrip.map(({ json, markdown, name }) => { const adf = JSON.parse(json) diff --git a/browser-tests/run.js b/browser-tests/run.js index 64bffa0..c6fc9cf 100644 --- a/browser-tests/run.js +++ b/browser-tests/run.js @@ -70,7 +70,25 @@ async function whenDriverAnswers() { } } +// Text the browser's innerHTML spells other than adfToHtml does (WHATWG HTML §13.3): a raw ",  , and in an attribute < and > escaped or not by version. +const text = (value, ...marks) => (marks.length === 0 ? { text: value, type: 'text' } : { marks, text: value, type: 'text' }) +const browserDocuments = [ + { + json: JSON.stringify({ + content: [ + { content: [text('Say "hi"\u00a0now & '), text(' link', { attrs: { href: 'https://example.com/?ac&d="e"', title: 'A "title"\u00a0' }, type: 'link' })], type: 'paragraph' }, + { attrs: { language: 'js' }, content: [text('if (a < b) say("x\u00a0y")\n done()')], type: 'codeBlock' }, + { content: [{ attrs: { id: '0199f4c2-8b1e-7d03-9a6f-2c5e8d71b4a0', text: 'Say "hi"\u00a0now' }, type: 'mention' }], type: 'paragraph' }, + { attrs: { panelType: 'info' }, content: [{ content: [text('A "quote"\u00a0here', { type: 'underline' }, { attrs: { href: 'https://example.com/?a', title: 'T"\u00a0' }, type: 'link' })], type: 'paragraph' }], type: 'panel' }, + ], + type: 'doc', + version: 1, + }), + name: 'quotes, no-break spaces and angle brackets', + }, +] const corpus = { + browserDocuments, errors: fixtureNames('errors', '.md').map((name) => ({ markdown: fixture(name, '.md'), name })), normalization: fixtureNames('normalization', '.md').map((name) => ({ markdown: fixture(name, '.md'), name })), realPayloads: fixtureNames('real-payloads', '.json').map((name) => ({ json: fixture(name, '.json'), name })), @@ -135,6 +153,16 @@ for (const [index, { json, name }] of corpus.realPayloads.entries()) { assert.deepEqual(result.html, adfToHtml(JSON.parse(json)), 'it wrote other HTML than Node wrote') assert.ok(result.htmlParsed.ok, `its HTML did not read back — ${refusal(result.htmlParsed)}`) assert.deepEqual(revived(result.htmlParsed.value), JSON.parse(json)) + assert.ok(result.innerHtml.parsed.ok, `its innerHTML did not read back — ${refusal(result.innerHtml.parsed)}`) + assert.deepEqual(revived(result.innerHtml.parsed.value), JSON.parse(json)) + }) +} + +for (const [index, { json, name }] of corpus.browserDocuments.entries()) { + const result = results.browserDocuments[index] + checking(name, result, () => { + assert.ok(result.parsed.ok, `its innerHTML did not read back — ${refusal(result.parsed)}`) + assert.deepEqual(revived(result.parsed.value), JSON.parse(json)) }) } @@ -147,5 +175,5 @@ for (const [index, { name }] of corpus.errors.entries()) { } console.log( - `Firefox ${session.capabilities.browserVersion} converted ${corpus.roundTrip.length} round-trip, ${corpus.normalization.length} normalization and ${corpus.errors.length} error fixtures and ${corpus.realPayloads.length} real payloads`, + `Firefox ${session.capabilities.browserVersion} converted ${corpus.roundTrip.length} round-trip, ${corpus.normalization.length} normalization and ${corpus.errors.length} error fixtures, ${corpus.realPayloads.length} real payloads and ${corpus.browserDocuments.length} documents through innerHTML`, ) diff --git a/docs/decisions.md b/docs/decisions.md index 508c44b..6015f17 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -446,13 +446,17 @@ own: it puts the second panel's `` around the text a card shows. ## `htmlToAdf` also reads the dialect as a browser serializes it 2026-10-28, the maintainer. Goal 6. Valid while a viewer/editor app may show `adfToHtml`'s HTML in a -browser and post the page back. Lands with `todo.md` item 150. +browser and post the page back. A browser does not write the dialect back as `adfToHtml` wrote it. `htmlToAdf` reads both spellings: a page's `innerHTML`, as WHATWG HTML §Serializing HTML fragments spells it, and a form post, which ends each line in `\r\n`. So text's raw `"` reads as `"` does, ` ` as a no-break space and -`\r\n` as a line feed, in what a carry element shows too. Every element the dialect writes reads this -way, those later items add included. HTML no browser writes from the dialect stays refused. +`\r\n` as a line feed, in what a carry element shows too. An attribute value's raw `<` and `>` read as +`<` and `>` do: the serialization algorithm escapes them now, and a browser from before that +writes them raw. These are every difference §Serializing HTML fragments leaves between a browser's +text and `adfToHtml`'s, checked against Firefox's own `innerHTML` in the browser leg. Every element +the dialect writes reads this way, those later items add included. HTML no browser writes from the +dialect stays refused, a lone `\r` among it. A refusal's position counts the HTML as handed in. ## No runtime dependencies @@ -740,7 +744,8 @@ resolver maps them, under `NodeNext` alone; a `.d.ts` reader that is not `tsc` s SpiderMonkey. A headless Firefox loads `dist/index.js` over HTTP and converts the round-trip, normalization and -error fixtures and the real payloads — the `commonmark-spec` sort is the Node suite's to check — +error fixtures and the real payloads, and reads the HTML its own `innerHTML` writes back from +`adfToHtml`'s — the `commonmark-spec` sort is the Node suite's to check — which is the browser half of §Any ES2022 engine and the only SpiderMonkey there is — the gate's other engines are two V8s and a JavaScriptCore that is not Safari's. A WebDriver session is what carries a verdict back out, the driver and the page's server sharing one network namespace so each diff --git a/spec/html.md b/spec/html.md index 4b4640a..c0bba0b 100644 --- a/spec/html.md +++ b/spec/html.md @@ -119,14 +119,24 @@ later release writes it as its own element. ## Reading -`htmlToAdf` reads exactly the HTML above, so `htmlToAdf(adfToHtml(doc))` deep-equals `doc`. It -reads a carry element's node from its `data-adf`, and what the element shows, through its closing -tag, must be what `adfToHtml` shows for that node: an edit there would be lost, so it is +`htmlToAdf` reads exactly the HTML above, so `htmlToAdf(adfToHtml(doc))` deep-equals `doc`. It also +reads the HTML above as a browser serializes it (WHATWG HTML §13.3) and as a form post sends it, +anywhere the HTML above has the character, a carry element's `data-adf` and what it shows included: + +| The HTML above writes | Also read | +| --- | --- | +| `"` in text | a raw `"` | +| a no-break space | ` ` | +| `<` and `>` in an attribute value | a raw `<` and `>`, as older browsers write them | +| a line feed | `\r\n`; a refusal's position counts the HTML as handed in | + +It reads a carry element's node from its `data-adf`, and what the element shows, through its +closing tag, must be what `adfToHtml` shows for that node: an edit there would be lost, so it is `unmappable-html`. - Any other HTML is `unmappable-html`: another element, attribute or escape, attributes in another - order, a block on the line of another, a carriage return anywhere, or non-empty HTML not ending - in a line feed. + order, a block on the line of another, a carriage return not followed by a line feed, or + non-empty HTML not ending in a line feed. - A `data-*` attribute that its element does not take, such as `data-level` on `

`, or that holds a value outside its attribute's kind or not written as compact JSON, is `unsupported-node-shape`. - A carry element's `data-adf` holding invalid JSON is `unmappable-html`. JSON that is no ADF node, diff --git a/src/html/escaping.ts b/src/html/escaping.ts index a39db0b..75e91c9 100644 --- a/src/html/escaping.ts +++ b/src/html/escaping.ts @@ -1,17 +1,35 @@ -const entities: ReadonlyMap = new Map([ +export type Spelling = 'attribute' | 'text' + +const escapable = /[&<>"]/g +const escapes: ReadonlyMap = new Map([ + ['"', '"'], + ['&', '&'], + ['<', '<'], + ['>', '>'], +]) +const writtenEntities = [...escapes.values()] +const entity = /&(?:amp|gt|lt|nbsp|quot);/g +const entityValues: ReadonlyMap = new Map([ ['&', '&'], ['>', '>'], ['<', '<'], + [' ', '\u00a0'], ['"', '"'], ]) -const entity = new RegExp([...entities.keys()].join('|'), 'g') -const escapable = new RegExp(`[${[...entities.values()].join('')}]`, 'g') -const escapes: ReadonlyMap = new Map([...entities].map(([spelled, character]) => [character, spelled])) // What HTML text and attribute values cannot hold (WHATWG HTML §13.1.3): any control but a tab, line feed or form feed, a carriage return among them, since a parser reads it as a line feed; a noncharacter; and a lone surrogate, which UTF-8 cannot encode. const unwritable = /(?![\t\n\f ])[\p{Cc}\p{Cs}\p{Noncharacter_Code_Point}]/u const unwritables = new RegExp(unwritable.source, 'gu') -// A character `escapeHtml` escapes, other than the `&` opening an entity, or one HTML cannot hold. -const misspelling = new RegExp(`(?!${entity.source})${escapable.source}|${unwritable.source}`, 'u') +const strayAmpersand = '&(?!(?:amp|gt|lt|nbsp|quot);)' +// What reads wrongly in a value: `&` outside an entity, a character HTML cannot hold, and in text a `<` or `>`. An attribute value may hold `<` and `>` raw, as older browsers write it, and text may hold `"`. +const misspellings: Readonly> = { + attribute: new RegExp(`${strayAmpersand}|${unwritable.source}`, 'u'), + text: new RegExp(`${strayAmpersand}|[<>]|${unwritable.source}`, 'u'), +} +// What `adfToHtml` spells with an entity, a browser (WHATWG HTML §13.3) spells raw in text, or in an attribute; a no-break space it spells ` ` in both. +const browserSpellings: Readonly>> = { + attribute: new Map([['>', '>'], ['<', '<'], ['\u00a0', ' ']]), + text: new Map([['"', '"'], ['\u00a0', ' ']]), +} // The carry's JSON, which spells each character HTML cannot hold as a JSON escape reading back to the same string. export function holdableJson(json: string): string { @@ -22,9 +40,9 @@ export function jsonValue(json: string): string { return escapeHtml(holdableJson(json)) } -// The index of the first character `writableValue` would not write so, -1 where none stands in `html`. -export function misspelledAt(html: string): number { - return html.search(misspelling) +// The index of the first character that reads wrongly in `html`, -1 where none stands in it. +export function misspelledAt(html: string, spelling: Spelling): number { + return html.search(misspellings[spelling]) } // ` name="value"`, `''` where the value is undefined, `undefined` where HTML cannot hold it. @@ -38,9 +56,27 @@ export function shownValue(value: string, lineFeed: string): string { return escapeHtml(value.replace(unwritables, '')).replaceAll('\n', lineFeed) } -// The value an escaped HTML text or attribute stands for, `undefined` where `writableValue` would not write it so. -export function unescapedValue(html: string): string | undefined { - return misspelledAt(html) < 0 ? html.replace(entity, (spelled) => entities.get(spelled) ?? spelled) : undefined +// Where `html` reads as `shown` from `index`: `whole`, and the index after it, or where it first reads otherwise. `shown` is spelled as `adfToHtml` writes it, `html` as a browser may too. +export function readsAsShown(html: string, index: number, shown: string): { at: number; whole: boolean } { + if (html.startsWith(shown, index)) return { at: index + shown.length, whole: true } + let at = index + let spelling: Spelling = 'text' + for (let next = 0; next < shown.length; ) { + const token = writtenEntities.find((spelled) => shown.startsWith(spelled, next)) ?? shown.charAt(next) + // Only a tag holds a raw `<` or `>`, and only its attribute values an escape. + if (token === '<') spelling = 'attribute' + if (token === '>') spelling = 'text' + const read = [token, browserSpellings[spelling].get(token)].find((spelled) => spelled !== undefined && html.startsWith(spelled, at)) + if (read === undefined) return { at, whole: false } + at += read.length + next += token.length + } + return { at, whole: true } +} + +// The value an escaped HTML text or attribute stands for, `undefined` where `misspelledAt` finds a character that reads wrongly. +export function unescapedValue(html: string, spelling: Spelling): string | undefined { + return misspelledAt(html, spelling) < 0 ? html.replace(entity, (spelled) => entityValues.get(spelled) ?? spelled) : undefined } // `undefined` where HTML cannot hold the value, so its node rides the carry element. diff --git a/src/html/parse/carry-element.ts b/src/html/parse/carry-element.ts index 3523bb5..ee9fc17 100644 --- a/src/html/parse/carry-element.ts +++ b/src/html/parse/carry-element.ts @@ -4,7 +4,7 @@ import { carriedNesting, carriedNode, carriedValue } from '../../adf/carried-jso import { carryContent } from '../carry-content.ts' import { carryJsonOffset, readCarryAttributes } from '../carry-attributes.ts' import { foundAt, refused, type Place, type ReadNode, type StartTag } from './source.ts' -import { holdableJson } from '../escaping.ts' +import { holdableJson, readsAsShown } from '../escaping.ts' import { spellsCanonicalJson } from '../../canonical-json.ts' import { success } from '../../result.ts' @@ -18,10 +18,9 @@ export function readCarry(html: string, tag: StartTag, element: 'div' | 'span', const node = heldNode(json, levels, html, jsonAt) if (node.fault !== undefined) return refused(node.fault.code, node.fault.message, html, jsonAt, place) const shown = `${carryContent(node.value, element)}` - if (html.startsWith(shown, tag.end)) return success({ end: tag.end + shown.length, node: node.value }) - let at = tag.end - while (html[at] === shown[at - tag.end]) at += 1 - return refused('unmappable-html', `a carry element shows exactly what adfToHtml writes for its node, then ; to change the node, edit its ADF and write the HTML again: ${foundAt(html, at)}`, html, at, place) + const read = readsAsShown(html, tag.end, shown) + if (read.whole) return success({ end: read.at, node: node.value }) + return refused('unmappable-html', `a carry element shows exactly what adfToHtml writes for its node, then ; to change the node, edit its ADF and write the HTML again: ${foundAt(html, read.at)}`, html, read.at, place) } function heldNode(json: string, levels: number, html: string, jsonAt: number): Read { diff --git a/src/html/parse/html-to-adf.test.ts b/src/html/parse/html-to-adf.test.ts index 4f4038b..488e58e 100644 --- a/src/html/parse/html-to-adf.test.ts +++ b/src/html/parse/html-to-adf.test.ts @@ -52,7 +52,7 @@ const rule = JSON.stringify({ type: 'unknownBlock' }).replaceAll('"', '"') // The carry element of `node`, showing `shown`. function carry(tag: 'div' | 'span', node: unknown, shown: string): string { - return `<${tag} class="adf-carry" data-adf="${JSON.stringify(node).replaceAll('"', '"')}">${shown}` + return `<${tag} class="adf-carry" data-adf="${JSON.stringify(node).replaceAll('&', '&').replaceAll('<', '<').replaceAll('>', '>').replaceAll('"', '"')}">${shown}` } function nested(value: unknown, levels: number, wrap: (inner: unknown) => unknown): unknown { @@ -83,6 +83,53 @@ test('reads escaped text, and keeps tabs, line feeds and form feeds', () => { assert.deepEqual(read(htmlToAdf('

a < b && c > "d" \'e\'\tf\ng\fh &lt;

\n')), document(paragraph(text('a < b && c > "d" \'e\'\tf\ng\fh <')))) }) +test('reads text as a browser\'s innerHTML spells it: a raw ",   for a no-break space, and in an attribute a raw < or >', () => { + assert.deepEqual(read(htmlToAdf('

a "b" "c" d e

\n')), document(paragraph(text('a "b" "c" d e')))) + assert.deepEqual(read(htmlToAdf('
say "a" b
\n')), document({ content: [text('say "a" b')], type: 'codeBlock' })) + const link: AdfMark = { attrs: { href: 'https://x.test/?ac&d', title: 'a b' }, type: 'link' } + assert.deepEqual(read(htmlToAdf('

x

\n')), document(paragraph(text('x', link)))) + assert.deepEqual(read(htmlToAdf('

x

\n')), document(paragraph(text('x', link)))) + assert.deepEqual(read(htmlToAdf('
\n')), document({ attrs: { style: 'a "b' }, type: 'rule' })) +}) + +test('reads a line ending of CRLF as a line feed wherever the dialect has one, and keeps a lone carriage return refused', () => { + const crlf = (html: string): string => html.replaceAll('\n', '\r\n') + assert.deepEqual(read(htmlToAdf(crlf('

a\nb

\n
\n
c\n  d
\n'))), document(paragraph(text('a\nb')), { type: 'rule' }, { content: [text('c\n d')], type: 'codeBlock' })) + assert.deepEqual(read(htmlToAdf(crlf('
\n
    \n
  • Bolt\n

    b

    \n
    • \n
\n
\n'))), document({ content: [{ content: [item(paragraph(text('Bolt')), paragraph(text('b')))], type: 'bulletList' }], type: 'blockquote' })) + assert.deepEqual(read(htmlToAdf(crlf(`${carry('div', panel, '

Check the collation.

')}\n`))), document(panel)) + assert.deepEqual(read(htmlToAdf(crlf(`${documentCarry}\n`))), { type: 'doc', version: 1 }) + assert.deepEqual(read(htmlToAdf('
\n')), document({ attrs: { style: 'a\nb' }, type: 'rule' })) + for (const html of ['

a\r

\n', '

a

\r', '

a

\r\r\n', '

a\r\r\nb

\n']) assert.equal(refusal(htmlToAdf(html)).startsWith('unmappable-html'), true, JSON.stringify(html)) +}) + +test('places a refusal in HTML ending its lines in CRLF at the start of the line it stands on, counted in the HTML handed in', () => { + assert.equal(refusal(htmlToAdf('

a

\r\nx\r\n')), 'unmappable-html at /content/1 line 2 offset 10') + assert.equal(refusal(htmlToAdf('

a

\r\n

b

\r\n

cd

\r\n')), 'unmappable-html at /content/2 line 3 offset 20') + assert.equal(refusal(htmlToAdf('

a

\r\n

b

')), 'unmappable-html at / line 2 offset 10') +}) + +test('reads a carry element showing what a browser spells it as, in any mix with the spelling adfToHtml writes', () => { + const said: AdfNode = { attrs: { id: ids.mention, text: 'Say "hi" now' }, type: 'mention' } + for (const shown of ['Say "hi" now', 'Say "hi" now', 'Say "hi" now', 'Say "hi" now']) { + assert.deepEqual(read(htmlToAdf(`

${carry('span', said, shown)}

\n`)), document(paragraph(said)), shown) + } + const underlined: AdfNode = text('ad', title: 'T" ' }, type: 'link' }) + for (const opener of ['', '']) { + assert.deepEqual(read(htmlToAdf(`

${carry('span', underlined, `${opener}a<b`)}

\n`)), document(paragraph(underlined)), opener) + } + const block: AdfNode = { attrs: { panelType: 'info' }, content: [paragraph(text('A "quote" here'))], type: 'panel' } + assert.deepEqual(read(htmlToAdf(`${carry('div', block, '

A "quote" here

')}\r\n`)), document(block)) +}) + +test('refuses an edit to what a carry element shows even where a browser would spell the same text so', () => { + const said: AdfNode = { attrs: { id: ids.mention, text: 'Say "hi"' }, type: 'mention' } + const underlined: AdfNode = text('a', { type: 'underline' }, { attrs: { href: 'x', title: 'T"' }, type: 'link' }) + assert.equal(refusal(htmlToAdf(`

${carry('span', said, 'Say "hi')}

\n`)), 'unmappable-html at /content/0/content/0 line 1 offset 0') + assert.equal(refusal(htmlToAdf(`

${carry('span', said, 'Say &quot;hi&quot;')}

\n`)), 'unmappable-html at /content/0/content/0 line 1 offset 0') + assert.equal(refusal(htmlToAdf(`

${carry('span', underlined, 'a')}

\n`)), 'unmappable-html at /content/0/content/0 line 1 offset 0') + assert.equal(refusal(htmlToAdf(`

${carry('span', underlined, 'a')}

\n`)), 'read') +}) + test('reads lists, an item\'s first paragraph on its line and its other blocks a line each', () => { assert.deepEqual(read(htmlToAdf('
    \n
  • Bolt
    • \n
  • Nut
    • \n
\n')), document({ content: [item(paragraph(text('Bolt'))), item(paragraph(text('Nut')))], type: 'bulletList' })) const listed: AdfNode = { attrs: { order: 3 }, content: [item(paragraph(text('Bolt')), { content: [item(paragraph(text('M8')))], type: 'bulletList' }), item(paragraph(text('a')), paragraph(text('b')))], type: 'orderedList' } @@ -279,9 +326,9 @@ test('refuses a tag of three million attributes, past what one pattern over them const messages: readonly (readonly [string, string])[] = [ ['

ab

\n', ' as written here is no inline element of the dialect: found "b

\\n"'], - ['

a > b

\n', 'text escapes &, <, > and " as &, <, > and ", and holds no character HTML cannot hold: found "> b

\\n"'], + ['

a > b

\n', 'text escapes & as &, < as < and > as >, and holds no character HTML cannot hold: found "> b

\\n"'], ['

a

\n', ' holds its attributes as name="value", each value escaped as text is, then >: found "b\\">a

\\n"'], - ['

a \u00a0

\n', 'text escapes &, <, > and " as &, <, > and ", and holds no character HTML cannot hold: found " \\u00a0

\\n"'], + ['

a 

\n', 'text escapes & as &, < as < and > as >, and holds no character HTML cannot hold: found " 

\\n"'], ['
a
: found "x\n', ' as written here is no code element of the dialect, which writes or : found "\n
  • a\n', 'an item\'s line ends at
    • , or at a line feed before the item\'s next block, and its text spells < as <: found "\\n"'], diff --git a/src/html/parse/html-to-adf.ts b/src/html/parse/html-to-adf.ts index 2b891b4..ad59120 100644 --- a/src/html/parse/html-to-adf.ts +++ b/src/html/parse/html-to-adf.ts @@ -7,7 +7,7 @@ import { carryContent } from '../carry-content.ts' import { centeredImage, externalMedia } from '../../adf/plain-image.ts' import { containerElements, headingElements, headingLevel, readListStart, type ContainerType } from '../block-elements.ts' import { firstParagraphForm } from '../first-paragraph.ts' -import { foundAt, misread, readStartTag, refused, startTagName, type Place, type ReadNode, type StartTag } from './source.ts' +import { foundAt, misread, positionIn, readStartTag, refused, startTagName, type Place, type ReadNode, type StartTag } from './source.ts' import { largestNesting } from '../../nesting.ts' import { misspelledAt, unescapedValue } from '../escaping.ts' import { misspelledText, readInline } from './inline.ts' @@ -59,8 +59,15 @@ export function htmlToAdf(html: string): Result { ) } +// A form post ends each line in `\r\n`, which reads as the line feed `adfToHtml` writes; a refusal's position still counts the HTML as handed in. function readHtml(html: string): Result { if (typeof html !== 'string') return notAString(html, 'HTML') + const read = readLineFeeds(html.replaceAll('\r\n', '\n')) + if (read.ok || read.error.position === undefined) return read + return { error: { ...read.error, position: positionIn(html, read.error.position) }, ok: false } +} + +function readLineFeeds(html: string): Result { if (html === '') return success({ content: [], type: 'doc', version: 1 }) return readDocumentCarry(html) ?? readBlocks(html) } @@ -191,8 +198,8 @@ function readCodeBlock(html: string, tag: StartTag, place: Place): Result(html: string, index: number, place: Place): Result { - return refused('unmappable-html', `text escapes &, <, > and " as &, <, > and ", and holds no character HTML cannot hold: ${foundAt(html, index)}`, html, index, place) + return refused('unmappable-html', `text escapes & as &, < as < and > as >, and holds no character HTML cannot hold: ${foundAt(html, index)}`, html, index, place) } function markOf(tag: StartTag): AdfMark | undefined { diff --git a/src/html/parse/source.ts b/src/html/parse/source.ts index 5a8f8bf..20d7607 100644 --- a/src/html/parse/source.ts +++ b/src/html/parse/source.ts @@ -45,6 +45,13 @@ export function foundAt(html: string, index: number): string { return `found ${spelled}${index + found.length < html.length ? '…' : ''}` } +// The start of the same line in `html`, where the position was counted after its `\r\n` became line feeds. +export function positionIn(html: string, { line }: SourcePosition): SourcePosition { + let offset = 0 + for (let passed = 1; passed < line; passed += 1) offset = html.indexOf('\n', offset) + 1 + return { line, offset } +} + // At the start of the line `index` stands on. export function refused(code: ConvertErrorCode, message: string, html: string, index: number, place: Place | undefined): Result { return { error: { code, message, path: pathOf(place), position: lineStart(html, index) }, ok: false } @@ -61,14 +68,14 @@ function scanStartTag(html: string, index: number): CutTag | StartTag | undefine const match = attribute.exec(html) if (match === null) return html.startsWith('>', at) ? { attributes, end: at + 1, name, start: index } : { name, stopped: at } const [whole, key = '', spelled = ''] = match - const value = unescapedValue(spelled) - if (value === undefined) return { name, stopped: at + whole.length - 1 - spelled.length + misspelledAt(spelled) } + const value = unescapedValue(spelled, 'attribute') + if (value === undefined) return { name, stopped: at + whole.length - 1 - spelled.length + misspelledAt(spelled, 'attribute') } attributes.push([key, value]) at += whole.length } } -// Only a line feed ends a line: the reader refuses a carriage return wherever it stands. +// Only a line feed ends a line: readHtml has made each \r\n one, and a lone carriage return is refused. function lineStart(html: string, index: number): SourcePosition { let line = 1 let offset = 0 diff --git a/todo.md b/todo.md index 812c74f..df0e772 100644 --- a/todo.md +++ b/todo.md @@ -12,13 +12,12 @@ | ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score | |---|---|---|---|---|---|---|---|---|---|---|---| -| 150 | 0.3.0 | | decision | **Read `adfToHtml`'s HTML in `htmlToAdf` as a browser serializes it too — a raw `"` in text, ` `, `\r\n` line endings — so a page's `innerHTML` or a form post reads back.** | 2 | 2 | 2 | 4 | 4 | 6 by 5 | 5.0 | | 115 | 0.3.0 | | defect | **Build text holding no `border` mark in `losslessMarkdownToAdf`, and have `adfToLosslessMarkdown` carry text holding one.** | 2 | 1 | 2 | 3 | 3 | 4 by 3 | 4.6 | | 91 | 0.3.0 | | | **Spell an emphasis once across the code spans, an emphasis or link once across the hard breaks, dates, emoji, inline cards, mentions and statuses between two nodes it marks, and an emphasis across inline media, in both writers.** | 3 | 2 | 2 | 5 | 5 | 6 by 6 | 3.6 | | 133 | 0.3.0 | | decision | **Write mentions, emoji, dates, statuses and inline cards as semantic HTML in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 4 | 6 by 3 | 3.0 | | 132 | 0.3.0 | | | **Write tables as `` in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 5 | 6 by 4 | 2.4 | -| 73 | 0.3.0 | 91, 115, 132, 133, 150 | | **Announce the package where someone needing an ADF converter already reads: JRACLOUD-77436, the Atlassian developer community and Stack Overflow's ADF-to-markdown questions.** | 1 | 2 | 2 | 3 | 2 | 3 by 8 | 16.9 | -| 72 | 0.3.0 | 91, 115, 132, 133, 150 | decision | **Re-create `gitea.larvit.se/larvit/adf-codec` as a read-only pull mirror of GitHub, its issues and Actions off and its description pointing at GitHub.** | 1 | 2 | 1 | 2 | 1 | 3 by 1 | 2.5 | +| 73 | 0.3.0 | 91, 115, 132, 133 | | **Announce the package where someone needing an ADF converter already reads: JRACLOUD-77436, the Atlassian developer community and Stack Overflow's ADF-to-markdown questions.** | 1 | 2 | 2 | 3 | 2 | 3 by 8 | 16.9 | +| 72 | 0.3.0 | 91, 115, 132, 133 | decision | **Re-create `gitea.larvit.se/larvit/adf-codec` as a read-only pull mirror of GitHub, its issues and Actions off and its description pointing at GitHub.** | 1 | 2 | 1 | 2 | 1 | 3 by 1 | 2.5 | | 153 | 0.4.0 | | question | **Decide whether `src/markdown/` root may import `emit/`, `parse/` or `portable/`, and hold the answer in `.oxlintrc.json` and §The source parts by ADF and format.** | 1 | 2 | 2 | 3 | 2 | code health by 3 | 26.2 | | 156 | 0.4.0 | | | **Stop a leg's container when its script is interrupted, so Ctrl-C ends `ci.sh` or `mutation-tests/run.sh` at once.** | 1 | 2 | 1 | 2 | 1 | code health by 2 | 20.9 | | 128 | 0.4.0 | | | **Pin the portable flavour's block reading and writing with tests: callout titles past one line, a link beside other marks in a title, the nesting limit of callouts and task lists, refusals inside tasks and their paths, and empty or non-paragraph tasks.** | 2 | 1 | 1 | 2 | 3 | code health by 3 | 19.6 | @@ -69,16 +68,6 @@ ## Details -### 150. Read `adfToHtml`'s HTML in `htmlToAdf` as a browser serializes it too — a raw `"` in text, ` `, `\r\n` line endings — so a page's `innerHTML` or a form post reads back. - -`htmlToAdf` refuses all three today. A browser's `innerHTML` writes text's `"` raw and a no-break -space as ` `, and a form post turns every line feed into `\r\n`, so a viewer/editor app -sending its page back gets `unmappable-html`. `docs/decisions.md` §`htmlToAdf` also reads the -dialect as a browser serializes it records the maintainer's yes. Take every difference from WHATWG -HTML §Serializing HTML fragments, not only these three, and check them against a browser's own -`innerHTML`. README §HTML and `spec/html.md` say `\r\n` is refused; update both. The comprehension -panel found `misspelling` in `src/html/escaping.ts` hard to read: make it easier. - ### 115. Build text holding no `border` mark in `losslessMarkdownToAdf`, and have `adfToLosslessMarkdown` carry text holding one. `!adf:border[x]{color="#091e42" size=2}` builds text holding `border`, which