diff --git a/AGENTS.md b/AGENTS.md index f3c0fa3..fa24a39 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -66,7 +66,8 @@ In `docs/decisions.md`: - The CommonMark suite checks three ways - The flavour spec is read as a source - The node tables answer to Atlassian's schema -- Markdown past V8's longest string is refused on every engine +- Output past V8's longest string is refused on every engine +- A URL a browser would run is carried, never written - Nothing recurses unbounded - JSON text is read by `parseJsonText` - No value `JSON.parse` or `structuredClone` builds loops a call diff --git a/CHANGELOG.md b/CHANGELOG.md index 8873ff6..34d3906 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ ## Unreleased +- `adfToHtml` writes ADF as HTML: paragraphs, headings, lists, quotes, code blocks, rules, hard + breaks, external images, and `em`, `strong`, `code` and links as their elements. Every other node + is an element of class `adf-carry` holding the node's JSON in `data-adf` and showing its text. A + link or image whose URL a browser would run, such as `javascript:`, rides that element too. - **Breaking:** `losslessMarkdownToAdf` gives a `date`, `emoji`, `inlineCard`, `mention`, `status` or `mediaInline` only the marks Atlassian's schema lets it hold: `**!adf:mention[@A]{id=…}**` now reads as a plain mention. A link or directive mark around only nodes that cannot take it is diff --git a/README.md b/README.md index a489831..3c7c375 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,9 @@ -# @larvit/adf-codec — ADF to markdown, markdown to ADF +# @larvit/adf-codec — ADF to markdown and HTML, markdown to ADF [](https://www.npmjs.com/package/@larvit/adf-codec) [](https://github.com/larvit/adf-codec/actions/workflows/ci.yml) -Convert Atlassian Document Format (ADF), the rich text of Jira and Confluence, to and from markdown, with a lossless round-trip. +Convert Atlassian Document Format (ADF), the rich text of Jira and Confluence, to and from markdown, with a lossless round-trip, and to HTML. For application developers whose app shows, edits, exports or generates the rich text Atlassian products store as ADF. @@ -19,6 +19,9 @@ products store as ADF. and Obsidian render — GitHub Flavored Markdown's alerts, task lists and pipe tables, Obsidian Flavored Markdown's callouts and highlights. It keeps the text, images by URL and link targets and drops the rest. +- **HTML out:** `adfToHtml` writes paragraphs, headings, lists, quotes, code blocks, images and + links as their HTML elements, and every other node as an element holding its JSON and showing its + text. - **No throws:** every call returns a result; a refusal carries a code from a closed list. - **Runs anywhere:** pure functions with no runtime dependencies, in Node, Deno, Bun and browsers. TypeScript types ship beside the JavaScript. @@ -67,6 +70,18 @@ if (portable.ok) console.log(portable.value) // > Check the collation before importing. ``` +### ADF to HTML + +Full reference: [HTML](#html). + +```ts +import { adfToHtml } from '@larvit/adf-codec' + +const html = adfToHtml(doc) // the panel above +if (html.ok) console.log(html.value) +//
` | +| `bulletList`, `orderedList`, `listItem` | ``, `
`, `
` for an `order` of 3, and `
- `; an item's first paragraph stands bare, `
- Bolt
` | +| `codeBlock` | ``, no `class` without a language | +| `hardBreak` | `
` | +| `heading` | `` to `
` | +| `link` | `` | +| `mediaSingle` holding an external `media`, centered, uncaptioned | `
`, the image inside `` where a link marks it | +| `paragraph` | `
` | +| `rule` | `
` | +| `text` marked `code`, `em`, `strong` | ``, ``, ``, the first mark outermost | +| any other node, or a node holding what its element cannot | `its text`, a `` inline | + +- Text and attribute values are escaped: `&`, `<`, `>`, `"` and a carriage return. +- The carry element shows a mention, status or emoji by its text, a date as `2026-09-13` in UTC, a + card by its URL, an image by its `alt`, else its URL, and an expand by its title. +- A link or image whose URL a browser would run or read as a file, such as `javascript:`, rides + the carry element, so the HTML follows no such URL. + ## The errors Serves [Goals](#goals) 1. An ADF node type this version does not know is not an error: the lossless @@ -475,7 +522,7 @@ Which reader raises which code: [The calls](#the-calls). Codes every direction r ### Emitting -`adfToLosslessMarkdown` and `adfToPortableMarkdown` raise these, and the codes under +`adfToLosslessMarkdown`, `adfToPortableMarkdown` and `adfToHtml` raise these, and the codes under [Either direction](#either-direction): | Code | Fires when | What you can do | @@ -514,7 +561,8 @@ Every call can raise these: - **Writing, in `adfToPortableMarkdown`**: a line or task list the portable writer cannot spell. This is a bug in this library: report it. - **Past the engine's longest string**: - - a writer's markdown would pass 536870888 characters, V8's longest string, on every engine; + - a writer's markdown or HTML would pass 536870888 characters, V8's longest string, on every + engine; - in `adfToPortableMarkdown`, the document's text alone would, edge whitespace included; - in `losslessMarkdownToAdf`, a directive's plain CommonMark would; - in every call, a conversion passes V8's or SpiderMonkey's longest string. @@ -616,6 +664,7 @@ Contract: [`docs/decisions.md`](https://github.com/larvit/adf-codec/blob/main/do | `ParseError` | type | the error a markdown reader returns, its `position` always set | | `Result` | type | every call's return value | | `SourcePosition` | type | an error's place in the markdown | +| `adfToHtml` | function | ADF to HTML | | `adfToLosslessMarkdown` | function | ADF to the lossless flavour | | `adfToPortableMarkdown` | function | ADF to portable markdown, lossy | | `commonMarkToAdf` | function | strict CommonMark to ADF | diff --git a/docs/decisions.md b/docs/decisions.md index 946e4cc..beb75d0 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -398,6 +398,13 @@ themselves. The HTML dialect mirrors the markdown flavour: semantic elements, stable `adf-*` classes, `data-*` for what HTML cannot express, text always escaped. No stylesheet ships. +The spellings, `spec/html.md`, are a writer panel's (2026-10-23): a product manager, an engineer and +a support agent picked, 3 of 3, an image standing alone as `` over `
` and a +bare ``, a list item's first paragraph bare, `
- Bolt
`, over `- `, +`class="language-js"` over `lang-js`, and `
Bolt
` over `
`. For the carry element, seven readers +picked `data-adf` 6 of 7 over `data-adf-node`. Four of them wanted the class `adf-carry` and three +no class; this entry's stable `adf-*` classes settle it. + ## No runtime dependencies 2026-08-23, the maintainer. Goal 8. Valid while ~20 lines of own code, or a vendored table, do each @@ -775,13 +782,13 @@ node takes. Value sets stay documentation, since any value round-trips. What the attribute as a gap, a type as carried — so a re-pin adding either goes red until someone spells it or pins it. -## Markdown past V8's longest string is refused on every engine +## Output past V8's longest string is refused on every engine -2026-10-11, the maintainer. Goals 1 and 6. Valid while V8's longest string, 2^29 − 24 characters, -is the shortest of the engines the library claims. +2026-10-11, HTML 2026-10-23, the maintainer. Goals 1 and 6. Valid while V8's longest string, +2^29 − 24 characters, is the shortest of the engines the library claims. -Every writer, and `losslessMarkdownToAdf`'s `commonMarkSpelling` ask, refuses markdown longer than -2^29 − 24 characters (`longestMarkdown`), so the same document converts alike everywhere. A writer +Every writer, and `losslessMarkdownToAdf`'s `commonMarkSpelling` ask, refuses markdown or HTML +longer than 2^29 − 24 characters (`longestOutput`), so the same document converts alike everywhere. A writer first refuses a document whose text alone passes the bound: the lossless writer spells every character of it, and the portable reduction refuses on the caller's text, edge whitespace and all, since merging that text first could build past the bound. Each block, the writer's result and the @@ -794,6 +801,19 @@ carry's two-space JSON, which indents every value its depth over, stops at the b JavaScriptCore builds a joined block up to its own longest string, 2^31 − 1, before the check. Past that, JavaScriptCore throws `Out of memory`, which is `engine-limit` (§Which code a cause takes). +## A URL a browser would run is carried, never written + +2026-10-23, written with `adfToHtml`, for the maintainer's review. Goals 1 and 6: an export tool's +HTML runs no script. Valid while a consumer may show the HTML in a browser without sanitizing it. + +`adfToHtml` writes no `href` or `src` a browser would run as script or read as a local file: the +link's text or the image rides the carry element, whose `data-adf` no browser follows, so nothing +is lost. The list is cmark's, the CommonMark reference implementation's, which leaves these URLs +out of its HTML by default: `javascript:`, `vbscript:`, `file:`, and `data:` other than a PNG, GIF, +JPEG or WebP image, any case. A browser drops tabs and line breaks anywhere in a URL, and leading +control characters and spaces, before it reads the scheme (the WHATWG URL standard), so the check +reads the URL so too: a tab inside `javascript:` still runs. + ## Nothing recurses unbounded 2026-08-25, the directive form's count 2026-09-18, the maintainer. Goal 1. Valid while an engine's diff --git a/docs/reading-cost.md b/docs/reading-cost.md index 816acfc..95dae7c 100644 --- a/docs/reading-cost.md +++ b/docs/reading-cost.md @@ -478,3 +478,7 @@ Read most (opens and searches): ## Reading cost, chunk fc577244d095, finished 2026-10-23T10:08:26Z 89461 new input tokens before the first commit. + +## Reading cost, chunk f305730665ab, finished 2026-10-23T10:31:43Z + +78044 new input tokens before the first commit. diff --git a/package-tests/consumer.ts b/package-tests/consumer.ts index 25a6f6e..bf389cd 100644 --- a/package-tests/consumer.ts +++ b/package-tests/consumer.ts @@ -1,8 +1,9 @@ -import { adfToLosslessMarkdown, adfToPortableMarkdown, commonMarkToAdf, isAdfDocument, losslessMarkdownToAdf, portableMarkdownToAdf, type AdfDocument, type ConvertErrorCode, type ParseError, type Result } from '@larvit/adf-codec' +import { adfToHtml, adfToLosslessMarkdown, adfToPortableMarkdown, commonMarkToAdf, isAdfDocument, losslessMarkdownToAdf, portableMarkdownToAdf, type AdfDocument, type ConvertErrorCode, type ParseError, type Result } from '@larvit/adf-codec' const document: AdfDocument = { content: [{ content: [{ text: 'x', type: 'text' }], type: 'paragraph' }], type: 'doc', version: 1 } const emitted: Result= adfToLosslessMarkdown(document) +const htmlEmitted: Result = adfToHtml(document) const parsed: Result = losslessMarkdownToAdf('x\n') const portableEmitted: Result = adfToPortableMarkdown(document) const portableParsed: Result = portableMarkdownToAdf('x\n') @@ -11,4 +12,4 @@ const guarded: boolean = isAdfDocument(document) const code: ConvertErrorCode | undefined = emitted.ok ? undefined : emitted.error.code const line: number | undefined = parsed.ok ? undefined : parsed.error.position.line -export const surface = { code, commonMarkParsed, guarded, line, portableEmitted, portableParsed } +export const surface = { code, commonMarkParsed, guarded, htmlEmitted, line, portableEmitted, portableParsed } diff --git a/package.json b/package.json index bf9f18d..26d913f 100644 --- a/package.json +++ b/package.json @@ -1,9 +1,10 @@ { "name": "@larvit/adf-codec", "version": "0.2.0", - "description": "Convert Atlassian Document Format (ADF), the rich text of Jira and Confluence, to and from markdown, with a lossless round-trip.", + "description": "Convert Atlassian Document Format (ADF), the rich text of Jira and Confluence, to and from markdown, with a lossless round-trip, and to HTML.", "keywords": [ "adf", + "adf-to-html", "adf-to-markdown", "atlassian", "atlassian-document-format", @@ -11,6 +12,7 @@ "confluence", "converter", "gfm", + "html", "jira", "lossless", "markdown", diff --git a/spec/html.md b/spec/html.md new file mode 100644 index 0000000..8aaea9e --- /dev/null +++ b/spec/html.md @@ -0,0 +1,78 @@ +# The HTML dialect + +The HTML `adfToHtml` writes: semantic elements for the nodes and marks below, and the carry element +for every other node (`docs/decisions.md` §The HTML dialect). The written form is contract +(`docs/decisions.md` §The formats are API). + +## Blocks + +One block a line: blocks are separated by a line feed, and a document holding any ends with one. A +document whose `content` is an empty array is the empty string. + +| ADF | HTML | +| --- | --- | +| `paragraph` | ` …
` | +| `heading`, `level` 1 to 6 | `…
` to `…
` | +| `blockquote` | ``, a line feed, its blocks, a line feed, `` | +| `bulletList` | ``, a line feed, its items a line each, a line feed, `
` | +| `orderedList` | as `bulletList` with ``; `
` for an `order` of 3, a safe integer; no `start` where the list holds no `order` | +| `listItem` | `
- …
`: a first paragraph holding content bare, its inline content straight after `- `, then each other block after a line feed and a line feed before `
`; with no bare paragraph, `- `, a line feed, its blocks, a line feed, `
` | +| `codeBlock` | ``, the language one token holding no whitespace; no `class` where it holds no `language` | +| `rule` | `…
` | +| `mediaSingle`, centered, holding one external `media` | ``, no `alt` where the media holds none; `` where a link marks it | + +A container holding no `content` key is its element holding nothing: ``, +`
`, ``; so is a paragraph or heading, ``. + +## Inline + +| ADF | HTML | +| --- | --- | +| `text` | its text, escaped | +| `em`, `strong`, `code` | ``, ``, `
` around the text | +| `link`, holding `href` and an optional `title` | `` around the text | +| `hardBreak` | `
` | + +A text node's marks nest in the order its `marks` array lists them, the first outermost: +`manual`. Each text node writes its own elements, so two text nodes a link +marks are two `` elements. + +## Escaping + +Text and attribute values escape `&`, `<`, `>` and `"` as `&`, `<`, `>` and `"`, +and a carriage return as ` `, which an HTML parser would otherwise read as a line feed. +Attribute values are double-quoted. + +## The carry element + +A node the sections above cannot write where it stands is the carry element: `` in block +position, `` inline, of class `adf-carry`, holding the node's JSON in `data-adf` and showing +the node's text. + +```html +\n'), true) +}) + +test('carries a node holding what its element cannot', () => { + const carried = (node: AdfNode): boolean => html(adfToHtml(document(node))).startsWith('Check the collation.+Ask @Mikael
+``` + +- `data-adf` holds the node's JSON compact, keys sorted, as `!adf:carry` does in markdown, its type + included. A document holding no `content` key is the block carry of `{"type":"doc","version":1}`. +- The text shown is every text node's text below the node, and what a node holding no text shows + in its place: a mention's text, else `@` and its id; an emoji's text, else its `shortName`; a + status's text; a date as `2026-09-13` in UTC; a card's URL, else its name, else the URL its data holds; media's `alt`, else an + external one's URL; an extension's text; an expand's title. A line feed parts two blocks and + stands for a hard break. Null characters are dropped. +- A node rides the carry where it holds an attribute, mark or key its element does not write, an + empty `attrs`, `content` or `marks`, or a value its element cannot hold: + - text holding no character, or a null character, which HTML cannot hold; text holding a mark + twice, or a mark with attributes besides a link's `href` and `title`; + - an unmarked text node straight after another, which would read back as one; + - a link or an image whose URL a browser would run or read as a file: a `javascript:`, + `vbscript:` or `file:` URL, or `data:` other than a PNG, GIF, JPEG or WebP image, read after + dropping tabs and line breaks and a leading control character or space (`docs/decisions.md` + §A URL a browser would run is carried, never written); + - a code block holding more than one text node, a marked one, or a language holding whitespace; + - a list holding a node that is no `listItem`, or an item holding attributes or marks: the whole + list rides the carry; + - a `listItem` outside a list, an inline node in block position and a block node inline. +- A block carry's JSON nests at most 500 levels less the depth it stands at, an inline carry's at + most 500, as in markdown. diff --git a/src/adf/carried-json.ts b/src/adf/carried-json.ts new file mode 100644 index 0000000..2b2f22a --- /dev/null +++ b/src/adf/carried-json.ts @@ -0,0 +1,17 @@ +import type { AdfNode } from './document.ts' +import type { JsonSpelling } from '../canonical-json.ts' +import { canonicalJsonPieces } from '../canonical-json.ts' +import { failure, success, type ConvertErrorPath, type Result } from '../result.ts' +import { isJsonValue, nestingDepth } from '../json-value.ts' +import { joinedWithin } from '../length-bound.ts' + +// `spelled` is the node as the carry holds it, `type` dropped where the format names it apart; `levels` the nesting its position leaves. +export function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, path: ConvertErrorPath, levels: number): Result<{ headroom: number; json: string }> { + const headroom = levels - nestingDepth(node) + if (!isJsonValue(spelled) || headroom < 0) { + return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path) + } + // Two-space JSON indents each value by its depth, so a short compact value can spell past the bound: join only up to it. + const json = joinedWithin(canonicalJsonPieces(spelled, spelling)) + return json.ok ? success({ headroom, json: json.value }) : json +} diff --git a/src/adf/external-image.ts b/src/adf/external-image.ts new file mode 100644 index 0000000..0800b1b --- /dev/null +++ b/src/adf/external-image.ts @@ -0,0 +1,29 @@ +import type { AdfNode } from './document.ts' +import { linkTarget, linkTargetMark, type LinkTarget } from './mark-attributes.ts' +import { nodeAttrs, nodeContent, nodeMarks } from './document.ts' +import { serializeCanonicalJson } from '../canonical-json.ts' + +// `alt` undefined where the media holds none: an empty alt builds no `alt`, so a media holding an empty one is no plain image. +export type ExternalImage = { alt: string | undefined; link: LinkTarget | undefined; url: string } + +// The plain image, the one shape each format writes as its own image element: a centered `mediaSingle` holding one external `media`, which a link may mark. +export function centeredImage(content: AdfNode[]): AdfNode { + return { attrs: { layout: 'center' }, content, type: 'mediaSingle' } +} + +export function externalMedia(alt: string, url: string, link?: LinkTarget): AdfNode { + const media: AdfNode = { attrs: alt === '' ? { type: 'external', url } : { alt, type: 'external', url }, type: 'media' } + return link === undefined ? media : { ...media, marks: [linkTargetMark(link)] } +} + +// `undefined` unless the node is exactly the plain image its parts build. +export function plainImage(node: AdfNode): ExternalImage | undefined { + const [media] = nodeContent(node) + if (media === undefined) return undefined + const { alt, url } = nodeAttrs(media) + if (typeof url !== 'string' || (alt !== undefined && typeof alt !== 'string')) return undefined + const [mark] = nodeMarks(media) + const link = mark === undefined ? undefined : linkTarget(nodeAttrs(mark)) + const read = centeredImage([externalMedia(alt ?? '', url, link)]) + return serializeCanonicalJson(node, 'compact') === serializeCanonicalJson(read, 'compact') ? { alt, link, url } : undefined +} diff --git a/src/adf/mark-attributes.ts b/src/adf/mark-attributes.ts index db24e2e..9a53955 100644 --- a/src/adf/mark-attributes.ts +++ b/src/adf/mark-attributes.ts @@ -1,3 +1,4 @@ +import type { AdfAttributes, AdfMark } from './document.ts' import type { AttributeVocabulary } from './attribute-vocabulary.ts' export const blockMarkAttributes = { @@ -26,3 +27,17 @@ export type MarkType = keyof typeof markAttributes export function isMarkType(type: string): type is MarkType { return Object.hasOwn(markAttributes, type) } + +export type LinkTarget = { href: string; title?: string } + +export function linkTargetMark({ href, title }: LinkTarget): AdfMark { + return { attrs: title === undefined ? { href } : { href, title }, type: 'link' } +} + +// `undefined` where `href` is no string, or `title` is there and no string; any other key is the caller's to check. +export function linkTarget(attrs: AdfAttributes): LinkTarget | undefined { + const { href, title } = attrs + if (typeof href !== 'string') return undefined + if (title === undefined) return { href } + return typeof title === 'string' ? { href, title } : undefined +} diff --git a/src/adf/shown-text.ts b/src/adf/shown-text.ts new file mode 100644 index 0000000..143f3f6 --- /dev/null +++ b/src/adf/shown-text.ts @@ -0,0 +1,60 @@ +import type { AdfNode } from './document.ts' +import type { JsonValue } from '../json-value.ts' +import { nodeAttrs } from './document.ts' + +// The text a node shows in place of itself, read from the attribute holding its content; `undefined` where it holds none. +export function shownText(node: AdfNode): string | undefined { + const attrs = nodeAttrs(node) + switch (node.type) { + case 'blockCard': + case 'embedCard': + case 'inlineCard': + return visibleText(attrs['url']) ?? visibleText(cardData(node)['name']) ?? cardUrl(node) + case 'date': + return isoDate(attrs['timestamp']) + case 'emoji': + return visibleText(attrs['text']) ?? stringValue(attrs['shortName']) + case 'expand': + case 'nestedExpand': + case 'status': + return stringValue(node.type === 'status' ? attrs['text'] : attrs['title']) + case 'extension': + case 'inlineExtension': + return visibleText(attrs['text']) + case 'media': + case 'mediaInline': + return visibleText(attrs['alt']) ?? (attrs['type'] === 'external' ? visibleText(attrs['url']) : undefined) + case 'mention': { + const id = attrs['id'] + return visibleText(attrs['text']) ?? (typeof id === 'string' && id !== '' ? `@${id}` : undefined) + } + default: + return undefined + } +} + +export function cardUrl(card: AdfNode): string | undefined { + return visibleText(nodeAttrs(card)['url']) ?? visibleText(cardData(card)['url']) +} + +// `undefined` where the value is no string or holds only whitespace and null characters. +export function visibleText(value: unknown): string | undefined { + return typeof value === 'string' && /[^\s\0]/.test(value) ? value : undefined +} + +function cardData(card: AdfNode): Readonly> { + const data = nodeAttrs(card)['data'] + return typeof data === 'object' && data !== null && !Array.isArray(data) ? data : {} +} + +function isoDate(timestamp: unknown): string | undefined { + const milliseconds = typeof timestamp === 'string' && /^-?\d+$/.test(timestamp) ? Number(timestamp) : Number.NaN + const date = new Date(milliseconds) + if (Number.isNaN(date.getTime())) return undefined + const iso = date.toISOString() + return iso.slice(0, iso.indexOf('T')) +} + +function stringValue(value: unknown): string | undefined { + return typeof value === 'string' ? value : undefined +} diff --git a/src/conformance/adf-property.test.ts b/src/conformance/adf-property.test.ts index dbb0ddc..3b4d56e 100644 --- a/src/conformance/adf-property.test.ts +++ b/src/conformance/adf-property.test.ts @@ -4,6 +4,7 @@ import test from 'node:test' import type { AdfNode } from '../adf/document.ts' import { adfDocument, blockTaskItemChildren, propertyRuns, propertyTimeout } from './property-harness.ts' +import { adfToHtml } from '../html/emit/adf-to-html.ts' import { adfToLosslessMarkdown } from '../markdown/emit/adf-to-markdown.ts' import { adfToPortableMarkdown, reduceToPortable } from '../markdown/portable/adf-to-portable-markdown.ts' import { losslessMarkdownToAdf } from '../markdown/parse/markdown-to-adf.ts' @@ -55,6 +56,16 @@ test('a generated document writes portable markdown refusing only what the guard ) }) +test('a generated document writes HTML', { timeout: propertyTimeout }, () => { + fc.assert( + fc.property(adfDocument, (document) => { + const written = adfToHtml(document) + assert.ok(written.ok, written.ok ? '' : `${written.error.code}: ${written.error.message}`) + }), + propertyRuns(gateRuns), + ) +}) + test('a generated document writes the portable markdown its editor-normal form writes', { timeout: propertyTimeout }, () => { fc.assert( fc.property(adfDocument, (document) => { diff --git a/src/conformance/property-harness.ts b/src/conformance/property-harness.ts index d213b16..2d87d53 100644 --- a/src/conformance/property-harness.ts +++ b/src/conformance/property-harness.ts @@ -10,7 +10,8 @@ import type { LinkDefinition } from '../markdown/commonmark/link-syntax.ts' import { blockArgument } from '../markdown/directive/block.ts' import { blockMarkAttributes, markAttributes } from '../adf/mark-attributes.ts' import { blockNodes } from '../adf/block-nodes.ts' -import { centeredImage, externalMedia } from '../markdown/external-image.ts' +import { centeredImage, externalMedia } from '../adf/external-image.ts' +import { definitionTarget } from '../markdown/mark-spellings.ts' import { directivePrefix } from '../markdown/directive/syntax.ts' import { emptyKeys, mergeAdjacentText, nodeContent } from '../adf/document.ts' import { inlineNodes } from '../adf/inline-nodes.ts' @@ -95,7 +96,7 @@ function occasionallyApart(arbitrary: Arbitrary ')) + } + assert.equal(shown({ attrs: { timestamp: '1700000000000' }, type: 'date' }), '2023-11-14') + assert.equal(shown({ attrs: { shortName: ':smile:', text: '😄' }, type: 'emoji' }), '😄') + assert.equal(shown({ attrs: { id: '5b10a2' }, type: 'mention' }), '@5b10a2') + assert.equal(shown({ attrs: { color: 'green', text: 'Done' }, type: 'status' }), 'Done') + assert.equal(shown({ attrs: { url: 'https://example.com/): Arbitrary ): string { + return result.ok ? result.value : `${result.error.code}: ${result.error.message}` +} + +function paragraph(...content: AdfNode[]): AdfNode { + return content.length === 0 ? { type: 'paragraph' } : { content, type: 'paragraph' } +} + +function text(value: string, ...marks: AdfMark[]): AdfNode { + return marks.length === 0 ? { text: value, type: 'text' } : { marks, text: value, type: 'text' } +} + +function item(...content: AdfNode[]): AdfNode { + return { content, type: 'listItem' } +} + +test('writes paragraphs, headings and rules as their elements, one block a line', () => { + const heading = (level: number): AdfNode => ({ attrs: { level }, content: [text(`Level ${level}`)], type: 'heading' }) + assert.equal(html(adfToHtml(document(heading(1), paragraph(text('Read the manual.')), { type: 'rule' }, heading(6)))), ' Level 1
\nRead the manual.
\n
\nLevel 6
\n') + assert.equal(html(adfToHtml(document(paragraph(), { attrs: { level: 2 }, type: 'heading' }))), '\n\n') +}) + +test('writes the marks CommonMark spells, the first mark outermost', () => { + const link: AdfMark = { attrs: { href: 'https://example.com/a?b=1&c=2', title: 'The "A" page' }, type: 'link' } + const line = paragraph(text('Read '), text('the', { type: 'em' }), text(' '), text('manual', { type: 'strong' }, { type: 'em' }), text(': '), text('npm ci', { type: 'code' }, link), text(' or '), text('this', link)) + assert.equal( + html(adfToHtml(document(line))), + 'Read the manual:
\n', + ) + assert.equal(html(adfToHtml(document(paragraph(text('Line one'), { type: 'hardBreak' }, text('Line two'))))), 'npm cior thisLine one
\n') +}) + +test('escapes text and attribute values, and a carriage return so an HTML parser keeps it', () => { + assert.equal(html(adfToHtml(document(paragraph(text('a < b && c > "d" \'e\'\r\nf'))))), '
Line twoa < b && c > "d" \'e\' \nf
\n') +}) + +test('writes lists tight, an item holding more than one paragraph or a list in elements after the first', () => { + assert.equal(html(adfToHtml(document({ content: [item(paragraph(text('Bolt'))), item(paragraph(text('Nut')))], type: 'bulletList' }))), '\n
\n') + const nested: AdfNode = { attrs: { order: 3 }, content: [item(paragraph(text('Bolt')), { content: [item(paragraph(text('M8')))], type: 'bulletList' }), item(paragraph(text('a')), paragraph(text('b')))], type: 'orderedList' } + assert.equal(html(adfToHtml(document(nested))), '- Bolt
\n- Nut
\n\n
\n') + assert.equal(html(adfToHtml(document({ content: [item({ type: 'rule' }), { type: 'listItem' }], type: 'orderedList' }))), '- Bolt\n
\n\n
\n- M8
\n- a\n
\nb
\n\n
\n') + assert.equal(html(adfToHtml(document({ content: [item(paragraph())], type: 'bulletList' }))), '- \n
\n\n
\n\n
\n') +}) + +test('writes quotes and code blocks, the language a class', () => { + assert.equal(html(adfToHtml(document({ content: [paragraph(text('Quoted'))], type: 'blockquote' }))), '- \n\n
\n\n\n') + assert.equal(html(adfToHtml(document({ attrs: { language: 'javascript' }, content: [text('if (a < b)\n run()')], type: 'codeBlock' }))), 'Quoted
\n\n') + assert.equal(html(adfToHtml(document({ content: [text('plain')], type: 'codeBlock' }, { type: 'codeBlock' }))), 'if (a < b)\n run()\nplain\n') +}) + +test('writes an external image standing alone in a paragraph, a link around it where one marks it', () => { + const image = (media: AdfNode): AdfNode => ({ attrs: { layout: 'center' }, content: [media], type: 'mediaSingle' }) + assert.equal(html(adfToHtml(document(image({ attrs: { alt: 'The moon', type: 'external', url: 'https://example.com/moon.jpg' }, type: 'media' })))), '\n') + const linked = image({ attrs: { type: 'external', url: 'https://example.com/moon.jpg' }, marks: [{ attrs: { href: 'https://example.com/' }, type: 'link' }], type: 'media' }) + assert.equal(html(adfToHtml(document(linked))), '\n') +}) + +test('carries every other node in an adf-carry element holding its JSON and showing its text', () => { + const panel: AdfNode = { attrs: { panelType: 'info' }, content: [paragraph(text('Check the collation.')), paragraph(text('Then import.'))], type: 'panel' } + assert.equal( + html(adfToHtml(document(panel))), + '
Check the collation.\nThen import.\n', + ) + const mention: AdfNode = { attrs: { id: '5b10a2', text: '@Mikael' }, type: 'mention' } + assert.equal(html(adfToHtml(document(paragraph(text('Ask '), mention)))), 'Ask @Mikael
\n') +}) + +test('shows the content a carried node holds in an attribute', () => { + const shown = (node: AdfNode): string => { + const written = html(adfToHtml(document(paragraph(node)))) + return written.slice(written.indexOf('">') + 2, written.indexOf('' }, type: 'inlineCard' }), 'https://example.com/<card>') + assert.equal(shown(text('underlined', { type: 'underline' })), 'underlined') + assert.equal(html(adfToHtml(document({ attrs: { title: 'Details' }, content: [paragraph(text('Inside'))], type: 'expand' }))).endsWith('">Details\nInside { + const carriedSpans = (...content: AdfNode[]): number => html(adfToHtml(document(paragraph(...content)))).split(' { + const linked = (href: string): string => html(adfToHtml(document(paragraph(text('x', { attrs: { href }, type: 'link' }))))) + for (const href of ['javascript:alert(1)', 'JavaScript:alert(1)', ' \u0001java\tscript:alert(1)', 'vbscript:x', 'file:///etc/passwd', 'data:text/html,x', 'data:image/svg+xml,x']) { + assert.equal(linked(href).startsWith('` +} + +function containerBlock(pending: Pending[], { depth, node, path }: Block, tag: string, attributes: string): Result{ + if (node.content === undefined) return success(`<${tag}${attributes}>${tag}>`) + scheduleBlocks(pending, node.content, path, depth + 1, `<${tag}${attributes}>\n`, `\n${tag}>`) + return success(undefined) +} + +// The list's element, `undefined` where the list or an item holds what the elements cannot, so the whole list rides the carry. +function listElement(node: AdfNode): { attributes: string; tag: string } | undefined { + if (node.type === 'bulletList' && holdsOnlyAttributes(node, [])) return listItemsWritable(node) ? { attributes: '', tag: 'ul' } : undefined + if (node.type !== 'orderedList' || !holdsOnlyAttributes(node, ['order']) || !listItemsWritable(node)) return undefined + const order = nodeAttrs(node)['order'] + if (order === undefined) return { attributes: '', tag: 'ol' } + return typeof order === 'number' && Number.isSafeInteger(order) && !Object.is(order, -0) ? { attributes: ` start="${order}"`, tag: 'ol' } : undefined +} + +function listItemsWritable(list: AdfNode): boolean { + return nodeContent(list).every((item) => item.type === 'listItem' && holdsOnlyAttributes(item, [])) +} + +function scheduleList(pending: Pending[], block: Block, { attributes, tag }: { attributes: string; tag: string }): Result { + const { depth, node, path } = block + if (node.content === undefined) return success(`<${tag}${attributes}>${tag}>`) + const items: { opening: string; rest: Pending[] }[] = [] + // The items' blocks sit one level below the list, as a markdown list's do. + for (const [index, item] of node.content.entries()) { + const rest: Pending[] = [] + const opening = writeItem(rest, item, [...path, 'content', index], depth + 1) + if (!opening.ok) return opening + items.push({ opening: `${index === 0 ? '' : '\n'}${opening.value}`, rest }) + } + pending.push(`\n${tag}>`) + for (const { opening, rest } of items.reverse()) { + for (const piece of rest) pending.push(piece) + pending.push(opening) + } + pending.push(`<${tag}${attributes}>\n`) + return success(undefined) +} + +// Returns the item's opening, its first paragraph written bare where it holds content, and pushes the rest; panel pick, docs/decisions.md §The HTML dialect. +function writeItem(pending: Pending[], item: AdfNode, path: ConvertErrorPath, depth: number): Result { + if (depth > largestNesting) return tooDeep(path) + if (item.content === undefined) return success('') + const [first] = item.content + const bare = first !== undefined && first.type === 'paragraph' && holdsOnlyAttributes(first, []) && nodeContent(first).length > 0 + const rest = bare ? item.content.slice(1) : item.content + scheduleBlocks(pending, rest, path, depth, rest.length === 0 ? '' : '\n', rest.length === 0 ? '' : '\n') + if (!bare) return success(' - ') + const inline = inlineHtml(nodeContent(first), [...path, 'content', 0]) + return inline.ok ? success(`
- ${inline.value}`) : inline +} + +function inlineHtml(nodes: readonly AdfNode[], path: ConvertErrorPath): Result
{ + let html = '' + let afterPlainText = false + for (const [index, node] of nodes.entries()) { + // Two unmarked text nodes side by side read back as one, so the second rides the carry. + const written: string | undefined = afterPlainText && isUnmarkedSpellableText(node) ? undefined : tryInlineNode(node) + if (written === undefined) { + const carried = carriedInline(node, [...path, 'content', index]) + if (!carried.ok) return carried + html += carried.value + } else html += written + afterPlainText = written !== undefined && isUnmarkedSpellableText(node) + } + return success(html) +} + +function tryInlineNode(node: AdfNode): string | undefined { + if (node.type === 'hardBreak') return holdsOnlyAttributes(node, []) && nodeContent(node).length === 0 ? '
' : undefined + if (!isSpellableText(node) || !writableText(node)) return undefined + const marks = nodeMarks(node) + const openers: string[] = [] + const closers: string[] = [] + for (const mark of marks) { + if (marks.filter(({ type }) => type === mark.type).length > 1) return undefined + const element = markElement(mark) + if (element === undefined) return undefined + openers.push(element.opener) + closers.unshift(element.closer) + } + return `${openers.join('')}${escapeHtml(node.text)}${closers.join('')}` +} + +function markElement(mark: AdfMark): { closer: string; opener: string } | undefined { + const { type } = mark + if (elementMarks.includes(type)) return mark.attrs === undefined ? { closer: `${type}>`, opener: `<${type}>` } : undefined + if (type !== 'link' || Object.keys(nodeAttrs(mark)).some((key) => key !== 'href' && key !== 'title')) return undefined + const target = linkTarget(nodeAttrs(mark)) + const opener = target === undefined ? undefined : anchorOpener(target) + return opener === undefined ? undefined : { closer: '', opener } +} + +function anchorOpener({ href, title }: LinkTarget): string | undefined { + if (!writableUrl(href) || title?.includes('\0') === true) return undefined + return `` +} + +// HTML holds no null character, and no element holds empty text apart from no text. +function writableText(node: AdfNode): node is AdfNode & { text: string } { + return typeof node.text === 'string' && node.text !== '' && !node.text.includes('\0') +} + +// cmark's dangerous-URL rule: its schemes and its safe image data. A browser drops tabs and line breaks and a leading control or space before it reads the scheme, so the check does too. +function writableUrl(url: string): boolean { + if (url.includes('\0')) return false + const scheme = url.replace(/[\t\n\r]/g, '').replace(/^[\u0001- ]+/, '').toLowerCase() + if (/^(?:file|javascript|vbscript):/.test(scheme)) return false + return !scheme.startsWith('data:') || /^data:image\/(?:gif|jpeg|png|webp)/.test(scheme) +} + +function carriedBlock(node: AdfNode, path: ConvertErrorPath, depth: number): Result{ + const json = carriedJson(node, node, 'compact', path, largestNesting - depth) + return json.ok ? success(carryElement('div', json.value.json, carriedText(node))) : json +} + +function carriedInline(node: AdfNode, path: ConvertErrorPath): Result { + const json = carriedJson(node, node, 'compact', path, largestNesting) + return json.ok ? success(carryElement('span', json.value.json, carriedText(node))) : json +} + +function carryElement(tag: string, json: string, text: string): string { + return `<${tag} class="adf-carry" data-adf="${escapeHtml(json)}">${escapeHtml(text.replaceAll('\0', ''))}${tag}>` +} + +// The text a carried node shows, a line break between its blocks and for each hard break; a reader of the element reads the JSON alone. +function carriedText(node: AdfNode): string { + let text = '' + const pending: AdfNode[] = [node] + for (let next = pending.pop(); next !== undefined; next = pending.pop()) { + if (text !== '' && (next.type === 'hardBreak' || blockNodeModel(next.type) !== undefined)) text += '\n' + text += next.text ?? shownText(next) ?? '' + for (const child of [...nodeContent(next)].reverse()) pending.push(child) + } + return text +} + +// A carriage return is a character reference, since an HTML parser reads a raw one as a line feed. +function escapeHtml(text: string): string { + return text.replaceAll('&', '&').replaceAll('<', '<').replaceAll('>', '>').replaceAll('"', '"').replaceAll('\r', ' ') +} diff --git a/src/index.ts b/src/index.ts index 9fa5272..12b6a1c 100644 --- a/src/index.ts +++ b/src/index.ts @@ -2,6 +2,7 @@ export type { AdfAttributes, AdfDocument, AdfMark, AdfNode } from './adf/document.ts' export type { ConvertError, ConvertErrorCode, ConvertErrorPath, ParseError, Result, SourcePosition } from './result.ts' export type { JsonValue } from './json-value.ts' +export { adfToHtml } from './html/emit/adf-to-html.ts' export { adfToLosslessMarkdown } from './markdown/emit/adf-to-markdown.ts' export { adfToPortableMarkdown } from './markdown/portable/adf-to-portable-markdown.ts' export { commonMarkToAdf, losslessMarkdownToAdf } from './markdown/parse/markdown-to-adf.ts' diff --git a/src/markdown/length-bound.test.ts b/src/length-bound.test.ts similarity index 96% rename from src/markdown/length-bound.test.ts rename to src/length-bound.test.ts index 47eeaf1..524629a 100644 --- a/src/markdown/length-bound.test.ts +++ b/src/length-bound.test.ts @@ -1,9 +1,9 @@ import assert from 'node:assert/strict' import test from 'node:test' -import type { Result } from '../result.ts' +import type { Result } from './result.ts' import { joinedWithin, tooLong, withinEngineLimits, withinLongest } from './length-bound.ts' -import { success } from '../result.ts' +import { success } from './result.ts' function thrown(name: string, message: string): () => Result { return () => { diff --git a/src/markdown/length-bound.ts b/src/length-bound.ts similarity index 76% rename from src/markdown/length-bound.ts rename to src/length-bound.ts index b535593..ab844f6 100644 --- a/src/markdown/length-bound.ts +++ b/src/length-bound.ts @@ -1,9 +1,9 @@ -import { failure, success, type Result } from '../result.ts' +import { failure, success, type Result } from './result.ts' -// V8's longest string, refused on every engine so a document converts alike everywhere (docs/decisions.md §Markdown past V8's longest string is refused on every engine). -export const longestMarkdown = 2 ** 29 - 24 +// V8's longest string, refused on every engine so a document converts alike everywhere (docs/decisions.md §Output past V8's longest string is refused on every engine). +export const longestOutput = 2 ** 29 - 24 -// Measured on Firefox 155, Node 24 and Bun 1.4; SpiderMonkey's `invalid array length` is taken from its js.msg. SpiderMonkey's overflow and V8's string overflow both mean a string past longestMarkdown; the others name an engine's own limit or its memory running short. +// Measured on Firefox 155, Node 24 and Bun 1.4; SpiderMonkey's `invalid array length` is taken from its js.msg. SpiderMonkey's overflow and V8's string overflow both mean a string past longestOutput; the others name an engine's own limit or its memory running short. const pastLongestString = ['InternalError: allocation size overflow', 'RangeError: Invalid string length'] const pastEngineLimit = [ 'RangeError: Array buffer allocation failed', @@ -15,11 +15,11 @@ const pastEngineLimit = [ ] export function tooLong (): Result { - return failure('unsupported-node-shape', `the markdown for this document, or for a directive's plain CommonMark, would be longer than ${longestMarkdown} characters, the longest this library writes`, []) + return failure('unsupported-node-shape', `the markdown or HTML for this document, or a directive's plain CommonMark, would be longer than ${longestOutput} characters, the longest this library writes`, []) } export function withinLongest(text: string): Result { - return text.length > longestMarkdown ? tooLong() : success(text) + return text.length > longestOutput ? tooLong() : success(text) } export function joinedWithin(pieces: Iterable ): Result { @@ -27,7 +27,7 @@ export function joinedWithin(pieces: Iterable ): Result { let length = 0 for (const piece of pieces) { length += piece.length - if (length > longestMarkdown) return tooLong() + if (length > longestOutput) return tooLong() kept.push(piece) } return success(kept.join('')) diff --git a/src/markdown/emit/adf-to-markdown.test.ts b/src/markdown/emit/adf-to-markdown.test.ts index 56cb521..2f7f708 100644 --- a/src/markdown/emit/adf-to-markdown.test.ts +++ b/src/markdown/emit/adf-to-markdown.test.ts @@ -44,7 +44,7 @@ test('refuses a value that is not an ADF document, naming the check it failed', assert.equal(markdown(adfToLosslessMarkdown({ type: 'doc', version: Number.NaN })), 'not-an-adf-document: an ADF document holds a version number: found NaN') }) -const tooLong = 'unsupported-node-shape: the markdown for this document, or for a directive\'s plain CommonMark, would be longer than 536870888 characters, the longest this library writes' +const tooLong = 'unsupported-node-shape: the markdown or HTML for this document, or a directive\'s plain CommonMark, would be longer than 536870888 characters, the longest this library writes' // A code block of 300 lines; two of these together fall some 1000 characters short of V8's longest string, so only what their spelling adds passes it. function nearlyHalfTheBound(): AdfNode { diff --git a/src/markdown/emit/adf-to-markdown.ts b/src/markdown/emit/adf-to-markdown.ts index 19a3119..2671ca7 100644 --- a/src/markdown/emit/adf-to-markdown.ts +++ b/src/markdown/emit/adf-to-markdown.ts @@ -12,7 +12,7 @@ import { flavourClaims } from '../flavour.ts' import { holdsNullCharacter, isBlankLine, isThematicBreak, markerInterruptsParagraph } from '../commonmark/grammar.ts' import { languageSlot, type LanguageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' -import { longestMarkdown, tooLong, withinEngineLimits, withinLongest } from '../length-bound.ts' +import { longestOutput, tooLong, withinEngineLimits, withinLongest } from '../../length-bound.ts' import { spellBlockDirectiveOpener } from '../directive/block-spelling.ts' import { spellDirectiveCloser, spellDirectiveOpener } from '../directive/syntax.ts' import { tryImage } from './image.ts' @@ -55,7 +55,7 @@ export function writeMarkdown(document: AdfDocument, writtenAs: WrittenAs): Resu if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) if (document.content === undefined) return success(`${documentSpelling}\n`) // Every character of text is spelled at least once, so this much text cannot fit, and none of it is spelled. - if (heldTextLength(document.content) > longestMarkdown) return tooLong() + if (heldTextLength(document.content) > longestOutput) return tooLong() const walk = walkBlocks(document.content, [], 0, { ...writtenAs, memo: undefined }) if (!walk.ok) return walk const text = joinBlocks(walk.value.blocks, 'document') diff --git a/src/markdown/emit/image.ts b/src/markdown/emit/image.ts index 515965f..9bc8be8 100644 --- a/src/markdown/emit/image.ts +++ b/src/markdown/emit/image.ts @@ -1,18 +1,12 @@ import type { AdfNode } from '../../adf/document.ts' import type { WrittenFlavour } from '../flavour.ts' -import { centeredImage, externalMedia } from '../external-image.ts' -import { linkDefinition } from '../mark-spellings.ts' -import { nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' -import { serializeCanonicalJson } from '../../canonical-json.ts' +import { plainImage } from '../../adf/external-image.ts' +import { targetDefinition } from '../mark-spellings.ts' import { tryImageLine } from './inline-line.ts' export function tryImage(node: AdfNode, flavour: WrittenFlavour): string | undefined { - const [media] = nodeContent(node) - if (media === undefined) return undefined - const { alt, url } = nodeAttrs(media) - if (typeof url !== 'string' || (alt !== undefined && typeof alt !== 'string')) return undefined - const [mark] = nodeMarks(media) - const link = mark === undefined ? undefined : linkDefinition(nodeAttrs(mark)) - const read = centeredImage([externalMedia(alt ?? '', url, link)]) - return serializeCanonicalJson(node, 'compact') === serializeCanonicalJson(read, 'compact') ? tryImageLine(alt, url, flavour, link) : undefined + const image = plainImage(node) + if (image === undefined) return undefined + const { alt, link, url } = image + return tryImageLine(alt, url, flavour, link === undefined ? undefined : targetDefinition(link)) } diff --git a/src/markdown/external-image.ts b/src/markdown/external-image.ts deleted file mode 100644 index ec74b35..0000000 --- a/src/markdown/external-image.ts +++ /dev/null @@ -1,19 +0,0 @@ -import type { AdfNode } from '../adf/document.ts' -import type { LinkDefinition } from './commonmark/link-syntax.ts' -import { linkMark } from './mark-spellings.ts' - -// spec/flavour.md, The CommonMark image: `` reads to `centeredImage([externalMedia(alt, url)])`, the only shape `adfToLosslessMarkdown` writes as ``. -export function centeredImage(content: AdfNode[]): AdfNode { - return { attrs: { layout: 'center' }, content, type: 'mediaSingle' } -} - -// `` builds no `alt`, so a media holding an empty one takes the directive form. -export function externalMedia(alt: string, url: string, link?: LinkDefinition): AdfNode { - const media: AdfNode = { attrs: alt === '' ? { type: 'external', url } : { alt, type: 'external', url }, type: 'media' } - return link === undefined ? media : { ...media, marks: [linkMark(link)] } -} - -// The ` (image)` after a link for an image inside it, as the readers build it (spec/flavour.md, The CommonMark image) and `adfToPortableMarkdown` writes it; `link` builds the linked word `image`. -export function imageAside(link: (text: string) => AdfNode[]): AdfNode[] { - return [{ text: ' (', type: 'text' }, ...link('image'), { text: ')', type: 'text' }] -} diff --git a/src/markdown/image-aside.ts b/src/markdown/image-aside.ts new file mode 100644 index 0000000..ae1c767 --- /dev/null +++ b/src/markdown/image-aside.ts @@ -0,0 +1,6 @@ +import type { AdfNode } from '../adf/document.ts' + +// The ` (image)` after a link for an image inside it, as the readers build it (spec/flavour.md, The CommonMark image) and `adfToPortableMarkdown` writes it; `link` builds the linked word `image`. +export function imageAside(link: (text: string) => AdfNode[]): AdfNode[] { + return [{ text: ' (', type: 'text' }, ...link('image'), { text: ')', type: 'text' }] +} diff --git a/src/markdown/mark-spellings.ts b/src/markdown/mark-spellings.ts index 7fa53d7..ada6a69 100644 --- a/src/markdown/mark-spellings.ts +++ b/src/markdown/mark-spellings.ts @@ -7,7 +7,7 @@ import { escapeUnbalanced, spellLinkTarget } from './commonmark/link-syntax.ts' import { holdsDirectivePrefix, spellAttributes, spellVocabulary } from './directive/syntax.ts' import { holdsEntityReference } from './commonmark/entity-references.ts' import { isAutolink } from './commonmark/grammar.ts' -import { isMarkType, markAttributes } from '../adf/mark-attributes.ts' +import { isMarkType, linkTarget, linkTargetMark, markAttributes, type LinkTarget } from '../adf/mark-attributes.ts' import { spellEmptyKeys } from './directive/empty-keys.ts' import { vocabularyPairs } from '../adf/attribute-vocabulary.ts' @@ -37,16 +37,21 @@ export function markSpelling(type: string): MarkSpelling | undefined { return { attributes, kind: spelling.kind } } -export function linkMark({ destination, title }: LinkDefinition): AdfMark { - return { attrs: title === undefined ? { href: destination } : { href: destination, title }, type: 'link' } +export function linkMark(definition: LinkDefinition): AdfMark { + return linkTargetMark(definitionTarget(definition)) +} + +export function definitionTarget({ destination, title }: LinkDefinition): LinkTarget { + return title === undefined ? { href: destination } : { href: destination, title } } export function linkDefinition(attrs: AdfAttributes): LinkDefinition | undefined { - const destination = linkHref(attrs) - const title = attrs['title'] - if (destination === undefined) return undefined - if (title === undefined) return { destination } - return typeof title === 'string' ? { destination, title } : undefined + const target = linkTarget(attrs) + return target === undefined ? undefined : targetDefinition(target) +} + +export function targetDefinition({ href, title }: LinkTarget): LinkDefinition { + return title === undefined ? { destination: href } : { destination: href, title } } export function linkHref(attrs: AdfAttributes): string | undefined { diff --git a/src/markdown/opaque-carry.ts b/src/markdown/opaque-carry.ts index 21487d6..9757388 100644 --- a/src/markdown/opaque-carry.ts +++ b/src/markdown/opaque-carry.ts @@ -2,14 +2,14 @@ import type { AdfNode } from '../adf/document.ts' import type { DirectiveSpan } from './directive/syntax.ts' import type { JsonSpelling } from '../canonical-json.ts' import type { JsonValue } from '../json-value.ts' -import { canonicalJsonPieces, spellsCanonicalJson } from '../canonical-json.ts' +import { carriedJson } from '../adf/carried-json.ts' +import { spellsCanonicalJson } from '../canonical-json.ts' import { directiveEscape, malformedDirective, readSoleStringAttribute, spellAttributes, spellDirectiveOpener, spellInlineLeafDirective, spellStringAttribute, unsupportedNodeShape } from './directive/syntax.ts' -import { failure, success, type ConvertErrorPath, type Read, type Result } from '../result.ts' +import { success, type ConvertErrorPath, type Read, type Result } from '../result.ts' import { fencedCodeBlock } from './commonmark/backtick-runs.ts' import { infoStringCarries } from './commonmark/grammar.ts' import { isAdfNode } from '../adf/document.ts' -import { isJsonValue, nestingDepth, overNested } from '../json-value.ts' -import { joinedWithin } from './length-bound.ts' +import { overNested } from '../json-value.ts' import { largestNesting } from '../nesting.ts' import { parseJsonText } from '../json-text.ts' @@ -54,16 +54,6 @@ export function readCarriedInline(span: DirectiveSpan): Read | undefine return readCarriedJson(spelled.value, 'compact', largestNesting) } -function carriedJson(node: AdfNode, spelled: object, spelling: JsonSpelling, path: ConvertErrorPath, levels: number): Result<{ headroom: number; json: string }> { - const headroom = levels - nestingDepth(node) - if (!isJsonValue(spelled) || headroom < 0) { - return failure('unsupported-nesting-depth', `a carried node's JSON nests deeper than the ${levels} levels its position leaves`, path) - } - // Two-space JSON indents each value by its depth, so a short compact value can spell past the bound: join only up to it. - const json = joinedWithin(canonicalJsonPieces(spelled, spelling)) - return json.ok ? success({ headroom, json: json.value }) : json -} - // `type` is the one the fence's info string names, `undefined` for the inline carry. function readCarriedJson(raw: string, spelling: JsonSpelling, levels: number, type?: string): Read { const wayOut = type === undefined ? directiveEscape : fenceEscape(type) diff --git a/src/markdown/parse/inline-content.ts b/src/markdown/parse/inline-content.ts index d138545..ed71fd6 100644 --- a/src/markdown/parse/inline-content.ts +++ b/src/markdown/parse/inline-content.ts @@ -5,8 +5,9 @@ import type { EmphasisPairing } from '../commonmark/emphasis-matching.ts' import type { InlineSource, InlineToken } from '../inline-tokens.ts' import type { LineContainer } from '../line-container.ts' import type { LinkDefinition } from '../commonmark/link-syntax.ts' -import { centeredImage, externalMedia, imageAside } from '../external-image.ts' -import { commonMarkLink, linkDefinition, linkHref, linkMark } from '../mark-spellings.ts' +import { centeredImage, externalMedia } from '../../adf/external-image.ts' +import { imageAside } from '../image-aside.ts' +import { commonMarkLink, definitionTarget, linkDefinition, linkHref, linkMark } from '../mark-spellings.ts' import { decodeTextEscapes, trimTrailingSpace } from '../commonmark/grammar.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { flavourClaims } from '../flavour.ts' @@ -318,7 +319,7 @@ function imageLink(definition: LinkDefinition): ImageLink { function mediaSingle({ alt, definition, link }: Image): AdfNode { const { title } = definition const caption: AdfNode[] = title === undefined || title === '' ? [] : [{ content: [{ text: title, type: 'text' }], type: 'caption' }] - return centeredImage([externalMedia(alt, definition.destination, link), ...caption]) + return centeredImage([externalMedia(alt, definition.destination, link === undefined ? undefined : definitionTarget(link)), ...caption]) } // ADF holds no empty text node and a hard break takes no link, so the destination text carries the link ahead of the text's hard breaks; an empty destination leaves the breaks alone. diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index 094c411..c152065 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -364,7 +364,7 @@ test('refuses a blockquote directive whose CommonMark spelling would pass V8\'s const fence = `\`\`\`\n${`${'a'.repeat(894_782)}\n`.repeat(300)}\`\`\`\n` assert.equal( content(losslessMarkdownToAdf(`!adf:blockquote\n${fence}\n${fence}!adf:/blockquote\n`)), - 'unsupported-node-shape: the markdown for this document, or for a directive\'s plain CommonMark, would be longer than 536870888 characters, the longest this library writes', + 'unsupported-node-shape: the markdown or HTML for this document, or a directive\'s plain CommonMark, would be longer than 536870888 characters, the longest this library writes', ) }) diff --git a/src/markdown/parse/markdown-to-adf.ts b/src/markdown/parse/markdown-to-adf.ts index 6ccb158..6ba1b53 100644 --- a/src/markdown/parse/markdown-to-adf.ts +++ b/src/markdown/parse/markdown-to-adf.ts @@ -17,7 +17,7 @@ import { parseInlineContent } from './inline-content.ts' import { readBlockDirectiveNode } from '../directive/node-reading.ts' import { spellStringAttribute, unsupportedNodeShape } from '../directive/syntax.ts' import { spellsEmpty } from '../directive/empty-keys.ts' -import { withinEngineLimits } from '../length-bound.ts' +import { withinEngineLimits } from '../../length-bound.ts' // A flavour's own reading of a quote, and of a bullet list into one or more blocks; `path` is the container's, `index` the list's place among the container's blocks. export type FlavourBlockReaders = { diff --git a/src/markdown/portable/adf-to-portable-markdown.ts b/src/markdown/portable/adf-to-portable-markdown.ts index b9f6d39..c4a255b 100644 --- a/src/markdown/portable/adf-to-portable-markdown.ts +++ b/src/markdown/portable/adf-to-portable-markdown.ts @@ -1,7 +1,7 @@ import type { AdfDocument, AdfNode } from '../../adf/document.ts' import { adfDocumentFault, heldTextLength, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { blockNodeModel, isTaskItem } from '../../adf/block-nodes.ts' -import { centeredImage, externalMedia } from '../external-image.ts' +import { centeredImage, externalMedia } from '../../adf/external-image.ts' import { commonMarkSpelling, largestListMarker, writeMarkdown, type SpellingMemo } from '../emit/adf-to-markdown.ts' import { failure, faulted, success, type ConvertErrorPath, type Result } from '../../result.ts' import { flavourClaims } from '../flavour.ts' @@ -9,8 +9,8 @@ import { inlineLeaves, isBlockNodeType, oneLine, portableLink, reduceInline, wri import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { languageSlot } from '../code-language.ts' import { largestNesting } from '../../nesting.ts' -import { linkDefinition } from '../mark-spellings.ts' -import { longestMarkdown, tooLong, withinEngineLimits } from '../length-bound.ts' +import { linkTarget } from '../../adf/mark-attributes.ts' +import { longestOutput, tooLong, withinEngineLimits } from '../../length-bound.ts' import { portableWriting } from './block-spelling.ts' import { splitTaskList } from './task-list.ts' import { taskMarker } from './conventions.ts' @@ -63,7 +63,7 @@ export function reduceToPortable(document: AdfDocument): Result { if (fault !== undefined) return faulted(fault, []) if (document.version !== 1) return failure('unsupported-document-version', `no markdown spelling carries ADF version ${document.version}`, []) // Measured on the caller's text, edge whitespace and all: `toEditorNormal`'s merge could build past the bound. - if (heldTextLength(nodeContent(document)) > longestMarkdown) return tooLong() + if (heldTextLength(nodeContent(document)) > longestOutput) return tooLong() // Two documents the editor holds equal write the same portable markdown (docs/decisions.md §Equality is deep). const blocks = reduceBlocks(nodeContent(toEditorNormal(document)), { depth: 0, memo: new Map(), path: [] }) if (!blocks.ok) { @@ -367,5 +367,5 @@ function reduceMedia(media: AdfNode, reduction: Reduction): Result { const alt = typeof held === 'string' ? oneLine(held).trim() : '' const mark = nodeMarks(media).find(({ type }) => type === 'link') const link = mark === undefined ? undefined : portableLink(mark, 'paragraph') - return success([centeredImage([externalMedia(alt, writableHref(url), link === undefined ? undefined : linkDefinition(nodeAttrs(link)))])]) + return success([centeredImage([externalMedia(alt, writableHref(url), link === undefined ? undefined : linkTarget(nodeAttrs(link)))])]) } diff --git a/src/markdown/portable/inline-reduction.ts b/src/markdown/portable/inline-reduction.ts index f3d29fb..6a6058e 100644 --- a/src/markdown/portable/inline-reduction.ts +++ b/src/markdown/portable/inline-reduction.ts @@ -2,9 +2,10 @@ import type { AdfAttributes, AdfMark, AdfNode } from '../../adf/document.ts' import type { LineContainer } from '../line-container.ts' import type { MarkRun } from '../emit/line-escaping.ts' import { blockNodeModel } from '../../adf/block-nodes.ts' +import { cardUrl, shownText, visibleText } from '../../adf/shown-text.ts' import { codeTextMarkTypes } from '../../adf/inline-nodes.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' -import { imageAside } from '../external-image.ts' +import { imageAside } from '../image-aside.ts' import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from './editor-normal.ts' import { largestNesting } from '../../nesting.ts' import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' @@ -47,50 +48,38 @@ function nodeLeaves(node: AdfNode, container: LineContainer, path: ConvertErrorP const attrs = nodeAttrs(node) if (node.type === 'text') return success(textLeaves(node.text, marks, container)) if (node.type === 'hardBreak') return success([lineBreak(container)]) - if (node.type === 'date') return success(textLeaves(isoDate(attrs['timestamp']), marks, container)) - if (node.type === 'emoji') return success(textLeaves(nonEmpty(attrs['text']) ?? attrs['shortName'], marks, container)) + if (['date', 'emoji', 'mention', 'status'].includes(node.type)) return success(textLeaves(shownText(node), marks, container)) if (node.type === 'placeholder') return success([]) - if (node.type === 'mention') return success(textLeaves(nonEmpty(attrs['text']) ?? idMention(attrs['id']), marks, container)) - if (node.type === 'status') return success(textLeaves(attrs['text'], marks, container)) - if (['extension', 'inlineExtension'].includes(node.type)) return success(textLeaves(nonEmpty(attrs['text']), marks, container, noteName(attrs['extensionKey']) ?? 'extension')) + if (['extension', 'inlineExtension'].includes(node.type)) return success(textLeaves(shownText(node), marks, container, noteName(attrs['extensionKey']) ?? 'extension')) if (node.type === 'syncBlock') return success(noteLeaves('synced block')) if (['media', 'mediaInline'].includes(node.type)) return success(mediaLeaves(attrs, marks, container)) - if (['blockCard', 'embedCard', 'inlineCard'].includes(node.type)) return success(cardLeaves(attrs, marks, container)) - const own = textLeaves(node.text ?? (['expand', 'nestedExpand'].includes(node.type) ? attrs['title'] : undefined), marks, container) + if (['blockCard', 'embedCard', 'inlineCard'].includes(node.type)) return success(cardLeaves(node, marks, container)) + const own = textLeaves(node.text ?? shownText(node), marks, container) const held = inlineLeaves(nodeContent(node), container, path, depth + 1) if (!held.ok) return held return success(own.length > 0 && held.value.length > 0 ? [...own, textLeaf(' ', []), ...held.value] : [...own, ...held.value]) } -function cardLeaves(attrs: Readonly , marks: readonly AdfMark[], container: LineContainer): AdfNode[] { - const data = attrs['data'] - const held = typeof data === 'object' && data !== null && !Array.isArray(data) ? data : {} - const url = nonEmpty(attrs['url']) - const heldUrl = nonEmpty(held['url']) - const name = nonEmpty(held['name']) - const href = url ?? heldUrl - if (href === undefined) return textLeaves(name, marks, container, 'link card') - return linkedLeaves(url ?? name ?? href, href, marks, container) +function cardLeaves(card: AdfNode, marks: readonly AdfMark[], container: LineContainer): AdfNode[] { + const href = cardUrl(card) + if (href === undefined) return textLeaves(shownText(card), marks, container, 'link card') + return linkedLeaves(shownText(card) ?? href, href, marks, container) } function linkedLeaves(text: string, href: string, marks: readonly AdfMark[], container: LineContainer): AdfNode[] { return textLeaves(text, [...marks.filter((mark) => mark.type !== 'link'), { attrs: { href }, type: 'link' }], container) } -function idMention(id: unknown): string | undefined { - return typeof id === 'string' && id !== '' ? `@${id}` : undefined -} - // An image standing inline is a link to it: the reader builds no node for an inline image, reading it as its alt text linked, and inside a link as the link's text, then (image). function mediaLeaves(attrs: Readonly , marks: readonly AdfMark[], container: LineContainer): AdfNode[] { - const alt = nonEmpty(attrs['alt']) - const url = nonEmpty(attrs['url']) + const alt = visibleText(attrs['alt']) + const url = visibleText(attrs['url']) if (attrs['type'] !== 'external' || url === undefined) return textLeaves(alt, marks, container, 'image') const link = marks.find((mark) => mark.type === 'link') const href = link === undefined ? undefined : nodeAttrs(link)['href'] if (typeof href !== 'string') return linkedLeaves(alt ?? url, url, marks, container) if (href === url) return textLeaves(alt ?? url, marks, container) - return [...textLeaves(alt ?? nonEmpty(href) ?? url, marks, container), ...imageAside((text) => linkedLeaves(text, url, marks, container))] + return [...textLeaves(alt ?? visibleText(href) ?? url, marks, container), ...imageAside((text) => linkedLeaves(text, url, marks, container))] } function noteLeaves(name: string): AdfNode[] { @@ -98,25 +87,13 @@ function noteLeaves(name: string): AdfNode[] { } function noteName(value: unknown): string | undefined { - return nonEmpty(value) === undefined ? undefined : oneLine(String(value)).trim() + return visibleText(value) === undefined ? undefined : oneLine(String(value)).trim() } export function oneLine(text: string): string { return text.replace(/[\r\u0000]/g, '').replace(/\n/g, ' ') } -function nonEmpty(value: unknown): string | undefined { - return typeof value === 'string' && oneLine(value).trim() !== '' ? value : undefined -} - -function isoDate(timestamp: unknown): string | undefined { - const milliseconds = typeof timestamp === 'string' && /^-?\d+$/.test(timestamp) ? Number(timestamp) : Number.NaN - const date = new Date(milliseconds) - if (Number.isNaN(date.getTime())) return undefined - const iso = date.toISOString() - return iso.slice(0, iso.indexOf('T')) -} - function lineBreak(container: LineContainer): AdfNode { return container === 'paragraph' ? { type: 'hardBreak' } : textLeaf(' ', []) } diff --git a/src/markdown/portable/portable-markdown-to-adf.ts b/src/markdown/portable/portable-markdown-to-adf.ts index 1955ad5..7ebdd3a 100644 --- a/src/markdown/portable/portable-markdown-to-adf.ts +++ b/src/markdown/portable/portable-markdown-to-adf.ts @@ -9,7 +9,7 @@ import { mintTaskIds } from './task-ids.ts' import { parseInlineContent } from '../parse/inline-content.ts' import { pathTo, splitTaskList } from './task-list.ts' import { positioned, success } from '../../result.ts' -import { withinEngineLimits } from '../length-bound.ts' +import { withinEngineLimits } from '../../length-bound.ts' type Paragraph = Extract diff --git a/todo.md b/todo.md index 0db4fbf..758ca3f 100644 --- a/todo.md +++ b/todo.md @@ -6,14 +6,13 @@ `Bar = 0.1` -`Next ID = 141` +`Next ID = 142` ## Items | ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score | |---|---|---|---|---|---|---|---|---|---|---|---| -| 130 | 0.3.0 | | | **Ship `adfToHtml`, writing paragraphs, headings, lists, quotes, code blocks, rules, hard breaks, external images and the marks CommonMark spells as semantic HTML, and every other node as a carry element showing its text.** | 3 | 2 | 2 | 5 | 6 | 3 by 6 | 2.1 | -| 131 | 0.3.0 | 130 | | **Ship `htmlToAdf`, reading back exactly what `adfToHtml` writes and refusing any other HTML as `unmappable-html`.** | 4 | 1 | 2 | 5 | 7 | 3 by 3 | 0.6 | +| 131 | 0.3.0 | | | **Ship `htmlToAdf`, reading back exactly what `adfToHtml` writes and refusing any other HTML as `unmappable-html`.** | 4 | 1 | 2 | 5 | 7 | 3 by 3 | 0.6 | | 133 | 0.3.0 | 131 | | **Write mentions, emoji, dates, statuses and inline cards as semantic HTML in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 4 | 6 by 3 | 3.0 | | 132 | 0.3.0 | 131 | | **Write tables as ` ` in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 5 | 6 by 4 | 2.4 | | 128 | 0.4.0 | | | **Pin the portable flavour's block reading and writing with tests: callout titles past one line, a link beside other marks in a title, the nesting limit of callouts and task lists, refusals inside tasks and their paths, and empty or non-paragraph tasks.** | 2 | 1 | 1 | 2 | 3 | code health by 3 | 19.6 | @@ -31,6 +30,7 @@ | 112 | 0.4.0 | | | **Pad a GFM pipe table's short rows with empty cells, read its body rows holding no pipe, and read a one-column table without leading pipes in `portableMarkdownToAdf`.** | 2 | 1 | 2 | 3 | 3 | 4 by 4 | 6.2 | | 80 | 0.4.0 | | | **Name the node's path and the attribute key when `not-an-adf-document` refuses a node inside the document.** | 2 | 1 | 2 | 3 | 3 | 6 by 4 | 6.0 | | 81 | 0.4.0 | | | **Fail the gate unless the README's error tables list exactly the codes `ConvertErrorCode` holds.** | 1 | 1 | 1 | 1 | 2 | 3 by 2 | 4.2 | +| 141 | 0.4.0 | 131 | | **Write `strike` as HTML in `adfToHtml` and read it back in `htmlToAdf`, in place of the carry element.** | 1 | 1 | 1 | 1 | 2 | 6 by 2 | 4.0 | | 135 | 0.4.0 | 131 | | **Write media, media groups and captions as semantic HTML in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 4 | 6 by 2 | 2.0 | | 136 | 0.4.0 | 131 | | **Write underline, text and background colours, subscript, superscript, alignment and indentation as HTML in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 4 | 6 by 2 | 2.0 | | 134 | 0.4.0 | 131 | | **Write panels, expands, task lists and decision lists as semantic HTML in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 5 | 6 by 3 | 1.8 | @@ -62,19 +62,6 @@ ## Details -### 130. Ship `adfToHtml`, writing paragraphs, headings, lists, quotes, code blocks, rules, hard breaks, external images and the marks CommonMark spells as semantic HTML, and every other node as a carry element showing its text. - -The first, limited HTML: the dialect (`docs/decisions.md` §The HTML dialect) specified for this set, -and a writer panel picking its spellings. A code block is `
`, its language a class. Every -other node is a carry element holding the node's JSON in a `data-*` attribute and showing the node's -text, so an HTML reader loses no content (goal 7). The README documents HTML as it documents -markdown, its comparison table names HTML, and its tagline and `package.json`'s description and -keywords name HTML. - -Whatever `html/` shares with the carry (`opaque-carry.ts`) or the mark spellings -(`mark-spellings.ts`) splits: the ADF question moves to `adf/`, and each format keeps its own -spelling. - ### 131. Ship `htmlToAdf`, reading back exactly what `adfToHtml` writes and refusing any other HTML as `unmappable-html`. A hand-written HTML tokenizer, linear in the input, its nesting bounded (`docs/decisions.md` @@ -227,6 +214,11 @@ document's whole tree. Give the node's path, and name its type and the attribute leaves the README's tables a false contract under a green gate. Assert that the tables list exactly the union. Found by the technical-principles audit of item 78, 2026-10-05. +### 141. Write `strike` as HTML in `adfToHtml` and read it back in `htmlToAdf`, in place of the carry element. + +GFM spells `strike`, and no other HTML item writes it: item 130 wrote the marks CommonMark spells. +A writer panel picks `` or ``. + ### 135. Write media, media groups and captions as semantic HTML in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element. A writer panel picks the spellings; an external image already has one from item 130.