diff --git a/CHANGELOG.md b/CHANGELOG.md index da2047d..b424638 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,10 +19,10 @@ other than plain text nodes rides the block carry, where it was refused. - **Breaking:** `unspellable-character` and `unspellable-line-start` leave `ConvertErrorCode`, and `adfToMarkdown` no longer refuses `unspellable-whitespace`. A carriage return in text is spelled - ` ` and a null character `!adf:text{text="\u0000"}`; a code block holding either, a text node - holding no text or holding `content`, a paragraph line opening with a code span whose backticks - read back as a fence, and an `emoji`, `mention` or `status` whose `text` holds a line ending or a - null character ride the carry. See `MIGRATION.md`. + ` ` and a null character `!adf:text{text="\u0000"}`; a paragraph line opening with a code span + is spelled as one, its backticks opening no fence; a code block holding either character, a text + node holding no text or holding `content`, and an `emoji`, `mention` or `status` whose `text` holds + a line ending or a null character ride the carry. See `MIGRATION.md`. - Fix a link `title` holding a null character: it is written as `!adf:link[text]{attrs}`, where it read back as U+FFFD. - **Breaking:** `unspellable-link` leaves `ConvertErrorCode`; a link whose `href` or `title` no diff --git a/MIGRATION.md b/MIGRATION.md index ccdb7d6..2df4a66 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -71,7 +71,8 @@ a `switch` naming one stops compiling, and the document it named converts. | a carriage return in text, on emit | `unspellable-character` | spells ` ` | | a null character in text, on emit | `unspellable-character` | spells `!adf:text{text="\u0000"}` | | a carriage return or null character in a code block or code span, on emit | `unspellable-character` | rides the carry | -| a paragraph line opening with a code span whose backticks read back as a fence, on emit | `unspellable-line-start` | rides the inline carry | +| a paragraph line opening with a code span of three or more backticks, on emit | `unspellable-line-start` | spells the code span | +| a block directive whose CommonMark spelling holds one of the above, on parse | `unspellable-character`, `unspellable-line-start` | `unsupported-node-shape`; write the CommonMark spelling | | an `emoji`, `mention` or `status` whose `text` holds a line ending or a null character, on emit | `unspellable-whitespace`, `unspellable-character` | rides the inline carry | | a text node holding no text, or holding `content`, on emit | `unsupported-node-shape` | rides the carry | | a `codeBlock` holding other than plain text nodes, on emit | `unsupported-node-shape` | rides the block carry | diff --git a/README.md b/README.md index 3a9e3ac..6be82f8 100644 --- a/README.md +++ b/README.md @@ -224,8 +224,9 @@ Serves Goals 1, 3 and 4. text, which the spec requires and the reference itself breaks, nesting one `` in the other. The first three are pinned `pending` in `corpus/commonmark-spec/exceptions.json`; the suite holds no example of the fourth. -- Not every document converts back: `adfToMarkdown` refuses with `unsupported-nesting-depth` or - `unsupported-node-shape` as the tables above say. Show the refusal and keep the document +- Not every document converts back: `adfToMarkdown` refuses a document nesting past 500 levels, a + node carrying one mark type twice, a block or a non-text inline node holding `text`, and a leaf + block or a non-text inline node holding `content`. Show the refusal and keep the document read-only; saving markdown you could not produce is the loss the round-trip exists to stop. - The pipe table narrows GFM's twice: every row opens with a pipe, so GFM's bare form is an error result rather than the prose it reads as, and an alignment colon in the delimiter row is an diff --git a/corpus/README.md b/corpus/README.md index 3629e43..1c23a45 100644 --- a/corpus/README.md +++ b/corpus/README.md @@ -16,7 +16,7 @@ One directory per contract kind: is the suite; `refusals.json` pins each refusing example to its error `code`; `exceptions.json` pins each known divergence by `check`, `example`, `kind` and the exact `divergence`, with a `reason`. `kind` is `mark-model` (the permanent count divergence from ADF's mark-per-text-node - model), `unspellable` (parses but the flavour has no spelling) or `pending` (a parser gap). + model) or `pending` (a parser gap). JSON is two-space indent, keys sorted, and a document read back must deep-equal the fixture's (`docs/decisions.md` §Equality is deep). `spec.json` is the vendored, upstream machine-readable diff --git a/corpus/round-trip/block-nodes/code-block-carriage-return.json b/corpus/round-trip/block-nodes/code-block-carriage-return.json new file mode 100644 index 0000000..cd33f9b --- /dev/null +++ b/corpus/round-trip/block-nodes/code-block-carriage-return.json @@ -0,0 +1,15 @@ +{ + "content": [ + { + "content": [ + { + "text": "a\r\nb", + "type": "text" + } + ], + "type": "codeBlock" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/block-nodes/code-block-carriage-return.md b/corpus/round-trip/block-nodes/code-block-carriage-return.md new file mode 100644 index 0000000..0fd7d41 --- /dev/null +++ b/corpus/round-trip/block-nodes/code-block-carriage-return.md @@ -0,0 +1,10 @@ +```adf:codeBlock +{ + "content": [ + { + "text": "a\r\nb", + "type": "text" + } + ] +} +``` diff --git a/corpus/round-trip/commonmark-subset/code-span-line-start.json b/corpus/round-trip/commonmark-subset/code-span-line-start.json new file mode 100644 index 0000000..9233ba5 --- /dev/null +++ b/corpus/round-trip/commonmark-subset/code-span-line-start.json @@ -0,0 +1,20 @@ +{ + "content": [ + { + "content": [ + { + "marks": [ + { + "type": "code" + } + ], + "text": "``", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/commonmark-subset/code-span-line-start.md b/corpus/round-trip/commonmark-subset/code-span-line-start.md new file mode 100644 index 0000000..e116d88 --- /dev/null +++ b/corpus/round-trip/commonmark-subset/code-span-line-start.md @@ -0,0 +1 @@ +``` `` ``` diff --git a/corpus/round-trip/inline-nodes/text-carriage-return-null.json b/corpus/round-trip/inline-nodes/text-carriage-return-null.json new file mode 100644 index 0000000..f265cd6 --- /dev/null +++ b/corpus/round-trip/inline-nodes/text-carriage-return-null.json @@ -0,0 +1,43 @@ +{ + "content": [ + { + "content": [ + { + "text": "a\r\nb\u0000c", + "type": "text" + }, + { + "marks": [ + { + "attrs": { + "href": "https://example.com/", + "title": "t\u0000" + }, + "type": "link" + } + ], + "text": "link", + "type": "text" + } + ], + "type": "paragraph" + }, + { + "content": [ + { + "attrs": { + "color": "neutral", + "text": "a\nb" + }, + "type": "status" + }, + { + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/inline-nodes/text-carriage-return-null.md b/corpus/round-trip/inline-nodes/text-carriage-return-null.md new file mode 100644 index 0000000..c6702c2 --- /dev/null +++ b/corpus/round-trip/inline-nodes/text-carriage-return-null.md @@ -0,0 +1,3 @@ +a !adf:text{text="\n"}b!adf:text{text="\u0000"}c!adf:link[link]{href="https://example.com/" title="t\u0000"} + +!adf:carry{json="{\"attrs\":{\"color\":\"neutral\",\"text\":\"a\\nb\"},\"type\":\"status\"}"}!adf:carry{json="{\"type\":\"text\"}"} diff --git a/docs/decisions.md b/docs/decisions.md index 0cc7f2a..763c8c0 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -493,16 +493,18 @@ exits 0 on. 2026-10-03, the maintainer. KISS, a technical principle, and its comprehension floor of 7. Valid while `todo.md` items 59, 60, 61 and 62 are open. -A four-seat comprehension panel scored the project under the floor of 7. Its round-5 scores -(2026-10-03) are the baseline. Since 2026-10-04, a chunk whose panel scores any dimension lower -improves what the seats named and scores again, and after three such rounds asks the maintainer. +A four-seat comprehension panel scored the project under the floor of 7. Its last scoring run +(item 55, 2026-10-04) is the baseline. Since 2026-10-04, a chunk whose panel scores any dimension +lower improves what the seats named and scores again, and after three such rounds asks the +maintainer. | Seat | Navigation | Locality | Shape | Self-sufficiency | Overall | |---|---|---|---|---|---| -| Junior | 6 | 5 | 6 | 5 | 5.5 | -| Mid | 6.5 | 5 | 6 | 4.5 | 5.5 | -| Senior | 6.5 | 5.5 | 6.5 | 5.5 | 6 | -| Architect | 6.5 | 5.5 | 6 | 6 | 6 | +| Junior | 7 | 5.5 | 6.5 | 5.5 | 6 | +| Mid | 7 | 5.5 | 6.5 | 5.5 | 6 | +| Senior | 7 | 5.5 | 6.5 | 5.5 | 6 | +| Architect | 6.5 | 5.5 | 6 | 6.5 | 6 | +| Mean | 6.88 | 5.50 | 6.38 | 5.75 | 6.00 | Three panels on near-identical code scored overall means of 5.88, 5.75 and 5.75, and a seat moves ±0.5 between runs. diff --git a/spec/flavour.md b/spec/flavour.md index d0e3e1b..778c5c6 100644 --- a/spec/flavour.md +++ b/spec/flavour.md @@ -450,7 +450,8 @@ the nodes below, `emoji`, `mention` and `status` spell their `text` attribute in plain text: `[]` is the empty string, absent content is the absent attribute, non-empty content parsing to anything but one text node carrying neither marks, attributes nor content is a named error, and so is a `text` key in `{attrs}`. An enclosing mark spelling does not reach into the slot. -A node whose `text` holds a line ending or a null character rides the inline carry. +A node whose `text` holds a line ending or a null character rides the inline carry; a slot spelling +one is a named error. The rest take no content, `!adf:text` included; content on a node that takes none is a named error. - `date` — Attributes: `localId` (string), `timestamp` (string, epoch milliseconds). @@ -473,7 +474,7 @@ The rest take no content, `!adf:text` included; content on a node that takes non Shipped !adf:emoji[🎉]{shortName=":tada:"} on !adf:date{timestamp=1756080000000}. ``` -**Characters CommonMark cannot hold.** A carriage return in text is ` `, which CommonMark reads +**Text CommonMark cannot hold plainly.** A carriage return in text is ` `, which CommonMark reads back as the character. A newline inside a text node, a null character, which CommonMark reads as U+FFFD, and a space or tab where CommonMark strips or refuses one — a block's inline content edges, either side of a line break, an em, strong or strike spelling's inner edges, a pipe cell's edges — @@ -534,8 +535,8 @@ breaks at every node the emitter carries, so no emitted carry sits inside a mark An inline node whose marks no nesting spells — a mark type not listed here, an attrs key its spelling does not list, a value that is not the spelling's type, an attribute the spelling needs and the mark lacks, an empty `attrs` on a mark CommonMark spells, an order putting a code span outside -another mark, `code` over anything but a text node holding text, over text holding a line ending or -a null character, or opening a paragraph line its backticks would read back as a fence, or a spelling +another mark, `code` over anything but a text node holding text or over text holding a line ending +or a null character, or a spelling CommonMark's flanking rules cannot open or close where the run sits (`un**-real**istic`), or one CommonMark's matching pairs elsewhere — the intra-word `*` runs together with a neighbouring `**`, and the multiple-of-3 rule can leave the merged run's pairing to another delimiter — rides the diff --git a/src/adf/document.ts b/src/adf/document.ts index 6f45a23..8816ea8 100644 --- a/src/adf/document.ts +++ b/src/adf/document.ts @@ -65,7 +65,7 @@ export function emptyKeys(held: { attrs?: AdfAttributes; content?: AdfNode[]; ma return keys } -// A format spells this node as text; anything more rides a carry, save content, which is refused. +// A format spells this node as text; anything more rides a carry. export function isBareText(node: AdfNode): boolean { return node.type === 'text' && node.attrs === undefined && node.content === undefined && node.marks?.length !== 0 } diff --git a/src/conformance/adf-property.test.ts b/src/conformance/adf-property.test.ts index 96a8c83..0af5d37 100644 --- a/src/conformance/adf-property.test.ts +++ b/src/conformance/adf-property.test.ts @@ -34,11 +34,11 @@ function shownText(nodes: readonly AdfNode[]): string[] { return shown.filter((text) => text !== '') } -test('a generated document refuses to emit, or its markdown reads back to it', { timeout: propertyTimeout }, () => { +test('a generated document emits markdown that reads back to it', { timeout: propertyTimeout }, () => { fc.assert( fc.property(adfDocument, (document) => { const emitted = adfToMarkdown(document) - if (!emitted.ok) return + assert.ok(emitted.ok, emitted.ok ? '' : `${emitted.error.code}: ${emitted.error.message}`) const read = markdownToAdf(emitted.value) assert.ok(read.ok, read.ok ? '' : `${read.error.code}: ${read.error.message} — reading ${JSON.stringify(emitted.value)}`) assert.deepEqual(read.value, document, `reading ${JSON.stringify(emitted.value)}`) diff --git a/src/markdown/commonmark/grammar.ts b/src/markdown/commonmark/grammar.ts index 5d394e2..6ceb77c 100644 --- a/src/markdown/commonmark/grammar.ts +++ b/src/markdown/commonmark/grammar.ts @@ -83,10 +83,6 @@ export function backslashEscape(text: string, index: number): string | undefined return isAsciiPunctuation(escaped) ? escaped : undefined } -export function claimsLine(line: string, position: LinePosition): boolean { - return escapesLineClaim(line, 0, position) || orderedListOpener.test(line) -} - export function claimsPipeLine(line: string): boolean { return pipeClaim.test(line) } diff --git a/src/markdown/commonmark/link-syntax.ts b/src/markdown/commonmark/link-syntax.ts index d33fac5..8a89bbf 100644 --- a/src/markdown/commonmark/link-syntax.ts +++ b/src/markdown/commonmark/link-syntax.ts @@ -1,4 +1,4 @@ -import { backslashEscape, decodeTextEscapes, holdsControlCharacter } from './grammar.ts' +import { backslashEscape, decodeTextEscapes, holdsControlCharacter, holdsNullCharacter } from './grammar.ts' import { holdsEntityReference } from './entity-references.ts' export type LinkDefinition = { destination: string; title?: string } @@ -117,5 +117,5 @@ export function escapeUnbalanced(spelling: string, opener: string, closer: strin } function spellTitle(title: string): string | undefined { - return /[\n\r\0\\]/.test(title) || holdsEntityReference(title) ? undefined : ` "${title.replaceAll('"', '\\"')}"` + return /[\n\r\\]/.test(title) || holdsNullCharacter(title) || holdsEntityReference(title) ? undefined : ` "${title.replaceAll('"', '\\"')}"` } diff --git a/src/markdown/directive-syntax.ts b/src/markdown/directive-syntax.ts index e42ea98..621c820 100644 --- a/src/markdown/directive-syntax.ts +++ b/src/markdown/directive-syntax.ts @@ -1,7 +1,7 @@ import type { AttributeKind, VocabularyPair, VocabularyValue } from '../adf/attribute-vocabulary.ts' import type { ConvertFault } from '../result.ts' import type { JsonValue } from '../json-value.ts' -import { backslashEscape } from './commonmark/grammar.ts' +import { backslashEscape, holdsNullCharacter } from './commonmark/grammar.ts' import { backtickRun, closingBacktickRun } from './commonmark/backtick-runs.ts' import { isJsonValue, overNested } from '../json-value.ts' import { largestNesting } from '../nesting.ts' @@ -100,10 +100,10 @@ export function readSoleStringAttribute(span: DirectiveSpan, key: string): Read< return { value: spelled.decoded } } -// An inline directive never spans lines, so no content slot holds a line ending. -export function slotLineEndingFault(type: string, text: string): ConvertFault | undefined { - if (!/[\n\r]/.test(text)) return undefined - return { code: 'unspellable-whitespace', message: `the ${type} content slot holds a newline no inline directive spans` } +// What no content slot holds, which the emitter carries instead: an inline directive never spans lines, and CommonMark reads a null character as U+FFFD. +export function slotFault(type: string, text: string): ConvertFault | undefined { + if (/[\n\r]/.test(text)) return { code: 'unspellable-whitespace', message: `the ${type} content slot holds a newline no inline directive spans` } + return holdsNullCharacter(text) ? unsupportedNodeShape(`the ${type} content slot holds a null character: write the node as !adf:carry`) : undefined } export function spellAttributes(pairs: readonly (readonly [string, string])[]): string { diff --git a/src/markdown/emit/adf-to-markdown.test.ts b/src/markdown/emit/adf-to-markdown.test.ts index ca0d451..a876b39 100644 --- a/src/markdown/emit/adf-to-markdown.test.ts +++ b/src/markdown/emit/adf-to-markdown.test.ts @@ -153,11 +153,10 @@ function readsBack(doc: AdfDocument): string { return spelled } -test('carries a code span whose backticks would open its line as a fence', () => { - const fence: AdfNode = { marks: [{ type: 'code' }], text: '```', type: 'text' } - const carried = '!adf:carry{json="{\\"marks\\":[{\\"type\\":\\"code\\"}],\\"text\\":\\"\\u0060\\u0060\\u0060\\",\\"type\\":\\"text\\"}"}' - assert.equal(readsBack(document(paragraph(fence))), `${carried}\n`) - assert.equal(readsBack(document(paragraph({ text: 'a', type: 'text' }, { type: 'hardBreak' }, fence, fence))), `a\\\n${carried}\`\`\`\` \`\`\` \`\`\`\`\n`) +test('spells a code span opening a line, whose backticks open no fence', () => { + const fence: AdfNode = { marks: [{ type: 'code' }], text: '``', type: 'text' } + assert.equal(readsBack(document(paragraph(fence))), '``` `` ```\n') + assert.equal(readsBack(document(paragraph({ text: 'a', type: 'text' }, { type: 'hardBreak' }, fence))), 'a\\\n``` `` ```\n') }) test('escapes the delimiter row a hard break leaves opening a pipe table with no leading pipe', () => { diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index 4b8703f..c0533a6 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -4,17 +4,17 @@ import type { InlineNodeModel } from '../../adf/inline-nodes.ts' import type { LineContainer } from '../line-container.ts' import { assembleInlineLine, isSyntax, type InlineEscaping, type InlineSegment, type MarkRun, type NodeRange } from './line-escaping.ts' import { carriedInline } from '../opaque-carry.ts' -import { claimsLine, holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts' +import { holdsNullCharacter, trimTrailingSpace } from '../commonmark/grammar.ts' import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts' -import { identicalMark, isBareText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { escapeUnbalanced, spellDestination } from '../commonmark/link-syntax.ts' +import { identicalMark, isBareText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { highlightDelimiter } from '../plain/conventions.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { joinsWhenRead, textBreakSpelling } from '../adjacent-text.ts' import { largestNesting } from '../../nesting.ts' import { longestBacktickRun } from '../commonmark/backtick-runs.ts' -import { slotLineEndingFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts' +import { slotFault, spellInlineDirectiveOpener, spellInlineLeafDirective } from '../directive-syntax.ts' import { spellInlineNodeAttributes } from './inline-directive-spelling.ts' import { spellTextDirective } from '../text-directive.ts' @@ -38,7 +38,7 @@ type LineAttempt = { fallback: NodeRange | 'opening-link'; line?: undefined } | type LineFallbacks = { carried: Set; flavour: Flavour; openingLinkAsDirective: boolean } -export type PlainLineFallback = { kind: 'claimed-line'; line: number; text: string } | { kind: 'opening-link' } | { kind: 'unspellable-run'; runs: [MarkRun, ...MarkRun[]] } +export type PlainLineFallback = { kind: 'opening-link' } | { kind: 'unspellable-run'; runs: [MarkRun, ...MarkRun[]] } export function emitInlineLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: Flavour): Result { const emitted = emitLine(nodes, container, path, flavour) @@ -119,19 +119,7 @@ function attemptLine(segments: readonly InlineSegment[], container: LineContaine const verdict = lineVerdict(segments, container, flavour) if (verdict.kind === 'opening-link') return { fallback: 'opening-link' } if (verdict.kind === 'unspellable-run') return { fallback: verdict.runs[0] } - if (verdict.kind === 'line') return { line: verdict.text } - return { fallback: lineOpener(segments, verdict.line) } -} - -// A code span's backticks are what block parsing claims, and its segment names its node; a segment naming none leaves takeFallback to refuse the line. -function lineOpener(segments: readonly InlineSegment[], line: number): NodeRange { - let breaks = 0 - const opener = segments.find((segment) => { - const opens = breaks === line && segment.text !== '' - breaks += segment.text.split('\n').length - 1 - return opens - }) - return opener?.nodes ?? { first: 0, last: -1 } + return { line: verdict.text } } // The fallbacks in the order a line takes them, or the line where it takes none. @@ -139,10 +127,7 @@ function lineVerdict(segments: readonly InlineSegment[], container: LineContaine const assembled = assembleInlineLine(segments, container, flavour) if (assembled.openingLinkAsDirective) return { kind: 'opening-link' } const [run, ...others] = assembled.unspellableRuns - if (run !== undefined) return { kind: 'unspellable-run', runs: [run, ...others] } - const lines = assembled.line.split('\n') - const claimed = container === 'paragraph' ? lines.findIndex((single, index) => claimsLine(single, index === 0 ? 'first' : 'later')) : -1 - return claimed === -1 ? { kind: 'line', text: assembled.line } : { kind: 'claimed-line', line: claimed, text: lines[claimed] ?? '' } + return run === undefined ? { kind: 'line', text: assembled.line } : { kind: 'unspellable-run', runs: [run, ...others] } } // spec/flavour.md, Inline nodes. @@ -269,7 +254,7 @@ function emitInlineDirective(node: AdfNode, model: InlineNodeModel, index: numbe const slot = model.textAttribute === undefined ? undefined : nodeAttrs(node)[model.textAttribute] if (slot === undefined) return success({ segments: [syntax(spellInlineLeafDirective(node.type, attributes))] }) if (typeof slot !== 'string') return success({ carry: { first: index, last: index } }) - if (slotLineEndingFault(node.type, slot) !== undefined || holdsNullCharacter(slot)) return success({ carry: { first: index, last: index } }) + if (slotFault(node.type, slot) !== undefined) return success({ carry: { first: index, last: index } }) const content: InlineSegment[] = slot === '' ? [] : [{ escaping: 'bracketed', text: slot }] return success({ segments: [syntax(spellInlineDirectiveOpener(node.type)), ...content, syntax(`]${attributes}`)] }) } @@ -324,16 +309,14 @@ function emitHighlight(nodes: readonly AdfNode[], depth: number, range: NodeRang // Each node its own span: CommonMark reads two adjacent text nodes in one span back as one. function emitCodeSpan(nodes: readonly AdfNode[], depth: number, range: NodeRange): Emission { - const segments: InlineSegment[] = [] - for (const [offset, node] of nodes.entries()) { + const spans: string[] = [] + for (const node of nodes) { const { text } = node if (!isBareText(node) || nodeMarks(node).length !== depth + 1 || typeof text !== 'string' || text === '' || /[\n\r]/.test(text) || holdsNullCharacter(text)) return { carry: range } - if (offset > 0) segments.push(syntax(textBreakSpelling)) const fence = '`'.repeat(longestBacktickRun(text) + 1) - const index = range.first + offset - segments.push({ escaping: 'none', nodes: { first: index, last: index }, text: `${fence}${needsPadding(text) ? ` ${text} ` : text}${fence}` }) + spans.push(`${fence}${needsPadding(text) ? ` ${text} ` : text}${fence}`) } - return { segments } + return { segments: [syntax(spans.join(textBreakSpelling))] } } function needsPadding(text: string): boolean { diff --git a/src/markdown/parse/directive-nodes.ts b/src/markdown/parse/directive-nodes.ts index 19afd52..efeae1f 100644 --- a/src/markdown/parse/directive-nodes.ts +++ b/src/markdown/parse/directive-nodes.ts @@ -13,7 +13,7 @@ import { inlineMarkSpellingFault } from './directive-marks.ts' import { inlineNodeModel } from '../../adf/inline-nodes.ts' import { readEmptyKeys, spellsEmpty } from '../empty-keys.ts' import { readVocabulary } from './directive-attributes.ts' -import { slotLineEndingFault } from '../directive-syntax.ts' +import { slotFault } from '../directive-syntax.ts' import { textBreakName } from '../adjacent-text.ts' import { textDirectiveName } from '../text-directive.ts' @@ -68,8 +68,8 @@ export function readInlineDirectiveNode( if (text === undefined) { return failure('unsupported-node-shape', `the ${name} content slot holds one text node carrying neither marks, attributes nor content: this one holds something else`, path) } - const spans = slotLineEndingFault(name, text) - if (spans !== undefined) return faulted(spans, path) + const fault = slotFault(name, text) + if (fault !== undefined) return faulted(fault, path) attrs.value[slot] = text } return success(namedNode(name, attrs.value, undefined, empty.value.empty)) diff --git a/src/markdown/parse/markdown-to-adf.test.ts b/src/markdown/parse/markdown-to-adf.test.ts index c5a1aca..6672d37 100644 --- a/src/markdown/parse/markdown-to-adf.test.ts +++ b/src/markdown/parse/markdown-to-adf.test.ts @@ -304,7 +304,7 @@ test('names the directive form a node CommonMark spells refuses', () => { assert.deepEqual(content(markdownToAdf('!adf:bulletList\n!adf:listItem\n---\n!adf:/listItem\n!adf:/bulletList\n')), [bulletList(item({ type: 'rule' }))]) }) -test('refuses the directive form of a blockquote opening with a code span the carry spells', () => { +test('refuses the directive form of a blockquote opening with a code span', () => { assert.equal(content(markdownToAdf('!adf:blockquote\n` `` `\n!adf:/blockquote\n')), 'unsupported-node-shape: blockquote takes the CommonMark spelling, not the directive form') }) @@ -1005,6 +1005,7 @@ test('names the content slot no lone plain text node reads back from', () => { assert.equal(content(markdownToAdf('!adf:status[!adf:text{text="\\n"}]{color=yellow}\n')), spans) assert.equal(content(markdownToAdf('!adf:status[a b]{color=yellow}\n')), spans) assert.equal(content(markdownToAdf('!adf:status[a b]{color=yellow}\n')), spans) + assert.equal(content(markdownToAdf('!adf:status[!adf:text{text="\\u0000"}]{color=yellow}\n')), 'unsupported-node-shape: the status content slot holds a null character: write the node as !adf:carry') assert.equal(content(markdownToAdf('Part !adf:mention{id=b1c2 text=A}.\n')), 'unsupported-node-shape: mention spells its text attribute in the content slot, never in {attrs}') }) diff --git a/src/markdown/plain/adf-to-plain-markdown.test.ts b/src/markdown/plain/adf-to-plain-markdown.test.ts index d634fe0..2076b8e 100644 --- a/src/markdown/plain/adf-to-plain-markdown.test.ts +++ b/src/markdown/plain/adf-to-plain-markdown.test.ts @@ -313,12 +313,12 @@ test('breaks a line at a newline and trims whitespace at every edge CommonMark s assert.equal(plain(paragraph(text(' x ', code))), '` x `\n') assert.equal(plain(node('heading', { level: 1 }, text(' h\ni '))), '# h i\n') assert.equal(plain(paragraph(text('a\r\u0000b'))), 'ab\n') + assert.equal(plain(paragraph(text('x', link('/u', 't\r\u0000')))), '[x](/u "t")\n') }) -test('drops the code mark of a span opening a line with backticks that read as a fence', () => { - assert.equal(plain(paragraph(text('``` x', code))), '\\`\\`\\` x\n') - assert.equal(plain(paragraph(text('a\n'), text('``` x', code))), 'a\\\n\\`\\`\\` x\n') - assert.equal(plain(paragraph(text('a '), text('``` x', code))), 'a ```` ``` x ````\n') +test('keeps a code span opening a line, whose backticks open no fence', () => { + assert.equal(plain(paragraph(text('``` x', code))), '```` ``` x ````\n') + assert.equal(plain(paragraph(text('a\n'), text('``` x', code))), 'a\\\n```` ``` x ````\n') }) test('drops an empty paragraph and merges adjacent lists of one type', () => { diff --git a/src/markdown/plain/inline-reduction.ts b/src/markdown/plain/inline-reduction.ts index 882f732..10acb39 100644 --- a/src/markdown/plain/inline-reduction.ts +++ b/src/markdown/plain/inline-reduction.ts @@ -149,7 +149,7 @@ function plainMarks(marks: readonly AdfMark[], container: LineContainer, text: s function plainLink(mark: AdfMark, container: LineContainer): AdfMark | undefined { const attrs = nodeAttrs(mark) const held = attrs['href'] - const title = typeof attrs['title'] === 'string' ? attrs['title'].replace(/\r/g, '').replace(/\n/g, ' ') : undefined + const title = typeof attrs['title'] === 'string' ? attrs['title'].replace(/[\r\0]/g, '').replace(/\n/g, ' ') : undefined if (typeof held !== 'string') return undefined const href = writableHref(container === 'table-cell' ? held.replaceAll('|', '%7C') : held) if (title === undefined || spellLinkTarget(href, title) === undefined || (container === 'table-cell' && title.includes('|'))) return { attrs: { href }, type: 'link' } @@ -267,21 +267,13 @@ function spellableLine(leaves: AdfNode[], container: LineContainer, path: Conver function withoutFallback(leaves: readonly AdfNode[], fallback: PlainLineFallback): AdfNode[] | undefined { if (fallback.kind === 'unspellable-run') return withoutMarks(leaves, fallback.runs) - const first = fallback.kind === 'opening-link' ? 0 : lineStart(leaves, fallback.line) - const mark = nodeMarks(leaves[first] ?? {})[0] - if (mark === undefined || mark.type !== (fallback.kind === 'opening-link' ? 'link' : 'code')) return undefined - let last = first + const mark = nodeMarks(leaves[0] ?? {})[0] + if (mark === undefined || mark.type !== 'link') return undefined + let last = 0 while (sameMarkAt(nodeMarks(leaves[last + 1] ?? {}), [mark], 0)) last += 1 // A code span is what binds the `]` a link definition reads, and dropping it keeps the link target. - const spans = leaves.slice(first, last + 1).some((leaf) => nodeMarks(leaf).length > 1 && nodeMarks(leaf).at(-1)?.type === 'code') - if (mark.type === 'link' && spans) return leaves.map((leaf, index) => (index < first || index > last ? leaf : withMarks(leaf, nodeMarks(leaf).filter((held) => held.type !== 'code')))) - return withoutMarks(leaves, [{ depth: 0, first, last }]) -} - -function lineStart(leaves: readonly AdfNode[], line: number): number { - let index = 0 - for (let breaks = 0; breaks < line && index < leaves.length; index += 1) if (leaves[index]?.type === 'hardBreak') breaks += 1 - return index + const spans = leaves.slice(0, last + 1).some((leaf) => nodeMarks(leaf).length > 1 && nodeMarks(leaf).at(-1)?.type === 'code') + return spans ? leaves.map((leaf, index) => (index > last ? leaf : withMarks(leaf, nodeMarks(leaf).filter((held) => held.type !== 'code')))) : undefined } // The runs cover disjoint leaves, so one pass drops them all. diff --git a/todo.md b/todo.md index 44ef38d..e6ca6d9 100644 --- a/todo.md +++ b/todo.md @@ -6,7 +6,7 @@ `Bar = 9` -`Next ID = 67` +`Next ID = 69` | Goal | W | |---|---| @@ -26,11 +26,13 @@ |---|---|---|---|---|---|---|---|---|---| | 7 | 0.2.0 | | **Ship HTML: `adfToHtml`, `htmlToAdf`, and `markdownToHtml` / `htmlToMarkdown` composed through ADF.** | 6 | 9 | 9 | 9 | 2, 3 | 25.8 | | 45 | 0.2.0 | | **Replace `isAdfDocument` with a reader returning `Result`.** | 2 | 3 | 6 | 8 | 1 | 25.2 | +| 67 | 0.2.0 | defect | **Push an expand title's leaves one at a time, so a title carrying many text nodes returns a result.** | 1 | 1 | 4 | 9 | 1 | 24.8 | | 6 | 0.2.0 | decision | **Specify the HTML dialect.** | 2 | 6 | 7 | 8 | 2, 3 | 24.7 | | 64 | 0.2.0 | defect | **Read back unchanged on V8 every JSON key the emitter writes, with a fixture whose key holds `\`, `"` or a control character.** | 4 | 4 | 6 | 8 | 1, 7 | 23.0 | | 66 | 0.2.0 | defect | **Refuse an attribute value that is not a plain object, array or JSON primitive.** | 3 | 2 | 5 | 8 | 1 | 22.5 | | 43 | 0.2.0 | decision | **Give each markdown input its own reader, strict to its own standard.** | 6 | 7 | 8 | 9 | 3, 4 | 22.3 | | 65 | 0.2.0 | defect | **Read an image not alone in its paragraph as its alt text linked to its URL, and a titled image alone in its paragraph as the image captioned with its title.** | 5 | 5 | 7 | 7 | 3, 4, 6 | 18.7 | +| 68 | 0.2.0 | | **Carry a non-text inline node holding `content` or `text`, a block holding `text`, and a leaf block holding `content`.** | 3 | 3 | 4 | 7 | 1 | 18.2 | | 49 | 0.2.0 | | **Read a list whose bullet or ordered delimiter changes as two lists, in both readers, and retire `!adf:listBreak`.** | 4 | 5 | 5 | 8 | 3, 4 | 17.2 | | 61 | 0.2.0 | decision | **Have the emitter ask the inline reader how a line reads back, in place of `line-escaping.ts` predicting it.** | 6 | 8 | 3 | 8 | 1 | 14.0 | | 52 | 0.2.0 | | **Spell `colwidth` as a comma list, `colwidth="340,420"`.** | 3 | 3 | 5 | 6 | 5 | 13.0 | @@ -60,7 +62,7 @@ Lands after items 6, 59, 60, 61 and 62. The CommonMark spec suite also runs against `markdownToHtml`. The README documents HTML as it documents markdown, and its tagline and -`package.json`'s `description` regain HTML. +`package.json`'s `description` keep naming HTML. ### 45. Replace `isAdfDocument` with a reader returning `Result`. @@ -68,6 +70,13 @@ Goal 1 has every call return a result; the boolean guard is the one export that cannot say which branch refused, where `not-an-adf-document`'s message already does. Breaking: `MIGRATION.md` shows the guard's replacement. +### 67. Push an expand title's leaves one at a time, so a title carrying many text nodes returns a result. + +`quoteNode`'s title loop (`parse/markdown-to-adf.ts`) spreads `inlineLeaves`'s result into +`push`, against `docs/decisions.md` §Nothing spreads an unbounded array: an inline carry in a +folded callout's title holding an unknown node with ~125k text children throws a `RangeError` from +`plainMarkdownToAdf`. Found by the README-goals audit of item 55, 2026-10-04. + ### 6. Specify the HTML dialect. Element-by-element mapping, the `data-*` fidelity scheme, the opaque-carry form, and the documented @@ -111,6 +120,14 @@ its paragraph, it reads as the image with a `link` mark to `/uri`, which ADF's ` alone in its paragraph, the image reads as its alt text linked to `/uri`, and its URL drops. A writer panel chose this reading 3 of 3 (2026-10-04), by Goal 5, which outranks Goal 6. +### 68. Carry a non-text inline node holding `content` or `text`, a block holding `text`, and a leaf block holding `content`. + +`refuseContentAndText` (`emit/inline-line.ts`) and `emitDirectiveBlock` (`emit/adf-to-markdown.ts`) +refuse these with `unsupported-node-shape`, though `isAdfDocument` accepts them and the carry +round-trips them; `docs/decisions.md` §The code list gives a cause the carry answers no code. The +README's "Not every document converts back" bullet loses them. Found by the README-goals audit of +item 55, 2026-10-04. + ### 49. Read a list whose bullet or ordered delimiter changes as two lists, in both readers, and retire `!adf:listBreak`. Lands after item 43. Today `- a` then `+ b`, or `1.` then `1)`, reads as one list; CommonMark reads @@ -126,7 +143,9 @@ which counts one fewer. The comprehension panel's worst place: `escapeClaims` and `escapeClosedRuns` re-implement the reader's view — flanking, code-span closers, link-definition openings, highlight flanking — and -only the property tests catch drift. It caps the panel's Locality score. +only the property tests catch drift. It caps the panel's Locality score. The replacement runs in +time linear in the line: today `mergesWithSyntax`, `touchesSyntax` and `closesHeading` rescan a run +of one character from each of its characters. ### 52. Spell `colwidth` as a comma list, `colwidth="340,420"`. @@ -248,7 +267,11 @@ Technical principle "One owner per value": the `Walk` record passes through twel mutate it and return `void`; `walk.leaf` alone is written in seven places. `ContainerStack` in the same file shows the shape to follow. `inline-content.ts`'s `Scan` has the same shape: about fifteen functions write `pending`, `pieces`, `deactivatedBefore` and `openingSpellableLink` and return -`void`, and `parseInlineContent` reads a flag `scanInline` leaves on it. +`void`, and `parseInlineContent` reads a flag `scanInline` leaves on it. The same shape recurs in +`mintTaskIds` (`plain/task-ids.ts`), which writes `attrs` on the document `readDocument` built; +`takeFallback` (`emit/inline-line.ts`), which writes the record the next `lineSegments` pass reads; +`writeUnpaired`, `markPairings` and `markHighlights` (`inline-content.ts`), whose order changes the +output; and `nestIn` (`plain/adf-to-plain-markdown.ts`), which pushes into its caller's array. ### 57. Make each `ci.sh` leg build what it reads, so one leg run alone tests the current tree.