diff --git a/AGENTS.md b/AGENTS.md index 73fe356..247cf46 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -75,6 +75,7 @@ In `docs/decisions.md`: - No value `JSON.parse` or `structuredClone` builds loops a call - Nothing spreads an unbounded array - A retry loop checks its own termination +- A line reads back at most 32 times, then takes back every emphasis run - Readers scan by index - The spelling memo - Cost fixes are measured, never timed diff --git a/CHANGELOG.md b/CHANGELOG.md index 45e0076..d305035 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,8 +62,12 @@ strong: text `un` beside `-r` marked em and strong, repeated 1000 times, took up to 9 seconds, now under 170 ms, and `(` marked em, strike and strong beside a mention, repeated 1000 times, took 18 seconds in `adfToPortableMarkdown`, now 150 ms. A line where em shares a delimiter run - with strong takes time quadratic in its length, as in `(` marked em and strong beside `(` marked - em, repeated. + with strong takes time linear in its length: ` ~~ ` marked em and strong, `)` marked em, strike + and strong, `a`, then `.unun` marked strike and strong, repeated 1000 times, took 33 seconds in + `adfToLosslessMarkdown` and 39 in `adfToPortableMarkdown`, now 2.6 and 1.5 seconds. A line whose + emphasis still reads back wrong after 32 tries is written with every node holding em, strike or + strong as `!adf:carry{json="…"}` in `adfToLosslessMarkdown`, and without those marks in + `adfToPortableMarkdown`. - `adfToLosslessMarkdown` and `adfToPortableMarkdown` take time linear in the markdown when lists and quotes nest deeply, and so does `losslessMarkdownToAdf`: a list nested 480 levels around 64 KB of paragraphs took 2 seconds to write, now 100 ms. A `!adf:blockquote` holding 2 MB of lines diff --git a/docs/decisions.md b/docs/decisions.md index 4e26e1a..0789d6a 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -908,6 +908,18 @@ fine. A loop retrying an input until a fallback spells it refuses the pass taking no fallback, so its termination is the loop's own check. +## A line reads back at most 32 times, then takes back every emphasis run + +2026-10-30, the maintainer. Goals 9 and 1. Valid while a writer finds a misread run only by reading +the line back. + +The lossless and portable writers read a line back after each fallback, and the sweep takes back +the misread pairs it can prove fall together. Where a strong opener captures the `*` em leaves over, +the sweep can't predict it, so one pass took back one strong, and a line of 1000 such repeats took +33 seconds. After 32 passes the line takes back every em, strong and strike run at once: the +lossless writer carries every node holding one, and the portable writer drops those marks. That +line keeps fewer marks than the pass-by-pass result would, in exchange for time linear in the line. + ## Readers scan by index 2026-08-30, the kept scan 2026-09-18, the maintainer. Goal 9. Valid while the pipeline persona diff --git a/docs/reading-cost.md b/docs/reading-cost.md index 8d59a25..87676dd 100644 --- a/docs/reading-cost.md +++ b/docs/reading-cost.md @@ -550,3 +550,18 @@ Read most (opens and searches): - `src/markdown/portable/adf-to-portable-markdown.test.ts` 3 - `README.md` 2 - `ci.sh` 2 + +## Reading cost, chunk 13332c44ccd0, finished 2026-10-30T15:27:59Z + +132270 new input tokens before the first commit. + +Read most (opens and searches): + +- `todo.md` 10 +- `src/markdown/emit/inline-line.ts` 4 +- `src/markdown/emit/line-escaping.ts` 4 +- `src/markdown/portable/inline-reduction.ts` 4 +- `CHANGELOG.md` 2 +- `docs/score-anchors.md` 1 +- `src/markdown/emit/inline-line.test.ts` 1 +- `src/markdown/portable/inline-reduction.test.ts` 1 diff --git a/src/markdown/emit/adf-to-markdown.test.ts b/src/markdown/emit/adf-to-markdown.test.ts index 1e1ef3a..ebc6478 100644 --- a/src/markdown/emit/adf-to-markdown.test.ts +++ b/src/markdown/emit/adf-to-markdown.test.ts @@ -452,6 +452,14 @@ test('carries the mark spelling that cannot close where it sits, and keeps the l assert.equal(emitted(marked('^', { type: 'strike' }), b, marked('c', { type: 'strike' })), '!adf:carry{json="{\\"marks\\":[{\\"type\\":\\"strike\\"}],\\"text\\":\\"^\\",\\"type\\":\\"text\\"}"}b~~c~~\n') }) +test('carries every node holding em, strong or strike in a line still misread after 32 passes', () => { + const marked = (text: string, ...types: string[]): AdfNode => ({ marks: types.map((type) => ({ type })), text, type: 'text' }) + const carry = (node: AdfNode): string => `!adf:carry{json="${JSON.stringify(node).replaceAll('"', '\\"')}"}` + const repeat = [marked(' ~~ ', 'em', 'strong'), marked(')', 'em', 'strike', 'strong'), { text: 'a', type: 'text' }, marked('.unun', 'strike', 'strong')] + const written = repeat.map((node) => (node.marks === undefined ? 'a' : carry(node))).join('') + assert.equal(markdown(adfToLosslessMarkdown(document(paragraph(...Array.from({ length: 40 }, () => repeat).flat())))), `${written.repeat(40)}\n`) +}) + test('refuses a node carrying one mark type twice', () => { const em: AdfMark = { type: 'em' } assert.equal(code(adfToLosslessMarkdown(document(paragraph({ marks: [em, em], text: 'x', type: 'text' })))), 'unsupported-node-shape') diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index b26938e..9189a8c 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -5,7 +5,7 @@ import type { LinkDefinition, TitleEscapes } from '../commonmark/link-syntax.ts' import type { WrittenFlavour } from '../flavour.ts' import { assembleInlineLine, isSyntax, type InlineEscaping, type InlineSegment, type MarkRun, type NodeRange, type RunFallback } from './line-escaping.ts' import { carriedInline } from '../opaque-carry.ts' -import { commonMarkLink, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts' +import { commonMarkLink, isEmphasis, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts' import { escapeUnbalanced, spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { flavourClaims } from '../flavour.ts' @@ -47,6 +47,9 @@ type LineAttempt = { fallback: Fallback; line?: undefined } | { fallback?: undef type LineFallbacks = { carried: Set; flavour: WrittenFlavour; openingLinkAsDirective: boolean } +// docs/decisions.md §A line reads back at most 32 times, then takes back every emphasis run. +export const readBackPasses = 32 + export type PortableLineFallback = { kind: 'opening-link' } | { kind: 'unspellable-run'; runs: [MarkRun, ...MarkRun[]] } export function emitInlineLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: WrittenFlavour, perPass: FallbacksPerPass = 'every'): Result { @@ -92,7 +95,7 @@ export function tryImageLine(alt: string | undefined, url: string, flavour: Writ function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: ConvertErrorPath, flavour: WrittenFlavour, perPass: FallbacksPerPass): Result { const fallbacks = withheldMarkFallbacks(nodes, flavour) // Terminates because takeFallback refuses a pass that took no new fallback. - for (;;) { + for (let pass = 1; ; pass += 1) { const emission = lineSegments(nodes, container, path, fallbacks) if (!emission.ok) return emission const { carries, segments } = emission.value @@ -100,6 +103,7 @@ function emitLine(nodes: readonly AdfNode[], container: LineContainer, path: Con if (attempt.fallback === undefined) return success({ line: attempt.line, openingLinkAsDirective: fallbacks.openingLinkAsDirective, segments: attempt.segments }) const taken = takeFallback(fallbacks, takenThisPass(attempt.fallback, perPass), path) if (!taken.ok) return taken + if (pass === readBackPasses) for (const [index, node] of nodes.entries()) if (nodeMarks(node).some(isEmphasis)) fallbacks.carried.add(index) } } diff --git a/src/markdown/mark-spellings.ts b/src/markdown/mark-spellings.ts index c23a401..49fbe80 100644 --- a/src/markdown/mark-spellings.ts +++ b/src/markdown/mark-spellings.ts @@ -37,6 +37,10 @@ export function markSpelling(type: string): MarkSpelling | undefined { return { attributes, kind: spelling.kind } } +export function isEmphasis(mark: AdfMark): boolean { + return markSpelling(mark.type)?.kind === 'emphasis' +} + export function linkMark(definition: LinkDefinition): AdfMark { return linkMarkFrom(toLinkAttrs(definition)) } diff --git a/src/markdown/portable/adf-to-portable-markdown.test.ts b/src/markdown/portable/adf-to-portable-markdown.test.ts index ec48a68..c3f6216 100644 --- a/src/markdown/portable/adf-to-portable-markdown.test.ts +++ b/src/markdown/portable/adf-to-portable-markdown.test.ts @@ -401,6 +401,13 @@ test('drops the mark of the run that cannot close where it sits, and keeps the l assert.equal(portable(paragraph(text('~', highlight), text('b'), text('日本', highlight))), '~b==日本==\n') }) +test('drops every em, strong and strike mark in a line still misread after 32 passes', () => { + const strike: AdfMark = { type: 'strike' } + const repeat = [text(' ~~ ', em, strong), text(')', em, strike, strong), text('a'), text('.unun', strike, strong)] + const line = Array.from({ length: 40 }, () => repeat).flat() + assert.equal(portable(paragraph(...line)), `${Array.from({ length: 40 }, () => '~~ )a.unun').join(' ')}\n`) +}) + test('gives up every highlight run whose own delimiter cannot flank in one fallback, so a line of them re-emits once however many it holds', () => { const highlight: AdfMark = { type: 'backgroundColor' } const line = [text('a '), text('x', highlight), text('a '), text('x', highlight), text('a '), text('x', highlight), text('a')] diff --git a/src/markdown/portable/inline-reduction.ts b/src/markdown/portable/inline-reduction.ts index d428caf..a064feb 100644 --- a/src/markdown/portable/inline-reduction.ts +++ b/src/markdown/portable/inline-reduction.ts @@ -4,11 +4,12 @@ import type { MarkRun } from '../emit/line-escaping.ts' import { codeTextMarkTypes } from '../../adf/inline-nodes.ts' import { failure, success, type ConvertErrorPath, type Result } from '../../result.ts' import { imageAside } from '../image-aside.ts' +import { isEmphasis } from '../mark-spellings.ts' import { joinsWhenEditorNormal, sameMarkWhenEditorNormal } from './editor-normal.ts' import { largestNesting } from '../../nesting.ts' import { mergeAdjacentText, nodeAttrs, nodeContent, nodeMarks } from '../../adf/document.ts' import { partsShownText, referenceName, shownImage, shownLink, shownText, shownUrl } from '../../adf/shown-text.ts' -import { portableLineFallback, type FallbacksPerPass, type PortableLineFallback } from '../emit/inline-line.ts' +import { portableLineFallback, readBackPasses, type FallbacksPerPass, type PortableLineFallback } from '../emit/inline-line.ts' import { spellDestination } from '../commonmark/link-syntax.ts' const highlight = 'backgroundColor' @@ -219,13 +220,13 @@ function sameMarkAt(marks: readonly AdfMark[], others: readonly AdfMark[], index } function spellableLine(leaves: AdfNode[], container: LineContainer, path: ConvertErrorPath, perPass: FallbacksPerPass): Result { - for (let current = leaves; ; ) { + for (let current = leaves, pass = 1; ; pass += 1) { const fallback = portableLineFallback(current, container, path, (run) => dropsAlone(current, run)) if (!fallback.ok) return fallback if (fallback.value === undefined) return success(current) const fixed = withoutFallback(current, fallback.value, perPass) if (fixed === undefined) return failure('unsupported-node-shape', 'a portable line keeps a spelling that dropping a mark does not change', path) - current = trimmedEdges(fixed) + current = trimmedEdges(pass === readBackPasses ? fixed.map((leaf) => withMarks(leaf, nodeMarks(leaf).filter((mark) => !isEmphasis(mark)))) : fixed) } } diff --git a/todo.md b/todo.md index b85417c..6068937 100644 --- a/todo.md +++ b/todo.md @@ -12,8 +12,7 @@ | ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score | |---|---|---|---|---|---|---|---|---|---|---|---| -| 161 | 0.3.0 | | defect | **Write a line where em shares a delimiter run with strong in time linear in its mark runs, in `adfToLosslessMarkdown` and `adfToPortableMarkdown`.** | 4 | 1 | 2 | 5 | 5 | 9 by 3 | 7.8 | -| 160 | 0.3.0 | 161 | decision | **Make the block-text tree and the batched mark fallbacks read as easily as the code before items 33, 109 and 158, and keep every switch only tests use out of production signatures.** | 3 | 1 | 2 | 4 | 6 | code health by 6 | 8.7 | +| 160 | 0.3.0 | | decision | **Make the block-text tree and the batched mark fallbacks read as easily as the code before items 33, 109 and 158, and keep every switch only tests use out of production signatures.** | 3 | 1 | 2 | 4 | 6 | code health by 6 | 8.7 | | 150 | 0.3.0 | | decision | **Read `adfToHtml`'s HTML in `htmlToAdf` as a browser serializes it too — a raw `"` in text, ` `, `\r\n` line endings — so a page's `innerHTML` or a form post reads back.** | 2 | 2 | 2 | 4 | 4 | 6 by 5 | 5.0 | | 115 | 0.3.0 | | defect | **Build text holding no `border` mark in `losslessMarkdownToAdf`, and have `adfToLosslessMarkdown` carry text holding one.** | 2 | 1 | 2 | 3 | 3 | 4 by 3 | 4.6 | | 91 | 0.3.0 | | | **Spell an emphasis once across the code spans, an emphasis or link once across the hard breaks, dates, emoji, inline cards, mentions and statuses between two nodes it marks, and an emphasis across inline media, in both writers.** | 3 | 2 | 2 | 5 | 5 | 6 by 6 | 3.6 | @@ -70,23 +69,6 @@ ## Details -### 161. Write a line where em shares a delimiter run with strong in time linear in its mark runs, in `adfToLosslessMarkdown` and `adfToPortableMarkdown`. - -The stability review of item 158 found it, on main before item 158 as well. Em's `*` and strong's -`**` form one delimiter run, so once that strong closes, the next strong opener that can close -captures the `*` left over. One run per pass then takes back one strong per pass, and every pass -must write the same. At 1000 repeats: - -- `(` marked em and strong beside `(` marked em, then `a`: 7.4 s in `adfToPortableMarkdown`; -- `~** ` marked em and strong beside `- _` marked em, then `r`: 8.6 s in `adfToPortableMarkdown`; -- ` ~~ ` marked em and strong beside `)` marked em, strike and strong, then `a` and `.unun` marked - strike and strong: 33 s in `adfToLosslessMarkdown`, 39 s in `adfToPortableMarkdown`. - -v0.2.0 wrote each in under 100 ms at 400 repeats: lossless carried the whole line, and portable -kept strong on every `(`. `unspellableRuns` judges pair by pair, so it cannot predict these captures -without CommonMark's delimiter stack across passes. Dropping a doomed em before the strongs it holds -may be linear and keep more marks, but it changes what the portable writer keeps: ask first. - ### 160. Make the block-text tree and the batched mark fallbacks read as easily as the code before items 33, 109 and 158, and keep every switch only tests use out of production signatures. PR #106's paired ruling read worse than c7bbb04: both seats ruled Locality and Shape worse, the