From 42fc6d97fb5e10dd65a41eb747acc3e21f628d28 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Fri, 30 Oct 2026 22:11:47 +0100 Subject: [PATCH] Move the emphasis sweep into unspellable-runs.ts with named standings, derive its rule in docs/decisions.md, and close item 160 --- AGENTS.md | 1 + docs/decisions.md | 52 ++++- src/markdown/emit/inline-line.ts | 3 +- src/markdown/emit/line-escaping.ts | 226 +------------------- src/markdown/emit/unspellable-runs.ts | 244 ++++++++++++++++++++++ src/markdown/portable/inline-reduction.ts | 2 +- todo.md | 29 +-- 7 files changed, 300 insertions(+), 257 deletions(-) create mode 100644 src/markdown/emit/unspellable-runs.ts diff --git a/AGENTS.md b/AGENTS.md index 572319b..94aa8f0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -76,6 +76,7 @@ In `docs/decisions.md`: - Nothing spreads an unbounded array - A retry loop checks its own termination - After 32 passes, a line carries or drops every em, strike and strong mark +- Beside the first misread emphasis run, a pass takes back only runs one run per pass would take back too - Readers scan by index - The spelling memo - Cost fixes are measured, never timed diff --git a/docs/decisions.md b/docs/decisions.md index 2df5ba5..bb3bbda 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -692,16 +692,14 @@ Drafts describe the finished release, so they wait for every item in it. Valid w ## A chunk ruled worse merges where a plan item of its release already restores that code's reading, and that item takes the ruling's places -2026-10-30, the maintainer, merging PR #107 only because `todo.md` item 160 fixes exactly the -places its paired ruling voted worse. Serves code health. Valid while the comprehension check runs -once per chunk. +2026-10-30, the maintainer, merging PR #107 only because a plan item of its release would fix +exactly the places its paired ruling voted worse. Serves code health. Valid while the comprehension +check runs once per chunk. The comprehension check runs once per chunk, after review. A restructure made after it goes unmeasured, and past a pass's round cap it goes unreviewed. Where an item of the same release reworks that code again, restructuring it inside the chunk does the work twice. Before the chunk -merges, that item's detail names every place the ruling voted worse. Until item 160 ships, the -places PR #106's and PR #107's rulings named read worse than their bases, so this entry works -against code health, and item 160 ends it. +merges, that item's detail names every place the ruling voted worse. ## No schema validation @@ -922,7 +920,47 @@ and a line repeating such a pattern 1000 times took 33 seconds lossless and 39 p passes, both writers fall back on every em, strike and strong run at once. Such a line keeps fewer marks than one strong per pass would leave, and takes time linear in its length. 32 clears the most passes any line in the suite needs, 22 portable and 5 lossless, with room to spare. The -`first` mode, which only a property uses, takes no cap. +properties' reference, which takes back one run per pass, takes no cap. + +## Beside the first misread emphasis run, a pass takes back only runs one run per pass would take back too + +2026-10-29, the maintainer. Goals 9 and 6. Valid while a writer finds a misread run only by reading +the line back. + +Taking back one misread run per pass costs a pass per run. So with the first misread run, a pass +takes back every run `unspellableRuns` proves one run per pass would take back later. The proof is +sound, not complete: a run it misses costs one more pass, but a run it wrongly takes back loses a +spelling the line could keep. The properties beside `inline-line.ts` and `inline-reduction.ts` +check that every line comes out as one run per pass writes it. + +The sweep reads each run as a pair, its opener and its closer, and orders the pairs by where they +close. A pair falls with the first misread pair where both hold: + +- **It falls cleanly.** Under a drop, no run merges at its ends. Under a carry, it holds no pair + closing before it over fewer nodes, whose carry would split it. +- **It cannot read, however it is written.** CommonMark judges a delimiter by the characters either + side of its whole delimiter run. Either the opener cannot open, or the closer cannot close and the + pair holds no pair, other than those settled to fall, that may be written with its character. An + opener that may open can pair with such a delimiter inside, and one run per pass would take that + inner pair back first. + +Em has no fixed character: `resolveEmphasis` writes `_`, and `*` beside a word character. So a +verdict tries every character the pairs beside it leave possible. It reads each other pair as that +pair stands when one run per pass reaches the judged one: + +- **A pair whose ends a drop may merge proves nothing**, since the merge may move its delimiters. +- **Past a judged pair closing after the first, every pair stands.** One run per pass takes back + the first misread pair in closing order. While a pair settled to fall before the judged one + stands, it or one before it is misread, and once they are gone the judged pair is. So no pair + closing after the judged one goes first. +- **Under a carry, a pair over the judged pair's nodes stands**, carried with it. Any other proves + nothing, since a carry may split it. +- **Under a drop, a pair settled to fall is gone.** One run per pass takes it back, so the judged + pair need only fail to read once every such pair is gone. Any other pair may stand or be gone, + and the verdict must hold both ways. + +Each verdict reads the verdicts settled before it, so the sweep settles forward from the first +pair, then back. ## Readers scan by index diff --git a/src/markdown/emit/inline-line.ts b/src/markdown/emit/inline-line.ts index ee2dd31..55bbfde 100644 --- a/src/markdown/emit/inline-line.ts +++ b/src/markdown/emit/inline-line.ts @@ -2,8 +2,9 @@ import type { AdfMark, AdfNode } from '../../adf/document.ts' import type { InlineNodeModel } from '../../adf/inline-nodes.ts' import type { LineContainer } from '../line-container.ts' import type { LinkDefinition, TitleEscapes } from '../commonmark/link-syntax.ts' +import type { RunFallback } from './unspellable-runs.ts' import type { WrittenFlavour } from '../flavour.ts' -import { assembleInlineLine, isSyntax, type InlineEscaping, type InlineSegment, type MarkRun, type NodeRange, type RunFallback } from './line-escaping.ts' +import { assembleInlineLine, isSyntax, type InlineEscaping, type InlineSegment, type MarkRun, type NodeRange } from './line-escaping.ts' import { carriedInline } from '../opaque-carry.ts' import { commonMarkLink, isEmphasis, linkHref, markSpelling, spellMarkAttributes } from '../mark-spellings.ts' import { escapeUnbalanced, spellDestination, spellLinkTarget } from '../commonmark/link-syntax.ts' diff --git a/src/markdown/emit/line-escaping.ts b/src/markdown/emit/line-escaping.ts index 6064776..4e539c9 100644 --- a/src/markdown/emit/line-escaping.ts +++ b/src/markdown/emit/line-escaping.ts @@ -1,16 +1,16 @@ -import type { AdfNode } from '../../adf/document.ts' import type { Claims, WrittenFlavour } from '../flavour.ts' import type { InlineReading } from '../inline-content.ts' import type { InlineSource, InlineToken } from '../inline-tokens.ts' import type { LineContainer } from '../line-container.ts' -import { delimiterFlags, isWordCharacter, runLength } from '../commonmark/emphasis-matching.ts' -import { delimiterRunToken, highlightFlanking, inlineSource, readInlineToken } from '../inline-tokens.ts' +import type { RunFallback } from './unspellable-runs.ts' +import { delimiterRunToken, inlineSource, readInlineToken } from '../inline-tokens.ts' import { escapesLineClaim, type LinePosition } from '../commonmark/grammar.ts' import { flavourClaims } from '../flavour.ts' import { isBareDelimiterRow } from '../pipe-table-syntax.ts' -import { nodeMarks } from '../../adf/document.ts' +import { isWordCharacter, runLength } from '../commonmark/emphasis-matching.ts' import { opensLinkDefinition } from '../commonmark/link-reference-definitions.ts' import { readLineBack } from '../inline-content.ts' +import { unspellableRuns } from './unspellable-runs.ts' export type DelimiterRole = 'close' | 'open' @@ -30,32 +30,12 @@ export type InlineSegment = export type AssembledLine = { line: string; openingLinkAsDirective?: true; unspellableRuns: MarkRun[] } -// How the caller takes a run back: a carry swaps its nodes for one carry directive, splitting every run holding it; a drop removes its mark from the line's `nodes`. -export type RunFallback = { kind: 'carry' } | { kind: 'drop'; nodes: readonly AdfNode[] } - type ScanLine = { position: LinePosition; start: number; text: string } -// An emphasis run's delimiters, written alike, by index into the placed segments. -type EmphasisPair = { closer: number; delimiter: string; nodes: MarkRun; opener: number } - type Escapings = (index: number) => InlineEscaping | undefined -// Which emphasis pairs fall: go back in this pass with the first misread one. `fallen` flags each pair settled to fall. -type PairSweep = { - readonly fallback: RunFallback - readonly fallen: Uint8Array - readonly first: number - // The ends of every run whose drop is not alone: dropping it can merge the runs meeting there. - readonly merging: ReadonlySet - readonly output: string - // The pair each placed segment delimits, -1 where it delimits none. - readonly pairAt: Int32Array - readonly pairs: readonly EmphasisPair[] - readonly placed: readonly PlacedSegment[] -} - // Where a segment's first character landed in the output. -type PlacedSegment = { segment: InlineSegment; start: number } +export type PlacedSegment = { segment: InlineSegment; start: number } // The escaped line once reading it back asks for no further escape. type ReadBack = { output: string; placed: PlacedSegment[]; reading: InlineReading } @@ -63,9 +43,6 @@ type ReadBack = { output: string; placed: PlacedSegment[]; reading: InlineReadin // Which of a run's characters escape: every escapable one, or every one before `end`. type RunEscapes = { end: number; escapable: false } | { escapable: true } -// How a pair stands while another falls: `either` where it may fall before the judged one. -type Standing = 'either' | 'gone' | 'there' - // Which text characters escape wherever they stand: one of them opened a construct when the line was read back. type TextEscapes = { backticks: boolean; brackets: boolean } @@ -292,199 +269,6 @@ function claimsCharacter({ container, escapings, headingCloser, inline: { text: } } -// The runs this pass takes back: the first emphasis run the line misreads and every run that falls with it, else every misflanked highlight run. One misread emphasis pair can re-pair the delimiters after it. -function unspellableRuns(placed: readonly PlacedSegment[], reading: InlineReading, output: string, fallback: RunFallback): MarkRun[] { - const { pairAt, pairs } = emphasisPairs(placed) - const first = pairs.findIndex((pair) => !readAsWritten(reading, placed, pair)) - const firstPair = pairs[first] - if (firstPair === undefined) return misflankedHighlights(placed, output) - const merging = new Set() - if (fallback.kind === 'drop') for (const { nodes } of pairs) if (!dropsAlone(fallback.nodes, nodes)) merging.add(nodes.first).add(nodes.last + 1) - const sweep: PairSweep = { fallback, fallen: new Uint8Array(pairs.length), first, merging, output, pairAt, pairs, placed } - sweep.fallen[first] = 1 - // Each verdict reads the verdicts settled before it, so the sweep settles outward from the first pair. - for (let judged = first + 1; judged < pairs.length; judged += 1) if (fallsWithFirst(sweep, judged)) sweep.fallen[judged] = 1 - for (let judged = first - 1; judged >= 0; judged -= 1) if (fallsWithFirst(sweep, judged)) sweep.fallen[judged] = 1 - const runs = [firstPair.nodes] - for (const [index, pair] of pairs.entries()) if (index !== first && sweep.fallen[index] === 1) runs.push(pair.nodes) - return runs -} - -// Dropping the run merges no runs and moves no run's end, whatever else drops: at each end, the node outside holds no mark type the end node holds inside the run's mark. -function dropsAlone(nodes: readonly AdfNode[], run: MarkRun): boolean { - return endsAlone(nodes[run.first], nodes[run.first - 1], run.depth) && endsAlone(nodes[run.last], nodes[run.last + 1], run.depth) -} - -function endsAlone(edge: AdfNode | undefined, beside: AdfNode | undefined, depth: number): boolean { - const besideTypes = new Set(nodeMarks(beside ?? {}).map((mark) => mark.type)) - return nodeMarks(edge ?? {}) - .slice(depth + 1) - .every((mark) => !besideTypes.has(mark.type)) -} - -// The pairs in closing order. -function emphasisPairs(placed: readonly PlacedSegment[]): { pairAt: Int32Array; pairs: EmphasisPair[] } { - const pairAt = new Int32Array(placed.length).fill(-1) - const pairs: EmphasisPair[] = [] - const openers: { at: number; nodes: MarkRun }[] = [] - for (const [at, { segment }] of placed.entries()) { - if (segment.emphasis === 'open') openers.push({ at, nodes: segment.nodes }) - const opener = segment.emphasis === 'close' ? openers.pop() : undefined - if (opener === undefined) continue - pairAt[opener.at] = pairs.length - pairAt[at] = pairs.length - pairs.push({ closer: at, delimiter: segment.text, nodes: opener.nodes, opener: opener.at }) - } - return { pairAt, pairs } -} - -// No `==` in text can open or close, and highlights never nest, so a highlight run reads back where both its delimiters flank. -function misflankedHighlights(placed: readonly PlacedSegment[], output: string): MarkRun[] { - const misflanked: MarkRun[] = [] - for (const { segment, start } of placed) { - if (segment.highlight !== undefined && !highlightFlanks(output, start, segment.highlight) && misflanked.at(-1) !== segment.nodes) misflanked.push(segment.nodes) - } - return misflanked -} - -// A pair falls with the first where one run per pass would take it back too: one of its delimiters cannot flank however the unsettled pairs fall. -function fallsWithFirst(sweep: PairSweep, judged: number): boolean { - if (!fallsCleanly(sweep, judged)) return false - const spellings = pairSpellings(sweep, judged, judged) - return spellings !== undefined && spellings.every((character) => !mayFlank(sweep, judged, character, 'opener') || (!mayFlank(sweep, judged, character, 'closer') && !holdsAlike(sweep, judged, character))) -} - -// An opener that may open can pair with a delimiter of its own character inside, so one run per pass may take back a pair standing there first. -function holdsAlike({ fallen, pairs }: PairSweep, judged: number, character: string): boolean { - const pair = pairs[judged] - for (let held = judged - 1; pair !== undefined && held >= 0; held -= 1) { - const inner = pairs[held] - if (inner === undefined || inner.closer < pair.opener) return false - if (fallen[held] === 0 && (inner.delimiter.length === 1 ? character !== '~' : inner.delimiter.startsWith(character))) return true - } - return false -} - -// A carry splits every pair holding it, so a pair falls cleanly under a carry only where it holds no pair spanning fewer nodes, and under a drop only where no run merges at its ends. -function fallsCleanly({ fallback, merging, pairs }: PairSweep, judged: number): boolean { - const pair = pairs[judged] - if (pair === undefined) return false - return fallback.kind === 'drop' ? !mergesAt(merging, pair) : !holdsEarlier(pairs, judged, pair) -} - -function mergesAt(merging: ReadonlySet, { nodes }: EmphasisPair): boolean { - return merging.has(nodes.first) || merging.has(nodes.last + 1) -} - -// CommonMark judges a delimiter by the characters either side of its whole run of `character`. -function mayFlank(sweep: PairSweep, judged: number, character: string, end: 'closer' | 'opener'): boolean { - const at = sweep.pairs[judged]?.[end] ?? -1 - const before = runEdge(sweep, judged, at, -1, character) - const after = runEdge(sweep, judged, at, 1, character) - if (before === undefined || after === undefined) return true - return before.some((left) => - after.some((right) => { - const flags = delimiterFlags(character, left, right) - return end === 'opener' ? flags.canOpen : flags.canClose - }), - ) -} - -// The characters that may stand past the run of `character` holding the delimiter at `from`; `undefined` where a pair beside may move. -function runEdge(sweep: PairSweep, judged: number, from: number, step: -1 | 1, character: string): string[] | undefined { - const found: string[] = [] - // Past the run stands the character after its last delimiter: a backslash where the text beside is escaped, which the next segment's start skips. - let edge = delimiterEdge(sweep.placed, from, step) - for (let at = from + step; ; at += step) { - const segment = sweep.placed[at]?.segment - const other = sweep.pairAt[at] ?? -1 - if (segment === undefined || (other === -1 && segment.text !== '')) break - if (other === -1) continue - const standing = pairStanding(sweep, judged, other) - const spellings = standing === 'gone' ? [] : pairSpellings(sweep, judged, other) - if (standing === undefined || spellings === undefined) return undefined - for (const spelling of spellings) if (spelling !== character) found.push(spelling) - if (standing === 'there' && !spellings.includes(character)) return found - edge = delimiterEdge(sweep.placed, at, step) - } - found.push(sweep.output.charAt(step === 1 ? edge : edge - 1)) - return found -} - -function delimiterEdge(placed: readonly PlacedSegment[], at: number, step: -1 | 1): number { - const delimiter = placed[at] - if (delimiter === undefined) return 0 - return step === 1 ? delimiter.start + delimiter.segment.text.length : delimiter.start -} - -// How the other pair stands when the judged one falls. Where the judged pair closes after the first, every pair closing after it still stands, since the judged pair is then the first that cannot read as written; `undefined` where a carry or a merge may move the other's delimiters. -function pairStanding({ fallback, fallen, first, merging, pairs }: PairSweep, judged: number, other: number): Standing | undefined { - const pair = pairs[other] - if (other === judged) return 'there' - if (pair === undefined || mergesAt(merging, pair)) return undefined - if (other > judged && judged > first) return 'there' - // Under a carry, pairs spanning the same nodes stand and fall together. - if (fallback.kind === 'carry') return spansAlike(pair, pairs[judged]) ? 'there' : undefined - return fallen[other] === 1 ? 'gone' : 'either' -} - -// The characters the other pair may be written with when the judged one falls: em takes `_`, turning `*` beside a word character. -function pairSpellings(sweep: PairSweep, judged: number, other: number): string[] | undefined { - const pair = sweep.pairs[other] - if (pair === undefined) return undefined - if (pair.delimiter.length > 1) return [pair.delimiter.charAt(0)] - // The carry's punctuation stands where the split pair meets it. - if (sweep.fallback.kind === 'carry' && other !== judged && holdsEarlier(sweep.pairs, judged, pair)) return ['*', '_'] - const before = wordBeside(sweep, judged, pair.opener, -1) - const after = wordBeside(sweep, judged, pair.closer, 1) - if (before === undefined || after === undefined) return undefined - const spellings: string[] = [] - if (before.includes(true) || after.includes(true)) spellings.push('*') - if (before.includes(false) && after.includes(false)) spellings.push('_') - return spellings -} - -// Whether a word character may stand beside the delimiter at `from` as resolveEmphasis reads the line, for each way the unsettled pairs fall. -function wordBeside(sweep: PairSweep, judged: number, from: number, step: -1 | 1): boolean[] | undefined { - const found: boolean[] = [] - for (let at = from + step; ; at += step) { - const segment = sweep.placed[at]?.segment - const other = sweep.pairAt[at] ?? -1 - if (segment !== undefined && segment.text === '') continue - if (segment === undefined || other === -1) { - found.push(segment !== undefined && isWordCharacter(segment.text.charAt(step === 1 ? 0 : segment.text.length - 1))) - return found - } - const standing = pairStanding(sweep, judged, other) - if (standing === undefined) return undefined - if (standing === 'gone') continue - found.push(false) - if (standing === 'there') return found - } -} - -// Whether the pair holds one a carry may take before the judged one. Pairs close in order, so the judged pair, or one holding it, holds such a pair only where it holds the last of them to close. -function holdsEarlier(pairs: readonly EmphasisPair[], judged: number, pair: EmphasisPair): boolean { - let earlier = judged - 1 - while (spansAlike(pairs[earlier], pairs[judged])) earlier -= 1 - const held = pairs[earlier] - return held !== undefined && pair.nodes.first <= held.nodes.first && held.nodes.last <= pair.nodes.last -} - -function spansAlike(pair: EmphasisPair | undefined, other: EmphasisPair | undefined): boolean { - return pair !== undefined && other !== undefined && pair.nodes.first === other.nodes.first && pair.nodes.last === other.nodes.last -} - -function highlightFlanks(output: string, start: number, role: DelimiterRole): boolean { - const flanking = highlightFlanking(output, start) - return role === 'open' ? flanking.opens : flanking.closes -} - -function readAsWritten(reading: InlineReading, placed: readonly PlacedSegment[], pair: EmphasisPair): boolean { - const read = reading.emphasis.get(placed[pair.opener]?.start ?? -1) - return read !== undefined && read.closer === placed[pair.closer]?.start && read.width === pair.delimiter.length -} - function isEscapable(escaping: InlineEscaping | undefined): boolean { return escaping === 'backslash' || escaping === 'bracketed' } diff --git a/src/markdown/emit/unspellable-runs.ts b/src/markdown/emit/unspellable-runs.ts new file mode 100644 index 0000000..b1eb280 --- /dev/null +++ b/src/markdown/emit/unspellable-runs.ts @@ -0,0 +1,244 @@ +import type { AdfNode } from '../../adf/document.ts' +import type { DelimiterRole, MarkRun, PlacedSegment } from './line-escaping.ts' +import type { InlineReading } from '../inline-content.ts' +import { delimiterFlags, isWordCharacter } from '../commonmark/emphasis-matching.ts' +import { highlightFlanking } from '../inline-tokens.ts' +import { nodeMarks } from '../../adf/document.ts' + +// How the caller takes a run back: a carry swaps its nodes for one carry directive, splitting every run holding it; a drop removes its mark from the line's `nodes`. +export type RunFallback = { kind: 'carry' } | { kind: 'drop'; nodes: readonly AdfNode[] } + +// An emphasis run's delimiters, written alike, by index into the placed segments. +type EmphasisPair = { closer: number; delimiter: string; nodes: MarkRun; opener: number } + +// The pairs in closing order, and what the sweep has settled about them. +type PairSweep = { + readonly fallback: RunFallback + // 1 where the pair is settled to fall. + readonly fallen: Uint8Array + readonly first: number + // The node index at each end of a run whose drop is not alone. + readonly mergingEnds: ReadonlySet + readonly output: string + // The pair each placed segment delimits, -1 where it delimits none. + readonly pairAt: Int32Array + readonly pairs: readonly EmphasisPair[] + readonly placed: readonly PlacedSegment[] +} + +// How another pair stands when the judged one falls: `moves` where a carry or a merge may move its delimiters, `unsettled` where it may stand or be gone. +type Standing = 'fallen' | 'moves' | 'stands' | 'unsettled' + +// The runs this pass takes back: the first emphasis run the line misreads and every run that falls with it, else every misflanked highlight run. docs/decisions.md §Beside the first misread emphasis run, a pass takes back only runs one run per pass would take back too. +export function unspellableRuns(placed: readonly PlacedSegment[], reading: InlineReading, output: string, fallback: RunFallback): MarkRun[] { + const { pairAt, pairs } = emphasisPairs(placed) + const first = pairs.findIndex((pair) => !readAsWritten(reading, placed, pair)) + const firstPair = pairs[first] + if (firstPair === undefined) return misflankedHighlights(placed, output) + const sweep: PairSweep = { fallback, fallen: new Uint8Array(pairs.length), first, mergingEnds: mergingEnds(pairs, fallback), output, pairAt, pairs, placed } + sweep.fallen[first] = 1 + // Each verdict reads the verdicts settled before it, so the sweep settles outward from the first pair. + for (let judged = first + 1; judged < pairs.length; judged += 1) if (fallsWithFirst(sweep, judged)) sweep.fallen[judged] = 1 + for (let judged = first - 1; judged >= 0; judged -= 1) if (fallsWithFirst(sweep, judged)) sweep.fallen[judged] = 1 + const runs = [firstPair.nodes] + for (const [index, pair] of pairs.entries()) if (index !== first && sweep.fallen[index] === 1) runs.push(pair.nodes) + return runs +} + +// The pairs in closing order. +function emphasisPairs(placed: readonly PlacedSegment[]): { pairAt: Int32Array; pairs: EmphasisPair[] } { + const pairAt = new Int32Array(placed.length).fill(-1) + const pairs: EmphasisPair[] = [] + const openers: { at: number; nodes: MarkRun }[] = [] + for (const [at, { segment }] of placed.entries()) { + if (segment.emphasis === 'open') openers.push({ at, nodes: segment.nodes }) + const opener = segment.emphasis === 'close' ? openers.pop() : undefined + if (opener === undefined) continue + pairAt[opener.at] = pairs.length + pairAt[at] = pairs.length + pairs.push({ closer: at, delimiter: segment.text, nodes: opener.nodes, opener: opener.at }) + } + return { pairAt, pairs } +} + +function readAsWritten(reading: InlineReading, placed: readonly PlacedSegment[], pair: EmphasisPair): boolean { + const read = reading.emphasis.get(placed[pair.opener]?.start ?? -1) + return read !== undefined && read.closer === placed[pair.closer]?.start && read.width === pair.delimiter.length +} + +// No `==` in text can open or close, and highlights never nest, so a highlight run reads back where both its delimiters flank. +function misflankedHighlights(placed: readonly PlacedSegment[], output: string): MarkRun[] { + const misflanked: MarkRun[] = [] + for (const { segment, start } of placed) { + if (segment.highlight !== undefined && !highlightFlanks(output, start, segment.highlight) && misflanked.at(-1) !== segment.nodes) misflanked.push(segment.nodes) + } + return misflanked +} + +function highlightFlanks(output: string, start: number, role: DelimiterRole): boolean { + const flanking = highlightFlanking(output, start) + return role === 'open' ? flanking.opens : flanking.closes +} + +function mergingEnds(pairs: readonly EmphasisPair[], fallback: RunFallback): Set { + const ends = new Set() + if (fallback.kind === 'drop') for (const { nodes } of pairs) if (!dropsAlone(fallback.nodes, nodes)) ends.add(nodes.first).add(nodes.last + 1) + return ends +} + +// Dropping the run merges no runs and moves no run's end, whatever else drops: at each end, the node outside holds no mark type the end node holds inside the run's mark. +function dropsAlone(nodes: readonly AdfNode[], run: MarkRun): boolean { + return endsAlone(nodes[run.first], nodes[run.first - 1], run.depth) && endsAlone(nodes[run.last], nodes[run.last + 1], run.depth) +} + +function endsAlone(edge: AdfNode | undefined, beside: AdfNode | undefined, depth: number): boolean { + const besideTypes = new Set(nodeMarks(beside ?? {}).map((mark) => mark.type)) + return nodeMarks(edge ?? {}) + .slice(depth + 1) + .every((mark) => !besideTypes.has(mark.type)) +} + +// A pair falls with the first only where one run per pass would take it back too: it falls cleanly, and cannot read however it is written. +function fallsWithFirst(sweep: PairSweep, judged: number): boolean { + if (!fallsCleanly(sweep, judged)) return false + const spellings = pairSpellings(sweep, judged, judged) + return spellings !== undefined && spellings.every((character) => cannotRead(sweep, judged, character)) +} + +// A carry splits every pair holding it, so a pair falls cleanly under a carry only where it holds no pair spanning fewer nodes, and under a drop only where no run merges at its ends. +function fallsCleanly({ fallback, mergingEnds, pairs }: PairSweep, judged: number): boolean { + const pair = pairs[judged] + if (pair === undefined) return false + return fallback.kind === 'drop' ? !mergesAt(mergingEnds, pair) : !holdsPairBefore(pairs, judged, pair) +} + +function mergesAt(mergingEnds: ReadonlySet, { nodes }: EmphasisPair): boolean { + return mergingEnds.has(nodes.first) || mergingEnds.has(nodes.last + 1) +} + +// An opener that may open can pair with a delimiter of its own character inside, so one run per pass may take back a pair standing there first. +function cannotRead(sweep: PairSweep, judged: number, character: string): boolean { + if (!mayFlank(sweep, judged, character, 'opener')) return true + return !mayFlank(sweep, judged, character, 'closer') && !holdsAlike(sweep, judged, character) +} + +// Whether the judged pair holds a pair not settled to fall that may be written with `character`. Pairs close in order, so the pairs it holds close just before it. +function holdsAlike({ fallen, pairs }: PairSweep, judged: number, character: string): boolean { + const pair = pairs[judged] + for (let held = judged - 1; pair !== undefined && held >= 0; held -= 1) { + const inner = pairs[held] + if (inner === undefined || inner.closer < pair.opener) return false + if (fallen[held] === 0 && mayBeWrittenWith(inner, character)) return true + } + return false +} + +// Em may be written `*` or `_`, strong and strike only with their own character. +function mayBeWrittenWith(pair: EmphasisPair, character: string): boolean { + return pair.delimiter.length === 1 ? character !== '~' : pair.delimiter.startsWith(character) +} + +// CommonMark judges a delimiter by the characters either side of its whole run of `character`. +function mayFlank(sweep: PairSweep, judged: number, character: string, end: 'closer' | 'opener'): boolean { + const at = sweep.pairs[judged]?.[end] ?? -1 + const before = runEdge(sweep, judged, at, -1, character) + const after = runEdge(sweep, judged, at, 1, character) + if (before === undefined || after === undefined) return true + return before.some((left) => + after.some((right) => { + const flags = delimiterFlags(character, left, right) + return end === 'opener' ? flags.canOpen : flags.canClose + }), + ) +} + +// The characters that may stand past the run of `character` holding the delimiter at `from`; `undefined` where a pair beside may move. +function runEdge(sweep: PairSweep, judged: number, from: number, step: -1 | 1, character: string): string[] | undefined { + const found: string[] = [] + const { delimiters } = touching(sweep, from, step) + for (const at of delimiters) { + const other = sweep.pairAt[at] ?? -1 + const standing = pairStanding(sweep, judged, other) + if (standing === 'moves') return undefined + const spellings = standing === 'fallen' ? [] : pairSpellings(sweep, judged, other) + if (spellings === undefined) return undefined + for (const spelling of spellings) if (spelling !== character) found.push(spelling) + if (standing === 'stands' && !spellings.includes(character)) return found + } + // Past the run stands the character after its last delimiter: a backslash where the text beside is escaped, which the next segment's start skips. + const edge = delimiterEdge(sweep.placed, delimiters.at(-1) ?? from, step) + found.push(sweep.output.charAt(step === 1 ? edge : edge - 1)) + return found +} + +// Whether a word character may stand beside the delimiter at `from` as resolveEmphasis reads the line, for each way the unsettled pairs fall. +function wordBeside(sweep: PairSweep, judged: number, from: number, step: -1 | 1): boolean[] | undefined { + const found: boolean[] = [] + const { delimiters, next } = touching(sweep, from, step) + for (const at of delimiters) { + const standing = pairStanding(sweep, judged, sweep.pairAt[at] ?? -1) + if (standing === 'moves') return undefined + if (standing === 'fallen') continue + found.push(false) + if (standing === 'stands') return found + } + const text = sweep.placed[next]?.segment.text ?? '' + found.push(isWordCharacter(text.charAt(step === 1 ? 0 : text.length - 1))) + return found +} + +// The delimiters touching the one at `from`, nearest first, and the segment holding text past them; an empty segment touches both sides. +function touching({ pairAt, placed }: PairSweep, from: number, step: -1 | 1): { delimiters: number[]; next: number } { + const delimiters: number[] = [] + let next = from + step + for (let segment = placed[next]?.segment; segment !== undefined; segment = placed[next]?.segment) { + if ((pairAt[next] ?? -1) !== -1) delimiters.push(next) + else if (segment.text !== '') break + next += step + } + return { delimiters, next } +} + +function delimiterEdge(placed: readonly PlacedSegment[], at: number, step: -1 | 1): number { + const delimiter = placed[at] + if (delimiter === undefined) return 0 + return step === 1 ? delimiter.start + delimiter.segment.text.length : delimiter.start +} + +// How the other pair stands when the judged one falls; docs/decisions.md §Beside the first misread emphasis run, a pass takes back only runs one run per pass would take back too derives each case. +function pairStanding({ fallback, fallen, first, mergingEnds, pairs }: PairSweep, judged: number, other: number): Standing { + const pair = pairs[other] + if (other === judged) return 'stands' + if (pair === undefined || mergesAt(mergingEnds, pair)) return 'moves' + if (other > judged && judged > first) return 'stands' + if (fallback.kind === 'carry') return spanSameNodes(pair, pairs[judged]) ? 'stands' : 'moves' + return fallen[other] === 1 ? 'fallen' : 'unsettled' +} + +// The characters the other pair may be written with when the judged one falls: em takes `_`, turning `*` beside a word character. +function pairSpellings(sweep: PairSweep, judged: number, other: number): string[] | undefined { + const pair = sweep.pairs[other] + if (pair === undefined) return undefined + if (pair.delimiter.length > 1) return [pair.delimiter.charAt(0)] + // The carry's punctuation stands where the split pair meets it. + if (sweep.fallback.kind === 'carry' && other !== judged && holdsPairBefore(sweep.pairs, judged, pair)) return ['*', '_'] + const before = wordBeside(sweep, judged, pair.opener, -1) + const after = wordBeside(sweep, judged, pair.closer, 1) + if (before === undefined || after === undefined) return undefined + const spellings: string[] = [] + if (before.includes(true) || after.includes(true)) spellings.push('*') + if (before.includes(false) && after.includes(false)) spellings.push('_') + return spellings +} + +// Whether the pair holds one closing before the judged one over other nodes: one a carry may take back first. Pairs close in order, so it holds such a pair only where it holds the last of them to close. +function holdsPairBefore(pairs: readonly EmphasisPair[], judged: number, pair: EmphasisPair): boolean { + let earlier = judged - 1 + while (spanSameNodes(pairs[earlier], pairs[judged])) earlier -= 1 + const held = pairs[earlier] + return held !== undefined && pair.nodes.first <= held.nodes.first && held.nodes.last <= pair.nodes.last +} + +function spanSameNodes(pair: EmphasisPair | undefined, other: EmphasisPair | undefined): boolean { + return pair !== undefined && other !== undefined && pair.nodes.first === other.nodes.first && pair.nodes.last === other.nodes.last +} diff --git a/src/markdown/portable/inline-reduction.ts b/src/markdown/portable/inline-reduction.ts index 3c9842c..5f697d2 100644 --- a/src/markdown/portable/inline-reduction.ts +++ b/src/markdown/portable/inline-reduction.ts @@ -255,7 +255,7 @@ function withoutEmphasis(leaves: readonly AdfNode[]): AdfNode[] { return leaves.map((leaf) => withMarks(leaf, nodeMarks(leaf).filter((mark) => !isEmphasis(mark)))) } -// Pairs that fall together may hold one another, so a leaf drops its mark at each depth a run over it names. +// The runs one pass takes back may nest, so a leaf drops its mark at each depth a run over it names. function withoutMarks(leaves: readonly AdfNode[], runs: readonly MarkRun[]): AdfNode[] { const depths = new Map() for (const run of runs) { diff --git a/todo.md b/todo.md index aff9844..088737c 100644 --- a/todo.md +++ b/todo.md @@ -12,14 +12,13 @@ | ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score | |---|---|---|---|---|---|---|---|---|---|---|---| -| 160 | 0.3.0 | | decision | **Make the block-text tree and the batched mark fallbacks read as easily as the code before items 33, 109 and 158, and keep every switch only tests use out of production signatures.** | 4 | 1 | 2 | 5 | 6 | code health by 6 | 8.7 | | 150 | 0.3.0 | | decision | **Read `adfToHtml`'s HTML in `htmlToAdf` as a browser serializes it too — a raw `"` in text, ` `, `\r\n` line endings — so a page's `innerHTML` or a form post reads back.** | 2 | 2 | 2 | 4 | 4 | 6 by 5 | 5.0 | | 115 | 0.3.0 | | defect | **Build text holding no `border` mark in `losslessMarkdownToAdf`, and have `adfToLosslessMarkdown` carry text holding one.** | 2 | 1 | 2 | 3 | 3 | 4 by 3 | 4.6 | | 91 | 0.3.0 | | | **Spell an emphasis once across the code spans, an emphasis or link once across the hard breaks, dates, emoji, inline cards, mentions and statuses between two nodes it marks, and an emphasis across inline media, in both writers.** | 3 | 2 | 2 | 5 | 5 | 6 by 6 | 3.6 | | 133 | 0.3.0 | | decision | **Write mentions, emoji, dates, statuses and inline cards as semantic HTML in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 4 | 6 by 3 | 3.0 | | 132 | 0.3.0 | | | **Write tables as `` in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 5 | 6 by 4 | 2.4 | -| 73 | 0.3.0 | 91, 115, 132, 133, 150, 160 | | **Announce the package where someone needing an ADF converter already reads: JRACLOUD-77436, the Atlassian developer community and Stack Overflow's ADF-to-markdown questions.** | 1 | 2 | 2 | 3 | 2 | 3 by 8 | 16.9 | -| 72 | 0.3.0 | 91, 115, 132, 133, 150, 160 | decision | **Re-create `gitea.larvit.se/larvit/adf-codec` as a read-only pull mirror of GitHub, its issues and Actions off and its description pointing at GitHub.** | 1 | 2 | 1 | 2 | 1 | 3 by 1 | 2.5 | +| 73 | 0.3.0 | 91, 115, 132, 133, 150 | | **Announce the package where someone needing an ADF converter already reads: JRACLOUD-77436, the Atlassian developer community and Stack Overflow's ADF-to-markdown questions.** | 1 | 2 | 2 | 3 | 2 | 3 by 8 | 16.9 | +| 72 | 0.3.0 | 91, 115, 132, 133, 150 | decision | **Re-create `gitea.larvit.se/larvit/adf-codec` as a read-only pull mirror of GitHub, its issues and Actions off and its description pointing at GitHub.** | 1 | 2 | 1 | 2 | 1 | 3 by 1 | 2.5 | | 153 | 0.4.0 | | question | **Decide whether `src/markdown/` root may import `emit/`, `parse/` or `portable/`, and hold the answer in `.oxlintrc.json` and §The source parts by ADF and format.** | 1 | 2 | 2 | 3 | 2 | code health by 3 | 26.2 | | 156 | 0.4.0 | | | **Stop a leg's container when its script is interrupted, so Ctrl-C ends `ci.sh` or `mutation-tests/run.sh` at once.** | 1 | 2 | 1 | 2 | 1 | code health by 2 | 20.9 | | 128 | 0.4.0 | | | **Pin the portable flavour's block reading and writing with tests: callout titles past one line, a link beside other marks in a title, the nesting limit of callouts and task lists, refusals inside tasks and their paths, and empty or non-paragraph tasks.** | 2 | 1 | 1 | 2 | 3 | code health by 3 | 19.6 | @@ -69,30 +68,6 @@ ## Details -### 160. Make the block-text tree and the batched mark fallbacks read as easily as the code before items 33, 109 and 158, and keep every switch only tests use out of production signatures. - -PR #106's paired ruling read worse than c7bbb04: both seats ruled Locality and Shape worse, the -maintainer chose to merge it and file this. Make each place they named easier to read: -`writtenText` and `Shape` in `block-text.ts` (a stack walk and eight hand-kept measures), -`FallbacksPerPass` threaded through `emitInlineLine` and `reduceInline` only for two properties, -`unspellableRuns`'s pair sweep in `line-escaping.ts`, which needs CommonMark's flanking rules open -beside it, `takenThisPass` and `checkedCarry` in `inline-line.ts`, `tryListItemLines` reading -`block-text.ts`'s measures, `holdsOnlyRuleCharacters` holding BlockText's rule in the grammar, and -`markerOpening` repeating the marker regexes' `[`. - -PR #107's paired ruling read worse than 41fd106: both seats ruled Locality, Shape and -Self-sufficiency worse. The maintainer merged it only because this item fixes exactly the places -they named (`docs/decisions.md` §A chunk ruled worse merges): in `line-escaping.ts`, -`unspellableRuns`'s sweep over `PairSweep.fallen`, where each verdict reads those settled before -it; `fallsWithFirst`, whose calls run five levels deep, need CommonMark's delimiter-run rules beside -them, and rest on a property test for their rule; and `holdsAlike`, `holdsEarlier`, `spansAlike` -and `Standing`, defined only by comments. In `inline-reduction.ts` they named `spellableLine`, -which passes `dropsAlone` through four functions into the escaper's `RunFallback` and `merging`, -and `withoutMarks`, whose runs nest for a reason that lives in the sweep. - -No output may change, and the timings the cost entries under `CHANGELOG.md` → `## Unreleased` give -for both writers must hold. - ### 150. Read `adfToHtml`'s HTML in `htmlToAdf` as a browser serializes it too — a raw `"` in text, ` `, `\r\n` line endings — so a page's `innerHTML` or a form post reads back. `htmlToAdf` refuses all three today. A browser's `innerHTML` writes text's `"` raw and a no-break