Compare commits

..

5 Commits

12 changed files with 115 additions and 20 deletions
+2 -1
View File
@@ -229,7 +229,8 @@ editor wrote; the CommonMark spec suite against `markdownToAdf` and `markdownToH
Beside the corpus, properties run over documents generated from the node tables and over generated Beside the corpus, properties run over documents generated from the node tables and over generated
markdown, on a fixed seed in the gate; `PROPERTY_RUNS=<runs>` raises the runs and randomizes the markdown, on a fixed seed in the gate; `PROPERTY_RUNS=<runs>` raises the runs and randomizes the
seed for local digging, and a counterexample found becomes a round-trip fixture. seed for local digging, and a counterexample found becomes a round-trip fixture. The generators and
run parameters properties share live in `src/property-harness.ts`, outside the build and coverage.
`spec/flavour.md` is read as a source too, so the node tables cannot drift from the prose they `spec/flavour.md` is read as a source too, so the node tables cannot drift from the prose they
copy: each `- ` bullet in `## Block nodes`, `## Inline nodes` and `## Marks` declares the nodes copy: each `- ` bullet in `## Block nodes`, `## Inline nodes` and `## Marks` declares the nodes
@@ -0,0 +1,45 @@
{
"content": [
{
"content": [
{
"marks": [
{
"attrs": {
"href": "/u"
},
"type": "link"
},
{
"type": "code"
}
],
"text": "]: a",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"attrs": {
"timestamp": "]:a"
},
"marks": [
{
"attrs": {
"href": "/u"
},
"type": "link"
}
],
"type": "date"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1,3 @@
:adf{json="{\"marks\":[{\"attrs\":{\"href\":\"/u\"},\"type\":\"link\"},{\"type\":\"code\"}],\"text\":\"]: a\",\"type\":\"text\"}"}
:adf{json="{\"attrs\":{\"timestamp\":\"]:a\"},\"marks\":[{\"attrs\":{\"href\":\"/u\"},\"type\":\"link\"}],\"type\":\"date\"}"}
+1 -1
View File
@@ -23,7 +23,7 @@
}, },
"scripts": { "scripts": {
"build": "tsc -p tsconfig.build.json", "build": "tsc -p tsconfig.build.json",
"test": "node --test --experimental-test-coverage --test-coverage-exclude=\"src/**/*.test.ts\" --test-coverage-exclude=src/property-generators.ts --test-coverage-branches=98 --test-coverage-functions=100 --test-coverage-lines=100 \"src/**/*.test.ts\"", "test": "node --test --experimental-test-coverage --test-coverage-exclude=\"src/**/*.test.ts\" --test-coverage-exclude=src/property-harness.ts --test-coverage-branches=98 --test-coverage-functions=100 --test-coverage-lines=100 \"src/**/*.test.ts\"",
"typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.build.json" "typecheck": "tsc --noEmit && tsc --noEmit -p tsconfig.build.json"
}, },
"devDependencies": { "devDependencies": {
+3 -2
View File
@@ -56,8 +56,9 @@ normalizes to it through the round-trip.
the leading delimiter of a construct that would otherwise open, re-scan from there, and repeat. the leading delimiter of a construct that would otherwise open, re-scan from there, and repeat.
A backtick run escapes whole, and a lone backtick escapes wherever an escaped one follows it in A backtick run escapes whole, and a lone backtick escapes wherever an escaped one follows it in
the same inline content: CommonMark reads no escape inside a code span, so `` \` `` closes one. the same inline content: CommonMark reads no escape inside a code span, so `` \` `` closes one.
A paragraph's opening `[` escapes wherever its line reads as a link reference definition, which Where a paragraph opens with what reads as a link reference definition, which resolves before
resolves before any inline construct binds: a `]` inside a code span or `{attrs}` counts. any inline construct binds (a `]` inside a code span or `{attrs}` counts), an opening text `[`
escapes and an opening link's nodes ride the carry.
An emphasis delimiter run in text escapes where CommonMark can open **or** close with it, so An emphasis delimiter run in text escapes where CommonMark can open **or** close with it, so
`*not emphasis*` is `\*not emphasis\*` — no delimiter the emitter did not write reaches the `*not emphasis*` is `\*not emphasis\*` — no delimiter the emitter did not write reaches the
matching below, which is what lets the emitter decide its own pairings. matching below, which is what lets the emitter decide its own pairings.
+1 -1
View File
@@ -2,7 +2,7 @@ import fc from 'fast-check'
import assert from 'node:assert/strict' import assert from 'node:assert/strict'
import test from 'node:test' import test from 'node:test'
import { adfDocument, propertyRuns, propertyTimeout } from './property-generators.ts' import { adfDocument, propertyRuns, propertyTimeout } from './property-harness.ts'
import { adfToMarkdown } from './markdown/emit/adf-to-markdown.ts' import { adfToMarkdown } from './markdown/emit/adf-to-markdown.ts'
import { markdownToAdf } from './markdown/parse/markdown-to-adf.ts' import { markdownToAdf } from './markdown/parse/markdown-to-adf.ts'
import { toEditorNormal } from './adf/editor-normal.ts' import { toEditorNormal } from './adf/editor-normal.ts'
+34 -8
View File
@@ -5,7 +5,8 @@ import test from 'node:test'
import type { Arbitrary, DepthIdentifier } from 'fast-check' import type { Arbitrary, DepthIdentifier } from 'fast-check'
import type { AttributeVocabulary } from './adf/attribute-vocabulary.ts' import type { AttributeVocabulary } from './adf/attribute-vocabulary.ts'
import type { JsonValue } from './json-value.ts' import type { JsonValue } from './json-value.ts'
import { adfDocument, attributes, jsonKey, jsonValue, markdownPieces, propertyRuns, propertyTimeout, textOf } from './property-generators.ts' import type { Result } from './result.ts'
import { adfDocument, attributes, jsonKey, jsonValue, markdownPieces, propertyRuns, propertyTimeout, textOf } from './property-harness.ts'
import { adfToMarkdown } from './markdown/emit/adf-to-markdown.ts' import { adfToMarkdown } from './markdown/emit/adf-to-markdown.ts'
import { blockArgument } from './markdown/block-directive-arguments.ts' import { blockArgument } from './markdown/block-directive-arguments.ts'
import { blockDirectives } from './adf/block-directives.ts' import { blockDirectives } from './adf/block-directives.ts'
@@ -27,6 +28,13 @@ type Choice = { arbitrary: Arbitrary<string>; hostile?: true; weight: number }
type Edit = [at: number, removed: number, inserted: string] type Edit = [at: number, removed: number, inserted: string]
type InlineMarkdown = { destination: Arbitrary<string>; inlines: Arbitrary<string>; label: Arbitrary<string>; oneLine: Arbitrary<string>; text: Arbitrary<string>; word: Arbitrary<string> }
type LeafMarkdown = { fencedCode: Arbitrary<string>; leafBlock: Arbitrary<string> }
const directiveShapedFloor = 350
const directiveSpelling = /(?<!\\):[a-z][A-Za-z0-9]*[[{]|^[\t >*+.)0-9-]*::+[a-z]/m
const fixpointFloor = 600
const gateRuns = 1000 const gateRuns = 1000
const vocabularies = [...Object.values(blockDirectives).map((directive) => directive.attributes), ...Object.values(inlineDirectives).map((directive) => directive.attributes), ...Object.values(markAttributes)] const vocabularies = [...Object.values(blockDirectives).map((directive) => directive.attributes), ...Object.values(inlineDirectives).map((directive) => directive.attributes), ...Object.values(markAttributes)]
@@ -179,7 +187,11 @@ const indentDrift = fc.oneof({ arbitrary: fc.constant(0), weight: 6 }, { arbitra
const closerDrift = fc.option(fc.integer({ max: 5, min: 2 }), { freq: 6 }) const closerDrift = fc.option(fc.integer({ max: 5, min: 2 }), { freq: 6 })
function markdownOf(hostile: boolean): Arbitrary<string> { function markdownOf(hostile: boolean): Arbitrary<string> {
const blockDepth = fc.createDepthIdentifier() const inline = inlineMarkdown(hostile)
return blockMarkdown(hostile, inline, leafBlocks(hostile, inline))
}
function inlineMarkdown(hostile: boolean): InlineMarkdown {
const inlineDepth = fc.createDepthIdentifier() const inlineDepth = fc.createDepthIdentifier()
const text = hostile ? fc.oneof(cleanText, textOf(0)) : cleanText const text = hostile ? fc.oneof(cleanText, textOf(0)) : cleanText
const word = fc.oneof({ arbitrary: prose, weight: 3 }, { arbitrary: text.filter((held) => held !== ''), weight: 2 }) const word = fc.oneof({ arbitrary: prose, weight: 3 }, { arbitrary: text.filter((held) => held !== ''), weight: 2 })
@@ -228,8 +240,10 @@ function markdownOf(hostile: boolean): Arbitrary<string> {
), ),
inlines: fc.array(tie('inline'), { depthIdentifier: inlineDepth, maxLength: 4, minLength: 1 }).map((parts) => parts.join('')), inlines: fc.array(tie('inline'), { depthIdentifier: inlineDepth, maxLength: 4, minLength: 1 }).map((parts) => parts.join('')),
})) }))
return { destination, inlines, label, oneLine: inlines.map((held) => held.replace(/[\n\r]/g, ' ')), text, word }
}
const oneLine = inlines.map((held) => held.replace(/[\n\r]/g, ' ')) function leafBlocks(hostile: boolean, { destination, inlines, label, oneLine, text, word }: InlineMarkdown): LeafMarkdown {
const fencedCode = fc const fencedCode = fc
.tuple( .tuple(
fc.constantFrom('```', '```', '~~~', '````', '``'), fc.constantFrom('```', '```', '~~~', '````', '``'),
@@ -287,7 +301,11 @@ function markdownOf(hostile: boolean): Arbitrary<string> {
}, },
{ arbitrary: fc.constantFrom(':::', '::', '::::', ':::panel', '::: panel', ':::panel info extra'), hostile: true, weight: 1 }, { arbitrary: fc.constantFrom(':::', '::', '::::', ':::panel', '::: panel', ':::panel info extra'), hostile: true, weight: 1 },
]) ])
return { fencedCode, leafBlock }
}
function blockMarkdown(hostile: boolean, { inlines, oneLine }: InlineMarkdown, { fencedCode, leafBlock }: LeafMarkdown): Arbitrary<string> {
const blockDepth = fc.createDepthIdentifier()
const { blocks } = fc.letrec<{ block: string; blocks: string }>((tie) => { const { blocks } = fc.letrec<{ block: string; blocks: string }>((tie) => {
const bodyByModel = { block: fc.oneof(tie('blocks'), fc.constant('')), code: fencedCode, inline: fc.oneof(oneLine, fc.constant('')) } const bodyByModel = { block: fc.oneof(tie('blocks'), fc.constant('')), code: fencedCode, inline: fc.oneof(oneLine, fc.constant('')) }
const tableDirectives = Object.entries(blockDirectives).map(([name, directive]) => { const tableDirectives = Object.entries(blockDirectives).map(([name, directive]) => {
@@ -346,10 +364,10 @@ function markdownOf(hostile: boolean): Arbitrary<string> {
const cleanMarkdown = markdownOf(false) const cleanMarkdown = markdownOf(false)
const hostileMarkdown = markdownOf(true) const hostileMarkdown = markdownOf(true)
const canonical = adfDocument.map((document) => { const canonical = adfDocument
const emitted = adfToMarkdown(document) .map((document) => adfToMarkdown(document))
return emitted.ok ? emitted.value : '' .filter((emitted): emitted is Extract<Result<string>, { ok: true }> => emitted.ok)
}) .map((emitted) => emitted.value)
function edited(markdown: string, edits: readonly Edit[]): string { function edited(markdown: string, edits: readonly Edit[]): string {
let text = markdown let text = markdown
@@ -375,12 +393,17 @@ const document = fc
.map(([held, ending, trailing]) => `${held}${trailing}`.replaceAll('\n', ending)) .map(([held, ending, trailing]) => `${held}${trailing}`.replaceAll('\n', ending))
test('generated markdown refuses, or what it parses to refuses to emit, or its spelling reads back and spells itself', { timeout: propertyTimeout }, () => { test('generated markdown refuses, or what it parses to refuses to emit, or its spelling reads back and spells itself', { timeout: propertyTimeout }, () => {
const parameters = propertyRuns(gateRuns)
let directiveShaped = 0
let fixpoints = 0
fc.assert( fc.assert(
fc.property(document, (input) => { fc.property(document, (input) => {
const parsed = markdownToAdf(input) const parsed = markdownToAdf(input)
if (!parsed.ok) return if (!parsed.ok) return
const emitted = adfToMarkdown(parsed.value) const emitted = adfToMarkdown(parsed.value)
if (!emitted.ok) return if (!emitted.ok) return
fixpoints += 1
if (directiveSpelling.test(emitted.value)) directiveShaped += 1
const read = markdownToAdf(emitted.value) const read = markdownToAdf(emitted.value)
assert.ok(read.ok, read.ok ? '' : `${read.error.code}: ${read.error.message} — reading ${JSON.stringify(emitted.value)}`) assert.ok(read.ok, read.ok ? '' : `${read.error.code}: ${read.error.message} — reading ${JSON.stringify(emitted.value)}`)
assert.deepEqual(toEditorNormal(read.value), toEditorNormal(parsed.value), `reading ${JSON.stringify(emitted.value)}`) assert.deepEqual(toEditorNormal(read.value), toEditorNormal(parsed.value), `reading ${JSON.stringify(emitted.value)}`)
@@ -388,7 +411,10 @@ test('generated markdown refuses, or what it parses to refuses to emit, or its s
assert.ok(respelled.ok, respelled.ok ? '' : `${respelled.error.code}: ${respelled.error.message} — spelling ${JSON.stringify(emitted.value)} again`) assert.ok(respelled.ok, respelled.ok ? '' : `${respelled.error.code}: ${respelled.error.message} — spelling ${JSON.stringify(emitted.value)} again`)
assert.equal(respelled.value, emitted.value) assert.equal(respelled.value, emitted.value)
}), }),
propertyRuns(gateRuns), parameters,
) )
if (parameters.seed === undefined) return
assert.ok(fixpoints >= fixpointFloor, `${fixpoints} of ${gateRuns} runs reached the fixpoint, under the floor of ${fixpointFloor}`)
assert.ok(directiveShaped >= directiveShapedFloor, `${directiveShaped} runs reaching the fixpoint spelled a directive, under the floor of ${directiveShapedFloor}`)
}) })
+1 -1
View File
@@ -292,6 +292,6 @@ function emitLink(nodes: readonly AdfNode[], mark: AdfMark, depth: number, range
if (!inner.ok) return inner if (!inner.ok) return inner
if (inner.value.carry !== undefined) return inner if (inner.value.carry !== undefined) return inner
const spelledTarget: InlineSegment = context.bracketed ? { escaping: 'bracketed-link-target', text: escapeUnbalanced(target.value, '[', ']') } : syntax(target.value) const spelledTarget: InlineSegment = context.bracketed ? { escaping: 'bracketed-link-target', text: escapeUnbalanced(target.value, '[', ']') } : syntax(target.value)
return success({ segments: [syntax('['), ...inner.value.segments, syntax(']('), spelledTarget, syntax(')')] }) return success({ segments: [{ escaping: 'none', nodes: range, text: '[' }, ...inner.value.segments, syntax(']('), spelledTarget, syntax(')')] })
} }
+8 -3
View File
@@ -14,7 +14,8 @@ export type NodeRange = { first: number; last: number }
export type InlineSegment = export type InlineSegment =
| { emphasis: EmphasisRole; escaping: 'none'; nodes: NodeRange; text: string } | { emphasis: EmphasisRole; escaping: 'none'; nodes: NodeRange; text: string }
| { emphasis?: undefined; escaping: InlineEscaping; text: string } | { emphasis?: undefined; escaping: 'none'; nodes: NodeRange; text: string }
| { emphasis?: undefined; escaping: InlineEscaping; nodes?: undefined; text: string }
export type AssembledLine = { line: string; unspellableRun: NodeRange | undefined } export type AssembledLine = { line: string; unspellableRun: NodeRange | undefined }
@@ -75,8 +76,12 @@ function escape(segments: readonly InlineSegment[], container: LineContainer): A
placements.push(output.length) placements.push(output.length)
output += scan.charAt(index) output += scan.charAt(index)
} }
const line = container === 'paragraph' && opensLinkDefinition(output) ? `\\${output}` : output if (container === 'paragraph' && opensLinkDefinition(output)) {
return { line, unspellableRun: unspellableRun(segments, output, placements) } const opener = segments[0]?.nodes
if (opener !== undefined) return { line: output, unspellableRun: opener }
return { line: `\\${output}`, unspellableRun: unspellableRun(segments, output, placements) }
}
return { line: output, unspellableRun: unspellableRun(segments, output, placements) }
} }
function escapedIndexes(scan: string, escapings: readonly InlineEscaping[], container: LineContainer): Set<number> { function escapedIndexes(scan: string, escapings: readonly InlineEscaping[], container: LineContainer): Set<number> {
+16 -2
View File
@@ -59,6 +59,18 @@ proves 12, 13 spells 11's gaps in 12's grammar, and 12 rewrites code 4b and 4c c
- [ ] **4.3 — The markdown property.** Generated markdown through `markdownToAdf` never throws, - [ ] **4.3 — The markdown property.** Generated markdown through `markdownToAdf` never throws,
and the runs fit the budget; where it parses and `adfToMarkdown` spells the result, that and the runs fit the budget; where it parses and `adfToMarkdown` spells the result, that
spelling parses and emits to itself byte for byte (§2). spelling parses and emits to itself byte for byte (§2).
**Settled** (the maintainer, 2026-09-15): the generators and run parameters 4.2 and 4.3
share live in one test-only module in `src/`, kept out of the build and coverage, with
10c's properties as its third user. Under the gate seed the property asserts floors on the
runs reaching the fixpoint and on the directive-shaped ones. It lands when hunts of several
hundred thousand runs per engine pass clean, since the breaks hit once per ~150,000 runs,
past a 10,000-run bar. Two breaks, both shipped in `0.1.0`, are fixed inside it. An escaped
backtick closed an earlier lone backtick's code span, since CommonMark reads no escape
inside one: a backtick run now escapes whole, and a lone backtick escapes wherever an
escaped one follows it in the same inline content. A paragraph's opening read as a link
reference definition across a `]` the emitter spelled: the emitter now escapes the opening
`[` exactly when the parser's own definition reader accepts the paragraph, and a link
opening it rides the carry until 13b.
- [x] **4.4 — The real payloads.** - [x] **4.4 — The real payloads.**
- [ ] **4b — The block walk's retry (`0.2.0`).** `emitBlock` walks a subtree twice wherever - [ ] **4b — The block walk's retry (`0.2.0`).** `emitBlock` walks a subtree twice wherever
`readableBlock` reads it whole and then gives up — a list item whose first line reads back `readableBlock` reads it whole and then gives up — a list item whose first line reads back
@@ -237,7 +249,7 @@ proves 12, 13 spells 11's gaps in 12's grammar, and 12 rewrites code 4b and 4c c
`src/adf/block-directives.ts` + `inline-directives.ts`, `src/markdown/`'s `src/adf/block-directives.ts` + `inline-directives.ts`, `src/markdown/`'s
`directive-syntax.ts`, `opaque-carry.ts` and the `emit/` + `parse/` readers, every corpus `directive-syntax.ts`, `opaque-carry.ts` and the `emit/` + `parse/` readers, every corpus
fixture (round-trip, normalization and `errors/`), the prose reader over `spec/flavour.md`, fixture (round-trip, normalization and `errors/`), the prose reader over `spec/flavour.md`,
and the README's examples. the markdown property's generator, and the README's examples.
**Settled** (the maintainer, 2026-09-13): **Settled** (the maintainer, 2026-09-13):
- A line opening `!adf:name` is a block line when a space or the line's end follows the name, - A line opening `!adf:name` is a block line when a space or the line's end follows the name,
and a paragraph when `[` or `{` does. Claiming stays syntactic and structure comes from the and a paragraph when `[` or `{` does. Claiming stays syntactic and structure comes from the
@@ -292,7 +304,9 @@ proves 12, 13 spells 11's gaps in 12's grammar, and 12 rewrites code 4b and 4c c
exceptions it cures re-derived, and the README's code table and its "not every document exceptions it cures re-derived, and the README's code table and its "not every document
converts back" guarantee following; the gap list is empty. A round-trip fixture holds the converts back" guarantee following; the gap list is empty. A round-trip fixture holds the
shape 4.2's review left refused until then: an autolink-shaped link under a directive mark shape 4.2's review left refused until then: an autolink-shaped link under a directive mark
whose href holds `\:name{`. whose href holds `\:name{`. A link opening a paragraph whose opening reads as a link
reference definition, which 4.3 leaves riding the carry, takes the directive link too, with
its round-trip fixture (the maintainer, 2026-09-15).
## The ADF inventory to cover ## The ADF inventory to cover
+1 -1
View File
@@ -27,6 +27,6 @@
"forceConsistentCasingInFileNames": true, "forceConsistentCasingInFileNames": true,
"skipLibCheck": true "skipLibCheck": true
}, },
"exclude": ["src/**/*.test.ts", "src/property-generators.ts"], "exclude": ["src/**/*.test.ts", "src/property-harness.ts"],
"include": ["src"] "include": ["src"]
} }