Corpus 1d1: the CommonMark subset pairs, and the 1d split #6

Merged
lilleman merged 4 commits from corpus-start into main 2026-08-24 12:54:02 +02:00
37 changed files with 84 additions and 75 deletions
Showing only changes of commit fc92bf8bf5 - Show all commits
+12 -5
View File
@@ -1,8 +1,15 @@
# The corpus
`<name>.json` is an ADF document; `<name>.md` is the markdown `adfToMarkdown` must emit for it,
byte for byte including the trailing newline, and that `markdownToAdf` must read back to that same
document (AGENTS.md §2). Directories mirror `spec/flavour.md`'s sections.
One directory per contract kind:
The JSON is editor-normal — empty `attrs`, `marks` and `content` as the absent key, adjacent
identical-mark text nodes merged — two-space indent, keys sorted.
- `round-trip/``<name>.json` + `<name>.md`: the markdown `adfToMarkdown` must emit for that
document, byte for byte, and that `markdownToAdf` must read back to it (AGENTS.md §2). Grouped
by node family: `commonmark-subset/`, `block-nodes/`, `inline-nodes/`, `opaque-carry/`,
`combinations/`.
- `normalization/``<name>.md` + `<name>.json`: markdown input, and the document
`markdownToAdf` must build from it. One-way; the markdown is not canonical.
- `errors/``<name>.md` + `<name>.error`: markdown input that must not convert.
- `real-payloads/``<name>.json`: sanitized live ADF, round-tripped ADF→markdown→ADF. No
expected markdown.
JSON is editor-normal (AGENTS.md §2), two-space indent, keys sorted.
-53
View File
@@ -1,53 +0,0 @@
{
"content": [
{
"content": [
{
"content": [
{
"content": [
{
"text": "Bolt M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
},
{
"content": [
{
"content": [
{
"text": "Nut M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
},
{
"content": [
{
"content": [
{
"text": "Washer M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "orderedList"
}
],
"type": "doc",
"version": 1
}
-3
View File
@@ -1,3 +0,0 @@
1. Bolt M8
2. Nut M8
3. Washer M8
@@ -0,0 +1,18 @@
{
"content": [
{
"attrs": {
"language": "bash"
},
"content": [
{
"text": "echo \"```\" >> notes.md",
"type": "text"
}
],
"type": "codeBlock"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1,3 @@
````bash
echo "```" >> notes.md
````
@@ -1,6 +1,9 @@
{
"content": [
{
"attrs": {
"language": "markdown"
},
"content": [
{
"text": "```\ncode\n```",
@@ -1,4 +1,4 @@
````
````markdown
```
code
```
@@ -0,0 +1,4 @@
{
"type": "doc",
"version": 1
}
@@ -35,6 +35,15 @@
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "*not emphasis*",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
@@ -5,3 +5,5 @@
2 * 3 * 4 = 24
snake_case_name
\*not emphasis*
+7 -3
View File
@@ -23,7 +23,8 @@ normalizes to it through the round-trip.
- Blockquotes prefix lines with `> `; a blank line inside a blockquote is a bare `>`.
- ATX headings (`#``######`); setext input normalizes to ATX.
- Code fences ``` with the node's language as info string, the fence lengthened past any backtick
run in the content; indented-code input normalizes to fences.
run in the content — the longest run anywhere plus one, counting mid-line runs no closing fence
could match; indented-code input normalizes to fences.
- Code spans: a backtick string one longer than the longest backtick run in the text, the text
padded with one space on each side where it begins or ends with a backtick, or begins and ends
with a space without being all spaces. The content is literal — inline parsing does not see
@@ -37,8 +38,11 @@ normalizes to it through the round-trip.
CommonMark autolink (absolute URI).
- Paragraphs on one line — no soft wrapping; a soft line break in input becomes a single space.
- Entity references in input decode to their characters; output backslash-escapes only where text
would otherwise parse as syntax.
- Blocks separated by one blank line, no trailing whitespace, single trailing newline.
would otherwise parse as syntax: escape the leading delimiter of a construct that would otherwise
open, re-scan from there, and repeat — with the opener literal the closer parses as text, so
`*not emphasis*` is `\*not emphasis*`, one backslash.
- Blocks separated by one blank line, no trailing whitespace, single trailing newline; a document
with no blocks is the empty string.
## Directives
+25 -10
View File
@@ -20,11 +20,18 @@ detail is settled at its own milestone.
escape-based, never literal, since pipe cells trim and pad. At `mediaInline`, check real
payloads for external-URL support — if it exists, revisit the media section's
mid-text-image error and its "no slot" ground.
- [ ] **1d — Corpus start** (§10): checked-in ADF ↔ canonical-markdown fixture pairs per spec'd
node, in `corpus/`, one directory per `spec/flavour.md` section.
- [ ] **1d1 — Canonical form**: the plain-CommonMark subset — blockquote, bulletList,
codeBlock, heading, orderedList, paragraph, rule, listItem, hardBreak, text, code spans,
and the `code`, `em`, `link`, `strike` and `strong` marks.
- [ ] **1d — Corpus start** (§10): checked-in fixtures per spec'd node, in `corpus/`, one
directory per contract kind (`corpus/README.md`).
**Blocked on the maintainer** (§15), not to be guessed: Canonical form has no totality
guard — a `paragraph`, `heading`, `blockquote` or `codeBlock` carrying a `localId`, or a
`blockquote` carrying marks, has no spelling that keeps it, and picking one (directive
sections for the six CommonMark block nodes, or the opaque carry) is a permanent format
decision (§8). Its three collision sites stay out of the corpus until then: an `orderedList`
starting at 1, a `codeBlock` whose info string is empty, and `media` with an empty `alt`
each a choice between the absent attribute and the empty value.
- [ ] **1d1 — The CommonMark subset**: blockquote, bulletList, codeBlock, heading, orderedList,
paragraph, rule, listItem, hardBreak, text, code spans, and the `code`, `em`, `link`,
`strike` and `strong` marks — one mark per text node; nesting is 1d5's.
- [ ] **1d2 — Block nodes**: panel, expand/nestedExpand, the media family and the CommonMark
image shape, both table forms, task and decision lists, layout, extensions, syncBlock —
with the reserved `marks` attribute and the fence lengths nesting forces.
@@ -38,14 +45,22 @@ detail is settled at its own milestone.
combining nodes rather than isolating one.
- [ ] **1d6 — Input normalization**: one-way markdown→ADF fixtures, not pairs — setext
headings, indented code, loose lists, `*`/`+` bullets, entity references, soft wraps.
- [ ] **1d7 — Error input**: also one-way, a markdown input per named error. Waits on milestone
3 naming them; 1d's pairs are valid documents only.
- [ ] **2 — `adfToMarkdown`.** First real code — decide here where §10's coverage check lives.
- [ ] **1d7 — Error input**: also one-way, a markdown input per named error, asserting only
that conversion fails — malformed directives, the image gap, a claimed pipe-table line
that does not parse, the content slot, raw HTML with no mapping. Which error each returns
is pinned at milestone 3, where they are named.
- [ ] **2 — `adfToMarkdown`.** First real code — decide here where §10's coverage check lives, and
gate that every `corpus/**/*.json` re-serializes to itself under the library's own canonical
serializer: one implementation, keys sorted, two spellings — two-space indent for the corpus
files and the block carry's body, compact for the inline carry.
- [ ] **3 — `markdownToAdf`.** The CommonMark parser is the largest single component. The raw-HTML
element mapping is empty until milestone 6, so at `0.1.0` every raw-HTML construct in input
is an error result.
is an error result. The CommonMark spec suite runs against it from here (§10), and
`corpus/errors/` gains the error each fixture must return (1d7).
- [ ] **4 — Round-trip property tests** over the corpus, both ways — the thing that proves 2 and
3. Generators emit editor-normal ADF (§2).
3. Generators emit editor-normal ADF (§2). Real sanitized ADF from live Atlassian APIs lands
here too (§10), in `corpus/real-payloads/`: an ADF→markdown→ADF check with no expected
markdown, the payloads supplied by the maintainer.
- [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret,
the repo made public first (§6). `0.1.0` is the markdown round-trip: both markdown
directions, the types, `isAdfDocument`. The build lands here: a build tsconfig emitting JS