Corpus 1d1: the CommonMark subset pairs, and the 1d split #6

Merged
lilleman merged 4 commits from corpus-start into main 2026-08-24 12:54:02 +02:00
37 changed files with 84 additions and 75 deletions
Showing only changes of commit fc92bf8bf5 - Show all commits
+12 -5
View File
@@ -1,8 +1,15 @@
# The corpus # The corpus
`<name>.json` is an ADF document; `<name>.md` is the markdown `adfToMarkdown` must emit for it, One directory per contract kind:
byte for byte including the trailing newline, and that `markdownToAdf` must read back to that same
document (AGENTS.md §2). Directories mirror `spec/flavour.md`'s sections.
The JSON is editor-normal — empty `attrs`, `marks` and `content` as the absent key, adjacent - `round-trip/``<name>.json` + `<name>.md`: the markdown `adfToMarkdown` must emit for that
identical-mark text nodes merged — two-space indent, keys sorted. document, byte for byte, and that `markdownToAdf` must read back to it (AGENTS.md §2). Grouped
by node family: `commonmark-subset/`, `block-nodes/`, `inline-nodes/`, `opaque-carry/`,
`combinations/`.
- `normalization/``<name>.md` + `<name>.json`: markdown input, and the document
`markdownToAdf` must build from it. One-way; the markdown is not canonical.
- `errors/``<name>.md` + `<name>.error`: markdown input that must not convert.
- `real-payloads/``<name>.json`: sanitized live ADF, round-tripped ADF→markdown→ADF. No
expected markdown.
JSON is editor-normal (AGENTS.md §2), two-space indent, keys sorted.
-53
View File
@@ -1,53 +0,0 @@
{
"content": [
{
"content": [
{
"content": [
{
"content": [
{
"text": "Bolt M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
},
{
"content": [
{
"content": [
{
"text": "Nut M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
},
{
"content": [
{
"content": [
{
"text": "Washer M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "orderedList"
}
],
"type": "doc",
"version": 1
}
-3
View File
@@ -1,3 +0,0 @@
1. Bolt M8
2. Nut M8
3. Washer M8
@@ -0,0 +1,18 @@
{
"content": [
{
"attrs": {
"language": "bash"
},
"content": [
{
"text": "echo \"```\" >> notes.md",
"type": "text"
}
],
"type": "codeBlock"
}
],
"type": "doc",
"version": 1
}
@@ -0,0 +1,3 @@
````bash
echo "```" >> notes.md
````
@@ -1,6 +1,9 @@
{ {
"content": [ "content": [
{ {
"attrs": {
"language": "markdown"
},
"content": [ "content": [
{ {
"text": "```\ncode\n```", "text": "```\ncode\n```",
@@ -1,4 +1,4 @@
```` ````markdown
``` ```
code code
``` ```
@@ -0,0 +1,4 @@
{
"type": "doc",
"version": 1
}
@@ -35,6 +35,15 @@
} }
], ],
"type": "paragraph" "type": "paragraph"
},
{
"content": [
{
"text": "*not emphasis*",
"type": "text"
}
],
"type": "paragraph"
} }
], ],
"type": "doc", "type": "doc",
@@ -5,3 +5,5 @@
2 * 3 * 4 = 24 2 * 3 * 4 = 24
snake_case_name snake_case_name
\*not emphasis*
+7 -3
View File
@@ -23,7 +23,8 @@ normalizes to it through the round-trip.
- Blockquotes prefix lines with `> `; a blank line inside a blockquote is a bare `>`. - Blockquotes prefix lines with `> `; a blank line inside a blockquote is a bare `>`.
- ATX headings (`#``######`); setext input normalizes to ATX. - ATX headings (`#``######`); setext input normalizes to ATX.
- Code fences ``` with the node's language as info string, the fence lengthened past any backtick - Code fences ``` with the node's language as info string, the fence lengthened past any backtick
run in the content; indented-code input normalizes to fences. run in the content — the longest run anywhere plus one, counting mid-line runs no closing fence
could match; indented-code input normalizes to fences.
- Code spans: a backtick string one longer than the longest backtick run in the text, the text - Code spans: a backtick string one longer than the longest backtick run in the text, the text
padded with one space on each side where it begins or ends with a backtick, or begins and ends padded with one space on each side where it begins or ends with a backtick, or begins and ends
with a space without being all spaces. The content is literal — inline parsing does not see with a space without being all spaces. The content is literal — inline parsing does not see
@@ -37,8 +38,11 @@ normalizes to it through the round-trip.
CommonMark autolink (absolute URI). CommonMark autolink (absolute URI).
- Paragraphs on one line — no soft wrapping; a soft line break in input becomes a single space. - Paragraphs on one line — no soft wrapping; a soft line break in input becomes a single space.
- Entity references in input decode to their characters; output backslash-escapes only where text - Entity references in input decode to their characters; output backslash-escapes only where text
would otherwise parse as syntax. would otherwise parse as syntax: escape the leading delimiter of a construct that would otherwise
- Blocks separated by one blank line, no trailing whitespace, single trailing newline. open, re-scan from there, and repeat — with the opener literal the closer parses as text, so
`*not emphasis*` is `\*not emphasis*`, one backslash.
- Blocks separated by one blank line, no trailing whitespace, single trailing newline; a document
with no blocks is the empty string.
## Directives ## Directives
+25 -10
View File
@@ -20,11 +20,18 @@ detail is settled at its own milestone.
escape-based, never literal, since pipe cells trim and pad. At `mediaInline`, check real escape-based, never literal, since pipe cells trim and pad. At `mediaInline`, check real
payloads for external-URL support — if it exists, revisit the media section's payloads for external-URL support — if it exists, revisit the media section's
mid-text-image error and its "no slot" ground. mid-text-image error and its "no slot" ground.
- [ ] **1d — Corpus start** (§10): checked-in ADF ↔ canonical-markdown fixture pairs per spec'd - [ ] **1d — Corpus start** (§10): checked-in fixtures per spec'd node, in `corpus/`, one
node, in `corpus/`, one directory per `spec/flavour.md` section. directory per contract kind (`corpus/README.md`).
- [ ] **1d1 — Canonical form**: the plain-CommonMark subset — blockquote, bulletList, **Blocked on the maintainer** (§15), not to be guessed: Canonical form has no totality
codeBlock, heading, orderedList, paragraph, rule, listItem, hardBreak, text, code spans, guard — a `paragraph`, `heading`, `blockquote` or `codeBlock` carrying a `localId`, or a
and the `code`, `em`, `link`, `strike` and `strong` marks. `blockquote` carrying marks, has no spelling that keeps it, and picking one (directive
sections for the six CommonMark block nodes, or the opaque carry) is a permanent format
decision (§8). Its three collision sites stay out of the corpus until then: an `orderedList`
starting at 1, a `codeBlock` whose info string is empty, and `media` with an empty `alt`
each a choice between the absent attribute and the empty value.
- [ ] **1d1 — The CommonMark subset**: blockquote, bulletList, codeBlock, heading, orderedList,
paragraph, rule, listItem, hardBreak, text, code spans, and the `code`, `em`, `link`,
`strike` and `strong` marks — one mark per text node; nesting is 1d5's.
- [ ] **1d2 — Block nodes**: panel, expand/nestedExpand, the media family and the CommonMark - [ ] **1d2 — Block nodes**: panel, expand/nestedExpand, the media family and the CommonMark
image shape, both table forms, task and decision lists, layout, extensions, syncBlock — image shape, both table forms, task and decision lists, layout, extensions, syncBlock —
with the reserved `marks` attribute and the fence lengths nesting forces. with the reserved `marks` attribute and the fence lengths nesting forces.
@@ -38,14 +45,22 @@ detail is settled at its own milestone.
combining nodes rather than isolating one. combining nodes rather than isolating one.
- [ ] **1d6 — Input normalization**: one-way markdown→ADF fixtures, not pairs — setext - [ ] **1d6 — Input normalization**: one-way markdown→ADF fixtures, not pairs — setext
headings, indented code, loose lists, `*`/`+` bullets, entity references, soft wraps. headings, indented code, loose lists, `*`/`+` bullets, entity references, soft wraps.
- [ ] **1d7 — Error input**: also one-way, a markdown input per named error. Waits on milestone - [ ] **1d7 — Error input**: also one-way, a markdown input per named error, asserting only
3 naming them; 1d's pairs are valid documents only. that conversion fails — malformed directives, the image gap, a claimed pipe-table line
- [ ] **2 — `adfToMarkdown`.** First real code — decide here where §10's coverage check lives. that does not parse, the content slot, raw HTML with no mapping. Which error each returns
is pinned at milestone 3, where they are named.
- [ ] **2 — `adfToMarkdown`.** First real code — decide here where §10's coverage check lives, and
gate that every `corpus/**/*.json` re-serializes to itself under the library's own canonical
serializer: one implementation, keys sorted, two spellings — two-space indent for the corpus
files and the block carry's body, compact for the inline carry.
- [ ] **3 — `markdownToAdf`.** The CommonMark parser is the largest single component. The raw-HTML - [ ] **3 — `markdownToAdf`.** The CommonMark parser is the largest single component. The raw-HTML
element mapping is empty until milestone 6, so at `0.1.0` every raw-HTML construct in input element mapping is empty until milestone 6, so at `0.1.0` every raw-HTML construct in input
is an error result. is an error result. The CommonMark spec suite runs against it from here (§10), and
`corpus/errors/` gains the error each fixture must return (1d7).
- [ ] **4 — Round-trip property tests** over the corpus, both ways — the thing that proves 2 and - [ ] **4 — Round-trip property tests** over the corpus, both ways — the thing that proves 2 and
3. Generators emit editor-normal ADF (§2). 3. Generators emit editor-normal ADF (§2). Real sanitized ADF from live Atlassian APIs lands
here too (§10), in `corpus/real-payloads/`: an ADF→markdown→ADF check with no expected
markdown, the payloads supplied by the maintainer.
- [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret, - [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret,
the repo made public first (§6). `0.1.0` is the markdown round-trip: both markdown the repo made public first (§6). `0.1.0` is the markdown round-trip: both markdown
directions, the types, `isAdfDocument`. The build lands here: a build tsconfig emitting JS directions, the types, `isAdfDocument`. The build lands here: a build tsconfig emitting JS