larvit/adf-codec
3
0
Fork 0
Code Pull Requests 1 Actions Activity

Corpus 1d1: the CommonMark subset pairs, and the 1d split #6

Merged
lilleman merged 4 commits from corpus-start into main 2026-08-24 12:54:02 +02:00
Conversation 0 Commits 4 Files Changed 37
+925 -1
32 changed files with 925 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 7195cd9f49 - Show all commits
corpus/README.md
+8
View File
@@ -0,0 +1,8 @@
# The corpus
`<name>.json` is an ADF document; `<name>.md` is the markdown `adfToMarkdown` must emit for it,
byte for byte including the trailing newline, and that `markdownToAdf` must read back to that same
document (AGENTS.md §2). Directories mirror `spec/flavour.md`'s sections.
The JSON is editor-normal — empty `attrs`, `marks` and `content` as the absent key, adjacent
identical-mark text nodes merged — two-space indent, keys sorted.
corpus/canonical-form/blockquote.json
+29
View File
@@ -0,0 +1,29 @@
{
"content": [
{
"content": [
{
"content": [
{
"text": "Ship it.",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Then tell them.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "blockquote"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/blockquote.md
+3
View File
@@ -0,0 +1,3 @@
> Ship it.
>
> Then tell them.
corpus/canonical-form/bullet-list.json
+72
View File
@@ -0,0 +1,72 @@
{
"content": [
{
"content": [
{
"content": [
{
"content": [
{
"text": "Bolt M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
},
{
"content": [
{
"content": [
{
"text": "Nut M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
},
{
"content": [
{
"content": [
{
"text": "Washer M8",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"content": [
{
"content": [
{
"text": "Fibre",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "bulletList"
}
],
"type": "listItem"
}
],
"type": "bulletList"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/bullet-list.md
+4
View File
@@ -0,0 +1,4 @@
- Bolt M8
- Nut M8
- Washer M8
- Fibre
corpus/canonical-form/code-block-backtick-run.json
+15
View File
@@ -0,0 +1,15 @@
{
"content": [
{
"content": [
{
"text": "```\ncode\n```",
"type": "text"
}
],
"type": "codeBlock"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/code-block-backtick-run.md
+5
View File
@@ -0,0 +1,5 @@
````
```
code
```
````
corpus/canonical-form/code-block.json
+18
View File
@@ -0,0 +1,18 @@
{
"content": [
{
"attrs": {
"language": "sql"
},
"content": [
{
"text": "SELECT id\nFROM part\nWHERE qty > 0;",
"type": "text"
}
],
"type": "codeBlock"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/code-block.md
+5
View File
@@ -0,0 +1,5 @@
```sql
SELECT id
FROM part
WHERE qty > 0;
```
corpus/canonical-form/code-span.json
+118
View File
@@ -0,0 +1,118 @@
{
"content": [
{
"content": [
{
"text": "Run ",
"type": "text"
},
{
"marks": [
{
"type": "code"
}
],
"text": "npm ci",
"type": "text"
},
{
"text": " first.",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Nested backticks: ",
"type": "text"
},
{
"marks": [
{
"type": "code"
}
],
"text": "run `date` twice",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "A span holding ",
"type": "text"
},
{
"marks": [
{
"type": "code"
}
],
"text": "`code`",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Held: ",
"type": "text"
},
{
"marks": [
{
"type": "code"
}
],
"text": " spaced ",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Three: ",
"type": "text"
},
{
"marks": [
{
"type": "code"
}
],
"text": " ",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Literal: ",
"type": "text"
},
{
"marks": [
{
"type": "code"
}
],
"text": "~~not strike~~",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/code-span.md
+11
View File
@@ -0,0 +1,11 @@
Run `npm ci` first.
Nested backticks: ``run `date` twice``
A span holding `` `code` ``
Held: ` spaced `
Three: ` `
Literal: `~~not strike~~`
corpus/canonical-form/em-strong-strike.json
+76
View File
@@ -0,0 +1,76 @@
{
"content": [
{
"content": [
{
"text": "Read the ",
"type": "text"
},
{
"marks": [
{
"type": "em"
}
],
"text": "manual",
"type": "text"
},
{
"text": " before ",
"type": "text"
},
{
"marks": [
{
"type": "strong"
}
],
"text": "wiring",
"type": "text"
},
{
"text": " the ",
"type": "text"
},
{
"marks": [
{
"type": "strike"
}
],
"text": "relay",
"type": "text"
},
{
"text": ".",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "An un",
"type": "text"
},
{
"marks": [
{
"type": "em"
}
],
"text": "real",
"type": "text"
},
{
"text": "istic goal.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/em-strong-strike.md
+3
View File
@@ -0,0 +1,3 @@
Read the _manual_ before **wiring** the ~~relay~~.
An un*real*istic goal.
corpus/canonical-form/hard-break.json
+53
View File
@@ -0,0 +1,53 @@
{
"content": [
{
"content": [
{
"text": "Line one",
"type": "text"
},
{
"type": "hardBreak"
},
{
"text": "Line two",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Trailing",
"type": "text"
},
{
"type": "hardBreak"
}
],
"type": "paragraph"
},
{
"attrs": {
"level": 2
},
"content": [
{
"text": "Two",
"type": "text"
},
{
"type": "hardBreak"
},
{
"text": "lines",
"type": "text"
}
],
"type": "heading"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/hard-break.md
+6
View File
@@ -0,0 +1,6 @@
Line one\
Line two
Trailing:hardBreak{}
## Two:hardBreak{}lines
corpus/canonical-form/heading.json
+78
View File
@@ -0,0 +1,78 @@
{
"content": [
{
"attrs": {
"level": 1
},
"content": [
{
"text": "Assembly",
"type": "text"
}
],
"type": "heading"
},
{
"attrs": {
"level": 2
},
"content": [
{
"text": "Parts",
"type": "text"
}
],
"type": "heading"
},
{
"attrs": {
"level": 3
},
"content": [
{
"text": "Fasteners",
"type": "text"
}
],
"type": "heading"
},
{
"attrs": {
"level": 4
},
"content": [
{
"text": "Bolts",
"type": "text"
}
],
"type": "heading"
},
{
"attrs": {
"level": 5
},
"content": [
{
"text": "Sizes",
"type": "text"
}
],
"type": "heading"
},
{
"attrs": {
"level": 6
},
"content": [
{
"text": "M8",
"type": "text"
}
],
"type": "heading"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/heading.md
+11
View File
@@ -0,0 +1,11 @@
# Assembly
## Parts
### Fasteners
#### Bolts
##### Sizes
###### M8
corpus/canonical-form/link.json
+116
View File
@@ -0,0 +1,116 @@
{
"content": [
{
"content": [
{
"text": "See ",
"type": "text"
},
{
"marks": [
{
"attrs": {
"href": "https://example.com/changelog"
},
"type": "link"
}
],
"text": "the changelog",
"type": "text"
},
{
"text": ".",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"marks": [
{
"attrs": {
"href": "https://example.com/"
},
"type": "link"
}
],
"text": "https://example.com/",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Read ",
"type": "text"
},
{
"marks": [
{
"attrs": {
"href": "https://example.com/guide",
"title": "Setup guide"
},
"type": "link"
}
],
"text": "the guide",
"type": "text"
},
{
"text": ".",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Open ",
"type": "text"
},
{
"marks": [
{
"attrs": {
"href": "https://example.com/the plan.pdf"
},
"type": "link"
}
],
"text": "the plan",
"type": "text"
},
{
"text": ".",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"marks": [
{
"attrs": {
"href": "/parts/m8"
},
"type": "link"
}
],
"text": "/parts/m8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/link.md
+9
View File
@@ -0,0 +1,9 @@
See [the changelog](https://example.com/changelog).
<https://example.com/>
Read [the guide](https://example.com/guide "Setup guide").
Open [the plan](<https://example.com/the plan.pdf>).
[/parts/m8](/parts/m8)
corpus/canonical-form/ordered-list-order.json
+65
View File
@@ -0,0 +1,65 @@
{
"content": [
{
"attrs": {
"order": 9
},
"content": [
{
"content": [
{
"content": [
{
"text": "Bolt M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
},
{
"content": [
{
"content": [
{
"text": "Nut M8",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Zinc-plated.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
},
{
"content": [
{
"content": [
{
"text": "Washer M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "orderedList"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/ordered-list-order.md
+5
View File
@@ -0,0 +1,5 @@
9. Bolt M8
10. Nut M8
Zinc-plated.
11. Washer M8
corpus/canonical-form/ordered-list.json
+53
View File
@@ -0,0 +1,53 @@
{
"content": [
{
"content": [
{
"content": [
{
"content": [
{
"text": "Bolt M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
},
{
"content": [
{
"content": [
{
"text": "Nut M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
},
{
"content": [
{
"content": [
{
"text": "Washer M8",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "listItem"
}
],
"type": "orderedList"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/ordered-list.md
+3
View File
@@ -0,0 +1,3 @@
1. Bolt M8
2. Nut M8
3. Washer M8
corpus/canonical-form/paragraph-empty.json
+27
View File
@@ -0,0 +1,27 @@
{
"content": [
{
"content": [
{
"text": "First.",
"type": "text"
}
],
"type": "paragraph"
},
{
"type": "paragraph"
},
{
"content": [
{
"text": "Second.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/paragraph-empty.md
+5
View File
@@ -0,0 +1,5 @@
First.
::paragraph
Second.
corpus/canonical-form/paragraph.json
+24
View File
@@ -0,0 +1,24 @@
{
"content": [
{
"content": [
{
"text": "The converter emits every paragraph on a single line, however long it runs, because a soft line break in input carries no meaning ADF can hold.",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "Blocks are separated by exactly one blank line.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/paragraph.md
+3
View File
@@ -0,0 +1,3 @@
The converter emits every paragraph on a single line, however long it runs, because a soft line break in input carries no meaning ADF can hold.
Blocks are separated by exactly one blank line.
corpus/canonical-form/rule.json
+27
View File
@@ -0,0 +1,27 @@
{
"content": [
{
"content": [
{
"text": "Before.",
"type": "text"
}
],
"type": "paragraph"
},
{
"type": "rule"
},
{
"content": [
{
"text": "After.",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/rule.md
+5
View File
@@ -0,0 +1,5 @@
Before.
---
After.
corpus/canonical-form/text-escaping.json
+42
View File
@@ -0,0 +1,42 @@
{
"content": [
{
"content": [
{
"text": "# Not a heading",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "- not a bullet",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "2 * 3 * 4 = 24",
"type": "text"
}
],
"type": "paragraph"
},
{
"content": [
{
"text": "snake_case_name",
"type": "text"
}
],
"type": "paragraph"
}
],
"type": "doc",
"version": 1
}
corpus/canonical-form/text-escaping.md
+7
View File
@@ -0,0 +1,7 @@
\# Not a heading
\- not a bullet
2 * 3 * 4 = 24
snake_case_name
todo.md
+19 -1
View File
@@ -21,7 +21,25 @@ detail is settled at its own milestone.
payloads for external-URL support — if it exists, revisit the media section's payloads for external-URL support — if it exists, revisit the media section's
mid-text-image error and its "no slot" ground. mid-text-image error and its "no slot" ground.
- [ ] **1d — Corpus start** (§10): checked-in ADF ↔ canonical-markdown fixture pairs per spec'd - [ ] **1d — Corpus start** (§10): checked-in ADF ↔ canonical-markdown fixture pairs per spec'd
node. node, in `corpus/`, one directory per `spec/flavour.md` section.
- [ ] **1d1 — Canonical form**: the plain-CommonMark subset — blockquote, bulletList,
codeBlock, heading, orderedList, paragraph, rule, listItem, hardBreak, text, code spans,
and the `code`, `em`, `link`, `strike` and `strong` marks.
- [ ] **1d2 — Block nodes**: panel, expand/nestedExpand, the media family and the CommonMark
image shape, both table forms, task and decision lists, layout, extensions, syncBlock —
with the reserved `marks` attribute and the fence lengths nesting forces.
- [ ] **1d3 — Inline nodes and marks**: date, emoji, inlineCard, mediaInline, mention, status;
border, subsup, textColor, underline; the content slot's `text` attribute and the
`:text{text="…"}` whitespace spelling.
- [ ] **1d4 — Opaque carry** (§3): an unknown node in both positions, the reserved `adf` info
string, and the `codeBlock` whose language is `adf`.
- [ ] **1d5 — Carve-outs and combinations**: the three carve-outs and their escapes, mark
nesting order and the runs a carry breaks, attribute canonicalization, and documents
combining nodes rather than isolating one.
- [ ] **1d6 — Input normalization**: one-way markdown→ADF fixtures, not pairs — setext
headings, indented code, loose lists, `*`/`+` bullets, entity references, soft wraps.
- [ ] **1d7 — Error input**: also one-way, a markdown input per named error. Waits on milestone
3 naming them; 1d's pairs are valid documents only.
- [ ] **2 — `adfToMarkdown`.** First real code — decide here where §10's coverage check lives. - [ ] **2 — `adfToMarkdown`.** First real code — decide here where §10's coverage check lives.
- [ ] **3 — `markdownToAdf`.** The CommonMark parser is the largest single component. The raw-HTML - [ ] **3 — `markdownToAdf`.** The CommonMark parser is the largest single component. The raw-HTML
element mapping is empty until milestone 6, so at `0.1.0` every raw-HTML construct in input element mapping is empty until milestone 6, so at `0.1.0` every raw-HTML construct in input