Merge pull request 'Corpus 2e1: the carve-out escapes and the claimed line' (#19) from carve-outs into main

corpus/README.md

@@ -4,7 +4,7 @@ One directory per contract kind, each landing with its milestone:
 
 - `round-trip/` — `<name>.json` + `<name>.md`: the markdown `adfToMarkdown` must emit for that
   document, byte for byte, and that `markdownToAdf` must read back to it (AGENTS.md §2). Grouped
-  by node family.
+  by what the fixture exercises.
 - `normalization/` — `<name>.md` + `<name>.json`: markdown input, and the document
   `markdownToAdf` must build from it. One-way; the markdown is not canonical.
 - `errors/` — `<name>.md`: markdown input that must not convert. A `<name>.error` beside it

corpus/round-trip/combinations/carve-out-directive.json

@@ -0,0 +1,73 @@
+{
+  "content": [
+    {
+      "content": [
+        {
+          "text": "Write :mention[@Mikael]{id=5b10a2} to tag someone.",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    },
+    {
+      "content": [
+        {
+          "text": "::syncBlock {localId=x}",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    },
+    {
+      "content": [
+        {
+          "text": ":::panel warning",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    },
+    {
+      "content": [
+        {
+          "text": "Stand-up at 10:30, shipped :tada:, {maxLevel} deep, a ::syncBlock mid-line.",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    },
+    {
+      "content": [
+        {
+          "attrs": {
+            "text": "In review"
+          },
+          "type": "status"
+        },
+        {
+          "text": "{color} decides.",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    },
+    {
+      "content": [
+        {
+          "text": ":status[nope]{color=red} vs ",
+          "type": "text"
+        },
+        {
+          "attrs": {
+            "color": "red",
+            "text": "yep"
+          },
+          "type": "status"
+        }
+      ],
+      "type": "paragraph"
+    }
+  ],
+  "type": "doc",
+  "version": 1
+}

corpus/round-trip/combinations/carve-out-directive.md

@@ -0,0 +1,11 @@
+Write \:mention[@Mikael]{id=5b10a2} to tag someone.
+
+\::syncBlock {localId=x}
+
+\:::panel warning
+
+Stand-up at 10:30, shipped :tada:, {maxLevel} deep, a ::syncBlock mid-line.
+
+:status[In review]\{color} decides.
+
+\:status[nope]{color=red} vs :status[yep]{color=red}

corpus/round-trip/combinations/carve-out-pipe-table.json

@@ -0,0 +1,60 @@
+{
+  "content": [
+    {
+      "content": [
+        {
+          "text": "| Part | Qty |",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    },
+    {
+      "content": [
+        {
+          "text": "Pass a | b to the shell.",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    },
+    {
+      "content": [
+        {
+          "content": [
+            {
+              "content": [
+                {
+                  "text": "| --- | --- |",
+                  "type": "text"
+                }
+              ],
+              "type": "paragraph"
+            }
+          ],
+          "type": "listItem"
+        }
+      ],
+      "type": "bulletList"
+    },
+    {
+      "attrs": {
+        "panelType": "info"
+      },
+      "content": [
+        {
+          "content": [
+            {
+              "text": "| a | b |",
+              "type": "text"
+            }
+          ],
+          "type": "paragraph"
+        }
+      ],
+      "type": "panel"
+    }
+  ],
+  "type": "doc",
+  "version": 1
+}

corpus/round-trip/combinations/carve-out-pipe-table.md

@@ -0,0 +1,9 @@
+\| Part | Qty |
+
+Pass a | b to the shell.
+
+- \| --- | --- |
+
+:::panel info
+\| a | b |
+:::

corpus/round-trip/combinations/carve-out-strike.json

@@ -0,0 +1,69 @@
+{
+  "content": [
+    {
+      "content": [
+        {
+          "text": "~~not strike~~",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    },
+    {
+      "content": [
+        {
+          "marks": [
+            {
+              "type": "strike"
+            }
+          ],
+          "text": "gone",
+          "type": "text"
+        },
+        {
+          "text": " but ~~kept~~",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    },
+    {
+      "content": [
+        {
+          "text": "A ~ B ~ C",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    },
+    {
+      "content": [
+        {
+          "text": "~",
+          "type": "text"
+        },
+        {
+          "marks": [
+            {
+              "type": "strike"
+            }
+          ],
+          "text": "a",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    },
+    {
+      "content": [
+        {
+          "text": "~~~not~~~",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    }
+  ],
+  "type": "doc",
+  "version": 1
+}

corpus/round-trip/combinations/carve-out-strike.md

@@ -0,0 +1,9 @@
+\~~not strike~~
+
+~~gone~~ but \~~kept~~
+
+A ~ B ~ C
+
+\~~~a~~
+
+\~\~~not~~~

corpus/round-trip/combinations/closing-fence-line.json

@@ -0,0 +1,83 @@
+{
+  "content": [
+    {
+      "attrs": {
+        "panelType": "info"
+      },
+      "content": [
+        {
+          "content": [
+            {
+              "content": [
+                {
+                  "content": [
+                    {
+                      "text": ":::",
+                      "type": "text"
+                    }
+                  ],
+                  "type": "paragraph"
+                }
+              ],
+              "type": "listItem"
+            }
+          ],
+          "type": "bulletList"
+        },
+        {
+          "content": [
+            {
+              "content": [
+                {
+                  "text": ":::",
+                  "type": "text"
+                }
+              ],
+              "type": "paragraph"
+            }
+          ],
+          "type": "blockquote"
+        },
+        {
+          "content": [
+            {
+              "text": ":::",
+              "type": "text"
+            }
+          ],
+          "type": "paragraph"
+        },
+        {
+          "content": [
+            {
+              "text": "::: x",
+              "type": "text"
+            }
+          ],
+          "type": "paragraph"
+        },
+        {
+          "content": [
+            {
+              "text": "::",
+              "type": "text"
+            }
+          ],
+          "type": "paragraph"
+        }
+      ],
+      "type": "panel"
+    },
+    {
+      "content": [
+        {
+          "text": ":::",
+          "type": "text"
+        }
+      ],
+      "type": "paragraph"
+    }
+  ],
+  "type": "doc",
+  "version": 1
+}

corpus/round-trip/combinations/closing-fence-line.md

@@ -0,0 +1,13 @@
+:::panel info
+- \:::
+
+> \:::
+
+\:::
+
+\::: x
+
+\::
+:::
+
+\:::

src/corpus.test.ts

@@ -13,7 +13,7 @@ const corpusRoot = join(dirname(fileURLToPath(import.meta.url)), '..', 'corpus')
 const roundTripRoot = join(corpusRoot, 'round-trip')
 const unspellableRoot = join(corpusRoot, 'unspellable')
 
-const emittingDirectories = ['block-nodes', 'commonmark-subset', 'inline-nodes', 'opaque-carry']
+const emittingDirectories = ['block-nodes', 'combinations', 'commonmark-subset', 'inline-nodes', 'opaque-carry']
 
 function directoryNames(root: string): string[] {
   return readdirSync(root, { withFileTypes: true })
@@ -86,6 +86,8 @@ function fenceNestingFault(markdown: string): string | undefined {
     const run = colons[1]?.length ?? 0
     const enclosing = open[open.length - 1]
     if (colons[2] === '') {
+      if (enclosing === undefined) return `${JSON.stringify(line)} closes no open container`
+      if (run < enclosing) return `${JSON.stringify(line)} is shorter than the ${enclosing} colons it would close`
       open.pop()
       continue
     }
@@ -99,6 +101,8 @@ test('the fence nesting check catches a fence a container cannot hold', () => {
   assert.equal(fenceNestingFault(':::panel info\n- :::panel warning\n  B\n  :::\n:::'), '"- :::panel warning" sits in a container fenced with 3 colons')
   assert.equal(fenceNestingFault(':::panel info\n- - :::panel warning\n    B\n    :::\n:::'), '"- - :::panel warning" sits in a container fenced with 3 colons')
   assert.equal(fenceNestingFault(':::panel info\n10. :::panel warning\n    B\n    :::\n:::'), '"10. :::panel warning" sits in a container fenced with 3 colons')
+  assert.equal(fenceNestingFault('::::panel info\n:::\n::::'), '":::" is shorter than the 4 colons it would close')
+  assert.equal(fenceNestingFault('Text\n:::\n'), '":::" closes no open container')
   assert.equal(fenceNestingFault('::::panel info\n- - :::panel warning\n    B\n    :::\n::::'), undefined)
   assert.equal(fenceNestingFault(':::tableCell\n```text\n:::::::panel warning\n:::\n```\n:::'), undefined)
 })

todo.md

@@ -84,37 +84,41 @@ detail is settled at its own milestone.
         attribute fallback 2e owes, since the reservation leaves that node no other spelling
         whatever 1d decides for its `localId`.
   - [ ] **2e — Carve-outs and combinations.** Fixtures and emitter together, into
-        `corpus/round-trip/combinations/`: the three carve-outs and their escapes, mark runs — the
+        `corpus/round-trip/combinations/`.
-        longest-run rule, attributes included — and the runs a carry breaks, a mark spelling that
+    - [ ] **2e1 — The carve-outs and the claimed line.** The three carve-outs
-        cannot open where it sits (`un**-real**istic`; the spec owes the carry a trigger),
+          and their escapes, and a paragraph line inside a container body shaped like a closing
-        attribute canonicalization, a pipe cell's whitespace edges and `\u007c` for a `|` inside a
+          fence (`:::`, `::: x`). Guard `fenceNestingFault`'s bare-run pop here too — a run shorter
-        quoted attribute value, documents combining nodes rather than isolating one, and a
+          than the open fence is a fault, not a close — which today's emitter cannot reach.
-        paragraph line inside a container body shaped like a closing fence (`:::`, `::: x`).
+    - [ ] **2e2 — Mark runs and the runs a carry breaks.** The longest-run rule, attributes
-        Guard `fenceNestingFault`'s bare-run pop here too — a run shorter than the open fence is a
+          included, and a mark spelling that cannot open where it sits (`un**-real**istic`; the spec
-        fault, not a close — which today's emitter cannot reach.
+          owes the carry a trigger). One mark vocabulary lands here, before 2e3 changes the
-        Two moves land before the attribute spelling changes. One mark vocabulary:
+          attribute spelling: `emphasisSpellings`, `linkAttributes` and the `code`/`link` names join
-        `emphasisSpellings`, `linkAttributes` and the `code`/`link` names join
           `inline-directives.ts`, which holds four of the nine marks while the rest are branch
           literals in the emitter — and the parser (3) needs every name to make `:em[x]` the named
-        error `spec/flavour.md` promises. And `escaping: 'attribute'` earns its keep at the
+          error `spec/flavour.md` promises.
-        `\u007c` rule or collapses into `none`: nothing the escaper does tells the two apart
+    - [ ] **2e3 — Attribute canonicalization and the quoted value's escape.** Attribute
-        today, since a carried segment holds only spaces, tabs and newlines.
+          canonicalization, a pipe cell's whitespace edges, and `\u007c` for a `|` inside a quoted
-        The carry's fallback triggers land here too: `spec/flavour.md` carries a node its section
+          attribute value. The rule is wider than the pipe: a quoted value spells `` ` ``, `&` and
+          `<` raw today, so a backtick pair in an `inlineCard`'s `data`, a `status`'s `style` or the
+          inline carry's `json` reads back as a code span, an entity or raw HTML inside the value.
+          One escape settles all four, and the pick decides whether a directive or a code span wins
+          where they overlap (3). `escaping: 'attribute'` earns its keep at that rule or collapses
+          into `none`: nothing the escaper does tells the two apart today, since a carried segment
+          holds only spaces, tabs and newlines.
+    - [ ] **2e4 — The carry's fallback triggers.** `spec/flavour.md` carries a node its section
           cannot spell — an attrs key no section lists, a value that is not the section's type, an
           arg slot holding no bare token, marks no nesting spells — where the emitter still refuses,
-        which leaves the refusals a container's own spelling owns. The `\u007c` rule is wider than
+          which leaves the refusals a container's own spelling owns. The carry spells
-        the pipe: a quoted attribute value spells `` ` ``, `&` and `<` raw today, so a backtick
+          `escaping: 'none'`, which is what makes `tryPipeCell` refuse the pipe form for a carry
-        pair in an `inlineCard`'s `data`, a `status`'s `style` or the inline carry's `json` reads
+          whose JSON holds a pipe.
-        back as a code span, an entity or raw HTML inside the value. One escape settles all four,
+    - [ ] **2e5 — Combined documents and the collision property.** Documents combining nodes rather
-        and the pick decides whether a directive or a code span wins where they overlap (3). The
+          than isolating one, and the gate's collision property: no two corpus documents may emit
-        carry spells `escaping: 'none'` meanwhile, which is what makes `tryPipeCell` refuse the
+          the same bytes — one spelling for two documents is a round-trip break no parser can undo,
-        pipe form for a carry whose JSON holds a pipe.
+          and it is provable without one. It also settles the emitter's one known approximation:
-        The gate gains the collision property here: no two corpus documents may emit the same
+          delimiter flanking is exact, but CommonMark's *matching* — the multiple-of-3 rule and the
-        bytes — one spelling for two documents is a round-trip break no parser can undo, and it is
+          way a run splits across several openers — is not modelled. No reachable violation has been
-        provable without one. It also settles the emitter's one known approximation: delimiter
+          found by hand; the property test is what decides it, and 2e1's `carve-out-strike` pins a
-        flanking is exact, but CommonMark's *matching* — the multiple-of-3 rule and the way a run
+          second backslash only flanking-without-matching emits.
-        splits across several openers — is not modelled. No reachable violation has been found by
-        hand; the property test is what decides it.
   - [ ] **2f — The attributes CommonMark cannot hold.** 1d's settled answer: the block nodes
         CommonMark spells — `blockquote`, `bulletList`, `codeBlock`, `heading`, `listItem`,
         `orderedList`, `paragraph`, `rule` — get directive sections in `spec/flavour.md` carrying
@@ -137,7 +141,10 @@ detail is settled at its own milestone.
       assumes — CommonMark flanking, as for `*` — which `spec/flavour.md` does not yet pin, and
       the precedence between a directive and the constructs a raw attribute value opens inside it —
       a code span, an entity, raw HTML — which one directive alone already reaches until 2e's
-      escape lands.
+      escape lands. Whether a claimed line interrupts a paragraph is pinned for the plain case and
+      unstated for the lazy one: a closing fence on the line after a blockquote's open paragraph is
+      lazy continuation in CommonMark, which would swallow the fence and leave the container
+      unclosed. 2e1's `closing-fence-line` orders its blockquote away from that edge meanwhile.
       `src/` gets its hierarchy at the same split — `adf/`,
       `markdown/`, `html/`, the grammar module shared inside `markdown/` — while the rename is
       still mechanical. Three files do not move whole: `block-directives.ts` and