Flavour spec 1a: the directive grammar, canonical form, opaque carry #3

Merged
lilleman merged 7 commits from flavour-grammar into main 2026-08-24 00:49:26 +02:00
3 changed files with 49 additions and 25 deletions
Showing only changes of commit a4e5718a87 - Show all commits
+2 -2
View File
@@ -22,8 +22,8 @@ Round-trip equality is a property tested over a corpus, not a claim made in pros
## 3. Unknown input policy ## 3. Unknown input policy
- Unknown ADF node: carried opaquely — raw JSON rides a dedicated syntax in both formats, restored - Unknown ADF node: carried opaquely — raw JSON rides a dedicated syntax in both formats and
byte-for-byte. The round-trip holds for documents newer than the library. restores to a deep-equal node. The round-trip holds for documents newer than the library.
- Unmappable foreign HTML element: error result naming the element — never a silent drop. - Unmappable foreign HTML element: error result naming the element — never a silent drop.
- Bare `@name` / `:smile:` in typed text: stays a text node. Only directives produce - Bare `@name` / `:smile:` in typed text: stays a text node. Only directives produce
mention/emoji/media nodes; resolving names to ids needs I/O, which is the consumer's job. mention/emoji/media nodes; resolving names to ids needs I/O, which is the consumer's job.
+44 -22
View File
@@ -1,9 +1,9 @@
# The markdown flavour # The markdown flavour
The grammar of the extended markdown `adfToMarkdown` emits and `markdownToAdf` parses. Plain The grammar of the extended markdown `adfToMarkdown` emits and `markdownToAdf` parses. Plain
CommonMark is a subset: the flavour adds directives, never changes CommonMark meaning. The emitted CommonMark is a subset with one carve-out: literal text that matches directive syntax below is
form is contract (AGENTS.md §8). Per-node syntaxes build on this grammar in sections that follow claimed by the flavour (escape the `:` to keep it literal). The emitted form is contract
(todo.md 1b1c). (AGENTS.md §8). Per-node syntaxes build on this grammar in sections that follow (todo.md 1b1c).
## Canonical form ## Canonical form
@@ -12,15 +12,19 @@ normalizes to it through the round-trip.
- Emphasis `_em_`, strong `**strong**`; `*` replaces `_` only where `_` cannot parse - Emphasis `_em_`, strong `**strong**`; `*` replaces `_` only where `_` cannot parse
(intra-word). (intra-word).
- Bullet lists `- `, two-space continuation indent. Ordered lists incrementing `1.` `2.` `3.`, - Bullet lists `- `; ordered lists incrementing `1.` `2.` `3.`, the first number taken from the
the first number taken from the node's `order` attribute. node's `order` attribute. Continuation lines align with the first character after the marker
(two spaces for `- `, three for `1. `). Lists are tight — blank lines between items normalize
away; ADF does not record tightness.
- Blockquotes prefix every line with `> `.
- ATX headings (`#``######`); setext input normalizes to ATX. - ATX headings (`#``######`); setext input normalizes to ATX.
- Code fences ``` with the node's language as info string, the fence lengthened past any backtick - Code fences ``` with the node's language as info string, the fence lengthened past any backtick
run in the content; indented-code input normalizes to fences. run in the content; indented-code input normalizes to fences.
- Thematic break `---`. - Thematic break `---`.
- Hard break: backslash at end of line (survives editors that trim trailing spaces). - Hard break: backslash at end of line (survives editors that trim trailing spaces).
- Links `[text](url)`; `<…>` around a destination containing spaces; title in double quotes; - Links `[text](url)`; `<…>` around a destination containing spaces; title in double quotes.
`<url>` autolink when the link text equals its destination. `<url>` autolink form only when the text equals the destination and the destination is a valid
CommonMark autolink (absolute URI).
- Paragraphs on one line — no soft wrapping; soft line breaks in input collapse per CommonMark. - Paragraphs on one line — no soft wrapping; soft line breaks in input collapse per CommonMark.
- Entity references in input decode to their characters; output backslash-escapes only where text - Entity references in input decode to their characters; output backslash-escapes only where text
would otherwise parse as syntax. would otherwise parse as syntax.
@@ -28,11 +32,19 @@ normalizes to it through the round-trip.
## Directives ## Directives
One grammar for everything CommonMark lacks. Names are the ADF node names (camelCase). One grammar for everything CommonMark lacks. A directive name is `[a-z][A-Za-z0-9]*` — the ADF
node names. Recognition is syntactic and name-set-independent: anything matching the forms below
parses as a directive regardless of whether the name is known, and an unknown name is an error
result naming it — so output an old emitter escaped stays escaped, and erroring input gaining
meaning later is MINOR, never a reparse (§8). The name `adf` is reserved for the opaque carry, as
both directive name and fence info string.
**Inline**: `:name[content]{attrs}`. `[content]` is inline markdown; each node's section says **Inline**: `:name[content]{attrs}`, on one line — an inline directive never spans lines.
whether content and attrs are required. `:` opens a directive only when the name is followed `[content]` is inline markdown; brackets inside balance as in CommonMark link text, `\]` for a
immediately by `[` or `{` — anything else (`10:30`, `:smile:`) is literal text. literal bracket. Each node's section says whether content and attrs are required. `:` opens a
directive only when the name is followed immediately by `[` or `{`, and `{attrs}` must follow
`]` (or the name) with no gap — anything else (`10:30`, `:smile:`, a stray `{…}` in text) is
literal text.
**Container block**: **Container block**:
@@ -45,25 +57,35 @@ block content
The fence is three or more colons. `arg` is one optional bare token whose meaning each node The fence is three or more colons. `arg` is one optional bare token whose meaning each node
defines (e.g. the panel type). The body is block markdown. The closing fence is a line of at defines (e.g. the panel type). The body is block markdown. The closing fence is a line of at
least the opening's length, and a container's fence is longer than every directive fence line in least the opening's length, and a container's fence is longer than every directive fence line in
its body, so closers are unambiguous — the code-fence rule. Canonical form uses minimal lengths. its body — counting only lines that parse as directive fences in the body's block structure; a
colon run inside a code fence or opaque carry is content. Canonical form uses minimal lengths.
**Leaf block**: `::name {attrs}` — a block-position node with no body. **Leaf block**: `::name {attrs}` — a block-position node with no body.
**Attributes**: `{key=value key2="two words"}`. A bare value matches `[A-Za-z0-9_-]+`; anything **Attributes**: `{key=value key2="two words"}`. A bare value matches `[A-Za-z0-9_-]+`; any other
else is double-quoted with `\"` and `\\` as the only escapes. All values are strings at the value is double-quoted with JSON string escaping (`\"` `\\` `\n` `\t` `\uXXXX`, …) — total over
grammar level; each node's section assigns types. Canonical form orders keys alphabetically and Unicode, and raw newlines never appear inside quotes. All values are strings at the grammar
spells values bare wherever allowed. level; each node's section assigns types. Canonical form orders keys alphabetically and spells
values bare wherever allowed.
**Escaping**: the emitter backslash-escapes a `:` whose text would otherwise parse as a **Escaping**: the emitter backslash-escapes whatever literal text would otherwise parse as
directive; a backslash before `:` in input always yields a literal colon. directive syntax — the leading `:` of a would-be directive, `]` inside content; a backslash
before `:` in input always yields a literal colon.
**Malformed directives are error results**, named: an unclosed container at end of input, a body
fence line of the container's length or longer, unparseable or duplicate-keyed attrs, invalid
JSON in an opaque carry. Never a silent literal-text fallback — a typo that reparses as prose is
the silent loss §2 refuses.
## The opaque carry (AGENTS.md §3) ## The opaque carry (AGENTS.md §3)
A node type the library does not know rides as its raw JSON and restores to a deep-equal node: A node type the library does not know rides as its raw JSON and restores to a deep-equal node.
Block and inline positions canonicalize differently, each fitting where it sits:
- **Block position**: a fenced code block with info string `adf`, body = the node's JSON, - **Block position**: a fenced code block with info string `adf`, body = the node's JSON
serialized canonically — two-space indent, object keys sorted. two-space indent, object keys sorted.
- **Inline position**: `:adf{json="…"}`, same serialization, quote-escaped. - **Inline position**: `:adf{json="…"}` — compact serialization (keys sorted, no whitespace),
JSON-string-escaped into the attribute.
The info string `adf` is reserved: a genuine `codeBlock` whose `language` is exactly `adf` is The info string `adf` is reserved: a genuine `codeBlock` whose `language` is exactly `adf` is
itself emitted through the opaque carry, so the reservation stays absolute and stays lossless. itself emitted through the opaque carry, so the reservation stays absolute and stays lossless.
+3 -1
View File
@@ -19,7 +19,9 @@ detail is settled at its own milestone.
- [ ] **1d — Corpus start** (§10): checked-in ADF ↔ canonical-markdown fixture pairs per spec'd - [ ] **1d — Corpus start** (§10): checked-in ADF ↔ canonical-markdown fixture pairs per spec'd
node. node.
- [ ] **2 — `adfToMarkdown`.** First real code — decide here where §10's coverage check lives. - [ ] **2 — `adfToMarkdown`.** First real code — decide here where §10's coverage check lives.
- [ ] **3 — `markdownToAdf`.** The CommonMark parser is the largest single component. - [ ] **3 — `markdownToAdf`.** The CommonMark parser is the largest single component. The raw-HTML
element mapping is empty until milestone 6, so at `0.1.0` every raw-HTML construct in input
is an error result.
- [ ] **4 — Round-trip property tests** over the corpus, both ways — the thing that proves 2 and 3. - [ ] **4 — Round-trip property tests** over the corpus, both ways — the thing that proves 2 and 3.
- [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret, - [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change (§9), `NPM_TOKEN` secret,
the repo made public first (§6). `0.1.0` is the markdown round-trip: both markdown the repo made public first (§6). `0.1.0` is the markdown round-trip: both markdown