From 1f86f248705ea7519248bb95cd3c87e17f1916f9 Mon Sep 17 00:00:00 2001 From: Lilleman auf Larv Date: Thu, 5 Nov 2026 11:14:27 +0100 Subject: [PATCH] Publish what each call writes for chosen corpus examples, and fail the gate where the page is stale --- AGENTS.md | 1 + CHANGELOG.md | 3 + README.md | 7 +- ci.sh | 2 + .../round-trip/block-nodes/table-plain.json | 110 ++++ corpus/round-trip/block-nodes/table-plain.md | 4 + .../round-trip/combinations/bold-command.json | 14 + .../round-trip/combinations/bold-command.md | 1 + docs/decisions.md | 10 + docs/feature-table.md | 590 ++++++++++++++++++ feature-table/build.mjs | 16 + feature-table/examples.json | 17 + feature-table/render.mjs | 41 ++ feature-table/render.test.mjs | 43 ++ feature-table/write.sh | 9 + todo.md | 54 +- 16 files changed, 876 insertions(+), 46 deletions(-) create mode 100644 corpus/round-trip/block-nodes/table-plain.json create mode 100644 corpus/round-trip/block-nodes/table-plain.md create mode 100644 corpus/round-trip/combinations/bold-command.json create mode 100644 corpus/round-trip/combinations/bold-command.md create mode 100644 docs/feature-table.md create mode 100644 feature-table/build.mjs create mode 100644 feature-table/examples.json create mode 100644 feature-table/render.mjs create mode 100644 feature-table/render.test.mjs create mode 100755 feature-table/write.sh diff --git a/AGENTS.md b/AGENTS.md index 5632d0a..9719b3f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -89,6 +89,7 @@ In `docs/decisions.md`: - The source parts by ADF and format - The comparison table lists every converter that starts and converts - The real-world corpus holds anonymized documents from sites whose licence it records +- The feature table is a section per corpus example, led by a summary table ## 1. Nothing about any consumer in the library diff --git a/CHANGELOG.md b/CHANGELOG.md index 5f7b3b7..4acd89a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,9 @@ ## Unreleased +- `docs/feature-table.md` sets real documents beside what `adfToLosslessMarkdown`, + `adfToPortableMarkdown` and `adfToHtml` write for them, and says whether each output reads back to + the same document. - **Breaking:** `adfToHtml`, `adfToPortableMarkdown`, `htmlToAdf` and `losslessMarkdownToAdf` refuse a block or non-text inline node holding `text`, and a leaf block or non-text inline node holding `content`, as `adfToLosslessMarkdown` already did outside a carried node, which it now also diff --git a/README.md b/README.md index 9e8a1b3..fd17b30 100644 --- a/README.md +++ b/README.md @@ -300,7 +300,8 @@ documentation says, on 2026-10-28. ## The calls Serves [Goals](#goals) 1, 2 and 8. Pure functions, each taking a whole document and returning a -whole result; no I/O, no configuration. +whole result; no I/O, no configuration. [What each call writes](https://github.com/larvit/adf-codec/blob/main/docs/feature-table.md) +sets real documents beside each writer's output and says whether that output reads back the same. ```ts adfToLosslessMarkdown(doc: AdfDocument): Result @@ -771,9 +772,11 @@ Contract: [`docs/decisions.md`](https://github.com/larvit/adf-codec/blob/main/do ## Changing the source -Both commands need only Docker, git and bash 5. +All three commands need only Docker, git and bash 5. - `./ci.sh` runs every check CI runs. +- `./feature-table/write.sh` rewrites `docs/feature-table.md` from `feature-table/examples.json` and + the build. `./ci.sh` fails where a change leaves that page stale. - `./mutation-tests/run.sh []` mutation-tests the lines changed since ``, `origin/main` by default, in source files git tracks. It needs 6 GB of memory for Docker, and lists the mutants that survived. Survivors don't fail it; a mutant whose test run crashed does. diff --git a/ci.sh b/ci.sh index 4afa0f6..0037fd2 100755 --- a/ci.sh +++ b/ci.sh @@ -60,8 +60,10 @@ rm -f "$test_log" leg "tests ($deno_image)" in_image "$deno_image" deno test --allow-env=PROPERTY_RUNS --allow-read --no-check src/ leg "tests ($bun_image)" in_image "$bun_image" bun test src/ leg "corpus-gathering tests ($node_image)" in_image "$node_image" node --test corpus-gathering/ +leg "feature table tests ($node_image)" in_image "$node_image" node --test feature-table/ leg "build ($node_image)" in_image "$node_image" npm run build +leg "feature table is current ($node_image)" in_image "$node_image" node feature-table/build.mjs --check leg "pack and install the tarball ($node_image)" in_image "$node_image" sh -c 'set -e rm -rf package-tests/node_modules npm pack --pack-destination /tmp diff --git a/corpus/round-trip/block-nodes/table-plain.json b/corpus/round-trip/block-nodes/table-plain.json new file mode 100644 index 0000000..dc44612 --- /dev/null +++ b/corpus/round-trip/block-nodes/table-plain.json @@ -0,0 +1,110 @@ +{ + "content": [ + { + "content": [ + { + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Part", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "tableHeader" + }, + { + "content": [ + { + "content": [ + { + "text": "Stock", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "tableHeader" + } + ], + "type": "tableRow" + }, + { + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Bolt M8", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "tableCell" + }, + { + "content": [ + { + "content": [ + { + "text": "120", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "tableCell" + } + ], + "type": "tableRow" + }, + { + "content": [ + { + "content": [ + { + "content": [ + { + "text": "Washer M8", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "tableCell" + }, + { + "content": [ + { + "content": [ + { + "text": "45", + "type": "text" + } + ], + "type": "paragraph" + } + ], + "type": "tableCell" + } + ], + "type": "tableRow" + } + ], + "type": "table" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/block-nodes/table-plain.md b/corpus/round-trip/block-nodes/table-plain.md new file mode 100644 index 0000000..380d471 --- /dev/null +++ b/corpus/round-trip/block-nodes/table-plain.md @@ -0,0 +1,4 @@ +| Part | Stock | +| --- | --- | +| Bolt M8 | 120 | +| Washer M8 | 45 | diff --git a/corpus/round-trip/combinations/bold-command.json b/corpus/round-trip/combinations/bold-command.json new file mode 100644 index 0000000..126879c --- /dev/null +++ b/corpus/round-trip/combinations/bold-command.json @@ -0,0 +1,14 @@ +{ + "content": [ + { + "content": [ + { "marks": [{ "type": "strong" }], "text": "Run ", "type": "text" }, + { "marks": [{ "type": "code" }], "text": "npm ci", "type": "text" }, + { "marks": [{ "type": "strong" }], "text": " first", "type": "text" } + ], + "type": "paragraph" + } + ], + "type": "doc", + "version": 1 +} diff --git a/corpus/round-trip/combinations/bold-command.md b/corpus/round-trip/combinations/bold-command.md new file mode 100644 index 0000000..c02f8cb --- /dev/null +++ b/corpus/round-trip/combinations/bold-command.md @@ -0,0 +1 @@ +**Run `npm ci` first** diff --git a/docs/decisions.md b/docs/decisions.md index f8ddd8b..9b0f6b3 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -1221,3 +1221,13 @@ the site, applying them is an inference, which `corpus/README.md` tells the read None of the 22,773 fetched documents holds a type in `typesNoFetchedDocumentHolds` (`src/conformance/corpus.test.ts`), and neither did up to 40 pages per site with "decision" in the title. A regenerated corpus holding one of those types fails that test until the type leaves the list. + +## The feature table is a section per corpus example, led by a summary table + +2026-11-05, a developer panel, 3 of 3. Goal 3. Valid while the page lists about fifteen examples. + +Readers from the viewer/editor, export/indexing and LLM/agent pipeline personas picked labelled code +blocks stacked under a heading per example, with a table of the round-trip results on top, over one +wide pipe table and over one wide HTML table with a `
` per cell. Examples come from
+`corpus/round-trip/` and are the ones `feature-table/examples.json` lists, never
+`corpus/real-world/`, whose licence is not the package's.
diff --git a/docs/feature-table.md b/docs/feature-table.md
new file mode 100644
index 0000000..cfe8aa0
--- /dev/null
+++ b/docs/feature-table.md
@@ -0,0 +1,590 @@
+# What each call writes
+
+`feature-table/build.mjs` writes this page from the corpus and the built package, and `./ci.sh` fails where the page differs.
+
+Each output is read back by its format's reader. `holds` means that gives the same document, `loses` a different one, and `refused` an error whose code the example names.
+
+| Example | Lossless | Portable | HTML |
+| --- | --- | --- | --- |
+| [Bold instruction with a command in it](#bold-instruction-with-a-command-in-it) | holds | holds | holds |
+| [Emphasis and strikethrough](#emphasis-and-strikethrough) | holds | holds | holds |
+| [Headings](#headings) | holds | holds | holds |
+| [Bullet list](#bullet-list) | holds | holds | holds |
+| [Code block](#code-block) | holds | holds | holds |
+| [Quote](#quote) | holds | holds | holds |
+| [Link](#link) | holds | holds | holds |
+| [Table](#table) | holds | holds | holds |
+| [Panels](#panels) | holds | loses | holds |
+| [Task list](#task-list) | holds | loses | holds |
+| [Image](#image) | holds | holds | holds |
+| [Mention](#mention) | holds | loses | holds |
+| [Status](#status) | holds | loses | holds |
+| [Date](#date) | holds | loses | holds |
+| [Node types this version does not know](#node-types-this-version-does-not-know) | holds | loses | holds |
+
+## Bold instruction with a command in it
+
+ADF:
+
+```json
+{"content":[{"content":[{"marks":[{"type":"strong"}],"text":"Run ","type":"text"},{"marks":[{"type":"code"}],"text":"npm ci","type":"text"},{"marks":[{"type":"strong"}],"text":" first","type":"text"}],"type":"paragraph"}],"type":"doc","version":1}
+```
+
+`adfToLosslessMarkdown`, round-trip holds:
+
+```markdown
+**Run `npm ci` first**
+```
+
+`adfToPortableMarkdown`, round-trip holds:
+
+```markdown
+**Run `npm ci` first**
+```
+
+`adfToHtml`, round-trip holds:
+
+```html
+

Run npm ci first

+``` + +## Emphasis and strikethrough + +ADF: + +```json +{"content":[{"content":[{"text":"Read the ","type":"text"},{"marks":[{"type":"em"}],"text":"manual","type":"text"},{"text":" before ","type":"text"},{"marks":[{"type":"strong"}],"text":"wiring","type":"text"},{"text":" the ","type":"text"},{"marks":[{"type":"strike"}],"text":"relay","type":"text"},{"text":".","type":"text"}],"type":"paragraph"},{"content":[{"text":"An un","type":"text"},{"marks":[{"type":"em"}],"text":"real","type":"text"},{"text":"istic goal.","type":"text"}],"type":"paragraph"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +Read the _manual_ before **wiring** the ~~relay~~. + +An un*real*istic goal. +``` + +`adfToPortableMarkdown`, round-trip holds: + +```markdown +Read the _manual_ before **wiring** the ~~relay~~. + +An un*real*istic goal. +``` + +`adfToHtml`, round-trip holds: + +```html +

Read the manual before wiring the relay.

+

An unrealistic goal.

+``` + +## Headings + +ADF: + +```json +{"content":[{"attrs":{"level":1},"content":[{"text":"Assembly","type":"text"}],"type":"heading"},{"attrs":{"level":2},"content":[{"text":"Parts","type":"text"}],"type":"heading"},{"attrs":{"level":3},"content":[{"text":"Fasteners","type":"text"}],"type":"heading"},{"attrs":{"level":4},"content":[{"text":"Bolts","type":"text"}],"type":"heading"},{"attrs":{"level":5},"content":[{"text":"Sizes","type":"text"}],"type":"heading"},{"attrs":{"level":6},"content":[{"text":"M8","type":"text"}],"type":"heading"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +# Assembly + +## Parts + +### Fasteners + +#### Bolts + +##### Sizes + +###### M8 +``` + +`adfToPortableMarkdown`, round-trip holds: + +```markdown +# Assembly + +## Parts + +### Fasteners + +#### Bolts + +##### Sizes + +###### M8 +``` + +`adfToHtml`, round-trip holds: + +```html +

Assembly

+

Parts

+

Fasteners

+

Bolts

+
Sizes
+
M8
+``` + +## Bullet list + +ADF: + +```json +{"content":[{"content":[{"content":[{"content":[{"text":"Bolt M8","type":"text"}],"type":"paragraph"}],"type":"listItem"},{"content":[{"content":[{"text":"Nut M8","type":"text"}],"type":"paragraph"}],"type":"listItem"},{"content":[{"content":[{"text":"Washer M8","type":"text"}],"type":"paragraph"},{"content":[{"content":[{"content":[{"text":"Fibre","type":"text"}],"type":"paragraph"}],"type":"listItem"}],"type":"bulletList"}],"type":"listItem"}],"type":"bulletList"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +- Bolt M8 +- Nut M8 +- Washer M8 + - Fibre +``` + +`adfToPortableMarkdown`, round-trip holds: + +```markdown +- Bolt M8 +- Nut M8 +- Washer M8 + - Fibre +``` + +`adfToHtml`, round-trip holds: + +```html +
    +
  • Bolt M8
  • +
  • Nut M8
  • +
  • Washer M8 +
      +
    • Fibre
    • +
    +
  • +
+``` + +## Code block + +ADF: + +```json +{"content":[{"attrs":{"language":"sql"},"content":[{"text":"SELECT id\nFROM part\nWHERE qty > 0;\n- 1. the markers a container start would claim","type":"text"}],"type":"codeBlock"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +````markdown +```sql +SELECT id +FROM part +WHERE qty > 0; +- 1. the markers a container start would claim +``` +```` + +`adfToPortableMarkdown`, round-trip holds: + +````markdown +```sql +SELECT id +FROM part +WHERE qty > 0; +- 1. the markers a container start would claim +``` +```` + +`adfToHtml`, round-trip holds: + +```html +
SELECT id
+FROM part
+WHERE qty > 0;
+- 1. the markers a container start would claim
+``` + +## Quote + +ADF: + +```json +{"content":[{"content":[{"content":[{"text":"Ship it.","type":"text"}],"type":"paragraph"},{"content":[{"text":"Then tell them.","type":"text"}],"type":"paragraph"}],"type":"blockquote"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +> Ship it. +> +> Then tell them. +``` + +`adfToPortableMarkdown`, round-trip holds: + +```markdown +> Ship it. +> +> Then tell them. +``` + +`adfToHtml`, round-trip holds: + +```html +
+

Ship it.

+

Then tell them.

+
+``` + +## Link + +ADF: + +```json +{"content":[{"content":[{"text":"See ","type":"text"},{"marks":[{"attrs":{"href":"https://example.com/changelog"},"type":"link"}],"text":"the changelog","type":"text"},{"text":".","type":"text"}],"type":"paragraph"},{"content":[{"marks":[{"attrs":{"href":"https://example.com/"},"type":"link"}],"text":"https://example.com/","type":"text"}],"type":"paragraph"},{"content":[{"text":"Read ","type":"text"},{"marks":[{"attrs":{"href":"https://example.com/guide","title":"Setup guide"},"type":"link"}],"text":"the guide","type":"text"},{"text":".","type":"text"}],"type":"paragraph"},{"content":[{"text":"Open ","type":"text"},{"marks":[{"attrs":{"href":"https://example.com/the plan.pdf"},"type":"link"}],"text":"the plan","type":"text"},{"text":".","type":"text"}],"type":"paragraph"},{"content":[{"marks":[{"attrs":{"href":"/parts/m8"},"type":"link"}],"text":"/parts/m8","type":"text"}],"type":"paragraph"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +See [the changelog](https://example.com/changelog). + + + +Read [the guide](https://example.com/guide "Setup guide"). + +Open [the plan](). + +[/parts/m8](/parts/m8) +``` + +`adfToPortableMarkdown`, round-trip holds: + +```markdown +See [the changelog](https://example.com/changelog). + + + +Read [the guide](https://example.com/guide "Setup guide"). + +Open [the plan](). + +[/parts/m8](/parts/m8) +``` + +`adfToHtml`, round-trip holds: + +```html +

See the changelog.

+

https://example.com/

+

Read the guide.

+

Open the plan.

+

/parts/m8

+``` + +## Table + +ADF: + +```json +{"content":[{"content":[{"content":[{"content":[{"content":[{"text":"Part","type":"text"}],"type":"paragraph"}],"type":"tableHeader"},{"content":[{"content":[{"text":"Stock","type":"text"}],"type":"paragraph"}],"type":"tableHeader"}],"type":"tableRow"},{"content":[{"content":[{"content":[{"text":"Bolt M8","type":"text"}],"type":"paragraph"}],"type":"tableCell"},{"content":[{"content":[{"text":"120","type":"text"}],"type":"paragraph"}],"type":"tableCell"}],"type":"tableRow"},{"content":[{"content":[{"content":[{"text":"Washer M8","type":"text"}],"type":"paragraph"}],"type":"tableCell"},{"content":[{"content":[{"text":"45","type":"text"}],"type":"paragraph"}],"type":"tableCell"}],"type":"tableRow"}],"type":"table"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +| Part | Stock | +| --- | --- | +| Bolt M8 | 120 | +| Washer M8 | 45 | +``` + +`adfToPortableMarkdown`, round-trip holds: + +```markdown +| Part | Stock | +| --- | --- | +| Bolt M8 | 120 | +| Washer M8 | 45 | +``` + +`adfToHtml`, round-trip holds: + +```html + + + + + + + + +
PartStock
Bolt M8120
Washer M845
+``` + +## Panels + +ADF: + +```json +{"content":[{"attrs":{"panelType":"warning"},"content":[{"content":[{"text":"Check the collation before importing.","type":"text"}],"type":"paragraph"}],"type":"panel"},{"attrs":{"localId":"01a0337c-561a-75aa-b629-3801f68839ad","panelType":"info"},"content":[{"content":[{"text":"Backups run at 02:00.","type":"text"}],"type":"paragraph"}],"type":"panel"},{"attrs":{"panelColor":"#c9372c","panelIcon":":rainbow:","panelIconId":"1f308","panelIconText":"🌈","panelType":"custom"},"content":[{"content":[{"text":"Weather permitting.","type":"text"}],"type":"paragraph"}],"type":"panel"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +!adf:panel warning +Check the collation before importing. +!adf:/panel + +!adf:panel info {localId=01a0337c-561a-75aa-b629-3801f68839ad} +Backups run at 02:00. +!adf:/panel + +!adf:panel custom {panelColor="#c9372c" panelIcon=":rainbow:" panelIconId=1f308 panelIconText="🌈"} +Weather permitting. +!adf:/panel +``` + +`adfToPortableMarkdown`, round-trip loses: + +```markdown +> [!WARNING] +> +> Check the collation before importing. + +> [!NOTE] +> +> Backups run at 02:00. + +> [!NOTE] +> +> Weather permitting. +``` + +`adfToHtml`, round-trip holds: + +```html +

Check the collation before importing.

+

Backups run at 02:00.

+

Weather permitting.

+``` + +## Task list + +ADF: + +```json +{"content":[{"attrs":{"localId":"0198f3a2-7c41-7f2e-9b3a-4d8e2c1a6b90"},"content":[{"attrs":{"localId":"0198f3a2-8d52-70b1-8c4f-5e9f3d2b7ca1","state":"DONE"},"content":[{"text":"Write the spec","type":"text"}],"type":"taskItem"},{"attrs":{"localId":"0198f3a2-9e63-7d80-a15b-6fa04e3c8db2","state":"TODO"},"content":[{"text":"Ship it","type":"text"}],"type":"taskItem"},{"attrs":{"localId":"01a033b2-95c8-7d62-9b9c-dee7ad510982","state":"TODO"},"content":[{"content":[{"text":"Check the torque.","type":"text"}],"type":"paragraph"},{"content":[{"text":"Then sign off.","type":"text"}],"type":"paragraph"}],"type":"blockTaskItem"},{"attrs":{"localId":"01a033b3-d1fa-7489-8c9f-f5846d2c249d"},"content":[{"attrs":{"localId":"01a033bd-a22d-7fb5-be9d-89d1aa261112","state":"TODO"},"content":[{"text":"Nested step","type":"text"}],"type":"taskItem"},{"attrs":{"state":"TODO"},"type":"taskItem"}],"type":"taskList"}],"type":"taskList"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +!adf:taskList {localId=0198f3a2-7c41-7f2e-9b3a-4d8e2c1a6b90} +!adf:taskItem DONE {localId=0198f3a2-8d52-70b1-8c4f-5e9f3d2b7ca1} +Write the spec +!adf:/taskItem +!adf:taskItem TODO {localId=0198f3a2-9e63-7d80-a15b-6fa04e3c8db2} +Ship it +!adf:/taskItem +!adf:blockTaskItem TODO {localId=01a033b2-95c8-7d62-9b9c-dee7ad510982} +Check the torque. + +Then sign off. +!adf:/blockTaskItem +!adf:taskList {localId=01a033b3-d1fa-7489-8c9f-f5846d2c249d} +!adf:taskItem TODO {localId=01a033bd-a22d-7fb5-be9d-89d1aa261112} +Nested step +!adf:/taskItem +!adf:taskItem TODO +!adf:/taskItem +!adf:/taskList +!adf:/taskList +``` + +`adfToPortableMarkdown`, round-trip loses: + +```markdown +- [x] Write the spec +- [ ] Ship it +- [ ] Check the torque. + + Then sign off. + - [ ] Nested step + - [ ] +``` + +`adfToHtml`, round-trip holds: + +```html +

Write the spec

Ship it

Check the torque.

Then sign off.

Nested step

+``` + +## Image + +ADF: + +```json +{"content":[{"content":[{"text":"Taken from the roof last night.","type":"text"}],"type":"paragraph"},{"attrs":{"layout":"center"},"content":[{"attrs":{"alt":"The moon, at night.","type":"external","url":"https://example.com/moon.png"},"type":"media"}],"type":"mediaSingle"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +Taken from the roof last night. + +![The moon, at night.](https://example.com/moon.png) +``` + +`adfToPortableMarkdown`, round-trip holds: + +```markdown +Taken from the roof last night. + +![The moon, at night.](https://example.com/moon.png) +``` + +`adfToHtml`, round-trip holds: + +```html +

Taken from the roof last night.

+

The moon, at night.

+``` + +## Mention + +ADF: + +```json +{"content":[{"content":[{"text":"Ping ","type":"text"},{"attrs":{"accessLevel":"CONTAINER","id":"01a032c3-7a7c-775f-a730-2d79351338b4","localId":"01a033ad-ecb7-7530-aa1c-ff25031e96e2","text":"@Mikael","userType":"DEFAULT"},"type":"mention"},{"text":" before merging.","type":"text"}],"type":"paragraph"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +Ping !adf:mention[@Mikael]{accessLevel=CONTAINER id=01a032c3-7a7c-775f-a730-2d79351338b4 localId=01a033ad-ecb7-7530-aa1c-ff25031e96e2 userType=DEFAULT} before merging. +``` + +`adfToPortableMarkdown`, round-trip loses: + +```markdown +Ping @Mikael before merging. +``` + +`adfToHtml`, round-trip holds: + +```html +

Ping @Mikael before merging.

+``` + +## Status + +ADF: + +```json +{"content":[{"content":[{"text":"The rebuild is ","type":"text"},{"attrs":{"color":"yellow","localId":"01a033a7-d68a-71a9-a58a-edf5757ad176","style":"bold","text":"In review"},"type":"status"},{"text":" until Friday.","type":"text"}],"type":"paragraph"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +The rebuild is !adf:status[In review]{color=yellow localId=01a033a7-d68a-71a9-a58a-edf5757ad176 style=bold} until Friday. +``` + +`adfToPortableMarkdown`, round-trip loses: + +```markdown +The rebuild is In review until Friday. +``` + +`adfToHtml`, round-trip holds: + +```html +

The rebuild is In review until Friday.

+``` + +## Date + +ADF: + +```json +{"content":[{"content":[{"text":"Due ","type":"text"},{"attrs":{"timestamp":"1756080000000"},"type":"date"},{"text":", reviewed ","type":"text"},{"attrs":{"localId":"01a033ab-781c-7f79-9c8a-fb63356c1cad","timestamp":"1735689600000"},"type":"date"},{"text":".","type":"text"}],"type":"paragraph"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +```markdown +Due !adf:date{timestamp=1756080000000}, reviewed !adf:date{localId=01a033ab-781c-7f79-9c8a-fb63356c1cad timestamp=1735689600000}. +``` + +`adfToPortableMarkdown`, round-trip loses: + +```markdown +Due 2025-08-25, reviewed 2025-01-01. +``` + +`adfToHtml`, round-trip holds: + +```html +

Due , reviewed .

+``` + +## Node types this version does not know + +ADF: + +```json +{"content":[{"attrs":{"url":"https://example.com/roadmap"},"type":"blockCard"},{"attrs":{"panelType":"info"},"content":[{"content":[{"text":"The card below has no spelling yet.","type":"text"}],"type":"paragraph"},{"attrs":{"layout":"wide","url":"https://example.com/board","width":100},"type":"embedCard"}],"type":"panel"}],"type":"doc","version":1} +``` + +`adfToLosslessMarkdown`, round-trip holds: + +````markdown +```adf:blockCard +{ + "attrs": { + "url": "https://example.com/roadmap" + } +} +``` + +!adf:panel info +The card below has no spelling yet. + +```adf:embedCard +{ + "attrs": { + "layout": "wide", + "url": "https://example.com/board", + "width": 100 + } +} +``` +!adf:/panel +```` + +`adfToPortableMarkdown`, round-trip loses: + +```markdown + + +> [!NOTE] +> +> The card below has no spelling yet. +> +> +``` + +`adfToHtml`, round-trip holds: + +```html + +

The card below has no spelling yet.

https://example.com/board

+``` diff --git a/feature-table/build.mjs b/feature-table/build.mjs new file mode 100644 index 0000000..0cd8295 --- /dev/null +++ b/feature-table/build.mjs @@ -0,0 +1,16 @@ +import { readFileSync, writeFileSync } from 'node:fs' +import * as library from '@larvit/adf-codec' + +import { renderFeatureTable } from './render.mjs' + +const pagePath = new URL('../docs/feature-table.md', import.meta.url) +const readJson = (url) => JSON.parse(readFileSync(url, 'utf8')) + +const examples = readJson(new URL('examples.json', import.meta.url)).map(({ id, name }) => ({ doc: readJson(new URL(`../corpus/round-trip/${id}.json`, import.meta.url)), name })) +const page = renderFeatureTable(examples, library) + +if (!process.argv.includes('--check')) writeFileSync(pagePath, page) +else if (readFileSync(pagePath, 'utf8') !== page) { + console.error('docs/feature-table.md differs from what the examples and the build give: run ./feature-table/write.sh') + process.exitCode = 1 +} diff --git a/feature-table/examples.json b/feature-table/examples.json new file mode 100644 index 0000000..dbc02f5 --- /dev/null +++ b/feature-table/examples.json @@ -0,0 +1,17 @@ +[ + { "id": "combinations/bold-command", "name": "Bold instruction with a command in it" }, + { "id": "commonmark-subset/em-strong-strike", "name": "Emphasis and strikethrough" }, + { "id": "commonmark-subset/heading", "name": "Headings" }, + { "id": "commonmark-subset/bullet-list", "name": "Bullet list" }, + { "id": "commonmark-subset/code-block", "name": "Code block" }, + { "id": "commonmark-subset/blockquote", "name": "Quote" }, + { "id": "commonmark-subset/link", "name": "Link" }, + { "id": "block-nodes/table-plain", "name": "Table" }, + { "id": "block-nodes/panel", "name": "Panels" }, + { "id": "block-nodes/task-list", "name": "Task list" }, + { "id": "block-nodes/image", "name": "Image" }, + { "id": "inline-nodes/mention", "name": "Mention" }, + { "id": "inline-nodes/status", "name": "Status" }, + { "id": "inline-nodes/date", "name": "Date" }, + { "id": "opaque-carry/unknown-block", "name": "Node types this version does not know" } +] diff --git a/feature-table/render.mjs b/feature-table/render.mjs new file mode 100644 index 0000000..7c762cb --- /dev/null +++ b/feature-table/render.mjs @@ -0,0 +1,41 @@ +import { isDeepStrictEqual } from 'node:util' + +const writers = [ + { column: 'Lossless', fence: 'markdown', reader: 'losslessMarkdownToAdf', writer: 'adfToLosslessMarkdown' }, + { column: 'Portable', fence: 'markdown', reader: 'portableMarkdownToAdf', writer: 'adfToPortableMarkdown' }, + { column: 'HTML', fence: 'html', reader: 'htmlToAdf', writer: 'adfToHtml' }, +] + +const intro = `# What each call writes + +\`feature-table/build.mjs\` writes this page from the corpus and the built package, and \`./ci.sh\` fails where the page differs. + +Each output is read back by its format's reader. \`holds\` means that gives the same document, \`loses\` a different one, and \`refused\` an error whose code the example names. +` + +// `library` holds the package's calls, so a test can hand in what a real document cannot make them do. +export function renderFeatureTable(examples, library) { + const written = examples.map(({ doc, name }) => ({ doc, name, outcomes: writers.map((writer) => outcome(doc, writer, library)) })) + const rows = written.map(({ name, outcomes }) => `| [${name}](#${name.toLowerCase().replaceAll(' ', '-')}) | ${outcomes.map(({ result }) => result).join(' | ')} |`) + const head = `| Example | ${writers.map(({ column }) => column).join(' | ')} |\n| --- | --- | --- | --- |` + return [intro, [head, ...rows].join('\n') + '\n', ...written.map(section)].join('\n') +} + +function outcome(doc, { fence, reader, writer }, library) { + const output = library[writer](doc) + if (!output.ok) return { label: `\`${writer}\` refuses with \`${output.error.code}\`.\n`, result: 'refused' } + const read = library[reader](output.value) + if (!read.ok) return { label: `\`${writer}\`, round-trip refused with \`${read.error.code}\`:\n\n${fenced(fence, output.value)}`, result: 'refused' } + const result = isDeepStrictEqual(read.value, doc) ? 'holds' : 'loses' + return { label: `\`${writer}\`, round-trip ${result}:\n\n${fenced(fence, output.value)}`, result } +} + +function section({ doc, name, outcomes }) { + return [`## ${name}\n`, `ADF:\n\n${fenced('json', `${JSON.stringify(doc)}\n`)}`, ...outcomes.map(({ label }) => label)].join('\n') +} + +function fenced(language, text) { + const longest = Math.max(2, ...[...text.matchAll(/`+/g)].map(([run]) => run.length)) + const fence = '`'.repeat(longest + 1) + return `${fence}${language}\n${text}${fence}\n` +} diff --git a/feature-table/render.test.mjs b/feature-table/render.test.mjs new file mode 100644 index 0000000..3674d12 --- /dev/null +++ b/feature-table/render.test.mjs @@ -0,0 +1,43 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' + +import { renderFeatureTable } from './render.mjs' + +const doc = { content: [], type: 'doc', version: 1 } +const held = { ok: true, value: doc } +const refused = { error: { code: 'unsupported-node-shape' }, ok: false } + +function library(overrides) { + const writes = (value) => () => ({ ok: true, value }) + return { adfToHtml: writes('

a

\n'), adfToLosslessMarkdown: writes('a\n'), adfToPortableMarkdown: writes('a\n'), htmlToAdf: () => held, losslessMarkdownToAdf: () => held, portableMarkdownToAdf: () => held, ...overrides } +} + +test('leads with a row per example linking its section, then the section labels each output', () => { + const page = renderFeatureTable([{ doc, name: 'Some example' }], library()) + assert.ok(page.includes('| [Some example](#some-example) | holds | holds | holds |\n')) + assert.ok(page.includes(`## Some example\n\nADF:\n\n\`\`\`json\n${JSON.stringify(doc)}\n\`\`\`\n\n\`adfToLosslessMarkdown\`, round-trip holds:\n\n\`\`\`markdown\na\n\`\`\`\n`)) + assert.ok(page.includes('`adfToHtml`, round-trip holds:\n\n```html\n

a

\n```\n')) +}) + +test('says the round-trip loses where the output reads back to another document', () => { + const page = renderFeatureTable([{ doc, name: 'Some example' }], library({ portableMarkdownToAdf: () => ({ ok: true, value: { ...doc, content: [{ type: 'rule' }] } }) })) + assert.ok(page.includes('| holds | loses | holds |')) + assert.ok(page.includes('`adfToPortableMarkdown`, round-trip loses:')) +}) + +test('compares documents whatever their key order', () => { + const page = renderFeatureTable([{ doc, name: 'Some example' }], library({ htmlToAdf: () => ({ ok: true, value: { version: 1, type: 'doc', content: [] } }) })) + assert.ok(page.includes('| holds | holds | holds |')) +}) + +test('names the code where a writer or a reader refuses', () => { + const page = renderFeatureTable([{ doc, name: 'Some example' }], library({ adfToHtml: () => refused, losslessMarkdownToAdf: () => refused })) + assert.ok(page.includes('| refused | holds | refused |')) + assert.ok(page.includes('`adfToLosslessMarkdown`, round-trip refused with `unsupported-node-shape`:\n\n```markdown\na\n```\n')) + assert.ok(page.includes('`adfToHtml` refuses with `unsupported-node-shape`.\n')) +}) + +test('fences an output in more backticks than it holds', () => { + const page = renderFeatureTable([{ doc, name: 'Some example' }], library({ adfToLosslessMarkdown: () => ({ ok: true, value: '````js\nx\n````\n' }) })) + assert.ok(page.includes('`````markdown\n````js\nx\n````\n`````\n')) +}) diff --git a/feature-table/write.sh b/feature-table/write.sh new file mode 100755 index 0000000..8cbad2c --- /dev/null +++ b/feature-table/write.sh @@ -0,0 +1,9 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")/.." +source ./docker-runner.sh + +in_image_memory=4g +leg "install ($node_image)" in_image "$node_image" npm ci +leg "build ($node_image)" in_image "$node_image" npm run build +leg "write the feature table ($node_image)" in_image "$node_image" node feature-table/build.mjs diff --git a/todo.md b/todo.md index 967f7b2..81d204c 100644 --- a/todo.md +++ b/todo.md @@ -6,18 +6,16 @@ `Bar = 0.1` -`Next ID = 168` +`Next ID = 169` ## Items | ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score | |---|---|---|---|---|---|---|---|---|---|---|---| -| 164 | 0.3.0 | | | **Publish a feature table setting chosen corpus examples beside what each call writes for them, readable at a glance by people and parseable by LLMs, and kept current by the gate.** | 3 | 2 | 1 | 4 | 5 | 3 by 6, 6 by 4 | 6.2 | -| 165 | 0.3.0 | 164 | | **Have a reader panel seated from `## Audience` judge each feature-table row reasonable or not, and file an item for each row it rejects.** | 2 | 2 | 1 | 3 | 3 | 6 by 5 | 7.5 | -| 132 | 0.3.0 | | | **Write tables as `` in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element.** | 2 | 2 | 2 | 4 | 5 | 6 by 4 | 2.4 | -| 73 | 0.3.0 | 132, 165 | | **Announce the package where someone needing an ADF converter already reads: JRACLOUD-77436, the Atlassian developer community and Stack Overflow's ADF-to-markdown questions.** | 1 | 2 | 2 | 3 | 2 | 3 by 8 | 16.9 | -| 167 | 0.3.0 | | decision | **Refuse in `refusedShape` the schema-forbidden shapes `adfToLosslessMarkdown` refuses in its walk, and drop them from that writer.** | 3 | 1 | 2 | 4 | 5 | code health by 5 | 13.1 | -| 72 | 0.3.0 | 132 | decision | **Re-create `gitea.larvit.se/larvit/adf-codec` as a read-only pull mirror of GitHub, its issues and Actions off and its description pointing at GitHub.** | 1 | 2 | 1 | 2 | 1 | 3 by 1 | 2.5 | +| 168 | 0.3.0 | | | **Lower the reading cost of the files a chunk reads most before its first commit.** | 2 | 1 | 1 | 2 | 4 | code health by 5 | 21.8 | +| 165 | 0.3.0 | | | **Have a reader panel seated from `## Audience` judge each feature-table row reasonable or not, and file an item for each row it rejects.** | 2 | 2 | 1 | 3 | 3 | 6 by 5 | 7.5 | +| 73 | 0.3.0 | 165 | | **Announce the package where someone needing an ADF converter already reads: JRACLOUD-77436, the Atlassian developer community and Stack Overflow's ADF-to-markdown questions.** | 1 | 2 | 2 | 3 | 2 | 3 by 8 | 16.9 | +| 72 | 0.3.0 | | decision | **Re-create `gitea.larvit.se/larvit/adf-codec` as a read-only pull mirror of GitHub, its issues and Actions off and its description pointing at GitHub.** | 1 | 2 | 1 | 2 | 1 | 3 by 1 | 2.5 | | 153 | 0.4.0 | | question | **Decide whether `src/markdown/` root may import `emit/`, `parse/` or `portable/`, and hold the answer in `.oxlintrc.json` and §The source parts by ADF and format.** | 1 | 2 | 2 | 3 | 2 | code health by 3 | 26.2 | | 156 | 0.4.0 | | | **Stop a leg's container when its script is interrupted, so Ctrl-C ends `ci.sh` or `mutation-tests/run.sh` at once.** | 1 | 2 | 1 | 2 | 1 | code health by 2 | 20.9 | | 128 | 0.4.0 | | | **Pin the portable flavour's block reading and writing with tests: callout titles past one line, a link beside other marks in a title, the nesting limit of callouts and task lists, refusals inside tasks and their paths, and empty or non-paragraph tasks.** | 2 | 1 | 1 | 2 | 3 | code health by 3 | 19.6 | @@ -67,19 +65,17 @@ ## Details -### 164. Publish a feature table setting chosen corpus examples beside what each call writes for them, readable at a glance by people and parseable by LLMs, and kept current by the gate. +### 168. Lower the reading cost of the files a chunk reads most before its first commit. -The examples are picked by an explicit list of corpus ids, each with a one-line name such as "bold instruction with a command in it". For each one, the table shows the ADF, `adfToLosslessMarkdown`, `adfToPortableMarkdown` and `adfToHtml` output, and whether the round-trip holds. A script builds the table from the built package. The gate fails when the committed table differs from what the script writes, so the table always shows what the release does. A writer panel from `## Audience` picks the layout, rendered or source, before it ships. +The last 5 chunks' median was 215876 new input tokens before their first commit, and the 5 before +them 147097. Most read: `todo.md` 36, `docs/decisions.md` 25, `README.md` 21, `spec/flavour.md` 16, +`ci.sh` 15, `src/html/parse/html-to-adf.test.ts` 13, `src/markdown/emit/adf-to-markdown.test.ts` 11, +`CHANGELOG.md` 9, `src/conformance/corpus.test.ts` 8, `MIGRATION.md` 7. ### 165. Have a reader panel seated from `## Audience` judge each feature-table row reasonable or not, and file an item for each row it rejects. The panel follows AGENTS.md §7's reader-panel rule: readers, one per persona the row serves, see only `## Audience` and the row, and say whether the output is what they expect. A rejected row becomes an item naming the expected output. The verdicts land in `docs/decisions.md`. -### 132. Write tables as `
` in `adfToHtml` and read them back in `htmlToAdf`, in place of the carry element. - -Header cells as `
`; a cell's span, width and background in HTML attributes or `data-*`, as a -writer panel picks. - ### 73. Announce the package where someone needing an ADF converter already reads: JRACLOUD-77436, the Atlassian developer community and Stack Overflow's ADF-to-markdown questions. Blocked by every other `0.3.0` item (the maintainer, 2026-10-14): the posts announce the finished @@ -96,36 +92,6 @@ solves it, then links the GitHub repository, the README's comparison table and t opening. The maintainer posts them once `0.3.0` is published (`docs/decisions.md` §Posts announcing a release wait for its publish). Found while planning discoverability, 2026-10-04. -### 167. Refuse in `refusedShape` the schema-forbidden shapes `adfToLosslessMarkdown` refuses in its walk, and drop them from that writer. - -The maintainer's call, 2026-11-04 (`docs/decisions.md` §Schema refusals are made in the first step -alone): the first step holds every schema check, "moved there from wherever else in the code they -are". It replaces item 68, which would have carried these shapes. Move into `refusedShape`: - -- `emitDirectiveBlock` in `emit/adf-to-markdown.ts`: a block holding `text`, and a leaf block - holding `content`. -- `emitLeaf` in `emit/inline-line.ts`: a non-text inline node holding `content` or `text`. - -Atlassian's schema gives each such node no `text` or `content` (`additionalProperties` is false). -Breaking: `adfToHtml` and `adfToPortableMarkdown` refuse these shapes too, and so does -`adfToLosslessMarkdown` inside a node that rides the carry, where it writes them today. Both carry -readers refuse them too, so a stored markdown or HTML document whose carry holds one stops -reading; `MIGRATION.md` says to convert it once with the last version that reads it. Say so in -`CHANGELOG.md` and `MIGRATION.md`. Search the writers for any other refusal of a shape the schema -forbids, and `carriedNode` for one the readers still build. - -The aim is the same behaviour in and out, with the checks alike in every format. The breaking -change above is what "alike" costs: `refusedShape` takes no format, so it cannot spare a node that -only one writer carries. The maintainer accepted it (2026-11-04); the item adds no other refusal. - -Stays where it is: a node holding one mark type twice (the schema allows it, no spelling holds it), -nesting depth, output length, a portable task list holding what no list item spells, and a -reader's refusal of markdown or HTML it cannot read. A node type, mark or attribute the schema does -not name, a known node out of place and a mark a node cannot hold keep riding the carry. Drop the -refused shapes from the README's `unsupported-node-shape` cause under writing in -`adfToLosslessMarkdown` and add them to the cause common to the writers, and from `spec/flavour.md`; -the property generators build none of them. - ### 72. Re-create `gitea.larvit.se/larvit/adf-codec` as a read-only pull mirror of GitHub, its issues and Actions off and its description pointing at GitHub. Blocked by every other `0.3.0` item but 73 (the maintainer, 2026-10-05; moved to `0.3.0` 2026-10-28,