diff --git a/AGENTS.md b/AGENTS.md index aa7dde3..7a91234 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,69 +1,118 @@ # Working in this repo -Decisions that a reader would otherwise relitigate. Everything about *using* the library is in -`README.md`; what is still to build, and what is still undecided, is in `todo.md`. +Decisions a reader would otherwise relitigate. Everything about *using* the library is in +`README.md`; what is still to build is in `todo.md`. -## 1. Two formats, never three +## 1. Three formats, ADF is the hub -ADF and one markdown flavour. **No HTML** — not as an output, not as an intermediate, not as a -convenience export. A consumer that wants HTML renders the markdown itself, with its own escaping -and its own stylesheet; a consumer that wants neither shows the markdown verbatim, which is what the -first one does. - -Three formats would mean six directions to keep lossless instead of two. +ADF, one markdown flavour, one HTML dialect. Six directions are exposed, but markdown↔HTML compose +through ADF, so four conversions exist to keep correct — never write a fifth. A fourth format +(wiki markup, anything else) is refused: each one doubles the directions again. ## 2. The round-trip is the product -`markdownToAdf(adfToMarkdown(doc))` must equal `doc`. Anything less and a consumer that lets someone -edit a ticket destroys what it could not represent — a panel, a mention, an attachment — in a -document it did not author. +`markdownToAdf(adfToMarkdown(doc))` and `htmlToAdf(adfToHtml(doc))` must equal `doc`. Anything less +and a consumer that lets someone edit a document destroys what it could not represent — a panel, a +mention, an attachment — in a document it did not author. -That is why the flavour is *extended*: markdown has no syntax for most of what ADF holds, so the -flavour invents it. Designing that syntax is the first real task, and it is open (`todo.md`). +When losslessness and readability conflict, losslessness wins. A rare node may look ugly; nothing +is ever dropped for looks. -Two consequences to settle before any node is implemented, not after: - -- **What happens to a node the library does not know.** The documented ADF node set is not the whole - schema, and Atlassian adds to it. Whether an unknown node is carried opaquely, refused, or dropped - is a correctness decision for the whole library, and it decides the return shape of both functions. -- **Whether a lossless document must stay readable to a plain markdown reader.** Anything the flavour - invents is noise to a reader that does not know it. How much noise is acceptable bounds the syntax. +The other direction is a **canonical fixpoint**, not byte-identity: human markdown normalizes +(`*em*` and `_em_` become one ADF doc), converting back yields the library's canonical spelling, +and that spelling round-trips byte-identically. Byte-identity both ways would mean storing the +author's formatting choices in ADF, which has no place for them. Round-trip equality is a property to test over a corpus, not a claim to make in prose. -## 3. Zero runtime dependencies +## 3. Unknown input policy + +- **Unknown ADF node** (Atlassian ships undocumented types and adds more): carried opaquely — the + raw JSON rides a dedicated syntax in markdown and HTML, and converts back byte-for-byte. The + round-trip holds even for documents newer than the library. +- **Unmappable foreign HTML element**: error result naming the element. Arbitrary HTML can express + what ADF cannot hold, so import maps a documented element set and refuses the rest — never a + silent drop. +- **Bare `@name` / `:smile:` typed as plain text**: stays a text node. Only the directive syntax + produces mention/emoji/media nodes — resolving names to ids needs I/O, which is the consumer's + job (e.g. an autocomplete that inserts the directive). + +## 4. The flavour + +- **Directives**, one grammar for everything markdown lacks: `:::panel info` … `:::` for blocks, + `:mention[@Mikael]{id=5b10a2}` style inline. Prior art: CommonMark's generic-directives proposal. +- **Plain CommonMark is a subset.** Anything a human types converts per the spec; the flavour only + adds syntax, never changes CommonMark meaning. +- **Tables**: a simple table (one header row, plain inline cells — no spans, widths, colours, block + content) emits a pipe table; anything richer emits the directive form. +- **Identity-bearing nodes** (`mention`, `media`, `emoji`) carry their ids in attributes. A + converted document is therefore only portable within the site it came from — accepted. + +The HTML dialect mirrors this: semantic elements, stable `adf-*` class names, `data-*` attributes +for what HTML cannot express, text content always escaped. No stylesheet ships — styling is the +consumer's. + +## 5. Zero runtime dependencies Nothing in `dependencies`, ever. TypeScript and whatever the tests need are `devDependencies`, and -they never reach a consumer. A markdown parser is exactly the dependency this rule exists to refuse: -the flavour is not CommonMark, so a general parser would have to be extended into one anyway. +they never reach a consumer. This means the CommonMark parser and the (well-formed) HTML parser are +written in this repo — a general parser would have to be extended into the flavour anyway. -## 4. The package contract +## 6. The package contract - **ESM only.** No CommonJS build, no dual-package hazard. -- **Two entrypoints.** The built JavaScript for ordinary consumers, and the TypeScript source for - consumers that run TypeScript directly through Node's type stripping — the first consumer is one, - which is why this exists. -- **Types for both.** The JavaScript entrypoint ships `.d.ts` beside it; the TypeScript entrypoint is - its own types. -- **Published to public npmjs as `@larvit/atlassian-adf-converter`**, matching `@larvit/log`. Public - means the source is public: the Gitea repo starts private, and going public — with the LICENSE in +- **Two entrypoints.** Built JavaScript for ordinary consumers; TypeScript source for consumers + running TypeScript directly through Node's type stripping. +- **Types for both.** The JavaScript entrypoint ships `.d.ts` beside it; the TypeScript entrypoint + is its own types. +- **Published to public npmjs as `@larvit/atlassian-adf-converter`**, matching `@larvit/log`. + Public means the source is public: the Gitea repo starts private, and going public — LICENSE in place — is a step before the first publish, not after it. - **Exact versions.** `save-exact=true` in `.npmrc`, as in every other repo here. -## 5. Nothing about any consumer +## 7. Nothing about any consumer -No Jira, no HTTP, no REST response shapes, no plainpages, no issue keys. The library takes a document -tree and returns a string, or the reverse. A consumer's concern that leaks in here is a seam nobody -declared — and the reason this is a library at all rather than a file in the client that needed it. +No Jira client, no HTTP, no REST response shapes, no issue keys — and no actual consumer named +anywhere in this repo. Design against the README's personas, not against a product. A consumer's +concern that leaks in here is a seam nobody declared. -## 6. Tests first, in Docker +## 8. Semver: the formats are API + +The emitted markdown and HTML are contracts, not just the TypeScript surface. After 1.0: a change +that makes previously-emitted output parse differently, or stop parsing, is MAJOR; teaching the +parsers new syntax while old output still round-trips is MINOR. Documents in flight — stored, +cached, mid-edit — survive upgrades. Pre-1.0, normal 0.x rules apply. + +## 9. Release automation + +- `package.json` version on `main` is the source of truth. CI on every `main` push: tests green + and version differs from npm's latest → publish and tag `vX.Y.Z`. A merge without a bump (docs, + CI, dep bumps) deploys nothing; the bump in each shipping PR is a deliberate semver judgment. +- **Renovate** watches npm devDependencies, Docker image pins and action tags, and automerges + every update — majors included — when CI is green. + +## 10. Tests first, in Docker Write the test for the behaviour wanted, then implement until it passes. `node --test`, beside the -code. Node, tsc and npm never run on the host — a compose service or a `docker run` against a -**full patch version** image tag (`node:24.19.0-alpine3.24`, never `node:24`), so the same commit -builds the same thing on a different day. +code. Node, tsc and npm never run on the host — a compose service or `docker run` against a +**full patch version** image tag (`node:24.19.0-alpine3.24`, never `node:24`). -## 7. Style +The corpus, all checked in: hand-built fixtures per node and combination; real (sanitized) ADF +captured from live Atlassian APIs, including undocumented nodes; property-based generated ADF trees +(generator is a devDependency); the official CommonMark spec suite, run against `markdownToAdf` +(mapped) and `markdownToHtml`. -Two-space indent, alphabetically sorted object keys, strict TypeScript. Failures are values, not -exceptions: a function that both returns a result and throws for some inputs has two error channels. +## 11. Style + +Two-space indent, alphabetically sorted object keys, strict TypeScript. Failures are values: +every conversion returns `Result` — `{ ok: true; value } | { ok: false; error: ConvertError }` — +and nothing throws. Input validation is structural only: what the walk needs (a node is an object +with a string `type`, known nodes have the attrs read from them), nothing more. + +## 12. Non-goals + +Stated so nobody builds them in: no wiki markup (§1), no network or filesystem I/O, no name→id +resolution (§3), no ADF schema validation or exported validator — whether Atlassian accepts a tree +is Atlassian's call — no shipped CSS (§4), no streaming APIs and no performance budget: conversions +are O(n) with no pathological blowups, and real documents are kilobytes. A CLI is a later goal +(`todo.md`), not a non-goal. diff --git a/README.md b/README.md index 11071ae..a7868b4 100644 --- a/README.md +++ b/README.md @@ -1,42 +1,72 @@ # @larvit/atlassian-adf-converter -Lossless conversion between **Atlassian Document Format** (ADF) and an extended markdown flavour -that can carry the nodes plain markdown has no syntax for. +Lossless conversion between **Atlassian Document Format** (ADF), an extended markdown flavour, and +an HTML dialect. -**Status: specification only. No code is implemented yet.** `todo.md` holds the plan and the design -questions still open; `AGENTS.md` holds the decisions already made. +**Status: specification only. No code is implemented yet.** `todo.md` holds the plan; `AGENTS.md` +holds the decisions already made. ## What it is for -Jira Cloud's REST v3 API hands out issue descriptions and comment bodies as ADF — a JSON node tree, -ProseMirror-shaped — and takes them back the same way. There is no Atlassian endpoint that converts -it: `pf-editor-service/convert` was decommissioned and +Atlassian Cloud REST APIs (Jira v3, Confluence) hand out rich text — issue descriptions, comments, +pages — as ADF, a JSON node tree, ProseMirror-shaped, and take it back the same way. There is no +Atlassian endpoint that converts it: `pf-editor-service/convert` was decommissioned and [JRACLOUD-77436](https://jira.atlassian.com/browse/JRACLOUD-77436) is still an open request. The npm ecosystem covers one direction each, drops what markdown cannot express, and none of it round-trips. -A client that shows a ticket and lets someone edit it needs both directions, and needs them +A consumer that shows a document and lets someone edit it needs both directions, and needs them lossless — otherwise saving an edit silently destroys the panels, mentions and attachments that were -in someone else's ticket. That is what this library is. +in someone else's document. That is what this library is. -## The intended shape +## The shape -Two pure functions and their types. No I/O, no network, no configuration: +Pure functions and their types. No I/O, no network, no configuration. ADF is the hub: the +markdown↔HTML directions compose through it. ```ts -adfToMarkdown(document: AdfDocument): string -markdownToAdf(markdown: string): AdfDocument +adfToMarkdown(doc: AdfDocument): Result +markdownToAdf(markdown: string): Result +adfToHtml(doc: AdfDocument): Result +htmlToAdf(html: string): Result +markdownToHtml(markdown: string): Result // via ADF +htmlToMarkdown(html: string): Result // via ADF +isAdfDocument(v: unknown): v is AdfDocument ``` -The published package is ESM only, has **no runtime dependencies**, and offers two entrypoints — the -built JavaScript for ordinary consumers, and the TypeScript source for consumers that run TypeScript -directly (Node's type stripping), with exported types either way. `AGENTS.md` §4 has the contract. +`Result` is `{ ok: true; value: T } | { ok: false; error: ConvertError }` — nothing throws. -## The first consumer +## The guarantees -[`plainpages-plugin-fastjira`](https://gitea.larvit.se/larvit/plainpages-plugin-fastjira) — a -server-rendered Jira client. Its read-only ticket view shows the markdown this library produces -verbatim, with no HTML rendering anywhere; its later write paths post back what this library -converts the other way. **That view is blocked on `0.1.0`,** and it needs `adfToMarkdown` first. +- **`markdownToAdf(adfToMarkdown(doc))` equals `doc`** — including node types the library has never + seen, which are carried opaquely (AGENTS.md §3). +- **`htmlToAdf(adfToHtml(doc))` equals `doc`** — fidelity HTML cannot express rides `data-*` + attributes. +- **Plain CommonMark is valid input** to `markdownToAdf`: any ordinary markdown a human types + converts per the CommonMark spec. Converting back yields the library's canonical spelling, which + then round-trips byte-identically. +- **Foreign HTML** (not emitted by this library) maps a documented element set; an unmappable + element is an error result, never a silent drop. The parser takes well-formed HTML, not + WHATWG tag-soup recovery. +- **The emitted formats are semver surface** (AGENTS.md §8): after 1.0, output an old version + emitted always parses under a newer one within the same major. -The library knows nothing about that consumer. No Jira, no HTTP, no REST shapes, no plainpages — a -document tree in, a string out, and the reverse. +## Who it is for + +No actual consumer is named here or anywhere in this repo (AGENTS.md §7). The personas the design +serves: + +- **A viewer/editor app** — shows a document as markdown or HTML, lets a human edit, posts the + result back. Needs losslessness above all. +- **A bot posting content** — generates ordinary markdown (templates, LLM output) and converts it + to ADF. Needs the CommonMark input promise; never reads ADF back. +- **An export/indexing tool** — bulk-converts ADF to markdown or HTML for archives, search, static + sites. Read-only; needs readable output. +- **An LLM/agent pipeline** — feeds documents to a model as markdown, converts the model's edits + back. Needs the round-trip plus markdown that stays legible to a reader that half-knows the + flavour. + +## The package + +ESM only, **no runtime dependencies**, published to public npmjs. Two entrypoints — built +JavaScript for ordinary consumers, TypeScript source for consumers running Node's type stripping — +with exported types either way. `AGENTS.md` §5–6 have the contract. diff --git a/todo.md b/todo.md index 1792a11..fc4287a 100644 --- a/todo.md +++ b/todo.md @@ -1,40 +1,33 @@ # Todo -The plan, in order. Nothing here is built yet. - -## Open design questions — settle these first - -None of them have an answer yet, and each one changes what every later milestone implements. - -- [ ] **The flavour's syntax.** Markdown has no syntax for most of ADF. Every node in the inventory - below that is not plain markdown needs one, and the set has to be internally consistent rather - than invented node by node. Prior art worth reading before choosing: CommonMark's generic - directives proposal, MDX, Obsidian's and Pandoc's extensions, and what Atlassian's own - `editor-markdown-transformer` does (it is lossy — read it for the failure modes, not the design). -- [ ] **The unknown-node policy** (AGENTS.md §2). Carried opaquely, refused, or dropped — it decides - the return shape of both functions, so it cannot be retrofitted. -- [ ] **How readable a converted document must stay** to a reader that does not know the flavour. -- [ ] **Whether plain CommonMark is valid input** to `markdownToAdf`. A human typing ordinary - markdown into a comment box is the second consumer's whole write path. -- [ ] **Table fidelity.** ADF tables carry column widths, colspan, rowspan, header rows and cell - background colours; markdown tables carry none of it. -- [ ] **Identity-bearing nodes.** `mention` holds an account id, `media` an attachment id, `emoji` a - shortcode plus an id. The rendered text is not enough to reconstruct them, so the syntax has to - carry the id — and then a document is only portable within the site it came from. +The plan, in order. Nothing here is built yet. The design questions that used to be open are +settled in `AGENTS.md`; what remains open below is spec detail, settled at its own milestone. ## Milestones -- [ ] **0 — Scaffold.** `package.json` with the §4 contract, `tsconfig.json`, `.npmrc`, LICENSE (MIT, - Larv IT AB), the Docker tooling setup, and `.gitea/workflows/ci.yml` gating branches. Mirror - `plainpages` for the workflow shape: `runs-on: docker-host`, actions pinned to semver tags. -- [ ] **1 — Settle the flavour.** Write the syntax down as this repo's specification before - implementing it, and make the round-trip corpus from it. -- [ ] **2 — `adfToMarkdown`.** The direction the first consumer needs. Ships `0.1.0`. -- [ ] **3 — `markdownToAdf`.** -- [ ] **4 — Round-trip property tests** over a corpus of real Jira documents, both ways. Not a - milestone that follows 2 and 3 so much as the thing that proves them. -- [ ] **5 — Release pipeline.** Tag-triggered publish to public npmjs, `NPM_TOKEN` secret, the repo - made public with the LICENSE in place first (AGENTS.md §4). +- [ ] **0 — Scaffold.** `package.json` with the §6 contract, `tsconfig.json`, `.npmrc` + (`save-exact=true`), the Docker tooling setup, `renovate.json` (automerge-on-green, §9), and + `.gitea/workflows/ci.yml` gating branches: `runs-on: docker-host`, actions pinned to semver + tags. +- [ ] **1 — The flavour spec.** Write the markdown flavour down as this repo's specification before + implementing it: the exact directive grammar (attributes, escaping, nesting), each node's + syntax from the inventory below, the opaque-carry spelling for unknown nodes, and the + pipe-vs-directive table rule. Open detail to settle here: what CommonMark's raw-HTML + constructs become in ADF, which has no raw-HTML node — likely the §3 element mapping, error + otherwise. Start the corpus (§10) from this spec. +- [ ] **2 — `adfToMarkdown`.** +- [ ] **3 — `markdownToAdf`.** The CommonMark parser is the largest single component. +- [ ] **4 — Round-trip property tests** over the corpus, both ways. Not a milestone that follows + 2 and 3 so much as the thing that proves them. +- [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change on `main` (§9), + `NPM_TOKEN` secret, the repo made public first (§6). `0.1.0` is the markdown round-trip: + both markdown directions, the types, `isAdfDocument`. +- [ ] **6 — The HTML dialect spec.** Element-by-element mapping, the `data-*` fidelity scheme, the + opaque-carry form, and the documented foreign-element set `htmlToAdf` accepts. +- [ ] **7 — HTML, ship `0.2.0`.** `adfToHtml`, `htmlToAdf`, and the composed `markdownToHtml` / + `htmlToMarkdown`. CommonMark spec suite runs against `markdownToHtml` from here (§10). +- [ ] **8 — CLI.** A later goal, deliberately unspecified until the library exists: shape it + around the personas then. ## The ADF inventory to cover @@ -43,7 +36,8 @@ reference](https://developer.atlassian.com/cloud/jira/platform/apis/document/str not the whole schema** — real payloads also carry `taskList`/`taskItem`, `decisionList`/`decisionItem`, `layoutSection`/`layoutColumn`, `blockCard`/`embedCard`, `extension`/`bodiedExtension`/`inlineExtension` and `placeholder`, none of which are documented there. Treat the documented set as the floor, not the -ceiling, and see the unknown-node policy above. +ceiling — the floor gets designed syntax, everything else rides the opaque carry (§3) until it too +gets syntax. | | | | --- | --- |