Settle goals, guarantees, HTML scope and release automation in the spec docs
This commit is contained in:
@@ -1,69 +1,118 @@
|
||||
# Working in this repo
|
||||
|
||||
Decisions that a reader would otherwise relitigate. Everything about *using* the library is in
|
||||
`README.md`; what is still to build, and what is still undecided, is in `todo.md`.
|
||||
Decisions a reader would otherwise relitigate. Everything about *using* the library is in
|
||||
`README.md`; what is still to build is in `todo.md`.
|
||||
|
||||
## 1. Two formats, never three
|
||||
## 1. Three formats, ADF is the hub
|
||||
|
||||
ADF and one markdown flavour. **No HTML** — not as an output, not as an intermediate, not as a
|
||||
convenience export. A consumer that wants HTML renders the markdown itself, with its own escaping
|
||||
and its own stylesheet; a consumer that wants neither shows the markdown verbatim, which is what the
|
||||
first one does.
|
||||
|
||||
Three formats would mean six directions to keep lossless instead of two.
|
||||
ADF, one markdown flavour, one HTML dialect. Six directions are exposed, but markdown↔HTML compose
|
||||
through ADF, so four conversions exist to keep correct — never write a fifth. A fourth format
|
||||
(wiki markup, anything else) is refused: each one doubles the directions again.
|
||||
|
||||
## 2. The round-trip is the product
|
||||
|
||||
`markdownToAdf(adfToMarkdown(doc))` must equal `doc`. Anything less and a consumer that lets someone
|
||||
edit a ticket destroys what it could not represent — a panel, a mention, an attachment — in a
|
||||
document it did not author.
|
||||
`markdownToAdf(adfToMarkdown(doc))` and `htmlToAdf(adfToHtml(doc))` must equal `doc`. Anything less
|
||||
and a consumer that lets someone edit a document destroys what it could not represent — a panel, a
|
||||
mention, an attachment — in a document it did not author.
|
||||
|
||||
That is why the flavour is *extended*: markdown has no syntax for most of what ADF holds, so the
|
||||
flavour invents it. Designing that syntax is the first real task, and it is open (`todo.md`).
|
||||
When losslessness and readability conflict, losslessness wins. A rare node may look ugly; nothing
|
||||
is ever dropped for looks.
|
||||
|
||||
Two consequences to settle before any node is implemented, not after:
|
||||
|
||||
- **What happens to a node the library does not know.** The documented ADF node set is not the whole
|
||||
schema, and Atlassian adds to it. Whether an unknown node is carried opaquely, refused, or dropped
|
||||
is a correctness decision for the whole library, and it decides the return shape of both functions.
|
||||
- **Whether a lossless document must stay readable to a plain markdown reader.** Anything the flavour
|
||||
invents is noise to a reader that does not know it. How much noise is acceptable bounds the syntax.
|
||||
The other direction is a **canonical fixpoint**, not byte-identity: human markdown normalizes
|
||||
(`*em*` and `_em_` become one ADF doc), converting back yields the library's canonical spelling,
|
||||
and that spelling round-trips byte-identically. Byte-identity both ways would mean storing the
|
||||
author's formatting choices in ADF, which has no place for them.
|
||||
|
||||
Round-trip equality is a property to test over a corpus, not a claim to make in prose.
|
||||
|
||||
## 3. Zero runtime dependencies
|
||||
## 3. Unknown input policy
|
||||
|
||||
- **Unknown ADF node** (Atlassian ships undocumented types and adds more): carried opaquely — the
|
||||
raw JSON rides a dedicated syntax in markdown and HTML, and converts back byte-for-byte. The
|
||||
round-trip holds even for documents newer than the library.
|
||||
- **Unmappable foreign HTML element**: error result naming the element. Arbitrary HTML can express
|
||||
what ADF cannot hold, so import maps a documented element set and refuses the rest — never a
|
||||
silent drop.
|
||||
- **Bare `@name` / `:smile:` typed as plain text**: stays a text node. Only the directive syntax
|
||||
produces mention/emoji/media nodes — resolving names to ids needs I/O, which is the consumer's
|
||||
job (e.g. an autocomplete that inserts the directive).
|
||||
|
||||
## 4. The flavour
|
||||
|
||||
- **Directives**, one grammar for everything markdown lacks: `:::panel info` … `:::` for blocks,
|
||||
`:mention[@Mikael]{id=5b10a2}` style inline. Prior art: CommonMark's generic-directives proposal.
|
||||
- **Plain CommonMark is a subset.** Anything a human types converts per the spec; the flavour only
|
||||
adds syntax, never changes CommonMark meaning.
|
||||
- **Tables**: a simple table (one header row, plain inline cells — no spans, widths, colours, block
|
||||
content) emits a pipe table; anything richer emits the directive form.
|
||||
- **Identity-bearing nodes** (`mention`, `media`, `emoji`) carry their ids in attributes. A
|
||||
converted document is therefore only portable within the site it came from — accepted.
|
||||
|
||||
The HTML dialect mirrors this: semantic elements, stable `adf-*` class names, `data-*` attributes
|
||||
for what HTML cannot express, text content always escaped. No stylesheet ships — styling is the
|
||||
consumer's.
|
||||
|
||||
## 5. Zero runtime dependencies
|
||||
|
||||
Nothing in `dependencies`, ever. TypeScript and whatever the tests need are `devDependencies`, and
|
||||
they never reach a consumer. A markdown parser is exactly the dependency this rule exists to refuse:
|
||||
the flavour is not CommonMark, so a general parser would have to be extended into one anyway.
|
||||
they never reach a consumer. This means the CommonMark parser and the (well-formed) HTML parser are
|
||||
written in this repo — a general parser would have to be extended into the flavour anyway.
|
||||
|
||||
## 4. The package contract
|
||||
## 6. The package contract
|
||||
|
||||
- **ESM only.** No CommonJS build, no dual-package hazard.
|
||||
- **Two entrypoints.** The built JavaScript for ordinary consumers, and the TypeScript source for
|
||||
consumers that run TypeScript directly through Node's type stripping — the first consumer is one,
|
||||
which is why this exists.
|
||||
- **Types for both.** The JavaScript entrypoint ships `.d.ts` beside it; the TypeScript entrypoint is
|
||||
its own types.
|
||||
- **Published to public npmjs as `@larvit/atlassian-adf-converter`**, matching `@larvit/log`. Public
|
||||
means the source is public: the Gitea repo starts private, and going public — with the LICENSE in
|
||||
- **Two entrypoints.** Built JavaScript for ordinary consumers; TypeScript source for consumers
|
||||
running TypeScript directly through Node's type stripping.
|
||||
- **Types for both.** The JavaScript entrypoint ships `.d.ts` beside it; the TypeScript entrypoint
|
||||
is its own types.
|
||||
- **Published to public npmjs as `@larvit/atlassian-adf-converter`**, matching `@larvit/log`.
|
||||
Public means the source is public: the Gitea repo starts private, and going public — LICENSE in
|
||||
place — is a step before the first publish, not after it.
|
||||
- **Exact versions.** `save-exact=true` in `.npmrc`, as in every other repo here.
|
||||
|
||||
## 5. Nothing about any consumer
|
||||
## 7. Nothing about any consumer
|
||||
|
||||
No Jira, no HTTP, no REST response shapes, no plainpages, no issue keys. The library takes a document
|
||||
tree and returns a string, or the reverse. A consumer's concern that leaks in here is a seam nobody
|
||||
declared — and the reason this is a library at all rather than a file in the client that needed it.
|
||||
No Jira client, no HTTP, no REST response shapes, no issue keys — and no actual consumer named
|
||||
anywhere in this repo. Design against the README's personas, not against a product. A consumer's
|
||||
concern that leaks in here is a seam nobody declared.
|
||||
|
||||
## 6. Tests first, in Docker
|
||||
## 8. Semver: the formats are API
|
||||
|
||||
The emitted markdown and HTML are contracts, not just the TypeScript surface. After 1.0: a change
|
||||
that makes previously-emitted output parse differently, or stop parsing, is MAJOR; teaching the
|
||||
parsers new syntax while old output still round-trips is MINOR. Documents in flight — stored,
|
||||
cached, mid-edit — survive upgrades. Pre-1.0, normal 0.x rules apply.
|
||||
|
||||
## 9. Release automation
|
||||
|
||||
- `package.json` version on `main` is the source of truth. CI on every `main` push: tests green
|
||||
and version differs from npm's latest → publish and tag `vX.Y.Z`. A merge without a bump (docs,
|
||||
CI, dep bumps) deploys nothing; the bump in each shipping PR is a deliberate semver judgment.
|
||||
- **Renovate** watches npm devDependencies, Docker image pins and action tags, and automerges
|
||||
every update — majors included — when CI is green.
|
||||
|
||||
## 10. Tests first, in Docker
|
||||
|
||||
Write the test for the behaviour wanted, then implement until it passes. `node --test`, beside the
|
||||
code. Node, tsc and npm never run on the host — a compose service or a `docker run` against a
|
||||
**full patch version** image tag (`node:24.19.0-alpine3.24`, never `node:24`), so the same commit
|
||||
builds the same thing on a different day.
|
||||
code. Node, tsc and npm never run on the host — a compose service or `docker run` against a
|
||||
**full patch version** image tag (`node:24.19.0-alpine3.24`, never `node:24`).
|
||||
|
||||
## 7. Style
|
||||
The corpus, all checked in: hand-built fixtures per node and combination; real (sanitized) ADF
|
||||
captured from live Atlassian APIs, including undocumented nodes; property-based generated ADF trees
|
||||
(generator is a devDependency); the official CommonMark spec suite, run against `markdownToAdf`
|
||||
(mapped) and `markdownToHtml`.
|
||||
|
||||
Two-space indent, alphabetically sorted object keys, strict TypeScript. Failures are values, not
|
||||
exceptions: a function that both returns a result and throws for some inputs has two error channels.
|
||||
## 11. Style
|
||||
|
||||
Two-space indent, alphabetically sorted object keys, strict TypeScript. Failures are values:
|
||||
every conversion returns `Result<T>` — `{ ok: true; value } | { ok: false; error: ConvertError }` —
|
||||
and nothing throws. Input validation is structural only: what the walk needs (a node is an object
|
||||
with a string `type`, known nodes have the attrs read from them), nothing more.
|
||||
|
||||
## 12. Non-goals
|
||||
|
||||
Stated so nobody builds them in: no wiki markup (§1), no network or filesystem I/O, no name→id
|
||||
resolution (§3), no ADF schema validation or exported validator — whether Atlassian accepts a tree
|
||||
is Atlassian's call — no shipped CSS (§4), no streaming APIs and no performance budget: conversions
|
||||
are O(n) with no pathological blowups, and real documents are kilobytes. A CLI is a later goal
|
||||
(`todo.md`), not a non-goal.
|
||||
|
||||
@@ -1,42 +1,72 @@
|
||||
# @larvit/atlassian-adf-converter
|
||||
|
||||
Lossless conversion between **Atlassian Document Format** (ADF) and an extended markdown flavour
|
||||
that can carry the nodes plain markdown has no syntax for.
|
||||
Lossless conversion between **Atlassian Document Format** (ADF), an extended markdown flavour, and
|
||||
an HTML dialect.
|
||||
|
||||
**Status: specification only. No code is implemented yet.** `todo.md` holds the plan and the design
|
||||
questions still open; `AGENTS.md` holds the decisions already made.
|
||||
**Status: specification only. No code is implemented yet.** `todo.md` holds the plan; `AGENTS.md`
|
||||
holds the decisions already made.
|
||||
|
||||
## What it is for
|
||||
|
||||
Jira Cloud's REST v3 API hands out issue descriptions and comment bodies as ADF — a JSON node tree,
|
||||
ProseMirror-shaped — and takes them back the same way. There is no Atlassian endpoint that converts
|
||||
it: `pf-editor-service/convert` was decommissioned and
|
||||
Atlassian Cloud REST APIs (Jira v3, Confluence) hand out rich text — issue descriptions, comments,
|
||||
pages — as ADF, a JSON node tree, ProseMirror-shaped, and take it back the same way. There is no
|
||||
Atlassian endpoint that converts it: `pf-editor-service/convert` was decommissioned and
|
||||
[JRACLOUD-77436](https://jira.atlassian.com/browse/JRACLOUD-77436) is still an open request. The npm
|
||||
ecosystem covers one direction each, drops what markdown cannot express, and none of it round-trips.
|
||||
|
||||
A client that shows a ticket and lets someone edit it needs both directions, and needs them
|
||||
A consumer that shows a document and lets someone edit it needs both directions, and needs them
|
||||
lossless — otherwise saving an edit silently destroys the panels, mentions and attachments that were
|
||||
in someone else's ticket. That is what this library is.
|
||||
in someone else's document. That is what this library is.
|
||||
|
||||
## The intended shape
|
||||
## The shape
|
||||
|
||||
Two pure functions and their types. No I/O, no network, no configuration:
|
||||
Pure functions and their types. No I/O, no network, no configuration. ADF is the hub: the
|
||||
markdown↔HTML directions compose through it.
|
||||
|
||||
```ts
|
||||
adfToMarkdown(document: AdfDocument): string
|
||||
markdownToAdf(markdown: string): AdfDocument
|
||||
adfToMarkdown(doc: AdfDocument): Result<string>
|
||||
markdownToAdf(markdown: string): Result<AdfDocument>
|
||||
adfToHtml(doc: AdfDocument): Result<string>
|
||||
htmlToAdf(html: string): Result<AdfDocument>
|
||||
markdownToHtml(markdown: string): Result<string> // via ADF
|
||||
htmlToMarkdown(html: string): Result<string> // via ADF
|
||||
isAdfDocument(v: unknown): v is AdfDocument
|
||||
```
|
||||
|
||||
The published package is ESM only, has **no runtime dependencies**, and offers two entrypoints — the
|
||||
built JavaScript for ordinary consumers, and the TypeScript source for consumers that run TypeScript
|
||||
directly (Node's type stripping), with exported types either way. `AGENTS.md` §4 has the contract.
|
||||
`Result<T>` is `{ ok: true; value: T } | { ok: false; error: ConvertError }` — nothing throws.
|
||||
|
||||
## The first consumer
|
||||
## The guarantees
|
||||
|
||||
[`plainpages-plugin-fastjira`](https://gitea.larvit.se/larvit/plainpages-plugin-fastjira) — a
|
||||
server-rendered Jira client. Its read-only ticket view shows the markdown this library produces
|
||||
verbatim, with no HTML rendering anywhere; its later write paths post back what this library
|
||||
converts the other way. **That view is blocked on `0.1.0`,** and it needs `adfToMarkdown` first.
|
||||
- **`markdownToAdf(adfToMarkdown(doc))` equals `doc`** — including node types the library has never
|
||||
seen, which are carried opaquely (AGENTS.md §3).
|
||||
- **`htmlToAdf(adfToHtml(doc))` equals `doc`** — fidelity HTML cannot express rides `data-*`
|
||||
attributes.
|
||||
- **Plain CommonMark is valid input** to `markdownToAdf`: any ordinary markdown a human types
|
||||
converts per the CommonMark spec. Converting back yields the library's canonical spelling, which
|
||||
then round-trips byte-identically.
|
||||
- **Foreign HTML** (not emitted by this library) maps a documented element set; an unmappable
|
||||
element is an error result, never a silent drop. The parser takes well-formed HTML, not
|
||||
WHATWG tag-soup recovery.
|
||||
- **The emitted formats are semver surface** (AGENTS.md §8): after 1.0, output an old version
|
||||
emitted always parses under a newer one within the same major.
|
||||
|
||||
The library knows nothing about that consumer. No Jira, no HTTP, no REST shapes, no plainpages — a
|
||||
document tree in, a string out, and the reverse.
|
||||
## Who it is for
|
||||
|
||||
No actual consumer is named here or anywhere in this repo (AGENTS.md §7). The personas the design
|
||||
serves:
|
||||
|
||||
- **A viewer/editor app** — shows a document as markdown or HTML, lets a human edit, posts the
|
||||
result back. Needs losslessness above all.
|
||||
- **A bot posting content** — generates ordinary markdown (templates, LLM output) and converts it
|
||||
to ADF. Needs the CommonMark input promise; never reads ADF back.
|
||||
- **An export/indexing tool** — bulk-converts ADF to markdown or HTML for archives, search, static
|
||||
sites. Read-only; needs readable output.
|
||||
- **An LLM/agent pipeline** — feeds documents to a model as markdown, converts the model's edits
|
||||
back. Needs the round-trip plus markdown that stays legible to a reader that half-knows the
|
||||
flavour.
|
||||
|
||||
## The package
|
||||
|
||||
ESM only, **no runtime dependencies**, published to public npmjs. Two entrypoints — built
|
||||
JavaScript for ordinary consumers, TypeScript source for consumers running Node's type stripping —
|
||||
with exported types either way. `AGENTS.md` §5–6 have the contract.
|
||||
|
||||
@@ -1,40 +1,33 @@
|
||||
# Todo
|
||||
|
||||
The plan, in order. Nothing here is built yet.
|
||||
|
||||
## Open design questions — settle these first
|
||||
|
||||
None of them have an answer yet, and each one changes what every later milestone implements.
|
||||
|
||||
- [ ] **The flavour's syntax.** Markdown has no syntax for most of ADF. Every node in the inventory
|
||||
below that is not plain markdown needs one, and the set has to be internally consistent rather
|
||||
than invented node by node. Prior art worth reading before choosing: CommonMark's generic
|
||||
directives proposal, MDX, Obsidian's and Pandoc's extensions, and what Atlassian's own
|
||||
`editor-markdown-transformer` does (it is lossy — read it for the failure modes, not the design).
|
||||
- [ ] **The unknown-node policy** (AGENTS.md §2). Carried opaquely, refused, or dropped — it decides
|
||||
the return shape of both functions, so it cannot be retrofitted.
|
||||
- [ ] **How readable a converted document must stay** to a reader that does not know the flavour.
|
||||
- [ ] **Whether plain CommonMark is valid input** to `markdownToAdf`. A human typing ordinary
|
||||
markdown into a comment box is the second consumer's whole write path.
|
||||
- [ ] **Table fidelity.** ADF tables carry column widths, colspan, rowspan, header rows and cell
|
||||
background colours; markdown tables carry none of it.
|
||||
- [ ] **Identity-bearing nodes.** `mention` holds an account id, `media` an attachment id, `emoji` a
|
||||
shortcode plus an id. The rendered text is not enough to reconstruct them, so the syntax has to
|
||||
carry the id — and then a document is only portable within the site it came from.
|
||||
The plan, in order. Nothing here is built yet. The design questions that used to be open are
|
||||
settled in `AGENTS.md`; what remains open below is spec detail, settled at its own milestone.
|
||||
|
||||
## Milestones
|
||||
|
||||
- [ ] **0 — Scaffold.** `package.json` with the §4 contract, `tsconfig.json`, `.npmrc`, LICENSE (MIT,
|
||||
Larv IT AB), the Docker tooling setup, and `.gitea/workflows/ci.yml` gating branches. Mirror
|
||||
`plainpages` for the workflow shape: `runs-on: docker-host`, actions pinned to semver tags.
|
||||
- [ ] **1 — Settle the flavour.** Write the syntax down as this repo's specification before
|
||||
implementing it, and make the round-trip corpus from it.
|
||||
- [ ] **2 — `adfToMarkdown`.** The direction the first consumer needs. Ships `0.1.0`.
|
||||
- [ ] **3 — `markdownToAdf`.**
|
||||
- [ ] **4 — Round-trip property tests** over a corpus of real Jira documents, both ways. Not a
|
||||
milestone that follows 2 and 3 so much as the thing that proves them.
|
||||
- [ ] **5 — Release pipeline.** Tag-triggered publish to public npmjs, `NPM_TOKEN` secret, the repo
|
||||
made public with the LICENSE in place first (AGENTS.md §4).
|
||||
- [ ] **0 — Scaffold.** `package.json` with the §6 contract, `tsconfig.json`, `.npmrc`
|
||||
(`save-exact=true`), the Docker tooling setup, `renovate.json` (automerge-on-green, §9), and
|
||||
`.gitea/workflows/ci.yml` gating branches: `runs-on: docker-host`, actions pinned to semver
|
||||
tags.
|
||||
- [ ] **1 — The flavour spec.** Write the markdown flavour down as this repo's specification before
|
||||
implementing it: the exact directive grammar (attributes, escaping, nesting), each node's
|
||||
syntax from the inventory below, the opaque-carry spelling for unknown nodes, and the
|
||||
pipe-vs-directive table rule. Open detail to settle here: what CommonMark's raw-HTML
|
||||
constructs become in ADF, which has no raw-HTML node — likely the §3 element mapping, error
|
||||
otherwise. Start the corpus (§10) from this spec.
|
||||
- [ ] **2 — `adfToMarkdown`.**
|
||||
- [ ] **3 — `markdownToAdf`.** The CommonMark parser is the largest single component.
|
||||
- [ ] **4 — Round-trip property tests** over the corpus, both ways. Not a milestone that follows
|
||||
2 and 3 so much as the thing that proves them.
|
||||
- [ ] **5 — Release pipeline, ship `0.1.0`.** Publish-on-version-change on `main` (§9),
|
||||
`NPM_TOKEN` secret, the repo made public first (§6). `0.1.0` is the markdown round-trip:
|
||||
both markdown directions, the types, `isAdfDocument`.
|
||||
- [ ] **6 — The HTML dialect spec.** Element-by-element mapping, the `data-*` fidelity scheme, the
|
||||
opaque-carry form, and the documented foreign-element set `htmlToAdf` accepts.
|
||||
- [ ] **7 — HTML, ship `0.2.0`.** `adfToHtml`, `htmlToAdf`, and the composed `markdownToHtml` /
|
||||
`htmlToMarkdown`. CommonMark spec suite runs against `markdownToHtml` from here (§10).
|
||||
- [ ] **8 — CLI.** A later goal, deliberately unspecified until the library exists: shape it
|
||||
around the personas then.
|
||||
|
||||
## The ADF inventory to cover
|
||||
|
||||
@@ -43,7 +36,8 @@ reference](https://developer.atlassian.com/cloud/jira/platform/apis/document/str
|
||||
not the whole schema** — real payloads also carry `taskList`/`taskItem`, `decisionList`/`decisionItem`,
|
||||
`layoutSection`/`layoutColumn`, `blockCard`/`embedCard`, `extension`/`bodiedExtension`/`inlineExtension`
|
||||
and `placeholder`, none of which are documented there. Treat the documented set as the floor, not the
|
||||
ceiling, and see the unknown-node policy above.
|
||||
ceiling — the floor gets designed syntax, everything else rides the opaque carry (§3) until it too
|
||||
gets syntax.
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
|
||||
Reference in New Issue
Block a user