CI: sync release images to Docker Hub on vX.Y.Z tag push

.gitea/workflows/registry-cleanup.yml

@@ -0,0 +1,22 @@
+name: Registry cleanup
+on:
+  schedule:
+    - cron: '43 3 * * *'
+  workflow_dispatch:
+
+jobs:
+  prune-stale-images:
+    runs-on: docker-host
+    steps:
+      - uses: actions/checkout@v4.2.2
+      - name: Delete hash images that are neither release-tagged nor a branch head
+        env:
+          REGISTRY_TOKEN: ${{ secrets.DOCKER_REGISTRY_TOKEN }}
+          REGISTRY_USER: ${{ vars.DOCKER_REGISTRY_USER }}
+          REPO_TOKEN: ${{ github.token }}
+          REPOSITORY: ${{ github.repository }}
+          SERVER_URL: ${{ github.server_url }}
+        run: |
+          docker run --rm -v "$PWD:/repo" -w /repo \
+            -e REGISTRY_TOKEN -e REGISTRY_USER -e REPO_TOKEN -e REPOSITORY -e SERVER_URL \
+            node:24.16.0-alpine3.24 node registry-cleanup/cleanup.ts

.gitea/workflows/release.yml

@@ -24,6 +24,25 @@ jobs:
             docker tag "$REPO:$COMMIT" "$REPO:$TAG"
             docker push "$REPO:$TAG"
           done
-      - name: Log out of the registry
+      - name: Sync the release tags to Docker Hub
+        env:
+          DOCKERHUB_REPO: docker.io/${{ github.repository }}
+          DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
+          DOCKERHUB_USER: ${{ vars.DOCKERHUB_USER }}
+          GIT_TAG: ${{ github.ref_name }}
+          REPO: gitea.larvit.se/${{ github.repository }}
+        run: |
+          COMMIT=$(git rev-parse 'HEAD^{commit}')
+          VERSION=${GIT_TAG#v}
+          [ -n "$DOCKERHUB_USER" ] && [ -n "$DOCKERHUB_TOKEN" ] \
+            || { echo "Set the DOCKERHUB_USER variable + DOCKERHUB_TOKEN secret (README -> CI/CD)"; exit 1; }
+          printf '%s' "$DOCKERHUB_TOKEN" | docker login docker.io -u "$DOCKERHUB_USER" --password-stdin
+          for TAG in "$VERSION" "${VERSION%.*}" "${VERSION%%.*}" latest; do
+            docker tag "$REPO:$COMMIT" "$DOCKERHUB_REPO:$TAG"
+            docker push "$DOCKERHUB_REPO:$TAG"
+          done
+      - name: Log out of the registries
         if: always()
-        run: docker logout gitea.larvit.se
+        run: |
+          docker logout gitea.larvit.se
+          docker logout docker.io

AGENTS.md

@@ -86,6 +86,11 @@ them. Revisit only if the stated reason stops holding.
   `tsconfig.include` and resolve the host surface via `#`-imports, so each example typechecks
   in place *and* copies across unchanged. Never commit real plugins/config into the root
   mount dirs (`plugins/`, `config/`) — they ship empty (`.gitkeep`, git-ignored otherwise).
+- **CI docker logins share the runner host's Docker config.** The act_runner is host-mode, so
+  `docker login`/`logout` in the workflows mutate one shared `~/.docker/config.json`:
+  concurrent jobs can race (one job's logout can 401 another's push — recover by re-running),
+  and tokens sit in that file between login and logout. Accepted for a single-maintainer
+  cadence; serialize with a workflow `concurrency` group if it ever bites.
 
 ## Docker only — no host tooling
 

README.md

@@ -1174,8 +1174,9 @@ Gitea Actions (`.gitea/workflows/`) runs the pipeline; the test job runs
 | Workflow | Trigger | Does |
 | --- | --- | --- |
 | `ci.yml` | push, any branch except `main` | the full gate (`bash ci.sh`), then build + push the app image |
-| `release.yml` | push of a `vX.Y.Z` tag | re-tag that commit's image as `X.Y.Z`, `X.Y`, `X`, `latest` |
+| `release.yml` | push of a `vX.Y.Z` tag | re-tag that commit's image as `X.Y.Z`, `X.Y`, `X`, `latest`; sync those tags to Docker Hub |
 | `mirror.yml` | push to `main` or any tag, or manual | force-push `main` + tags to the [GitHub mirror](https://github.com/larvit/plainpages) |
+| `registry-cleanup.yml` | nightly cron, or manual | delete registry images that are neither release-tagged nor a branch head |
 
 `main` is not re-tested on push — its commits are meant to arrive already green from a
 gated branch, so the status check to gate a merge on is `CI / full-gate (push)`.
@@ -1193,21 +1194,37 @@ and pushed by that exact commit's branch gate; nothing is rebuilt after merge (b
 promote by re-tagging). One-time setup: on an account with package write in the `larvit` org,
 create a Gitea access token with `read:package` + `write:package`, and store the account name
 as the Actions **variable** `DOCKER_REGISTRY_USER` and the token as the Actions **secret**
-`DOCKER_REGISTRY_TOKEN` (a `GITEA_` prefix is rejected — reserved, like `GITHUB_`). Because
+`DOCKER_REGISTRY_TOKEN` (a `GITEA_` prefix is rejected — reserved, like `GITHUB_`). The
+package is **org-owned** (the image path starts with `larvit/`), so it lists under
+`larvit/-/packages`, not the repo — link it once to the repo's Packages tab:
+`POST /api/v1/packages/larvit/container/plainpages/-/link/plainpages`. Because
 this step runs
 inside the required gate, a missing/expired token (or registry outage) fails every branch's
 gate and blocks **all** merges until restored — set the secrets before this lands, and use a
-non-expiring token or track its expiry. Retention: hash tags
+non-expiring token or track its expiry. Retention: hash tags accumulate one image per gated
-accumulate one image per gated push, so an org-level package **cleanup rule**
+push, so the nightly `registry-cleanup.yml` prunes them precisely
-(org Settings → Packages, applied by Gitea's daily cleanup cron) prunes them — type
+([`registry-cleanup/cleanup.ts`](registry-cleanup/cleanup.ts), run in a `node:24` container):
-`container`, remove versions matching `^[0-9a-f]{40}$` older than 30 days, keep the 10 most
+a hash tag survives only while its commit is a **branch head** or carries a **`vX.Y.Z`
-recent and anything matching `^\d+(\.\d+){0,2}$|^latest$` (release tags are never pruned).
+release tag**; deleted alongside are the untagged `sha256:…` child manifests (arch image +
+provenance) that no surviving tag references. Named tags (`1.2.3`, `latest`, …) are never
+touched. It reuses `DOCKER_REGISTRY_USER`/`DOCKER_REGISTRY_TOKEN` — no extra setup. Don't
+add a pattern-based org cleanup rule for this package (and remove it if one exists): its
+age/count heuristics can't see branch heads or release tags and would delete images the
+workflow protects.
 
 **Releases** — pushing a semver git tag (`git tag v1.2.3 && git push origin v1.2.3`) runs
 `release.yml`, which pulls that commit's hash image from the registry and re-tags it as
 `1.2.3`, `1.2`, `1`, and `latest` — nothing is rebuilt, the released image is byte-identical
 to the gated one. It fails loud if no hash image exists: release tags must point at a commit
-that went through the gate (in practice, any `main` commit).
+that went through the gate (in practice, any `main` commit). The same four tags are then
+synced to [Docker Hub](https://hub.docker.com/r/larvit/plainpages) (`larvit/plainpages`) —
+releases only, no hash tags. One-time setup: create the public `larvit/plainpages`
+repository on Docker Hub, generate a read/write access token **scoped to that repository**
+(an organization access token, or a token on a dedicated single-purpose account — an
+account-wide PAT can push to every repo under the account), and store the account name as
+the Actions **variable** `DOCKERHUB_USER` and the token as the Actions **secret**
+`DOCKERHUB_TOKEN`. Until they exist, a release run fails at the Docker Hub step — after the
+Gitea re-tag has succeeded — so set them, then re-run the workflow.
 
 **GitHub mirror** — [github.com/larvit/plainpages](https://github.com/larvit/plainpages) is a
 read-only mirror; after every merge, `mirror.yml` force-pushes `main` and all tags there,

package.json

@@ -15,7 +15,7 @@
     "dev": "node --watch src/server.ts",
     "gen-jwks": "node src/auth/gen-jwks.ts",
     "typecheck": "tsc --noEmit",
-    "test": "node --test \"src/**/*.test.ts\" \"plugins/**/*.test.ts\" \"examples/**/*.test.ts\""
+    "test": "node --test \"src/**/*.test.ts\" \"plugins/**/*.test.ts\" \"examples/**/*.test.ts\" \"registry-cleanup/**/*.test.ts\""
   },
   "dependencies": {
     "@larvit/log": "2.3.0",

registry-cleanup/cleanup.ts

@@ -0,0 +1,98 @@
+// Prunes the app's container package in the Gitea registry: a commit-hash tag survives only
+// while its commit is a branch head or carries a vX.Y.Z release tag. Untagged sha256:* package
+// versions are CHILD manifests (arch image + provenance) of the tagged OCI indexes — deleting
+// one that a surviving tag still references breaks that image, so only the ones no kept tag
+// references are removed. Named tags (1.2.3, latest, …) are never touched.
+import { planHashTagDeletions, selectOrphanedManifests } from "./select-versions.ts";
+
+function env(name: string): string {
+  const value = process.env[name];
+  if (value === undefined || value === "") throw new Error(`Missing env var ${name}`);
+  return value;
+}
+
+const registryToken = env("REGISTRY_TOKEN");
+const registryUser = env("REGISTRY_USER");
+const repoToken = env("REPO_TOKEN");
+const repository = env("REPOSITORY");
+const serverUrl = env("SERVER_URL").replace(/\/+$/, "");
+
+const [owner, name] = repository.split("/");
+if (owner === undefined || name === undefined || owner === "" || name === "") {
+  throw new Error(`REPOSITORY must be owner/name, got "${repository}"`);
+}
+
+async function fetchOk(url: string, init: RequestInit): Promise<Response> {
+  const res = await fetch(url, init);
+  if (!res.ok) throw new Error(`${init.method ?? "GET"} ${url} -> ${res.status}: ${await res.text()}`);
+  return res;
+}
+
+async function apiGetAllPages<T>(path: string, token: string): Promise<T[]> {
+  const all: T[] = [];
+  const limit = 50;
+  for (let page = 1; ; page++) {
+    const sep = path.includes("?") ? "&" : "?";
+    const res = await fetchOk(`${serverUrl}/api/v1${path}${sep}limit=${limit}&page=${page}`, {
+      headers: { authorization: `token ${token}` },
+    });
+    const batch = (await res.json()) as T[];
+    // Stop on an empty page, not a short one — a lowered server page-size cap would otherwise
+    // silently truncate the list, and a truncated branch/tag list deletes protected images.
+    if (batch.length === 0) return all;
+    all.push(...batch);
+  }
+}
+
+async function childDigests(tag: string): Promise<string[]> {
+  const res = await fetchOk(`${serverUrl}/v2/${owner}/${name}/manifests/${tag}`, {
+    headers: {
+      accept: [
+        "application/vnd.oci.image.index.v1+json",
+        "application/vnd.docker.distribution.manifest.list.v2+json",
+        "application/vnd.oci.image.manifest.v1+json",
+        "application/vnd.docker.distribution.manifest.v2+json",
+      ].join(", "),
+      authorization: `Basic ${Buffer.from(`${registryUser}:${registryToken}`).toString("base64")}`,
+    },
+  });
+  const manifest = (await res.json()) as { manifests?: { digest: string }[] };
+  return (manifest.manifests ?? []).map((m) => m.digest);
+}
+
+async function deleteVersion(version: string): Promise<void> {
+  const url = `${serverUrl}/api/v1/packages/${owner}/container/${name}/${encodeURIComponent(version)}`;
+  const res = await fetch(url, { headers: { authorization: `token ${registryToken}` }, method: "DELETE" });
+  if (res.status === 404) {
+    console.log(`already gone: ${version}`);
+    return;
+  }
+  if (!res.ok) throw new Error(`DELETE ${url} -> ${res.status}: ${await res.text()}`);
+  console.log(`deleted: ${version}`);
+}
+
+const branches = await apiGetAllPages<{ commit: { id: string } }>(`/repos/${owner}/${name}/branches`, repoToken);
+if (branches.length === 0) throw new Error("no branches returned — refusing to prune with an empty keep-set");
+const tags = await apiGetAllPages<{ commit: { sha: string }; name: string }>(`/repos/${owner}/${name}/tags`, repoToken);
+const packages = await apiGetAllPages<{ name: string; version: string }>(
+  `/packages/${owner}?type=container&q=${encodeURIComponent(name)}`,
+  registryToken,
+);
+
+const versions = packages.filter((p) => p.name === name).map((p) => p.version);
+const plan = planHashTagDeletions({
+  branchHeadShas: branches.map((b) => b.commit.id),
+  releaseTagShas: tags.filter((t) => /^v\d+\.\d+\.\d+$/.test(t.name)).map((t) => t.commit.sha),
+  versions,
+});
+
+const referenced = new Set<string>();
+for (const tag of plan.keptTags) {
+  for (const digest of await childDigests(tag)) referenced.add(digest);
+}
+const orphans = selectOrphanedManifests({ referencedDigests: [...referenced], versions });
+
+for (const version of [...plan.deletions, ...orphans]) await deleteVersion(version);
+console.log(
+  `kept ${plan.keptTags.length} tags; deleted ${plan.deletions.length} hash tags and ${orphans.length} orphaned manifests`,
+);

registry-cleanup/select-versions.test.ts

@@ -0,0 +1,48 @@
+import assert from "node:assert/strict";
+import { test } from "node:test";
+import { planHashTagDeletions, selectOrphanedManifests } from "./select-versions.ts";
+
+const headSha = "0582809000000000000000000000000000000001";
+const releaseSha = "50006dd000000000000000000000000000000002";
+const staleSha = "c8981c1000000000000000000000000000000003";
+
+test("planHashTagDeletions deletes hash tags that are neither a branch head nor release-tagged", () => {
+  const plan = planHashTagDeletions({
+    branchHeadShas: [headSha],
+    releaseTagShas: [releaseSha],
+    versions: [headSha, releaseSha, staleSha],
+  });
+  assert.deepEqual(plan.deletions, [staleSha]);
+  assert.deepEqual(plan.keptTags, [headSha, releaseSha]);
+});
+
+test("planHashTagDeletions keeps named tags and excludes sha256 manifest versions from keptTags", () => {
+  const plan = planHashTagDeletions({
+    branchHeadShas: [],
+    releaseTagShas: [],
+    versions: ["0.0.1", "0", "latest", "some-manual-tag", `sha256:${"a".repeat(64)}`, staleSha],
+  });
+  assert.deepEqual(plan.deletions, [staleSha]);
+  assert.deepEqual(plan.keptTags, ["0.0.1", "0", "latest", "some-manual-tag"]);
+});
+
+test("planHashTagDeletions with no versions plans nothing", () => {
+  const plan = planHashTagDeletions({ branchHeadShas: [headSha], releaseTagShas: [], versions: [] });
+  assert.deepEqual(plan.deletions, []);
+  assert.deepEqual(plan.keptTags, []);
+});
+
+test("selectOrphanedManifests picks only sha256 versions unreferenced by kept tags", () => {
+  const referenced = `sha256:${"b".repeat(64)}`;
+  const orphaned = `sha256:${"c".repeat(64)}`;
+  const orphans = selectOrphanedManifests({
+    referencedDigests: [referenced],
+    versions: ["0.0.1", "latest", headSha, referenced, orphaned],
+  });
+  assert.deepEqual(orphans, [orphaned]);
+});
+
+test("selectOrphanedManifests with nothing referenced orphans every sha256 version", () => {
+  const manifest = `sha256:${"d".repeat(64)}`;
+  assert.deepEqual(selectOrphanedManifests({ referencedDigests: [], versions: [manifest, "latest"] }), [manifest]);
+});

registry-cleanup/select-versions.ts

@@ -0,0 +1,31 @@
+const hashTag = /^[0-9a-f]{40}$/;
+const manifestVersion = /^sha256:[0-9a-f]{64}$/;
+
+export type HashTagPlan = {
+  deletions: string[];
+  keptTags: string[];
+};
+
+export function planHashTagDeletions(input: {
+  branchHeadShas: string[];
+  releaseTagShas: string[];
+  versions: string[];
+}): HashTagPlan {
+  const keep = new Set([...input.branchHeadShas, ...input.releaseTagShas]);
+  const deletions: string[] = [];
+  const keptTags: string[] = [];
+  for (const version of input.versions) {
+    if (manifestVersion.test(version)) continue;
+    if (hashTag.test(version) && !keep.has(version)) deletions.push(version);
+    else keptTags.push(version);
+  }
+  return { deletions, keptTags };
+}
+
+export function selectOrphanedManifests(input: {
+  referencedDigests: string[];
+  versions: string[];
+}): string[] {
+  const referenced = new Set(input.referencedDigests);
+  return input.versions.filter((v) => manifestVersion.test(v) && !referenced.has(v));
+}

todo.md

@@ -8,7 +8,7 @@
 - [x] CI/CD - Sync up to github after every successful merge to main, URL: git@github.com:larvit/plainpages.git - also note the true home top of the README. Force push to github, it should only ever be a mirror of the gitea.larvit.se repository. (`.gitea/workflows/mirror.yml` force-pushes main + tags over HTTPS with a dedicated account's PAT in the `MIRROR_GITHUB_TOKEN` secret; setup documented in README → CI/CD.)
 - [x] CI/CD - Build docker images as part of the requirements to be able to merge to main. Push them with the git commit hash as docker tag. Push to container registry at Gitea. (`ci.yml` builds + pushes `gitea.larvit.se/larvit/plainpages:<commit hash>` after a green gate — with ff-only merges that is the main commit's image; auth via the `DOCKER_REGISTRY_USER` variable + `DOCKER_REGISTRY_TOKEN` secret, retention via an org cleanup rule; documented in README → CI/CD.)
 - [x] CI/CD - Re-tag docker images from git hash to semver when a semver git tag is pushed. (`release.yml` on a `vX.Y.Z` tag pulls the commit-hash image and re-tags it `X.Y.Z`/`X.Y`/`X`/`latest`, failing loud if the gated image is missing; tag pushes also trigger the GitHub mirror; documented in README → CI/CD.)
-- [ ] CI/CD - Sync docker images to docker hub after each re-tag to git tags.
+- [x] CI/CD - Sync docker images to docker hub after each re-tag to git tags. (`release.yml` pushes the same `X.Y.Z`/`X.Y`/`X`/`latest` tags to `docker.io/larvit/plainpages` after the Gitea re-tag — releases only, no hash tags; auth via the `DOCKERHUB_USER` variable + `DOCKERHUB_TOKEN` secret; documented in README → CI/CD.)
 - [ ] CI/CD - Setup renovate bot.
 - [ ] Add an e2e test for the admin plugin's OAuth2-clients (Hydra) screen. The full-flow e2e suite runs without Hydra (compose.full.yml), so /admin/clients register/detail/delete is only unit-covered (src/http/app.test.ts); wire Hydra into an e2e stack and drive the screen in the browser.
 - [ ] Build and publish docker image as CI/CD.

tsconfig.json

@@ -24,5 +24,5 @@
     "forceConsistentCasingInFileNames": true,
     "skipLibCheck": true
   },
-  "include": ["config", "examples/config", "examples/plugins", "plugins", "src"]
+  "include": ["config", "examples/config", "examples/plugins", "plugins", "registry-cleanup", "src"]
 }