Discover plugins at boot (todo §2); scan plugins/, import + validate each plugin.ts default export, fail loud on bad plugin/conflict

README.md

@@ -249,9 +249,10 @@ but a bind mount matches the edit-and-reload loop. For a **baked** production im
 just keep the plugin in the build context and it's `COPY`'d in at build time — pinned
 and reproducible; mount a volume only to add plugins to an already-built image.
 
-> _(Planned, §2.)_ Discovery — scanning `plugins/`, importing each `plugin.ts`, and
+> Discovery — scanning `plugins/`, importing each `plugin.ts` default export, and validating
-> validating it — is the next plugin-host work. The mount mechanics above are how the
+> it (id, `apiVersion`, conflicts) — runs at boot (`src/discovery.ts`); a bad plugin stops
-> files get into the container regardless of how discovery lands.
+> startup with a precise message. _(Planned, §2:)_ the router that mounts the discovered routes
+> is next. The mount mechanics above are how the files get into the container either way.
 
 ## The menu system _(planned)_
 
@@ -429,7 +430,8 @@ src/icons.ts         Used-icon registry + sprite builder from lucide-static (reg
 src/list-query.ts    parseListQuery(): read a list URL → { q, filters, sort, page, pageSize }
 src/nav.ts           composeNav(): merge plugin nav fragments + central override, role-filter → nav-tree model
 src/paginate.ts      paginate(total,page,pageSize): page model (counts, row window, ellipsis sequence) for pagination.ejs
-src/plugin.ts        Plugin contract: manifest types, definePlugin(), version + conflict rules (discovery/router planned, §2)
+src/plugin.ts        Plugin contract: manifest types, definePlugin(), version + conflict rules (router planned, §2)
+src/discovery.ts     discoverPlugins(): scan plugins/, import + validate each plugin.ts default export, fail loud at boot (§2)
 views/               Core EJS templates (index = the app-shell People dashboard, 403/404/500, partials/ incl. app shell, nav tree, filter bar, data table, pagination, form field, auth card, menu/popover, theme switch, icon sprite)
 public/              Static assets under /public/ (css/styles.css + auth.css, favicon, robots.txt)
 config/menu.ts       Central menu override + branding                      (planned)

src/discovery.test.ts

@@ -0,0 +1,64 @@
+import assert from "node:assert/strict";
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { dirname, join } from "node:path";
+import { test, type TestContext } from "node:test";
+import { discoverPlugins } from "./discovery.ts";
+
+// Write a throwaway plugins/ tree of `relpath → source` and clean it up after the test. Fixtures
+// default-export plain objects — definePlugin is identity, so a literal is an equivalent manifest.
+function scaffold(t: TestContext, files: Record<string, string>): string {
+  const dir = mkdtempSync(join(tmpdir(), "pp-plugins-"));
+  t.after(() => rmSync(dir, { force: true, recursive: true }));
+  for (const [rel, content] of Object.entries(files)) {
+    const full = join(dir, rel);
+    mkdirSync(dirname(full), { recursive: true });
+    writeFileSync(full, content);
+  }
+  return dir;
+}
+
+const full = (id: string): string =>
+  `export default { apiVersion: "1.0.0", nav: [{ id: "${id}:root", label: "${id}" }], ` +
+  `routes: [{ method: "GET", path: "/", handler: () => ({ html: "${id}" }) }] };`;
+
+test("a missing plugins/ dir means zero plugins, not an error (clean clone)", async () => {
+  assert.deepEqual(await discoverPlugins({ dir: join(tmpdir(), "pp-does-not-exist-xyz") }), []);
+});
+
+test("discovers each folder's manifest, sorted, id derived from the folder name", async (t) => {
+  const dir = scaffold(t, { "beta/plugin.ts": full("beta"), "alpha/plugin.ts": full("alpha") });
+  const plugins = await discoverPlugins({ dir });
+
+  assert.deepEqual(plugins.map((p) => p.id), ["alpha", "beta"]); // deterministic order
+  assert.equal(plugins[0]?.apiVersion, "1.0.0");
+  assert.equal(plugins[0]?.nav?.[0]?.label, "alpha");
+  assert.equal(typeof plugins[0]?.routes?.[0]?.handler, "function"); // handlers survive import
+});
+
+// Every per-plugin problem and every error-level conflict aborts boot with a message naming it.
+const badCases: Array<{ name: string; files: Record<string, string>; match: RegExp }> = [
+  { name: "invalid folder name", files: { "Bad_Name/plugin.ts": full("x") }, match: /Bad_Name/ },
+  { name: "missing plugin.ts", files: { "broken/readme.txt": "x" }, match: /broken.*plugin\.ts/s },
+  { name: "no default export", files: { "named-only/plugin.ts": "export const x = 1;" }, match: /named-only.*default/s },
+  { name: "import throws", files: { "explodes/plugin.ts": "throw new Error('boom');" }, match: /explodes.*boom/s },
+  { name: "incompatible apiVersion", files: { "future/plugin.ts": `export default { apiVersion: "2.0.0" };` }, match: /future.*apiVersion/s },
+  { name: "non-array routes", files: { "weird/plugin.ts": `export default { apiVersion: "1.0.0", routes: "nope" };` }, match: /weird.*routes.*array/s },
+  { name: "duplicate nav id across plugins", files: { "a/plugin.ts": full("a").replace("a:root", "dup"), "b/plugin.ts": full("b").replace("b:root", "dup") }, match: /nav id "dup"/ },
+];
+
+for (const c of badCases) {
+  test(`fails loud: ${c.name}`, async (t) => {
+    await assert.rejects(discoverPlugins({ dir: scaffold(t, c.files) }), c.match);
+  });
+}
+
+test("a shared permission token only warns — both plugins still load", async (t) => {
+  const perm = `export default { apiVersion: "1.0.0", permissions: [{ token: "shared:read" }] };`;
+  const dir = scaffold(t, { "x/plugin.ts": perm, "y/plugin.ts": perm });
+  const warnings: string[] = [];
+  const plugins = await discoverPlugins({ dir, logger: { warn: (m) => warnings.push(String(m)) } });
+
+  assert.equal(plugins.length, 2);
+  assert.ok(warnings.some((w) => /shared:read/.test(w)), "expected a permission-conflict warning");
+});

src/discovery.ts

@@ -0,0 +1,96 @@
+// Plugin discovery (todo §2): scan plugins/, import each folder's plugin.ts default export,
+// validate it, assemble the loaded Plugin[]. The imperative shell over the pure rules in
+// plugin.ts (isValidPluginId, checkApiVersion, findConflicts). Fails loud: every per-plugin
+// problem and every error-level conflict is collected and thrown as one boot-stopping Error;
+// warn-level diagnostics (older-minor apiVersion, shared permission token) are logged, load
+// continues. The folder name is the id; mount/router wiring is the next §2 item.
+
+import { existsSync, readdirSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { fileURLToPath, pathToFileURL } from "node:url";
+import { checkApiVersion, findConflicts, isValidPluginId, type Plugin, type PluginManifest } from "./plugin.ts";
+
+const rootDir = join(dirname(fileURLToPath(import.meta.url)), "..");
+
+// Default scan root — <repo>/plugins, i.e. the /app/plugins the container mounts (README).
+export const PLUGINS_DIR = join(rootDir, "plugins");
+
+export interface DiscoverOptions {
+  dir?: string;
+  logger?: Pick<Console, "warn">; // warn-level diagnostics; defaults to console
+}
+
+export async function discoverPlugins(options: DiscoverOptions = {}): Promise<Plugin[]> {
+  const dir = options.dir ?? PLUGINS_DIR;
+  const logger = options.logger ?? console;
+  if (!existsSync(dir)) return []; // a clean clone has no plugins/ yet — zero plugins is valid
+
+  const errors: string[] = [];
+  const plugins: Plugin[] = [];
+
+  for (const id of pluginFolders(dir)) {
+    const fail = (msg: string): number => errors.push(`plugins/${id}: ${msg}`);
+
+    if (!isValidPluginId(id)) {
+      errors.push(`"${id}" is not a valid plugin folder name (lowercase a–z, digits, dashes)`);
+      continue;
+    }
+    const file = join(dir, id, "plugin.ts");
+    if (!existsSync(file)) { fail("no plugin.ts found"); continue; }
+
+    let mod: { default?: unknown };
+    try {
+      mod = (await import(pathToFileURL(file).href)) as { default?: unknown };
+    } catch (err) {
+      fail(`failed to import plugin.ts — ${messageOf(err)}`);
+      continue;
+    }
+
+    const manifest = asManifest(mod.default);
+    if (!manifest) { fail("plugin.ts must default-export a manifest object"); continue; }
+
+    const version = checkApiVersion(manifest.apiVersion);
+    if (version.level === "refuse") { fail(version.message); continue; }
+    if (version.level === "warn") logger.warn(`[plugins] ${id}: ${version.message}`);
+
+    const shape = shapeError(manifest);
+    if (shape) { fail(shape); continue; }
+
+    plugins.push({ ...manifest, id }); // identity is the folder, not the manifest
+  }
+
+  for (const conflict of findConflicts(plugins)) {
+    if (conflict.level === "error") errors.push(conflict.message);
+    else logger.warn(`[plugins] ${conflict.message}`);
+  }
+
+  if (errors.length) {
+    throw new Error(`Plugin discovery failed:\n${errors.map((e) => `  - ${e}`).join("\n")}`);
+  }
+  return plugins;
+}
+
+// Subfolders of plugins/, sorted for deterministic load order + stable conflict messages. Hidden
+// entries (.git, .DS_Store, …) and non-directories are skipped — only folders are plugins.
+function pluginFolders(dir: string): string[] {
+  return readdirSync(dir, { withFileTypes: true })
+    .filter((e) => e.isDirectory() && !e.name.startsWith("."))
+    .map((e) => e.name)
+    .sort();
+}
+
+function asManifest(value: unknown): PluginManifest | null {
+  return typeof value === "object" && value !== null && !Array.isArray(value) ? (value as PluginManifest) : null;
+}
+
+// The collection fields feed findConflicts, which iterates them — a non-array crashes it opaquely.
+function shapeError(manifest: PluginManifest): string | null {
+  for (const field of ["nav", "permissions", "routes"] as const) {
+    if (manifest[field] !== undefined && !Array.isArray(manifest[field])) return `"${field}" must be an array`;
+  }
+  return null;
+}
+
+function messageOf(err: unknown): string {
+  return err instanceof Error ? err.message : String(err);
+}

src/server.ts

@@ -1,8 +1,12 @@
 import { createApp } from "./app.ts";
 import { loadConfig } from "./config.ts";
+import { discoverPlugins } from "./discovery.ts";
 
 const config = loadConfig(); // validates the env (incl. enforced secrets) — fails loud at boot
 
+const plugins = await discoverPlugins(); // scans plugins/, validates — fails loud on a bad plugin (router wiring is next §2)
+console.log(`Discovered ${plugins.length} plugin(s)${plugins.length ? `: ${plugins.map((p) => p.id).join(", ")}` : ""}`);
+
 const server = createApp({ cache: config.cacheTemplates }).listen(config.port, () => {
   console.log(`Listening on http://localhost:${config.port}`);
 });

todo.md

@@ -46,7 +46,7 @@ everything via Docker.
 
 ## 2. Plugin host
 - [x] **Specify the plugin contract** (big job, do first — it's the product's main API surface). Write it down as the authoritative reference: the full manifest shape; the `RequestContext` handed to handlers and what's guaranteed stable; **contract versioning** (a `apiVersion`/`engines`-style field so a plugin declares the host it targets, and the host refuses or warns on mismatch); **conflict rules** (two plugins claiming the same `basePath`, nav slot, or `permission` name → defined, loud resolution, not last-write-wins); the **local dev/test story** (how an author runs + tests one plugin in isolation against the host). Audience is experienced devs: optimise for a powerful, predictable, clearly-documented API. Crash-isolation (a bad plugin can't take down the host) is a *nice-to-have*, not a blocker — fail loud at boot/discovery over sandboxing at runtime. It is a target that plugins should be able to overload as much as possible. Hooks on actions in the system is not bad either, if it is possible. → `src/plugin.ts` is the typed, machine-readable contract (single source of truth: authored `PluginManifest` + folder-derived `Plugin`, `Route`/`RouteResult`/`RouteHandler`, `PermissionDecl`, `PluginHooks`, `definePlugin()`, `HOST_API_VERSION`) plus the pure rules the §2 host enforces — `isValidPluginId` (URL-safe folder name: lowercase/digits/dashes), `checkApiVersion` (semver via `parseSemver`/official regex, no dep: same major+minor→ok, older minor→warn, newer minor/major-mismatch/malformed→refuse) and `findConflicts` (id/route = error, duplicate nav-id = error, shared permission token = warn; never last-write-wins). Identity is the folder: id = folder name, mount = `/<id>` — neither is in the manifest, so mount-path uniqueness is structural (no basePath rule). `apiVersion` is a literal a plugin pins (never imports `HOST_API_VERSION`). nav `icon` = Lucide sprite id. `docs/plugin-contract.md` is the prose reference (anatomy/identity, manifest fields, handler/RouteResult, `RequestContext` stability guarantee, nav/permission namespacing, versioning, conflicts, hooks, dev/test story). README links it. Tests-first (`plugin.test.ts`); typecheck + 82 units green. Discovery/router/view-resolver/static stay as the next §2 items that wire this to FS+HTTP.
-- [ ] Discovery: scan `plugins/`, import each `plugin.ts` default export, validate.
+- [x] Discovery: scan `plugins/`, import each `plugin.ts` default export, validate. → `src/discovery.ts` (`discoverPlugins`): the imperative shell over plugin.ts's pure rules. Scans `plugins/` (sorted, skips dotfiles/non-dirs; missing dir ⇒ `[]` for a clean clone), derives `id` from the folder, dynamically imports each `plugin.ts` default export and validates it — `isValidPluginId`, default-export-is-a-manifest, `checkApiVersion`, array-shape of nav/routes/permissions, then `findConflicts` across the set. Fails loud: every per-plugin problem + every error-level conflict is collected and thrown as one boot-stopping Error naming the plugin(s); warns (older-minor apiVersion, shared permission token) log and load continues. Wired into `server.ts` boot (logs the loaded ids). `discovery.test.ts` covers empty/happy/each failure mode + the warn path (temp-dir fixtures). Router/view-resolver/static are the next §2 items.
 - [ ] Router: match method+path under `basePath`, resolve path params, run permission gate, call handler with context.
 - [ ] Per-plugin view resolver (`plugins/<id>/views/*.ejs`).
 - [ ] Per-plugin static serving: `plugins/<id>/public/` → `/public/<id>/`.