feat(mcpedia): Phase 4 — operability + correctness hardening
Reinterpreted from the plan's YAGNI 'Scale-out' (OpenSearch/object-storage /multi-tenant deferred at KB scale). Phase 4 = make the Phase 3 async + revision system correct, secure, observable, deployable. - T1 (correctness bug): restoreRevision now rebuilds semantic chunks via new @mcpedia/core reindexChunks(slug) so semantic/hybrid search stay consistent after a restore (previously document_chunks held the NEW body while documents.body held the restored OLD body -> stale search). - T2 (security): /hooks/* git-sync webhooks now require x-webhook-secret header matching WEBHOOK_SECRET (401 otherwise); API fails fast at startup if unset. Added WEBHOOK_SECRET to @mcpedia/config + .env.example; set real secret in .env. - T3 (UX): web doc page shows a History panel (revision no/reason/date/length) with per-revision Restore; app/api/revisions/restore/route.ts calls restoreRevision + revalidatePath (server-component only, no client JS). - T4: listRevisions gains offset paging; summary never includes body. - T5 (ops): deploy/mcpedia-api.service + deploy/mcpedia-worker.service systemd units (Restart=on-failure, EnvironmentFile=.env). Not auto-enabled on host. Verified against live imrnes Redis + Postgres: turbo typecheck+build green; restore-rebuilds-chunks (marker present -> gone after restore); webhook 401/200; web restore route redirects to doc + reverts body; revisions API returns summary (no body); systemd-analyze verify passes.
This commit is contained in:
asepharyana
@@ -0,0 +1,105 @@
|
||||
# MCPedia Phase 4 — Operability & Correctness Hardening
|
||||
|
||||
> Reinterpretation: the PHASES.md "Scale-out" items (OpenSearch, object storage,
|
||||
> multi-tenant, distributed workers) are YAGNI at KB scale (4 docs). Phase 4 =
|
||||
> make the Phase 3 async + revision machinery **correct, secure, observable, and
|
||||
> deployable** — not speculative infra. Each task below fixes a real gap found
|
||||
> by reading the code, not a hypothetical need.
|
||||
|
||||
## Tasks
|
||||
|
||||
### T1 — `restoreRevision` must rebuild semantic chunks (CORRECTNESS BUG)
|
||||
**Root cause:** `packages/core/src/revision.service.ts` `restoreRevision` writes
|
||||
the old body back into `documents` but never calls `indexChunks(slug, body)`.
|
||||
So after a restore, keyword search (FTS on `documents.body`) is correct but
|
||||
`document_chunks`/embeddings stay on the *new* body → semantic + hybrid search
|
||||
return stale/ghost chunks.
|
||||
|
||||
**Fix:**
|
||||
- Add `reindexChunks(slug)` to `@mcpedia/core` that re-runs `indexChunks(slug, body)`
|
||||
using the live `documents.body` (the new body after the update).
|
||||
- Call it inside `restoreRevision` after the `documents` update (wrap in try/catch
|
||||
like `indexContentFile` so embed failure doesn't abort the restore).
|
||||
- Add a unit-style assertion to the MCP smoke test or a small script: restore →
|
||||
`document_chunks` count matches re-chunked body.
|
||||
|
||||
**Files:** `packages/core/src/revision.service.ts`, `packages/core/src/index.ts`,
|
||||
`packages/core/src/document.service.ts` (export existing `indexChunks` if needed).
|
||||
|
||||
### T2 — Secure the git-sync webhook (SECURITY)
|
||||
**Root cause:** `apps/api/src/index.ts` `/hooks/reindex` and `/hooks/index` accept
|
||||
any request with no `WEBHOOK_SECRET` check — `.env.example` defines `WEBHOOK_SECRET`
|
||||
but the router never reads it.
|
||||
|
||||
**Fix:**
|
||||
- In `apps/api/src/index.ts`, compare `c.req.header("x-webhook-secret")` (or
|
||||
`?secret=`) against `WEBHOOK_SECRET` (from `@mcpedia/config`). If unset/mismatch →
|
||||
`401`. If `WEBHOOK_SECRET` env is empty, reject at startup with a clear log
|
||||
(fail-fast, don't run an open endpoint).
|
||||
- Add `WEBHOOK_SECRET` to `packages/config/src/index.ts` export.
|
||||
- Document the header in README + verify with curl (401 without secret, 200 with).
|
||||
|
||||
**Files:** `apps/api/src/index.ts`, `packages/config/src/index.ts`, README.
|
||||
|
||||
### T3 — Web UI revisions view (UX)
|
||||
**Root cause:** Web UI (server components) calls `@mcpedia/core` directly; there is
|
||||
no revisions surface even though `revisions`/`restoreRevision` tRPC + MCP resource
|
||||
exist.
|
||||
|
||||
**Fix (server-component only, no client JS):**
|
||||
- On the doc page (`apps/web/app/[section]/[...slug]/page.tsx`), fetch
|
||||
`listRevisions(fullSlug, 10)` and render a "History" panel: revision number,
|
||||
reason, createdAt, body length, and a `/api/revisions/restore` link/POST that
|
||||
calls the tRPC `restoreRevision` mutation via a server action or a form POST to
|
||||
a small route handler. Simplest: a `<form method="post" action="/api/revisions/restore">`
|
||||
with hidden `id` + a route handler in `apps/web` calling `restoreRevision`.
|
||||
Keep it read-mostly; restore is a deliberate action.
|
||||
- Add `apps/web/app/api/revisions/restore/route.ts` (POST) → `restoreRevision(id)`
|
||||
→ `revalidatePath` the doc.
|
||||
|
||||
**Files:** `apps/web/app/[section]/[...slug]/page.tsx`,
|
||||
`apps/web/app/api/revisions/restore/route.ts`.
|
||||
|
||||
### T4 — Paginate `listRevisions` / `revisions` API (PERF)
|
||||
**Root cause:** `revision.service.ts` `listRevisions` does `select length(body)`
|
||||
(fine) but the tRPC `revisions` and MCP resource return *full* revision rows
|
||||
including the body in some callers; list endpoints should never carry bodies.
|
||||
|
||||
**Fix:**
|
||||
- Ensure `listRevisions` summary excludes `body` (it already does — `bodyLength`
|
||||
only). Add `offset` param for paging. Confirm MCP resource uses the summary.
|
||||
- No behavior change for the doc page (uses summary).
|
||||
|
||||
**Files:** `packages/core/src/revision.service.ts` (add `offset`), router unchanged.
|
||||
|
||||
### T5 — Deployable as supervised services (OPS)
|
||||
**Root cause:** `apps/api` and `apps/worker` run only ad-hoc; the host already runs
|
||||
`zeavis` via Nix/systemd. Phase-4 operability = provide a systemd unit (or Nix
|
||||
service) so `mcpedia-api` + `mcpedia-worker` start on boot and restart on failure.
|
||||
|
||||
**Fix (Nix-first, per MEMORY):**
|
||||
- Write `mcpedia-api.service` + `mcpedia-worker.service` systemd unit files under
|
||||
`deploy/` (bun run api / bun run worker, `WorkingDirectory`, `Restart=on-failure`,
|
||||
`EnvironmentFile` pointing at `.env`, `After=network-online.target`).
|
||||
- README section "Run as a service" with `cp deploy/*.service /etc/systemd/system && systemctl daemon-reload && systemctl enable --now mcpedia-api mcpedia-worker`.
|
||||
- **Do NOT** `systemctl` on the host without user confirmation (changing live
|
||||
services). Provide the files + instructions only; user runs enable.
|
||||
|
||||
**Files:** `deploy/mcpedia-api.service`, `deploy/mcpedia-worker.service`, README.
|
||||
|
||||
## Verification (all real, against imrnes Redis + Postgres)
|
||||
1. `turbo run typecheck` + `turbo run build` green.
|
||||
2. T1: script — edit a doc, reindex (new revision + new chunks), restore rev #1,
|
||||
assert `document_chunks` count for that slug now matches re-chunk of rev #1 body
|
||||
and `semanticSearch` on a term unique to rev #1 returns it.
|
||||
3. T2: `curl -XPOST localhost:4020/hooks/reindex` → 401; with
|
||||
`-H "x-webhook-secret: $WEBHOOK_SECRET"` → 200 + jobId.
|
||||
4. T3: `next build` includes the History panel; restore form rebuilds chunks
|
||||
(verified via T1 path through the route handler).
|
||||
5. T4: `revisions` API returns summaries without body; offset paging works.
|
||||
6. T5: `systemd-analyze verify deploy/*.service` passes (off-host safe check);
|
||||
README documents enable steps.
|
||||
|
||||
## Out of scope (YAGNI, keep deferred per PHASES.md)
|
||||
OpenSearch/Elasticsearch, object storage, multi-tenant, distributed workers,
|
||||
pgvector migration. Revisit only when corpus > ~10k docs or query latency bites.
|
||||
Reference in New Issue
Block a user