asepharyana
9397303f01
- @mcpedia/embeddings: OpenRouter provider (9router /v1, encoding_format float), chunkText + embedChunks; EMBED_DIM=2048 - document_chunks table (real[] embedding) — pgvector NOT available on shared imrnes Postgres, so cosine is computed in-app (KB-scale fine); pgvector deferred - indexer: chunk + embed + upsert per document - @mcpedia/search: semanticSearch (cosine) + hybridSearch (FTS+cosine RRF) - apps/api: Hono + tRPC v11 (6 procedures), serve on :4020 - MCP: semantic_search + hybrid_search tools (6 total) - web: keyword/hybrid toggle; .env.example + README + PHASES updated
3.0 KiB
3.0 KiB
MCPedia — Phase Status
Legend: ✅ built · 🟡 partial · ⬜ deferred
Phase 1 — MVP (✅ DONE)
| Capability | Status | Notes |
|---|---|---|
| Monorepo (bun + Turbo) | ✅ | apps/{web,mcp}, packages/{types,config,db,parser,search,core}, scripts |
| Content as Markdown | ✅ | content/{docs,writeups,research,notes}/, Git-tracked |
| Frontmatter parsing | ✅ | @mcpedia/parser (gray-matter) |
| Postgres metadata | ✅ | @mcpedia/db Drizzle, documents table |
| Postgres FTS | ✅ | weighted tsvector (title A / body B), GIN index, ts_rank+ts_headline |
| Core services | ✅ | Document / Content / Search — single business-logic layer |
| Indexer | ✅ | scripts/indexer.ts walks content/ → upserts |
| Web UI (Next 16) | ✅ | home (list), doc view (SSG), search (dynamic). react-markdown render |
| MCP server (stdio) | ✅ | 4 tools; in-memory smoke test passing |
| Hybrid/semantic search | ⬜ | Phase 2 |
Phase 2 — Semantic + API
packages/embeddings—EmbeddingProviderinterface + OpenRouter provider (via 9router/v1,encoding_format:"float");chunkText+embedChunksbatcher.EMBED_DIM=2048discovered live.- Schema
document_chunks(id, document_id→documents.id cascade, slug, chunk_index, content,embedding real[]). Stored asreal[]because pgvector is not installed on the shared imrnes Postgres (installing needs host-level apt — deferred). Cosine computed in-app; instant for a KB-sized corpus. scripts/indexer.ts— chunks + embeds + upserts (per-doc replace).@mcpedia/search—semanticSearch(cosine) +hybridSearch(FTS + cosine, RRF fusion).keywordSearchunchanged.apps/api— Hono + tRPC v11 (@trpc/serverfetch adapter,@hono/node-serveron :4020):search,semanticSearch,hybridSearch,getDocument,listDocuments,related.- MCP server — added
semantic_search+hybrid_searchtools (6 total). - Web search — keyword/hybrid toggle (
?mode=hybrid), hybrid reaches semantically-related docs keyword misses.
Phase 3 — Async + Scale
- Redis + BullMQ background indexing / embedding workers
- Git synchronization hook (auto-reindex on push)
- Document revision system (
document_revisions) - MCP Resources (
mcpedia://docs/...) in addition to tools
Phase 4 — Scale-out (only if needed)
- Dedicated search engine (OpenSearch/Elasticsearch) — YAGNI until FTS is insufficient
- Object storage for assets
- Advanced ranking, distributed workers, observability, multi-tenant
Decisions locked (from initial planning)
- Tooling: bun workspaces + Turborepo (repo already used bun; pnpm rejected to minimize churn).
- DB: imrnes Postgres
100.121.180.82:6432/mcpediafor both dev and deploy; driverprepare:false(PgBouncer). Docker Compose reserved for future prod. - Phase 1 scope: Core + Web + MCP only. tRPC/Hono API, pgvector, auth, BullMQ deferred (YAGNI).