Compare commits
3
Commits
0eb089d9dc
...
e0a383c5b3
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e0a383c5b3 | ||
|
|
43f2f8449d | ||
|
|
b3418ad799 |
No files matched your search
@@ -0,0 +1,100 @@
|
||||
# GMW — Agent Guide
|
||||
|
||||
GMW (Guild Moderation Watcher) is a Discord bot + web dashboard for AI-powered moderation. A monorepo with three services: a selfbot gateway that captures messages/voice and runs LLM moderation, an Express/oRPC backend that serves the dashboard API, and a Next.js 16 SSR frontend. They communicate via Redis pub/sub (gateway→backend) and WebSocket (backend→browser).
|
||||
|
||||
## Quick reference
|
||||
|
||||
```bash
|
||||
# Per-service — cd into the service first
|
||||
pnpm install # install deps (pnpm 11, not npm or bun)
|
||||
pnpm typecheck # tsc --noEmit
|
||||
pnpm lint # biome check
|
||||
pnpm format # biome format --write
|
||||
pnpm build # gateway/backend: tsc + fix-imports.mjs; frontend: next build
|
||||
pnpm test # vitest run (gateway & backend only — frontend has no tests)
|
||||
```
|
||||
|
||||
No monorepo-level scripts exist. Run each command from inside the service directory.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
services/
|
||||
├── discord-gateway/ Event-driven selfbot. No HTTP (except :4016 metrics).
|
||||
│ ├── src/
|
||||
│ │ ├── app/ Bootstrap, shutdown, retention
|
||||
│ │ ├── modules/ Feature modules — each self-contained
|
||||
│ │ └── shared/ Config, DB (Drizzle), logger, errors, utils
|
||||
│ ├── tests/ Vitest tests (colocated, not inside src/)
|
||||
│ ├── drizzle/migrations/ DB migrations
|
||||
│ └── scripts/fix-imports.mjs Post-build: rewrites @/ aliases → relative .js
|
||||
│
|
||||
├── backend/ Express HTTP + WebSocket + oRPC server (:4001).
|
||||
│ ├── src/
|
||||
│ │ ├── modules/ Feature modules (schema→repo→service→controller→routes)
|
||||
│ │ ├── http/ Express app setup
|
||||
│ │ ├── ws/ WebSocket server + Redis bridge
|
||||
│ │ ├── orpc/ oRPC router definition
|
||||
│ │ └── shared/ Config, DB, errors, Redis, logger
|
||||
│ └── tests/ Vitest tests
|
||||
│
|
||||
└── frontend/ Next.js 16 App Router, React 19, Tailwind v4 (:4017).
|
||||
├── src/
|
||||
│ ├── app/ Pages — route groups under (dashboard)/
|
||||
│ ├── components/ UI components (primitives, shell, charts, etc.)
|
||||
│ ├── hooks/ React hooks
|
||||
│ └── lib/ API clients, types, utils, WebSocket, audio
|
||||
└── pnpm-workspace.yaml Build-script approvals (sharp only)
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
### Package manager & runtime
|
||||
|
||||
- **pnpm** (v11), not npm or bun. Lockfiles are committed. Node ≥ 22.
|
||||
- ESM throughout (`"type": "module"` in all package.json files).
|
||||
|
||||
### Import style
|
||||
|
||||
Source files use `@/*` path aliases (mapped in tsconfig to `./src/*`). Relative imports **must include the `.js` extension** (e.g., `from "./embed.js"`). The `moduleResolution: "bundler"` tsconfig setting allows bare specifiers during dev, but `tsc` emits them as-is. A post-build script (`scripts/fix-imports.mjs`) rewrites both `@/` aliases and extensionless imports in `dist/` so Node ESM can resolve them at runtime.
|
||||
|
||||
### Error handling
|
||||
|
||||
Both gateway and backend define an `AppError` base class in `@/shared/errors/index` with subclasses: `ValidationError` (400), `NotFoundError` (404), `UnauthorizedError` (401), `DatabaseError` (500), `ConfigError` (500). Services throw these; callers or middleware map them to HTTP status codes.
|
||||
|
||||
### Logging
|
||||
|
||||
Use `createChildLogger('module-name')` from `@/shared/logger/index`. Never use raw `console`. It wraps pino; in development it pretty-prints via `pino-pretty`.
|
||||
|
||||
### Config
|
||||
|
||||
Environment variables are validated with Zod at startup in `shared/config/index.ts` of both gateway and backend. Do not read `process.env` directly outside config modules.
|
||||
|
||||
### Module boundaries
|
||||
|
||||
- Gateway: each feature lives in `src/modules/<name>/` with its own `index.ts` barrel. Modules register event listeners and are composed in `src/app/bootstrap.ts`.
|
||||
- Backend: `modules/<name>/` follows schema → repository → service → controller → routes. Data flows up only. No cross-module repository imports.
|
||||
- Frontend: `page.tsx` is a server component that fetches data via `src/lib/api/server.ts` (oRPC over HTTP, server-side only) and passes it to a `view.tsx` client component. Browser code uses `src/lib/orpc/client.ts` (oRPC over WebSocket via partysocket). Never import the server API client from a client component.
|
||||
|
||||
### API layer
|
||||
|
||||
The backend exposes an oRPC router mounted at `/trpc` (both HTTP and WebSocket). The frontend does **not** use a REST `/api/*` layer — all data goes through oRPC procedures. The server-side client (`src/lib/api/server.ts`) uses a fetch-based RPCLink; the browser client uses a WebSocket-based RPCLink backed by partysocket for auto-reconnection. Results are asserted to the frontend's local types at each call site (the backend's router type is not imported into the frontend).
|
||||
|
||||
### Testing
|
||||
|
||||
- **Gateway**: Vitest, tests in `tests/` at the service root. Config sets env vars (`DISCORD_TOKEN`, `DATABASE_URL`, etc.) so tests run without real services.
|
||||
- **Backend**: Vitest, tests in `tests/` and `src/`. Includes an `e2e.test.ts` excluded from CI (needs a live backend).
|
||||
- **Frontend**: No test runner configured.
|
||||
- Tests are pure-function / unit-level. Mock external dependencies; do not start real DB/Redis in tests.
|
||||
|
||||
### Formatting
|
||||
|
||||
Biome, 2-space indent, spaces. Config at repo root `biome.json`. `lint` uses `--diagnostic-level=error`; `format` auto-writes.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Never commit without running `fix-imports.mjs` after `tsc`** — gateway and backend builds will produce ESM that crashes at startup (`ERR_MODULE_NOT_FOUND`).
|
||||
2. **Don't add `@discordjs/opus` build-from-source flags** — it ships prebuilt binaries for Node 22. Forcing source builds in CI/Nix will fail or add minutes of compile time.
|
||||
3. **Frontend SSR fetches are always `cache: "no-store"`** — the server API client bypasses Next.js fetch cache. Do not add caching without understanding the live dashboard requirement.
|
||||
4. **oRPC types are loosely coupled** — the frontend casts oRPC results to its own types with `as unknown`. Adding a field to the backend schema does not automatically update the frontend type. Update both sides.
|
||||
5. **Gateway is a selfbot** (`discord.js-selfbot-v13`) — it uses a user token, not a bot token. It must not be deployed as a standard Discord bot.
|
||||
@@ -0,0 +1,9 @@
|
||||
# Roadmap
|
||||
|
||||
What is built, what is next, and what has been deliberately declined.
|
||||
|
||||
---
|
||||
|
||||
## Next
|
||||
|
||||
_Empty._
|
||||
@@ -0,0 +1,23 @@
|
||||
# TODO
|
||||
|
||||
Next up. One item, one outcome, verifiable when done.
|
||||
|
||||
Longer-term direction lives in [ROADMAP.md](ROADMAP.md).
|
||||
|
||||
---
|
||||
|
||||
## Now
|
||||
|
||||
_Empty._
|
||||
|
||||
---
|
||||
|
||||
## Next
|
||||
|
||||
_Empty._
|
||||
|
||||
---
|
||||
|
||||
## Maintenance
|
||||
|
||||
_Empty._
|
||||
+131
@@ -0,0 +1,131 @@
|
||||
# docs/ — Spec-Driven Development Hub
|
||||
|
||||
Direktori ini adalah pusat dari workflow **spec-driven development** di GMW.
|
||||
Semua perubahan signifikan dimulai dari satu spec, **sebelum kode ditulis**,
|
||||
dan setiap tugas yang berjalan dilacak lewat `todo.md`.
|
||||
|
||||
## Struktur
|
||||
|
||||
```
|
||||
docs/
|
||||
├── README.md # Dokumen ini
|
||||
├── spec-template.md # Template standar untuk semua spec
|
||||
├── todo-template.md # Template todo list
|
||||
└── specs/ # Semua spec dan implementation plan
|
||||
├── YYYY-MM-DD_<slug>-spec.md # Spec (apa & mengapa)
|
||||
└── YYYY-MM-DD_<slug>.md # Plan/fix (bisa langsung spec+plan)
|
||||
```
|
||||
|
||||
### Naming convention
|
||||
|
||||
```
|
||||
YYYY-MM-DD_<slug>-spec.md # Spec baru untuk fitur/fix
|
||||
YYYY-MM-DD_<slug>.md # Plan yang sudah include spec di dalamnya
|
||||
```
|
||||
|
||||
Contoh:
|
||||
- `2026-08-30_recordings-v2-features-spec.md` — spec untuk Recordings v2
|
||||
- `2026-08-24-attachment-delay-fix.md` — spec + plan untuk attachment delay fix
|
||||
|
||||
### Service-specific docs
|
||||
|
||||
```
|
||||
services/<service>/docs/specs/ # Spec yang spesifik untuk 1 service
|
||||
```
|
||||
|
||||
## Workflow: Spec-Driven Development
|
||||
|
||||
### Prinsip utama
|
||||
|
||||
> **No code without a spec.** Semua perubahan signifikan harus punya spec
|
||||
> terlebih dahulu. Spec adalah kontrak: apa yang akan dibangun, mengapa,
|
||||
> dan bagaimana memverifikasinya.
|
||||
|
||||
### Todo list (`todo.md`) — WAJIB
|
||||
|
||||
Setiap tugas berjalan (implementasi fitur/fix) harus punya `todo.md`:
|
||||
|
||||
```markdown
|
||||
# Todo — <judul tugas>
|
||||
|
||||
## Task
|
||||
- [ ] Tulis spec
|
||||
- [ ] Verifikasi facts (baca kode: file:line)
|
||||
- [ ] Implementasi tahap 1: ...
|
||||
- [ ] Implementasi tahap 2: ...
|
||||
- [ ] Verifikasi: pnpm typecheck / lint / build / test
|
||||
- [ ] Commit + push
|
||||
```
|
||||
|
||||
Aturan `todo.md`:
|
||||
- Task **konkret & verifiable** — "ubah X di file Y", bukan "fix bug".
|
||||
- Satu task `[in_progress]` pada satu waktu; ceklis `[x]` segera setelah selesai.
|
||||
- Gunakan `docs/todo-template.md` sebagai template.
|
||||
- Simpan `todo.md` di `docs/` (tugas lintas-service) atau di
|
||||
`services/<service>/docs/` (tugas satu service).
|
||||
|
||||
### Kapan perlu spec + todo
|
||||
|
||||
| Perlu spec + todo | Tidak perlu |
|
||||
|---|---|
|
||||
| Fitur baru | Typo fix |
|
||||
| Bug fix non-trivial | Dependency bump (dependabot) |
|
||||
| Refactor yang mengubah behavior | Format/lint auto-fix |
|
||||
| Perubahan DB schema | Test-only change |
|
||||
| Perubahan API contract | README/doc update |
|
||||
| Perubahan arsitektur | Variable rename |
|
||||
|
||||
### Workflow step-by-step
|
||||
|
||||
1. **Tulis spec** — Salin `docs/spec-template.md`, isi semua section yang relevan.
|
||||
- **Verified facts**: baca kode yang terpengaruh, catat temuan dengan file:line.
|
||||
- **Root cause**: analisis sebab (jangan hanya describe symptom).
|
||||
- **Decisions**: pilih pendekatan, jelaskan alternatif yang ditolak.
|
||||
- **Verification**: command executable, bukan "should work".
|
||||
- Simpan sebagai `docs/specs/YYYY-MM-DD_<slug>-spec.md`.
|
||||
|
||||
2. **Tulis `todo.md`** — Pecah spec jadi task konkret yang bisa diceklis
|
||||
(gunakan `docs/todo-template.md`), dengan urutan implementasi yang logis.
|
||||
|
||||
3. **Review spec** — Baca ulang spec sendiri. Cek:
|
||||
- Apakah verified facts benar-benar verified (bukan asumsi)?
|
||||
- Apakah file changes lengkap (tidak ada yang terlewat)?
|
||||
- Apakah verification steps executable?
|
||||
- Jika spec untuk user request: pastikan user setuju dengan approach.
|
||||
|
||||
4. **Implement** — Ikuti spec + todo step-by-step. Ceklis `[x]` setiap task
|
||||
yang selesai. Jika menemukan sesuatu yang berubah dari asumsi spec,
|
||||
**update spec dulu**, baru implement.
|
||||
|
||||
5. **Verify** — Jalankan semua verification steps di spec. Catat hasilnya.
|
||||
|
||||
6. **Commit** — Reference spec di commit message:
|
||||
```
|
||||
feat(module): deskripsi singkat
|
||||
|
||||
Ref: docs/specs/YYYY-MM-DD_<slug>-spec.md
|
||||
```
|
||||
|
||||
### Tips menulis spec yang baik
|
||||
|
||||
- **Evidence-based**: setiap klaim harus ada sumbernya (file:line, log, error).
|
||||
- **Actionable**: orang lain (atau AI agent) harus bisa implementasi dari spec saja.
|
||||
- **Verifiable**: setiap requirement harus bisa di-test secara eksplisit.
|
||||
- **Scoped**: satu spec = satu perubahan terfokus. Jangan campur 3 fitur dalam 1 spec.
|
||||
- **Bahasa campuran**: narrative boleh Indonesia/English, istilah teknis pakai English.
|
||||
|
||||
### Role of AI agents
|
||||
|
||||
AI coding agents (Shiro Neko / Hermes) **harus**:
|
||||
- Membaca spec sebelum menulis kode
|
||||
- Membuat & meng-update `todo.md` untuk setiap tugas berjalan
|
||||
- Verifikasi fakta dari kode aktual (bukan dari asumsi training data)
|
||||
- Update spec jika temuan baru mengubah approach
|
||||
- Jalankan verification steps setelah implementasi
|
||||
- Reference spec di commit message
|
||||
|
||||
AI agents **tidak boleh**:
|
||||
- Langsung implement tanpa membaca/menulis spec
|
||||
- Mengabaikan verified facts yang bertentangan dengan asumsi
|
||||
- Skip verification steps
|
||||
- Mengerjakan tugas tanpa todo list yang jelas
|
||||
@@ -0,0 +1,80 @@
|
||||
# Spec: <Judul singkat, aktif, deskriptif>
|
||||
|
||||
Status: **DRAFT** | **APPROVED** | **IN PROGRESS** | **DONE**
|
||||
Date: YYYY-MM-DD
|
||||
Author: <nama/agent>
|
||||
Related: <link ke spec/plan terkait jika ada>
|
||||
Todo: <link ke todo.md untuk tugas ini, jika ada>
|
||||
|
||||
## Problem
|
||||
|
||||
<Deskripsi masalah atau keinginan. Apa yang rusak? Apa yang belum ada?
|
||||
Sebutkan siapa yang melaporkan (user/bot/audit) dan konteksnya.
|
||||
Gunakan **evidence**: log, error, metrik, atau observasi langsung.>
|
||||
|
||||
## Root cause
|
||||
|
||||
<Analisis teknis mengapa masalah ini terjadi. Jika bug: trace dari symptom ke cause.
|
||||
Jika fitur baru: jelaskan gap saat ini. Referensikan file + line number yang relevan.>
|
||||
|
||||
## Behavior target
|
||||
|
||||
<Deskripsi eksplisit perilaku yang diharapkan setelah fix/fitur.
|
||||
Buat list bernomor. Setiap item harus bisa di-verify.>
|
||||
|
||||
## Verified facts
|
||||
|
||||
>Fakta-fakta teknis yang **sudah diverifikasi** dari pembacaan kode, log produksi,
|
||||
>atau eksperimen langsung. Setiap fakta harus citation ke file:line.
|
||||
>Jika belum diverifikasi, tulis `UNVERIFIED` dan rencana verifikasi.
|
||||
|
||||
- `path/to/file.ts:42` — <apa yang terjadi di sini>
|
||||
- `path/to/file.ts:88` — <apa yang terjadi di sini>
|
||||
- Kolom DB `table.column` — <tipe, constraint, index>
|
||||
- Config `ENV_VAR` default <value> — <dibaca di mana>
|
||||
|
||||
## Keputusan desain
|
||||
|
||||
<Pilihan desain yang dibuat, beserta rationale (mengapa bukan alternatif lain).
|
||||
Format: nomor, judul singkat, penjelasan.>
|
||||
|
||||
1. **<Judul keputusan>**: <penjelasan + rationale>
|
||||
- Alternatif yang ditolak: <apa + mengapa ditolak>
|
||||
|
||||
## Perubahan file
|
||||
|
||||
>Daftar **semua file** yang perlu diubah/ditambah/dihapus, dikelompokkan per service.
|
||||
>Untuk setiap file: sebutkan **apa yang berubah** (bukan copy-paste kode).
|
||||
|
||||
### Gateway (`services/discord-gateway/`)
|
||||
- `src/modules/<module>/<file>.ts` — <ringkasan perubahan>
|
||||
|
||||
### Backend (`services/backend/`)
|
||||
- `src/modules/<module>/<file>.ts` — <ringkasan perubahan>
|
||||
|
||||
### Frontend (`services/frontend/`)
|
||||
- `src/<path>/<file>.ts|tsx` — <ringkasan perubahan>
|
||||
|
||||
### Database / Config
|
||||
- <migration file atau config change jika ada>
|
||||
|
||||
## Schema/type changes
|
||||
|
||||
<Perubahan tipe, interface, atau DB schema. Jika tidak ada, tulis "TIDAK ada perubahan.">
|
||||
|
||||
## Verification
|
||||
|
||||
>**Harus spesifik dan executable.** Jangan tulis "works correctly".
|
||||
>Tulis command yang bisa dijalankan dan expected outcome.
|
||||
|
||||
- **Typecheck**: `<service> pnpm typecheck` — clean
|
||||
- **Lint**: `<service> pnpm lint` — no errors
|
||||
- **Build**: `<service> pnpm build` — compiles
|
||||
- **Test**: `<service> pnpm test` — all pass (+ test baru jika ada)
|
||||
- **Smoke test**: <langkah manual untuk verifikasi visual/behavioral>
|
||||
- **DB**: <query untuk cek data jika applicable>
|
||||
- **Deploy**: CI green → deploy → cek <specific observable>
|
||||
|
||||
## Notes (opsional)
|
||||
|
||||
<Catatan tambahan: risiko, fallback, future work, atau hal yang sengaja di-scope-out.>
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
+2
-2
@@ -3,8 +3,8 @@
|
||||
Status: **P1–P3 DONE + 4th CRITICAL FIX deployed (f1a7b0c2); DAVE Ready + MLS handshake CONFIRMED live; P4 = waiting on active streamer to confirm video-burst→mp4**
|
||||
Date: 2026-08-31
|
||||
Author: Hermes
|
||||
Related: `.hermes/plans/2026-08-31_video-receive-eager-selfbot-connection-spec.md` (superseded by this)
|
||||
`.hermes/plans/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, selfbot path — dead)
|
||||
Related: `docs/specs/2026-08-31_video-receive-eager-selfbot-connection-spec.md` (superseded by this)
|
||||
`docs/specs/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, selfbot path — dead)
|
||||
|
||||
## Problem / Ground truth (established from live logs 2026-08-31)
|
||||
GMW must record OTHER members' screen-share + camera video in a voice channel it
|
||||
+1
-1
@@ -3,7 +3,7 @@
|
||||
Status: PLANNED
|
||||
Date: 2026-08-31
|
||||
Author: Hermes
|
||||
Related: `.hermes/plans/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, made Option A this fix)
|
||||
Related: `docs/specs/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, made Option A this fix)
|
||||
|
||||
## Symptom (from live logs, 2026-08-31 ~12:34)
|
||||
A user was actively screen-sharing + on camera in the recorded voice channel.
|
||||
+1
-1
@@ -3,7 +3,7 @@
|
||||
Status: PLANNED (not built)
|
||||
Date: 2026-08-31
|
||||
Author: Hermes
|
||||
Related: `.hermes/plans/2026-08-30_video-record-receive-spec.md` (Phase A/B — raw UDP hook, superseded for receive)
|
||||
Related: `docs/specs/2026-08-30_video-record-receive-spec.md` (Phase A/B — raw UDP hook, superseded for receive)
|
||||
|
||||
## TL;DR — what changed vs Phase A/B
|
||||
|
||||
File renamed without changes.
+1
-1
@@ -3,7 +3,7 @@
|
||||
Status: PLANNED (not yet built)
|
||||
Date: 2026-09-02
|
||||
Author: Hermes
|
||||
Related: `.hermes/plans/2026-08-31_video-receive-phaseC-spec.md` (auto-receive, superseded
|
||||
Related: `docs/specs/2026-08-31_video-receive-phaseC-spec.md` (auto-receive, superseded
|
||||
for selfbot), `gmw-ops/references/selfbot-presence-detection-limits.md`,
|
||||
`gmw-ops/references/discord-voice-fork-video-receive.md`
|
||||
|
||||
File renamed without changes.
@@ -0,0 +1,27 @@
|
||||
# Todo — <judul tugas>
|
||||
|
||||
Status: **IN PROGRESS** | **BLOCKED** | **DONE**
|
||||
Spec: <link ke spec: `docs/specs/YYYY-MM-DD_<slug>-spec.md`>
|
||||
Date: YYYY-MM-DD
|
||||
|
||||
## Task
|
||||
|
||||
> Urutan implementasi sesuai spec. Task harus konkret & verifiable
|
||||
> ("ubah X di file Y"), bukan "fix bug". Satu task dikerjakan pada satu
|
||||
> waktu; ceklis `[x]` segera setelah selesai.
|
||||
|
||||
- [ ] Tulis spec
|
||||
- [ ] Verifikasi facts (baca kode: `file:line`)
|
||||
- [ ] Implementasi tahap 1: <apa + file mana>
|
||||
- [ ] Implementasi tahap 2: <apa + file mana>
|
||||
- [ ] Implementasi tahap 3: <apa + file mana>
|
||||
- [ ] Typecheck: `pnpm typecheck` — clean
|
||||
- [ ] Lint: `pnpm lint` — no errors
|
||||
- [ ] Test: `pnpm test` — all pass
|
||||
- [ ] Build: `pnpm build` — compiles
|
||||
- [ ] Daftar ulang hasil verifikasi di spec
|
||||
- [ ] Commit + push (reference spec di commit message)
|
||||
|
||||
## Notes
|
||||
|
||||
<Blocking issue, temuan yang mengubah approach, atau hal yang perlu dicatat.>
|
||||
@@ -0,0 +1,117 @@
|
||||
# Backend Service — Agent Guide
|
||||
|
||||
> **Read `../../AGENTS.md` first.** This file adds backend-specific conventions.
|
||||
|
||||
Backend service: Express HTTP server + WebSocket, serves the GMW dashboard API.
|
||||
|
||||
## Quick reference
|
||||
|
||||
```bash
|
||||
pnpm typecheck # tsc --noEmit
|
||||
pnpm lint # biome check --diagnostic-level=error .
|
||||
pnpm build # tsc
|
||||
pnpm test # vitest run
|
||||
pnpm format # biome format --write .
|
||||
```
|
||||
|
||||
## Architecture (Modular MVC)
|
||||
|
||||
```
|
||||
src/
|
||||
├── shared/ # Infrastructure (no business logic)
|
||||
│ ├── config/index.ts # Zod-validated env
|
||||
│ ├── database/ # Drizzle ORM + pg Pool
|
||||
│ ├── errors/index.ts # AppError hierarchy
|
||||
│ ├── logger/index.ts # pino + createChildLogger()
|
||||
│ ├── middlewares/index.ts # errorHandler, asyncHandler, rateLimit
|
||||
│ └── utils/ # Pagination, messageMapper
|
||||
├── modules/ # Feature modules
|
||||
│ └── <module>/
|
||||
│ ├── <module>.schema.ts # Zod validation schemas
|
||||
│ ├── <module>.repository.ts # DB operations only
|
||||
│ ├── <module>.service.ts # Business logic
|
||||
│ ├── <module>.controller.ts # HTTP handlers
|
||||
│ └── routes/index.ts # Express router
|
||||
├── http/ # app.ts (factory) + server.ts (startup)
|
||||
├── ws/ # WebSocket server + Redis bridge
|
||||
└── index.ts # Entry point
|
||||
```
|
||||
|
||||
## Dependency rules
|
||||
|
||||
- Controller → Service → Repository → Database
|
||||
- No cross-module repo imports (each module owns its data)
|
||||
- No HTTP in Service layer (no req/res)
|
||||
- No DB in Controller layer
|
||||
- Any layer → Config, Logger, Errors
|
||||
|
||||
## API: oRPC + Express
|
||||
|
||||
- **oRPC** (`src/orpc/router.ts` + `src/orpc/ws.ts`): type-safe procedures for frontend
|
||||
- **Express routes** (`src/modules/*/routes/`): REST endpoints under `/api/`
|
||||
- **WebSocket** (`src/ws/`): Redis bridge → broadcast to connected browsers
|
||||
- Never invent endpoints. Match what the frontend calls (`src/lib/api/`).
|
||||
|
||||
## Key modules
|
||||
|
||||
| Module | Purpose | DB tables |
|
||||
|---|---|---|
|
||||
| messages | Store & query Discord messages | `messages`, `ai_moderations`, `ai_moderation_flags` |
|
||||
| moderation | Moderation actions & metrics | `ai_moderations`, `moderation_actions` |
|
||||
| media | Media file management | `media_attachments` |
|
||||
| voice | Live speakers + recordings | `voice_recordings` |
|
||||
| recordings | Recordings API | `voice_recordings` |
|
||||
| dashboard | Stats aggregation | Various (read-only) |
|
||||
| knowledge | Semantic search | Qdrant vector DB |
|
||||
| chatbot | AI chatbot with tools | `chatbot_history` |
|
||||
| health | Health checks + metrics | Various |
|
||||
| analysis | Text analysis cache | `text_analysis_cache` |
|
||||
| ui-state | Persist UI preferences | `ui_state` |
|
||||
|
||||
## Config (env vars)
|
||||
|
||||
All validated via Zod in `shared/config/index.ts`. Key vars:
|
||||
|
||||
- `WEBSERVER_PORT` (default 4001)
|
||||
- `DATABASE_URL` or individual `DATABASE_HOST/PORT/NAME/USER/PASSWORD`
|
||||
- `REDIS_URL` — for pub/sub with gateway
|
||||
- `MONITOR_GUILD_ID` — primary Discord guild
|
||||
- `ADMIN_PASSWORD` — admin endpoints
|
||||
|
||||
## Data contract with frontend
|
||||
|
||||
The frontend fetches via `src/lib/api/server.ts` (SSR, server-side) and
|
||||
`src/lib/api/client.ts` (browser, same-origin through proxy).
|
||||
|
||||
**Do not change response shapes without updating both sides.** Check
|
||||
`services/frontend/src/lib/types/` for frontend type definitions.
|
||||
|
||||
## Redis channels (inbound from gateway)
|
||||
|
||||
```
|
||||
discord:message:{created,updated,deleted,analyzed}
|
||||
discord:attachment:{created,uploaded}
|
||||
discord:voice:{started,stopped,uploaded,active_user,pcm,analyzed}
|
||||
discord:analysis:queue_status
|
||||
discord:reaction:{added,removed}
|
||||
discord:thread:{created,deleted,updated}
|
||||
discord:channel_topic:updated
|
||||
discord:presence:updated
|
||||
discord:guild_member:{added,removed}
|
||||
```
|
||||
|
||||
Canonical names: `src/shared/redis-channels.ts`.
|
||||
|
||||
## Testing
|
||||
|
||||
- Vitest for unit tests
|
||||
- Mock database and external services
|
||||
- Test files: `src/modules/<module>/*.test.ts` or `tests/*.test.ts`
|
||||
- Run: `pnpm test`
|
||||
|
||||
## Common pitfalls
|
||||
|
||||
- **oRPC vs REST**: check both routers when adding an endpoint
|
||||
- **Redis channel mismatch**: gateway publishes → backend subscribes. Channel
|
||||
names must match exactly (see `redis-channels.ts` in BOTH services)
|
||||
- **DB pool**: backend uses one pool (main thread). Gateway has per-piscina-thread pools.
|
||||
@@ -0,0 +1,153 @@
|
||||
# Discord Gateway — Agent Guide
|
||||
|
||||
> **Read `../../AGENTS.md` first.** This file adds gateway-specific conventions.
|
||||
|
||||
Event-driven microservice: captures Discord events, runs AI moderation, publishes to Redis.
|
||||
|
||||
## Quick reference
|
||||
|
||||
```bash
|
||||
pnpm typecheck # tsc --noEmit
|
||||
pnpm lint # biome check --diagnostic-level=error .
|
||||
pnpm build # tsc
|
||||
pnpm test # vitest run
|
||||
pnpm format # biome format --write .
|
||||
```
|
||||
|
||||
## Architecture (Event-driven)
|
||||
|
||||
```
|
||||
src/
|
||||
├── index.ts # Entry point → initializeDiscordGateway()
|
||||
├── app/
|
||||
│ ├── bootstrap.ts # Wires client, DB, Redis, workers, schedulers
|
||||
│ ├── shutdown.ts # Graceful shutdown
|
||||
│ └── retention.ts # Expired-record cleanup
|
||||
├── shared/
|
||||
│ ├── config/index.ts # Zod-validated env (SINGLE source of truth)
|
||||
│ ├── database/ # Drizzle ORM + pg Pool + migrations
|
||||
│ ├── logger/index.ts # pino + createChildLogger()
|
||||
│ ├── errors/index.ts # AppError hierarchy
|
||||
│ ├── utils/ # retry, pagination
|
||||
│ ├── discord/clientOptions.ts # discord.js-selfbot-v13 options
|
||||
│ ├── uploader.ts # Attachment upload helper
|
||||
│ ├── redis-channels.ts # Redis channel-name constants
|
||||
│ └── moderation-types.ts # Shared AI analysis types
|
||||
├── modules/
|
||||
│ ├── ai-moderation/ # LLM moderation pipeline (largest module)
|
||||
│ ├── message-capture/ # Discord event listeners + DB store
|
||||
│ ├── voice-recording/ # Voice connect + Opus→OGG recording
|
||||
│ │ └── recorder/ # decoder, segment, session, uploader, oggCrc
|
||||
│ ├── voice-pcm-ws/ # Real-time PCM → backend WebSocket
|
||||
│ ├── attachment-upload/ # Download + sharp resize + upload
|
||||
│ ├── event-broadcaster/ # Redis pub/sub publisher
|
||||
│ ├── command-handler/ # Backend→gateway Redis commands
|
||||
│ ├── reaction-tracking/ # Reaction events
|
||||
│ ├── thread-tracking/ # Thread events
|
||||
│ ├── user-presence/ # Presence/status events
|
||||
│ ├── channel-topic/ # Channel topic events
|
||||
│ ├── guild-member-events/ # Member join/leave
|
||||
│ └── gateway-metrics/ # Prometheus /metrics (port 4016)
|
||||
└── tests/ # Vitest suites
|
||||
```
|
||||
|
||||
## Key invariants (DO NOT BREAK)
|
||||
|
||||
1. **LLM is the only judge.** Failed LLM → `status:"error"` + recovery retry.
|
||||
**Never** reintroduce regex/heuristic content classification.
|
||||
2. **Discord tokens sanitized** before reaching LLM (`discordTokens.ts`).
|
||||
3. **Semantic cache is batched** — one embed call + one Qdrant batch search.
|
||||
4. **Streaming is mandatory** against the omniroute base URL.
|
||||
|
||||
## AI moderation pipeline
|
||||
|
||||
```
|
||||
aiAnalyzer.ts → batchScheduler.ts → batchProcessor.ts → individualFallbackProcessor.ts
|
||||
↓ ↓ ↓ ↓
|
||||
moderationOrchestrator.ts → (hash cache → Qdrant → LLM)
|
||||
↓ ↓ ↓
|
||||
textBatchProcessor.ts mediaBatchProcessor.ts llmClient.ts
|
||||
embeddingClient.ts
|
||||
qdrantClient.ts
|
||||
```
|
||||
|
||||
- Entry: `aiAnalyzer.ts` (`queueMessageAnalysis`, `startPendingAIAnalysisWorker`)
|
||||
- Concurrency: LLM semaphore (`AI_LLM_MAX_CONCURRENT`, default 5)
|
||||
- Piscina: text pool (4 threads) + media pool (2 threads)
|
||||
- **Each worker thread has its own pg Pool** (min 0, grows to `POSTGRES_POOL_MAX`)
|
||||
|
||||
## Voice recording pipeline
|
||||
|
||||
```
|
||||
receiver.speaking "start" → speakingHandler(userId)
|
||||
→ collectUserMetadata → receiver.subscribe → PacketFilter → oggPacketStream
|
||||
→ SegmentManager.open → OggLogicalBitstream → .ogg file
|
||||
→ data: rotateIfNeeded + decoder.write
|
||||
→ end: finalizeSegment → upload + transcribe
|
||||
```
|
||||
|
||||
Key files:
|
||||
- `voiceController.ts` — connect/disconnect/list
|
||||
- `recorder.ts` — orchestration
|
||||
- `recorder/segment.ts` — segment rotation
|
||||
- `recorder/sessionRecording.ts` — session management
|
||||
- `recorder/uploader.ts` — upload to storage
|
||||
- `voiceTranscriber.ts` — Whisper transcription (if enabled)
|
||||
|
||||
## Module: message-capture
|
||||
|
||||
- `messageCapture.ts` — Discord event listeners (messageCreate/Update/Delete)
|
||||
- `messageStore.ts` — DB operations
|
||||
- `messageMetadata.ts` — metadata extraction
|
||||
- `messagesDb.ts` / `messagesCrud.ts` — DB schema operations
|
||||
- `archiveEmbedder.ts` — Qdrant embedding (respect age-restricted guard)
|
||||
- `retentionDb.ts` / `reviewsDb.ts` / `attachmentsDb.ts` — auxiliary tables
|
||||
|
||||
## Redis channels (outbound to backend)
|
||||
|
||||
See `src/shared/redis-channels.ts` for canonical names. Examples:
|
||||
```
|
||||
discord:message:created, discord:voice:active_user, discord:attachment:uploaded
|
||||
```
|
||||
|
||||
## Config (env vars)
|
||||
|
||||
All validated via Zod in `shared/config/index.ts`. Critical:
|
||||
|
||||
- `DISCORD_TOKEN` — selfbot token
|
||||
- `MONITOR_GUILD_ID` — primary guild
|
||||
- `DATABASE_URL` — PostgreSQL
|
||||
- `REDIS_URL` — pub/sub to backend
|
||||
- `AI_ANALYSIS_ENABLED` — master toggle for AI moderation
|
||||
- `AI_LLM_BASE_URL` / `AI_LLM_API_KEY` — LLM router
|
||||
- `AI_VOICE_TRANSCRIPTION_ENABLED` — toggle Whisper transcription
|
||||
- `PISCINA_MAX_THREADS` / `PISCINA_MEDIA_MAX_THREADS` — worker pool sizing
|
||||
|
||||
## Concurrency model
|
||||
|
||||
- Main thread: event loop + LLM semaphore
|
||||
- Text Piscina pool: `PISCINA_MAX_THREADS` (default 4)
|
||||
- Media Piscina pool: `PISCINA_MEDIA_MAX_THREADS` (default 2)
|
||||
- Batch routed to media pool if ANY message has attachment/sticker/embed
|
||||
- Each worker thread initializes own pg Pool (min 0, grows to `POSTGRES_POOL_MAX`)
|
||||
- MemoryMax: 1G (service systemd limit)
|
||||
|
||||
## Testing
|
||||
|
||||
- Vitest. Test files: `tests/<name>.test.ts`
|
||||
- Run: `pnpm test`
|
||||
- Key test areas: batch operations, cache guards, image/video handling, context enrichment
|
||||
- Unit tests for pure helpers (classifier, normalize, parse), integration for pipeline stages
|
||||
|
||||
## Common pitfalls
|
||||
|
||||
- **Piscina pool isolation**: worker threads are NOT the main thread. Cannot
|
||||
share state via module-level variables. Use DB or Redis for cross-thread state.
|
||||
- **AfterSilence race**: `@discordjs/voice` AfterSilence can fail to emit "end"
|
||||
on disconnect. Always have a watchdog/timeout.
|
||||
- **Cache eviction**: LRU caches (user metadata, term glossary) evict at max size.
|
||||
Don't assume cache hit after eviction.
|
||||
- **Circuit breaker**: per-conversation CB opens after repeated failures.
|
||||
Check `circuitBreaker.ts` state when debugging "missing analysis".
|
||||
- **Gateway≠Backend schema**: both have `redis-channels.ts` and `moderation-types.ts`.
|
||||
Keep them in sync manually.
|
||||
@@ -1,3 +1,5 @@
|
||||
@../../AGENTS.md
|
||||
|
||||
# Bete Frontend — Project Overview
|
||||
|
||||
Next.js 16 (App Router), React 19, TypeScript strict, Tailwind v4, shadcn/ui, base-ui.
|
||||
|
||||
File renamed without changes.
File renamed without changes.
Reference in new issue
Block a user