Author SHA1 Message Date
asepharyana 588e750ede fix: switch GMW AI source from omniroute to 9router
- Change AI_LLM_BASE_URL default from omniroute.imrnes.team to 9router.asepharyana.my.id
- Update AI_LLM_MODEL default from 'text' to 'claude-opus-5' (bare model name
  compatible with 9router/OpenAI-compatible router)
- Update .env.example and inline comments to reflect 9router
- discord-gateway config now matches backend (which already uses 9router)
2026-08-25 20:22:40 +07:00
asepharyana 31e303c187 feat(fe): optimize error handling, state consistency, and WS feedback
- Add global SWR config with exponential backoff retry (swr-config.ts)
- Add ErrorBoundary component for React crash recovery (error-boundary.tsx)
- Standardize ErrorState onRetry on all 6 dashboard views (dashboard, media,
  messages, moderation, recordings, voice)
- Fix useAction: expose resetError + onError callback
- Refactor useSpeakers to SWR-backed state (was local useState) for
  consistent cache/revalidate semantics with other hooks
- Remove polling in useReview (15s refreshInterval); replace with
  useReviewWsSync subscribing to WS moderation_action events
- Extract hashUserId to lib/hash.ts (de-dup with ambient-canvas)
- WS context: add reconnect/error toast feedback via onStatusChange
- WS connection: expose reconnectAttemptCount getter

All typecheck + lint clean, Next 16 build passes.
2026-08-25 13:46:15 +07:00
asepharyana 796c6390ac perf(gmw-ai): cache-hit optimization — bare-key Qdrant upsert + hit_count bump 2026-08-25 12:03:47 +07:00
asepharyana ecbf2617e4 fix(ai-moderation): matikan hot requeue loop saat upload attachment in-flight
Batch race guard balikin {ok:true, rows:[]} tanpa sinyal saat semua target
masih upload-pending -> processor klasifikasi semua incomplete -> fanout ke
individual queue -> di situ requeue + reschedule 250ms -> balik ke batch:
hot loop ~300ms sepanjang upload (10 siklus/3 dtk di log prod 08:13).

Fix: worker batch kini return uploadPendingIds eksplisit; classifier pure
baru (partitionBatchOutcome) partisi completed/upload_pending/incomplete/
parse_failed/api_failed; target upload-pending DEFERRED dengan poll backoff
linear (AI_ANALYSIS_UPLOAD_POLL_MS 1500 base, cap AI_ANALYSIS_MAX_UPLOAD_POLL_MS
8000), tidak pernah masuk fanout; tail shouldScheduleNext tak menimpa defer.
Test: tests/batchOutcomeClassifier.test.ts (8 kasus, pure tanpa DB/Piscina).
2026-08-25 10:37:11 +07:00
asepharyana 9ef7d005fb style(llmClient): explicit type utk let completion (noImplicitAnyLet) 2026-08-24 22:07:33 +07:00
asepharyana 842610b1af fix(ai-moderation): bedah delay attachment ~330s -> target <20s
Root cause (trace msg 1541417073245290638):
- Race-guard upload-pending balik results:[] diperlakukan sbg SUKSES
  -> row yatam 'processing' sampai cleanup 300s mengembalikan
- Vision gagal 3x utk GIF besar (SSE truncation) tanpa fallback

Fix:
- Sinyal eksplisit uploadPending dari worker race guard
- Classifier murni classifyIndividualWorkerResult(): upload_pending ->
  requeue pending + reschedule segera (250ms), bukan error palsu;
  empty-results ok:true kini error transien (bug silent-success mati)
- llmVision fallback stream:false sekali saat SSE truncation
- Safety-net cleanup stuck processing 300s -> 120s
2026-08-24 22:05:28 +07:00
asepharyana fca96396b9 style(ai-moderation): sort import moderationOrchestrator (biome organizeImports) 2026-08-24 20:20:26 +07:00
asepharyana ccf3fa260e perf(ai-moderation): dual-key write-back — clean verdict ikut di-cache global
Analisis pertama tetap berkonteks (chat history) demi akurasi, tapi
verdict clean non-actionable (conf>=0.85) juga ditulis di bare key
tanpa konteks. Repeat teks sama di channel lain -> exact cache HIT,
bukan LLM call baru. Guard sama dgn read path; dedupe LRU per proses;
bare row tanpa embedding (tier semantic sudah global).
2026-08-24 20:18:14 +07:00
asepharyana 1accfd9390 perf(ai-moderation): naikkan cache hit dgn guard akurasi
- Fase-1 exact-cache lookup: N query serial -> SATU query ANY($1::text[])
- Global reuse utk bare key legacy, HANYA verdict non-actionable
  (clean/flagless/action=none, conf>=0.85, umur<=72h) — flagged/warn
  tetap context-scoped
- Semantic cache dua-band: clean band 0.92 default, actionable tetap
  0.97; di antara band -> LLM (fail-open ke akurasi)
- hit_count kini di-increment (bulk UPDATE per batch) -> hit-rate terukur
- Cache hasil wikipediaSearch di Redis (6h, hanya hasil non-kosong)
- Memoize fetchUrlSafely utk type=text (LRU 30m + in-flight dedupe)
- makeImageCacheKey strip query CDN Discord (?ex/is/hm, format/width)
  -> attachment sama = satu key vision, skip re-download+re-vision

Spec: .hermes/plans/2026-08-24-ai-analysis-cache-optimization.md
Tests: +33 (cacheGuards, discordImageKeyNormalize, cacheBatchLookup)
2026-08-24 18:51:23 +07:00
asepharyana 440ec41da8 polish(frontend): perkuat bracket & marker segitiga game-menu (nudge anim) 2026-08-24 17:52:50 +07:00
asepharyana 1c8c0ca081 fix(frontend): normalisasi trailing slash pada isActivePath agar nav aktif terdeteksi 2026-08-24 17:44:42 +07:00
asepharyana 5f42c17caa feat(frontend): tema monokrom + sidebar animasi game-menu + gate ringan mobile 2026-08-24 17:35:51 +07:00
asepharyana eda5c752b7 revert(frontend): kembalikan shell usable — hapus total eksperimen constellation (three/d3-force dihapus) 2026-08-24 16:32:16 +07:00
asepharyana 25d5097edb fix(frontend): definite-height overlay chain so absolute panels anchor to viewport 2026-08-24 16:15:55 +07:00
asepharyana d3e3b4764a fix(frontend): let empty overlay areas click through to the constellation sky 2026-08-24 16:02:57 +07:00
asepharyana 33a557c761 fix(frontend): soften constellation glow for light theme 2026-08-24 15:48:15 +07:00
asepharyana 32de2819df feat(frontend): a11y constellation mirror — sr/keyboard-accessible node list 2026-08-24 15:47:19 +07:00
asepharyana a21d252e9b feat(frontend): eased camera fly-to on scene transitions (reduced-motion aware) 2026-08-24 15:46:08 +07:00
asepharyana 84a766db0c feat(frontend): voice stage-orbit, media queue-spiral & recordings timeline-ring scenes 2026-08-24 15:44:10 +07:00
asepharyana 0581dc3485 feat(frontend): messages orbital-belt & moderation verdict-hub scenes 2026-08-24 15:35:51 +07:00
asepharyana 3d6c07bd91 feat(frontend): glossary satellite ring + analysis search console scenes 2026-08-24 15:29:21 +07:00
asepharyana 60ae1fb5c3 feat(frontend): dashboard & channels constellation scenes — publish bridge + floating overlays 2026-08-24 15:22:27 +07:00
asepharyana 1fafebb16d refactor(frontend): remove classic dashboard chrome (topbar/navrail/mobilenav) 2026-08-24 15:07:40 +07:00
asepharyana a9e09c38e9 feat(frontend): constellation stage + floating chrome replace classic shell 2026-08-24 15:04:31 +07:00
asepharyana 3d4236e8df feat(frontend): constellation lib scaffold — graph/layout/camera pure modules (tested) 2026-08-24 14:46:38 +07:00
47 changed files with 2674 additions and 239 deletions

No files matched your search

+3 -3
View File
@@ -84,9 +84,9 @@ BACKLOG_SYNC_BATCH_SIZE=100 # Messages per backlog batch, max 100 (d
# === AI Analysis ===
AI_ANALYSIS_ENABLED=false # Enable AI content moderation (default: false)
# AI_LLM_API_KEY= # REQUIRED if AI_ANALYSIS_ENABLED=true. LLM API key
AI_LLM_BASE_URL=http://100.121.180.82:20128/api/v1 # LLM API base URL (omniroute on imrnes; /api/v1 exposes OpenAI-compatible chat+embeddings)
AI_LLM_MODEL=text # LLM text model name (default: text)
AI_LLM_API_KEY= # REQUIRED if AI_ANALYSIS_ENABLED=true. LLM API key
AI_LLM_BASE_URL=https://9router.asepharyana.my.id/v1 # LLM API base URL (9router — OpenAI-compatible router, replaces omniroute)
AI_LLM_MODEL=claude-opus-5 # LLM text model name (default: claude-opus-5)
# AI_LLM_VISION_MODEL= # Vision model for image analysis (falls back to AI_LLM_MODEL)
# AI_LLM_EMBEDDING_MODEL= # Embedding model for semantic moderation cache (optional; enables near-duplicate text reuse to save LLM calls)
# AI_LLM_EMBEDDING_MIN_SIMILARITY=0.97 # Min cosine similarity to reuse a cached verdict (default: 0.97)
@@ -0,0 +1,111 @@
# AI Analysis Flow — Audit & Optimization (discord-gateway)
**Goal:** Analisis alur AI analysis end-to-end, temukan bug/inconsistency yang merusak kualitas verdict, lalu perbaiki root cause-nya.
## Scope
- `services/discord-gateway/src/modules/ai-moderation/**`
- Tidak menyentuh chatbot backend / frontend.
## Alur saat ini (hasil tracing)
```
message capture → aiAnalyzer.queueMessageAnalysis(messageId)
→ batchScheduler.scheduleConversationAnalysis(conversationKey) [debounce 250ms, CB gate]
→ messageStore.getPendingMessagesByConversation(≤200)
→ skipAgeRestrictedMessages
→ pickBatchWithinBudget(14000 tokens, 50/msg)
→ processBatch [Piscina worker, ≤4 threads]
→ ai-analysis-worker.processBatch
→ getConversationContextBefore(20 msgs) + attachments
→ attachment-upload race guard (pending upload → skip)
→ runModerationAnalysis
→ Phase 1: exact-hash cache (PG text_analysis_cache, per channel/thread)
→ Phase 2: semantic cache (embedTexts → Qdrant batch search; PG fallback)
→ split text-only vs media
→ runTextOnlyBatch: URL fetch + wiki search + glossary (paralel)
→ dedup short messages → sub-batches (60/sub-batch)
→ vision evidence utk URL images (hoisted, 15s cap per image)
→ callModerationLLM per sub-batch (stream:true, retries 3, JSON parse + correction retry)
→ runMediaBatch: download → vision per image (cache LRU→DB→live, lock) → 1 LLM call
→ setCachedTextModeration (PG + Qdrant upsert w/ embedding)
→ normalizeResult (confidence clamp, fallback analysis)
→ updateMessagesAIAnalysisBulk → broadcast + scheduleAutoDelete
→ recovery worker tiap 10s: pending keys → re-schedule; incomplete → individual fallback queue
→ individual fallback: 1 msg = 1 worker job (context + full LLM)
→ cache prune tiap 6 jam (PG expired + Qdrant expired points)
```
## Temuan audit (ranked)
### F1 — Cache hit menghapus status "warn" (BUG AKURASI)
`moderationOrchestrator.ts` Phase-2 semantic hit & PG-fallback memetakan status via
`parseQdrantVerdict`: storedStatus bukan "warn"/"flagged" → dipaksa "clean".
TAPI exact-hash lookup (`getCachedTextModeration`, textCacheStore.ts:288-295) lebih parah:
hanya menerima "clean"|"flagged" — **"warn" jatuh ke branch flags.length===0 ? clean : flagged**
→ warn dengan flags=["conflict_instigation"] dibaca sebagai FLAGGED.
Efek: auto-delete eligibility (butuh recommendedAction delete/escalate + severity list) salah baca;
dashboard menampilkan flagged padahal verdict asli warn. Root cause: type narrowing legacy
(`status: "clean" | "flagged"`) tidak diupdate ketika "warn" ditambahkan ke schema.
### F2 — Exact-cache key mengabaikan edit (BUG EVASION)
Key = sha256(content)+context. Pesan yang DIEDIT (`edited_content`) menghasilkan hash berbeda,
tapi verdict lama utk konten pre-edit tetap hidup; lebih penting: pesan edited="true" adalah sinyal
evasion di prompt, sedangkan cache bisa menyajikan verdict dari konten lama jika content sama.
(Minor, tapi konsistensi: `resolveIsEdited` ada di prompt, tidak ada di cache key.)
### F3 — `pickBatchWithinBudget` skip-bukan-break (LATENSI/KUALITAS)
Loop `if (usedTokens + msgTokens <= maxTokens) {push}` — pesan BESAR di tengah list dilewati
dan iterasi lanjut mencoba msg berikutnya. Efek: batch berisi "lubang" (msg pending tetap pending,
dianalisis di gelombang berikutnya = LLM call tambahan). Ini by-design tolerable, tapi ada bug halus:
pesan >budget tunggal tidak pernah masuk (scheduler sudah punya fallback slice(0,1), OK).
Keputusan: biarkan (bukan bug nyata), catat saja.
### F4 — `callModerationLLM` max_tokens 16384 hardcoded (COST)
Sub-batch 60 pesan × output ~150 token/pesan ≈ 9k token cukup; 16k aman. Biarkan.
### F5 — Dead code builder user-profile/reputation
`buildUserProfilesBlock`, `buildUserProfileRef`, `UserProfileEntry` di moderationBuilders.ts
tidak dipakai lagi sejak context minimization (hanya tests). `<user_history>` juga tak pernah
di-inject (rules masih menyebutnya — misleading bagi model). Bersihkan referensi prompt.
### F6 — rules.ts menyebut `<user_history>` yang tidak pernah ada di payload
Model diberi instruksi tentang blok yang tak pernah muncul → pemborosan token + potensi
kelakuan aneh ("menunggu" data yang tak ada). Hapus/ubah kalimat.
### F7 — system.ts "Blok Data" menyebut `<term_glossary> (SearXNG)` — STALE
Sumber sudah Wikipedia. Komentar kode & teks prompt menyebut SearXNG. Perbaiki teks (kecil).
### F8 — output.ts typo "secifik", baris tabel `-|-` rusak
Kualitas prompt: typo + markdown table broken (`||-`) di beberapa baris. Rapikan.
### F9 — llmCaller parse-error correction tail hanya di SYSTEM
Correction tail ditambahkan ke system prompt; provider caching fine, tapi preview invalid
content (800 char) ikut SYSTEM — ok. Skip.
### F10 — `getLlmSemaphore` race kecil saat config berubah di tengah flight
Non-issue praktis (config statis per proses). Skip.
## Keputusan perbaikan (yang dieksekusi sekarang)
1. **F1 (utama):** normalisasi status di SATU tempat — `normalizeStoredStatus()` di
textCacheStore.ts yang menerima clean/warn/flagged; pakai di getCachedTextModeration
DAN parseQdrantVerdict; perluas return types ke union penuh. Orchestrator tinggal pakai.
2. **F6+F7+F8:** bersihkan stale references di prompts (user_history, SearXNG, typo).
3. **F5:** hapus dead builders + test-nya (biome/tsc yang jaga).
4. Regression test untuk F1 (vitest): warn tersimpan → warn terbaca (exact + qdrant path).
## Files touched
- services/discord-gateway/src/modules/ai-moderation/textCacheStore.ts (F1)
- services/discord-gateway/src/modules/ai-moderation/moderationOrchestrator.ts (type only)
- services/discord-gateway/src/modules/ai-moderation/prompts/rules.ts (F6)
- services/discord-gateway/src/modules/ai-moderation/prompts/system.ts (F7)
- services/discord-gateway/src/modules/ai-moderation/prompts/output.ts (F8)
- services/discord-gateway/src/modules/ai-moderation/moderationBuilders.ts (F5)
- services/discord-gateway/tests/contextEnrichment.test.ts (F5 test cleanup + F1 regression test baru)
## Verification
```
cd services/discord-gateway
npx tsc --noEmit
npx biome check --diagnostic-level=error .
npx vitest run
```
Semua harus hijau sebelum commit. Deploy via GHA (push main) — user konfirmasi belakangan.
@@ -0,0 +1,127 @@
# Spec: Optimasi AI Analysis GMW — Naikkan Cache Hit Tanpa Kehilangan Akurasi
Tanggal: 2026-08-24 · Repo: `~/GMW` (branch `main`) · Service: `services/discord-gateway`
## Latar & Evidence (audit 2026-08-24)
State produksi:
- Qdrant `gmw_text_moderation`: **1.550 poin, status green** (vectors size 2048, Cosine).
- PG `text_analysis_cache`: 1.634 row `user_moderation`, 277 `vision_llm`; **sum(hit_count) = 0** →
hit-rate tidak pernah terukur.
- Embedding aktif (`AI_LLM_EMBEDDING_MODEL` set, Nemotron-embed, dim 2048), `AI_LLM_EMBEDDING_MIN_SIMILARITY`
tidak diset di BWS → default **0.97** (sangat konservatif).
- Messages: 9.375 total; 643 status `error` (banyak retry), 49 pending.
Temuan audit alur (`moderationOrchestrator.ts` → `textCacheStore.ts` → `qdrantClient.ts`,
`textBatchProcessor.ts`, `urlFetcher.ts`, `wikipediaClient.ts`, `visionAnalyzer.ts`):
| # | Temuan | Dampak |
|---|--------|--------|
| F1 | Exact-hash cache key menyertakan context (channel/thread) → teks sama di channel lain selalu miss | Killer hit-rate #1 |
| F2 | Semantic tier TIDAK memfilter context (Qdrant payload tak punya context) — sudah global tapi hanya aman krn sim 0.97 ketat | Inkonsisten dgn exact tier |
| F3 | Phase-1 lookup loop `await getCachedTextModeration(key)` per pesan → N round-trip PgBouncer per batch (60 msg = 60 query serial) | Latensi + beban DB |
| F4 | Verdict actionable (flagged/warn) dan clean sama-sama boleh di-serve semantic; toleransi akurasi beda | Risiko akurasi |
| F5 | `hit_count` tidak pernah di-increment oleh reader manapun | Hit-rate tak terukur |
| F6 | `wikipediaSearch()` (blok `<web_searches>`) tanpa cache — re-fetch tiap batch utk query sama | Latensi + spam ke WP |
| F7 | `fetchUrlSafely()` tanpa cache — link sama di batch berikutnya di-download lagi penuh | Latensi + bandwidth |
| F8 | Vision cache key dari data-URL base64 hasil resize → attachment sama via jalur berbeda (URL vs embed) = key beda → re-download + re-vision | Duplikasi kerja vision |
Non-goals: mengubah pipeline enforcement (auto-mute/ban trust-store writes), mengubah prompt
kebijakan moderasi, mengubah model/embedding provider.
## Desain
Semua perubahan degrade gracefully — cache gagal → perilaku lama (LLM). Akurasi dilindungi
asimetris: **hemat boleh untuk verdict non-actionable, konservatif untuk yang memicu aksi.**
### D1 — Cache metrics (F5)
- `textCacheStore.getCachedTextModeration()`: saat hit valid, increment `hit_count`
(`UPDATE ... SET hit_count = hit_count + 1`) fire-and-forget (`.catch(()=>{})`), jangan blokir return.
- Log info periodik ringkas di orchestrator sudah ada ("User moderation cache applied") — cukup.
### D2 — Batched exact-cache lookup (F3)
- Fungsi baru `getCachedTextModerations(keys: string[]): Promise<Map<string, StoredModerationVerdict>>`
di `textCacheStore.ts`: **satu** `SELECT ... WHERE text = ANY($1)` (chunk 200 key/query),
parse + `normalizeStoredStatus` per row (reuse helper existing).
- Orchestrator fase-1: kumpulkan semua key unik → satu call batched → distribusi hasil.
- Semantik identik dengan loop lama (row expired/error-artifact tetap miss); hanya jumlah round-trip
yang turun N→1.
### D3 — Global exact reuse untuk verdict non-actionable (F1)
- Key scoped-context TETAP ditulis (kompatibel, invalidasi moderator tetap presisi).
- Reader tambahan: kalau key `<ctx>:<hash>` miss, coba key legacy global `text_mod:<hash>` (bare).
- Guard akurasi (WAJIB semua terpenuhi):
- `status === "clean"` DAN `flags.length === 0`;
- `confidence >= AI_CACHE_GLOBAL_REUSE_MIN_CONFIDENCE` (default 0.85);
- `recommendedAction === "none"`;
- umur entry ≤ `AI_CACHE_GLOBAL_REUSE_MAX_AGE_H` (default 72h) — cek `analyzed_at`.
- Flag baru `policyVersion: "cached-global-clean-2026-08"` supaya terlacak di dashboard/log.
- Verdict flagged/warn TETAP context-scoped (tidak pernah lintas channel).
### D4 — Semantic dua-band similarity (F2+F4)
- Config baru: `AI_LLM_EMBEDDING_MIN_SIMILARITY_ACTIONABLE` default **0.97** (perilaku lama),
`AI_LLM_EMBEDDING_MIN_SIMILARITY_CLEAN` default **0.92**, keduanya coerce number 0..1.
- Satu Qdrant batch search pakai threshold RENDAH (0.92). Per hit, klasifikasi ulang:
- verdict non-actionable (clean, no flags, action=none): terima jika `score >= CLEAN_BAND`;
- verdict actionable (warn/flagged atau flags ada / action != none): terima hanya jika
`score >= ACTIONABLE_BAND` (0.97 — persis gate lama);
- di antara dua band → buang hit, pesan lanjut ke LLM (fail-open ke akurasi).
- Legacy PG fallback path: filter serupa di `findSimilarTextModeration` via parameter band.
### D5 — Cache Wikipedia search (F6)
- `wikipediaClient.wikipediaSearch(query)`: cek `cacheGet(makeCacheKey("wikisearch", q))` dulu;
miss → fetch (timeout existing) → sukses & hasil non-kosong → `cacheSet(..., TTL 6h)`.
Hasil kosong TIDAK di-cache (biar retry nanti). Redis down → langsung fetch (no-op cache).
### D6 — Cache URL text fetch (F7)
- `urlFetcher.fetchUrlSafely(url)`: wrapper async memoize in-process LRU (max 500, TTL 30 menit)
untuk `type === "text"` saja (image tetap selalu fresh-download karena dipakai sbg bukti vision
+ buffer besar; error tidak di-cache).
- Import `LRUCache` dari `lru-cache` (sudah dep gateway).
### D7 — Unified vision cache key (F8)
- `makeImageCacheKey(imageUrl)` di `textCacheStore.ts`: sebelum hash, strip query Discord CDN
(`?ex=&is=&hm=` signed tokens, `format/width/height/size`) — regex `(\?[^#]*)$` dibuang bila host
CDN discord (`cdn.discordapp.com`, `media.discordapp.net`, `images-ext-*.discordapp.net`);
URL non-Discord: hash full URL seperti sekarang.
- Efek: attachment sama yang lolos lewat jalur embed vs inline vs re-fetch dgn token beda → SATU
entry cache → skip download+vision kedua kali. Data-URL base64 tetap di-hash apa adanya.
## File yang disentuh
1. `src/shared/config/index.ts` — 3 config baru (D3×2, D4×2 — total 4 nilai, 3 baris zod + deskripsi).
2. `src/modules/ai-moderation/textCacheStore.ts` — hit_count inc (D1), batched getter (D2),
global-reuse guard helper (D3), image-key normalize (D7).
3. `src/modules/ai-moderation/moderationOrchestrator.ts` — pakai batched getter (D2),
global bare-key fallback (D3), dua-band semantic accept (D4).
4. `src/modules/ai-moderation/qdrantClient.ts` — `searchQdrantBatch` menerima threshold rendah
(sudah parametrik — mungkin tanpa perubahan; verifikasi).
5. `src/modules/ai-moderation/wikipediaClient.ts` — cache layer (D5).
6. `src/modules/ai-moderation/urlFetcher.ts` — LRU text-fetch memoize (D6).
## Schema/type changes
- Tidak ada migrasi DB (kolom `hit_count`, `analyzed_at`, `expires_at` sudah ada).
- Tidak ada perubahan kontrak WS/oRPC/frontend.
- Type baru: none public; internal `StoredModerationVerdict` dipakai ulang.
## Verification
1. Unit tests baru (`tests/`):
- `cacheBatchLookup.test.ts`: batched getter — hit/miss/expired/error-artifact mapping,
chunking >200 keys (mock executeAll), hit_count increment called.
- `globalReuseGuard.test.ts`: guard menerima clean+conf≥0.85+action none+umur ≤72h;
menolak flagged/warn/conf rendah/action≠none/stale.
- `semanticBands.test.ts`: clean @0.93 diterima, flagged @0.93 ditolak, flagged @0.98 diterima.
- `imageKeyNormalize.test.ts`: URL Discord dgn/ex token → key sama; non-Discord beda query → beda.
2. Gate service: `pnpm typecheck && pnpm exec biome check --diagnostic-level=error . && pnpm exec vitest run`.
3. Deploy via GHA (`git push origin main`) → watch `Build & Deploy (Nix)` → verifikasi
`systemctl show gmw-discord-gateway -p ActiveEnterTimestamp` baru.
4. Runtime probe pasca-deploy: journalctl level 30 normal; beberapa jam kemudian
`SELECT sum(hit_count) FROM text_analysis_cache WHERE source='user_moderation'` > 0 membuktikan
metrics jalan; log "User moderation cache applied" menunjukkan hits>0 pada traffic ramai.
## Rollback
Semua fitur behind config defaults yang mempertahankan perilaku lama pada nilai konservatif;
rollback = redeploy commit sebelumnya (tanpa migrasi DB, tanpa state eksternal).
@@ -0,0 +1,56 @@
# Spec: Perbaiki Delay Attachment 162s→<20s (GMW AI Analysis)
Tanggal: 2026-08-24 · Repo `~/GMW` · Service discord-gateway
## Evidence (audit produksi)
Klaster pesan attachment delay ~330–400 detik. Trace pesan `1541417073245290638` (.gif):
19:01:08 dibuat → 19:01:09 batch incomplete → fan-out individual → **guard upload-pending
mengembalikan `results:[]`** → diperalakukan sukses (`complete ... (undefined)`) → row
tertahan `ai_status='processing'` **tanpa penanggung jawab** → 19:06:12 cleanup mengembalikan
ke `pending` (tepat 300s) → baru dianalisis. Plus vision gagal 3× utk GIF besar
("Stream ended before producing a non-ping SSE event") → degradasi teks.
## Root causes
- **A (fatal)**: `individualFallbackProcessor.processIndividualFallback` memperlakukan
`ok:true + results:[]` sebagai sukses. Race-guard upload di `ai-analysis-worker.processIndividual`
sengaja balik `results:[]` (desain lama) → pesan yatim `processing` sampai cleanup 300s.
- **B**: `llmVision` hanya mencoba `stream:true`; kegagalan SSE truncation pada gambar besar
= 3 retry sia-sia (semua jalur sama) → bukti media hilang.
- **C**: safety-net cleanup 300s terlalu lambat sbg satu-satunya pemulih `processing`.
## Fix
1. **F1 — sinyal eksplisit upload-pending**: `IndividualOkResponse` + field opsional
`uploadPending?: boolean`. Worker set `uploadPending:true` saat race guard kena.
2. **F2 — processor menangani 3 kondisi** via helper murni baru
`classifyIndividualWorkerResult(result): "success" | "upload_pending" | "incomplete" | "error"`
(modul baru `fallbackResultClassifier.ts`, zero-dep agar mudah dites):
- `upload_pending` → tulis ulang row ke `pending` (pola sama dgn revert apiFailed di
batchProcessor) + broadcast + **re-schedule analisis percakapan segera**
(dynamic import batchScheduler, pola anti-siklus yg sudah ada) → retry dalam ~250ms
begitu upload beres. Bukan error, tidak naikkan CB counter.
- `incomplete` (flags analysis_incomplete) → perilaku lama (exhausted path).
- `error` / `results kosong tanpa penjelasan` → throw transien (retry oleh recovery),
BUKAN sukses palsu. Log "(undefined)" hilang.
3. **F3 — vision non-stream fallback**: di `llmVision`, jika error match
`/Stream ended before producing a non-ping SSE|stream ended/i` → coba SEKALI lagi dengan
`stream:false` (router agregasi penuh; timeout tetap 60s). Konversi hard-fail jadi sukses.
4. **F4 — turunkan safety net**: default `revertStuckProcessingMessages` 300000 → 120000 ms.
## File disentuh
- `src/modules/ai-moderation/fallbackResultClassifier.ts` (BARU, pure)
- `src/modules/ai-moderation/ai-analysis-worker.ts` (tipe + set flag uploadPending)
- `src/modules/ai-moderation/individualFallbackProcessor.ts` (konsumsi classifier + reschedule)
- `src/modules/ai-moderation/llmClient.ts` (fallback non-stream di llmVision)
- `src/modules/message-capture/messagesCleanup.ts` (default 120s)
## Verifikasi
- Test baru `tests/fallbackResultClassifier.test.ts` (4 klasifikasi + edge kosong).
- Gate: tsc --noEmit, biome error-level, vitest run semua hijau.
- Deploy GHA sukses; pasca-deploy: pesan attachment baru p50 < 20s
(`SELECT percentile_cont(0.5) ... WHERE metadata attachments>0 AND created_at > deploy`),
tidak ada lagi "complete ... (undefined)".
@@ -0,0 +1,205 @@
# GMW FE Rombak — "Constellation Ops" Implementation Plan
> **For Hermes:** Use subagent-driven-development skill to implement this plan task-by-task.
> Setiap task = edit → lint → build/test → smoke → commit (author `asepharyana`, tanpa Co-Authored-By).
**Goal:** Rombak total FE GMW dari template dashboard klasik menjadi **Constellation Node Graph** — graf node interaktif (three.js force layout) di mana channel/event/moderasi adalah node bintang dan navigasi adalah terbang antar node — untuk semua 9 route sekaligus.
**Architecture:** Hapus total shell template (TopBar + NavRail + MobileNav + main scroll container). Ganti dengan satu `ConstellationStage`: canvas full-bleed fixed di belakang seluruh app yang me-render force-directed graph per-route (scene), plus overlay HTML mengambang (tanpa card box standar) untuk konten detail. Router tetap Next App Router; perpindahan route = kamera "fly-to" ke scene baru. Data tetap lewat pola SSR seed (`page.tsx` server fetch → `view.tsx` client + SWR fallbackData) Line truncated
**Tech Stack:** Next.js 16 App Router · React 19 · TypeScript strict · Tailwind v4 (token oklch existing) · three r180 (+ `@types/three`) · d3-force (+ `@types/d3-force`) untuk simulasi layout deterministik · motion v12 (transisi antar-scene) · biome (0 error **dan** 0 warning) · bun:test untuk modul murni.
---
## Keputusan Terkunci (2026-08-24, user MythEclipse)
| Keputusan | Pilihan user |
|---|---|
| Cakupan | **Semua 9 route sekali jalan** (dashboard, voice, media, recordings, moderation, messages, analysis, channels, glossary) |
| Arah struktural | **Constellation Node Graph** — navigasi = terbang antar node |
| Tema | **Pertahankan toggle light/dark** (jangan rebuild sistem tema — harden saja) |
## Kondisi Saat Ini (fakta repo, 2026-08-24)
- Repo: `/home/code/GMW/services/frontend` (monorepo GMW, deploy Nix-first via GHA "Build & Deploy (Nix)" ke `imphnen.asepharyana.my.id`; nginx gmw-proxy :4009 → Next standalone :4017).
- Shell aktif: `src/components/shell/ambient-app.tsx` → `AppFrame` = NavRail + TopBar + `<main>` scroll + MobileNav + MiniPlayer, dipakai SEMUA route via `(dashboard)/layout.tsx`. AmbientCanvas (233 l) masih ada sebagai background di belakang chrome template.
- 9 route di `src/app/(dashboard)/`: dashboard (368 l), voice (311), media (293), moderation (424), messages (611), analysis (196), recordings (201), channels (23), glossary (15).
- `next.config.ts`: `output: "standalone"` + `trailingSlash: true` → **nav via `router.push`/Link bisa no-op; pakai `<a href>` polos** (pitfall terverifikasi).
- Token desain ada di `globals.css` `@theme`: `--color-canvas/-2`, `--color-ink/-soft/-faint`, `--color-signal(+glow)`, amber, vermilion, hairline; font Bricolage Grotesque/JetBrains Mono; `.light` class override + next-themes sudah jalan — JANGAN dibangun ulang.
- Sudah ada & dipertahankan: CommandPalette (⌘K), Chatbot, MiniPlayer, primitives (GlassPanel/GlassCard/Badge/…), shared states+skeletons, `lib/format.ts`, `lib/ai-status.ts`, WS typed events (`message_created/updated/deleted/analyzed/snapshot*`, `voice_state`, dll).
- Tooling lokal: `pnpm lint|build|format`, `pnpm exec biome check --write --unsafe .` buat import order; bun 1.3.14 tersedia untuk unit test modul murni.
- Smoke test SELALU di port non-prod (mis. 4024), cek `ss -ltnp | grep <port>` dulu; JANGAN pernah bind 4017.
## Struktur Target
```
src/components/shell/
constellation-frame.tsx # pengganti AppFrame: tanpa topbar/rail/bottom-bar
constellation-stage.tsx # canvas three.js full-bleed + hit-test + kamera
route-scenes.ts # map pathname → scene config (nodes/edges/focus)
floating-chrome.tsx # brand mark + status dot + route switcher minim + palette hint
(hapus setelah migrasi): topbar.tsx, nav-rail.tsx, mobile-nav.tsx, ambient-app.tsx*
src/lib/constellation/
graph.ts # tipe GraphNode/GraphEdge + builder dari data API (murni)
layout.ts # wrapper d3-force deterministik (seeded) + fallback radial (murni)
camera.ts # interpolasi kamera fly-to (murni, testable)
*.test.ts # bun:test utk modul murni di atas
```
*ambient-canvas/context dievaluasi: bisa di-refactor jadi bagian dari stage atau dihapus.
### Scene per route (bahasa visual satu kesatuan)
| Route | Scene |
|---|---|
| `/dashboard/` | Guild = bintang pusat; channel = node orbit; event WS = pulse edge; metric cluster floating (tanpa box) |
| `/channels/` | Semua channel sebagai konstelasi; klik channel = fly-to + panel detail mengambang |
| `/moderation/` | Pesan flagged = node merah (vermilion) mengelilingi hub AI-verdict; LiveModerationFeed jadi ribbon ticker |
| `/messages/` | Stream pesan = sabuk orbital; node baru masuk lewat animasi dari tepi |
| `/voice/` | Stage node di tengah; speaker aktif = node mengorbit dengan glow intensitas |
| `/media/` | Galeri spiral node (thumbnail via div bg-image); klik = MiniPlayer handoff |
| `/recordings/` | Variasi spiral media, timeline density sebagai cincin |
| `/analysis/` | Hub analitik; chart existing dirender sebagai panel translusen mengambang ter-anchor ke node |
| `/glossary/` | Term KB = cincin satelit; search memfilter node real-time |
### Checklist anti-default (wajib lolos sebelum commit akhir)
- [ ] Tidak ada top bar / nav rail / side panel / bottom prompt bar
- [ ] Tidak ada card grid standar sebagai struktur utama
- [ ] Layout beda SPASIAL dari versi lama (bukan ganti warna)
- [ ] Verifikasi visual sungguhan: browser_navigate + browser_vision → "bukan template dashboard"
---
## Tasks
### Phase 0 — Fondasi
#### Task 1: Dependencies + scaffold direktori
**Files:** `package.json`, create `src/lib/constellation/`
1. `cd /home/code/GMW/services/frontend && pnpm add three @types/three d3-force @types/d3-force && pnpm add -d @types/bun`
2. Buat 4 file kosong bertipe sesuai struktur target (ekspor placeholder bertipe agar tsc lolos).
3. Run: `pnpm exec tsc --noEmit` → expected PASS.
4. Commit: `chore(frontend): scaffold constellation lib + deps`.
> Catatan: jika `@types/bun` bentrok dengan tsconfig Next (duplikat globals), fallback: exclude `"**/*.test.ts"` dari `tsconfig.json` dan jalankan test hanya via `bun test` (tanpa typecheck). Putuskan saat eksekusi, dokumentasikan di commit body.
#### Task 2 (TDD): Modul murni graph + layout + camera
**Files:** `src/lib/constellation/{graph,layout,camera}.ts` + `*.test.ts`
1. **Tulis test dulu** (bun:test):
- `graph.test.ts`: builder `dashboardToGraph()` menghasilkan node guild pusat + N channel + edges benar dari fixture `/api/dashboard/stats|channels` shape (lihat AGENTS.md; nama channel di `message.metadata.channelName`).
- `layout.test.ts`: `computeLayout(nodes, edges, {seed:42})` DETERMINISTIK (dua pemanggilan = posisi identik), semua posisi finite, radius dalam bound; `radialLayout()` fallback untuk reduced-motion.
- `camera.test.ts`: `flyTo(from,to,t)` easing monotonic, t=0→from, t=1→to.
2. Run: `bun test src/lib/constellation` → expected FAIL (belum diimplementasi).
3. Implementasi minimal sampai PASS.
4. Run: `bun test src/lib/constellation && pnpm exec tsc --noEmit` → PASS.
5. Commit: `feat(frontend): constellation graph/layout/camera pure modules (tested)`.
### Phase 1 — Shell Baru
#### Task 3: `ConstellationStage` (canvas renderer)
**Files:** `src/components/shell/constellation-stage.tsx`
- `<canvas>` fixed inset-0 `-z-10`; three.js orthographic 2D-ish render node+edge (sprite/glow shader sederhana — jangan over-engineer; titik + garis + glow cukup).
- Interaksi: drag = pan, wheel/pinch = zoom, hover = highlight node + tooltip title, click node ber-`href` = navigasi via `<a>` semantics (window.location assign — hindari router.push no-op).
- DPR-aware resize; pause render loop saat `document.hidden`; `prefers-reduced-motion` → render statis tanpa animasi.
- Sampling warna dari CSS var (`getComputedStyle`) + `MutationObserver` pada `documentElement.classList` agar ikut toggle dark/light.
- Verify: `pnpm lint && pnpm build` PASS → commit `feat(frontend): constellation stage canvas renderer`.
#### Task 4: `route-scenes.ts` + `ConstellationFrame`
**Files:** `src/components/shell/{route-scenes,constellation-frame,floating-chrome}.tsx`
- `route-scenes.ts`: `usePathname()` → config scene (builder mana, focus node, overlay slots).
- `ConstellationFrame`: render `ConstellationStage` + `children` (overlay HTML absolute, BUKAN flex column bertingkat) + `floating-chrome` (brand mark kiri-atas dgn status dot WS, switcher route minimal kanan-bawah, hint ⌘K). MiniPlayer + Chatbot + CommandPalette tetap termounting.
- Switcher & semua nav internal pakai `<a href="/<route>/">` polos (trailingSlash!). Verifikasi klik di browser nanti, bukan cuma baca kode.
- Wire `(dashboard)/layout.tsx`: ganti `AppFrame` → `ConstellationFrame` (provider Ambient/WS tetap).
- Smoke: `ss -ltnp | grep 4024` kosong → `PORT=4024 pnpm start` → curl semua 9 route = 200 → kill server.
- Commit: `feat(frontend): replace classic shell with constellation frame`.
#### Task 5: Hapus chrome lama + bersih-bersih
**Files:** hapus `shell/topbar.tsx`, `shell/nav-rail.tsx`, `shell/mobile-nav.tsx`, update `shell/index.ts`; eval `ambient-*` (refactor jadi layer stage ATAU hapus).
- `grep -rn "TopBar\|NavRail\|MobileNav\|AppFrame" src/` → 0 referensi tersisa.
- `pnpm exec biome check --write --unsafe .` → import order rapi.
- Commit: `refactor(frontend): remove classic dashboard chrome`.
### Phase 2 — Migrasi 9 Route (batch, tiap batch ship hijau)
Urutan sengaja dari yang paling graph-native ke paling padat konten:
#### Task 6: `/dashboard/` + `/channels/`
- Dashboard view: guild-star + channel orbit + WS pulse; metric cluster floating (reuse MetricTile tapi tanpa card-box — ubah jadi teks besar + hairline).
- Channels: konstelasi channel, klik = fly-to + panel detail (data dari hook `use-channels`/server fetch existing — jangan karang endpoint).
- Lint+build+smoke 2 route → commit `feat(frontend): dashboard & channels constellation scenes`.
#### Task 7: `/glossary/` + `/analysis/`
- Glossary: cincin satelit term + search filter node (client-side filter pada node label).
- Analysis: hub node + panel chart translusen mengambang (reuse komponen charts/* existing, anchor posisi ke node).
- Commit: `feat(frontend): glossary & analysis scenes`.
#### Task 8: `/messages/` + `/moderation/`
- Messages: sabuk orbital stream (reuse sortMessages/logika WS `use-messages.ts` — jangan duplikasi); pesan baru animate-in dari tepi.
- Moderation: flagged nodes vermilion + hub verdict; LiveModerationFeed jadi ticker ribbon bawah (bukan panel).
- Commit: `feat(frontend): messages & moderation scenes`.
#### Task 9: `/voice/` + `/media/` + `/recordings/`
- Voice: speaker aktif orbit dgn glow (seed dari snapshot `activeSpeakers` server-authoritative — pertahankan perilaku useSpeakers).
- Media/Recordings: spiral galeri; thumbnail pakai div background-image (hindari `noImgElement`; kalau terpaksa `<img>` → `// biome-ignore lint/performance/noImgElement: <alasan>`); MiniPlayer handoff tidak boleh regress.
- Commit: `feat(frontend): voice, media & recordings scenes`.
### Phase 3 — Polish, A11y, Tema
#### Task 10: Motion & micro-interaction
- Fly-to antar scene pakai `motion` v12 (interpolasi kamera dari `lib/constellation/camera.ts`).
- Overlay: stagger masuk (pattern `animate-stagger` + `staggerDelay(i)` existing); tambah semua `animate-*` baru ke kill-list `prefers-reduced-motion` di `globals.css`.
- Commit: `feat(frontend): scene transitions + motion polish`.
#### Task 11: A11y & fallback semantik
- Setiap scene punya fallback semantik: `<nav aria-label="Routes">` visually-hidden berisi link semua route + `<ul>` node (label + href) yang ter-sync dgn graph state.
- Keyboard: Tab ke daftar route; panah memindahkan fokus antar node; Enter aktivasi; `focus-visible` ring pakai `--color-ring`.
- Kontras ink-on-canvas dicek (WCAG AA utk teks overlay).
- Hooks-after-early-return pitfall: compute flags dulu, guard di dalam useEffect.
- Commit: `feat(frontend): a11y semantic fallback for constellation`.
#### Task 12: Hardening tema light/dark
- Pastikan canvas sampling var `.light` benar saat toggle (MutationObserver dari Task 3); harden `.light`: `color-scheme: light`, scrollbar/selection, glow opacity turun.
- JANGAN menyentuh sistem next-themes.
- Commit: `fix(frontend): light theme parity for constellation stage`.
### Phase 4 — Gerbang Ship (non-negotiable)
#### Task 13: Verifikasi penuh + deploy
1. `pnpm lint` → **Found 0 errors. Found 0 warnings** (warning pun harus bersih).
2. `bun test src/lib/constellation` → all PASS; `pnpm exec tsc --noEmit` PASS; `pnpm build` sukses.
3. Smoke: port 4024 bersih → start → curl 9 route (dgn trailing slash) semua **200** → kill server.
4. `git status` audit: hanya file scope yang ter-commit (`git checkout --` file liar hasil auto-format).
5. Push `origin/main` → `gh run watch` "Build & Deploy (Nix)" → hijau.
6. Live: curl `https://imphnen.asepharyana.my.id/<route>/` × 9 → 200.
7. **Verifikasi visual sungguhan**: browser_navigate ke live URL → browser_vision: "Apakah ini masih template dashboard?" — jawaban HARUS bukan; screenshot dikirim ke user (`MEDIA:<path>`).
8. Checklist anti-default dicentang satu per satu di PR/commit message akhir.
---
## Files Likely to Change
```
package.json, pnpm-lock.yaml
next.config.ts # hanya jika perlu (tidak diharapkan)
src/app/(dashboard)/layout.tsx # wire frame baru
src/app/(dashboard)/{9 route}/page.tsx|view.tsx # scene migration
src/components/shell/** # frame baru + hapus lama
src/lib/constellation/** # baru (murni + test)
src/app/globals.css # tokens tambahan + reduced-motion kill list
biome.json # hanya jika rule intentional baru perlu "off"
```
## Risks / Tradeoffs
| Risiko | Mitigasi |
|---|---|
| Perf GPU di device lemah (full-page WebGL) | Node count kecil (<200/route), pause saat tab hidden, fallback radial statis utk reduced-motion; jika tetap berat → downgrade ke Canvas2D renderer dgn API sama |
| router.push no-op (standalone+trailingSlash) | Semua nav pakai `<a href>`; verifikasi klik nyata di browser |
| Konten padat (messages 611 l) tak muat di metafora graf | Prinsip: graf = struktur/navigasi; DETAIL tetap lewat overlay panel mengambang — konten tidak dikorbankan, hanya wadahnya yang berubah |
| Bun test + Next tsconfig konflik types | Exclude `*.test.ts` dari tsconfig (keputusan Task 1) |
| Regress fitur existing (theme toggle, palette, chatbot, mini-player, WS sync) | Dipertahankan apa adanya; smoke + klik verifikasi per phase; sortMessages & snapshot voice TIDAK disentuh logikanya |
## Open Questions
- Tidak ada blocker. Satu preferensi opsional: apakah brand mark perlu jam/uptime seperti TopBar lama — default: tidak (minimal). Keputusan bisa diambil saat review Task 4.
@@ -0,0 +1,39 @@
# GMW FE — Monokrom Hitam-Putih + Sidebar Ala Menu Game + Ringan di Mobile
Tanggal: 2026-08-24 · Basis: `eda5c75` (shell usable hasil revert)
## Tujuan
1. Tema **monokrom murni** (hitam-putih, tanpa warna) di dark & light.
2. Sidebar (desktop NavRail + mobile dock) beranimasi **ala menu game** — corner
brackets, sweep, stagger masuk, marker segitiga.
3. **Ringan di mobile**: matikan WebGL ambient di layar kecil, kurangi biaya
blur/backdrop, animasi transform/opacity saja.
## Non-goals
- Tidak menyentuh backend, endpoint, hooks/data-flow, struktur route.
- Tidak menambah dependensi baru (CSS murni untuk semua animasi).
## File yang disentuh
| File | Perubahan |
|---|---|
| `src/app/globals.css` | Token mono (dark+light): signal/amber/vermilion → skala putih-abu; `.glass` blur adaptif; kelas baru `.game-nav-item` (bracket ::before/::after, sweep, stagger via `--i`), `.game-frame` (panel sudut terpotong + garis tergambar), keyframes `sweep-x`, `draw-line`, `nav-in`; media query `<md`: blur 18→8px, hambat animasi berat |
| `src/components/shell/nav-rail.tsx` | Item pakai `.game-nav-item` + `style={{'--i': n}}`; marker aktif jadi segitiga ▸ putih; hapus box-shadow glow besar (ganti sweep) |
| `src/components/shell/mobile-nav.tsx` | Dock mono: tab aktif = bar atas putih + sweep sekali; target sentuh ≥44px; hapus glow blob |
| `src/components/shell/topbar.tsx` | Aksen mono + `.game-frame` pada container (cek markup dulu) |
| `src/components/ambient/ambient-canvas.tsx` | Early-return WebGL bila `(pointer: coarse)` / lebar <768 / `saveData` / core ≤4; fallback statik CSS tetap |
| `src/components/ambient/status/signal tone` (`SIGNAL_RGB`) | Semua tone jadi grayscale (putih; intensitas beda per tone) |
| `src/app/(dashboard)/dashboard/view.tsx` | Hero + kartu metrik pakai `.game-frame`/cut-corner sebagai showcase |
## Keputusan desain
- **Full monokrom termasuk danger**: flag/moderation tidak lagi merah —
ditandai badge putih-di-atlas-hitam inversi + pulse. Kalau user kangen merah,
tinggal isi ulang `--color-vermilion`.
- Semua animasi hanya `transform`/`opacity` (compositor-friendly), hormati
`prefers-reduced-motion` (sudah ada kill-switch global).
## Verifikasi (gerbang)
1. `tsc --noEmit` bersih; biome 0 error 0 warning.
2. `pnpm build` sukses; smoke lokal 4024 → 9 route 200.
3. Push → GHA "Build & Deploy (Nix)" hijau → live 9×200.
4. Visual check live: desktop (rail game-menu terlihat) + cek rule mobile
(media query & gate kode) — screenshot disimpan.
@@ -93,6 +93,12 @@ type BatchOkResponse = {
ok: true;
conversationKey: string;
rows: MessageRecord[];
/**
* Race-guard signal (2026-08-25): target ids whose attachment upload is
* still in-flight — NO analysis ran for them. The processor must defer
* these (requeue + poll), never fan them out as failures.
*/
uploadPendingIds?: string[];
};
type BatchErrorResponse = {
ok: false;
@@ -100,7 +106,16 @@ type BatchErrorResponse = {
rows: MessageRecord[];
error: string;
};
type IndividualOkResponse = { ok: true; results: AnalysisResult[] };
type IndividualOkResponse = {
ok: true;
results: AnalysisResult[];
/**
* Race-guard signal (2026-08-24): the message's attachment upload is still
* in-flight — NO analysis ran. The processor must re-queue the message as
* `pending` and re-schedule, never treat this as a completed moderation.
*/
uploadPending?: boolean;
};
type IndividualErrorResponse = {
ok: false;
results: AnalysisResult[];
@@ -310,7 +325,16 @@ async function processBatch(job: {
? messages
: messages.filter((m) => !pendingUploadTargetIds.has(m.id));
if (readyMessages.length === 0) {
return { ok: true, conversationKey, rows: [] };
// Explicit signal (2026-08-25): every target is still upload-pending.
// Returning bare {ok:true, rows:[]} made the processor classify all of
// them "incomplete" and fan out to the individual queue — a hot ~300ms
// requeue loop for the whole upload duration.
return {
ok: true,
conversationKey,
rows: [],
uploadPendingIds: messages.map((m) => m.id),
};
}
// The orchestrator handles text/media split + caching + parallel paths
@@ -415,7 +439,7 @@ async function processIndividual(job: {
(a) => a.message_id === message.id && a.upload_status === "pending",
);
if (uploadStillPending) {
return { ok: true, results: [] };
return { ok: true, results: [], uploadPending: true };
}
try {
@@ -0,0 +1,108 @@
/**
* batchOutcomeClassifier.ts
*
* Pure partitioner of the batch worker response (2026-08-25).
*
* Bug history: the batch race guard returned `{ok:true, rows:[]}` when every
* target's attachment upload was still in-flight. The processor classified all
* of them as "incomplete" and fanned out to the individual queue, where the
* guard there requeued + rescheduled at the 250ms debounce — a hot ~300ms loop
* for the entire upload duration (~10 cycles in 3s in prod logs). Root fix:
* the worker now reports `uploadPendingIds` explicitly and this pure function
* partitions the outcome so upload-pending targets NEVER enter the fanout.
*/
export interface BatchRowLike {
id?: string;
ai_status?: string | null;
ai_moderation_flags?: string | null;
}
export interface BatchWorkerResponseLike {
ok?: boolean;
rows?: BatchRowLike[];
/** Explicit race-guard signal from the worker (2026-08-25). */
uploadPendingIds?: string[];
error?: string;
}
/** One target's per-message disposition after a batch attempt. */
export type BatchTargetKind =
| "completed"
| "upload_pending"
| "incomplete"
| "parse_failed"
| "api_failed";
function flagsOf(row: { ai_moderation_flags?: string | null }): string[] {
if (!row.ai_moderation_flags) return [];
try {
const parsed = JSON.parse(row.ai_moderation_flags) as unknown;
return Array.isArray(parsed) ? (parsed as string[]) : [];
} catch {
return [] as string[];
}
}
/**
* Partition the input message ids into per-message dispositions for one batch
* worker response. Pure: no DB/Piscina/logger — unit-testable directly.
*
* Priority per id: explicit uploadPendingIds → completed row → flag-based
* failure kinds → unexplained missing (treated like incomplete).
*/
export function partitionBatchOutcome(
messages: ReadonlyArray<{ id: string }>,
response: BatchWorkerResponseLike,
): Map<string, BatchTargetKind> {
const pendingSet = new Set(response.uploadPendingIds ?? []);
const rowsById = new Map(
(response.rows ?? [])
.filter((r): r is BatchRowLike & { id: string } => Boolean(r?.id))
.map((r) => [r.id, r]),
);
const out = new Map<string, BatchTargetKind>();
for (const msg of messages) {
if (pendingSet.has(msg.id)) {
out.set(msg.id, "upload_pending");
continue;
}
const row = rowsById.get(msg.id);
if (!row) {
// Unexplained drop: LLM silently omitted it. Same retryable bucket as
// analysis_incomplete — never a silent success.
out.set(msg.id, "incomplete");
continue;
}
if (row.ai_status !== "error") {
out.set(msg.id, "completed");
continue;
}
const flags = flagsOf(row);
if (flags.includes("analysis_incomplete")) {
out.set(msg.id, "incomplete");
} else if (flags.includes("analysis_parse_failed")) {
out.set(msg.id, "parse_failed");
} else if (flags.includes("analysis_api_failed")) {
out.set(msg.id, "api_failed");
} else {
out.set(msg.id, "incomplete");
}
}
return out;
}
/**
* Linear backoff ramp for consecutive upload-pending polls:
* poll N (1-based) waits min(base × N, cap). Keeps latency low for fast
* uploads while bounding total polling cost for long uploads.
*/
export function computeUploadPollDelayMs(
consecutivePolls: number,
baseMs: number,
capMs: number,
): number {
const n = Math.max(1, Math.floor(consecutivePolls));
return Math.min(Math.round(baseMs * n), Math.round(capMs));
}
@@ -4,6 +4,10 @@ import { isAgeRestrictedMetadata } from "../message-capture/messageMetadata.js";
import { messageStore } from "../message-capture/messageStore.js";
import type { MessageRecord } from "../message-capture/types.js";
import { pickBatchWithinBudget as pickBatchWithinBudgetPure } from "./batchBudget.js";
import {
computeUploadPollDelayMs,
partitionBatchOutcome,
} from "./batchOutcomeClassifier.js";
import { workerPool } from "./circuitBreaker.js";
import { estimateTokens } from "./conversationContext.js";
import {
@@ -21,11 +25,20 @@ import {
const logger = createChildLogger("batch-processor");
/**
* Consecutive upload-pending poll counter per conversation (2026-08-25).
* Drives the linear backoff ramp while attachments are still uploading;
* cleared as soon as a batch comes back with no upload-pending targets.
*/
const conversationUploadPolls = new Map<string, number>();
export interface AnalysisWorkerResponse {
ok: boolean;
conversationKey: string;
rows: MessageRecord[];
error?: string;
/** Explicit upload-in-flight signal from the batch race guard (2026-08-25). */
uploadPendingIds?: string[];
}
// ---------------------------------------------------------------------------
@@ -141,6 +154,8 @@ export async function processBatch(
activeRequests++;
let shouldScheduleNext = false;
/** Set when upload-pending targets defer the next cycle by this many ms. */
let deferredUploadRescheduleMs: number | null = null;
try {
const result = (await workerPool.run({
type: "batch",
@@ -199,37 +214,85 @@ export async function processBatch(
return;
}
// Batch succeeded -- check for messages the LLM silently dropped or failed
const incompleteMessages: MessageRecord[] = [];
const parseFailedMessages: MessageRecord[] = [];
// Batch succeeded -- partition per-message outcome explicitly (2026-08-25).
// upload_pending targets are DEFERRED (never fanned out): the old code
// treated them as incomplete -> individual queue -> requeue+250ms
// reschedule -> hot ~300ms loop for the whole upload duration.
const outcomeById = partitionBatchOutcome(messages, result);
const messagesForIndividualQueue: MessageRecord[] = [];
const apiFailedMessages: MessageRecord[] = [];
const uploadPendingMessages: MessageRecord[] = [];
for (const msg of messages) {
const row = result.rows.find((r) => r.id === msg.id);
if (!row) {
incompleteMessages.push(msg);
continue;
}
if (row.ai_status === "error") {
let flags: string[] = [];
try {
flags = JSON.parse(row.ai_moderation_flags ?? "[]") as string[];
} catch {}
if (flags.includes("analysis_incomplete")) {
incompleteMessages.push(msg);
} else if (flags.includes("analysis_parse_failed")) {
parseFailedMessages.push(msg);
} else if (flags.includes("analysis_api_failed")) {
switch (outcomeById.get(msg.id)) {
case "upload_pending":
uploadPendingMessages.push(msg);
break;
case "api_failed":
// Preserve the dedicated api-failure semantics below: revert +
// conversation cooldown instead of an immediate individual retry.
apiFailedMessages.push(msg);
}
break;
default:
// incomplete / parse_failed / unexplained drops stay retryable via
// the individual fallback queue (same semantics as before).
messagesForIndividualQueue.push(msg);
break;
}
}
const messagesForIndividualQueue = [
...incompleteMessages,
...parseFailedMessages,
];
if (uploadPendingMessages.length > 0) {
const polls = (conversationUploadPolls.get(conversationKey) ?? 0) + 1;
conversationUploadPolls.set(conversationKey, polls);
const delayMs = computeUploadPollDelayMs(
polls,
config.AI_ANALYSIS_UPLOAD_POLL_MS,
config.AI_ANALYSIS_MAX_UPLOAD_POLL_MS,
);
logger.debug(
{
conversationKey,
count: uploadPendingMessages.length,
ids: uploadPendingMessages.map((m) => m.id),
pollAttempt: polls,
delayMs,
},
"Attachment upload in-flight for batch targets — deferring with poll backoff",
);
// Put the rows back to `pending` so the scheduler owns them again.
await messageStore
.updateMessagesAIAnalysisBulk(
uploadPendingMessages.map((msg) => ({
messageId: msg.id,
result: {
status: "pending",
flags: null,
score: null,
analysis: null,
categories: null,
severity: null,
confidence: null,
recommendedAction: null,
analyzedAt: null,
error: null,
},
})),
)
.catch((err: unknown) => {
logger.error(
{ error: String(err), ids: uploadPendingMessages.map((m) => m.id) },
"Failed to revert upload-pending batch targets to pending",
);
return [] as MessageRecord[];
});
// Poll backoff instead of the 250ms debounce: the finally-block
// schedules the next cycle after this delay instead of immediately.
deferredUploadRescheduleMs = delayMs;
} else {
conversationUploadPolls.delete(conversationKey);
}
if (messagesForIndividualQueue.length > 0) {
logger.warn(
@@ -307,7 +370,11 @@ export async function processBatch(
resetConversationBatchFailures(conversationKey);
conversationErrorCooldown.delete(conversationKey);
}
// Upload-pending defer owns the next-cycle timing; don't let the default
// immediate schedule override it.
if (deferredUploadRescheduleMs === null) {
shouldScheduleNext = true;
}
} catch (error) {
recordConversationBatchFailure(conversationKey);
@@ -344,7 +411,17 @@ export async function processBatch(
if (conversationProcessing.get(conversationKey) === processingStartedAt) {
conversationProcessing.delete(conversationKey);
}
if (shouldScheduleNext) {
if (deferredUploadRescheduleMs !== null) {
// Upload still in-flight: re-schedule after the backoff delay instead of
// immediately (the old path hot-looped at ~250-300ms per cycle).
const delayMs = deferredUploadRescheduleMs;
setTimeout(() => {
// Dynamic import to avoid circular dependency at module scope
import("./batchScheduler.js").then((m) =>
m.scheduleConversationAnalysis(conversationKey),
);
}, delayMs).unref();
} else if (shouldScheduleNext) {
setImmediate(() => {
// Dynamic import to avoid circular dependency at module scope
import("./batchScheduler.js").then((m) =>
@@ -0,0 +1,65 @@
/**
* fallbackResultClassifier.ts
*
* Pure classifier for the individual-fallback worker response.
*
* Bug history (2026-08-24): the worker's upload-pending race guard returned
* `{ ok: true, results: [] }` (a legacy "no results yet" signal), but the
* processor treated ANY `ok:true` as a successful moderation. Empty results
* meant nothing was written to the DB — the message stayed stuck in
* `ai_status='processing'` with nobody watching it until the 300s cleanup
* reverted it. That single gap produced the ~330-400s attachment delay
* cluster. Classification now happens in ONE pure function so every outcome
* has an explicit, testable owner.
*/
export type WorkerResultKind =
| "success"
| "upload_pending"
| "incomplete"
| "error";
export interface ClassifiableWorkerResult {
ok?: boolean;
/** Upload-pending marker set by ai-analysis-worker's race guard. */
uploadPending?: boolean;
results?: Array<{ status?: string; flags?: string[] | string } | undefined>;
error?: string;
}
function flagsOf(r: { flags?: string[] | string }): string[] {
if (!r.flags) return [];
if (Array.isArray(r.flags)) return r.flags;
try {
const parsed = JSON.parse(r.flags) as unknown;
return Array.isArray(parsed) ? (parsed as string[]) : [];
} catch {
return [];
}
}
/**
* Classify an individual-fallback worker response:
* - "upload_pending": explicit race-guard signal — retry shortly, NOT an error.
* - "success": at least one result and none is analysis_incomplete.
* - "incomplete": LLM ran but dropped/failed this message after retries
* (analysis_incomplete flag) — terminal exhausted path.
* - "error": anything else (ok:false, or ok:true with NO explainable
* results). The old code silently succeeded here — never again.
*/
export function classifyIndividualWorkerResult(
result: ClassifiableWorkerResult,
): WorkerResultKind {
if (result.uploadPending === true) return "upload_pending";
const results = (result.results ?? []).filter(
(r): r is NonNullable<typeof r> => Boolean(r),
);
if (results.length === 0) return "error";
if (result.ok !== true) return "error";
for (const r of results) {
const flags = flagsOf(r);
if (flags.includes("analysis_incomplete")) return "incomplete";
if ((r.status ?? "") === "") return "error";
}
return "success";
}
@@ -8,6 +8,7 @@ import type {
} from "../message-capture/types.js";
import { getConversationKey, workerPool } from "./circuitBreaker.js";
import { fireAlert } from "./conversationState.js";
import { classifyIndividualWorkerResult } from "./fallbackResultClassifier.js";
import {
broadcastAnalysisCompleted,
LAST_ERROR,
@@ -78,26 +79,82 @@ async function processIndividualFallback(
message,
skipNormalAnalysis: false,
} as unknown)) as
| { ok: true; results: AnalysisResult[] }
| { ok: true; results: AnalysisResult[]; uploadPending?: boolean }
| { ok: false; results: AnalysisResult[]; error: string };
// Explicit outcome classification (2026-08-24): the old code treated any
// ok:true as a completed moderation, so the upload-pending race guard's
// empty results left messages stuck in `processing` until the 300s
// cleanup reverted them — the root cause of the ~330s attachment delays.
const kind = classifyIndividualWorkerResult(workerResult);
if (kind === "upload_pending") {
// Attachment still uploading — put the row back to `pending` and
// re-schedule this conversation immediately. The next scheduler cycle
// (~debounce 250ms) re-fetches; once upload_status flips to done the
// race guard passes and analysis proceeds. NOT an error: never touches
// the circuit breaker counters.
const revertedRows = await messageStore
.updateMessagesAIAnalysisBulk([
{
messageId,
result: {
status: "pending",
flags: null,
score: null,
analysis: null,
categories: null,
severity: null,
confidence: null,
recommendedAction: null,
analyzedAt: null,
error: null,
},
},
])
.catch((dbErr: unknown) => {
logger.error(
{ messageId, error: String(dbErr) },
"Failed to revert upload-pending message to pending",
);
return [] as MessageRecord[];
});
for (const row of revertedRows) {
broadcastAnalysisCompleted(row);
}
logger.debug(
{ messageId, conversationKey },
"Individual fallback: attachment upload in-flight — requeued as pending + rescheduled",
);
setImmediate(() => {
import("./batchScheduler.js")
.then((m) => m.scheduleConversationAnalysis(conversationKey))
.catch(() => {});
});
return;
}
let analysisResult: { results: AnalysisResult[] } | null = null;
if (workerResult.ok) {
const stillIncomplete = workerResult.results.some((r) =>
r.flags.includes("analysis_incomplete"),
);
if (stillIncomplete) {
if (kind === "success") {
analysisResult = workerResult;
} else if (kind === "incomplete") {
exhaustedOnIncomplete = true;
analysisResult = null;
} else {
analysisResult = workerResult;
}
// "error" — includes ok:true with unexplainable empty results (the old
// silent-success bug). Throw so it is treated as a transient failure.
throw new Error(
(workerResult as { error?: string }).error ??
"Individual worker returned no explainable results",
);
}
// No heuristic fallback: an incomplete/errored LLM result stays a
// retryable error — the recovery worker picks it up later. Producing a
// regex/wordlist verdict here would reintroduce false positives.
// (incomplete keeps its exhausted flag so the catch writes the terminal
// individual_analysis_exhausted status.)
if (!analysisResult) {
throw new Error(`LLM analysis failed for message ${messageId}`);
}
@@ -83,7 +83,7 @@ export async function callModerationLLM(
jsonResponse: { type: "json_object" },
retries: 0,
signal,
// Router (9router/omniroute) always streams SSE even when the
// Router (9router / formerly omniroute) always streams SSE even when the
// request omits `stream`. In non-stream mode the OpenAI SDK waits
// for the FULL body before parsing, so slow/long upstream streams
// hit the 30s/60s timeout and abort mid-generation. Streaming mode
@@ -356,10 +356,10 @@ export async function llmVision(
promptText: string,
imageUrl: { url: string },
): Promise<string | null> {
const completion = await llmChat({
const params = {
messages: [
{
role: "user",
role: "user" as const,
content: [
{ type: "text" as const, text: promptText },
{ type: "image_url" as const, image_url: imageUrl },
@@ -371,9 +371,26 @@ export async function llmVision(
temperature: 0.1,
top_p: 0.9,
retries: 0,
stream: true, // router always streams SSE; non-stream waits for full body and times out
timeout: config.AI_LLM_VISION_ANALYSIS_TIMEOUT_MS ?? 60_000,
});
};
// Streaming first (the router always streams SSE; a non-stream request
// waits for the full body and times out on slow models). Fallback (2026-08-24):
// large GIFs/images sometimes get their SSE stream truncated mid-flight by
// the upstream ("Stream ended before producing a non-ping SSE event") — all
// streaming retries fail identically, so retry ONCE with stream:false where
// the router assembles the complete response server-side.
let completion: Awaited<ReturnType<typeof llmChat>>;
try {
completion = await llmChat({ ...params, stream: true });
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
if (/stream ended before producing a non-ping sse/i.test(msg)) {
completion = await llmChat({ ...params, stream: false });
} else {
throw err;
}
}
if (!completion) return null;
return completion.choices[0]?.message?.content?.trim() ?? null;
@@ -4,6 +4,8 @@
* Orchestrates LLM-based moderation analysis — manages batch splitting,
* parallel text+media analysis, LLM calls with retry, and cache handling.
*/
import { LRUCache } from "lru-cache";
import { createChildLogger } from "@/shared/logger/index";
import { config } from "../../shared/config/config.js";
import { extractMessageMediaEvidence } from "../message-capture/messageMetadata.js";
@@ -20,16 +22,29 @@ import { isQdrantConfigured, searchQdrantBatch } from "./qdrantClient.js";
import { logCacheEvent } from "./responseLogger.js";
import { runTextOnlyBatch } from "./textBatchProcessor.js";
import {
bumpTextModerationHitCounts,
ERROR_ARTIFACT_FLAGS,
findSimilarTextModeration,
getCachedTextModeration,
getCachedTextModerations,
isGloballyReusableCleanVerdict,
isSemanticBandAccepted,
makeModerationContextKey,
makeTextModerationCacheKey,
parseQdrantVerdict,
type StoredModerationVerdict,
setCachedTextModeration,
upsertBareKeyToQdrant,
} from "./textCacheStore.js";
const log = createChildLogger("moderationOrchestrator");
/**
* Bare keys already written this process (dual-key write-back dedupe).
* LRU-bounded so a long-lived gateway can't grow it without limit; the DB
* upsert underneath is idempotent anyway — this just avoids redundant writes.
*/
const globalBareKeysWritten = new LRUCache<string, true>({ max: 5000 });
// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------
@@ -73,46 +88,74 @@ export async function runModerationAnalysis(
initCacheStore(config.REDIS_URL);
if (!targets.length) throw new Error("No targets provided for analysis");
// ── Phase 1: exact-hash cache (per conversation context) ────────────────
// ── Phase 1: exact-hash cache — ONE batched DB query ────────────────────
// Key is content + conversation context (channel/thread). On a scoped miss
// we also probe the legacy bare key: verdicts that CANNOT trigger an action
// (clean, flagless, action=none) may be reused across channels under strict
// freshness + confidence guards — flagged/warn verdicts never leave their
// conversation. This replaced the old N-sequential-query loop (60-message
// burst = 60 PgBouncer round-trips before).
const cacheHits: AnalysisResult[] = [];
const uncachedTargets: MessageRecord[] = [];
// cacheKey → result for identical-content dedupe within one batch
// cacheKey → representative result for identical-content dedupe
const hitByKey = new Map<string, AnalysisResult>();
// Embedding per exact cache key — computed once during lookup, reused
// when the fresh LLM verdict is written back to the semantic cache.
const embeddingsByKey = new Map<string, number[]>();
interface ExactCandidate {
target: MessageRecord;
scopedKey: string;
bareKey: string;
}
const candidates: ExactCandidate[] = [];
for (const target of targets) {
const hasMedia = hasMediaContent(target, attachments);
if (hasMedia) {
if (hasMediaContent(target, attachments)) {
uncachedTargets.push(target);
continue;
}
const rawContent = target.edited_content ?? target.content;
if (!rawContent.trim()) {
uncachedTargets.push(target);
continue;
}
const cacheKey = makeTextModerationCacheKey(
candidates.push({
target,
scopedKey: makeTextModerationCacheKey(
rawContent,
makeModerationContextKey(target),
);
const seen = hitByKey.get(cacheKey);
if (seen) {
// Same content already resolved this batch — reuse the verdict.
cacheHits.push({ ...seen, messageId: target.id });
continue;
),
bareKey: makeTextModerationCacheKey(rawContent),
});
}
try {
const cached = await getCachedTextModeration(cacheKey);
if (cached) {
// Identical content within one batch resolves once (representative).
const firstByScopedKey = new Map<string, ExactCandidate>();
for (const c of candidates) {
if (!firstByScopedKey.has(c.scopedKey))
firstByScopedKey.set(c.scopedKey, c);
}
// Single round-trip for every key we might serve from (scoped + bare).
const storedEntries = await getCachedTextModerations([
...firstByScopedKey.keys(),
...Array.from(firstByScopedKey.values(), (c) => c.bareKey),
]);
// Keys actually served — bumped in one UPDATE at the end for metrics.
const servedCacheKeys = new Set<string>();
/** Validate + admit one stored verdict for a candidate. */
const acceptExactVerdict = (
candidate: ExactCandidate,
cacheKey: string,
entry: { verdict: StoredModerationVerdict },
policyVersion: string,
): boolean => {
const { verdict } = entry;
const hasMediaInMeta =
target.metadata &&
candidate.target.metadata &&
(() => {
const ev = extractMessageMediaEvidence(target.metadata);
const ev = extractMessageMediaEvidence(candidate.target.metadata);
return (
ev.attachments.length > 0 ||
ev.stickers.length > 0 ||
@@ -122,48 +165,87 @@ export async function runModerationAnalysis(
if (hasMediaInMeta) {
log.debug(
{ messageId: target.id, cacheKey },
{ messageId: candidate.target.id, cacheKey },
"Cache entry but message has media — treating as miss",
);
} else if (
cached.flags.some((f) =>
[
"analysis_api_failed",
"analysis_parse_failed",
"analysis_incomplete",
].includes(f),
return false;
}
if (
verdict.flags.some((f) =>
(ERROR_ARTIFACT_FLAGS as readonly string[]).includes(f),
)
) {
log.warn(
{ messageId: target.id, cacheKey },
{ messageId: candidate.target.id, cacheKey },
"Cache entry contains error artifact — treating as miss",
);
} else {
const hit: AnalysisResult = {
messageId: target.id,
status: cached.status,
flags: cached.flags,
score: cached.score,
analysis: cached.analysis,
categories: cached.categories,
severity: cached.severity as AnalysisResult["severity"],
confidence: cached.confidence,
recommendedAction:
cached.recommendedAction as AnalysisResult["recommendedAction"],
policyVersion: "cached-user-moderation-2026-06",
evidence: [],
};
cacheHits.push(hit);
hitByKey.set(cacheKey, hit);
logCacheEvent("hit", cacheKey, "text");
continue;
}
}
} catch {
/* proceed */
return false;
}
uncachedTargets.push(target);
hitByKey.set(candidate.scopedKey, {
messageId: candidate.target.id,
status: verdict.status,
flags: verdict.flags,
score: verdict.score,
analysis: verdict.analysis,
categories: verdict.categories,
severity: verdict.severity as AnalysisResult["severity"],
confidence: verdict.confidence,
recommendedAction:
verdict.recommendedAction as AnalysisResult["recommendedAction"],
policyVersion,
evidence: [],
});
servedCacheKeys.add(cacheKey);
logCacheEvent("hit", cacheKey, "text");
return true;
};
for (const candidate of firstByScopedKey.values()) {
const scopedEntry = storedEntries.get(candidate.scopedKey);
if (
scopedEntry &&
acceptExactVerdict(
candidate,
candidate.scopedKey,
scopedEntry,
"cached-user-moderation-2026-06",
)
) {
continue;
}
// Context-free fallback: ONLY non-actionable clean verdicts qualify
// (guard enforces status/flags/action/confidence/freshness). The bare
// key equals the scoped key for context-less messages, so the guard
// also prevents double-serving the same row.
const bareEntry = storedEntries.get(candidate.bareKey);
if (
bareEntry &&
candidate.bareKey !== candidate.scopedKey &&
isGloballyReusableCleanVerdict(
bareEntry.verdict,
bareEntry.analyzedAt ?? undefined,
)
) {
acceptExactVerdict(
candidate,
candidate.bareKey,
bareEntry,
"cached-global-clean-2026-08",
);
}
}
// Fan-out: every candidate (representative + in-batch duplicates) gets its
// own copy of the representative verdict; unresolved ones stay queued.
for (const candidate of candidates) {
const representative = hitByKey.get(candidate.scopedKey);
if (representative) {
cacheHits.push({ ...representative, messageId: candidate.target.id });
} else {
uncachedTargets.push(candidate.target);
}
}
// ── Phase 2: semantic cache — batched (one embed call + one Qdrant
@@ -199,10 +281,12 @@ export async function runModerationAnalysis(
}
if (isQdrantConfigured()) {
// ONE batch search at the LOOSER threshold; per-hit re-classification
// enforces the strict band for actionable verdicts.
const batchHits = await searchQdrantBatch(
embeddings,
config.AI_LLM_EMBEDDING_MAX_CANDIDATES,
config.AI_LLM_EMBEDDING_MIN_SIMILARITY,
config.AI_LLM_EMBEDDING_MIN_SIMILARITY_CLEAN,
);
for (let i = 0; i < semanticCandidates.length; i++) {
const { target, cacheKey } = semanticCandidates[i];
@@ -210,6 +294,7 @@ export async function runModerationAnalysis(
if (hits.length === 0) continue;
const verdict = parseQdrantVerdict(hits[0].payload, hits[0].score);
if (!verdict) continue;
if (!isSemanticBandAccepted(verdict, verdict.similarity)) continue;
log.debug(
{
messageId: target.id,
@@ -234,6 +319,7 @@ export async function runModerationAnalysis(
};
cacheHits.push(hit);
hitByKey.set(cacheKey, hit);
servedCacheKeys.add(cacheKey); // bump hit_count for metrics
logCacheEvent("hit", cacheKey, "text");
}
} else {
@@ -242,10 +328,12 @@ export async function runModerationAnalysis(
const { target, cacheKey } = semanticCandidates[i];
const semantic = await findSimilarTextModeration(
embeddings[i],
config.AI_LLM_EMBEDDING_MIN_SIMILARITY,
config.AI_LLM_EMBEDDING_MIN_SIMILARITY_CLEAN,
config.AI_LLM_EMBEDDING_MAX_CANDIDATES,
);
if (!semantic) continue;
if (!isSemanticBandAccepted(semantic, semantic.similarity))
continue;
log.debug(
{
messageId: target.id,
@@ -270,6 +358,7 @@ export async function runModerationAnalysis(
};
cacheHits.push(hit);
hitByKey.set(cacheKey, hit);
servedCacheKeys.add(cacheKey); // bump hit_count for metrics
logCacheEvent("hit", cacheKey, "text");
}
}
@@ -290,11 +379,14 @@ export async function runModerationAnalysis(
}
if (cacheHits.length > 0) {
// Metrics: one bulk UPDATE for every exact-cache key actually served.
bumpTextModerationHitCounts(Array.from(servedCacheKeys));
log.info(
{
cacheHits: cacheHits.length,
uncached: uncachedTargets.length,
total: targets.length,
servedKeys: servedCacheKeys.size,
},
"User moderation cache applied",
);
@@ -355,9 +447,7 @@ export async function runModerationAnalysis(
rawContent,
makeModerationContextKey(target),
);
setCachedTextModeration(
cacheKey,
{
const stored = {
flags: result.flags ?? [],
score: result.score ?? 0,
analysis: result.analysis ?? "",
@@ -366,9 +456,50 @@ export async function runModerationAnalysis(
confidence: result.confidence ?? result.score ?? 0,
recommendedAction: result.recommendedAction ?? "none",
status: result.status,
},
};
setCachedTextModeration(
cacheKey,
stored,
embeddingsByKey.get(cacheKey),
).catch(() => {});
// Dual-key write-back (2026-08-24): the FIRST analysis of a message runs
// WITH conversation context (accurate), but its verdict is also stored
// under the context-free bare key so repeats in OTHER channels hit the
// exact cache instead of paying a new LLM call. Same guard as the read
// path — only non-actionable clean verdicts may cross channels.
//
// 2026-08-25 cache-hit fix: the bare key is ALSO upserted to Qdrant
// (via upsertBareKeyToQdrant) with the SAME embedding already computed
// at lookup time. Previously the bare key was only PG-written with
// embedding=null — bare clean verdicts were DB-only and invisible to
// searchQdrantBatch, capping the semantic hit-rate below the exact-cache
// hit-rate for cross-channel repeats.
const bareKey = makeTextModerationCacheKey(rawContent);
if (
bareKey !== cacheKey &&
!globalBareKeysWritten.has(bareKey) &&
isGloballyReusableCleanVerdict(
{
status: stored.status,
flags: stored.flags,
score: stored.score,
analysis: stored.analysis,
categories: stored.categories,
severity: stored.severity,
confidence: stored.confidence,
recommendedAction: stored.recommendedAction,
},
undefined,
)
) {
globalBareKeysWritten.set(bareKey, true);
setCachedTextModeration(bareKey, stored, null).catch(() => {});
const bareEmbedding = embeddingsByKey.get(cacheKey);
if (bareEmbedding && bareEmbedding.length > 0) {
upsertBareKeyToQdrant(bareKey, stored, bareEmbedding).catch(() => {});
}
}
}
const allResults = [
@@ -1,5 +1,6 @@
import { createHash } from "node:crypto";
import { createChildLogger } from "@/shared/logger/index";
import { config } from "../../shared/config/config.js";
import { executeAll, executeGet } from "../../shared/database/drizzle.js";
import { findBestEmbeddingMatch } from "./embeddingClient.js";
import {
@@ -84,10 +85,43 @@ export function makeImageCacheKey(imageUrl: string): string {
// size is 8191"), which fails acquireMediaAnalysisLock and silently skips
// every media analysis. A 32-char sha256 keeps the key well under the limit
// and is still deterministic (same attachment → same key).
const hash = createHash("sha256").update(imageUrl).digest("hex").slice(0, 32);
const hash = createHash("sha256")
.update(normalizeDiscordImageUrl(imageUrl))
.digest("hex")
.slice(0, 32);
return `image:${hash}`;
}
/**
* Strip volatile query params from Discord CDN URLs so the SAME attachment
* always maps to ONE vision-cache key regardless of how it reached us
* (signed `?ex=&is=&hm=` tokens rotate per fetch; render variants differ by
* `format/width/height/size`). Previously each token variant hashed to its
* own key → the same image was re-downloaded and re-analyzed by the vision
* model once per variant. Non-Discord URLs and data: URLs are returned
* untouched (their query can be semantically meaningful).
*/
export function normalizeDiscordImageUrl(imageUrl: string): string {
try {
const parsed = new URL(imageUrl);
if (parsed.protocol !== "https:" && parsed.protocol !== "http:") {
return imageUrl;
}
const host = parsed.hostname;
const isAttachmentCdn = host === "cdn.discordapp.com";
const isRenderOrPreview =
host === "media.discordapp.net" ||
/^images-ext-\d+\.discordapp\.net$/.test(host);
if (!isAttachmentCdn && !isRenderOrPreview) return imageUrl;
if (!parsed.search) return imageUrl;
// Path IS the stable identity of the attachment; everything after "?" is
// signing or a render variant.
return `${parsed.origin}${parsed.pathname}`;
} catch {
return imageUrl;
}
}
/**
* Lookup a cached media analysis result.
* Returns the full cached text (the analysis summary string) or null if not found or expired.
@@ -294,10 +328,71 @@ export interface StoredModerationVerdict {
recommendedAction: string;
}
/** Raw DB row shape needed to rebuild a StoredModerationVerdict. */
interface VerdictRow {
flags: string;
analyzed_at?: number;
}
/**
* Parse one `text_analysis_cache` row into a StoredModerationVerdict.
* Shared by the single-key and batched getters so their semantics can never
* drift apart (status normalization lives in exactly one place).
*/
export function parseStoredVerdictRow(
row: VerdictRow,
): StoredModerationVerdict | null {
let parsed: Record<string, unknown>;
try {
parsed = JSON.parse(row.flags) as Record<string, unknown>;
} catch {
return null;
}
if (!parsed || typeof parsed !== "object") return null;
const flags = Array.isArray(parsed.flags) ? (parsed.flags as string[]) : [];
const status = normalizeStoredStatus(
parsed.status as string | undefined,
flags,
);
return {
status,
flags,
score: (parsed.score as number) ?? 0,
analysis: (parsed.analysis as string) ?? "",
categories: (parsed.categories as string[]) ?? [],
severity: (parsed.severity as string) ?? "none",
confidence: (parsed.confidence as number) ?? 0,
recommendedAction: (parsed.recommendedAction as string) ?? "none",
};
}
/**
* Increment the hit counter for a cache key (fire-and-forget).
*
* Bug history: `hit_count` was written as 0 on insert and never updated by
* any reader, so cache effectiveness was unmeasurable. This is best-effort
* observability — a failed bump must never affect the read path.
*/
function bumpHitCount(cacheKey: string): void {
executeAll(
`UPDATE text_analysis_cache SET hit_count = hit_count + 1 WHERE text = $1`,
[cacheKey],
).catch(() => {});
}
/** Error-artifact flags that make a cached verdict unusable. */
export const ERROR_ARTIFACT_FLAGS = [
"analysis_api_failed",
"analysis_parse_failed",
"analysis_incomplete",
] as const;
/**
* Lookup a cached moderation result for a text content.
* Returns the stored result fields or null.
*/
export async function getCachedTextModeration(
cacheKey: string,
): Promise<StoredModerationVerdict | null> {
@@ -311,25 +406,11 @@ export async function getCachedTextModeration(
if (!row) return null;
const parsed = JSON.parse(row.flags) as Record<string, unknown>;
const flags = (parsed.flags as string[]) ?? [];
// Use stored status if available (new entries), otherwise derive from
// flags (legacy compatibility). "warn" must survive the round-trip.
const status = normalizeStoredStatus(
parsed.status as string | undefined,
flags,
);
const verdict = parseStoredVerdictRow(row);
if (!verdict) return null;
return {
status,
flags,
score: (parsed.score as number) ?? 0,
analysis: (parsed.analysis as string) ?? "",
categories: (parsed.categories as string[]) ?? [],
severity: (parsed.severity as string) ?? "none",
confidence: (parsed.confidence as number) ?? 0,
recommendedAction: (parsed.recommendedAction as string) ?? "none",
};
bumpHitCount(cacheKey);
return verdict;
} catch (error) {
logger.error(
{ error: error instanceof Error ? error.message : String(error) },
@@ -339,6 +420,128 @@ export async function getCachedTextModeration(
}
}
/**
* Batched exact-hash lookup: ONE query for N keys.
*
* Semantics are identical to calling `getCachedTextModeration` per key
* (unexpired rows only, shared row parser). Per-key hit-count bumps are NOT
* issued here — the orchestrator logs an aggregate "cache applied" line
* instead, keeping a 60-message burst at exactly one round-trip.
* `analyzedAt` is surfaced so callers can apply freshness guards.
*/
export interface BatchedVerdictEntry {
verdict: StoredModerationVerdict;
analyzedAt: number | null;
}
export async function getCachedTextModerations(
cacheKeys: string[],
): Promise<Map<string, BatchedVerdictEntry>> {
const results = new Map<string, BatchedVerdictEntry>();
const uniqueKeys = Array.from(new Set(cacheKeys)).filter(Boolean);
if (uniqueKeys.length === 0) return results;
const CHUNK_SIZE = 200;
try {
for (let i = 0; i < uniqueKeys.length; i += CHUNK_SIZE) {
const chunk = uniqueKeys.slice(i, i + CHUNK_SIZE);
// Postgres has a 32k bind-parameter ceiling; ANY($1) keeps it at one
// array param per chunk regardless of chunk length.
const rows = await executeAll(
`SELECT text, flags, analyzed_at
FROM text_analysis_cache
WHERE text = ANY($1::text[]) AND expires_at > $2`,
[chunk, Date.now()],
);
for (const row of rows ?? []) {
if (results.has(row.text)) continue;
const verdict = parseStoredVerdictRow(row);
if (!verdict) continue;
results.set(row.text, {
verdict,
analyzedAt:
typeof row.analyzed_at === "number" ? row.analyzed_at : null,
});
}
}
} catch (error) {
logger.error(
{ error: error instanceof Error ? error.message : String(error) },
"Failed batched text moderation lookup",
);
}
return results;
}
/**
* Fire-and-forget bulk hit-count bump for keys actually served as hits.
* Companion to the batched getter (which skips per-row bumps): one UPDATE
* per analysis batch keeps hit-rate metrics working at zero extra latency
* cost per message.
*/
export function bumpTextModerationHitCounts(cacheKeys: string[]): void {
const uniqueKeys = Array.from(new Set(cacheKeys)).filter(Boolean);
if (uniqueKeys.length === 0) return;
executeAll(
`UPDATE text_analysis_cache SET hit_count = hit_count + 1 WHERE text = ANY($1::text[])`,
[uniqueKeys],
).catch(() => {});
}
// ---------------------------------------------------------------------------
// Semantic two-band acceptance
// ---------------------------------------------------------------------------
/**
* True when a semantic-cache hit may be reused given its verdict class.
* Two bands (2026-08-24): non-actionable verdicts (clean / flagless /
* action=none) are accepted from the LOOSER clean band; actionable verdicts
* (warn/flagged or any flags/action) keep the strict historical gate.
* Between the bands → reject → the message falls through to the LLM
* (fail-open toward accuracy).
*/
export function isSemanticBandAccepted(
verdict: StoredModerationVerdict,
similarity: number,
): boolean {
const isNonActionable =
verdict.status === "clean" &&
verdict.flags.length === 0 &&
(verdict.recommendedAction ?? "none") === "none";
return isNonActionable
? similarity >= config.AI_LLM_EMBEDDING_MIN_SIMILARITY_CLEAN
: similarity >= config.AI_LLM_EMBEDDING_MIN_SIMILARITY;
}
// ---------------------------------------------------------------------------
// Global exact-cache reuse guard (context-free fallback)
// ---------------------------------------------------------------------------
/**
* True when a stored verdict is safe to reuse OUTSIDE its original channel:
* only verdicts that cannot trigger an action and carry no flags qualify,
* and they must be confident + fresh. Flagged/warn verdicts are NEVER
* globally reused — enforcement is context-sensitive by design.
*/
export function isGloballyReusableCleanVerdict(
verdict: Omit<StoredModerationVerdict, "status"> & { status: string },
analyzedAtMs: number | undefined,
): boolean {
if (verdict.status !== "clean") return false;
if (verdict.flags.length > 0) return false;
if ((verdict.recommendedAction ?? "none") !== "none") return false;
if (!(verdict.confidence >= config.AI_CACHE_GLOBAL_REUSE_MIN_CONFIDENCE))
return false;
if (
typeof analyzedAtMs === "number" &&
Date.now() - analyzedAtMs >
config.AI_CACHE_GLOBAL_REUSE_MAX_AGE_H * 60 * 60 * 1000
) {
return false;
}
return true;
}
/**
* Parse a Qdrant verdict payload into the result shape shared by the
* semantic cache lookups. Returns null on malformed payloads (callers then
@@ -353,31 +556,13 @@ export function parseQdrantVerdict(
similarity: number;
})
| null {
let parsed: Record<string, unknown>;
try {
parsed = JSON.parse(payload.flags) as Record<string, unknown>;
} catch {
return null;
}
if (!parsed || typeof parsed !== "object") return null;
const flags = (parsed.flags as string[]) ?? [];
const status = normalizeStoredStatus(
parsed.status as string | undefined,
flags,
);
const parsed = parseStoredVerdictRow({ flags: payload.flags });
if (!parsed) return null;
return {
...parsed,
text: payload.text,
similarity,
status,
flags,
score: (parsed.score as number) ?? 0,
analysis: (parsed.analysis as string) ?? "",
categories: (parsed.categories as string[]) ?? [],
severity: (parsed.severity as string) ?? "none",
confidence: (parsed.confidence as number) ?? 0,
recommendedAction: (parsed.recommendedAction as string) ?? "none",
};
}
@@ -479,6 +664,55 @@ export async function findSimilarTextModeration(
}
}
/**
* Upsert a bare (context-free) clean verdict to the Qdrant vector store,
* making global-reuse clean verdicts discoverable by semantic search.
*
* Why: the main `setCachedTextModeration` writes bare-key rows to Postgres
* with embedding=null (deliberate — no duplicate PG embedding column), but
* a bare clean verdict that never reaches Qdrant is invisible to
* searchQdrantBatch. So two messages with identical clean content in
* DIFFERENT channels never match semantically — the semantic hit-rate is
* capped below the exact-cache hit-rate. This helper shares the embedding
* already computed at lookup time so the bare point is semantically
* findable.
*
* Guard: only non-actionable clean verdicts qualify (same guard as the
* read path and as the orchestrator's bare-key write-back). No-op when
* Qdrant is disabled or no embedding is available.
*/
export async function upsertBareKeyToQdrant(
bareKey: string,
result: {
status: string;
flags: string[];
score: number;
analysis: string;
categories: string[];
severity: string;
confidence: number;
recommendedAction: string;
},
embedding: number[] | null | undefined,
): Promise<void> {
if (!isQdrantConfigured() || !embedding || embedding.length === 0) return;
if (!isGloballyReusableCleanVerdict(result, undefined)) return;
const now = Date.now();
const USER_MOD_CACHE_TTL_MS = 24 * 60 * 60 * 1000;
await upsertQdrantPoint(bareKey, embedding, {
text: bareKey,
flags: JSON.stringify(result),
analyzed_at: now,
expires_at: now + USER_MOD_CACHE_TTL_MS,
content_hash: bareKey.split(":").pop() ?? "",
}).catch((err: unknown) => {
logger.error(
{ error: err instanceof Error ? err.message : String(err), bareKey },
"Failed to upsert bare-key clean verdict to Qdrant",
);
});
}
/**
* Store a moderation result for a (user, content) pair.
* The `flags` field stores the full result object as JSON.
@@ -1,5 +1,6 @@
import { resolve } from "node:dns/promises";
import { isIP } from "node:net";
import { LRUCache } from "lru-cache";
import { createChildLogger } from "@/shared/logger/index";
import { createAbortControllerWithTimeout } from "@/shared/utils/index";
@@ -150,6 +151,47 @@ function truncateAndCleanHtml(html: string, maxLen = 1000): string {
export async function fetchUrlSafely(
url: string,
depth = 0,
): Promise<FetchedUrlContext> {
// Text results are memoized (in-process, short TTL): the same link recurs
// across batches and re-downloading + re-parsing the page each time was
// pure latency. Images are NEVER cached here — they are vision evidence
// and multi-MB buffers don't belong in an LRU. Errors are not cached so a
// transient network blip retries on the next batch.
if (depth === 0) {
const memo = textFetchMemo.get(url);
if (memo) return memo;
// In-flight dedupe: concurrent callers share one live request.
const existing = textInFlight.get(url);
if (existing) return existing;
const promise = fetchUrlSafelyUncached(url, depth)
.then((fetched) => {
if (fetched.type === "text") textFetchMemo.set(url, fetched);
return fetched;
})
.finally(() => {
textInFlight.delete(url);
});
textInFlight.set(url, promise);
return promise;
}
return fetchUrlSafelyUncached(url, depth);
}
/** In-process memo of successful TEXT fetches (30 min TTL, bounded size). */
const textFetchMemo = new LRUCache<string, FetchedUrlContext>({
max: 500,
ttl: 30 * 60 * 1000,
});
/** Concurrent same-URL text fetches collapse into one live request. */
const textInFlight = new LRUCache<string, Promise<FetchedUrlContext>>({
max: 100,
ttl: 60_000,
});
async function fetchUrlSafelyUncached(
url: string,
depth = 0,
): Promise<FetchedUrlContext> {
if (depth > 1) {
return { url, type: "error", error: "Max redirect/meta depth reached" };
@@ -21,6 +21,7 @@
import { createChildLogger } from "@/shared/logger/index";
import { createAbortControllerWithTimeout } from "@/shared/utils/index";
import { config } from "../../shared/config/config.js";
import { cacheGet, cacheSet, makeCacheKey } from "./cacheStore.js";
const log = createChildLogger("wikipedia-client");
@@ -57,10 +58,19 @@ function stripHtml(snippet: string): string {
.trim();
}
/** Redis TTL for cached search results (6h — articles change slowly). */
const SEARCH_CACHE_TTL_SECONDS = 6 * 60 * 60;
/**
* Search Wikipedia for a query and return up to MAX_RESULTS structured hits.
* Uses the Action API `list=search` (srsearch) which is stable and returns
* title + HTML snippet. Graceful: returns [] on any failure.
*
* Cached in the shared Redis store: the same query recurs across batches
* (repeat slang, recurring topics), and an uncached re-search per batch was
* pure latency + Wikipedia rate-limit pressure. Only NON-EMPTY results are
* cached — an empty result may be a transient limiter/network blip, so it is
* retried on a later batch instead of being pinned for 6 hours.
*/
export async function wikipediaSearch(
query: string,
@@ -69,6 +79,32 @@ export async function wikipediaSearch(
const q = query.trim();
if (!q) return [];
const cacheKey = makeCacheKey("wikisearch", q);
const cached = await cacheGet(cacheKey);
if (cached) {
try {
const parsed = JSON.parse(cached) as SearchResult[];
if (Array.isArray(parsed) && parsed.length > 0) {
log.debug({ query: q }, "Wikipedia search cache HIT");
return parsed;
}
} catch {
// Malformed entry — fall through to live fetch.
}
}
const mapped = await wikipediaSearchLive(q, timeoutMs);
if (mapped.length > 0) {
cacheSet(cacheKey, JSON.stringify(mapped), SEARCH_CACHE_TTL_SECONDS);
}
return mapped;
}
/** Live (uncached) Action API search. Returns [] on any failure. */
async function wikipediaSearchLive(
q: string,
timeoutMs: number,
): Promise<SearchResult[]> {
const params = new URLSearchParams({
action: "query",
list: "search",
@@ -48,7 +48,13 @@ export class MessagesCleanup {
}
async revertStuckProcessingMessages(
timeoutMs: number = 300000,
// 2026-08-24: lowered from 300000 — this cleanup is the last-resort
// recoverer for rows stuck in `processing`. With the upload-pending
// race-guard now requeueing properly (fallbackResultClassifier), any row
// that still sits here for >2min is a genuine leak; reverting sooner
// bounds the worst-case delay without racing legitimate in-flight work
// (media batches can legitimately take ~60s+).
timeoutMs: number = 120000,
): Promise<number> {
this.logger.debug({ timeoutMs }, "revertStuckProcessingMessages entry");
try {
@@ -149,8 +149,8 @@ export const configSchema = z
AI_LLM_BASE_URL: z
.string()
.url()
.default("https://omniroute.imrnes.team/v1"),
AI_LLM_MODEL: z.string().default("text"),
.default("https://9router.asepharyana.my.id/v1"),
AI_LLM_MODEL: z.string().default("claude-opus-5"),
// Vision uses the SAME router/base URL as text moderation
// (AI_LLM_BASE_URL) but a different model alias. The dedicated NVIDIA
// multimodal endpoint was removed.
@@ -168,11 +168,21 @@ export const configSchema = z
.min(0)
.max(1)
.default(0.97),
// Two-band semantic acceptance (2026-08-24): non-actionable verdicts
// (clean, no flags, action=none) may be reused from a LOOSER similarity
// band than actionable ones (warn/flagged). Actionable verdicts keep the
// strict gate above; anything between the two bands falls through to the
// LLM (fail-open toward accuracy).
AI_LLM_EMBEDDING_MIN_SIMILARITY_CLEAN: z.coerce
.number()
.min(0)
.max(1)
.default(0.92),
AI_LLM_EMBEDDING_MAX_CANDIDATES: z.coerce
.number()
.int()
.positive()
.default(30),
.default(50),
// Qdrant vector store for the semantic moderation cache. When
// QDRANT_URL is set, embeddings are stored/searched there (Postgres
// embedding column remains as a legacy fallback).
@@ -244,9 +254,29 @@ export const configSchema = z
.positive()
.default(10000),
AI_ANALYSIS_ERROR_COOLDOWN_MS: z.coerce.number().positive().default(30000),
// Upload-pending batch poll (2026-08-25): when a batch is deferred because
// attachments are still uploading, the processor re-schedules with this
// base delay (linear ramp per consecutive poll, capped) instead of the
// 250ms debounce — the old path hot-looped ~300ms for the whole upload.
AI_ANALYSIS_UPLOAD_POLL_MS: z.coerce.number().positive().default(1500),
AI_ANALYSIS_MAX_UPLOAD_POLL_MS: z.coerce.number().positive().default(8000),
// ── AI Analysis Batch ───────────────────────────────────────────────
AI_ANALYSIS_MAX_BATCH_SIZE: z.coerce.number().int().positive().default(200),
// Global exact-cache reuse guard (2026-08-24): a context-scoped miss may
// fall back to the legacy bare (context-free) key, but ONLY for verdicts
// that cannot trigger an action and are fresh + confident. These knobs
// bound that reuse.
AI_CACHE_GLOBAL_REUSE_MIN_CONFIDENCE: z.coerce
.number()
.min(0)
.max(1)
.default(0.85),
AI_CACHE_GLOBAL_REUSE_MAX_AGE_H: z.coerce
.number()
.int()
.positive()
.default(120),
AI_ANALYSIS_MAX_CONTEXT_TOKENS: z.coerce.number().positive().default(8000),
AI_ANALYSIS_MAX_TARGET_TOKENS: z.coerce.number().positive().default(14000),
AI_ANALYSIS_CONTEXT_MESSAGE_LIMIT: z.coerce
@@ -0,0 +1,110 @@
// ═══════════════════════════════════════════════════════════════════════════
// partitionBatchOutcome — upload-pending defer vs fanout (2026-08-25)
// ═══════════════════════════════════════════════════════════════════════════
// Bug history: the batch worker's race guard returned {ok:true, rows:[]} while
// attachments were still uploading; every target was classified "incomplete",
// fanned out to the individual queue, requeued there, rescheduled at 250ms —
// a hot ~300ms loop for the whole upload duration (~10 cycles in 3s in prod).
import { describe, expect, it } from "vitest";
import {
computeUploadPollDelayMs,
partitionBatchOutcome,
} from "../src/modules/ai-moderation/batchOutcomeClassifier.js";
const msgs = (...ids: string[]) => ids.map((id) => ({ id }));
describe("partitionBatchOutcome", () => {
it("marks ALL targets upload_pending when the full-batch guard fires", () => {
const out = partitionBatchOutcome(msgs("a", "b"), {
ok: true,
rows: [],
uploadPendingIds: ["a", "b"],
});
expect(out.get("a")).toBe("upload_pending");
expect(out.get("b")).toBe("upload_pending");
});
it("never classifies an explicit upload_pending id as incomplete", () => {
// The regression this file exists for: uploadPendingIds must win over the
// missing-row heuristic.
const out = partitionBatchOutcome(msgs("a"), {
ok: true,
rows: [],
uploadPendingIds: ["a"],
});
expect(out.get("a")).not.toBe("incomplete");
expect(out.get("a")).toBe("upload_pending");
});
it("partitions a mixed batch: completed + upload-pending + missing", () => {
const out = partitionBatchOutcome(msgs("ok1", "up1", "gone1"), {
ok: true,
rows: [
{ id: "ok1", ai_status: "clean" },
// up1 has NO row but IS in uploadPendingIds -> deferred, not failed
],
uploadPendingIds: ["up1"],
});
expect(out.get("ok1")).toBe("completed");
expect(out.get("up1")).toBe("upload_pending");
expect(out.get("gone1")).toBe("incomplete"); // unexplained drop stays retryable
});
it("routes flag-based failures to their buckets", () => {
const out = partitionBatchOutcome(msgs("i", "p", "f"), {
ok: true,
rows: [
{
id: "i",
ai_status: "error",
ai_moderation_flags: JSON.stringify(["analysis_incomplete"]),
},
{
id: "p",
ai_status: "error",
ai_moderation_flags: JSON.stringify(["analysis_parse_failed"]),
},
{
id: "f",
ai_status: "error",
ai_moderation_flags: JSON.stringify(["analysis_api_failed"]),
},
],
});
expect(out.get("i")).toBe("incomplete");
expect(out.get("p")).toBe("parse_failed");
expect(out.get("f")).toBe("api_failed");
});
it("treats error rows with no known flag as retryable incomplete", () => {
const out = partitionBatchOutcome(msgs("x"), {
ok: true,
rows: [
{ id: "x", ai_status: "error", ai_moderation_flags: '["weird_flag"]' },
],
});
expect(out.get("x")).toBe("incomplete");
});
it("accepts DB-round-trip MessageRecord shape (null ai_status)", () => {
const out = partitionBatchOutcome([{ id: "r", ai_status: null }] as never, {
ok: true,
rows: [{ id: "r", ai_status: null }],
});
expect(out.get("r")).toBe("completed");
});
});
describe("computeUploadPollDelayMs", () => {
it("ramps linearly and respects the cap", () => {
expect(computeUploadPollDelayMs(1, 1500, 8000)).toBe(1500);
expect(computeUploadPollDelayMs(2, 1500, 8000)).toBe(3000);
expect(computeUploadPollDelayMs(3, 1500, 8000)).toBe(4500);
expect(computeUploadPollDelayMs(9, 1500, 8000)).toBe(8000); // capped
});
it("is safe on degenerate input", () => {
expect(computeUploadPollDelayMs(0, 1500, 8000)).toBe(1500);
expect(computeUploadPollDelayMs(-5, 1000, 4000)).toBe(1000);
});
});
@@ -0,0 +1,127 @@
// ═══════════════════════════════════════════════════════════════════════════
// Batched exact-cache lookup (getCachedTextModerations)
// ═══════════════════════════════════════════════════════════════════════════
// Design (2026-08-24): phase-1 cache lookups collapse N sequential per-key
// queries into ONE `text = ANY($1::text[])` query. Semantics must match the
// single-key getter: unexpired rows only, malformed rows skipped, verdicts
// normalized through the shared parser. The DB layer is mocked — no live
// Postgres in unit tests.
import { beforeEach, describe, expect, it, vi } from "vitest";
const executeAll = vi.fn();
const executeGet = vi.fn();
vi.mock("../src/shared/database/drizzle.js", () => ({
executeAll: (...args: unknown[]) => executeAll(...args),
executeGet: (...args: unknown[]) => executeGet(...args),
}));
import {
getCachedTextModerations,
parseStoredVerdictRow,
} from "../src/modules/ai-moderation/textCacheStore.js";
function rowFor(
text: string,
status: string,
analyzedAt = Date.now() - 1000,
): Record<string, unknown> {
return {
text,
flags: JSON.stringify({
status,
flags: [],
score: 0,
analysis: "ok",
categories: [],
severity: "none",
confidence: 0.9,
recommendedAction: "none",
}),
source: "user_moderation",
analyzed_at: analyzedAt,
expires_at: Date.now() + 3_600_000,
hit_count: 0,
};
}
beforeEach(() => {
executeAll.mockReset();
executeGet.mockReset();
});
describe("parseStoredVerdictRow", () => {
it("parses a valid stored verdict", () => {
const v = parseStoredVerdictRow({
flags: JSON.stringify({ status: "warn", flags: ["x"] }),
});
expect(v).not.toBeNull();
expect(v?.status).toBe("warn");
expect(v?.flags).toEqual(["x"]);
});
it("returns null on malformed JSON", () => {
expect(parseStoredVerdictRow({ flags: "{not-json" })).toBeNull();
});
it("derives legacy status from flags when status is absent", () => {
const v = parseStoredVerdictRow({
flags: JSON.stringify({ flags: ["a"] }),
});
expect(v?.status).toBe("flagged");
const v2 = parseStoredVerdictRow({ flags: JSON.stringify({}) });
expect(v2?.status).toBe("clean");
});
});
describe("getCachedTextModerations", () => {
it("returns an empty map for empty input and issues no query", async () => {
const result = await getCachedTextModerations([]);
expect(result.size).toBe(0);
expect(executeAll).not.toHaveBeenCalled();
});
it("dedupes keys and returns parsed verdicts keyed by cache key", async () => {
executeAll.mockResolvedValueOnce([
rowFor("text_mod:c1:aaa", "clean"),
rowFor("text_mod:c1:bbb", "warn"),
]);
const result = await getCachedTextModerations([
"text_mod:c1:aaa",
"text_mod:c1:aaa",
"text_mod:c1:bbb",
]);
expect(result.size).toBe(2);
expect(result.get("text_mod:c1:aaa")?.verdict.status).toBe("clean");
expect(result.get("text_mod:c1:bbb")?.verdict.status).toBe("warn");
// Exactly ONE batched round-trip.
expect(executeAll).toHaveBeenCalledTimes(1);
});
it("skips rows with malformed payloads instead of failing the batch", async () => {
executeAll.mockResolvedValueOnce([
{
text: "k_good",
flags: JSON.stringify({ status: "clean", flags: [] }),
analyzed_at: 1,
},
{ text: "k_bad", flags: "{broken" },
]);
const result = await getCachedTextModerations(["k_good", "k_bad"]);
expect(result.has("k_good")).toBe(true);
expect(result.has("k_bad")).toBe(false);
});
it("survives a DB failure and returns an empty map (fail-open)", async () => {
executeAll.mockRejectedValueOnce(new Error("connection refused"));
const result = await getCachedTextModerations(["k1", "k2"]);
expect(result.size).toBe(0);
});
it("chunks queries beyond 200 keys", async () => {
executeAll.mockResolvedValue([]);
const keys = Array.from({ length: 450 }, (_, i) => `k${i}`);
await getCachedTextModerations(keys);
expect(executeAll).toHaveBeenCalledTimes(3); // 200 + 200 + 50
});
});
@@ -0,0 +1,169 @@
// ═══════════════════════════════════════════════════════════════════════════
// Semantic two-band acceptance + global exact-cache reuse guard
// ═══════════════════════════════════════════════════════════════════════════
// Design (2026-08-24): cache hits may be served MORE aggressively for
// verdicts that cannot trigger enforcement actions, and NEVER more
// aggressively for actionable ones. Two layers enforce this:
// - isSemanticBandAccepted: similarity thresholds differ by verdict class
// (clean band 0.92 default vs strict actionable band 0.97 default).
// - isGloballyReusableCleanVerdict: context-free (cross-channel) reuse of
// the legacy bare key only for clean / flagless / action=none verdicts
// with high confidence and bounded age.
import { describe, expect, it } from "vitest";
import {
isGloballyReusableCleanVerdict,
type StoredModerationVerdict,
} from "../src/modules/ai-moderation/textCacheStore.js";
function makeVerdict(
overrides: Partial<StoredModerationVerdict> = {},
): StoredModerationVerdict {
return {
status: "clean",
flags: [],
score: 0,
analysis: "",
categories: [],
severity: "none",
confidence: 0.95,
recommendedAction: "none",
...overrides,
};
}
describe("isSemanticBandAccepted", () => {
it("accepts a non-actionable clean verdict at the loose clean band", () => {
// Default AI_LLM_EMBEDDING_MIN_SIMILARITY_CLEAN = 0.92.
expect(isBandAccept(makeVerdict(), 0.93)).toBe(true);
});
it("accepts a clean verdict exactly at the clean band boundary", () => {
expect(isBandAccept(makeVerdict({ confidence: 0.99 }), 0.92)).toBe(true);
});
it("rejects a clean verdict below the clean band", () => {
expect(isBandAccept(makeVerdict(), 0.91)).toBe(false);
});
it("rejects an actionable flagged verdict between the bands", () => {
// 0.93 >= clean band BUT < strict band → must NOT be served.
expect(
isBandAccept(
makeVerdict({ status: "flagged", flags: ["hate_speech"] }),
0.93,
),
).toBe(false);
});
it("accepts a flagged verdict at the strict band", () => {
expect(
isBandAccept(
makeVerdict({ status: "flagged", flags: ["hate_speech"] }),
0.98,
),
).toBe(true);
});
it("rejects a warn verdict below the strict band", () => {
expect(
isBandAccept(
makeVerdict({ status: "warn", recommendedAction: "warn" }),
0.96,
),
).toBe(false);
});
it("treats a clean verdict WITH flags as actionable (strict band)", () => {
expect(isBandAccept(makeVerdict({ flags: ["borderline"] }), 0.93)).toBe(
false,
);
});
it("treats a clean verdict with a non-none action as actionable", () => {
expect(
isBandAccept(makeVerdict({ recommendedAction: "review" }), 0.93),
).toBe(false);
});
});
// Import indirection so the describe block reads cleanly.
import { isSemanticBandAccepted as isBandAccept } from "../src/modules/ai-moderation/textCacheStore.js";
describe("isGloballyReusableCleanVerdict", () => {
it("accepts a fresh, confident, flagless clean verdict", () => {
const v = makeVerdict({ confidence: 0.9 });
expect(isGloballyReusableCleanVerdict(v, Date.now() - 60_000)).toBe(true);
});
it("rejects flagged / warn verdicts outright", () => {
expect(
isGloballyReusableCleanVerdict(
makeVerdict({ status: "flagged", flags: ["harassment"] }),
Date.now(),
),
).toBe(false);
expect(
isGloballyReusableCleanVerdict(
makeVerdict({ status: "warn", flags: ["mild"] }),
Date.now(),
),
).toBe(false);
});
it("rejects clean verdicts carrying flags", () => {
expect(
isGloballyReusableCleanVerdict(makeVerdict({ flags: ["x"] }), Date.now()),
).toBe(false);
});
it("rejects verdicts whose recommended action is not none", () => {
expect(
isGloballyReusableCleanVerdict(
makeVerdict({ recommendedAction: "delete" }),
Date.now(),
),
).toBe(false);
});
it("rejects low-confidence verdicts below the guard threshold", () => {
// Default AI_CACHE_GLOBAL_REUSE_MIN_CONFIDENCE = 0.85.
expect(
isGloballyReusableCleanVerdict(
makeVerdict({ confidence: 0.6 }),
Date.now(),
),
).toBe(false);
});
it("accepts confidence exactly at the guard threshold", () => {
expect(
isGloballyReusableCleanVerdict(
makeVerdict({ confidence: 0.85 }),
Date.now(),
),
).toBe(true);
});
it("rejects entries older than the freshness window", () => {
// Default AI_CACHE_GLOBAL_REUSE_MAX_AGE_H = 120h.
const tooOld = Date.now() - 121 * 60 * 60 * 1000;
expect(isGloballyReusableCleanVerdict(makeVerdict(), tooOld)).toBe(false);
const freshEnough = Date.now() - 119 * 60 * 60 * 1000;
expect(isGloballyReusableCleanVerdict(makeVerdict(), freshEnough)).toBe(
true,
);
});
it("skips the age check when analyzedAt is unknown", () => {
expect(isGloballyReusableCleanVerdict(makeVerdict(), undefined)).toBe(true);
});
it("rejects non-standard statuses such as processing write-backs", () => {
expect(
isGloballyReusableCleanVerdict(
{ ...makeVerdict(), status: "processing" },
undefined,
),
).toBe(false);
});
});
@@ -0,0 +1,89 @@
// ═══════════════════════════════════════════════════════════════════════════
// normalizeDiscordImageUrl — unified vision cache keys for Discord CDN URLs
// ═══════════════════════════════════════════════════════════════════════════
// Design (2026-08-24): the same attachment reached through different signed
// URLs (?ex=&is=&hm= tokens rotate per fetch) or render variants
// (?format=&width=) must map to ONE vision-cache key, otherwise the vision
// model re-downloads and re-analyzes the identical image once per variant.
import { describe, expect, it } from "vitest";
import {
makeImageCacheKey,
normalizeDiscordImageUrl,
} from "../src/modules/ai-moderation/textCacheStore.js";
describe("normalizeDiscordImageUrl", () => {
it("strips rotating signed tokens from cdn.discordapp.com URLs", () => {
const a = normalizeDiscordImageUrl(
"https://cdn.discordapp.com/attachments/1/2/img.png?ex=67a&is=67b&hm=tokA",
);
const b = normalizeDiscordImageUrl(
"https://cdn.discordapp.com/attachments/1/2/img.png?ex=78c&is=78d&hm=tokB",
);
expect(a).toBe("https://cdn.discordapp.com/attachments/1/2/img.png");
expect(a).toBe(b);
});
it("strips render variants from media.discordapp.net URLs", () => {
const a = normalizeDiscordImageUrl(
"https://media.discordapp.net/attachments/1/2/img.png?format=webp&width=400&height=300",
);
const b = normalizeDiscordImageUrl(
"https://media.discordapp.net/attachments/1/2/img.png?format=png&width=1024&height=768",
);
expect(a).toBe(b);
expect(a).toBe("https://media.discordapp.net/attachments/1/2/img.png");
});
it("strips query params from images-ext preview hosts", () => {
const a = normalizeDiscordImageUrl(
"https://images-ext-1.discordapp.net/external/X/https/example.com/cat.jpg?format=webp",
);
expect(a).toBe(
"https://images-ext-1.discordapp.net/external/X/https/example.com/cat.jpg",
);
});
it("leaves URLs without query untouched", () => {
const u = "https://cdn.discordapp.com/attachments/1/2/img.png";
expect(normalizeDiscordImageUrl(u)).toBe(u);
});
it("leaves non-Discord URLs untouched (query may be meaningful)", () => {
const u = "https://example.com/image?token=abc&id=1";
expect(normalizeDiscordImageUrl(u)).toBe(u);
});
it("leaves data: URLs untouched", () => {
const u = "data:image/png;base64,iVBORw0KGgoAAAANS?weird=query";
// new URL() parses data: with protocol "data:" → not http(s) → untouched.
expect(normalizeDiscordImageUrl(u)).toBe(u);
});
});
describe("makeImageCacheKey — Discord URL unification", () => {
it("produces the SAME key for the same attachment across token variants", () => {
const keyA = makeImageCacheKey(
"https://cdn.discordapp.com/attachments/9/8/pic.png?ex=111&is=222&hm=AAA",
);
const keyB = makeImageCacheKey(
"https://cdn.discordapp.com/attachments/9/8/pic.png?ex=333&is=444&hm=BBB",
);
expect(keyA).toBe(keyB);
});
it("still produces DIFFERENT keys for different attachments", () => {
const keyA = makeImageCacheKey(
"https://cdn.discordapp.com/attachments/9/8/a.png?ex=1",
);
const keyB = makeImageCacheKey(
"https://cdn.discordapp.com/attachments/9/8/b.png?ex=1",
);
expect(keyA).not.toBe(keyB);
});
it("preserves the historical full-hash behavior for non-Discord URLs", () => {
// Regression guard: external URLs keep pre-change keys.
const key = makeImageCacheKey("https://example.com/x.png");
expect(key).toBe(makeImageCacheKey("https://example.com/x.png"));
});
});
@@ -0,0 +1,79 @@
// ═══════════════════════════════════════════════════════════════════════════
// classifyIndividualWorkerResult — upload-pending vs success vs incomplete vs error
// ═══════════════════════════════════════════════════════════════════════════
// Bug (2026-08-24): the worker's upload-pending race guard returned
// {ok:true, results:[]}; the processor treated it as a completed moderation,
// leaving messages stuck in `processing` until the 300s cleanup reverted them.
import { describe, expect, it } from "vitest";
import { classifyIndividualWorkerResult } from "../src/modules/ai-moderation/fallbackResultClassifier.js";
describe("classifyIndividualWorkerResult", () => {
it("classifies the upload-pending race guard signal FIRST", () => {
expect(
classifyIndividualWorkerResult({
ok: true,
results: [],
uploadPending: true,
}),
).toBe("upload_pending");
});
it("classifies a normal verdict as success", () => {
expect(
classifyIndividualWorkerResult({
ok: true,
results: [{ status: "clean", flags: [] }],
}),
).toBe("success");
});
it("classifies analysis_incomplete as incomplete", () => {
expect(
classifyIndividualWorkerResult({
ok: true,
results: [{ status: "error", flags: ["analysis_incomplete"] }],
}),
).toBe("incomplete");
});
it("accepts flags as JSON string (DB round-trip shape)", () => {
expect(
classifyIndividualWorkerResult({
ok: true,
results: [
{ status: "error", flags: JSON.stringify(["analysis_incomplete"]) },
],
}),
).toBe("incomplete");
});
it("classifies ok:false as error", () => {
expect(
classifyIndividualWorkerResult({
ok: false,
results: [],
error: "boom",
}),
).toBe("error");
});
it("THE BUG: empty results with ok:true is an ERROR, not a success", () => {
// Old code silently succeeded here → stuck `processing` rows.
expect(classifyIndividualWorkerResult({ ok: true, results: [] })).toBe(
"error",
);
expect(
classifyIndividualWorkerResult({ ok: true, results: [undefined] }),
).toBe("error");
expect(classifyIndividualWorkerResult({})).toBe("error");
});
it("treats a result with no status as unexplainable (error)", () => {
expect(
classifyIndividualWorkerResult({
ok: true,
results: [{ flags: [] }],
}),
).toBe("error");
});
});
@@ -0,0 +1,91 @@
# FE Optimasi: Error Handling, State Refactor, Polish
**Scope:** `services/frontend/src` — Next.js 16, React 19, SWR, oRPC-over-WS.
**Branch:** `feat/fe-error-handling-state` (buat baru dari `main`).
**User:** MythEclipse — bahasa Indonesia/Inggris campur, "gas" = eksekusi langsung.
## 1. Masalah yang teridentifikasi
### 1.1 Error handling tidak seragam
- `ErrorState` (states.tsx:126) sudah ada dengan `onRetry?`, tapi **80% pemanggilan tidak pass `onRetry`** → user stuck, harus refresh manual.
- Pola `if (error && !data) return <ErrorState />` berulang di 5+ view, setiap satu hardcode logic-nya.
- `useAction` (use-action.ts) set `error` di state tapi komponen pakai `try/catch` lokal juga — **dua error states** (hook `action.error` + lokal `catch (e)`).
### 1.2 State duplication / double source of truth
- `useReview` (use-messages.ts:129) pakai `refreshInterval: 15_000` (polling) padahal WS sudah broadcast `moderation_action` real-time. **Polling + WS = double-fetch + race condition.**
- `page.tsx` dan `view.tsx` duplikat state `guildId`/`channelId`/`query` — logic selection ada di 2 tempat.
- Chatbot pakai local `failed` state + `setFailed(text)` untuk retry, tapi `useAction` sudah ada `error` field yang tidak dipakai.
- `useSpeakers` (use-voice.ts:50) pakai local `useState` untuk speakers, bukan SWR → tidak konsisten dengan pola SWR yang dipakai semua hook lain.
### 1.3 WebSocket error/user feedback lemah
- `WsConnection.onerror` (connection.ts:78) cuma `setStatus("error")` — tidak ada info ke user "reconnecting… (attempt N)".
- Tidak ada broadcast error ke komponen (tidak ada `ws_error` handler).
## 2. Rencana perubahan
### 2.1 Standardisasi ErrorBoundary + Retry (FE-only)
**File:** `src/components/shared/states.tsx`
- Tambahkan komponen `ErrorBoundary` (React error boundary untuk crash React, bukan fetch error).
- Refactor semua `<ErrorState error={error} />` → `<ErrorState error={error} onRetry={retryFn} />`.
- Tambahkan helper `withSWR` atau pattern: tiap view pakai `error` + `mutate`/`refetch` dari SWR dan pass ke ErrorState.
**File:** `src/app/(dashboard)/*/view.tsx` (6 files)
- Setiap `ErrorState` dapat `onRetry` yang memanggil `mutate`/`refetch`.
- `DashboardView`: `onRetry={() => void mutate(["dashboard-stats"])}` — tapi SWR keys tersebar. Solusi: export `mutate` via custom hook atau pakai `useSWR` config `onErrorRetry` global.
- **Keputusan:** gunakan pola **global SWR config** (`src/lib/swr-config.ts`) dengan `onErrorRetry` backoff, dan tiap view pass `onRetry` explicit ke ErrorState.
### 2.2 Hapus polling `useReview`, ganti WS-driven
**File:** `src/hooks/use-messages.ts`
- Hapus `refreshInterval: 15_000` dari `useReview`.
- Tambahkan `useReviewWsSync(ws)` — subscribe ke `moderation_action` WS event, mutate key `["messages-review", channelId]`.
- Backend sudah broadcast `moderation_action` via WS (redis-channels.ts:130). Review messages yang di-flag akan dapat `moderation_action` event. **Bisa pakai ini.**
**File:** `src/app/(dashboard)/messages/view.tsx`
- Tambahkan `useReviewWsSync(ws)` call.
### 2.3 Konsistensi state: `useSpeakers` → SWR
**File:** `src/hooks/use-voice.ts`
- Refactor `useSpeakers` agar pakai SWR key `["voice-speakers"]` + initialData dari `useVoiceStatus`. Ini memungkinkan revalidate + cache sharing.
- Tapi `voice_state` dan `voice_active_user` adalah WS events — perlu persist ke SWR cache via `mutate`. Refactor: `useSpeakers` subscribe WS + mutate SWR key.
### 2.4 Chatbot: gunakan `useAction` error state, hapus duplikat `failed`
**File:** `src/components/chatbot/chatbot.tsx`
- Hapus `const [failed, setFailed]` — sebalihoikan ke `useAction` return `error`.
- Tapi `useAction` `mutateAsync` throw — perlu catch. Refactor: pakai `mutate` (fire-and-forget) + `isPending` + `error`.
- **Note:** chatbot pakai `send` yang butuh pemuatan history — tetap pakai local state untuk msgs tapi gunakan `action.error` untuk display.
### 2.5 WS error feedback
**File:** `src/lib/ws/connection.ts`
- `onerror` emit kode/status ke status listeners.
- Tambahkan method `getReconnectAttempt()` atau expose via status change.
**File:** `src/lib/ws/context.tsx`
- Subscribe `onStatusChange` di `WsProvider`, toast "Reconnecting… (attempt N)" ketika status `error`/`connecting`.
## 3. Verification
- `npx tsc --noEmit` — compile OK
- `npx biome check src/` — lint OK
- `npm run build` — build OK (Next 16 SSG/SSR)
- Manual: refresh halaman, pastikan ErrorState muncul dengan tombol Retry yang bisa diklik.
## 4. Files yang disentuh
```
src/components/shared/states.tsx # ErrorBoundary + helper
src/lib/swr-config.ts # global SWR config (baru)
src/hooks/use-messages.ts # useReviewWsSync, hapus polling
src/hooks/use-voice.ts # useSpeakers → SWR
src/hooks/use-action.ts # expose resetError
src/lib/ws/connection.ts # reconnect info
src/lib/ws/context.tsx # WS error toast
src/app/(dashboard)/messages/view.tsx # useReviewWsSync
src/app/(dashboard)/messages/page.tsx # SSR error passthrough
src/app/(dashboard)/moderation/view.tsx # onRetry
src/app/(dashboard)/dashboard/view.tsx # onRetry
src/app/(dashboard)/voice/view.tsx # onRetry
src/app/(dashboard)/media/view.tsx # onRetry
src/app/(dashboard)/recordings/view.tsx # onRetry
src/components/chatbot/chatbot.tsx # pakai useAction error
```
## 5. Out of scope (BE)
- Regex heuristic di moderation.repository.ts (scam domain extraction) — ini BE task, catat tapi jangan sentuh kecuali diminta.
@@ -50,7 +50,12 @@ export function DashboardView({
initialStats?: DashboardStats;
initialActivity?: Awaited<ReturnType<typeof useActivity>>["data"];
}) {
const { data: stats, isLoading, error } = useStats(initialStats);
const {
data: stats,
isLoading,
error,
mutate: mutateStats,
} = useStats(initialStats);
const { data: activity } = useActivity(14, initialActivity as never);
const { data: reactors } = useTopReactors();
const { data: reactions } = useTopReactions();
@@ -65,7 +70,8 @@ export function DashboardView({
);
}, [stats, ambient]);
if (error && !stats) return <ErrorState error={error} />;
if (error && !stats)
return <ErrorState error={error} onRetry={() => void mutateStats()} />;
if (!stats && isLoading)
return (
<div className="space-y-5">
@@ -78,7 +84,13 @@ export function DashboardView({
</div>
</div>
);
if (!stats) return <ErrorState error={error ?? new Error("No data")} />;
if (!stats)
return (
<ErrorState
error={error ?? new Error("No data")}
onRetry={() => void mutateStats()}
/>
);
const s = stats;
const total = s.total_flagged + s.total_clean || 1;
@@ -87,7 +99,7 @@ export function DashboardView({
return (
<div className="space-y-5">
{/* Hero */}
<GlassPanel glow className="relative overflow-hidden">
<GlassPanel glow className="game-frame relative overflow-hidden">
<div className="scan-line absolute inset-x-0 top-0" />
<div className="flex flex-wrap items-end justify-between gap-4">
<div>
@@ -33,7 +33,12 @@ import { useWebSocket } from "@/lib/ws/context";
export function MediaView({ initialStatus }: { initialStatus?: MediaState }) {
const ws = useWebSocket();
const { data: media, isLoading, error } = useMediaState(initialStatus);
const {
data: media,
isLoading,
error,
mutate,
} = useMediaState(initialStatus);
const queue = useMediaQueue();
const skip = useMediaSkip();
const stop = useMediaStop();
@@ -79,7 +84,8 @@ export function MediaView({ initialStatus }: { initialStatus?: MediaState }) {
}
};
if (error && !media) return <ErrorState error={error} />;
if (error && !media)
return <ErrorState error={error} onRetry={() => void mutate()} />;
if (!media && isLoading)
return (
<div className="space-y-5">
@@ -39,6 +39,7 @@ import {
useMessagesStream,
useMessagesWsSync,
useRecentEdits,
useReviewWsSync,
useSemanticSearch,
} from "@/hooks";
import { aiTone } from "@/lib/ai-status";
@@ -94,6 +95,7 @@ export function MessagesView({
data: messages,
isLoading,
error,
refetch,
} = useMessages(
guildId ?? "",
channelId ?? undefined,
@@ -112,6 +114,7 @@ export function MessagesView({
const hasMore = pageInfo?.hasMore ?? false;
const loadMore = useLoadMore();
useMessagesWsSync(ws, guildId ?? "");
useReviewWsSync(ws);
const search = useMessageSearch(
query,
query.trim().length >= 2 && !semanticMode,
@@ -326,7 +329,7 @@ export function MessagesView({
}
/>
{error && !messages ? (
<ErrorState error={error} />
<ErrorState error={error} onRetry={() => refetch()} />
) : isLoading && !messages ? (
<SkeletonRows rows={8} />
) : list.length === 0 ? (
@@ -81,7 +81,12 @@ export function ModerationView({
initialStats?: ModerationStats;
initialActions?: ModerationAction[];
}) {
const { data: stats, isLoading, error } = useModerationStats(initialStats);
const {
data: stats,
isLoading,
error,
mutate: mutateStats,
} = useModerationStats(initialStats);
const [statusFilter, setStatusFilter] = useState<string>("");
const [typeFilter, setTypeFilter] = useState<string>("");
const { data: actions } = useModerationActions(
@@ -122,7 +127,8 @@ export function ModerationView({
);
}, [failedRate, ambient]);
if (error && !stats) return <ErrorState error={error} />;
if (error && !stats)
return <ErrorState error={error} onRetry={() => void mutateStats()} />;
if (!stats && isLoading)
return (
<div className="space-y-5">
@@ -131,7 +137,13 @@ export function ModerationView({
<SkeletonRows rows={6} />
</div>
);
if (!stats) return <ErrorState error={error ?? new Error("No data")} />;
if (!stats)
return (
<ErrorState
error={error ?? new Error("No data")}
onRetry={() => void mutateStats()}
/>
);
const statusOpts: SelectOption[] = [
{ value: "", label: "All statuses" },
@@ -33,7 +33,7 @@ export function RecordingsView({
initialItems?: VoiceRecording[];
}) {
const ws = useWebSocket();
const { data: items, isLoading, error } = useRecordings(initialItems);
const { data: items, isLoading, error, mutate } = useRecordings(initialItems);
const del = useDeleteRecording();
useRecordingsWsSync(ws);
const ambient = useAmbient();
@@ -56,7 +56,8 @@ export function RecordingsView({
}
};
if (error && !items) return <ErrorState error={error} />;
if (error && !items)
return <ErrorState error={error} onRetry={() => void mutate()} />;
if (!items && isLoading)
return (
<GlassPanel>
@@ -40,7 +40,12 @@ export function VoiceView({
initialGuilds?: Guild[];
}) {
const ws = useWebSocket();
const { data: status, isLoading, error } = useVoiceStatus(initialStatus);
const {
data: status,
isLoading,
error,
mutate,
} = useVoiceStatus(initialStatus);
const connect = useVoiceConnect();
const disconnect = useVoiceDisconnect();
const mic = useMicTransmit(ws);
@@ -68,7 +73,8 @@ export function VoiceView({
else ambient.set("vermilion", 0.35, "voice idle");
}, [status?.connected, ambient]);
if (error && !status) return <ErrorState error={error} />;
if (error && !status)
return <ErrorState error={error} onRetry={() => void mutate()} />;
if (!status && isLoading)
return (
<div className="space-y-5">
+148 -10
View File
@@ -17,7 +17,7 @@
/* ── Surfaces (dark-first; light overrides below) ── */
--color-canvas: oklch(0.12 0.014 70);
--color-canvas-2: oklch(0.16 0.02 70);
--color-surface: oklch(0.2 0.022 70 / 0.55);
--color-surface: oklch(0.2 0.022 70 / 0.45);
--color-surface-2: oklch(0.26 0.024 70 / 0.45);
/* ── Ink ── */
@@ -29,13 +29,13 @@
--color-hairline: oklch(1 0 0 / 0.1);
--hairline-w: 1px;
/* ── Semantic signals ── */
--color-signal: oklch(0.86 0.19 128);
--color-signal-ink: oklch(0.18 0.03 70);
--color-signal-glow: oklch(0.86 0.19 128 / 0.4);
--color-amber: oklch(0.85 0.15 72);
--color-vermilion: oklch(0.68 0.21 25);
--color-vermilion-glow: oklch(0.68 0.21 25 / 0.4);
/* ── Semantic signals (monochrome: tone = luminance steps) ── */
--color-signal: oklch(0.97 0 0);
--color-signal-ink: oklch(0.1 0 0);
--color-signal-glow: oklch(1 0 0 / 0.35);
--color-amber: oklch(0.72 0 0);
--color-vermilion: oklch(0.85 0 0);
--color-vermilion-glow: oklch(1 0 0 / 0.45);
--color-ring: var(--color-signal);
@@ -74,8 +74,8 @@
background: oklch(0.22 0.02 70 / 0.34);
}
.light ::selection {
background: var(--color-signal-glow);
color: var(--color-ink);
background: oklch(0.2 0 0 / 0.9);
color: oklch(1 0 0);
}
@layer base {
@@ -300,6 +300,144 @@
}
}
/* ═══ Game-menu nav (monochrome) ═══
Corner brackets + one-shot light sweep + staggered entrance.
Pure transform/opacity — compositor only. */
.game-nav-item {
--i: 0;
position: relative;
isolation: isolate;
animation: nav-in 0.45s cubic-bezier(0.22, 1, 0.36, 1) both;
animation-delay: calc(var(--i) * 45ms);
}
/* corner brackets on hover/active */
.game-nav-item::before,
.game-nav-item::after {
content: "";
position: absolute;
inset: 0;
opacity: 0;
transition: opacity 0.2s ease;
pointer-events: none;
}
.game-nav-item::before {
border-left: 2px solid currentColor;
border-top: 2px solid currentColor;
border-top-left-radius: 13px;
clip-path: polygon(0 0, 62% 0, 62% 30%, 30% 30%, 30% 100%, 0 100%);
}
.game-nav-item::after {
border-right: 2px solid currentColor;
border-bottom: 2px solid currentColor;
border-bottom-right-radius: 13px;
clip-path: polygon(100% 100%, 38% 100%, 38% 70%, 70% 70%, 70% 0, 100% 0);
}
.game-nav-item:hover::before,
.game-nav-item:hover::after,
.game-nav-item.is-active::before,
.game-nav-item.is-active::after {
opacity: 1;
}
/* one-shot sweep */
.game-nav-item .game-sweep {
position: absolute;
inset: 0;
overflow: hidden;
border-radius: inherit;
pointer-events: none;
}
.game-nav-item .game-sweep > i {
position: absolute;
top: -20%;
bottom: -20%;
width: 34%;
left: -60%;
background: linear-gradient(
105deg,
transparent,
oklch(1 0 0 / 0.16),
transparent
);
transform: skewX(-14deg);
}
.game-nav-item:hover .game-sweep > i,
.game-nav-item.is-active .game-sweep > i {
animation: sweep-x 0.55s ease-out both;
}
/* active triangle marker (game cursor) */
.game-marker {
position: absolute;
top: 50%;
width: 0;
height: 0;
border-block: 7px solid transparent;
border-left: 11px solid currentColor;
transform: translateY(-50%);
filter: drop-shadow(0 0 6px var(--color-signal-glow));
animation: marker-nudge 1.6s ease-in-out infinite;
}
@keyframes marker-nudge {
0%, 100% { transform: translateY(-50%) translateX(0); }
50% { transform: translateY(-50%) translateX(3px); }
}
@keyframes sweep-x {
from { left: -60%; }
to { left: 130%; }
}
@keyframes nav-in {
from { opacity: 0; transform: translateX(-10px); }
to { opacity: 1; transform: translateX(0); }
}
/* ── Game frame: panel with cut corners + drawn-in edge lines ── */
.game-frame {
position: relative;
}
.game-frame::before {
content: "";
position: absolute;
inset: 0;
border-radius: inherit;
pointer-events: none;
background:
linear-gradient(var(--color-hairline), var(--color-hairline)) top left /
26px 1.5px,
linear-gradient(var(--color-hairline), var(--color-hairline)) top left / 1.5px
26px,
linear-gradient(var(--color-hairline), var(--color-hairline)) top right /
26px 1.5px,
linear-gradient(var(--color-hairline), var(--color-hairline)) top right / 1.5px
26px,
linear-gradient(var(--color-hairline), var(--color-hairline)) bottom left /
26px 1.5px,
linear-gradient(var(--color-hairline), var(--color-hairline)) bottom left /
1.5px 26px,
linear-gradient(var(--color-hairline), var(--color-hairline)) bottom right /
26px 1.5px,
linear-gradient(var(--color-hairline), var(--color-hairline)) bottom right /
1.5px 26px;
background-repeat: no-repeat;
}
/* ── Mobile lightness (<md): cheaper blur, shorter animations ── */
@media (max-width: 767px) {
.glass {
backdrop-filter: blur(8px);
-webkit-backdrop-filter: blur(8px);
}
.game-nav-item {
animation-duration: 0.25s;
animation-delay: calc(var(--i) * 18ms);
}
.scan-line::after,
.animate-shimmer {
display: none;
}
}
@media (prefers-reduced-motion: reduce) {
.scan-line::after,
.scan-line,
@@ -67,6 +67,19 @@ export function AmbientCanvas({
const mount = mountRef.current;
if (!mount) return;
// Lightness gate: skip the WebGL layer entirely on small/coarse/data-saver
// devices — the static CSS fallback below stays. Desktop keeps the haze.
const nav = navigator as Navigator & {
connection?: { saveData?: boolean; deviceMemory?: number };
deviceMemory?: number;
};
const lightDevice =
window.matchMedia("(max-width: 767px)").matches ||
window.matchMedia("(pointer: coarse)").matches ||
nav.connection?.saveData === true ||
(nav.deviceMemory ?? 8) <= 4;
if (lightDevice) return;
let renderer: THREE.WebGLRenderer;
try {
renderer = new THREE.WebGLRenderer({
@@ -12,11 +12,14 @@ import { AmbientCanvas } from "./ambient-canvas";
export type SignalTone = "signal" | "amber" | "vermilion";
/** sRGB triplets for the three semantic signals (matches globals.css). */
/**
* sRGB triplets for the three semantic signals — MONOCHROME build:
* tone is expressed as luminance/intensity, not hue (black & white theme).
*/
export const SIGNAL_RGB: Record<SignalTone, [number, number, number]> = {
signal: [0.42, 1.0, 0.52],
amber: [1.0, 0.76, 0.28],
vermilion: [1.0, 0.34, 0.28],
signal: [0.92, 0.94, 0.96], // nominal — near-white
amber: [0.62, 0.64, 0.68], // warn — mid grey
vermilion: [0.78, 0.8, 0.84], // danger — bright flash grey
};
export interface AmbientState {
@@ -0,0 +1,56 @@
"use client";
import { Component, type ReactNode } from "react";
import { ErrorState } from "./states";
/**
* React error boundary — catches render/exception crashes in child trees
* (e.g. third-party lib throwing on unexpected payload shape) and surfaces
* a consistent ErrorState instead of unmounting the whole app shell.
*
* Usage: wrap leaf views in <ErrorBoundary><SomeView /></ErrorBoundary>.
*/
interface ErrorBoundaryProps {
children: ReactNode;
fallback?: (error: Error, reset: () => void) => ReactNode;
onReset?: () => void;
}
interface ErrorBoundaryState {
error: Error | null;
}
export class ErrorBoundary extends Component<
ErrorBoundaryProps,
ErrorBoundaryState
> {
constructor(props: ErrorBoundaryProps) {
super(props);
this.state = { error: null };
}
static getDerivedStateFromError(error: Error): ErrorBoundaryState {
return { error };
}
reset = () => {
this.setState({ error: null });
this.props.onReset?.();
};
override render() {
if (!this.state.error) return this.props.children;
if (this.props.fallback) {
return this.props.fallback(this.state.error, this.reset);
}
return (
<ErrorState
title="Something went wrong"
error={this.state.error}
onRetry={this.reset}
/>
);
}
}
@@ -2,6 +2,7 @@ export { GuildChannelPicker } from "./guild-picker";
export { MarkdownLite } from "./markdown";
export { PageTransition } from "./page-transition";
export { MetricTile, SectionHeader } from "./section";
export { ErrorBoundary } from "./error-boundary";
export {
EmptyState,
ErrorState,
@@ -24,16 +24,25 @@ export function MobileNav() {
aria-label={item.label}
aria-current={active ? "page" : undefined}
className={cn(
"relative flex flex-1 flex-col items-center justify-center rounded-[12px] rounded-t-[20px] px-2 pb-[calc(0.4rem+env(safe-area-inset-bottom))] pt-2 md:hidden",
"game-nav-item relative flex min-h-[44px] flex-1 flex-col items-center justify-center rounded-t-[20px] px-2 pt-2 pb-[calc(0.4rem+env(safe-area-inset-bottom))] md:hidden",
active
? "bg-signal/15 text-signal"
: "text-ink-faint hover:bg-white/5 hover:text-ink-soft",
? "is-active text-white"
: "text-ink-faint hover:text-ink-soft",
)}
>
{active && (
<span className="absolute -top-2 h-8 w-8 rounded-full bg-signal shadow-[0_0_12px_var(--color-signal-glow)]" />
<span
aria-hidden="true"
className="absolute inset-x-4 top-0 h-[2px] bg-white shadow-[0_0_8px_var(--color-signal-glow)]"
/>
)}
<item.icon className="size-[20px]" strokeWidth={active ? 2.4 : 2} />
<span className="game-sweep" aria-hidden="true">
<i />
</span>
<item.icon
className="relative z-10 size-[20px]"
strokeWidth={active ? 2.4 : 2}
/>
{item.label}
</Link>
);
@@ -9,30 +9,37 @@ function NavItem({
label,
active,
Icon,
index,
}: {
href: string;
label: string;
active: boolean;
Icon: React.ComponentType<React.SVGProps<SVGSVGElement>>;
index: number;
}) {
return (
<a
href={href}
aria-label={label}
aria-current={active ? "page" : undefined}
style={{ "--i": index } as React.CSSProperties}
className={cn(
"group relative flex size-11 items-center justify-center rounded-[13px] transition-all",
"game-nav-item group flex size-11 items-center justify-center rounded-[13px] transition-colors",
active
? "bg-signal/15 text-signal"
? "is-active bg-white/10 text-white"
: "text-ink-faint hover:bg-white/5 hover:text-ink-soft",
)}
>
{active && (
<span className="absolute -left-3 h-6 w-1 rounded-full bg-signal shadow-[0_0_12px_var(--color-signal-glow)]" />
)}
<Icon className="size-[18px]" strokeWidth={active ? 2.4 : 2} />
{active && <span className="game-marker -left-3.5" />}
<span className="game-sweep" aria-hidden="true">
<i />
</span>
<Icon
className="relative z-10 size-[18px]"
strokeWidth={active ? 2.4 : 2}
/>
{/* hover tooltip — labels are hidden in the rail, so surface on hover */}
<span className="pointer-events-none absolute left-full z-50 ml-3 hidden whitespace-nowrap rounded-[9px] border border-hairline bg-canvas-2 px-2.5 py-1.5 text-xs font-medium text-ink-soft opacity-0 shadow-lg transition-opacity group-hover:opacity-100 md:block">
<span className="pointer-events-none absolute left-full z-50 ml-3 hidden whitespace-nowrap rounded-[9px] border border-hairline bg-canvas-2 px-2.5 py-1.5 font-mono text-xs font-medium text-ink-soft opacity-0 shadow-lg transition-opacity group-hover:opacity-100 md:block">
{label}
</span>
</a>
@@ -46,11 +53,12 @@ export function NavRail() {
return (
<nav className="glass mb-[calc(0.75rem+env(safe-area-inset-bottom))] ml-[calc(0.75rem+env(safe-area-inset-left))] mt-[calc(0.75rem+env(safe-area-inset-top))] hidden w-[68px] flex-col items-center gap-1 rounded-[18px] py-4 md:flex">
<div className="flex flex-1 flex-col gap-1">
{navItems.map((item) => (
{navItems.map((item, i) => (
<NavItem
key={item.href}
href={item.href}
label={item.label}
index={i}
active={isActivePath(path, item.matchPrefix)}
Icon={item.icon}
/>
+1
View File
@@ -31,6 +31,7 @@ export {
useMessagesWsSync,
useRecentEdits,
useReview,
useReviewWsSync,
useSemanticSearch,
useTextChannels,
} from "./use-messages";
+14 -4
View File
@@ -7,17 +7,23 @@ export interface UseActionState {
/**
* A lightweight mutation hook with a TanStack-compatible surface
* ({ mutate, mutateAsync, isPending, error }) built on plain state —
* ({ mutate, mutateAsync, isPending, error, resetError }) built on plain state —
* the SWR replacement for useMutation. Fire-and-forget via `mutate`,
* await the result via `mutateAsync`.
*
* `onSuccess` receives (data, args) and may perform SWR cache updates
* (e.g. `mutate(key, data, { revalidate: false })`).
*
* `onError` receives the caught error and args — use it for side-effect
* logging without throwing (the error is also surfaced via `state.error`).
*
* `resetError` clears the error state without re-running the action.
*/
export function useAction<TArgs = void, TResult = unknown>(
fn: (args: TArgs) => Promise<TResult>,
options?: {
onSuccess?: (data: TResult, args: TArgs) => void | Promise<void>;
onError?: (error: Error, args: TArgs) => void | Promise<void>;
},
) {
const [state, setState] = useState<UseActionState>({
@@ -29,6 +35,8 @@ export function useAction<TArgs = void, TResult = unknown>(
fnRef.current = fn;
const onSuccessRef = useRef(options?.onSuccess);
onSuccessRef.current = options?.onSuccess;
const onErrorRef = useRef(options?.onError);
onErrorRef.current = options?.onError;
const run = useCallback(async (args?: TArgs): Promise<TResult> => {
setState({ isPending: true, error: null });
@@ -38,8 +46,10 @@ export function useAction<TArgs = void, TResult = unknown>(
setState({ isPending: false, error: null });
return data;
} catch (err) {
setState({ isPending: false, error: err as Error });
throw err;
const error = err as Error;
setState({ isPending: false, error });
await onErrorRef.current?.(error, args as TArgs);
throw error;
}
}, []);
@@ -50,6 +60,6 @@ export function useAction<TArgs = void, TResult = unknown>(
mutateAsync: run,
isPending: state.isPending,
error: state.error,
reset: () => setState({ isPending: false, error: null }),
resetError: useCallback(() => setState((s) => ({ ...s, error: null })), []),
};
}
+39 -11
View File
@@ -124,21 +124,43 @@ export function useImages(guildId: string) {
);
}
// ── Review ───────────────────────────────────────
// ── Review ────────────────────────────────────────
// WS-driven now (was: refreshInterval:15000 polling). The backend broadcasts
// `moderation_action` events over WS whenever a moderation action is created;
// the review list is revalidated on that event instead of polling.
export function useReview(channelId?: string) {
const key = msgKeys.review(channelId);
return useSWR<MessageRecord[]>(
msgKeys.review(channelId),
key,
async () => {
const result = await messagesApi.getReview(50, channelId || undefined);
return result.results;
},
{
refreshInterval: 15_000,
revalidateOnFocus: true,
// No more polling — WS sync handles real-time updates.
},
);
}
/**
* Subscribe to WS `moderation_action` events and invalidate the review
* SWR cache. Replaces the old 15-second polling interval that caused
* duplicate requests and race conditions with WS updates.
*/
export function useReviewWsSync(ws: WsHook, channelId?: string) {
const { mutate } = useSWRConfig();
const key = msgKeys.review(channelId);
useEffect(() => {
const unsub = ws.on("moderation_action", () => {
void mutate(key, undefined, { revalidate: true });
});
return unsub;
}, [ws, key, mutate]);
}
// ── Detail ───────────────────────────────────────
export function useMessageDetail(id: string | null) {
@@ -168,7 +190,11 @@ export function useMessageDetail(id: string | null) {
message: detail.data ?? null,
attachments: attachments.data ?? [],
loading: detail.isLoading || attachments.isLoading,
error: detail.error,
error: detail.error ?? attachments.error,
refetch: () => {
void detail.mutate();
void attachments.mutate();
},
};
}
@@ -216,7 +242,7 @@ export function useMessagesWsSync(ws: WsHook, guildId: string) {
const { mutate } = useSWRConfig();
useEffect(() => {
if (!guildId) return;
// Patch every message-list key for this guild (all channels + "**filtered**").
// Patch every message-list key for this guild (all channels + "__all__").
// The updater receives the SWR key so we can honor its channel filter:
// a live `message_created`/updated for channel B must NOT be prepended to
// a list that is filtered down to channel A.
@@ -255,13 +281,13 @@ export function useMessagesWsSync(ws: WsHook, guildId: string) {
});
const unsub2 = ws.on("message_updated", (data) => {
const msg = data as Partial<MessageRecord> & { id: string };
// The gateway broadcasts a PARTIAL update ({ id, edited_content,
// edited_at, ... }) — merge it over the existing record instead of
// replacing it, or the card would lose username/content/channel/etc.
// The gateway broadcasts a PARTIAL update ({ id } plus the changed fields
// — merge it over the existing record instead of replacing it, or the
// card would lose username/content/channel/etc.
patchLists(
(_k, m) =>
(m as Partial<MessageRecord>).channel_id === undefined ||
matchesFilter(_k as unknown[], m),
matchesFilter(_k as unknown[], m as { channel_id?: string }),
(old) =>
old
? {
@@ -271,7 +297,7 @@ export function useMessagesWsSync(ws: WsHook, guildId: string) {
),
}
: old,
msg,
msg as { channel_id?: string },
);
void mutate(
msgKeys.detail(msg.id),
@@ -321,7 +347,7 @@ export function useMessagesWsSync(ws: WsHook, guildId: string) {
* `message_snapshot` into the SWR list as it arrives, so the UI renders
* progressively. Falls back to the batched `messagesApi.list` if WS is down.
*
* Returns: { streaming, streamed, error }.
* Returns: { streaming, error }.
*/
export function useMessagesStream(
ws: WsHook,
@@ -414,3 +440,5 @@ export function useRecentEdits(
{ fallbackData: initialData },
);
}
export { msgKeys };
+38 -24
View File
@@ -4,19 +4,13 @@ import { useAction } from "@/hooks/use-action";
import { voiceApi } from "@/lib/api";
import { MicTransmitter } from "@/lib/audio/mic-transmit";
import { PcmPlayer } from "@/lib/audio/pcm-player";
import { hashUserId } from "@/lib/hash";
import type { ActiveSpeaker, Channel, VoiceStatus } from "@/lib/types";
import type { PcmChunk } from "@/lib/ws/types";
import type { WsHook } from "@/lib/ws-hook";
/** FNV-1a 32-bit — same hash the gateway uses to tag PCM frames. */
export function hashUserId(userId: string): number {
let hash = 0x811c9dc5;
for (let i = 0; i < userId.length; i++) {
hash ^= userId.charCodeAt(i);
hash = Math.imul(hash, 0x01000193);
}
return hash >>> 0;
}
// Re-export for components that still import hashUserId from this module.
export { hashUserId };
const STATUS_KEY = ["voice-status"] as const;
@@ -34,6 +28,8 @@ export function useVoiceChannels(guildId: string) {
);
}
const SPEAKERS_KEY = ["voice-speakers"] as const;
/**
* Live shared speaker state.
*
@@ -44,48 +40,66 @@ export function useVoiceChannels(guildId: string) {
* every client with the same list);
* - `voice_active_user` → incremental upsert of a single speaker delta.
*
* Now backed by SWR (consistent with all other hooks) so cache, revalidation,
* and deduping apply. WS events mutate the SWR cache directly
* ({ revalidate: false }) to avoid refetching the full status.
*
* This replaces the old per-browser model where each tab accumulated speakers
* only from events it happened to receive while mounted.
*/
export function useSpeakers(initialStatusActive?: ActiveSpeaker[]) {
const [speakers, setSpeakers] = useState<ActiveSpeaker[]>(
initialStatusActive ?? [],
);
const {
data: speakers,
error,
mutate,
isValidating,
} = useSWR<ActiveSpeaker[]>(SPEAKERS_KEY, () => Promise.resolve([]), {
fallbackData: initialStatusActive ?? [],
revalidateOnMount: false,
revalidateOnFocus: false,
revalidateOnReconnect: false,
});
const subscribe = useCallback((ws: WsHook) => {
const subscribe = useCallback(
(ws: WsHook) => {
const unsubSnapshot = ws.on("voice_state", (data) => {
const state = data as { activeSpeakers?: ActiveSpeaker[] };
if (Array.isArray(state?.activeSpeakers)) {
setSpeakers(state.activeSpeakers);
void mutate(state.activeSpeakers, { revalidate: false });
}
});
const unsub = ws.on("voice_active_user", (data) => {
const speaker = data as ActiveSpeaker;
setSpeakers((prev) => {
const idx = prev.findIndex((s) => s.userId === speaker.userId);
void mutate(
(prev: ActiveSpeaker[] | undefined) => {
const arr = prev ?? [];
const idx = arr.findIndex((s) => s.userId === speaker.userId);
if (idx >= 0) {
const next = [...prev];
const next = [...arr];
next[idx] = speaker;
return next;
}
return [...prev, speaker];
});
return [...arr, speaker];
},
{ revalidate: false },
);
});
return () => {
unsubSnapshot();
unsub();
setSpeakers([]);
};
}, []);
},
[mutate],
);
return { speakers, subscribe };
return { speakers: speakers ?? [], subscribe, error, isValidating };
}
function useStatusInvalidator() {
const { mutate } = useSWRConfig();
return () => {
return useCallback(() => {
void mutate(STATUS_KEY);
};
}, [mutate]);
}
export function useVoiceConnect() {
+12
View File
@@ -0,0 +1,12 @@
/**
* FNV-1a 32-bit hash — same hash the gateway uses to tag PCM frames.
* Shared between use-voice hooks and ambient-canvas.
*/
export function hashUserId(userId: string): number {
let hash = 0x811c9dc5;
for (let i = 0; i < userId.length; i++) {
hash ^= userId.charCodeAt(i);
hash = Math.imul(hash, 0x01000193);
}
return hash >>> 0;
}
+4 -2
View File
@@ -75,6 +75,8 @@ export const mobileNavItems: NavItem[] = navItems.filter((item) =>
export type NavItemId = (typeof navItems)[number]["href"];
export function isActivePath(pathname: string, matchPrefix: string): boolean {
if (matchPrefix === "/dashboard") return pathname === "/dashboard";
return pathname.startsWith(matchPrefix);
// trailingSlash builds surface "/x/" via usePathname — normalize first.
const path = pathname.replace(/\/+$/, "") || "/";
if (matchPrefix === "/dashboard") return path === "/dashboard";
return path.startsWith(matchPrefix);
}
+47
View File
@@ -0,0 +1,47 @@
/**
* Global SWR configuration for the GMW frontend.
*
* Centralizes revalidation policy, error retry with exponential backoff, and
* deduplication so every `useSWR` call across the app gets consistent behaviour
* without each hook re-declaring the same options.
*
* Import this in the root layout (or any component mounted once) via
* `SWRConfig` from "swr".
*/
import type { SWRConfiguration } from "swr";
/**
* Exponential backoff retry — starts at ~1s, doubles up to 30s, then flatlines.
* Matches the existing WsConnection reconnection philosophy.
*/
export const swrConfig: SWRConfiguration = {
// Re-validate on focus (tab switch) but not on every interval by default.
revalidateOnFocus: true,
// Dedupe rapid identical requests within 2s.
dedupingInterval: 2000,
// Never throw unhandled rejections — every hook handles `error` gracefully.
shouldRetryOnError: (error) => {
// Don't retry on 404 (NotFound) — it's a client expectation, not transient.
if (error?.statusCode === 404 || error?.code === "NOT_FOUND") return false;
// Retry network errors, 5xx, and oRPC transport failures.
return true;
},
onErrorRetry: (_error, _key, _config, revalidate, opts) => {
const attemptCount = (opts as { attemptCount?: number }).attemptCount ?? 0;
// Stop retrying after 3 attempts (≈ 1+2+4 = 7s max backoff).
if (attemptCount >= 3) return;
// Exponential backoff: 1000 * 2^attempt, capped at 30000ms.
const baseDelay = 1000 * 2 ** attemptCount;
const delay = Math.min(baseDelay, 30000);
// Jitter ±25% to avoid thundering herd on shared endpoints.
const jitter = delay * 0.25 * (Math.random() * 2 - 1);
const timeout = Math.round(delay + jitter);
setTimeout(revalidate, timeout);
},
// Sensible defaults for the "loading" state — most data loads in <500ms.
loadingTimeout: 15000,
};
@@ -37,6 +37,11 @@ export class WsConnection {
return this._status;
}
/** Current reconnect attempt count (0 = connected/first attempt). */
get reconnectAttemptCount(): number {
return this.reconnectAttempt;
}
onStatusChange(listener: (status: WsStatus) => void): () => void {
this.statusListeners.push(listener);
return () => {
+29 -11
View File
@@ -9,6 +9,7 @@ import {
useRef,
useState,
} from "react";
import { toast } from "@/components/primitives";
import { WsConnection } from "./connection";
import type { PcmChunk, WsEventHandler, WsEventType, WsStatus } from "./types";
@@ -29,16 +30,6 @@ interface WsContextValue {
const WsContext = createContext<WsContextValue | null>(null);
/** FNV-1a 32-bit hash matching the backend's hashUserId function */
function _hashUserId(userId: string): number {
let hash = 0x811c9dc5;
for (let i = 0; i < userId.length; i++) {
hash ^= userId.charCodeAt(i);
hash = Math.imul(hash, 0x01000193);
}
return hash >>> 0;
}
export function WsProvider({
children,
url,
@@ -48,6 +39,9 @@ export function WsProvider({
}) {
const connRef = useRef<WsConnection | null>(null);
const [status, setStatus] = useState<WsStatus>("disconnected");
// Tracks whether we've ever been connected — used to suppress the
// "reconnecting" toast on initial page load.
const wasConnected = useRef(false);
// Event handler registry — Ref so listeners survive re-renders without reconnect
// Using unknown as internal store; typed at the subscribe interface
@@ -85,7 +79,31 @@ export function WsProvider({
const conn = new WsConnection(url);
connRef.current = conn;
const unsubStatus = conn.onStatusChange(setStatus);
const unsubStatus = conn.onStatusChange((s) => {
setStatus(s);
// User feedback on WS lifecycle transitions.
if (s === "connecting") {
// Only toast if we were previously connected (i.e. a disconnect,
// not the initial connect on page load).
if (wasConnected.current) {
toast({
title: "Reconnecting…",
description: "WebSocket connection lost. Attempting to reconnect.",
tone: "neutral",
});
}
wasConnected.current = false;
} else if (s === "connected") {
wasConnected.current = true;
} else if (s === "error" && !wasConnected.current) {
toast({
title: "Connection error",
description:
"WebSocket failed to connect. Retrying in the background.",
tone: "vermilion",
});
}
});
const unsubEvent = conn.onEvent((event) => {
if (event.type === "text") {
handleJsonEvent(event.data);