fix(ai-moderation): bedah delay attachment ~330s -> target <20s

Root cause (trace msg 1541417073245290638):
- Race-guard upload-pending balik results:[] diperlakukan sbg SUKSES
  -> row yatam 'processing' sampai cleanup 300s mengembalikan
- Vision gagal 3x utk GIF besar (SSE truncation) tanpa fallback

Fix:
- Sinyal eksplisit uploadPending dari worker race guard
- Classifier murni classifyIndividualWorkerResult(): upload_pending ->
  requeue pending + reschedule segera (250ms), bukan error palsu;
  empty-results ok:true kini error transien (bug silent-success mati)
- llmVision fallback stream:false sekali saat SSE truncation
- Safety-net cleanup stuck processing 300s -> 120s
This commit is contained in:
asepharyana
2026-08-24 22:05:28 +07:00
parent fca96396b9
commit 842610b1af
7 changed files with 306 additions and 17 deletions
@@ -0,0 +1,56 @@
# Spec: Perbaiki Delay Attachment 162s→<20s (GMW AI Analysis)
Tanggal: 2026-08-24 · Repo `~/GMW` · Service discord-gateway
## Evidence (audit produksi)
Klaster pesan attachment delay ~330400 detik. Trace pesan `1541417073245290638` (.gif):
19:01:08 dibuat → 19:01:09 batch incomplete → fan-out individual → **guard upload-pending
mengembalikan `results:[]`** → diperalakukan sukses (`complete ... (undefined)`) → row
tertahan `ai_status='processing'` **tanpa penanggung jawab** → 19:06:12 cleanup mengembalikan
ke `pending` (tepat 300s) → baru dianalisis. Plus vision gagal 3× utk GIF besar
("Stream ended before producing a non-ping SSE event") → degradasi teks.
## Root causes
- **A (fatal)**: `individualFallbackProcessor.processIndividualFallback` memperlakukan
`ok:true + results:[]` sebagai sukses. Race-guard upload di `ai-analysis-worker.processIndividual`
sengaja balik `results:[]` (desain lama) → pesan yatim `processing` sampai cleanup 300s.
- **B**: `llmVision` hanya mencoba `stream:true`; kegagalan SSE truncation pada gambar besar
= 3 retry sia-sia (semua jalur sama) → bukti media hilang.
- **C**: safety-net cleanup 300s terlalu lambat sbg satu-satunya pemulih `processing`.
## Fix
1. **F1 — sinyal eksplisit upload-pending**: `IndividualOkResponse` + field opsional
`uploadPending?: boolean`. Worker set `uploadPending:true` saat race guard kena.
2. **F2 — processor menangani 3 kondisi** via helper murni baru
`classifyIndividualWorkerResult(result): "success" | "upload_pending" | "incomplete" | "error"`
(modul baru `fallbackResultClassifier.ts`, zero-dep agar mudah dites):
- `upload_pending` → tulis ulang row ke `pending` (pola sama dgn revert apiFailed di
batchProcessor) + broadcast + **re-schedule analisis percakapan segera**
(dynamic import batchScheduler, pola anti-siklus yg sudah ada) → retry dalam ~250ms
begitu upload beres. Bukan error, tidak naikkan CB counter.
- `incomplete` (flags analysis_incomplete) → perilaku lama (exhausted path).
- `error` / `results kosong tanpa penjelasan` → throw transien (retry oleh recovery),
BUKAN sukses palsu. Log "(undefined)" hilang.
3. **F3 — vision non-stream fallback**: di `llmVision`, jika error match
`/Stream ended before producing a non-ping SSE|stream ended/i` → coba SEKALI lagi dengan
`stream:false` (router agregasi penuh; timeout tetap 60s). Konversi hard-fail jadi sukses.
4. **F4 — turunkan safety net**: default `revertStuckProcessingMessages` 300000 → 120000 ms.
## File disentuh
- `src/modules/ai-moderation/fallbackResultClassifier.ts` (BARU, pure)
- `src/modules/ai-moderation/ai-analysis-worker.ts` (tipe + set flag uploadPending)
- `src/modules/ai-moderation/individualFallbackProcessor.ts` (konsumsi classifier + reschedule)
- `src/modules/ai-moderation/llmClient.ts` (fallback non-stream di llmVision)
- `src/modules/message-capture/messagesCleanup.ts` (default 120s)
## Verifikasi
- Test baru `tests/fallbackResultClassifier.test.ts` (4 klasifikasi + edge kosong).
- Gate: tsc --noEmit, biome error-level, vitest run semua hijau.
- Deploy GHA sukses; pasca-deploy: pesan attachment baru p50 < 20s
(`SELECT percentile_cont(0.5) ... WHERE metadata attachments>0 AND created_at > deploy`),
tidak ada lagi "complete ... (undefined)".