- Add examples for technical discussions (kinetic energy, drone weapon
engineering, physics simulations) that should be marked clean
- System rule: physics/engineering topics (kinetik, gravitasi, energi,
drone, senjata, drone warfare, CAD, CNC, 3D printing, robotics, aerospace)
are safe when in technical context — flag only if explicit threat
- Riwayat pengguna dengan pelanggaran sebelumnya tidak memengaruhi
penilaian pesan bersih yang terpisah dan tidak mengandung pelanggaran
- Removed getUserRecentInfractions usage in textBatchProcessor.ts and visionAnalyzer.ts
- Removed buildUserHistoryXml import and calls
- Messages are now evaluated standalone, not influenced by past violations in other channels
- Updated moderation prompts with clearer instructions about user_history usage
- Fixes issue where benign messages like 'tubuh manusia vs gravitasi' were incorrectly flagged due to carryover from previous drone weapons discussion
The user history context was causing the LLM to interpret unrelated current messages
as threats because it conflated them with past violations. Now each message is judged
on its own merit with only channel-specific context.
Add debug logging to trace cacheKey + messageId + content length on
every vision cache HIT and MISS, so we can detect if the vision model
returns duplicate analysis for different images (provider issue vs
cache collision). Includes the phash on cache miss (new analysis cached).
Follow-up to 9f7ce7d which fixed makeImageCacheKey to hash full data
URL instead of just first 128 chars (root cause of all images sharing
the same cached 'konten judi' verdict due to hash collision).
Root cause: makeImageCacheKey() only hashed the first 128 chars of the
data URL. Since all resized images use the same MIME prefix
('data:image/png;base64,') + identical base64 header bytes, nearly every
image got the same 16-char hash → 'image:<same-hash>' → all images reused
the first cached vision analysis (often a gambling-detection verdict).
Fix: hash the entire data URL instead of just the prefix. Verified
114 stale 'image:' entries + 745 stale 'phash:' entries purged from prod
DB. tsc --noEmit clean, 133 tests pass.
- Add term_glossary_cache table + migration 0014: resolved definitions are
stored permanently (definitions rarely change); misses stay ephemeral in
Redis/LRU with 1h TTL so transient failures get retried
- Lookup flow: LRU -> Redis -> Postgres (permanent) -> live SearXNG; DB hits
re-warm the fast caches; stale Redis miss sentinels no longer shadow DB
- Rate-limit-aware live lookups: concurrency 2 + stagger, retry once on empty
results, strict definition filter (Wikipedia preferred, rejects
disambiguation/ads/translate-homepages)
- Make SEARXNG_BASE_URL configurable via env (default unchanged)
Root cause (3rd layer after 50371bd + 4f4c435): a vision model run
(2026-08-10) returned 'Maaf, saya tidak melihat gambar apapun yang terlampir...'
and that text was cached as a VALID vision_llm result (image + phash keys,
24h/7d TTL). Every subsequent analysis of the same image (same hash/phash)
hit the poisoned cache, so image analysis looked broken forever even though
9router responded fine — the moderation LLM wrote 'lampiran yang gagal
terbaca' from a cache hit.
Also: mimo via 9router streams reasoning in delta.reasoning +
delta.reasoning_details[].text (content:"") — extractChunkText only read
delta.reasoning_content, so those runs aggregated empty → 'Vision API null
response' (observed 08:54/09:07/09:38).
Fixes:
- llmClient.extractChunkText: fall back to delta.reasoning and
reasoning_details[].text (mimo), on top of reasoning_content (gemma).
- visionAnalyzer: isNoImageSeenText() detects 'no image' style outputs;
such results are NEVER cached, and poisoned entries are purged when hit
(LRU/DB/phash) so re-analysis actually re-runs vision.
- Tests: reasoning/reasoning_details extraction + isNoImageSeenText
(Indonesian + English, no false positives on real descriptions).
Root cause (2nd layer after 50371bd): the analysis worker could pick up an
image message while its attachment upload was still in flight
(upload_status='pending'). downloadAndExtractFrame then fell back to the
Discord CDN URL (cdn.discordapp.com), which often 404s for old/purged links,
and 'if (!res.ok) return' silently dropped the image — no log, no vision
call, empty image map, and the LLM produced a text-only verdict like
'lampiran yang gagal terbaca oleh sistem'.
Fixes:
- ai-analysis-worker: skip targets whose attachment upload is still pending
(both batch + individual paths) — they stay ai_status='pending' and the
next 15s cycle analyzes them after the upload lands.
- mediaDownloader.downloadAndExtractFrame: try uploaded_url first, then
discord_url as fallback; log non-OK responses (status + host) instead of
silently returning; log when all candidate URLs fail.
Root cause: 9router combo 'multimodal' routes to cloudflare-ai/@cf/google/
gemma-4-26b-a4b-it which streams ALL output in delta.reasoning_content
(content:"") and finishes with 'length' at max_tokens. llmClient only read
delta.content, so llmVision returned empty → every image moderation fell back
to text-only analysis ('Meskipun analisis gambar gagal' in every ai_analysis).
Fix: extractChunkText() prefers delta.content then falls back to
delta.reasoning_content (also handles message/text/response fields), with
unit tests for the exact 9router chunk shape. Verified live against a real
DB image: oc/mimo-v2.5-free (new first model in the multimodal combo) returns
a proper description in delta.content.
- <message> targets now carry time (ISO), repetitions (N identical short texts = spam signal), bot and edited flags; escape id/user XML
- rich <user_reputation>: total_infractions, clean_streak, last_offense_days_ago, repeat_offender (7-day window)
- <user_history> with last flagged messages for repeat offenders (wires dead getUserRecentInfractions)
- <user_profile as_of> staleness signal; <location_context topic> from captured channel topic
- prompt framing + output instructions teach the LLM to use the new signals without treating history as proof
- tests: contextEnrichment.test.ts (13) + topic cases in conversationContext.test.ts
When the ONLY violation is offensive_username (message content clean):
- Message is NOT deleted (nickname-only violation bypasses auto-delete)
- Member's server nickname is reset to default username via
setNickname(null) (Discord shows the global username again)
- Action 'reset_nickname' logged to moderation_actions; cooldown
10min per guild:user (LRU) so repeated messages by same member
don't hammer the Discord PATCH
- Config: AUTO_NICKNAME_RESET_ENABLED / AUTO_NICKNAME_RESET_COOLDOWN_MS
- resolveDisplayName(): member.displayName from captured metadata,
falls back to global username
- Applied to context lines, target message blocks, and media message
blocks — LLM sees the name the channel actually sees (nickname can
carry moderation signal itself)
- Conversation context recency gates (GAP_MS/MAX_AGE_MS): drop stale
messages before silence gaps; cold_start anchor + flow descriptor
tells LLM whether conversation is ongoing or restarted
- [location] block: channel name, thread name, nsfw/age flags from
captured metadata (thread names instead of bare IDs)
- Link media -> multimodal: text-batch URL fetches that resolve to
images now run vision analysis (bounded 15s) and switch prompt to
mixed mode; <web_content> gains og:title for page context
- pnpm-workspace.yaml: approve sharp build script (unblocks install)
Audit lanjutan: 6x 'LLM API request failed: Request was aborted' per jam.
Root cause: 9router/omniroute SELALU balas SSE (data: chunks) walau request
tanpa stream:true — SDK OpenAI non-stream menunggu FULL body sebelum parse,
jadi batch moderasi besar yang upstream-nya lambat kena timeout 30-60s dan
di-abort. llmClient sudah punya agregasi streaming (chunks → ChatCompletion).
Fix: stream:true di llmCaller (moderasi batch/individual), llmVision,
cultureLearner, userProfileLearner. Verified: SDK stream test 806ms vs
sebelumnya abort. Caller lain (recovery worker dll) lewat llmCaller sama.
Audit log produksi (sejak deploy13:38) menemukan 3 isu:
1. mediaDownloader.ts spawn /usr/bin/ffprobe + /usr/bin/ffmpeg (path keras) —
ENOENT di Nix karena binary cuma di ffmpeg-headless closure. Pakai
PATH-resolved ('ffprobe'/'ffmpeg') seperti voice-recording module
(ffmpegProcess.ts/transmitter.ts) — 5 media warning hilang.
2. individualFallbackProcessor log error 'Success' di level50 tiap fallback
BERHASIL (logModerationError dengan new Error('Success')) — ganti
logger.info dengan verdict yang sama; error log cuma untuk error asli.
3. moderationResponseParser: strip frasa penutup generik ('Tidak ada
indikasi pelanggaran.') yang masih sering dikeluarkan LLM walau prompt
melarang (277/1486 analisis mengandung frasa, termasuk hari ini).
sanitizeGenericCleanCloser hanya mencocok frasa di AKHIR, teks substantif
tetap utuh. Unit test: 6/6 pass.
QoL lanjutan dari fix60084b3: content pesan mentah masih nampilin
snowflake (<@&roleid>, <@userid>, <:emoji:id>) di log moderasi dan
prompt LLM. Sekarang dirender ke nama yang bisa dibaca:
- Gateway capture: metadata menyimpan mentionedRoles + mentionedUsers
(id+name) dari message.mentions, disimpan ke metadata JSON
- renderDiscordMentions(): <@&id> -> @RoleName, <@id> -> @Username,
<:name:id> -> :name:, fallback @role/@user — dipakai di
conversationContext (konteks LLM) dan moderationBuilders
(getAnalysisContent) sehingga LLM lihat nama role/user beneran,
bukan placeholder generik
- Frontend renderMessageContent() (mirror gateway) dipasang di semua
tempat nampilin content: message-card, message-detail(-view),
search-overlay, search-panel, users/channels section, live-stream,
mod-queue, review list; sticker-only message tetap [Sticker: name],
pesan teks+sticker kini ikut nampilin nama sticker
- tsc --noEmit PASS di gateway & frontend; renderDiscordMentions
diverifikasi manual (6 kasus: role/user/emoji/unknown/plain)
- Deleted the text analysis prompt constants and helpers as they are no longer needed.
- Added batch search functionality for Qdrant to optimize vector searches.
- Implemented methods for deleting expired Qdrant points and invalidating cache based on content hash.
- Updated text batch processor to use new timeout configurations and modified content building for moderation prompts.
- Enhanced text cache store to support new Qdrant integration and improved cache invalidation logic.
- Introduced a new user reputation model with a more nuanced trust scoring system, including penalties and rewards for user behavior.
- Added unit tests for the new trust model to ensure correctness of penalty and trust gain calculations.
- Updated configuration schema to reflect new timeout settings and removed deprecated OpenAI moderation keys.
New qdrantClient.ts (zero-dep fetch REST): ensure collection with cosine
distance (auto-recreate on vector-size change), upsert point w/ verdict
payload, search w/ expires_at filter + score threshold.
textCacheStore: when QDRANT_URL set, embeddings are upserted to Qdrant
(primary) and searched there first; Postgres embedding column remains as
legacy fallback for pre-Qdrant rows. Config: QDRANT_URL/COLLECTION/API_KEY.
QDRANT_URL already in repo .env; added to VPS env + GATEWAY_ENV secret.
OpenAI SDK v6 defaults to encoding_format=base64; llama-nemotron-embed
(Nvidia-backed) returns 400 'do not support base64'. Semantic cache was
silently disabled in prod. encoding_format: 'float' fixes it.
TS compiled this fine, but the JS spec forbids mixing || and ??
without explicit parens; Node threw 'Unexpected token ??' at startup,
crash-looping gmw-discord-gateway (restart counter 250). Wrap the
fallback chain in parens so the expression is valid.
Root cause: ai-analysis-worker read llmResult.explanation and
llmResult.toxicityScore — fields the LLM pipeline never produces
(canonical AnalysisResult uses analysis/score). Every message fell back
to the bare template "Tidak ada indikasi pelanggaran." and the stored
score was always 0.
- Map analysis/score correctly; fallback now quotes the message content
- Prompt: ban generic analysis phrasing, require reply context
- LLM context: include replied-to message content (metadata.reference)
so the model can explain what the user is replying to
- Frontend: show thread/channel names from metadata instead of raw IDs
(message card, detail views, search overlay); detail panel now
displays the ai_analysis text
- Auto-delete log/DM include the descriptive analysis as the reason
Delete fastClassifier.ts (manual regex patterns for phone/email/IP/crypto/
spam/toxicity) and simpleFallback.ts. These hardcoded patterns were the
source of false positives (Discord emoji snowflakes matched phone_number,
URL digits matched phone, etc.) and produced heuristic verdicts whenever
the LLM failed.
New flow: Message → LLM (with conversation context, media evidence, user
reputation) → verdict. On LLM failure the message is marked 'error' and
retried by the recovery worker — no heuristic verdicts, ever.
Discord markdown tokens (custom emoji/mentions/timestamps) are normalized
to readable placeholders ([emoji:name], @user, @role, #channel, [time])
before reaching the LLM via discordTokens.ts.
Custom emoji (<:name:id>), user/role/channel mentions and timestamps embed
long numeric snowflakes that tripped the phone_number / personal_info /
ip_address_sharing patterns — e.g. <:mambotongue:1463255254220148939> was
flagged as phone_number. Strip Discord markdown tokens before pattern
matching and require phone matches to not sit inside a longer digit run.
Build & Deploy / build-and-push (backend) (push) Failing after 35s
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 25s
Build & Deploy / build-and-push (proxy) (push) Failing after 25s
- Remove pnpm workspace, moon repo, and all monorepo tooling
- Delete packages/shared/, embed shared code directly into each service
- Copy packages/shared/src/* -> services/backend/src/shared/ and services/discord-gateway/src/shared/
- Replace all @bete/shared imports with @/shared/ path alias
- Remove @bete/shared workspace dependency from both services
- Update root package.json scripts from --filter to --prefix
- Rewrite Dockerfiles to build each service standalone
- Clean up biome.json, .gitignore, remove root drizzle.config.ts
Build & Deploy / build-and-push (backend) (push) Successful in 25s
Build & Deploy / build-and-push (proxy) (push) Successful in 3m46s
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 4m42s
- Cast llmResult through unknown to handle type mismatch between
shared AnalysisResult and layer-specific local type
- Exclude src/**/archive/** from tsconfig to prevent dead code errors
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 28s
Build & Deploy / build-and-push (backend) (push) Successful in 1m46s
Build & Deploy / build-and-push (proxy) (push) Successful in 1m37s
\U escapes are not valid in JavaScript/TypeScript regex literals.
Use new RegExp() constructor to avoid TS parser issues.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-28 15:10:47 +07:00
DeveloperandClaude Opus 4.8 (1M context) <noreply@anthropic.com
Full frontend redesign with glassmorphic dark theme, floating top nav,
Live2D mascot, split-pane messages, and Ops Center dashboard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 2m22s
Build & Deploy / build-and-push (backend) (push) Failing after 3m22s
Build & Deploy / build-and-push (proxy) (push) Successful in 1m36s
Build & Deploy / deploy (push) Skipped
- Consolidate all DB schema definitions into packages/shared as single source of truth
- Migrate backend from raw SQL to Drizzle ORM across all modules
- Extract frontend inline UI into separate component files
- Refactor discord-gateway circuitBreaker into conversationState + moderationState
- Convert messageStore to Proxy singleton pattern
- Add validateBody/validateQuery middleware + Zod schemas for API endpoints
- Modernize Docker builds with multi-stage + pnpm deploy
- Migrate CI/CD from deployment to image-based pipeline
- Remove 60+ unused/dead files (~15K lines)
- Update color scheme from sky-blue to teal-cyan
- Move DB connection management to @bete/shared/database
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Added new dependencies for Next.js and lucide-react in pnpm-workspace.yaml.
- Refactored DashboardPage component to improve readability and error handling.
- Enhanced Header component to display error status with an alert icon.
- Updated MobileTabBar and Sidebar components to use a centralized tabs definition.
- Improved ChannelsView in dashboard-panel to handle channel fetching more cleanly.
- Fixed ActiveSpeaker type to use camelCase for userId.
- Updated MessagesPanel to handle guildId checks more gracefully.
- Adjusted API calls in dashboard and messages to align with backend expectations.
- Refined type definitions across various interfaces for consistency and clarity.
- Fix noImplicitAnyLet: add type to let match variable
- Fix noAssignInExpressions: use matchAll() + for-of instead of while
- Suppress useExhaustiveDependencies in mascot scroll effect
- Suppress useSemanticElements for message card click handler
Two-layer fix for forwarded messages showing as empty/clean:
Layer 1 (messageMetadata.ts): getReferencedMessageContent() now falls
back to message.messageSnapshots Collection when channel.messages.cache
lookup fails. Discord stores forward content in message_snapshots API
field, not in message.content.
Layer 2 (moderationBuilders.ts): buildReferenceXml() now parses msg.
metadata JSON to extract reference.content when DB getMessageById()
fails (cross-server forwards not in local DB).
Previously: forward messages captured with empty parentContent →
LLM saw no reference text → '99% confidence, pesan kosong'.
Now: forward content flows through capture → metadata → analysis.
- Previously: text+media message went ONLY to media array → text waited for vision
- Now: text part goes to text batch (immediate LLM analysis), media parallel
- DB update is idempotent — second write to same message_id overwrites
- User sees text moderation results instantly, media follows when ready
- text-only and media analysis now run concurrently via Promise.all
- text no longer blocks on media download + vision analysis
- each path independently saves to DB when its own results are ready
- same batch still uses single context fetch + attachment lookup
- Remove shouldSearchContent() trigger gate — search runs on all messages
- extractSearchQueries() now extracts from ANY message, not just trigger-matched
- Redis cache (24h TTL) prevents redundant searches for same query
- initSearxngCache() lazy-connects via config.REDIS_URL
- Cache miss→API, hit→skip — fire-and-forget writes
- Both text batch + media path simplified
- Remove hardcoded hentai title lists (Boku no Pico, Euphoria, etc.) from prompt
- Remove hardcoded SARA examples from prompt (Kitabonia, etc.)
- Prompt now tells LLM to use <web_searches> as evidence instead of hardcoded knowledge
- Evidence priority: searxng > web_content > media_analysis > internal model
- Code-side triggers in searxngSearch.ts still filter which messages to search
- LLM makes final decision based on search results, not static lists
- Video frame extraction via ffmpeg (4 key frames per video → vision LLM)
- Video display in FE MessageCard with HTML5 <video> player
- Reply/forward/crosspost indicator in FE + pipeline in DG/BE
- Fix: missing sanitizeAiContent + escapeXml in media path (prompt injection)
- Optimize: text-only batch results saved to DB immediately, no longer wait for media analysis
- BE mapper/schema/repo: add reference fields (is_reply, is_forward, etc.)
- userProfileLearner.ts: filter query to only clean messages (eq ai_status='clean')
to prevent profile contamination from flagged content. Also select channel_id
to group messages by channel in prompt, enabling channel-aware personality
summarization (user may behave differently across channels).
- llmModerationClient.ts (runSimpleTextFallback): inject user profile into
both the classify prompt and the reason prompt, so even the last-resort
fallback path has personality/memory context instead of being blind.
asepharyana