- mediaDownloader: flip URL candidate order so discord_url is tried
before uploaded_url (uploaded_url is archive-only fallback)
- ai-analysis-worker: remove upload-pending race guard that blocked
analysis until Tele upload completed; analysis now runs immediately
on the Discord CDN URL
- batchProcessor: remove upload-pending defer/poll-backoff logic
- individualFallbackProcessor: remove upload_pending requeue loop
- batchOutcomeClassifier/fallbackResultClassifier: drop upload_pending
classification (no longer needed)
- tests: update batchOutcomeClassifier + fallbackResultClassifier tests
to reflect removed upload_pending signal
Switch GMW's AI LLM base URL from 9router (https://9router.asepharyana.my.id/v1)
to omniroute on imrnes (http://100.121.180.82:20128/api/v1).
- Update default AI_LLM_BASE_URL in discord-gateway + backend config schemas
- Update .env.example documentation
- Update all 9router references in comments/docs/tests to omniroute
- Production BWS secret gmw_ai_llm_base_url already updated
Omniroute uses /api/v1 prefix (not /v1 like 9router), so the base URL
now correctly points at the right API path for the OpenAI SDK.
Rename server_name (guild name) to server_nick and populate it from
the member's server-specific display name (metadata.member.displayName)
at write time. This is what the moderation dashboard should show as
TARGET — e.g. server nick 'Bandar Togel「✔ ᵛᵉʳᶦᶠᶦᵉᵈ 」' for global
username '.nichiyobi'. Backfilled 210 existing actions from messages
metadata (reset_nickname rows now show 'Sarjana .jav', 'Penindas
Minoritas', etc). Frontend TARGET shows server nick with global
username as secondary context.
Replace static OFFENSIVE_USERNAME_KEYWORDS substring matching with a
lightweight LLM call that evaluates whether a global username violates
server rules (gambling, scam, NSFW, SARA, etc). Fail-open design:
if the LLM call fails/times out, the nickname reset still completes.
After resetting an offensive server nickname to the global username,
check the global username against gambling/scam keyword list. If it
also violates, generate a random 'UserXXXXX' nickname to prevent
circumvention via offensive global usernames.
Denormalize guild name alongside username so the moderation dashboard
shows both TARGET and server even after message table purges.
Migration 0018. Frontend displays 'username · server_name' in TARGET.
Two bugs causing 23 spurious 'error' logs after successful deletions:
1. batchProcessor switch missing 'completed' case: partitionBatchOutcome
returns 'completed' for successful messages, but the switch only handled
'upload_pending' and 'api_failed'. Successful messages fell through to
default → re-enqueued to individual fallback → re-analyzed → re-delete
attempt → error (message already gone from Discord). Now explicitly
skips 'completed' messages.
2. isAlreadyDeletedError only caught codes 10008/404. Discord also returns
10003 (Unknown Channel) and 50001 (Missing Access) when a message or
channel is gone. Added these codes plus text-based fallback matching
'Unknown Message'/'Unknown Channel'.
Impact: eliminates ~23 redundant error logs per day + stops wasted LLM
calls re-analyzing already-processed messages.
isEligibleForAutoDelete was rejecting messages where recommendedAction
was 'warn' or 'review' — only 'delete' and 'escalate' were accepted.
This caused 28+ medium-severity flagged messages to be logged as
'not_eligible' instead of being auto-deleted.
Changes:
- deriveRecommendedAction: return 'delete' for flagged+medium severity
(previously only critical/high triggered delete; medium got 'review')
- isEligibleForAutoDelete: accept 'warn' as valid recommendedAction
alongside 'delete' and 'escalate'
Impact: ~28 pending medium-severity flagged messages + all future
'warn'-action flagged messages will now be eligible for auto-deletion.
- Change AI_LLM_BASE_URL default from omniroute.imrnes.team to 9router.asepharyana.my.id
- Update AI_LLM_MODEL default from 'text' to 'claude-opus-5' (bare model name
compatible with 9router/OpenAI-compatible router)
- Update .env.example and inline comments to reflect 9router
- discord-gateway config now matches backend (which already uses 9router)
Batch race guard balikin {ok:true, rows:[]} tanpa sinyal saat semua target
masih upload-pending -> processor klasifikasi semua incomplete -> fanout ke
individual queue -> di situ requeue + reschedule 250ms -> balik ke batch:
hot loop ~300ms sepanjang upload (10 siklus/3 dtk di log prod 08:13).
Fix: worker batch kini return uploadPendingIds eksplisit; classifier pure
baru (partitionBatchOutcome) partisi completed/upload_pending/incomplete/
parse_failed/api_failed; target upload-pending DEFERRED dengan poll backoff
linear (AI_ANALYSIS_UPLOAD_POLL_MS 1500 base, cap AI_ANALYSIS_MAX_UPLOAD_POLL_MS
8000), tidak pernah masuk fanout; tail shouldScheduleNext tak menimpa defer.
Test: tests/batchOutcomeClassifier.test.ts (8 kasus, pure tanpa DB/Piscina).
Analisis pertama tetap berkonteks (chat history) demi akurasi, tapi
verdict clean non-actionable (conf>=0.85) juga ditulis di bare key
tanpa konteks. Repeat teks sama di channel lain -> exact cache HIT,
bukan LLM call baru. Guard sama dgn read path; dedupe LRU per proses;
bare row tanpa embedding (tier semantic sudah global).
- Fase-1 exact-cache lookup: N query serial -> SATU query ANY($1::text[])
- Global reuse utk bare key legacy, HANYA verdict non-actionable
(clean/flagless/action=none, conf>=0.85, umur<=72h) — flagged/warn
tetap context-scoped
- Semantic cache dua-band: clean band 0.92 default, actionable tetap
0.97; di antara band -> LLM (fail-open ke akurasi)
- hit_count kini di-increment (bulk UPDATE per batch) -> hit-rate terukur
- Cache hasil wikipediaSearch di Redis (6h, hanya hasil non-kosong)
- Memoize fetchUrlSafely utk type=text (LRU 30m + in-flight dedupe)
- makeImageCacheKey strip query CDN Discord (?ex/is/hm, format/width)
-> attachment sama = satu key vision, skip re-download+re-vision
Spec: .hermes/plans/2026-08-24-ai-analysis-cache-optimization.md
Tests: +33 (cacheGuards, discordImageKeyNormalize, cacheBatchLookup)
- pickBatchWithinBudget: stop di overflow pertama (break), bukan skip —
batch tetap prefix kronologis tanpa gap analisis di tengah timeline.
Diekstrak ke batchBudget.ts (pure, estimator di-inject) + regression test.
- callModerationLLM: param opsional maxTokens; text/media caller menghitung
ceiling dari estimasi prompt (floor 2048, cap 16384) — batch kecil tak
lagi reserve window completion 16k.
- getPending/IncompleteMessagesByConversation: sort hasil UPDATE..RETURNING
by created_at ASC — Postgres tak menjamin urutan, konsumen (anchor konteks
messages[0], prefix batch) bergantung pada urutan kronologis.
- normalizeStoredStatus(): exact-hash & semantic (Qdrant/PG) cache reader
sebelumnya menipiskan 'warn' jadi 'flagged'/'clean' (type narrowing
legacy clean|flagged) — merusak gating auto-delete & label dashboard.
Kini status tersimpan dipertahankan penuh (clean/warn/flagged).
- prompts: hapus referensi <user_history> yang tak pernah di-inject,
SearXNG -> Wikipedia (sudah migrasi), referensi section yang tak ada,
typo 'secifik', dan baris list rusak '|-'.
- moderationBuilders: buang dead code buildUserProfilesBlock/
buildUserProfileRef/UserProfileEntry/buildUserHistoryXml (tanpa caller
produksi sejak context minimization) + test-nya.
- test baru: tests/storedStatusNormalization.test.ts (regresi warn).
SearXNG was already replaced by Wikipedia REST/Action APIs (wikipediaClient.ts).
Update comments to reflect the current implementation: term glossary now
resolves definitions via Wikipedia → Redis → Postgres cache chain, with no
SearXNG dependency.
- Persist structured verdict (flags/severity/confidence/evidence) on
moderation_actions so the public web can show WHY a message was moderated.
- Add a persistent Qdrant archive collection (gmw_message_archive); embed
every captured message at capture time (fire-and-forget, best-effort).
- Public semantic search over the archive (backend oRPC + FE toggle on the
messages view). Both features are read-only/public and fully automatic.
Migration: 0015_add_moderation_explainability.sql
- Memoize buildSystemPrompt by (mode|channelCulture); identical signatures
now reuse the ~5k-token core instead of rebuilding per sub-batch call
(textBatchProcessor rebuilt it inside the loop; a 200-msg batch re-sent
the full system prompt ~4x). Correction tail stays per-attempt (uncached).
- Hoist URL-image -> vision evidence out of the per-sub-batch loop in
textBatchProcessor: it depends only on fetched images + full target set,
so compute once per whole batch, not per sub-batch.
- Compact system instructions: collapse 3x-duplicated 'evaluate by content
alone' statements into one standalone rule; trim output.ts channel-culture
+ context framing already covered by rules.ts/system.ts; drop duplicate
programming-error-log few-shot (id 17, covered by rules AMAN list).
- Fix misleading config default: AI_LLM_BASE_URL default -> omniroute
(gateway already runs omniroute via BWS; 9router was dead/misleading).
typecheck + lint + build green.
- resetOffensiveNickname: skip when target role sits above bot
(member.manageable) instead of hammering a doomed setNickname PATCH
that Discord rejects with 50013 'Missing Permissions'. Log the
Discord error code on failure for clear diagnosis.
- llmCaller: include contentPreview (first 200 chars) in the parse-
failure warning so non-JSON LLM responses are debuggable.
- Add wikipediaClient.ts: native fetch to Wikipedia REST/Action APIs
(search + summary), no extra npm dependency.
- Extract shared Redis cache into cacheStore.ts (decoupled from search).
- Term glossary now uses wikipediaSummary for direct article lookup.
- Remove searxngSearch.ts entirely; drop SEARXNG_BASE_URL config,
add WIKIPEDIA_LANG / WIKIPEDIA_TIMEOUT_MS.
- Rename backend searxngCalls metric to webSearchCalls.
User: 'jangan ada reputasi juga' — no profile, no reputation in the prompt,
raw messages only.
- textBatchProcessor: drop initializeUserReputation fetch + <user_reputation>
tag injection (kept the minimal <message> tag + reply/reference context).
- visionAnalyzer (prepareMediaMessage): same removal.
- prompts/system.ts + prompts/output.ts: replace <user_reputation>/<user_history>
instructions with an explicit 'no per-user profile/reputation context'
note so the LLM judges purely on message content + conversation/web/location.
- mediaBatchProcessor: fix stale comment.
Trust/infraction state is STILL written to the DB (userReputationsTable) for
enforcement — only the LLM context injection is removed, so moderation
actions (mute/ban via infraction thresholds) keep working.
Net: even smaller prompts (no per-user context at all) → more messages fit
per request, and one fewer DB round-trip per unique user per sub-batch.
tsc, biome, vitest (129) all clean.
User insight: personal profile summaries bloat the prompt (less room per
request) and add a per-user DB/Redis round-trip for little moderation signal.
Only the behavioural <user_reputation> history is kept.
- textBatchProcessor: stop fetching getUserProfile; remove <user_profiles>
block + <user_profile_ref> from message tags. Keep <user_reputation>.
- mediaBatchProcessor + visionAnalyzer: same removal (profile fetch + ref).
- prompts/system.ts + prompts/output.ts: drop stale <user_profiles>/
<user_profile_ref> instructions; point LLM at <user_reputation> instead.
- aiAnalyzer: gate userProfileLearner behind AI_USER_PROFILE_LEARNING_ENABLED
(default false) — generates profiles nobody reads, pure LLM/DB waste.
- Add AI_USER_PROFILE_LEARNING_ENABLED config knob.
Net: smaller prompts (more messages fit per request), fewer DB round-trips
per sub-batch, and no background LLM calls learning unused profiles.
tsc, biome, vitest (129) all clean.
- Parallelize per-user reputation/profile fetches in textBatchProcessor
(was a serial ~2N DB/Redis round-trip loop per sub-batch; now Promise.all
over unique users). Cuts per-batch latency, biggest win on small/quiet
batches.
- Make the LLM concurrency semaphore dynamic (cached per config value) instead
of frozen at import time, so AI_LLM_MAX_CONCURRENT is tunable without code
change and reflects current config.
- Bump AI_LLM_MAX_CONCURRENT default 5 -> 8 (gemini-flash-lite is cheap; helps
throughput when busy).
- Lower AI_ANALYSIS_DEBOUNCE_MS 500 -> 250 (snappier first-message analysis
when quiet).
- Lower AI_ANALYSIS_RECOVERY_INTERVAL_MS 15000 -> 10000 (stuck/errored
messages re-analyze sooner).
tsc, biome, vitest (129) all clean.
The standalone image analysis path (analyzeSingleMediaImage → llmVision →
llmChat) previously had no request-level timeout of its own — it silently
inherited the shared OpenAI client default (60s), and AI_LLM_MEDIA_ANALYSIS_
TIMEOUT_MS only governed the text+media *batch*, not a single vision call.
- Add AI_LLM_VISION_ANALYSIS_TIMEOUT_MS (default 60000) to config.
- llmChat now accepts an optional per-request `timeout` in LlmCallOpts,
forwarded to the OpenAI request options (falls back to the 60s client
default when omitted).
- llmVision passes config.AI_LLM_VISION_ANALYSIS_TIMEOUT_MS, so a single
image/sticker/emoji analysis gets a guaranteed 1-minute budget and is
independently tunable from the text path.
Verified: tsc + biome green, 129 gateway tests pass.
Co-Authored-By: Claude Opus 5 (Nous Research)
buildCorrectedFewShotExamples() (a getRecentCorrectedModerations(5)
DB hit) was called inside the per-sub-batch buildContent closure in
textBatchProcessor.ts — re-queried for every sub-batch (≈10× for a
200-msg burst) AND re-fired on each parse-error retry. mediaBatchProcessor
already hoisted it once. Mirror that: fetch once per runTextOnlyBatch,
reuse the cached string inside the closure.
No behavior change — identical content, fewer identical DB reads.
tsc + 129 tests + biome green.
Co-Authored-By: Claude Opus 5 (Nous Research)
The 32 few-shot examples each re-echoed score/confidence/
recommended_action/categories/policy_version inline (~150 chars ×
32). Those fields carry zero moderation-decision signal — the schema
and their ??-default coercion already live in OUTPUT_INSTRUCTIONS +
moderationResponseParser.ts. Removed 96 redundant key/value pairs.
Kept per-example: message_id, status, flags, severity, evidence,
analysis — the fields that actually teach decisions. Parser derives
the rest via ?? fallback, so real output shape is unchanged.
examples.ts: 21.7K→18.5K chars; FEW_SHOT(mixed) 15.3K→13.4K.
Total mixed system prompt now 33.9K (was 39.3K at audit start,
~14% leaner). tsc + 129 tests + biome green.
Co-Authored-By: Claude Opus 5 (Nous Research)
- prompts/system.ts: merge 3 overlapping framing blocks (Blok Data /
Konteks Pengguna / Framing Konteks vs Target) into 1 tight block —
same coverage, no duplicated "standalone judgment / profile-is-
reference-not-evidence" prose.
- prompts/output.ts: trim duplicated user_history/standalone paragraph
in PERSONALITY & MEMORI (keep concrete per-case lessons).
- prompts/examples.ts: drop 2 exact-duplicate-lesson few-shots (LGBT id=19
dup of id=30; weapons-tech id=33 dup of id=32). All teaching signals
retained via the surviving example of each lesson.
Static system prompt: text 32.7K→29.2K, mixed 39.3K→35.8K chars
(~10% smaller). No moderation rule, zero-tolerance category, or decision
tree altered — accuracy-controlling content untouched. tsc + 129 tests +
biome green.
Co-Authored-By: Claude Opus 5 (Nous Research)
Adds per-message AI moderation analysis time (ai_analysis_duration_ms)
so operators can see how long the LLM took to moderate each message.
Gateway:
- messagesTable: new ai_analysis_duration_ms (bigint) column.
- AIAnalysisUpdate + buildAIAnalysisSet: carry analysisDurationMs through
both single and bulk update paths.
- ai-analysis-worker: measure wall-clock time around runModerationAnalysis
and attach it to every result in the batch.
Backend:
- Mirror schema column; messageMapper maps ai_analysis_duration_ms;
moderation-types + MappedMessage expose it.
Frontend:
- message.ts type gains ai_analysis_duration_ms.
- AiBadge (messages view) shows 'status · 1.2s' when duration is present;
analysis view badge mirrors the same formatting.
DB:
- scripts/add-ai-analysis-duration.sql (idempotent ADD COLUMN IF NOT EXISTS).
No behavior change for moderation logic; null until new gateway build
records values.
Qdrant upserts were failing with 'This operation was aborted' ~32x/2h,
so semantic moderation cache entries were silently dropped. Root cause:
upsertQdrantPoint ran ensureQdrantCollection() on EVERY call — a GET
(and sometimes DELETE+PUT) round-trip — while the request AbortController
had only a 10s timeout. Under moderation load Qdrant is busy (the
gmw_text_moderation collection is not yet HNSW-indexed, so searches are
full-scans), the extra round-trips pushed the upsert past 10s, and the
client aborted it.
- Memoise ensureQdrantCollection() at module scope so the collection is
verified exactly once per process (resetQdrantCollectionCache() for
tests / config reload).
- Bump the upsert request timeout 10s -> 30s so a transiently busy
Qdrant no longer aborts the write.
Qdrant server itself is healthy (<100ms for direct upsert; collection is
green), so no server-side change is needed. Semantic cache should now
populate reliably.
Address every remaining biome lint/format warning across both services
so the codebase ships warning-free:
- textCacheStore: drop unused deleteExpiredQdrantPoints import; hash
image cache key (sha256[:32]) so long/base64 URLs no longer blow the
text_analysis_cache PK B-tree 8191-byte index (was aborting the media
analysis lock INSERT).
- bootstrap: drop unused unhandledRejection promise param.
- moderationOrchestrator: drop unused destructure at L197.
- mediaDownloader / textBatchProcessor / transmitter: replace non-null
assertions with proper null guards (stickerName ?? '', urlImages.get
guard, backpressureQueue.shift guard).
- backend utils: throw lastError ?? fallback instead of lastError!.
- message-capture: remove unused (retentionDb), (moderationActionsDb,
reviewsDb); simplify renderDiscordMentions guard to optional chain.
- transmitter: remove dead write-only field + its assignments.
No behavior change beyond the cache-key hashing (now deterministic
fixed-length) and the intentional null-safety guards.
Two root causes behind 'all image analysis failing':
1. imageResizer still emitted lossless PNG for vision input. A 1024px
Facebook photo balloons to multi-MB PNG base64 that the vision model
silently rejects ('Vision API null response'). Switch to JPEG q85
(no upscaling) — same photo drops to ~100-400KB, model processes fine.
Re-encodes even already-small images so raw originals never bloat the
data URL. Added tests/imageResizer.test.ts covering both cases.
2. acquireMediaAnalysisLock INSERT aborted with 'index row requires N
bytes, maximum size is 8191'. text_analysis_cache.text is the PK in a
B-tree index (8191-byte/row cap); callers pass the raw image URL as the
key, and base64 data URLs / very long URLs blow past the limit, so the
lock INSERT fails and every media analysis is skipped. Hash the URL in
makeImageCacheKey (image:<sha256[:32]>) — fixed-length, deterministic,
well under the limit. All store/get/lock/delete callers already route
through this function so lookup stays consistent.
- config: add AI_LLM_VISION_BASE_URL + AI_LLM_VISION_API_KEY (separate from text router)
- llmClient: llmVision() now calls dedicated vision endpoint when configured
(axios POST to integrate.api.nvidia.com, model nvidia/nemotron-3-nano-omni-30b-a3b-reasoning,
reasoning_budget 16384, non-stream), falls back to router combo otherwise
- keeps text/moderation on omniroute, vision on NVIDIA direct
- Add examples for technical discussions (kinetic energy, drone weapon
engineering, physics simulations) that should be marked clean
- System rule: physics/engineering topics (kinetik, gravitasi, energi,
drone, senjata, drone warfare, CAD, CNC, 3D printing, robotics, aerospace)
are safe when in technical context — flag only if explicit threat
- Riwayat pengguna dengan pelanggaran sebelumnya tidak memengaruhi
penilaian pesan bersih yang terpisah dan tidak mengandung pelanggaran
- Removed getUserRecentInfractions usage in textBatchProcessor.ts and visionAnalyzer.ts
- Removed buildUserHistoryXml import and calls
- Messages are now evaluated standalone, not influenced by past violations in other channels
- Updated moderation prompts with clearer instructions about user_history usage
- Fixes issue where benign messages like 'tubuh manusia vs gravitasi' were incorrectly flagged due to carryover from previous drone weapons discussion
The user history context was causing the LLM to interpret unrelated current messages
as threats because it conflated them with past violations. Now each message is judged
on its own merit with only channel-specific context.
Add debug logging to trace cacheKey + messageId + content length on
every vision cache HIT and MISS, so we can detect if the vision model
returns duplicate analysis for different images (provider issue vs
cache collision). Includes the phash on cache miss (new analysis cached).
Follow-up to 9f7ce7d which fixed makeImageCacheKey to hash full data
URL instead of just first 128 chars (root cause of all images sharing
the same cached 'konten judi' verdict due to hash collision).
Root cause: makeImageCacheKey() only hashed the first 128 chars of the
data URL. Since all resized images use the same MIME prefix
('data:image/png;base64,') + identical base64 header bytes, nearly every
image got the same 16-char hash → 'image:<same-hash>' → all images reused
the first cached vision analysis (often a gambling-detection verdict).
Fix: hash the entire data URL instead of just the prefix. Verified
114 stale 'image:' entries + 745 stale 'phash:' entries purged from prod
DB. tsc --noEmit clean, 133 tests pass.
- Add term_glossary_cache table + migration 0014: resolved definitions are
stored permanently (definitions rarely change); misses stay ephemeral in
Redis/LRU with 1h TTL so transient failures get retried
- Lookup flow: LRU -> Redis -> Postgres (permanent) -> live SearXNG; DB hits
re-warm the fast caches; stale Redis miss sentinels no longer shadow DB
- Rate-limit-aware live lookups: concurrency 2 + stagger, retry once on empty
results, strict definition filter (Wikipedia preferred, rejects
disambiguation/ads/translate-homepages)
- Make SEARXNG_BASE_URL configurable via env (default unchanged)
asepharyana
asepharyana
