- new tinyFishSearch module: GET api.search.tinyfish.ai with X-API-Key,
maps top-3 to SearchResult shape, never throws (all failure modes -> [])
- wikipediaSearch: on wiki miss, one tinyfish attempt; hits cached 6h
under the same key so fallback latency is paid once
- termGlossary: on summary miss, top tinyfish hit becomes the definition
(persisted permanently like wiki defs); miss keeps 1h sentinel
- config: TINYFISH_API_KEY (empty = fallback disabled), ENABLED,
BASE_URL, TIMEOUT_MS, LOCATION, LANGUAGE knobs
- tests: 6 coverage for disabled/mapping/non-OK/network/bad-json
API key NOT committed — set TINYFISH_API_KEY in BWS gmw secrets.
Verified: typecheck + lint clean, 216/216 tests pass, live probe
'gubernur jawa barat' returned 3 mapped results
- attachmentUploader: downloadDiscordAttachment retries CDN timeouts
via retryWithBackoff (ATTACHMENT_RETRY_ATTEMPTS, 1s-8s backoff);
AbortError normalized so 403/404 refresh path never fires on timeouts
- attachmentUploader: uploadAttachmentToTele uses ATTACHMENT_RETRY_ATTEMPTS
instead of retries:0 (Tele 5xx under load was failing outright)
- wikipediaClient: wikipediaSummary retries once on abort/timeout
(100% of prod summary errors were aborts); termGlossary drops its
redundant second call (was up to 4 reqs/term under miss+retry)
Verified: typecheck + lint clean, 210/210 tests pass
Jockie Music (user 411916947773587456) posts now-playing embeds/spotify links
~1347 captured messages — every one consumed a moderation LLM call for zero
signal and contributed to batch timeouts. Config AI_SKIP_ANALYSIS_USER_IDS
(default=Jockie) skips them at ALL three analysis paths:
- queueMessageAnalysis entry (direct skip-result like age-restricted)
- batchScheduler processing (pre-batch filter)
- individual recovery path (no fallback spam for already-skipped authors)
Skip-result mirrors age_restricted: status=clean, flags=[skip_analysis_user],
action=none — stays visible in the dashboard, never analyzed.
The text model behind omniroute/9router consistently takes >45s on long-context
batches. At 45s every such batch fell through to the individual-fallback
queue which re-runs with its own timeout, then exhausted to ai_status=error.
75s keeps the bounded budget while letting the first-pass batch succeed.
- AI_LLM_MEDIA_ANALYSIS_TIMEOUT_MS 60s→120s + vision 60s→120s: vision model
via router regularly exceeded 60s, dropping media batches into the
individual-fallback chain then exhausting into ai_status=error.
- Qdrant upserts: retryWithRetry() wraps PUT /points with exponential
backoff (3 attempts, jitter) for transient 408/abort/ECONNRESET — the
41 six-hour 'Qdrant upsert failed — semantic entry skipped' warnings were
single-hop timeouts on a healthy-but-loaded Qdrant.
- LLM caller: on parse failure, attempt extractJson() structural repair of
the raw content (models with thinking disabled sometimes emit JSON as
plain text) before giving up and re-requesting.
C2 full scope persistence:
- getRecentConversationContext now returns per-turn guildId/channelId
- processMessage merges historical scope when current request is unscoped
- chatbot remembers server context across all 8 history exchanges (not just 3)
A4+Metrics:
- Add incrementCounterBy(name, delta, labels) to gateway-metrics
- Token-usage counters: llm_tokens_total{model, type, label} per batch
- Cache hit counters: moderation_cache_hits{type} exact/semantic-qdrant/semantic-pg
- Cache miss counters: moderation_cache_misses per batch
- Prometheus /metrics now exposes cost + cache hit-rate for dashboards
Performance:
- Parallel tool execution within each chatbot round (Promise.all)
- All tool results collected before sending to model (ordering preserved)
- Tool failure now logged with structured warning (chatbot.tools.ts)
Verified: backend tsc+biome 37/37, discord-gateway tsc+biome 210/210
Even with the prompt firewall (username vs content), the LLM can still
occasionally mis-apply a content-level zero-tolerance flag (sara /
conflict_instigation) to a message whose ONLY violation is the username
(e.g. 'matikanetanyahu'). The auto-delete eligibility check only recognized
exact offensive_username flags, so such false positives still deleted the
message.
Add a belt-and-suspenders guard in isNicknameOnlyViolation: if the flag set
is entirely username-attributable (offensive_username/sara/conflict_instigation)
AND the analysis text corroborates that the violation is username-only with
clean message content, route to nickname-reset instead of message deletion.
Adds 6 test cases covering the real matikanetanyahu scenario and the
false-positive/negative boundaries.
Add explicit FIREWALL PENILAIAN rule separating username assessment from
message content assessment. Username containing political/religious terms
(matikanetanyahu etc) is assessed as offensive_username with severity low
— never triggers sara/conflict_instigation zero tolerance for content.
Changes:
- rules.ts: Add FIREWALL section before LARANGAN BERAT; clarify each
zero-tolerance rule applies to ISI PESAN only; update hierarchy #9
- examples.ts: Fix example #11 status flagged→warn for clean username;
add example #11b with matikanetanyahu case
- output.ts: Explicit status/action mapping for username-only violations
result.score was required by zod; the LLM (gemini-3.5-flash-lite via 9router)
occasionally omits it for media batches, hard-failing the whole batch parse
('Zod validation failed: expected number, received undefined' at
results[0].score). Callers already null-coalesce (result.score ?? 0) and the
parser clampScore()s it, so requiring it only caused parse failures.
Adds regression tests: media-batch without score parses (score->0), and
score-present responses still parse with the value.
A selfbot (user token) cannot auto-detect other members' camera/share
(no VOICE_STATE_UPDATE for others, 403 on member fetch). The only
selfbot-viable path to capture another member's SCREEN SHARE is an
operator-initiated STREAM_WATCH (gateway op 20, not gated on bot-vs-user).
Add video:watch / video:unwatch Redis commands routed via the existing
command handler to startStreamWatch/stopStreamWatch, which then does the
DAVE handshake + per-burst MP4 segmentation + DB insert + Tele upload
(already implemented in streamWatchReceiver).
- new VideoHandler (command-handler/video.handler.ts)
- register video:watch / video:unwatch in handler-registry + CommandHandler
- command constants COMMAND_VIDEO_WATCH / COMMAND_VIDEO_UNWATCH
- resolve active voice channel from voice controller + client cache
- 8 unit tests (videoHandler.test.ts)
- biome fixes for pre-existing test import ordering
All green: typecheck, build, lint (174 files), 200 tests.
Dump all WS raw event types after 10s (see if GUILD_CREATE exists at all),
and log broadcaster_user_ids + sessions.voice from READY payload.
Selfbot may RESUME (skip GUILD_CREATE) — voice data may only be in READY.
Temporary diagnostic to see whether GUILD_CREATE.voice_states actually
reaches the selfbot raw listener (selfbot-v13 emits everything via
WebSocketShard Events.RAW). Will remove once root cause confirmed.
Real root cause of 'tidak mendeteksi user yg sudah ada di voice':
The selfbot's discord.js-selfbot-v13 GUILD_CREATE handler only sends
GUILD_SUBSCRIPTIONS_BULK — it DROPS d.voice_states from the payload.
So when the gateway (re)starts, every user who was ALREADY in the voice
channel (with camera or screen-share on) is invisible: channel.members is
empty, and REST fallbacks don't work for user tokens (verified live:
GET /channels/{id}/voice-states -> 404, GET /guilds/{id}/members -> 403).
Fix: register our own raw listener for GUILD_CREATE, capture
d.voice_states per guild (buffer it), and consume it in
scanExistingStreamers when the channel gets tracked (voice join happens
after GUILD_CREATE). This is the ONLY user-token-compatible source of
'who is in voice with video right now'.
- videoRecorder: +pendingGuildCreateVoiceStates buffer, +raw GUILD_CREATE
listener, Path C consumes buffered states (self_video/self_stream,
matching channel, skip self) before the REST fallback
- scanExistingStreamers: Path A cache -> Path C GUILD_CREATE -> Path D
REST members.fetch() best-effort
- +1 test: GUILD_CREATE voice_states buffer -> watch camera+screen-share,
skip self/no-video/other-channel (192/192 pass)
- typecheck + build + biome clean (1 pre-existing warning)
Root cause: scanExistingStreamers only read channel.members, which is
EMPTY after a gateway restart because the selfbot's guild member cache
hasn't been populated yet. So any user who was ALREADY on camera /
screen-sharing when the bot (re)joined was never detected → no video.
Fix: when the member cache is empty, fall back to guild.members.fetch()
(REST GET /guilds/{id}/members — user-token compatible; the bot-only
GET /channels/{id}/voice-states returns 404 for selfbots, verified live)
which populates member.voice states, then re-scan channel.members for
streaming/selfVideo.
- scanExistingStreamers is now async; trackChannel fire-and-forgets it
- logs source=cache vs source=rest-members for observability
- +1 test: cold-start channel.members empty → REST fetch → watch camera user
- 191/191 tests pass, typecheck + build + biome clean
Previously forceSelfServerUnmuteUndeafen only ran on video-watch attempts and
after voice reconnects — so when an admin server-muted or server-deafened the
bot, it stayed muted/deafened for minutes (or forever if no streamer came on).
Add registerSelfVoiceStateGuard: a voiceStateUpdate listener that detects the
bot's own serverMute/serverDeaf transition to true and immediately re-issues
mute:false,deaf:false. Wired at client-ready in bootstrap.ts. Best-effort,
idempotent, never blocks the gateway.
videoReceiver.getSsrcInternalMap read asAny._map, but @discordjs/voice
0.19.x exposes the SSRC map as the public field 'map'. So inferVideoOwner
(e.g. matching a video RTP SSRC to a user via audio-SSRC proximity) always
returned null -> every video packet was silently dropped after decryption.
Accept both 'map' and '_map'. Unblocks camera/screen-share attribute when
op12 does not carry a videoSSRC stream entry.
streamWatchReceiver: DAVE decrypt for screen-share video packets was
returning null every time (VIDEO-PKT diag showed packets arriving but
no 'Video segment opened'). Three possible causes:
1. No VIDEO decryptor in MLS group (audio-only handshake)
2. GoLive stream tags video packets as AUDIO
3. Screen-share payloads are unencrypted above the AES layer
Fix: try MediaType.VIDEO first, then MediaType.AUDIO, then passthrough
(legacy-decrypted payload as-is). This covers all three modes without
breaking audio recording.
Fork @discordjs/voice 0.19.2 into vendor/discord-voice-fork and patch the
voice gateway handshake so Discord sends camera/screen-share RTP video:
- Identify payload now declares video:true + streams:[] (derived from
Discord-RE/Discord-video-stream) - this is what makes Discord deliver
H264 (PT 96-127) to the voice socket. Previously the client never
declared video capability, so Discord omitted all video SSRCs/packets.
- op-12 Speaking handler maps streams[].ssrc -> videoSSRC alongside the
audio SSRC, so ssrcMap carries camera/screen-share stream IDs.
- SSRCMap.get() now also resolves video SSRCs (video RTP arrives on a
different SSRC than audio), enabling attribution for videoReceiver.ts.
- Both dist/index.js (CJS) and dist/index.mjs (ESM) patched; 3 isolated
hunks vs upstream, verified by diff.
- package.json points @discordjs/voice -> file:vendor/discord-voice-fork
(pnpm lockfile updated, CI --frozen-lockfile compatible).
- .gitignore: replace global dist/ with per-service explicit patterns so
the vendored fork's dist/ is committed while build outputs stay ignored.
- tests: +3 SSRCMap videoSSRC cases (190 total, all pass).
The receive-side pipeline (H264Depacketizer -> .h264 -> muxToMp4 -> MP4)
already exists in videoReceiver.ts; this unblocks it by making Discord
actually deliver video packets.
Previously only member.voice.streaming (screen share, self_stream) triggered
video capture. Discord reports camera via self_video:true, so camera-only
users were never watched. Now scanExistingStreamers and handleVoiceStateUpdate
start a watch when streaming OR selfVideo is set, and stop it when both clear.
Video (camera + screen share) DAVE stream-watch now produces per-burst
MP4 segments instead of one long .h264 per watch:
- Detects VIDEO_SILENCE_MS (4000ms) of no H264 packets → closes the
current segment, muxes to MP4, registers in voice_recordings + uploads
to TeleUploader, then reopens for the next burst (mirrors voice AfterSilence).
- Per-watch segment counter + per-segment depacketizer reset + closing
guard + write-error swallow so races (silence close vs in-flight UDP
packet) never corrupt files or crash the gateway.
- Frontend: recordings deck renders a native <video> player for MP4 rows
(detected by filename), keeps single-playback registry across audio+video.
Gateway archive embedder now parses metadata.channel.{channelName,threadName}
from each message and stores channel_name/thread_name in the Qdrant payload.
Backend exposes them; the semantic results card renders the thread name (or
channel name) instead of a raw #snowflake, with the ID as a last-resort
fallback for legacy points. Matches the message feed's channel-label logic
(getMessageChannelLabel).
Archive payload now stores username, channel_id, guild_id, thread_id and the
real message created_at (not embed time). Backend searchArray accepts an
optional guildId and applies a Qdrant payload filter so results can be scoped
to the guild being viewed. API/frontend expose the new fields and the
semantic results card shows who said it, in which channel, and when —
turning bare text blobs into contextual results. Old points fall back to
analyzed_at and omit the new fields gracefully.
- AI_VOICE_TRANSCRIPTION_MODEL config (default whisper-1) so the model can be a provider-qualified id (openrouter/openai/whisper-1) that actually has credentials through 9router/omniroute — bare whisper-1 maps to the openai provider which has none
- response_format json (not text): 9router proxies only json/verbose_json transcription responses; text returns 400
- parse text from the json response object
- prod env updated: model=openrouter/openai/whisper-1 (still needs OpenRouter STT balance — 402 until funded)
- Normalize text before embedding (strip mentions/URLs/emoji/markdown/control chars, lowercase, truncate) on both write and query sides so vectors aren't diluted and tokens aren't wasted
- embeddingClient: retry embeddings (maxRetries 2), validate batch dimension consistency, preserve index alignment for empty-normalized texts
- archiveEmbedder: store normalized text in archive payload, skip empty-normalized content
- backend: normalize search queries, make archive search similarity threshold configurable (AI_LLM_EMBEDDING_ARCHIVE_MIN_SIMILARITY, default 0.6)
The bot's own VOICE_STATE_UPDATE showed server-level deaf:true — a
server-deafened member is NOT sent the streamer's audiovisual RTP by Discord,
which is the likely reason no H264 arrives despite the DAVE watch reaching
Ready. The previous fire-and-forget forceSelfServerUnmuteUndeafen() ran once
after the first Ready join and silently reverted on reconnect/restart.
- Export forceSelfServerUnmuteUndeafen from recorder.ts; re-assert it (with
read-back verification logging stillDeaf) at the START of every
startStreamWatch() before STREAM_WATCH is sent (dynamic import avoids the
recorder <-> videoRecorder <-> streamWatchReceiver module cycle).
- Re-assert it again after a successful voice reconnect.
- startStreamWatch() is now async; callers use void.
maxLen stayed 72 across 243 packets (no real H264, which is hundreds+ bytes) —
only 44-72-byte RTP packets on PT 76/72/73 arrive. Add ssrc + first-32-bytes
hex so we can identify exactly what Discord sends to the watch socket (control
packets vs stale video), which determines whether the gap is upstream routing
or whether large H264 packets are missing entirely.
Enhance watch-socket diagnostic to report distinct RTP payload types seen and
the max packet length, so we can distinguish 'only small control packets arrive
(no real H264)' from 'H264 arrives but decrypt fails'. Live already confirmed
dave=true ready=true with packets flowing but no burst — need to know if they're
tiny 52-byte control packets (PT 73) or large H264.
Instrument handleUdpMessage to log (rate-limited, first 3 then /20s) whether
video RTP packets actually reach the watch socket, and whether the DAVE session
is attached+ready and encryption key material present. Needed to diagnose why
no .h264 is written despite DAVE Ready + MLS: is the packet not arriving, or is
decrypt returning null?
CRITICAL: djs/voice stores the DAVESession wrapper at net.state.dave
(createDaveSession assigns to state.dave on op4 SessionDescription), NOT
inside connectionData. decryptVideoPacket looked up connectionData.dave which
is ALWAYS undefined -> every video packet hit '!dave?.session' guard and was
silently dropped, so no .h264/.mp4 ever got written despite the handshake
reaching Ready.
Fix: pass net.state.dave as a separate arg (the wrapper has .session ->
Davey.DAVESession) so the DAVE-layer decrypt (MediaType.VIDEO) actually runs.
Typecheck + build pass, lint clean (src/), 179/179 tests.
If someone is ALREADY sharing screen / camera on when the bot joins the
channel, no voiceStateUpdate with streaming:true fires for them, so the
bot never sent STREAM_WATCH and missed their video entirely. trackChannel
now scans channel.members and starts a watch for anyone already streaming
(ignoring the bot itself and non-streamers). Idempotent: startStreamWatch
no-ops if a watch already exists. +2 tests (9/9 in videoRecorder).
djs/voice Networking emits stateChange(oldState, newState), but the watch
handler declared (newState, oldState) -- reversed. So the code-4 Ready
branch (which attaches udp.on('message') + logs 'DAVE watch READY') never
fired when entering Ready; it only fired spuriously when LEAVING Ready.
Result: full DAVE handshake completed on the watch RTC (Ready + DAVE MLA +
video stream 21029 active 1920x1080@60) but no UDP listener -> no video
captured. Swap to (oldState, newState) so enter-Ready wires the socket.
Add watch-state N->M log on every djs/voice Networking stateChange (with
hasUdp flag) so the stream-watch connection's exact progression is visible:
OpeningWs(0)->Identifying(1)->UdpHandshaking(2)->SelectingProtocol(3)->
Ready(4). Pins down where the DAVE flow stalls instead of guessing from the
absence of logs. Pairs with debug:true + watch-djs-debug.
Add debug:true to watch Networking options and wire net.on('debug') to
logger.info so djs/voice internal WS/DAVE state transitions appear in
journal. Without this, the stream-watch connection went silent after
'Streaming DAVE Networking' — no ready/error/close visible. Needed to
diagnose why the WS to stream endpoint 'c-sin14-xxx:2083' produced no
events.
Live log (14:52) showed the stream-watch flow reaching STREAM_CREATE +
STREAM_SERVER_UPDATE but then DAVE processProposals threw
'ValidationError(WrongGroupId)' -- the Davey MLS session derived the wrong
group because connectionOptions.channelId was the guild voice channel id.
Per Discord-RE StreamConnection.daveChannelId = BigInt(serverId)-1n, the
stream-watch DAVE MLS group is keyed to rtc_server_id-1, not the vc channel.
Fix: pass BigInt(serverId)-1n as channelId to the watch Networking.
This error also surface as an uncaughtException that crashed the gateway
(systemd restarted it). Correct channel id prevents it at the root.
dependabot[bot]
asepharyana
asepharyana
mytheclipsebotreview