Test 00:32 confirmed the video pipeline WORKS (1410 frames @ 1280x720 sent,
ready=true, camera off) but the gateway crashed at stream stop:
unhandledRejection → graceful shutdown → systemd restart (bot offline).
Root cause candidates (both were fire-and-forget promises without .catch):
- BaseMediaConnection.setProtocols().then(...) — rejects when the PC is
closed while setProtocols is in flight (stream teardown)
- void webRtcConn.createOffer().then(...) — rejects when the PC closes
while the offer is still gathering
Fixes:
- .catch on both promise chains (log + continue; teardown is expected)
- unhandledRejection handler now treats transient stream errors (EPIPE,
ERR_STREAM_DESTROYED, ERR_STREAM_WRITE_AFTER_END, ECONNRESET) like
uncaughtException already does — warn + continue instead of shutting
down the whole gateway. Non-transient rejections still log + shutdown
(with String(reason) so the detail actually shows).
ROOT CAUSE of empty GoLive tile (finally): prepareStream ran with
includeAudio: true + output -f h264. The h264 muxer cannot mux audio
('h264 muxer does not support any stream of type audio') → header write
fails -22 → stdout empty → Demuxer ffmpeg 'Invalid data found when
processing input' → 0 frames → black tile. Reproduced locally end-to-end
(13s backpressure delay + prepareStream + demux).
Fixes:
- screenShareController: includeAudio: false (video-only GoLive; demux
path never delivers audio anyway)
- Demuxer: pin input format -f h264 for stream inputs (raw AnnexB H264
has no magic header → auto-detect unreliable on delayed pipes)
- Streamer.signalStream: self_video: false — stop flipping on the bot's
camera in Discord (user request; screen share ≠ camera)
Verified: local repro now emits 644 frames 1280x720 (was 0); tsc/biome/
vitest all green.
Root cause of 'tile appears but content empty': demux() spooled the live
NUT/H264 input to a temp file and awaited stream 'finish' — but the merge
ffmpeg output never ends during playback, so demux deadlocked, no probe,
no transcode, 0 frames sent.
- Demuxer: pipe input straight into ffmpeg stdin (-i pipe:0), parse NAL
frames live from stdout; parse video metadata from ffmpeg stderr with a
1.5s race (fall back to H264 defaults). No spool, no await-end.
- screenShareController: pass width/height/frameRate (1280x720@30) to
playStream — matches the prepareStream encode settings, so setVideoAttributes
gets real dimensions even when ffmpeg can't report metadata on an open pipe.
- Add tests/golive-demux-live-e2e.ts: proves frames flow while input is
still open (regression test for the deadlock).
- signalStream: flip voice state to self_video:true/self_deaf:false before
STREAM_CREATE (Discord silently ignores the request while video disabled)
- createStream: attach dispatch listeners before first signal (race), clean
up listeners on timeout, retry STREAM_CREATE every 3s up to 4 attempts
(upstream issue #217/#219 — Discord randomly drops the request)
- sendOpcode: direct [goLive:Streamer] log bypassing bootstrap debug filter
(proves op 18 is actually broadcast)
binding.gyp appended '/libdatachannel.so.0.24.0' to LDC_LIB; nixpkgs output
layout is <out>/lib/libdatachannel.so.0.24.1. Make LDC_LIB the complete
library path (env or default) and drop the append.
Track.setPacketizer(kind, ssrc, pt, clockRate, ...) builds the same
media-handler chain node-datachannel does for @dank074:
RtpPacketizer (Opus | H264 | H265 | AV1) → RtcpSrReporter →
RtcpNackResponder → PacingHandler(25Mbps, 1ms) for video
Track.sendFrame(encodedFrame) packetizes into RTP; addTimestamp(delta)
advances the RTP timestamp (node-datachannel contract).
Verified test-packetizer.js: two peers connected over tracks, real opus
frames + AnnexB H264 (SPS/PPS/IDR) flow through the chain without crash.
This removes the need for a JS RTP packetizer entirely — libdatachannel
0.24 has the full media stack built in.
Expose rtc::Track with send(binary) for raw RTP — verified:
- SDP from addTrack(audio)+addTrack(video) has m=audio (opus 120)
and m=video (H264 101 + H265/VP8/VP9/AV1 + RTX)
- libdatachannel Track::send() sends RAW RTP/RTCP when no media
handler is set (verified in src/track.cpp impl::Track::outgoing) —
so RTP packetization can live in pure JS, keeping the binding minimal
Also fix: Track class was missing from InitAll exports (crash on
TrackWrap::NewInstance — null FunctionReference).
Phase 0 of GoLive rewrite (drop @dank074/node-datachannel 771MB):
minimal N-API binding exposing PeerConnection/DataChannel/ICE/SDP,
built against libdatachannel 0.24.0 (from node-datachannel _deps source).
Verified: offer/answer/ICE/DataChannel roundtrip between two local
peers (test-handshake.js). Key findings:
- callbacks must be registered in ctor BEFORE createDataChannel
- SDP with candidates comes from localDescription() at gathering Complete
- answer auto-generates on setRemoteDescription(offer); do NOT call
setLocalDescription() after or role=actpass breaks the peer
prepareStream (from @dank074/discord-video-stream) unconditionally appends
audio filters 'volume@internal_lib' + 'azmq' that exist ONLY in its custom
node-av jellyfin-ffmpeg build. The Nix deployment runs plain ffmpeg-headless
on PATH, so fluent-ffmpeg died instantly with 'Filter not found' (exit 8),
the NUT output stream stayed empty, and playStream's node-av demux failed
with 'Failed to open input from Readable stream: Invalid data found when
processing input' — every screen share failed ~100ms after start.
Fix: pass customFfmpegFlags ['-filter:a','anull'] — ffmpeg applies the LAST
-filter:a for a stream, so the trailing no-op filter overrides the custom
chain (verified: command ends with '-filter:a anull', transcode runs, node-av
demux finds video+audio). Realtime volume control was already removed from
GMW (a690e5b), so dropping the filters is lossless.
Verified end-to-end with the failing URL (youtu.be/fONoh7Pc6VU, AV1+Opus
DASH): getDirectScreenInput → NUT merge → patched prepareStream → node-av
demux finds H264 video + Opus audio streams.
Root cause (3rd layer after 50371bd + 4f4c435): a vision model run
(2026-08-10) returned 'Maaf, saya tidak melihat gambar apapun yang terlampir...'
and that text was cached as a VALID vision_llm result (image + phash keys,
24h/7d TTL). Every subsequent analysis of the same image (same hash/phash)
hit the poisoned cache, so image analysis looked broken forever even though
9router responded fine — the moderation LLM wrote 'lampiran yang gagal
terbaca' from a cache hit.
Also: mimo via 9router streams reasoning in delta.reasoning +
delta.reasoning_details[].text (content:"") — extractChunkText only read
delta.reasoning_content, so those runs aggregated empty → 'Vision API null
response' (observed 08:54/09:07/09:38).
Fixes:
- llmClient.extractChunkText: fall back to delta.reasoning and
reasoning_details[].text (mimo), on top of reasoning_content (gemma).
- visionAnalyzer: isNoImageSeenText() detects 'no image' style outputs;
such results are NEVER cached, and poisoned entries are purged when hit
(LRU/DB/phash) so re-analysis actually re-runs vision.
- Tests: reasoning/reasoning_details extraction + isNoImageSeenText
(Indonesian + English, no false positives on real descriptions).
Root cause (2nd layer after 50371bd): the analysis worker could pick up an
image message while its attachment upload was still in flight
(upload_status='pending'). downloadAndExtractFrame then fell back to the
Discord CDN URL (cdn.discordapp.com), which often 404s for old/purged links,
and 'if (!res.ok) return' silently dropped the image — no log, no vision
call, empty image map, and the LLM produced a text-only verdict like
'lampiran yang gagal terbaca oleh sistem'.
Fixes:
- ai-analysis-worker: skip targets whose attachment upload is still pending
(both batch + individual paths) — they stay ai_status='pending' and the
next 15s cycle analyzes them after the upload lands.
- mediaDownloader.downloadAndExtractFrame: try uploaded_url first, then
discord_url as fallback; log non-OK responses (status + host) instead of
silently returning; log when all candidate URLs fail.
Root cause: 9router combo 'multimodal' routes to cloudflare-ai/@cf/google/
gemma-4-26b-a4b-it which streams ALL output in delta.reasoning_content
(content:"") and finishes with 'length' at max_tokens. llmClient only read
delta.content, so llmVision returned empty → every image moderation fell back
to text-only analysis ('Meskipun analisis gambar gagal' in every ai_analysis).
Fix: extractChunkText() prefers delta.content then falls back to
delta.reasoning_content (also handles message/text/response fields), with
unit tests for the exact 9router chunk shape. Verified live against a real
DB image: oc/mimo-v2.5-free (new first model in the multimodal combo) returns
a proper description in delta.content.
- <message> targets now carry time (ISO), repetitions (N identical short texts = spam signal), bot and edited flags; escape id/user XML
- rich <user_reputation>: total_infractions, clean_streak, last_offense_days_ago, repeat_offender (7-day window)
- <user_history> with last flagged messages for repeat offenders (wires dead getUserRecentInfractions)
- <user_profile as_of> staleness signal; <location_context topic> from captured channel topic
- prompt framing + output instructions teach the LLM to use the new signals without treating history as proof
- tests: contextEnrichment.test.ts (13) + topic cases in conversationContext.test.ts
When the ONLY violation is offensive_username (message content clean):
- Message is NOT deleted (nickname-only violation bypasses auto-delete)
- Member's server nickname is reset to default username via
setNickname(null) (Discord shows the global username again)
- Action 'reset_nickname' logged to moderation_actions; cooldown
10min per guild:user (LRU) so repeated messages by same member
don't hammer the Discord PATCH
- Config: AUTO_NICKNAME_RESET_ENABLED / AUTO_NICKNAME_RESET_COOLDOWN_MS
- resolveDisplayName(): member.displayName from captured metadata,
falls back to global username
- Applied to context lines, target message blocks, and media message
blocks — LLM sees the name the channel actually sees (nickname can
carry moderation signal itself)
- Conversation context recency gates (GAP_MS/MAX_AGE_MS): drop stale
messages before silence gaps; cold_start anchor + flow descriptor
tells LLM whether conversation is ongoing or restarted
- [location] block: channel name, thread name, nsfw/age flags from
captured metadata (thread names instead of bare IDs)
- Link media -> multimodal: text-batch URL fetches that resolve to
images now run vision analysis (bounded 15s) and switch prompt to
mixed mode; <web_content> gains og:title for page context
- pnpm-workspace.yaml: approve sharp build script (unblocks install)
A single long guild name was clipped (whitespace-nowrap + narrow min-width),
so the dropdown rendered as a tiny 144px box with truncated text. Size the
popup to fit-content up to the available width and let option text wrap.
Select dropdowns (voice tab, guild selector) rendered with a transparent
background because --color-popover/--color-input were undefined, and the
popup overlapped the trigger due to alignItemWithTrigger. Define the missing
theme tokens (popover, popover-foreground, input, secondary) and default the
select popup to open below the trigger.
- Implemented RadioGroup and RadioGroupItem for radio button functionality.
- Created ResizablePanelGroup, ResizablePanel, and ResizableHandle for resizable panels.
- Developed Sidebar component with context for state management and various subcomponents (SidebarTrigger, SidebarMenu, etc.).
- Added Spinner component for loading indicators.
- Introduced Table component with TableHeader, TableBody, TableFooter, and related subcomponents for structured data display.
- Built Toast component for notifications with customizable actions and icons.
- Implemented ToggleGroup and ToggleGroupItem for toggle button functionality with context support.
- Loop: toggle via POST /api/media/loop → COMMAND_MEDIA_LOOP; gateway
replays finished music track on natural end (queue untouched); status
payload exposes loop flag; FE tombol Loop di music-player + mini-player.
- Kualitas suara: music playback sekarang di-transcode sekali via ffmpeg ke
OggOpus 48kHz stereo 192kbps dengan volume di-bake ke encode — menghindari
double lossy encode (inlineVolume) yang bikin suara buram. Screen share
tetap pakai jalur lama.
- Backend: MediaState.loop, setLoop service, route + schema validation.
Volume sudah di-set default 0.3 di gateway (suara kecil saat play), dan
user bisa naikin sendiri di Discord (command media:volume) — jadi kontrol
volume lewat dashboard tak perlu. Hapus:
- BE: POST /api/media/volume route, mediaVolumeSchema, setVolume service
- FE: useMediaVolume hook, mediaApi.volume, slider volume di music-player
+ mini-player, field volume/setVolume di MediaPlayerProvider
Pertahankan COMMAND_MEDIA_VOLUME di gateway (masih dipakai command DC)
dan mic volume (terpisah, tetap di voice page).
Volume play music terlalu besar buat user — default sekarang 0.3 (30%)
di semua layer: player gateway (musicVolume=0.3), backend state/schema
(default 0.3), dan UI slider (fallback 0.3). User tetap bisa naikin
manual via slider volume di dashboard.
Rombak total alur data frontend: dari static-export CSR (tiap browser
fetch sendiri + akumulasi state voice per-tab) jadi server-side rendering.
Frontend (Next.js):
- next.config: output export -> standalone; halaman jadi server components
- server data layer baru src/lib/api/server.ts (GMW_BACKEND_URL, no window)
- dashboard/media/messages/moderation/recordings/voice page -> RSC yang
fetch backend di render-time, seed ke client view (SWR fallbackData)
- hook-hook utama terima initialData -> first paint data server, revalidate
SWR setelahnya, tanpa spinner-blank-load
- messages: guild/channel/tab/selected dibaca dari URL di server, page awal
di-fetch server-side
Shared realtime state (voice) server-authoritative:
- backend src/modules/voice/live-speaker.ts: agregat voice_active_user dari
gateway jadi snapshot authoritatif (single source of truth semua browser)
- GET /api/voice/status kini include activeSpeakers
- WS initial states kirim voice_state snapshot saat connect (late join
langsung dapat state yang sama, bukan daftar kosong)
- useSpeakers seed dari server snapshot + voice_state full-replace +
voice_active_user delta upsert
Deploy:
- flake.nix: frontend package build SSR standalone (server.js wrapper,
GMW_FRONTEND_PORT=4017); proxy nginx template proxy / -> Next server,
/api + /ws tetap ke backend :4001
Music playback produced no audio: with `-o -` yt-dlp streams media on
stdout and emits its `--print` title/duration headers on stderr, but
resolveMediaUrl read them from stdout — stripping two binary 'lines' off
the WebM container and corrupting the stream (player 'playing' but silent).
Now headers are read from stderr and the stdout media stream is returned
untouched.
Screenshare was failing with EACCES: getDirectScreenInput used --no-simulate,
making yt-dlp write .f*.part files into the read-only Nix store CWD. Dropped
it — simulate mode still returns requested_formats[].url in the JSON.
Adds tests/mediaResolve.test.ts (stderr-header + untouched-stream regression).
shutdown() awaited httpServer.close(), which waits for ALL open
connections. A lingering WS/keep-alive socket left the process zombie
forever after an uncaughtException (e.g. pg 'Connection terminated
unexpectedly' to imrnes) — no exit, so systemd Restart=always could
never revive it; /api/guilds returned 502 until manual restart.
Add 10s force-exit timer in shutdown(); clear it on clean completion.
getDirectVideoUrl used yt-dlp --get-url with bestvideo+bestaudio, which
prints the video-only and audio-only URLs on SEPARATE lines. Only the
first (video-only) line was used, so ffmpeg had no audio track and the
GoLive stream had no sound.
Replace with getDirectScreenInput which:
- uses --dump-single-json to fetch both fresh URLs in ONE yt-dlp run
(signature URLs expire quickly)
- returns the merged progressive URL directly when one exists
- otherwise merges the video-only + audio-only DASH URLs locally via a
child ffmpeg into a single NUT stream consumed as a Readable
- tracks the merge ffmpeg process in cleanup() so shutdown kills it too
Verified end-to-end with real YouTube URLs: yt-dlp pair → live ffmpeg
merge (NUT) → H264+opus transcode yields both streams. Added
tests/screenShareInput.test.ts covering URL / DASH-pair / error paths.
Gateway publishes {playing, activeMode, musicVolume, current, queue} but the
backend MediaState interface + normalizeMediaState dropped activeMode, so the
FE's 'Screen share active' / 'Music playing' badge never rendered. Carry it
through so the FE↔BE media contract stays in sync.
GoLive (dank074 Streamer) needs the single voice session; after the stream
ends, an automatic @discordjs/voice reconnect often races Discord's session
teardown and times out (AbortError). Restore is now best-effort with a 5s
delay; if it fails the FE shows disconnected and the user clicks Connect —
an accepted tradeoff for one-voice-session-per-user.
Immediate reconnect after Streamer.stop() races Discord's voice session
teardown → AbortError. Wait 4s so the old session is fully released before
re-joining with @discordjs/voice.
The restore callback previously read getVoiceStatus() AFTER disconnectGuild
had already cleared it, so it never knew which guild/channel to reconnect.
Now release/restore receive the status captured BEFORE the audio connection
is released, so reconnect actually happens after the GoLive stream ends.
The dank074 Streamer creates its own WebRTC voice connection, but Discord
allows only ONE voice session per user. When VoiceController (audio) was
already connected, the Streamer join hung forever (never got
VOICE_SERVER_UPDATE). Now:
1. ScreenShareController takes releaseVoice/restoreVoice callbacks.
2. Before joining, it disconnects the @discordjs audio connection via
VoiceController.disconnectGuild.
3. Streamer joins + streams GoLive.
4. After the stream ends, restoreVoice reconnects the audio connection so
mic/listen keep working.
5. media.handler wires these via a new setVoiceController accessor from
commandHandler; VoiceController is the single source of truth.
Also adds caller-bound timeouts & safe .catch() everywhere so a stream
failure can never become an unhandledRejection again.
Root cause: ScreenShareController created @dank074 Streamer but never called
streamer.joinVoiceChannel() — playStream threw 'Bot is not connected to a
voice channel', and since the code only used .finally() (no .catch), the
rejection became an unhandledRejection that took down the whole gateway
(graceful shutdown triggered, systemd restarted).
Fixes:
1. Resolve active channel + streamer.joinVoiceChannel(channel) before
prepareStream/playStream (dank074 needs its OWN WebRTC voice connection).
2. .catch() on the playStream done promise — log + kill ffmpeg instead of
crashing the process.
3. .catch() on playback.done in media.handler too.
4. stop() now kills ffmpeg AND stops the streamer's voice connection.
Audit voice (kirim/terima/music/screenshare) menemukan 3 masalah:
1. media:queue SILENT no-op — backend publish {source,mode} tapi gateway
handler baca payload.url → selalu 'received without a URL'. Backend
sekarang kirim {url,mode}, gateway terima url ATAU source (robust).
2. Voice connect gagal diam-diam saat user pilih channel tanpa permission
(joinable=false, contoh Music 32/64/128/256k). Backend+gateway sekarang
expose joinable; FE disable channel 'no akses' + empty state.
3. FE tidak kasih feedback saat connect gagal — tambah toast.error dengan
pesan dari backend.
Verified live: @discordjs/voice connect ke Lofi Radio joinable sukses
(VOICE READY, DAVE session OK) — pipeline voice sebenarnya sehat, masalah
utama UX. media:queue fix akan di-verify setelah deploy.
asepharyana