From 6738279bade24c32990ec7503fde55a0dc84de14 Mon Sep 17 00:00:00 2001 From: asepharyana Date: Sun, 30 Aug 2026 23:26:59 +0700 Subject: [PATCH] feat(gateway): record others' video via native selfbot watch/receive (Phase C) Root cause of why Phase A/B captured zero video: @discordjs/voice is audio-only and never sends the gateway STREAM_WATCH signal, so Discord never forwards a member's video RTP to the bot. Live diagnostic confirmed: while members were sharing, audio .ogg files flowed for many users but no non-opus RTP ever arrived. Correct path: discord.js-selfbot-v13 ships a complete native watch/record stack. New src/modules/voice-recording/videoRecorder.ts: - Detects streamers via voiceState.streaming on a single idempotent voiceStateUpdate listener. - client.voice.joinChannel() (reuses the single session alongside @discordjs/voice) + joinStreamConnection(userId) -> STREAM_WATCH (op 20). - receiver.createVideoStream(userId, path) -> PacketHandler -> Recorder (ffmpeg over UDP loopback) -> Matroska .mkv, decryption handled internally. Wired in recorder.ts (trackChannel/untrackChannel) + bootstrap.ts (setVideoRecorderClient/setVideoRecordingsDir). All best-effort; failures log and never break existing voice/audio. Unit tests 6/6 (single listener, watch handshake + mkv path, skip own video, idempotence, teardown). Full suite 20 files / 176 tests green; typecheck + build + biome clean. UNVERIFIED live yet: needs deploy + a streamer to confirm Recorder ready + playable .mkv. --- .../2026-08-31_video-receive-phaseC-spec.md | 97 +++++++ services/discord-gateway/src/app/bootstrap.ts | 10 + .../src/modules/voice-recording/recorder.ts | 5 + .../modules/voice-recording/videoRecorder.ts | 268 ++++++++++++++++++ .../tests/videoRecorder.test.ts | 120 ++++++++ 5 files changed, 500 insertions(+) create mode 100644 .hermes/plans/2026-08-31_video-receive-phaseC-spec.md create mode 100644 services/discord-gateway/src/modules/voice-recording/videoRecorder.ts create mode 100644 services/discord-gateway/tests/videoRecorder.test.ts diff --git a/.hermes/plans/2026-08-31_video-receive-phaseC-spec.md b/.hermes/plans/2026-08-31_video-receive-phaseC-spec.md new file mode 100644 index 00000000..f7acf566 --- /dev/null +++ b/.hermes/plans/2026-08-31_video-receive-phaseC-spec.md @@ -0,0 +1,97 @@ +# Spec: Record Other Users' Video (Camera / Screen Share) — Phase C + +Status: PLANNED (not built) +Date: 2026-08-31 +Author: Hermes +Related: `.hermes/plans/2026-08-30_video-record-receive-spec.md` (Phase A/B — raw UDP hook, superseded for receive) + +## TL;DR — what changed vs Phase A/B + +Phase A/B (commit `999c054b` etc.) hooked `@discordjs/voice`'s UDP socket to capture non-opus RTP and +depacketize H264 → mp4. **It captured ZERO video** because `@discordjs/voice` never authorizes the bot to +receive others' video (no STREAM_WATCH). This spec replaces that approach with the **native, selfbot-lib +receive path**, which is battle-tested and does the authorization + decryption + ffmpeg muxing for us. + +## Ground truth (verified in discord.js-selfbot-v13 3.7.1 source) + +1. `ClientVoiceManager.joinChannel(channel, config)` → a **selfbot `VoiceConnection`** with + `.receiver` (`VoiceReceiver` → `PacketHandler`). [ClientVoiceManager.js:102-118] +2. `VoiceConnection.receiver` is created in the constructor. [VoiceConnection.js:140] +3. `VoiceReceiver.createVideoStream(user, output)` → `PacketHandler.makeVideoStream` → **`Recorder`** + (ffmpeg that muxes H264+Opus RTP over UDP → **Matroska (.mkv)**). [Receiver.js, Recorder.js] +4. `PacketHandler` routes: video RTP → Recorder UDP 65506, opus RTP → UDP 65510; decodes all via + `connection.authentication.{secret_key, mode}` (supports `aead_aes256_gcm_rtpsize` and + `aead_xchacha20_poly1305_rtpsize` = DAVE-compatible). [PacketHandler.js:115-155, 195-240] +5. `StreamConnectionReadonly.joinStreamConnection(userId)` + `sendSignalScreenshare()` sends + gateway op `STREAM_WATCH` so Discord actually forwards the streamer's RTP to us. [VoiceConnection.js:1100-1240] +6. `VoiceState.streaming` = `data.self_stream ?? false` — lets us detect a streamer on voice state update. [VoiceState.js:94] + +## Problem / the crux + +The gateway's voice today is **`@discordjs/voice`** (audio + music + GoLive-send). The selfbot-lib +video-receive path lives on the **selfbot-lib `VoiceConnection`** — a separate voice stack. Two options: + +### Option A (RECOMMENDED): Parallel selfbot video-watch connection +Keep `@discordjs/voice` for everything it does today. Add a **second, selfbot-lib voice connection** +to the same channel whose ONLY job is to watch + record others' video. + +- Pros: zero regression risk to audio/music/screenshare-send; uses native `createVideoStream` → mk4. +- Cons: two voice connections for the same bot user in one channel. Need to verify Discord tolerates it + (real selfbots like Discord-RE do exactly this for multi-stream). The selfbot lib's `joinChannel` + reuses `ClientVoiceManager.connection` (it's a singleton) — see caveat below. + +### Option B: Migrate primary voice to selfbot lib +Make the selfbot `VoiceConnection` THE voice layer (it also does audio via `receiver.createStream`). +- Pros: one connection; video+audio unified. +- Cons: large refactor; high regression risk to the entire existing audio/music/GoLive stack. NOT chosen now. + +## CAVEAT — ClientVoiceManager.connection is a singleton +`ClientVoiceManager.connection` is a single `VoiceConnection`. The gateway's `@discordjs/voice` adapter and +the selfbot lib both drive the same client voice state. Need to verify whether `client.voice.joinChannel()` +can coexist with the active `@discordjs/voice` session, or whether we must create the selfbot VoiceConnection +manually / re-use the existing voice state. This is the #1 technical risk to validate in the spike before +committing to Option A. + +## Implementation plan (Option A) + +### 1. Streamer detector (new: `modules/voice-recording/videoRecorder.ts`) +- Listen to voice state updates (`client.on('voiceStateUpdate')` or the existing voice-state hook). +- When `voiceState.streaming === true` for a member in the bot's channel → candidate to record. +- Skip bot's own user id (unless we also want self-video; default skip). + +### 2. Watch + record wiring +- Ensure a selfbot-lib `VoiceConnection` exists for the channel (spike: `client.voice.joinChannel(channel)`, + fallback: build a `VoiceConnection` directly from the existing voice auth). +- `await selfbotVoiceConn.joinStreamConnection(userId)` → STREAM_WATCH op 20. +- `const recorder = selfbotVoiceConn.receiver.createVideoStream(userId, outPath)` where outPath points under + `//video--.mkv` (Recorder outputs MKV natively). +- On `recorder.on('ready')` → mark recording; `recorder.on('closed')` → finalize. +- Transcript later: MKV → mp4 via ffmpeg (Phase B `muxToMp4` can accept mkv) for dashboard playback. + +### 3. Teardown +- When `voiceState.streaming === false` / user leaves / channel emptied → `recorder.destroy()`, + `selfbotVoiceConn.streamWatchConnection.delete(userId)` / `sendStopScreenshare()`. + +### 4. Frontend (Phase UI, later) +- oRPC/backend list `.mkv` per call session + FE `