0 Fork 0

Files

T

asepharyana 419946f38e feat(gateway): record others' video (camera/screen share) — Phase A capture

@discordjs/voice only decrypts/forwards AUDIO (opus) — its onUdpMessage drops
every non-opus RTP packet (dist/index.mjs:2068 `!== RTP_OPUS_PAYLOAD_TYPE`, 120).
Video RTP (H264 camera + screen share, plus VP8/VP9/AV1) arrives on the same UDP
socket but was silently discarded.

New videoReceiver.ts wraps receiver.onUdpMessage (like screenShareAudio.ts):
- detects video payload types (96/98/101/102/106/116/126/127),
- decrypts them with the connection secret key/encryptionMode via the SAME
  receiver.parsePacket path @discordjs/voice uses for audio (so DAVE + voice
  encryption are handled identically),
- depacketizes H264 to AnnexB (single NAL, STAP-A, and FU-A fragmentation),
  waiting for a keyframe (SPS/PPS/IDR) before writing,
- writes a raw .h264 elementary stream per user per burst under
  <RECORDINGS_DIR>/<uid>/video-<ssrc>-<ts>.h264.

Attribution: a videoSSRC→user index is built from ssrcMap updates; a proximity
fallback mirrors screenShareAudio's inferScreenShareOwner. Bot's own video is
skipped.

Unit tests: tests/videoReceiver.test.ts (AnnexB start code, keyframe gating,
FU-A reassembly, orphan-fragment tolerance) — 5/5 green.

Phase A only (capture raw h264). Phase B (ffmpeg decode+mux to MP4/WebM +
persist) and Phase C (frontend playback) are follow-ups.

2026-08-30 21:20:32 +07:00

3.6 KiB

Raw Blame History

Spec: Record video (kamera/screenshare) orang lain — WebRTC receive (Request 2)

Date: 2026-08-30. Status: IN PROGRESS (Phase A).

Why this is hard (ground truth, verified from @discordjs/voice 0.19.2 source)

VoiceReceiver.onUdpMessage (dist/index.mjs:2059) drops EVERY non-opus RTP packet at line 2068: if ((msg[1] & 127) !== RTP_OPUS_PAYLOAD_TYPE) return;. So video (kamera H264? actually Discord uses VP8/H264; screenshare combines with video SSRC) is decrypted-capable but never forwarded. receiver.parsePacket (2033) DOES decrypt any payload type generically (audio + video) using connectionData.{encryptionMode, nonceBuffer, secretKey} — the only audio gate is the opus check inside onUdpMessage.

=> FIX: wrap receiver.onUdpMessage (like screenShareAudio.ts already does for screen-share AUDIO SSRCs): for packets whose payload type is a VIDEO type (payload 96 VP8, 101/102 H264, 106/116/126/127 AV1, VP9 98...), call receiver.parsePacket(...) myself to decrypt, then depacketize + write frames. Delegate opus (120) to the original handler. Delegate audio to original.

Audio already works (screenShareAudio.ts). We add VIDEO.

Science-of-the-changes below.

Phase A — capture + decrypt + depacketize to AnnexB h264 (THIS change)

Files (new): src/modules/voice-recording/videoReceiver.ts

Hook into recorder.startRecording alongside hookScreenShareAudio.
Wrap receiver.onUdpMessage:
- read ssrc = msg.readUInt32BE(8); userData = receiver.ssrcMap.get(ssrc)
- if payload type is video AND we have a "watching" subscription for that user (videoSSRC present), decrypt via receiver.parsePacket(...), then:
  - H264 (101/102 + payload 120 not): strip RTP header, reassemble FU-A fragments into AnnexB NALs (start-code prefixed), buffer until we have a full access unit (keyframe SPS/PPS/IDR or slices), append to a per- user-per-burst .h264 file.
- else delegate to original onUdpMessage.
Watch receiver.ssrcMap "create"/"update" for videoSSRC !== undefined → signal a video burst started for that user (like screenShareAudio does).
Per-user video files written to config.RECORDINGS_DIR/<uid>/video-<ts>.h264.
Guard: skip bot's own video (client.user.id).

Dependencies: NO new npm deps for Phase A (only crypto already in @discordjs/voice via parsePacket + Buffer). ffmpeg-headless (already in Nix buildInputs) used in Phase B for decode+mux.

Phase B — decode + mux to playable MP4/WebM (follow-up)

Pipe AnnexB h264 → ffmpeg -f h264 -i pipe:0 -c copy out.mp4 (or transcode).
Reuse muxer patterns from voice (muxer.ts) + session recording.
Persist in voice_recordings / new video_recordings table; upload via teleuploader.

Phase C — frontend playback + session grouping (follow-up)

Backend oRPC list video files; FE video player, group by call session like audio.

Verification

Phase A: join voice, have a member screen-share/camera, confirm .h264 file grows with NAL frames + keyframes; journal shows "video burst" logs.
Run vitest unit: RTP header strip + FU-A reassembly gives correct bytes.

Open questions / risks

Discord codec for camera = H264(101/102); screenshare uses H264 (101/103?) and can also be VP8/VP9. Handle H264 first (depacketize proven), VP8/VP9 in Phase B via ffmpeg RTP input.
Encryption: DAVE (dave_protocol_version) adds a session layer; parsePacket already applies daveSession.decrypt for audio — we must call the SAME parsePacket path so DAVE/encryption is handled identically.
ssrc↔user mapping during a broadcast: videoSSRC is in ssrcMap after the voice state; may need the STREAM_CREATE network events to key reliably.

3.6 KiB Raw Blame History