feat(gateway): record others' video (camera/screen share) — Phase A capture
@discordjs/voice only decrypts/forwards AUDIO (opus) — its onUdpMessage drops every non-opus RTP packet (dist/index.mjs:2068 `!== RTP_OPUS_PAYLOAD_TYPE`, 120). Video RTP (H264 camera + screen share, plus VP8/VP9/AV1) arrives on the same UDP socket but was silently discarded. New videoReceiver.ts wraps receiver.onUdpMessage (like screenShareAudio.ts): - detects video payload types (96/98/101/102/106/116/126/127), - decrypts them with the connection secret key/encryptionMode via the SAME receiver.parsePacket path @discordjs/voice uses for audio (so DAVE + voice encryption are handled identically), - depacketizes H264 to AnnexB (single NAL, STAP-A, and FU-A fragmentation), waiting for a keyframe (SPS/PPS/IDR) before writing, - writes a raw .h264 elementary stream per user per burst under <RECORDINGS_DIR>/<uid>/video-<ssrc>-<ts>.h264. Attribution: a videoSSRC→user index is built from ssrcMap updates; a proximity fallback mirrors screenShareAudio's inferScreenShareOwner. Bot's own video is skipped. Unit tests: tests/videoReceiver.test.ts (AnnexB start code, keyframe gating, FU-A reassembly, orphan-fragment tolerance) — 5/5 green. Phase A only (capture raw h264). Phase B (ffmpeg decode+mux to MP4/WebM + persist) and Phase C (frontend playback) are follow-ups.
This commit is contained in:
asepharyana
asepharyana
@@ -0,0 +1,66 @@
|
||||
# Spec: Record video (kamera/screenshare) orang lain — WebRTC receive (Request 2)
|
||||
|
||||
Date: 2026-08-30. Status: IN PROGRESS (Phase A).
|
||||
|
||||
## Why this is hard (ground truth, verified from @discordjs/voice 0.19.2 source)
|
||||
`VoiceReceiver.onUdpMessage` (dist/index.mjs:2059) drops EVERY non-opus RTP
|
||||
packet at line 2068: `if ((msg[1] & 127) !== RTP_OPUS_PAYLOAD_TYPE) return;`.
|
||||
So video (kamera H264? actually Discord uses VP8/H264; screenshare combines with
|
||||
video SSRC) is decrypted-capable but never forwarded. `receiver.parsePacket`
|
||||
(2033) DOES decrypt any payload type generically (audio + video) using
|
||||
`connectionData.{encryptionMode, nonceBuffer, secretKey}` — the only audio gate
|
||||
is the opus check inside onUdpMessage.
|
||||
|
||||
=> FIX: wrap `receiver.onUdpMessage` (like screenShareAudio.ts already does for
|
||||
screen-share AUDIO SSRCs): for packets whose payload type is a VIDEO type
|
||||
(payload 96 VP8, 101/102 H264, 106/116/126/127 AV1, VP9 98...), call
|
||||
`receiver.parsePacket(...)` myself to decrypt, then depacketize + write frames.
|
||||
Delegate opus (120) to the original handler. Delegate audio to original.
|
||||
|
||||
## Audio already works (screenShareAudio.ts). We add VIDEO.
|
||||
|
||||
## Science-of-the-changes below.
|
||||
|
||||
## Phase A — capture + decrypt + depacketize to AnnexB h264 (THIS change)
|
||||
Files (new): `src/modules/voice-recording/videoReceiver.ts`
|
||||
- Hook into `recorder.startRecording` alongside `hookScreenShareAudio`.
|
||||
- Wrap `receiver.onUdpMessage`:
|
||||
- read ssrc = msg.readUInt32BE(8); userData = receiver.ssrcMap.get(ssrc)
|
||||
- if payload type is video AND we have a "watching" subscription for that user
|
||||
(videoSSRC present), decrypt via receiver.parsePacket(...), then:
|
||||
- H264 (101/102 + payload 120 not): strip RTP header, reassemble FU-A
|
||||
fragments into AnnexB NALs (start-code prefixed), buffer until we have
|
||||
a full access unit (keyframe SPS/PPS/IDR or slices), append to a per-
|
||||
user-per-burst `.h264` file.
|
||||
- else delegate to original onUdpMessage.
|
||||
- Watch `receiver.ssrcMap` "create"/"update" for `videoSSRC !== undefined` →
|
||||
signal a video burst started for that user (like screenShareAudio does).
|
||||
- Per-user video files written to `config.RECORDINGS_DIR/<uid>/video-<ts>.h264`.
|
||||
- Guard: skip bot's own video (client.user.id).
|
||||
|
||||
Dependencies: NO new npm deps for Phase A (only crypto already in
|
||||
@discordjs/voice via parsePacket + Buffer). ffmpeg-headless (already in Nix
|
||||
buildInputs) used in Phase B for decode+mux.
|
||||
|
||||
## Phase B — decode + mux to playable MP4/WebM (follow-up)
|
||||
- Pipe AnnexB h264 → `ffmpeg -f h264 -i pipe:0 -c copy out.mp4` (or transcode).
|
||||
- Reuse muxer patterns from voice (muxer.ts) + session recording.
|
||||
- Persist in voice_recordings / new video_recordings table; upload via teleuploader.
|
||||
|
||||
## Phase C — frontend playback + session grouping (follow-up)
|
||||
- Backend oRPC list video files; FE video player, group by call session like audio.
|
||||
|
||||
## Verification
|
||||
- Phase A: join voice, have a member screen-share/camera, confirm `.h264` file
|
||||
grows with NAL frames + keyframes; journal shows "video burst" logs.
|
||||
- Run vitest unit: RTP header strip + FU-A reassembly gives correct bytes.
|
||||
|
||||
## Open questions / risks
|
||||
- Discord codec for camera = H264(101/102); screenshare uses H264 (101/103?)
|
||||
and can also be VP8/VP9. Handle H264 first (depacketize proven), VP8/VP9 in
|
||||
Phase B via ffmpeg RTP input.
|
||||
- Encryption: DAVE (dave_protocol_version) adds a session layer; parsePacket
|
||||
already applies daveSession.decrypt for audio — we must call the SAME
|
||||
parsePacket path so DAVE/encryption is handled identically.
|
||||
- ssrc↔user mapping during a broadcast: videoSSRC is in ssrcMap after the
|
||||
voice state; may need the STREAM_CREATE network events to key reliably.
|
||||
@@ -19,6 +19,7 @@ import {
|
||||
} from "./recorder/sessionRecording.js";
|
||||
import { createSpeakingHandler } from "./recorder/speakingHandler.js";
|
||||
import { hookScreenShareAudio } from "./screenShareAudio.js";
|
||||
import { hookVideoReceiver } from "./videoReceiver.js";
|
||||
|
||||
const logger = createChildLogger("recorder");
|
||||
|
||||
@@ -192,6 +193,16 @@ export async function startRecording(
|
||||
// to discover and register those SSRCs.
|
||||
hookScreenShareAudio(receiver, speakingHandler);
|
||||
|
||||
// ── Video capture (camera + screen share) ───────────────────────────
|
||||
// @discordjs/voice only decrypts/forwards audio (opus). Video RTP (H264,
|
||||
// VP8/VP9/AV1) arrives on the same UDP socket but is dropped by the opus
|
||||
// gate. hookVideoReceiver wraps onUdpMessage, decrypts video payload types
|
||||
// with the connection secret key (reusing receiver.parsePacket), then
|
||||
// depacketizes H264 to AnnexB and writes a raw .h264 stream per user.
|
||||
// Order matters: video hook must be installed AFTER the screen-share audio
|
||||
// hook so non-video packets delegate down the existing wrapper chain.
|
||||
hookVideoReceiver(receiver, client, recordingsDir);
|
||||
|
||||
// Handle unexpected disconnection
|
||||
connection.on(VoiceConnectionStatus.Disconnected, async () => {
|
||||
if (config.VERBOSE) {
|
||||
|
||||
@@ -0,0 +1,369 @@
|
||||
import { createWriteStream, mkdirSync, type WriteStream } from "node:fs";
|
||||
import path from "node:path";
|
||||
import type { VoiceReceiver } from "@discordjs/voice";
|
||||
import type { Client } from "discord.js-selfbot-v13";
|
||||
import { createChildLogger } from "@/shared/logger/index";
|
||||
|
||||
const logger = createChildLogger("video-receiver");
|
||||
|
||||
// ─── Discord RTP video payload types (real Discord clients) ───────────────
|
||||
// H264 is what Discord uses for the user's CAMERA; screen-share (GoLive) also
|
||||
// uses H264. VP8/VP9/AV1 appear for some share types / forced codecs.
|
||||
export const VIDEO_PAYLOAD_TYPES = new Set<number>([
|
||||
96, 98, 101, 102, 106, 116, 126, 127,
|
||||
]);
|
||||
|
||||
// H264 NAL unit types (single byte, after the 1-byte NAL header)
|
||||
const NAL_SPS = 7;
|
||||
const NAL_PPS = 8;
|
||||
const NAL_IDR = 5;
|
||||
|
||||
// RTP H264 packet types (byte 0 of the payload)
|
||||
const _H264_SINGLE_NAL = 1; // 0-23 → single NAL unit, type = nal_unit_type
|
||||
const H264_STAP_A = 24;
|
||||
const H264_FU_A = 28;
|
||||
|
||||
/**
|
||||
* Pure RTP-H264 depacketizer → AnnexB (prefix NALs with 00 00 00 01 start codes).
|
||||
*
|
||||
* Input: RTP payload bytes (after the 12-byte RTP header, after decryption).
|
||||
* Output: a list of AnnexB NAL buffers (each with start code) ready to append
|
||||
* to a `.h264` elementary stream file.
|
||||
*
|
||||
* Reassembles FU-A fragmented NALs and STAP-A aggregated NALs into standalone
|
||||
* NALs with start codes. Fragments that don't form a complete NAL are buffered.
|
||||
*/
|
||||
export class H264Depacketizer {
|
||||
/** True after we've seen an SPS/PPS/IDR so we only start writing on a keyframe. */
|
||||
private sawConfig = false;
|
||||
/** Buffer for an in-progress FU-A NAL. */
|
||||
private fuBuffer: number[] = [];
|
||||
|
||||
/** Handle one RTP payload; returns AnnexB NAL buffers to write (may be []). */
|
||||
push(payload: Buffer): Buffer[] {
|
||||
const out: Buffer[] = [];
|
||||
if (payload.length === 0) return out;
|
||||
|
||||
const packetType = payload[0] & 0x1f;
|
||||
|
||||
if (packetType >= 1 && packetType <= 23) {
|
||||
// Single NAL unit.
|
||||
const nalType = payload[0] & 0x1f;
|
||||
this.maybeEnableConfig(nalType);
|
||||
const annexB = this.toAnnexB(payload);
|
||||
if (this.sawConfig && annexB) out.push(annexB);
|
||||
return out;
|
||||
}
|
||||
|
||||
if (packetType === H264_FU_A) {
|
||||
// Fragmentation unit: payload[0] = FU indicator (F/NRI/type=28),
|
||||
// payload[1] = FU header (S/E/R/type).
|
||||
const fuHeader = payload[1];
|
||||
const start = (fuHeader & 0x80) !== 0;
|
||||
const end = (fuHeader & 0x40) !== 0;
|
||||
const nalType = fuHeader & 0x1f;
|
||||
|
||||
if (start) {
|
||||
// First fragment: reconstruct the full NAL header by preserving the
|
||||
// F and NRI bits from the FU indicator (payload[0]) OR'ed with the
|
||||
// NAL unit type from the FU header (payload[1] & 0x1F).
|
||||
const nalHeader = (payload[0] & 0xe0) | nalType;
|
||||
this.fuBuffer = [nalHeader, ...payload.subarray(2)];
|
||||
} else {
|
||||
// Continuation. (Ignore if no buffer — a lost first fragment.)
|
||||
if (this.fuBuffer.length === 0) {
|
||||
// We missed the start; the NAL type unknown, skip this fragment.
|
||||
return out;
|
||||
}
|
||||
if (payload.length > 2) {
|
||||
this.fuBuffer.push(...payload.subarray(2));
|
||||
}
|
||||
}
|
||||
|
||||
if (end) {
|
||||
const nal = Buffer.from(this.fuBuffer);
|
||||
this.fuBuffer = [];
|
||||
this.maybeEnableConfig(nalType);
|
||||
if (this.sawConfig && nal.length > 0) {
|
||||
out.push(this.toAnnexB(nal));
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
if (packetType === H264_STAP_A) {
|
||||
// Aggregation packet: list of (2-byte NAL size, NAL data).
|
||||
let offset = 1;
|
||||
while (offset < payload.length) {
|
||||
if (offset + 2 > payload.length) break;
|
||||
const naluLength = payload.readUInt16BE(offset);
|
||||
offset += 2;
|
||||
if (offset + naluLength > payload.length) break;
|
||||
const nalu = payload.subarray(offset, offset + naluLength);
|
||||
offset += naluLength;
|
||||
const nalType = nalu[0] & 0x1f;
|
||||
this.maybeEnableConfig(nalType);
|
||||
const annexB = this.toAnnexB(nalu);
|
||||
if (this.sawConfig && annexB) out.push(annexB);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private maybeEnableConfig(nalType: number): void {
|
||||
if (nalType === NAL_SPS || nalType === NAL_PPS || nalType === NAL_IDR) {
|
||||
this.sawConfig = true;
|
||||
}
|
||||
}
|
||||
|
||||
private toAnnexB(nal: Buffer): Buffer {
|
||||
const startCode = Buffer.from([0x00, 0x00, 0x00, 0x01]);
|
||||
return Buffer.concat([startCode, nal]);
|
||||
}
|
||||
|
||||
reset(): void {
|
||||
this.sawConfig = false;
|
||||
this.fuBuffer = [];
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Per-user video burst state ────────────────────────────────────────────
|
||||
|
||||
interface VideoBurst {
|
||||
userId: string;
|
||||
depacketizer: H264Depacketizer;
|
||||
/** video SSRC this burst is sourced from (may be camera or screenshare). */
|
||||
ssrc: number;
|
||||
createdAt: number;
|
||||
filePath: string;
|
||||
out: WriteStream;
|
||||
/** count of AnnexB NAL bytes written (for logging/health). */
|
||||
bytesWritten: number;
|
||||
lastPacketAt: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Listens for other users' VIDEO RTP (camera + screen share) on the same UDP
|
||||
* socket @discordjs/voice receives audio from, decrypts each video packet with
|
||||
* the connection's secret key (via receiver.parsePacket), depacketizes H264 to
|
||||
* AnnexB, and writes a raw `.h264` elementary stream per user-per-burst.
|
||||
*
|
||||
* Wraps `receiver.onUdpMessage` like screenShareAudio.ts. Packets with a video
|
||||
* payload type are consumed here; ALL other packets (opus mic + screen-share
|
||||
* audio) are delegated to the current handler (the existing wrapper chain).
|
||||
*/
|
||||
export function hookVideoReceiver(
|
||||
receiver: VoiceReceiver,
|
||||
client: Client,
|
||||
recordingsDir: string,
|
||||
): void {
|
||||
const original = receiver.onUdpMessage;
|
||||
const bursts = new Map<string, VideoBurst>();
|
||||
|
||||
// The UDP socket delivers video on SSRCs that are NOT the audioSSRC key of
|
||||
// ssrcMap. Build a videoSSRC→userId index from VoiceUserData as it updates so
|
||||
// we can attribute packets without SSRC-proximity guessing.
|
||||
const videoSsrcToUser = new Map<number, string>();
|
||||
|
||||
receiver.ssrcMap.on("create", (data) => {
|
||||
if (data.videoSSRC !== undefined) {
|
||||
videoSsrcToUser.set(data.videoSSRC, data.userId);
|
||||
logger.info(
|
||||
{ userId: data.userId, videoSSRC: data.videoSSRC },
|
||||
"Video SSRC appeared (camera/screenshare start)",
|
||||
);
|
||||
}
|
||||
});
|
||||
receiver.ssrcMap.on("update", (_old, neu) => {
|
||||
if (neu.videoSSRC !== undefined) {
|
||||
videoSsrcToUser.set(neu.videoSSRC, neu.userId);
|
||||
}
|
||||
});
|
||||
|
||||
/** Close a burst's file (flush + end). */
|
||||
const closeBurst = (userId: string): void => {
|
||||
const burst = bursts.get(userId);
|
||||
if (!burst) return;
|
||||
bursts.delete(userId);
|
||||
try {
|
||||
burst.out.end();
|
||||
} catch {
|
||||
// ignore write errors on close
|
||||
}
|
||||
logger.info(
|
||||
{
|
||||
userId,
|
||||
ssrc: burst.ssrc,
|
||||
bytes: burst.bytesWritten,
|
||||
durationMs: Date.now() - burst.createdAt,
|
||||
},
|
||||
"Video burst closed",
|
||||
);
|
||||
};
|
||||
|
||||
const openBurst = (userId: string, ssrc: number): VideoBurst | null => {
|
||||
try {
|
||||
const userDir = path.join(recordingsDir, userId);
|
||||
mkdirSync(userDir, { recursive: true });
|
||||
const filePath = path.join(userDir, `video-${ssrc}-${Date.now()}.h264`);
|
||||
const out = createWriteStream(filePath);
|
||||
const burst: VideoBurst = {
|
||||
userId,
|
||||
ssrc,
|
||||
depacketizer: new H264Depacketizer(),
|
||||
createdAt: Date.now(),
|
||||
filePath,
|
||||
out,
|
||||
bytesWritten: 0,
|
||||
lastPacketAt: Date.now(),
|
||||
};
|
||||
bursts.set(userId, burst);
|
||||
logger.info({ userId, ssrc, filePath }, "Video burst opened");
|
||||
return burst;
|
||||
} catch (err) {
|
||||
logger.warn(
|
||||
{ userId, ssrc, err: err instanceof Error ? err.message : String(err) },
|
||||
"Failed to open video burst file",
|
||||
);
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
receiver.onUdpMessage = (msg: Buffer) => {
|
||||
if (msg.length <= 8) {
|
||||
original.call(receiver, msg);
|
||||
return;
|
||||
}
|
||||
|
||||
const payloadType = msg[1] & 127;
|
||||
if (!VIDEO_PAYLOAD_TYPES.has(payloadType)) {
|
||||
// Not video — let the existing handler (audio + screen-share audio) take it.
|
||||
original.call(receiver, msg);
|
||||
return;
|
||||
}
|
||||
|
||||
// ── It's a video RTP packet. Decrypt + depacketize + write. ──
|
||||
const ssrc = msg.readUInt32BE(8);
|
||||
|
||||
// Attribute to a user.
|
||||
let userId = videoSsrcToUser.get(ssrc);
|
||||
if (!userId) {
|
||||
// Fallback: maybe we registered the SSRC under an audio entry via the
|
||||
// screen-share-audio hook, or proximity. Skip bot's own stream.
|
||||
const candidate = inferVideoOwner(msg, receiver);
|
||||
if (candidate && candidate.userId !== client.user?.id) {
|
||||
userId = candidate.userId;
|
||||
videoSsrcToUser.set(ssrc, candidate.userId);
|
||||
}
|
||||
}
|
||||
if (!userId || userId === client.user?.id) {
|
||||
return; // unknown or self — ignore
|
||||
}
|
||||
|
||||
// Decrypt the payload using @discordjs/voice's parser (handles DAVE +
|
||||
// encryption the same way it does for audio).
|
||||
let decrypted: Buffer;
|
||||
try {
|
||||
const receiverAny = receiver as unknown as {
|
||||
parsePacket: (
|
||||
buffer: Buffer,
|
||||
mode: string,
|
||||
nonce: Buffer,
|
||||
secretKey: Uint8Array,
|
||||
userId: string,
|
||||
) => Buffer;
|
||||
connectionData: {
|
||||
encryptionMode?: string;
|
||||
nonceBuffer?: Buffer;
|
||||
secretKey?: Uint8Array;
|
||||
};
|
||||
};
|
||||
const { encryptionMode, nonceBuffer, secretKey } =
|
||||
receiverAny.connectionData;
|
||||
if (!encryptionMode || !nonceBuffer || !secretKey) return;
|
||||
// decrypted payload = RTP header already stripped by parsePacket
|
||||
decrypted = receiverAny.parsePacket(
|
||||
msg,
|
||||
encryptionMode,
|
||||
nonceBuffer,
|
||||
secretKey,
|
||||
userId,
|
||||
);
|
||||
} catch {
|
||||
// Decryption failed — transient; skip this packet.
|
||||
return;
|
||||
}
|
||||
|
||||
// Depacketize the (video) RTP payload → AnnexB.
|
||||
let burst = bursts.get(userId);
|
||||
if (!burst) {
|
||||
const opened = openBurst(userId, ssrc);
|
||||
if (!opened) return;
|
||||
burst = opened;
|
||||
}
|
||||
burst.lastPacketAt = Date.now();
|
||||
|
||||
let nals: Buffer[];
|
||||
try {
|
||||
nals = burst.depacketizer.push(decrypted);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (nals.length === 0) return;
|
||||
|
||||
for (const nal of nals) {
|
||||
burst.bytesWritten += nal.length;
|
||||
if (!burst.out.write(nal)) {
|
||||
burst.out.once("drain", () => {});
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// If a user's video SSRC disappears, close their burst after a short grace so
|
||||
// the tail flushes.
|
||||
setInterval(() => {
|
||||
const now = Date.now();
|
||||
for (const [userId, burst] of bursts) {
|
||||
if (now - burst.lastPacketAt > 5000) {
|
||||
closeBurst(userId);
|
||||
}
|
||||
}
|
||||
}, 2000);
|
||||
// Prevent the interval from keeping the process alive.
|
||||
// (The gateway owns the process; a single 2s timer is negligible.)
|
||||
}
|
||||
|
||||
/**
|
||||
* Infer which user owns a video SSRC by proximity to their known audioSSRC
|
||||
* (Discord allocates a user's video SSRC near their audio SSRC).
|
||||
*/
|
||||
function inferVideoOwner(
|
||||
msg: Buffer,
|
||||
receiver: VoiceReceiver,
|
||||
): { userId: string } | null {
|
||||
const ssrc = msg.readUInt32BE(8);
|
||||
try {
|
||||
const map = getSsrcInternalMap(receiver.ssrcMap);
|
||||
if (!map) return null;
|
||||
for (const [, data] of map.entries()) {
|
||||
if (data.audioSSRC && Math.abs(data.audioSSRC - ssrc) < 400_000) {
|
||||
return { userId: data.userId };
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Access the private `_map` inside SSRCMap (mirrors screenShareAudio.ts). */
|
||||
function getSsrcInternalMap(
|
||||
ssrcMap: VoiceReceiver["ssrcMap"],
|
||||
):
|
||||
| Map<number, { userId: string; audioSSRC?: number; videoSSRC?: number }>
|
||||
| undefined {
|
||||
const asAny = ssrcMap as unknown as Record<string, unknown>;
|
||||
const m1 = asAny._map;
|
||||
if (m1 instanceof Map) return m1 as Map<number, never>;
|
||||
return undefined;
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { H264Depacketizer } from "../src/modules/voice-recording/videoReceiver.js";
|
||||
|
||||
// Build a single-NAL RTP payload: [NAL header, ...data]
|
||||
function singleNal(nalHeader: number, data: number[] = []): Buffer {
|
||||
return Buffer.from([nalHeader, ...data]);
|
||||
}
|
||||
|
||||
describe("H264Depacketizer", () => {
|
||||
it("prefixes a single NAL with an AnnexB start code once config seen", () => {
|
||||
const d = new H264Depacketizer();
|
||||
// SPS first (0x67 = nal ref idc 3, type 7)
|
||||
const sps = d.push(singleNal(0x67, [1, 2, 3]));
|
||||
expect(sps.length).toBe(1);
|
||||
expect(sps[0].subarray(0, 4)).toEqual(Buffer.from([0, 0, 0, 1]));
|
||||
expect(sps[0][4]).toBe(0x67); // start code + original NAL
|
||||
});
|
||||
|
||||
it("drops NALs before seeing SPS/PPS/IDR (waits for a keyframe)", () => {
|
||||
const d = new H264Depacketizer();
|
||||
// A non-IDR slice (type 1) before any config → dropped
|
||||
const res = d.push(singleNal(0x21, [9, 9, 9]));
|
||||
expect(res.length).toBe(0);
|
||||
});
|
||||
|
||||
it("emits frames after config", () => {
|
||||
const d = new H264Depacketizer();
|
||||
d.push(singleNal(0x67, [])); // SPS
|
||||
const slice = d.push(singleNal(0x65, [1, 2, 3, 4])); // IDR (type 5)
|
||||
expect(slice.length).toBe(1);
|
||||
expect(slice[0].equals(Buffer.from([0, 0, 0, 1, 0x65, 1, 2, 3, 4]))).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it("reassembles FU-A fragments into one NAL", () => {
|
||||
const d = new H264Depacketizer();
|
||||
d.push(singleNal(0x67, [])); // SPS config
|
||||
// FU-A: payload[0]=FU indicator (0x7C=type28), payload[1]=FU header
|
||||
// start fragment: S=1 E=0, NAL type 5 => 0x85 ; data [10, 11]
|
||||
const start = Buffer.from([0x7c, 0x85, 10, 11]);
|
||||
const middle = Buffer.from([0x7c, 0x05, 12, 13]);
|
||||
const end = Buffer.from([0x7c, 0x45, 14, 15]); // E=1
|
||||
|
||||
expect(d.push(start).length).toBe(0); // not complete yet
|
||||
expect(d.push(middle).length).toBe(0);
|
||||
const final = d.push(end);
|
||||
expect(final.length).toBe(1);
|
||||
// NAL header reconstructed as (0x7C & 0xE0) | 5 = 0x65, then data 10..15
|
||||
expect(
|
||||
final[0].equals(Buffer.from([0, 0, 0, 1, 0x65, 10, 11, 12, 13, 14, 15])),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("first fragment of a burst with unknown start is dropped gracefully", () => {
|
||||
const d = new H264Depacketizer();
|
||||
d.push(singleNal(0x67, [])); // SPS
|
||||
// A continuation fragment with no buffer → ignored, no crash
|
||||
const orphan = Buffer.from([0x7c, 0x05, 99]); // S=0
|
||||
expect(d.push(orphan).length).toBe(0);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user