Merge branch 'worktree-library-modernization'
This commit is contained in:
183
docs/superpowers/plans/2026-05-14-library-modernization.md
Normal file
183
docs/superpowers/plans/2026-05-14-library-modernization.md
Normal file
@@ -0,0 +1,183 @@
|
|||||||
|
# Library Modernization Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Modernize runtime and development dependencies while preserving Discord monitoring, recording, database migration, dashboard, and test behavior.
|
||||||
|
|
||||||
|
**Architecture:** Treat modernization as dependency classification plus small source refactors. Remove redundant validation libraries by moving `src/validation.ts` to Zod, replace `fluent-ffmpeg` with a tiny direct `ffmpeg` process wrapper for the muxer scripts, and convert database migration code to ESM-safe imports. Keep high-risk Discord/audio/native packages unless audit proves a safe replacement exists.
|
||||||
|
|
||||||
|
**Tech Stack:** Node.js, pnpm, TypeScript, Zod, Drizzle ORM, better-sqlite3, pg, Express, ws, Vite, React, Vitest, Biome, Discord voice/audio packages.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dependency Audit Baseline
|
||||||
|
|
||||||
|
- Usage audit confirms `class-transformer`/`class-validator` are only used by `src/validation.ts`.
|
||||||
|
- Usage audit confirms `fluent-ffmpeg` is only used by `src/muxer.ts` and `src/muxer-aup3.ts`.
|
||||||
|
- `pnpm outdated --format table` reports `discord.js-selfbot-v13` and `fluent-ffmpeg` as deprecated.
|
||||||
|
- Outdated packages reported: `tsx`, `@types/node`, `p-retry`, `pino`, `pino-pretty`, `sodium-native`.
|
||||||
|
- Direct dependency classification: remove `class-transformer`, `class-validator`, `fluent-ffmpeg`, `@types/fluent-ffmpeg`; replace validation with Zod and ffmpeg wrapper with `node:child_process`; upgrade outdated packages; keep high-risk voice/audio packages unless a compatible replacement is proven.
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
|
||||||
|
- Modify `package.json`: dependency upgrades, removals, and script additions if needed.
|
||||||
|
- Modify `pnpm-lock.yaml`: regenerated by `pnpm install`.
|
||||||
|
- Modify `src/validation.ts`: replace `class-transformer` and `class-validator` with Zod.
|
||||||
|
- Modify `src/database/migrate.ts`: remove dynamic CommonJS `require` and `any` cast.
|
||||||
|
- Create `src/audio/ffmpegProcess.ts`: small wrapper around `node:child_process` for direct ffmpeg execution.
|
||||||
|
- Modify `src/muxer.ts`: use `runFfmpeg()` instead of `fluent-ffmpeg`.
|
||||||
|
- Modify `src/muxer-aup3.ts`: use `runFfmpeg()` instead of `fluent-ffmpeg`.
|
||||||
|
- Modify `src/recorder/decoder.ts`: keep `createRequire()` for optional native probing unless a better ESM-safe probe is identified during implementation.
|
||||||
|
- Add or modify tests under `tests/`: validation, migration helper behavior, and ffmpeg argument construction.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Capture Dependency Audit Baseline
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `docs/superpowers/plans/2026-05-14-library-modernization.md`
|
||||||
|
- Inspect: `package.json`
|
||||||
|
- Inspect: `pnpm-lock.yaml`
|
||||||
|
- Inspect: `src/**/*.ts`
|
||||||
|
- Inspect: `tests/**/*.ts`
|
||||||
|
|
||||||
|
- [x] **Step 1: List direct dependency usage**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
grep -R "class-transformer\|class-validator\|fluent-ffmpeg\|@discordjs/opus\|@discordjs/voice\|@snazzah/davey\|discord.js-selfbot-v13\|libsodium-wrappers\|sodium-native\|prism-media\|drizzle-orm\|better-sqlite3\|pg\|express\|helmet\|p-retry\|pino\|pino-http\|prom-client\|react\|react-dom\|vite\|ws\|zod" -n src tests frontend package.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: output lists every direct package usage. Record the summary in the implementation notes during execution.
|
||||||
|
|
||||||
|
- [x] **Step 2: Check outdated dependencies**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm outdated --format table
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: command exits non-zero if packages are outdated; use the table as audit input, not as failure.
|
||||||
|
|
||||||
|
- [x] **Step 3: Classify direct dependencies**
|
||||||
|
|
||||||
|
Use this classification as the starting point, adjusting only if Step 1 proves a package is unused or irreplaceable:
|
||||||
|
|
||||||
|
```text
|
||||||
|
remove: class-transformer, class-validator, fluent-ffmpeg, @types/fluent-ffmpeg
|
||||||
|
replace: class-transformer/class-validator -> zod; fluent-ffmpeg -> node:child_process ffmpeg wrapper
|
||||||
|
upgrade: @vitejs/plugin-react, better-sqlite3, discord.js-selfbot-v13, dotenv, drizzle-orm, express, helmet, libsodium-wrappers, p-retry, pg, pino, pino-http, prom-client, react, react-dom, sodium-native, vite, ws, zod, @biomejs/biome, @types/*, drizzle-kit, pino-pretty, tsx, vitest
|
||||||
|
keep unless compatible alternative is proven: @discordjs/opus, @discordjs/voice, @snazzah/davey, prism-media
|
||||||
|
```
|
||||||
|
|
||||||
|
- [x] **Step 4: Commit audit note if this task changes files**
|
||||||
|
|
||||||
|
If only commands were run, do not commit. If the plan is updated with audit notes, run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add docs/superpowers/plans/2026-05-14-library-modernization.md
|
||||||
|
git commit -m "docs: record dependency modernization audit"
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: commit succeeds only if a file changed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: Replace Class Validator Stack With Zod
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/validation.ts`
|
||||||
|
- Test: `tests/validation.test.ts`
|
||||||
|
- Modify later: `package.json`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write failing validation tests**
|
||||||
|
- [ ] **Step 2: Run validation tests to establish baseline**
|
||||||
|
- [ ] **Step 3: Replace implementation with Zod**
|
||||||
|
- [ ] **Step 4: Run validation tests**
|
||||||
|
- [ ] **Step 5: Commit validation refactor**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: Convert Migration Code to ESM-Safe Drizzle Imports
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `src/database/migrate.ts`
|
||||||
|
- Test: `tests/database/migrate.test.ts`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Extract SQLite database creation for testing**
|
||||||
|
- [ ] **Step 2: Add migration helper test**
|
||||||
|
- [ ] **Step 3: Run migration test**
|
||||||
|
- [ ] **Step 4: Run typecheck for migration typing**
|
||||||
|
- [ ] **Step 5: Commit migration refactor**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 4: Replace Fluent FFmpeg With Direct Process Wrapper
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `src/audio/ffmpegProcess.ts`
|
||||||
|
- Modify: `src/muxer.ts`
|
||||||
|
- Modify: `src/muxer-aup3.ts`
|
||||||
|
- Test: `tests/audio/ffmpegProcess.test.ts`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add ffmpeg wrapper tests**
|
||||||
|
- [ ] **Step 2: Run ffmpeg wrapper test to verify it fails**
|
||||||
|
- [ ] **Step 3: Implement ffmpeg process wrapper**
|
||||||
|
- [ ] **Step 4: Refactor `src/muxer.ts`**
|
||||||
|
- [ ] **Step 5: Refactor `src/muxer-aup3.ts`**
|
||||||
|
- [ ] **Step 6: Run ffmpeg wrapper tests**
|
||||||
|
- [ ] **Step 7: Run typecheck**
|
||||||
|
- [ ] **Step 8: Commit ffmpeg refactor**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 5: Update Package Manifest and Lockfile
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `package.json`
|
||||||
|
- Modify: `pnpm-lock.yaml`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Remove replaced packages**
|
||||||
|
- [ ] **Step 2: Upgrade dependencies interactively-free**
|
||||||
|
- [ ] **Step 3: Ensure package manager remains pnpm 10**
|
||||||
|
- [ ] **Step 4: Run install to verify lockfile**
|
||||||
|
- [ ] **Step 5: Commit dependency manifest changes**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 6: Fix Upgrade Breakages
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify as needed: `src/**/*.ts`
|
||||||
|
- Modify as needed: `frontend/**/*.ts`
|
||||||
|
- Modify as needed: `frontend/**/*.tsx`
|
||||||
|
- Modify as needed: `tests/**/*.ts`
|
||||||
|
- Modify as needed: config files touched by upgraded tools
|
||||||
|
|
||||||
|
- [ ] **Step 1: Run typecheck**
|
||||||
|
- [ ] **Step 2: Run lint**
|
||||||
|
- [ ] **Step 3: Run tests**
|
||||||
|
- [ ] **Step 4: Run build**
|
||||||
|
- [ ] **Step 5: Commit breakage fixes**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 7: Final Verification and Manual Dashboard Check
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- No planned source changes
|
||||||
|
|
||||||
|
- [ ] **Step 1: Run full verification**
|
||||||
|
- [ ] **Step 2: Start dev server for dashboard check**
|
||||||
|
- [ ] **Step 3: Manually verify frontend build path if browser access is available**
|
||||||
|
- [ ] **Step 4: Check git status**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review
|
||||||
|
|
||||||
|
- Spec coverage: audit, dependency classification, replacement/removal, ESM migration, lockfile regeneration, verification, and dashboard manual check are covered.
|
||||||
|
- Placeholder scan: no `TBD`, `TODO`, or unspecified implementation steps remain.
|
||||||
|
- Type consistency: helper names are consistent across tasks: `validateUserStateUpdate`, `initializeMigrationSqliteDatabase`, `buildMuxFfmpegArgs`, and `runFfmpeg`.
|
||||||
18
package.json
18
package.json
@@ -25,26 +25,23 @@
|
|||||||
"@discordjs/voice": "^0.19.1",
|
"@discordjs/voice": "^0.19.1",
|
||||||
"@snazzah/davey": "^0.1.10",
|
"@snazzah/davey": "^0.1.10",
|
||||||
"@types/pg": "^8.20.0",
|
"@types/pg": "^8.20.0",
|
||||||
"@vitejs/plugin-react": "^6.0.1",
|
"@vitejs/plugin-react": "^6.0.2",
|
||||||
"better-sqlite3": "^12.10.0",
|
"better-sqlite3": "^12.10.0",
|
||||||
"class-transformer": "^0.5.1",
|
|
||||||
"class-validator": "^0.15.1",
|
|
||||||
"discord.js-selfbot-v13": "^3.7.1",
|
"discord.js-selfbot-v13": "^3.7.1",
|
||||||
"dotenv": "^17.4.2",
|
"dotenv": "^17.4.2",
|
||||||
"drizzle-orm": "^0.45.2",
|
"drizzle-orm": "^0.45.2",
|
||||||
"express": "^5.2.1",
|
"express": "^5.2.1",
|
||||||
"fluent-ffmpeg": "^2.1.3",
|
|
||||||
"helmet": "^8.1.0",
|
"helmet": "^8.1.0",
|
||||||
"libsodium-wrappers": "^0.8.2",
|
"libsodium-wrappers": "^0.8.2",
|
||||||
"p-retry": "^6.2.0",
|
"p-retry": "^8.0.0",
|
||||||
"pg": "^8.20.0",
|
"pg": "^8.20.0",
|
||||||
"pino": "^9.4.0",
|
"pino": "^10.3.1",
|
||||||
"pino-http": "^11.0.0",
|
"pino-http": "^11.0.0",
|
||||||
"prism-media": "2.0.0-alpha.0",
|
"prism-media": "2.0.0-alpha.0",
|
||||||
"prom-client": "^15.1.3",
|
"prom-client": "^15.1.3",
|
||||||
"react": "^19.2.6",
|
"react": "^19.2.6",
|
||||||
"react-dom": "^19.2.6",
|
"react-dom": "^19.2.6",
|
||||||
"sodium-native": "^4.3.2",
|
"sodium-native": "^5.1.0",
|
||||||
"vite": "^8.0.13",
|
"vite": "^8.0.13",
|
||||||
"ws": "^8.20.1",
|
"ws": "^8.20.1",
|
||||||
"zod": "^4.4.3"
|
"zod": "^4.4.3"
|
||||||
@@ -53,14 +50,13 @@
|
|||||||
"@biomejs/biome": "latest",
|
"@biomejs/biome": "latest",
|
||||||
"@types/better-sqlite3": "^7.6.13",
|
"@types/better-sqlite3": "^7.6.13",
|
||||||
"@types/express": "^5.0.6",
|
"@types/express": "^5.0.6",
|
||||||
"@types/fluent-ffmpeg": "^2.1.28",
|
"@types/node": "^25.8.0",
|
||||||
"@types/node": "^24.10.1",
|
|
||||||
"@types/react": "^19.2.14",
|
"@types/react": "^19.2.14",
|
||||||
"@types/react-dom": "^19.2.3",
|
"@types/react-dom": "^19.2.3",
|
||||||
"@types/ws": "^8.18.1",
|
"@types/ws": "^8.18.1",
|
||||||
"drizzle-kit": "^0.31.10",
|
"drizzle-kit": "^0.31.10",
|
||||||
"pino-pretty": "^10.3.1",
|
"pino-pretty": "^13.1.3",
|
||||||
"tsx": "^4.20.6",
|
"tsx": "^4.22.0",
|
||||||
"vitest": "latest"
|
"vitest": "latest"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
670
pnpm-lock.yaml
generated
670
pnpm-lock.yaml
generated
File diff suppressed because it is too large
Load Diff
61
src/audio/ffmpegProcess.ts
Normal file
61
src/audio/ffmpegProcess.ts
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
import { spawn } from "child_process";
|
||||||
|
|
||||||
|
export interface MuxFfmpegArgsOptions {
|
||||||
|
inputs: string[];
|
||||||
|
filter: string;
|
||||||
|
output: string;
|
||||||
|
codec: string;
|
||||||
|
audioFrequency?: number;
|
||||||
|
audioChannels?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Builds ffmpeg argument array for muxing audio clips.
|
||||||
|
*/
|
||||||
|
export function buildMuxFfmpegArgs(options: MuxFfmpegArgsOptions): string[] {
|
||||||
|
const args: string[] = ["-y"];
|
||||||
|
|
||||||
|
for (const input of options.inputs) {
|
||||||
|
args.push("-i", input);
|
||||||
|
}
|
||||||
|
|
||||||
|
args.push("-filter_complex", options.filter);
|
||||||
|
args.push("-map", "[out]");
|
||||||
|
args.push("-codec:a", options.codec);
|
||||||
|
|
||||||
|
if (options.audioFrequency !== undefined) {
|
||||||
|
args.push("-ar", String(options.audioFrequency));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (options.audioChannels !== undefined) {
|
||||||
|
args.push("-ac", String(options.audioChannels));
|
||||||
|
}
|
||||||
|
|
||||||
|
args.push(options.output);
|
||||||
|
|
||||||
|
return args;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Runs ffmpeg with the given arguments.
|
||||||
|
* Resolves on successful (code 0) exit, rejects on error or non-zero exit.
|
||||||
|
*/
|
||||||
|
export function runFfmpeg(args: string[]): Promise<void> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const proc = spawn("ffmpeg", args, {
|
||||||
|
stdio: ["ignore", "inherit", "inherit"],
|
||||||
|
});
|
||||||
|
|
||||||
|
proc.on("close", (code) => {
|
||||||
|
if (code === 0) {
|
||||||
|
resolve();
|
||||||
|
} else {
|
||||||
|
reject(new Error(`ffmpeg exited with code ${code}`));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
proc.on("error", (err) => {
|
||||||
|
reject(err);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -5,13 +5,11 @@ import { migrate as migrateSqlite } from "drizzle-orm/better-sqlite3/migrator";
|
|||||||
import { migrate as migratePostgres } from "drizzle-orm/node-postgres/migrator";
|
import { migrate as migratePostgres } from "drizzle-orm/node-postgres/migrator";
|
||||||
import { config } from "../config";
|
import { config } from "../config";
|
||||||
import { createChildLogger } from "../logger";
|
import { createChildLogger } from "../logger";
|
||||||
import { initializeDatabase } from "./drizzle";
|
import { closeDatabase, initializeDatabase } from "./drizzle";
|
||||||
|
|
||||||
const logger = createChildLogger("migrate");
|
const logger = createChildLogger("migrate");
|
||||||
|
|
||||||
export async function initializeMigrationSqliteDatabase(
|
export function initializeMigrationSqliteDatabase(path = ".muxer-queue.db") {
|
||||||
path = ".muxer-queue.db",
|
|
||||||
) {
|
|
||||||
const sqlite = new Database(path);
|
const sqlite = new Database(path);
|
||||||
sqlite.pragma("journal_mode = WAL");
|
sqlite.pragma("journal_mode = WAL");
|
||||||
return { sqlite, db: drizzleSqlite(sqlite) };
|
return { sqlite, db: drizzleSqlite(sqlite) };
|
||||||
@@ -23,17 +21,23 @@ export async function runMigrations(): Promise<void> {
|
|||||||
|
|
||||||
if (config.DATABASE_TYPE === "postgres") {
|
if (config.DATABASE_TYPE === "postgres") {
|
||||||
logger.info("Running PostgreSQL migrations");
|
logger.info("Running PostgreSQL migrations");
|
||||||
const db = await initializeDatabase();
|
const db = (await initializeDatabase()) as Parameters<
|
||||||
await migratePostgres(db as any, {
|
typeof migratePostgres
|
||||||
migrationsFolder: "./drizzle/migrations",
|
>[0];
|
||||||
});
|
try {
|
||||||
|
await migratePostgres(db, { migrationsFolder: "./drizzle/migrations" });
|
||||||
|
} finally {
|
||||||
|
await closeDatabase();
|
||||||
|
}
|
||||||
logger.info("PostgreSQL migrations completed successfully");
|
logger.info("PostgreSQL migrations completed successfully");
|
||||||
} else {
|
} else {
|
||||||
logger.info("Running SQLite migrations");
|
logger.info("Running SQLite migrations");
|
||||||
const { sqlite, db } = await initializeMigrationSqliteDatabase();
|
const { sqlite, db } = initializeMigrationSqliteDatabase();
|
||||||
migrateSqlite(db, { migrationsFolder: "./drizzle/migrations" });
|
try {
|
||||||
// Ensure the SQLite connection is closed after migrations
|
migrateSqlite(db, { migrationsFolder: "./drizzle/migrations" });
|
||||||
sqlite.close();
|
} finally {
|
||||||
|
sqlite.close();
|
||||||
|
}
|
||||||
logger.info("SQLite migrations completed successfully");
|
logger.info("SQLite migrations completed successfully");
|
||||||
}
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
@@ -45,7 +49,6 @@ export async function runMigrations(): Promise<void> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Run migrations if called directly
|
|
||||||
if (import.meta.url === `file://${process.argv[1]}`) {
|
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||||
runMigrations()
|
runMigrations()
|
||||||
.then(() => {
|
.then(() => {
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
import fs from "node:fs";
|
import fs from "fs";
|
||||||
import path from "node:path";
|
import path from "path";
|
||||||
import ffmpeg from "fluent-ffmpeg";
|
import { buildMuxFfmpegArgs, runFfmpeg } from "./audio/ffmpegProcess";
|
||||||
|
|
||||||
const recordingsDir = process.env.RECORDINGS_DIR ?? "./recordings";
|
const recordingsDir = process.env.RECORDINGS_DIR ?? "./recordings";
|
||||||
|
|
||||||
@@ -73,7 +73,7 @@ async function startMuxingToAup3() {
|
|||||||
|
|
||||||
// Check if OGG file has valid header (starts with "OggS")
|
// Check if OGG file has valid header (starts with "OggS")
|
||||||
const oggBuffer = fs.readFileSync(oggPath);
|
const oggBuffer = fs.readFileSync(oggPath);
|
||||||
const oggHeader = oggBuffer.slice(0, 4).toString();
|
const oggHeader = oggBuffer.subarray(0, 4).toString();
|
||||||
if (oggHeader !== "OggS") {
|
if (oggHeader !== "OggS") {
|
||||||
console.warn(
|
console.warn(
|
||||||
`[muxer-aup3] Skipping invalid OGG file (bad header): ${oggPath}`,
|
`[muxer-aup3] Skipping invalid OGG file (bad header): ${oggPath}`,
|
||||||
@@ -116,15 +116,12 @@ async function startMuxingToAup3() {
|
|||||||
`[muxer-aup3] Found ${clips.length} clips. Base timestamp: ${globalStartTime}`,
|
`[muxer-aup3] Found ${clips.length} clips. Base timestamp: ${globalStartTime}`,
|
||||||
);
|
);
|
||||||
|
|
||||||
const command = ffmpeg();
|
|
||||||
const filterParts: string[] = [];
|
const filterParts: string[] = [];
|
||||||
|
|
||||||
console.log(
|
console.log(
|
||||||
`[muxer-aup3] Creating audio filters for ${clips.length} clips...`,
|
`[muxer-aup3] Creating audio filters for ${clips.length} clips...`,
|
||||||
);
|
);
|
||||||
clips.forEach((clip, index) => {
|
clips.forEach((clip, index) => {
|
||||||
command.input(clip.oggPath);
|
|
||||||
|
|
||||||
// Calculate delay relative to the global start time
|
// Calculate delay relative to the global start time
|
||||||
const delayMs = clip.meta.startTime - globalStartTime;
|
const delayMs = clip.meta.startTime - globalStartTime;
|
||||||
|
|
||||||
@@ -154,33 +151,29 @@ async function startMuxingToAup3() {
|
|||||||
const timestamp = Date.now();
|
const timestamp = Date.now();
|
||||||
const wavFilename = path.join(recordingsDir, `muxed-${timestamp}.wav`);
|
const wavFilename = path.join(recordingsDir, `muxed-${timestamp}.wav`);
|
||||||
const aup3Filename = path.join(recordingsDir, `muxed-${timestamp}.aup3`);
|
const aup3Filename = path.join(recordingsDir, `muxed-${timestamp}.aup3`);
|
||||||
|
const inputs = clips.map((clip) => clip.oggPath);
|
||||||
|
|
||||||
console.log(
|
console.log(
|
||||||
`[muxer-aup3] Combining clips to WAV. This might take a while...`,
|
`[muxer-aup3] Combining clips to WAV. This might take a while...`,
|
||||||
);
|
);
|
||||||
|
|
||||||
// Using fluent-ffmpeg's complexFilter
|
try {
|
||||||
command
|
const args = buildMuxFfmpegArgs({
|
||||||
.complexFilter(filterParts, "out")
|
inputs,
|
||||||
.audioCodec("pcm_s16le")
|
filter: filterParts.join(";"),
|
||||||
.audioFrequency(44100)
|
output: wavFilename,
|
||||||
.audioChannels(2)
|
codec: "pcm_s16le",
|
||||||
.save(wavFilename)
|
audioFrequency: 44100,
|
||||||
.on("progress", (progress) => {
|
audioChannels: 2,
|
||||||
if (progress.percent) {
|
|
||||||
console.log(
|
|
||||||
`[muxer-aup3] WAV Progress: ${progress.percent.toFixed(2)}%`,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
})
|
|
||||||
.on("end", () => {
|
|
||||||
console.log(`[muxer-aup3] WAV file created: ${wavFilename}`);
|
|
||||||
console.log(`[muxer-aup3] Creating AUP3 project file...`);
|
|
||||||
createAup3Project(wavFilename, aup3Filename, clips, globalStartTime);
|
|
||||||
})
|
|
||||||
.on("error", (err) => {
|
|
||||||
console.error(`[muxer-aup3] FFmpeg Error:`, err);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
|
await runFfmpeg(args);
|
||||||
|
console.log(`[muxer-aup3] WAV file created: ${wavFilename}`);
|
||||||
|
console.log(`[muxer-aup3] Creating AUP3 project file...`);
|
||||||
|
createAup3Project(wavFilename, aup3Filename, clips, globalStartTime);
|
||||||
|
} catch (err) {
|
||||||
|
console.error(`[muxer-aup3] FFmpeg Error:`, err);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
function createAup3Project(
|
function createAup3Project(
|
||||||
|
|||||||
43
src/muxer.ts
43
src/muxer.ts
@@ -1,6 +1,6 @@
|
|||||||
import fs from "node:fs";
|
import fs from "fs";
|
||||||
import path from "node:path";
|
import path from "path";
|
||||||
import ffmpeg from "fluent-ffmpeg";
|
import { buildMuxFfmpegArgs, runFfmpeg } from "./audio/ffmpegProcess";
|
||||||
|
|
||||||
const recordingsDir = process.env.RECORDINGS_DIR ?? "./recordings";
|
const recordingsDir = process.env.RECORDINGS_DIR ?? "./recordings";
|
||||||
|
|
||||||
@@ -71,7 +71,7 @@ async function startMuxing() {
|
|||||||
|
|
||||||
// Check if OGG file has valid header (starts with "OggS")
|
// Check if OGG file has valid header (starts with "OggS")
|
||||||
const oggBuffer = fs.readFileSync(oggPath);
|
const oggBuffer = fs.readFileSync(oggPath);
|
||||||
const oggHeader = oggBuffer.slice(0, 4).toString();
|
const oggHeader = oggBuffer.subarray(0, 4).toString();
|
||||||
if (oggHeader !== "OggS") {
|
if (oggHeader !== "OggS") {
|
||||||
console.warn(
|
console.warn(
|
||||||
`[muxer] Skipping invalid OGG file (bad header): ${oggPath}`,
|
`[muxer] Skipping invalid OGG file (bad header): ${oggPath}`,
|
||||||
@@ -114,13 +114,10 @@ async function startMuxing() {
|
|||||||
`[muxer] Found ${clips.length} clips. Base timestamp: ${globalStartTime}`,
|
`[muxer] Found ${clips.length} clips. Base timestamp: ${globalStartTime}`,
|
||||||
);
|
);
|
||||||
|
|
||||||
const command = ffmpeg();
|
|
||||||
const filterParts: string[] = [];
|
const filterParts: string[] = [];
|
||||||
|
|
||||||
console.log(`[muxer] Creating audio filters for ${clips.length} clips...`);
|
console.log(`[muxer] Creating audio filters for ${clips.length} clips...`);
|
||||||
clips.forEach((clip, index) => {
|
clips.forEach((clip, index) => {
|
||||||
command.input(clip.oggPath);
|
|
||||||
|
|
||||||
// Calculate delay relative to the global start time
|
// Calculate delay relative to the global start time
|
||||||
const delayMs = clip.meta.startTime - globalStartTime;
|
const delayMs = clip.meta.startTime - globalStartTime;
|
||||||
|
|
||||||
@@ -148,27 +145,25 @@ async function startMuxing() {
|
|||||||
);
|
);
|
||||||
|
|
||||||
const outputFilename = path.join(recordingsDir, `muxed-${Date.now()}.mp3`);
|
const outputFilename = path.join(recordingsDir, `muxed-${Date.now()}.mp3`);
|
||||||
|
const inputs = clips.map((clip) => clip.oggPath);
|
||||||
|
|
||||||
console.log(`[muxer] Combining clips. This might take a while...`);
|
console.log(`[muxer] Combining clips. This might take a while...`);
|
||||||
|
|
||||||
// Using fluent-ffmpeg's complexFilter
|
try {
|
||||||
command
|
const args = buildMuxFfmpegArgs({
|
||||||
.complexFilter(filterParts, "out")
|
inputs,
|
||||||
.audioCodec("libmp3lame")
|
filter: filterParts.join(";"),
|
||||||
.save(outputFilename)
|
output: outputFilename,
|
||||||
.on("progress", (progress) => {
|
codec: "libmp3lame",
|
||||||
if (progress.percent) {
|
|
||||||
console.log(`[muxer] Progress: ${progress.percent.toFixed(2)}%`);
|
|
||||||
}
|
|
||||||
})
|
|
||||||
.on("end", () => {
|
|
||||||
console.log(
|
|
||||||
`[muxer] Successfully muxed! Output saved to: ${outputFilename}`,
|
|
||||||
);
|
|
||||||
})
|
|
||||||
.on("error", (err) => {
|
|
||||||
console.error(`[muxer] FFmpeg Error:`, err);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
|
await runFfmpeg(args);
|
||||||
|
console.log(
|
||||||
|
`[muxer] Successfully muxed! Output saved to: ${outputFilename}`,
|
||||||
|
);
|
||||||
|
} catch (err) {
|
||||||
|
console.error(`[muxer] FFmpeg Error:`, err);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
startMuxing();
|
startMuxing();
|
||||||
|
|||||||
@@ -32,7 +32,7 @@ export async function retryWithBackoff<T>(
|
|||||||
{
|
{
|
||||||
attempt: error.attemptNumber,
|
attempt: error.attemptNumber,
|
||||||
retriesLeft: error.retriesLeft,
|
retriesLeft: error.retriesLeft,
|
||||||
error: error.message,
|
error: error.error.message,
|
||||||
},
|
},
|
||||||
"Retry attempt",
|
"Retry attempt",
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
|
|
||||||
export const userStateUpdateSchema = z.object({
|
const userStateUpdateSchema = z.object({
|
||||||
userId: z.string(),
|
userId: z.string(),
|
||||||
username: z.string(),
|
username: z.string(),
|
||||||
avatar: z.string(),
|
avatar: z.string(),
|
||||||
|
|||||||
102
tests/audio/ffmpegProcess.test.ts
Normal file
102
tests/audio/ffmpegProcess.test.ts
Normal file
@@ -0,0 +1,102 @@
|
|||||||
|
import { spawn } from "node:child_process";
|
||||||
|
import { EventEmitter } from "node:events";
|
||||||
|
import { describe, expect, it, vi } from "vitest";
|
||||||
|
import { buildMuxFfmpegArgs, runFfmpeg } from "../../src/audio/ffmpegProcess";
|
||||||
|
|
||||||
|
vi.mock("node:child_process", () => ({
|
||||||
|
spawn: vi.fn(),
|
||||||
|
}));
|
||||||
|
|
||||||
|
const spawnMock = vi.mocked(spawn);
|
||||||
|
|
||||||
|
describe("buildMuxFfmpegArgs", () => {
|
||||||
|
it("builds args with multiple inputs and libmp3lame codec", () => {
|
||||||
|
const args = buildMuxFfmpegArgs({
|
||||||
|
inputs: ["a.ogg", "b.ogg"],
|
||||||
|
filter:
|
||||||
|
"[0:a]adelay=0|0[pad0];[1:a]adelay=1000|1000[pad1];[pad0][pad1]amix=inputs=2:dropout_transition=0[out]",
|
||||||
|
output: "out.mp3",
|
||||||
|
codec: "libmp3lame",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(args).toEqual([
|
||||||
|
"-y",
|
||||||
|
"-i",
|
||||||
|
"a.ogg",
|
||||||
|
"-i",
|
||||||
|
"b.ogg",
|
||||||
|
"-filter_complex",
|
||||||
|
"[0:a]adelay=0|0[pad0];[1:a]adelay=1000|1000[pad1];[pad0][pad1]amix=inputs=2:dropout_transition=0[out]",
|
||||||
|
"-map",
|
||||||
|
"[out]",
|
||||||
|
"-codec:a",
|
||||||
|
"libmp3lame",
|
||||||
|
"out.mp3",
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("includes audio frequency and channel options when provided", () => {
|
||||||
|
const args = buildMuxFfmpegArgs({
|
||||||
|
inputs: ["a.ogg"],
|
||||||
|
filter:
|
||||||
|
"[0:a]adelay=0|0[pad0];[pad0]amix=inputs=1:dropout_transition=0[out]",
|
||||||
|
output: "out.wav",
|
||||||
|
codec: "pcm_s16le",
|
||||||
|
audioFrequency: 44100,
|
||||||
|
audioChannels: 2,
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(args).toContain("-ar");
|
||||||
|
expect(args).toContain("44100");
|
||||||
|
expect(args).toContain("-ac");
|
||||||
|
expect(args).toContain("2");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not include -ar or -ac when audioFrequency and audioChannels are not provided", () => {
|
||||||
|
const args = buildMuxFfmpegArgs({
|
||||||
|
inputs: ["a.ogg"],
|
||||||
|
filter:
|
||||||
|
"[0:a]adelay=0|0[pad0];[pad0]amix=inputs=1:dropout_transition=0[out]",
|
||||||
|
output: "out.mp3",
|
||||||
|
codec: "libmp3lame",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(args).not.toContain("-ar");
|
||||||
|
expect(args).not.toContain("-ac");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("runFfmpeg", () => {
|
||||||
|
it("spawns ffmpeg and resolves on exit code 0", async () => {
|
||||||
|
const proc = new EventEmitter();
|
||||||
|
spawnMock.mockReturnValue(proc as ReturnType<typeof spawn>);
|
||||||
|
|
||||||
|
const result = runFfmpeg(["-version"]);
|
||||||
|
proc.emit("close", 0);
|
||||||
|
|
||||||
|
await expect(result).resolves.toBeUndefined();
|
||||||
|
expect(spawnMock).toHaveBeenCalledWith("ffmpeg", ["-version"], {
|
||||||
|
stdio: ["ignore", "inherit", "inherit"],
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects on non-zero exit code", async () => {
|
||||||
|
const proc = new EventEmitter();
|
||||||
|
spawnMock.mockReturnValue(proc as ReturnType<typeof spawn>);
|
||||||
|
|
||||||
|
const result = runFfmpeg(["-bad"]);
|
||||||
|
proc.emit("close", 1);
|
||||||
|
|
||||||
|
await expect(result).rejects.toThrow("ffmpeg exited with code 1");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects on spawn error", async () => {
|
||||||
|
const proc = new EventEmitter();
|
||||||
|
spawnMock.mockReturnValue(proc as ReturnType<typeof spawn>);
|
||||||
|
|
||||||
|
const result = runFfmpeg(["-version"]);
|
||||||
|
proc.emit("error", new Error("spawn ffmpeg ENOENT"));
|
||||||
|
|
||||||
|
await expect(result).rejects.toThrow("spawn ffmpeg ENOENT");
|
||||||
|
});
|
||||||
|
});
|
||||||
21
tests/database/migrate.test.ts
Normal file
21
tests/database/migrate.test.ts
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
import { mkdtempSync, rmSync } from "node:fs";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { initializeMigrationSqliteDatabase } from "../../src/database/migrate";
|
||||||
|
|
||||||
|
describe("initializeMigrationSqliteDatabase", () => {
|
||||||
|
it("creates a SQLite DB with WAL journal mode", () => {
|
||||||
|
const dir = mkdtempSync(join(tmpdir(), "bete-migrate-"));
|
||||||
|
const dbPath = join(dir, "test.db");
|
||||||
|
const { sqlite, db } = initializeMigrationSqliteDatabase(dbPath);
|
||||||
|
|
||||||
|
try {
|
||||||
|
expect(db).toBeDefined();
|
||||||
|
expect(sqlite.pragma("journal_mode", { simple: true })).toBe("wal");
|
||||||
|
} finally {
|
||||||
|
sqlite.close();
|
||||||
|
rmSync(dir, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -1,30 +1,35 @@
|
|||||||
import { expect, test } from "vitest";
|
import { describe, expect, it } from "vitest";
|
||||||
import { UserStateUpdate, validateUserStateUpdate } from "../src/validation";
|
import { validateUserStateUpdate } from "../src/validation";
|
||||||
|
|
||||||
test("valid object returns typed object", async () => {
|
describe("validateUserStateUpdate", () => {
|
||||||
const input = {
|
it("returns typed data for a valid user state update", async () => {
|
||||||
userId: "123",
|
const result = await validateUserStateUpdate({
|
||||||
username: "testuser",
|
userId: "123",
|
||||||
avatar: "avatar.png",
|
username: "aseph",
|
||||||
speaking: true,
|
avatar: "https://example.invalid/avatar.png",
|
||||||
};
|
speaking: true,
|
||||||
const result = await validateUserStateUpdate(input);
|
});
|
||||||
expect(result).toEqual(input as UserStateUpdate);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("non-object input returns null", async () => {
|
expect(result).toEqual({
|
||||||
// @ts-expect-error testing invalid input
|
userId: "123",
|
||||||
const result = await validateUserStateUpdate("not an object");
|
username: "aseph",
|
||||||
expect(result).toBeNull();
|
avatar: "https://example.invalid/avatar.png",
|
||||||
});
|
speaking: true,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
test("invalid field types return null", async () => {
|
it("returns null for non-object input", async () => {
|
||||||
const input = {
|
await expect(validateUserStateUpdate("bad")).resolves.toBeNull();
|
||||||
userId: "123",
|
});
|
||||||
username: "testuser",
|
|
||||||
avatar: "avatar.png",
|
it("returns null for invalid field types", async () => {
|
||||||
speaking: "true", // invalid type
|
const result = await validateUserStateUpdate({
|
||||||
};
|
userId: "123",
|
||||||
const result = await validateUserStateUpdate(input as unknown);
|
username: "aseph",
|
||||||
expect(result).toBeNull();
|
avatar: "https://example.invalid/avatar.png",
|
||||||
|
speaking: "true",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result).toBeNull();
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user