Compare commits

..

34 Commits

Author SHA1 Message Date
MythEclipse
203aa9a589 style: organize imports after dashboard rebuild 2026-05-14 21:19:43 +07:00
MythEclipse
196ca1b784 feat: serve React dashboard from public/app when available
Check for public/app/index.html before falling back to the old
public/index.html so the React moderation dashboard is served at
the root in production while preserving the legacy frontend during
the transition period.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 21:16:03 +07:00
MythEclipse
a812182218 fix: harden dashboard message state 2026-05-14 21:10:24 +07:00
MythEclipse
cb2cfc76f2 fix: import dashboard event type from websocket client 2026-05-14 21:03:38 +07:00
MythEclipse
ae02556f75 feat: add moderation review dashboard
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 21:01:57 +07:00
MythEclipse
fbf0f11db1 fix: align dashboard message api shape 2026-05-14 20:55:05 +07:00
MythEclipse
0a7965e1d4 fix: remove unused dashboard type import 2026-05-14 20:47:51 +07:00
MythEclipse
2d56012bf5 feat: add dashboard api clients 2026-05-14 20:46:21 +07:00
MythEclipse
c3f2d01e9b a 2026-05-14 20:27:35 +07:00
MythEclipse
298048aaaf fix: isolate frontend typescript config 2026-05-14 20:26:38 +07:00
MythEclipse
71a28085d2 feat: scaffold react dashboard 2026-05-14 20:24:41 +07:00
MythEclipse
da5e191b0f fix: make moderation index migration portable 2026-05-14 20:19:47 +07:00
MythEclipse
558d65342b feat: add moderation query indexes 2026-05-14 20:14:42 +07:00
MythEclipse
44368e646f fix: reanalyze edited messages 2026-05-14 20:10:16 +07:00
MythEclipse
dbfc6866f9 refactor: keep message capture on fast path 2026-05-14 20:03:02 +07:00
MythEclipse
d8aeefb739 fix: preserve ui state broadcasts in routes 2026-05-14 19:58:37 +07:00
MythEclipse
08cd515d8a fix: preserve ui state broadcasts in routes 2026-05-14 19:55:26 +07:00
MythEclipse
c81a499535 refactor: split api routes by concern 2026-05-14 19:46:47 +07:00
MythEclipse
3fb1fcb72c fix: remove unused analysis import 2026-05-14 19:41:18 +07:00
MythEclipse
243a18ecad fix: harden analysis queue scheduling 2026-05-14 19:39:25 +07:00
MythEclipse
f14e893cb7 feat: debounce ai analysis by conversation 2026-05-14 19:32:44 +07:00
MythEclipse
54cd4e0386 fix: type llm moderation test fixtures 2026-05-14 19:28:56 +07:00
MythEclipse
65ab5ecb32 fix: harden llm moderation parsing 2026-05-14 19:26:54 +07:00
MythEclipse
81253e4ffe feat: add strict llm moderation client 2026-05-14 19:21:15 +07:00
MythEclipse
ce35b335d0 test: cover conversation context edge cases 2026-05-14 19:16:46 +07:00
MythEclipse
2b4e2a7ab7 feat: add conversation context builder 2026-05-14 19:10:13 +07:00
MythEclipse
2d511e08db style: format message query changes 2026-05-14 19:06:17 +07:00
MythEclipse
13078e7c3c fix: resolve message query test diagnostics 2026-05-14 19:02:19 +07:00
MythEclipse
298fc968cf test: cover message query pagination 2026-05-14 19:01:01 +07:00
MythEclipse
7a8883f623 test: cover message query pagination 2026-05-14 19:00:03 +07:00
MythEclipse
0c54ac4ba8 test: cover message query pagination 2026-05-14 18:54:13 +07:00
MythEclipse
9f5f8a3090 feat: add cursor-based message queries 2026-05-14 18:48:02 +07:00
MythEclipse
8ab7aaa32d fix: harden moderation broadcaster 2026-05-14 18:44:47 +07:00
MythEclipse
01dc9b1836 feat: add typed moderation broadcaster
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 18:40:34 +07:00
41 changed files with 6914 additions and 683 deletions

30
.env.test Normal file
View File

@@ -0,0 +1,30 @@
DISCORD_TOKEN=test_token_for_testing
RECORDINGS_DIR=./recordings
RECORDING_SEGMENT_MS=5000
VERBOSE=false
DECODER_ROTATE_MS=5000
DECODER_COOLDOWN_MS=30000
AUDIO_STREAM_SILENCE_DURATION_MS=3000
PACKET_FILTER_MIN_SIZE=8
OPUS_FRAME_SIZE=960
AUDIO_SAMPLE_RATE=48000
AUDIO_CHANNELS=2
AVATAR_SIZE=64
WEBSERVER_PORT=3000
VOICE_CONNECTION_TIMEOUT_MS=15000
RECONNECT_TIMEOUT_MS=5000
LOG_LEVEL=info
NODE_ENV=test
MONITOR_GUILD_ID=test_guild_id
PICSER_UPLOAD_URL=https://picser.asepharyana.tech/api/upload
ATTACHMENT_UPLOAD_TIMEOUT_MS=30000
ATTACHMENT_MAX_SIZE_MB=100
ATTACHMENT_RETRY_ATTEMPTS=3
BACKLOG_SYNC_HOURS=24
BACKLOG_SYNC_BATCH_SIZE=100
AI_ANALYSIS_ENABLED=false
AI_LLM_API_KEY=test_key
AI_LLM_BASE_URL=https://9router.asepharyana.tech/v1
AI_LLM_MODEL=free
AI_ANALYSIS_TIMEOUT_MS=30000
DATABASE_TYPE=sqlite

4
.gitignore vendored
View File

@@ -2,4 +2,6 @@ node_modules
recordings recordings
.env .env
dist/ dist/
.muxer-queue.** public/app/
.muxer-queue.**
.claude/

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,264 @@
# AI Message Flow + React Dashboard Redesign
## Goal
Rebuild the moderation watcher flow so message capture is fast, AI analysis is contextual and reliable, APIs are split by concern, and the dashboard is maintainable. The implementation may fully replace the current static frontend and reorganize backend modules, while preserving existing voice functionality.
## Current Problems
- Message capture, AI queuing, DB access, and WebSocket broadcasting are tightly coupled.
- AI analysis batches depend on array ordering, which can mismatch results when the model returns malformed or partial JSON.
- Pending analysis is polled globally, not grouped by conversation, so context is weak and request efficiency is inconsistent.
- `/api/messages` mixes text/image concerns and uses offset pagination, which gets slower and less stable as rows grow.
- Frontend is a large static HTML file with inline state, API, WebSocket, rendering, and audio code.
- WebSocket broadcast uses untyped `globalThis` hooks across modules.
## Backend Architecture
### Message ingestion
`messageCapture` becomes a narrow ingestion layer:
1. Filter Discord events by guild and author.
2. Normalize message payload and metadata.
3. Upsert message/attachment records.
4. Set or reset `ai_status` to `pending` for new or edited text.
5. Emit typed domain events for WebSocket broadcasting and analysis queueing.
It should not build prompts, manage AI batches, or query unrelated DB state.
### Store/repository layer
`messageStore` becomes the single message/attachment query boundary. It should expose focused functions:
- `upsertMessage`
- `markMessageEdited`
- `markMessageDeleted`
- `listMessages`
- `listReviewMessages`
- `getConversationContext`
- `claimPendingMessagesForChannel`
- `saveAnalysisResults`
- `insertAttachments`
Queries should use cursor pagination based on `(created_at, id)` instead of offset pagination. Common filters: `guildId`, `channelId`, `threadId`, `status`, `userId`, `q`, `limit`, `cursor`.
### Analysis queue
Add an `analysisQueue` module. It owns async AI processing and keeps capture fast.
- Queue key: `thread_id ?? channel_id`.
- Debounce: 13 seconds per key to group nearby messages.
- Batch pending messages by conversation key and token budget.
- Only one or a small fixed number of active LLM requests.
- Backlog worker feeds the same queue; no separate analysis path.
- Edits reset a message to `pending` and enqueue its conversation key.
If the process restarts, pending rows are recovered by a periodic lightweight scanner grouped by conversation key.
### Conversation context builder
Add `conversationContext` module:
- Input: conversation key + target pending messages.
- Fetch context before the first target message, normally 20 prior messages.
- Include target messages and close neighboring messages when within budget.
- Mark target messages explicitly in the prompt.
- Keep context scoped to one channel/thread to avoid irrelevant noise.
### LLM moderation client
Add `llmModerationClient` module:
- Own request shape, timeout, retry, JSON extraction, and validation.
- Prompt returns JSON keyed by `message_id`, not positional arrays.
- Expected response shape:
```json
{
"results": [
{
"message_id": "string",
"status": "clean|warn|flagged",
"flags": ["string"],
"score": 0.0,
"analysis": "Bahasa Indonesia summary + reason + suggested action"
}
]
}
```
- Reject unknown IDs, invalid statuses, invalid scores, and missing target IDs.
- On partial model failure, retry once with smaller batch. If still invalid, mark only affected target messages as `error`.
- Store raw batch request/response in one run record if the DB migration is included; otherwise store compact raw metadata per message.
## API Design
Split API by use case so reads remain fast and obvious.
### Message read APIs
- `GET /api/messages`
- Query: `guildId`, `channelId`, `threadId`, `cursor`, `limit`, `status`, `userId`, `q`.
- Returns: `{ data, nextCursor }`.
- Uses indexed cursor pagination.
- `GET /api/messages/:id`
- Returns one message with attachments and AI analysis.
- `GET /api/review`
- Query: `guildId`, optional `channelId`, `status=warn,flagged,error`, `cursor`, `limit`.
- Optimized for moderator review panel.
- `GET /api/attachments`
- Query: `channelId`, `threadId`, `cursor`, `limit`, `type`.
- Replaces image mode inside `/api/messages`.
### Analysis APIs
- `POST /api/messages/:id/reanalyze`
- Sets message to `pending` and queues its conversation.
- Returns `202 Accepted` with current message status.
- `POST /api/analysis/requeue-pending`
- Admin/manual recovery endpoint for pending/error rows.
- Returns count queued.
- `GET /api/analysis/status`
- Returns queue depth, active requests, last error, and pending counts.
### Discord sync APIs
- `POST /api/backlog-sync`
- Stays async-friendly: starts sync for guild/channel/thread and returns `202` with a job id or immediate summary if small.
- Sync inserts messages, then queues analysis through the same `analysisQueue`.
### Voice/control APIs
Keep existing voice APIs working, but move route registration into route modules:
- `routes/voiceRoutes.ts`
- `routes/messageRoutes.ts`
- `routes/analysisRoutes.ts`
- `routes/syncRoutes.ts`
- `routes/uiStateRoutes.ts`
`webserver.ts` should only create Express/WS server, install middleware, register routes, and start listening.
## WebSocket Design
Replace ad-hoc globals with a typed broadcaster module.
Events:
- `ui_state`
- `user_state`
- `message_created`
- `message_updated`
- `message_deleted`
- `message_analyzed`
- `attachment_created`
- `analysis_queue_status`
Backend modules call broadcaster functions; they do not touch WebSocket clients directly.
## Database Changes
Add or verify indexes:
- messages `(channel_id, created_at, id)`
- messages `(thread_id, created_at, id)`
- messages `(ai_status, created_at, id)`
- messages `(guild_id, ai_status, created_at, id)`
- attachments `(channel_id, created_at, id)`
- attachments `(thread_id, created_at, id)`
Optional but preferred:
- `ai_analysis_runs` table:
- `id`
- `conversation_key`
- `target_message_ids`
- `model`
- `request_tokens_estimate`
- `response_raw`
- `status`
- `error`
- `created_at`
- `completed_at`
This avoids duplicating large raw LLM responses into every message row.
## React/Vite Frontend
Replace static inline dashboard code with a TypeScript React app.
Suggested structure:
- `frontend/src/api/` — typed REST clients
- `frontend/src/ws/` — WebSocket client and event types
- `frontend/src/state/` — small hooks for selected guild/channel, messages, review queue, voice state
- `frontend/src/components/voice/` — existing voice control/audio components
- `frontend/src/components/messages/` — feed, message card, filters, detail drawer
- `frontend/src/components/review/` — needs-review list and analysis status
- `frontend/src/components/layout/` — shell/sidebar/status cards
UI layout:
- Left sidebar: guild, voice channel, text channel/thread, connection state.
- Main area: message feed with filters and load-more cursor pagination.
- Right panel: review queue for `warn`, `flagged`, and `error` messages.
- Detail drawer/modal: message metadata, attachments, AI rationale, raw flags, reanalyze action.
Voice features stay functionally equivalent. Audio capture/playback code can be moved into React hooks but should not be behaviorally rewritten unless needed.
Build integration:
- Add Vite dev/build scripts.
- Express serves the built app from a stable public directory in production.
- During development, either run Vite separately or proxy API/WS to Express.
## Performance Rules
- Message capture must not wait on AI.
- Read APIs use cursor pagination and indexes.
- AI batches are bounded by token estimate and message count.
- UI fetches initial pages, then patches via WebSocket.
- Backlog sync should not block dashboard interactions.
- Avoid storing full raw LLM response per message when a batch table is available.
## Error Handling
- Capture errors log and do not crash Discord client event handlers.
- AI request failures mark target messages `error` with a short reason.
- Invalid LLM JSON triggers retry/split before marking errors.
- API validation returns 400 with structured error code.
- WebSocket reconnect logic stays client-side.
- Manual reanalysis provides recovery for bad AI results.
## Testing
Backend:
- Unit tests for conversation context selection.
- Unit tests for LLM response parser and validation.
- Unit tests for queue batching/debounce behavior.
- Integration tests for message cursor pagination and review filters.
- Existing voice tests remain unchanged.
Frontend:
- Typecheck and Vite build.
- Component-level smoke tests may be added if test tooling is already practical.
- Manual browser verification: channel select, message feed, review panel, WebSocket updates, reanalyze action, voice controls.
## Implementation Scope
This is a full redesign of the message/AI/dashboard path. Voice recording and live audio behavior should be preserved unless a change is required to integrate the React dashboard.
Implementation should proceed incrementally:
1. Backend boundaries and typed broadcaster.
2. Store/query improvements and indexes.
3. Analysis queue/context/client rewrite.
4. Split API routes.
5. React/Vite dashboard.
6. Verification and cleanup of old static dashboard code.

View File

@@ -0,0 +1,22 @@
CREATE TABLE "ai_analysis_runs" (
"id" text PRIMARY KEY NOT NULL,
"conversation_key" text NOT NULL,
"target_message_ids" text NOT NULL,
"model" text NOT NULL,
"request_tokens_estimate" integer,
"response_raw" text,
"status" text DEFAULT 'pending' NOT NULL,
"error" text,
"created_at" bigint NOT NULL,
"completed_at" bigint
);
--> statement-breakpoint
CREATE INDEX "idx_ai_analysis_runs_conversation_key" ON "ai_analysis_runs" ("conversation_key");--> statement-breakpoint
CREATE INDEX "idx_ai_analysis_runs_status" ON "ai_analysis_runs" ("status");--> statement-breakpoint
CREATE INDEX "idx_ai_analysis_runs_created_at" ON "ai_analysis_runs" ("created_at");--> statement-breakpoint
CREATE INDEX "idx_attachments_channel_created" ON "attachments" ("channel_id","created_at","id");--> statement-breakpoint
CREATE INDEX "idx_attachments_thread_created" ON "attachments" ("thread_id","created_at","id");--> statement-breakpoint
CREATE INDEX "idx_messages_channel_created" ON "messages" ("channel_id","created_at","id");--> statement-breakpoint
CREATE INDEX "idx_messages_thread_created" ON "messages" ("thread_id","created_at","id");--> statement-breakpoint
CREATE INDEX "idx_messages_ai_status_created" ON "messages" ("ai_status","created_at","id");--> statement-breakpoint
CREATE INDEX "idx_messages_guild_ai_status_created" ON "messages" ("guild_id","ai_status","created_at","id");

View File

@@ -0,0 +1,799 @@
{
"id": "79348c3a-b314-4bfd-88f3-7ddfbea4427e",
"prevId": "2b9e2347-dd99-4bf8-bbcb-f407af29ca83",
"version": "7",
"dialect": "postgresql",
"tables": {
"public.ai_analysis_runs": {
"name": "ai_analysis_runs",
"schema": "",
"columns": {
"id": {
"name": "id",
"type": "text",
"primaryKey": true,
"notNull": true
},
"conversation_key": {
"name": "conversation_key",
"type": "text",
"primaryKey": false,
"notNull": true
},
"target_message_ids": {
"name": "target_message_ids",
"type": "text",
"primaryKey": false,
"notNull": true
},
"model": {
"name": "model",
"type": "text",
"primaryKey": false,
"notNull": true
},
"request_tokens_estimate": {
"name": "request_tokens_estimate",
"type": "integer",
"primaryKey": false,
"notNull": false
},
"response_raw": {
"name": "response_raw",
"type": "text",
"primaryKey": false,
"notNull": false
},
"status": {
"name": "status",
"type": "text",
"primaryKey": false,
"notNull": true,
"default": "'pending'"
},
"error": {
"name": "error",
"type": "text",
"primaryKey": false,
"notNull": false
},
"created_at": {
"name": "created_at",
"type": "bigint",
"primaryKey": false,
"notNull": true
},
"completed_at": {
"name": "completed_at",
"type": "bigint",
"primaryKey": false,
"notNull": false
}
},
"indexes": {
"idx_ai_analysis_runs_conversation_key": {
"name": "idx_ai_analysis_runs_conversation_key",
"columns": [
{
"expression": "conversation_key",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_ai_analysis_runs_status": {
"name": "idx_ai_analysis_runs_status",
"columns": [
{
"expression": "status",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_ai_analysis_runs_created_at": {
"name": "idx_ai_analysis_runs_created_at",
"columns": [
{
"expression": "created_at",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
}
},
"foreignKeys": {},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
},
"public.attachments": {
"name": "attachments",
"schema": "",
"columns": {
"id": {
"name": "id",
"type": "text",
"primaryKey": true,
"notNull": true
},
"message_id": {
"name": "message_id",
"type": "text",
"primaryKey": false,
"notNull": true
},
"guild_id": {
"name": "guild_id",
"type": "text",
"primaryKey": false,
"notNull": true
},
"channel_id": {
"name": "channel_id",
"type": "text",
"primaryKey": false,
"notNull": true
},
"thread_id": {
"name": "thread_id",
"type": "text",
"primaryKey": false,
"notNull": false
},
"user_id": {
"name": "user_id",
"type": "text",
"primaryKey": false,
"notNull": true
},
"filename": {
"name": "filename",
"type": "text",
"primaryKey": false,
"notNull": true
},
"size": {
"name": "size",
"type": "integer",
"primaryKey": false,
"notNull": true
},
"type": {
"name": "type",
"type": "text",
"primaryKey": false,
"notNull": true
},
"discord_url": {
"name": "discord_url",
"type": "text",
"primaryKey": false,
"notNull": true
},
"uploaded_url": {
"name": "uploaded_url",
"type": "text",
"primaryKey": false,
"notNull": false
},
"upload_status": {
"name": "upload_status",
"type": "text",
"primaryKey": false,
"notNull": true,
"default": "'pending'"
},
"upload_error": {
"name": "upload_error",
"type": "text",
"primaryKey": false,
"notNull": false
},
"created_at": {
"name": "created_at",
"type": "bigint",
"primaryKey": false,
"notNull": true
},
"uploaded_at": {
"name": "uploaded_at",
"type": "bigint",
"primaryKey": false,
"notNull": false
}
},
"indexes": {
"idx_attachments_channel": {
"name": "idx_attachments_channel",
"columns": [
{
"expression": "channel_id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_attachments_message": {
"name": "idx_attachments_message",
"columns": [
{
"expression": "message_id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_attachments_status": {
"name": "idx_attachments_status",
"columns": [
{
"expression": "upload_status",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_attachments_channel_created": {
"name": "idx_attachments_channel_created",
"columns": [
{
"expression": "channel_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "created_at",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_attachments_thread_created": {
"name": "idx_attachments_thread_created",
"columns": [
{
"expression": "thread_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "created_at",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
}
},
"foreignKeys": {
"fk_attachments_message_id": {
"name": "fk_attachments_message_id",
"tableFrom": "attachments",
"tableTo": "messages",
"columnsFrom": [
"message_id"
],
"columnsTo": [
"id"
],
"onDelete": "cascade",
"onUpdate": "no action"
}
},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
},
"public.messages": {
"name": "messages",
"schema": "",
"columns": {
"id": {
"name": "id",
"type": "text",
"primaryKey": true,
"notNull": true
},
"guild_id": {
"name": "guild_id",
"type": "text",
"primaryKey": false,
"notNull": true
},
"channel_id": {
"name": "channel_id",
"type": "text",
"primaryKey": false,
"notNull": true
},
"thread_id": {
"name": "thread_id",
"type": "text",
"primaryKey": false,
"notNull": false
},
"user_id": {
"name": "user_id",
"type": "text",
"primaryKey": false,
"notNull": true
},
"username": {
"name": "username",
"type": "text",
"primaryKey": false,
"notNull": true
},
"avatar_url": {
"name": "avatar_url",
"type": "text",
"primaryKey": false,
"notNull": false
},
"content": {
"name": "content",
"type": "text",
"primaryKey": false,
"notNull": true
},
"edited_content": {
"name": "edited_content",
"type": "text",
"primaryKey": false,
"notNull": false
},
"created_at": {
"name": "created_at",
"type": "bigint",
"primaryKey": false,
"notNull": true
},
"edited_at": {
"name": "edited_at",
"type": "bigint",
"primaryKey": false,
"notNull": false
},
"deleted_at": {
"name": "deleted_at",
"type": "bigint",
"primaryKey": false,
"notNull": false
},
"type": {
"name": "type",
"type": "text",
"primaryKey": false,
"notNull": true,
"default": "'text'"
},
"metadata": {
"name": "metadata",
"type": "text",
"primaryKey": false,
"notNull": false
},
"ai_status": {
"name": "ai_status",
"type": "text",
"primaryKey": false,
"notNull": true,
"default": "'pending'"
},
"ai_moderation_flags": {
"name": "ai_moderation_flags",
"type": "text",
"primaryKey": false,
"notNull": false
},
"ai_moderation_score": {
"name": "ai_moderation_score",
"type": "real",
"primaryKey": false,
"notNull": false
},
"ai_moderation_raw": {
"name": "ai_moderation_raw",
"type": "text",
"primaryKey": false,
"notNull": false
},
"ai_analysis": {
"name": "ai_analysis",
"type": "text",
"primaryKey": false,
"notNull": false
},
"ai_analyzed_at": {
"name": "ai_analyzed_at",
"type": "bigint",
"primaryKey": false,
"notNull": false
},
"ai_error": {
"name": "ai_error",
"type": "text",
"primaryKey": false,
"notNull": false
}
},
"indexes": {
"idx_messages_channel": {
"name": "idx_messages_channel",
"columns": [
{
"expression": "channel_id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_messages_user": {
"name": "idx_messages_user",
"columns": [
{
"expression": "user_id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_messages_created": {
"name": "idx_messages_created",
"columns": [
{
"expression": "created_at",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_messages_thread": {
"name": "idx_messages_thread",
"columns": [
{
"expression": "thread_id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_messages_channel_created": {
"name": "idx_messages_channel_created",
"columns": [
{
"expression": "channel_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "created_at",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_messages_thread_created": {
"name": "idx_messages_thread_created",
"columns": [
{
"expression": "thread_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "created_at",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_messages_ai_status_created": {
"name": "idx_messages_ai_status_created",
"columns": [
{
"expression": "ai_status",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "created_at",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_messages_guild_ai_status_created": {
"name": "idx_messages_guild_ai_status_created",
"columns": [
{
"expression": "guild_id",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "ai_status",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "created_at",
"isExpression": false,
"asc": true,
"nulls": "last"
},
{
"expression": "id",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
}
},
"foreignKeys": {},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
},
"public.muxer_jobs": {
"name": "muxer_jobs",
"schema": "",
"columns": {
"id": {
"name": "id",
"type": "text",
"primaryKey": true,
"notNull": true
},
"data": {
"name": "data",
"type": "text",
"primaryKey": false,
"notNull": true
},
"status": {
"name": "status",
"type": "text",
"primaryKey": false,
"notNull": true,
"default": "'pending'"
},
"attempts": {
"name": "attempts",
"type": "integer",
"primaryKey": false,
"notNull": true,
"default": 0
},
"maxAttempts": {
"name": "maxAttempts",
"type": "integer",
"primaryKey": false,
"notNull": true,
"default": 3
},
"createdAt": {
"name": "createdAt",
"type": "bigint",
"primaryKey": false,
"notNull": true
},
"updatedAt": {
"name": "updatedAt",
"type": "bigint",
"primaryKey": false,
"notNull": true
},
"error": {
"name": "error",
"type": "text",
"primaryKey": false,
"notNull": false
}
},
"indexes": {
"idx_muxer_jobs_status": {
"name": "idx_muxer_jobs_status",
"columns": [
{
"expression": "status",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
},
"idx_muxer_jobs_createdAt": {
"name": "idx_muxer_jobs_createdAt",
"columns": [
{
"expression": "createdAt",
"isExpression": false,
"asc": true,
"nulls": "last"
}
],
"isUnique": false,
"concurrently": false,
"method": "btree",
"with": {}
}
},
"foreignKeys": {},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
},
"public.ui_state": {
"name": "ui_state",
"schema": "",
"columns": {
"key": {
"name": "key",
"type": "text",
"primaryKey": true,
"notNull": true
},
"value": {
"name": "value",
"type": "text",
"primaryKey": false,
"notNull": true
},
"updated_at": {
"name": "updated_at",
"type": "bigint",
"primaryKey": false,
"notNull": true
}
},
"indexes": {},
"foreignKeys": {},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
}
},
"enums": {},
"schemas": {},
"sequences": {},
"roles": {},
"policies": {},
"views": {},
"_meta": {
"columns": {},
"schemas": {},
"tables": {}
}
}

View File

@@ -8,6 +8,13 @@
"when": 1778750697764, "when": 1778750697764,
"tag": "0000_rare_kitty_pryde", "tag": "0000_rare_kitty_pryde",
"breakpoints": true "breakpoints": true
},
{
"idx": 1,
"version": "7",
"when": 1778764447718,
"tag": "0001_curious_zodiak",
"breakpoints": true
} }
] ]
} }

13
frontend/index.html Normal file
View File

@@ -0,0 +1,13 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Discord Moderation Dashboard</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>

121
frontend/src/App.tsx Normal file
View File

@@ -0,0 +1,121 @@
import { useEffect, useRef, useState } from "react";
import { listMessages, reanalyzeMessage } from "./api/client";
import { connectDashboardSocket } from "./ws/client";
import type { MessageRecord } from "./api/client";
import type { DashboardEvent } from "./ws/client";
import { MessageFeed } from "./components/messages/MessageFeed";
import { ReviewPanel } from "./components/review/ReviewPanel";
export default function App() {
const [messages, setMessages] = useState<MessageRecord[]>([]);
const [wsStatus, setWsStatus] = useState<string>("connecting");
const wsRef = useRef<WebSocket | null>(null);
useEffect(() => {
let cancelled = false;
listMessages(new URLSearchParams({ limit: "30" }))
.then((result) => {
if (!cancelled) {
setMessages(result.data);
}
})
.catch((err) => {
if (!cancelled) {
console.error("Failed to load messages:", err);
}
});
const ws = connectDashboardSocket((event: DashboardEvent) => {
switch (event.type) {
case "message_created":
setMessages((prev) => {
const existing = prev.some((message) => message.id === event.data.id);
if (existing) {
return prev.map((message) =>
message.id === event.data.id ? event.data : message,
);
}
return [event.data, ...prev].slice(0, 200);
});
break;
case "message_analyzed":
setMessages((prev) =>
prev.map((m) => (m.id === event.data.id ? event.data : m)),
);
break;
case "message_updated":
setMessages((prev) =>
prev.map((m) => (m.id === event.data.id ? { ...m, ...event.data } : m)),
);
break;
case "message_deleted":
setMessages((prev) =>
prev.map((m) =>
m.id === event.data.id ? { ...m, type: "deleted" as const } : m,
),
);
break;
}
});
wsRef.current = ws;
ws.addEventListener("open", () => setWsStatus("connected"));
ws.addEventListener("close", () => setWsStatus("disconnected"));
ws.addEventListener("error", () => setWsStatus("error"));
return () => {
cancelled = true;
ws.close();
wsRef.current = null;
};
}, []);
const handleReanalyze = async (id: string) => {
// Optimistic update
setMessages((prev) =>
prev.map((m) =>
m.id === id
? { ...m, ai_status: "pending" as const, ai_error: null, ai_analysis: null }
: m,
),
);
try {
await reanalyzeMessage(id);
} catch (err) {
console.error("Reanalyze failed:", err);
// Revert optimistic update on failure
setMessages((prev) =>
prev.map((m) =>
m.id === id ? { ...m, ai_status: "error" as const, ai_error: "Reanalyze failed" } : m,
),
);
}
};
return (
<div className="app">
<div className="sidebar">
<div className="sidebar-header">Moderation</div>
<div className="sidebar-placeholder">Channels placeholder</div>
</div>
<div className="main">
<div className="header">
<h1>Discord Moderation Dashboard</h1>
<span className="ws-status" data-status={wsStatus}>
{wsStatus}
</span>
</div>
<div className="content">
<MessageFeed messages={messages} onReanalyze={handleReanalyze} />
</div>
<ReviewPanel messages={messages} onReanalyze={handleReanalyze} />
</div>
</div>
);
}

View File

@@ -0,0 +1,92 @@
export type AIStatus = "pending" | "clean" | "warn" | "flagged" | "error";
export interface MessageRecord {
id: string;
guild_id: string;
channel_id: string;
thread_id: string | null;
user_id: string;
username: string;
avatar_url: string | null;
content: string;
edited_content: string | null;
created_at: number;
edited_at: number | null;
deleted_at: number | null;
type: "text" | "edited" | "deleted";
metadata: string | null;
ai_status?: AIStatus | null;
ai_moderation_flags?: string | null;
ai_moderation_score?: number | null;
ai_moderation_raw?: string | null;
ai_analysis?: string | null;
ai_analyzed_at?: number | null;
ai_error?: string | null;
}
export interface PageResult<T> {
data: T[];
nextCursor: string | null;
}
export type DashboardMessage = MessageRecord;
export interface Guild {
id: string;
name: string;
icon: string | null;
}
class ApiError extends Error {
code: string;
statusCode: number;
constructor(code: string, message: string, statusCode: number) {
super(message);
this.name = "ApiError";
this.code = code;
this.statusCode = statusCode;
}
}
async function request<T>(path: string, init?: RequestInit): Promise<T> {
const res = await fetch(path, {
headers: { "Content-Type": "application/json" },
...init,
});
if (!res.ok) {
let message = res.statusText;
let code = "REQUEST_FAILED";
try {
const body = (await res.json()) as { error?: string; message?: string };
if (body.message) message = body.message;
if (body.error) code = body.error;
} catch {
// ignore parse errors
}
throw new ApiError(code, message, res.status);
}
return res.json() as Promise<T>;
}
export async function listMessages(
params: URLSearchParams,
): Promise<PageResult<MessageRecord>> {
return request<PageResult<MessageRecord>>(`/api/messages?${params}`);
}
export async function listReview(
params: URLSearchParams,
): Promise<PageResult<MessageRecord>> {
return request<PageResult<MessageRecord>>(`/api/review?${params}`);
}
export async function reanalyzeMessage(id: string): Promise<void> {
await request<void>(`/api/messages/${id}/reanalyze`, { method: "POST" });
}
export async function getGuilds(): Promise<Guild[]> {
return request<Guild[]>("/api/guilds");
}

View File

@@ -0,0 +1,74 @@
import type { MessageRecord } from "../../api/client";
export interface MessageCardProps {
message: MessageRecord;
onReanalyze: (id: string) => void;
}
const STATUS_COLORS: Record<string, string> = {
pending: "#f9e2af",
clean: "#a6e3a1",
warn: "#fab387",
flagged: "#f38ba8",
error: "#f38ba8",
};
export function MessageCard({ message, onReanalyze }: MessageCardProps) {
const displayContent = message.edited_content ?? message.content;
const aiStatus = message.ai_status ?? "pending";
const statusColor = STATUS_COLORS[aiStatus] ?? "#6c7086";
return (
<div className={`message-card type-${message.type}`}>
<img
src={message.avatar_url ?? "/default-avatar.png"}
alt={message.username}
className="message-card-avatar"
width={32}
height={32}
/>
<div className="message-card-body">
<div className="message-card-meta">
<span className="message-card-username">{message.username}</span>
<span className="message-card-time">
{new Date(message.created_at).toLocaleString()}
</span>
{message.type === "edited" && (
<span className="badge badge-edited">edited</span>
)}
{message.type === "deleted" && (
<span className="badge badge-deleted">deleted</span>
)}
<span
className="badge badge-ai"
style={{ backgroundColor: statusColor }}
title={`AI: ${aiStatus}`}
>
{aiStatus}
</span>
</div>
<p className="message-card-content">{displayContent}</p>
{message.ai_analysis && (
<div className="message-card-analysis">{message.ai_analysis}</div>
)}
{message.ai_error && (
<div className="message-card-error">{message.ai_error}</div>
)}
<div className="message-card-actions">
<button
type="button"
className="btn-reanalyze"
onClick={() => onReanalyze(message.id)}
disabled={aiStatus === "pending"}
>
Reanalyze
</button>
</div>
</div>
</div>
);
}

View File

@@ -0,0 +1,25 @@
import type { MessageRecord } from "../../api/client";
import { MessageCard } from "./MessageCard";
export interface MessageFeedProps {
messages: MessageRecord[];
onReanalyze: (id: string) => void;
}
export function MessageFeed({ messages, onReanalyze }: MessageFeedProps) {
if (messages.length === 0) {
return (
<div className="empty-state">
<p>No messages yet</p>
</div>
);
}
return (
<div className="message-feed">
{messages.map((msg) => (
<MessageCard key={msg.id} message={msg} onReanalyze={onReanalyze} />
))}
</div>
);
}

View File

@@ -0,0 +1,37 @@
import type { MessageRecord } from "../../api/client";
import { MessageCard } from "../messages/MessageCard";
export interface ReviewPanelProps {
messages: MessageRecord[];
onReanalyze: (id: string) => void;
}
export function ReviewPanel({ messages, onReanalyze }: ReviewPanelProps) {
const reviewItems = messages.filter(
(m) =>
m.ai_status === "warn" ||
m.ai_status === "flagged" ||
m.ai_status === "error",
);
return (
<div className="review-panel">
<div className="review-header">
<h2>Needs Review</h2>
<span className="review-count">{reviewItems.length}</span>
</div>
{reviewItems.length === 0 ? (
<div className="empty-state">
<p>No items to review</p>
</div>
) : (
<div className="review-list">
{reviewItems.map((msg) => (
<MessageCard key={msg.id} message={msg} onReanalyze={onReanalyze} />
))}
</div>
)}
</div>
);
}

10
frontend/src/main.tsx Normal file
View File

@@ -0,0 +1,10 @@
import React from "react";
import ReactDOM from "react-dom/client";
import App from "./App";
import "./styles.css";
ReactDOM.createRoot(document.getElementById("root")!).render(
<React.StrictMode>
<App />
</React.StrictMode>
);

341
frontend/src/styles.css Normal file
View File

@@ -0,0 +1,341 @@
* {
box-sizing: border-box;
margin: 0;
padding: 0;
}
body {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
background: #1e1e2e;
color: #cdd6f4;
height: 100vh;
overflow: hidden;
}
#root {
height: 100%;
}
.app {
display: flex;
height: 100vh;
}
.sidebar {
width: 220px;
background: #181825;
border-right: 1px solid #313244;
display: flex;
flex-direction: column;
flex-shrink: 0;
}
.sidebar-header {
padding: 16px;
font-size: 14px;
font-weight: 700;
color: #cdd6f4;
border-bottom: 1px solid #313244;
}
.sidebar-placeholder {
padding: 16px;
font-size: 12px;
color: #6c7086;
flex: 1;
}
.main {
flex: 1;
display: flex;
flex-direction: column;
overflow: hidden;
}
.header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 12px 20px;
border-bottom: 1px solid #313244;
background: #181825;
flex-shrink: 0;
}
.header h1 {
font-size: 16px;
font-weight: 600;
color: #cdd6f4;
}
.ws-status {
font-size: 11px;
padding: 3px 8px;
border-radius: 4px;
font-weight: 500;
}
.ws-status[data-status="connected"] {
background: #a6e3a1;
color: #1e1e2e;
}
.ws-status[data-status="disconnected"],
.ws-status[data-status="error"] {
background: #f38ba8;
color: #1e1e2e;
}
.ws-status[data-status="connecting"] {
background: #f9e2af;
color: #1e1e2e;
}
.content {
flex: 1;
overflow-y: auto;
padding: 16px;
}
.message-list {
display: flex;
flex-direction: column;
gap: 8px;
}
.empty-state {
text-align: center;
color: #6c7086;
padding: 40px;
}
.message-item {
display: flex;
gap: 12px;
padding: 10px 12px;
background: #181825;
border-radius: 6px;
border: 1px solid #313244;
}
.message-item.type-deleted {
opacity: 0.6;
}
.message-avatar {
width: 32px;
height: 32px;
border-radius: 50%;
flex-shrink: 0;
}
.message-body {
flex: 1;
min-width: 0;
}
.message-username {
font-weight: 600;
color: #89b4fa;
margin-right: 8px;
}
.message-time {
font-size: 11px;
color: #6c7086;
}
.message-deleted {
font-size: 11px;
color: #f38ba8;
margin-left: 8px;
font-style: italic;
}
.message-content {
margin-top: 4px;
font-size: 13px;
color: #bac2de;
white-space: pre-wrap;
word-break: break-word;
}
.review-panel {
border-top: 1px solid #313244;
background: #181825;
flex-shrink: 0;
}
/* Message Card */
.message-card {
display: flex;
gap: 12px;
padding: 12px;
background: #181825;
border-radius: 6px;
border: 1px solid #313244;
}
.message-card.type-deleted {
opacity: 0.6;
}
.message-card-avatar {
width: 32px;
height: 32px;
border-radius: 50%;
flex-shrink: 0;
}
.message-card-body {
flex: 1;
min-width: 0;
}
.message-card-meta {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 8px;
}
.message-card-username {
font-weight: 600;
color: #89b4fa;
}
.message-card-time {
font-size: 11px;
color: #6c7086;
}
.badge {
font-size: 10px;
padding: 2px 6px;
border-radius: 4px;
font-weight: 500;
color: #1e1e2e;
}
.badge-edited {
background: #fab387;
color: #1e1e2e;
}
.badge-deleted {
background: #f38ba8;
color: #1e1e2e;
}
.message-card-content {
margin-top: 6px;
font-size: 13px;
color: #bac2de;
white-space: pre-wrap;
word-break: break-word;
}
.message-card-analysis {
margin-top: 8px;
padding: 8px;
background: #1e1e2e;
border-radius: 4px;
font-size: 12px;
color: #a6e3a1;
white-space: pre-wrap;
}
.message-card-error {
margin-top: 8px;
padding: 8px;
background: #1e1e2e;
border-radius: 4px;
font-size: 12px;
color: #f38ba8;
}
.message-card-actions {
margin-top: 8px;
}
.btn-reanalyze {
padding: 4px 10px;
font-size: 11px;
border: none;
border-radius: 4px;
background: #313244;
color: #cdd6f4;
cursor: pointer;
}
.btn-reanalyze:hover:not(:disabled) {
background: #45475a;
}
.btn-reanalyze:disabled {
opacity: 0.5;
cursor: not-allowed;
}
/* Message Feed */
.message-feed {
display: flex;
flex-direction: column;
gap: 8px;
}
/* Review Panel */
.review-panel {
border-top: 1px solid #313244;
background: #181825;
flex-shrink: 0;
max-height: 250px;
overflow-y: auto;
}
.review-header {
display: flex;
align-items: center;
gap: 8px;
padding: 12px 16px;
border-bottom: 1px solid #313244;
}
.review-header h2 {
font-size: 13px;
font-weight: 600;
color: #cdd6f4;
}
.review-count {
font-size: 11px;
padding: 2px 6px;
border-radius: 10px;
background: #f38ba8;
color: #1e1e2e;
font-weight: 600;
}
.review-list {
display: flex;
flex-direction: column;
gap: 8px;
padding: 12px;
}
.review-placeholder {
padding: 16px;
font-size: 12px;
color: #6c7086;
}
.review-panel .empty-state {
padding: 16px;
}
/* Keep existing styles */
.review-placeholder {
padding: 16px;
font-size: 12px;
color: #6c7086;
}

32
frontend/src/ws/client.ts Normal file
View File

@@ -0,0 +1,32 @@
import type { MessageRecord } from "../api/client";
export type DashboardEvent =
| { type: "message_created"; data: MessageRecord }
| { type: "message_updated"; data: Partial<MessageRecord> & { id: string } }
| { type: "message_deleted"; data: { id: string; deleted_at: number } }
| { type: "message_analyzed"; data: MessageRecord }
| { type: "analysis_queue_status"; data: unknown }
| { type: "ui_state"; state: unknown }
| { type: "user_state"; users: unknown[] };
export function connectDashboardSocket(
onEvent: (event: DashboardEvent) => void,
): WebSocket {
const protocol = window.location.protocol === "https:" ? "wss:" : "ws:";
const url = `${protocol}//${window.location.host}/ws`;
const ws = new WebSocket(url);
ws.addEventListener("message", (evt) => {
if (typeof evt.data === "string") {
try {
const event = JSON.parse(evt.data) as DashboardEvent;
onEvent(event);
} catch {
// ignore malformed JSON
}
}
// Binary frames (PCM audio) are ignored for now
});
return ws;
}

16
frontend/tsconfig.json Normal file
View File

@@ -0,0 +1,16 @@
{
"compilerOptions": {
"target": "ESNext",
"module": "ESNext",
"moduleResolution": "bundler",
"strict": true,
"jsx": "react-jsx",
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"types": ["vite/client"],
"noEmit": true,
"skipLibCheck": true,
"esModuleInterop": true,
"allowSyntheticDefaultImports": true
},
"include": ["src/**/*", "vite.config.ts"]
}

9
frontend/vite.config.ts Normal file
View File

@@ -0,0 +1,9 @@
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";
export default defineConfig({
plugins: [react()],
server: {
middlewareMode: false,
},
});

View File

@@ -6,8 +6,11 @@
"packageManager": "pnpm@10.25.0", "packageManager": "pnpm@10.25.0",
"scripts": { "scripts": {
"dev": "tsx watch src/index.ts", "dev": "tsx watch src/index.ts",
"dev:server": "tsx watch src/index.ts",
"dev:web": "vite --host 0.0.0.0 frontend",
"start": "tsx src/index.ts", "start": "tsx src/index.ts",
"build": "tsc --outDir dist", "build": "pnpm run build:web && tsc --outDir dist",
"build:web": "vite build frontend --outDir ../public/app --emptyOutDir",
"typecheck": "tsc --noEmit", "typecheck": "tsc --noEmit",
"lint": "biome check --diagnostic-level=error .", "lint": "biome check --diagnostic-level=error .",
"format": "biome format --write .", "format": "biome format --write .",
@@ -22,6 +25,7 @@
"@discordjs/voice": "^0.19.1", "@discordjs/voice": "^0.19.1",
"@snazzah/davey": "^0.1.10", "@snazzah/davey": "^0.1.10",
"@types/pg": "^8.20.0", "@types/pg": "^8.20.0",
"@vitejs/plugin-react": "^6.0.1",
"better-sqlite3": "^12.10.0", "better-sqlite3": "^12.10.0",
"class-transformer": "^0.5.1", "class-transformer": "^0.5.1",
"class-validator": "^0.15.1", "class-validator": "^0.15.1",
@@ -38,7 +42,10 @@
"pino-http": "^11.0.0", "pino-http": "^11.0.0",
"prism-media": "2.0.0-alpha.0", "prism-media": "2.0.0-alpha.0",
"prom-client": "^15.1.3", "prom-client": "^15.1.3",
"react": "^19.2.6",
"react-dom": "^19.2.6",
"sodium-native": "^4.3.2", "sodium-native": "^4.3.2",
"vite": "^8.0.13",
"ws": "^8.20.1", "ws": "^8.20.1",
"zod": "^4.4.3" "zod": "^4.4.3"
}, },
@@ -48,6 +55,8 @@
"@types/express": "^5.0.6", "@types/express": "^5.0.6",
"@types/fluent-ffmpeg": "^2.1.28", "@types/fluent-ffmpeg": "^2.1.28",
"@types/node": "^24.10.1", "@types/node": "^24.10.1",
"@types/react": "^19.2.14",
"@types/react-dom": "^19.2.3",
"@types/ws": "^8.18.1", "@types/ws": "^8.18.1",
"drizzle-kit": "^0.31.10", "drizzle-kit": "^0.31.10",
"pino-pretty": "^10.3.1", "pino-pretty": "^10.3.1",

237
pnpm-lock.yaml generated
View File

@@ -20,6 +20,9 @@ importers:
'@types/pg': '@types/pg':
specifier: ^8.20.0 specifier: ^8.20.0
version: 8.20.0 version: 8.20.0
'@vitejs/plugin-react':
specifier: ^6.0.1
version: 6.0.1(vite@8.0.13(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0))
better-sqlite3: better-sqlite3:
specifier: ^12.10.0 specifier: ^12.10.0
version: 12.10.0 version: 12.10.0
@@ -68,9 +71,18 @@ importers:
prom-client: prom-client:
specifier: ^15.1.3 specifier: ^15.1.3
version: 15.1.3 version: 15.1.3
react:
specifier: ^19.2.6
version: 19.2.6
react-dom:
specifier: ^19.2.6
version: 19.2.6(react@19.2.6)
sodium-native: sodium-native:
specifier: ^4.3.2 specifier: ^4.3.2
version: 4.3.3 version: 4.3.3
vite:
specifier: ^8.0.13
version: 8.0.13(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0)
ws: ws:
specifier: ^8.20.1 specifier: ^8.20.1
version: 8.20.1 version: 8.20.1
@@ -93,6 +105,12 @@ importers:
'@types/node': '@types/node':
specifier: ^24.10.1 specifier: ^24.10.1
version: 24.12.4 version: 24.12.4
'@types/react':
specifier: ^19.2.14
version: 19.2.14
'@types/react-dom':
specifier: ^19.2.3
version: 19.2.3(@types/react@19.2.14)
'@types/ws': '@types/ws':
specifier: ^8.18.1 specifier: ^8.18.1
version: 8.18.1 version: 8.18.1
@@ -107,7 +125,7 @@ importers:
version: 4.21.0 version: 4.21.0
vitest: vitest:
specifier: latest specifier: latest
version: 4.1.6(@opentelemetry/api@1.9.1)(@types/node@24.12.4)(vite@8.0.12(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0)) version: 4.1.6(@opentelemetry/api@1.9.1)(@types/node@24.12.4)(vite@8.0.13(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0))
packages: packages:
@@ -690,97 +708,97 @@ packages:
'@otplib/preset-v11@12.0.1': '@otplib/preset-v11@12.0.1':
resolution: {integrity: sha512-9hSetMI7ECqbFiKICrNa4w70deTUfArtwXykPUvSHWOdzOlfa9ajglu7mNCntlvxycTiOAXkQGwjQCzzDEMRMg==} resolution: {integrity: sha512-9hSetMI7ECqbFiKICrNa4w70deTUfArtwXykPUvSHWOdzOlfa9ajglu7mNCntlvxycTiOAXkQGwjQCzzDEMRMg==}
'@oxc-project/types@0.129.0': '@oxc-project/types@0.130.0':
resolution: {integrity: sha512-3oz8m3FGdr2nDXVqmFUw7jolKliC4MoyXYIG2c7gpjBnzUWQpUGIYcXYKxTdTi+N2jusvt610ckTMkxdwHkYEg==} resolution: {integrity: sha512-ibD2usx9JRu7f5pu2tMKMI4cpA4NgXJQoYRP4pQ7Pxmn1l6k/53qWtQWZayhYy3X4QZkt90Ot+mJEaeXouio6Q==}
'@pinojs/redact@0.4.0': '@pinojs/redact@0.4.0':
resolution: {integrity: sha512-k2ENnmBugE/rzQfEcdWHcCY+/FM3VLzH9cYEsbdsoqrvzAKRhUZeRNhAZvB8OitQJ1TBed3yqWtdjzS6wJKBwg==} resolution: {integrity: sha512-k2ENnmBugE/rzQfEcdWHcCY+/FM3VLzH9cYEsbdsoqrvzAKRhUZeRNhAZvB8OitQJ1TBed3yqWtdjzS6wJKBwg==}
'@rolldown/binding-android-arm64@1.0.0': '@rolldown/binding-android-arm64@1.0.1':
resolution: {integrity: sha512-TWMZnRLMe63C2Lhyicviu7ZHaU4kxa6PS3rofvc9GmcvptzNN11BcfQ4Sl7MwTOsisQoa2keB/EBdNCAnUo8vA==} resolution: {integrity: sha512-fJI3I0r3C3Oj/zdBCpaCmBRZYf07xpaq4yCfDDoSFm+beWNzbIl26puW8RraUdugoJw/95zerNOn6jasAhzSmg==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64] cpu: [arm64]
os: [android] os: [android]
'@rolldown/binding-darwin-arm64@1.0.0': '@rolldown/binding-darwin-arm64@1.0.1':
resolution: {integrity: sha512-6XcD+8k0gPVItNagEw78/qqcBDwKcwDYS8V2hRmVsfUSIrd8cWe/CBvRDI5toqFyPfj+FJr6t8U6Xj2P2prEew==} resolution: {integrity: sha512-cKnAhWEsV7TPcA/5EAteDp6KcJZBQ2G+BqE7zayMMi7kMvwRsbv7WT9aOnn0WNl4SKEIf43vjS31iUPu80nzXg==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64] cpu: [arm64]
os: [darwin] os: [darwin]
'@rolldown/binding-darwin-x64@1.0.0': '@rolldown/binding-darwin-x64@1.0.1':
resolution: {integrity: sha512-iN/tWVXRQDWvmZlKdceP1Dwug9GDpEymhb9p4xnEe6zvCg5lFmzVljl+1qR1NVx3yfGpr2Na+CuLmv5IU8uzfQ==} resolution: {integrity: sha512-YKrVwQjIRBPo+5G/u03wGjbdy4q7pyzCe93DK9VJ7zkVmeg8LJ7GbgsiHWdR4xSoe4CAXRD7Bcjgbtr64bkXNg==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64] cpu: [x64]
os: [darwin] os: [darwin]
'@rolldown/binding-freebsd-x64@1.0.0': '@rolldown/binding-freebsd-x64@1.0.1':
resolution: {integrity: sha512-jjQMDvvwSOuhOwMszD/klSOjyWMM3zI64hWTj9KT5x4MxRbZAf+7vLQ6qouRhtsLVFHr3f0ILaJAfgENPiQdAQ==} resolution: {integrity: sha512-z/oBsREo46SsFqBwYtFe0kpJeBijAT48O/WXLI4suiCLBkr03RTtTJMCzSdDd2znlh8VJizL09XVkQgk8IZonw==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64] cpu: [x64]
os: [freebsd] os: [freebsd]
'@rolldown/binding-linux-arm-gnueabihf@1.0.0': '@rolldown/binding-linux-arm-gnueabihf@1.0.1':
resolution: {integrity: sha512-d//Dtg2x6/m3mbV64yUGNnDGNZaDGRpDLLNGerHQUVObuNaIQaaDp25yUiqGXtHEXX+NP2d0wAlmKgpYgIAJ2A==} resolution: {integrity: sha512-ik8q7GM11zxvYxFc2PeDcT6TBvhCQMaUxfph/M5l9sKuTs/Sjg3L+Byw0F7w0ZVLBZmx30P+gG0ECzzN+MFcmQ==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm] cpu: [arm]
os: [linux] os: [linux]
'@rolldown/binding-linux-arm64-gnu@1.0.0': '@rolldown/binding-linux-arm64-gnu@1.0.1':
resolution: {integrity: sha512-n7Ofp0mx+aB2cC+Sdy5YtMnXtY9lchnHbY+3Yt0uq9JsWQExf4f5Whu0tK0R8Jdc9S6RchTHjIFY7uc92puOVQ==} resolution: {integrity: sha512-QoSx2EkyrrdZ6kcyE8stqZ62t0Yra8Fs5ia9lOxJrh6TMQJK7gQKmscdTHf7pOXKREKrVwOtJcQG3qVSfc866A==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64] cpu: [arm64]
os: [linux] os: [linux]
'@rolldown/binding-linux-arm64-musl@1.0.0': '@rolldown/binding-linux-arm64-musl@1.0.1':
resolution: {integrity: sha512-EIVjy2cgd7uuMMo94FVkBp7F6DhcZAUwNURkSG3RwUmvAXR6s0ISxM81U+IydcZByPG0pZIHsf1b6kTxoFDgJA==} resolution: {integrity: sha512-uwNwFpwKeNiZawfAWBgg0VIztPTV3ihhh1vV334h9ivnNLorxnQMU6Fz8wG1Zb4Qh9LC1/MkcyT3YlDXG3Rsgg==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64] cpu: [arm64]
os: [linux] os: [linux]
'@rolldown/binding-linux-ppc64-gnu@1.0.0': '@rolldown/binding-linux-ppc64-gnu@1.0.1':
resolution: {integrity: sha512-JEwwOPcwTLAcpDQlqSmjEmfs63xJnSiUNIGvLcDLUHCWK4XowpS/7c7tUsUH6uT/ct6bMUTdXKfI8967FYj6mg==} resolution: {integrity: sha512-zY1bul7OWr7DFBiJ++wofXvnr8B45ce3QsQUhKrIhXsygAh7bTkwyeM1bi1a2g5C/yC/N8TZyGDEoMfm/l9mpg==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [ppc64] cpu: [ppc64]
os: [linux] os: [linux]
'@rolldown/binding-linux-s390x-gnu@1.0.0': '@rolldown/binding-linux-s390x-gnu@1.0.1':
resolution: {integrity: sha512-0wjCFhLrihtAubnT9iA0N++0pSV0z5Hg7tNGdNJ4RFaINceHadoF+kiFGyY1qSSNVIAZtLotG8Ju1bgDPkjnFA==} resolution: {integrity: sha512-0frlsT/f4Ft6I7SMESTKnF3cZsdicQn1dCMkF/jT9wDLE+gGoiQfv1nmT9e+s7s/fekvvy6tZM2jHvI2tkbJDQ==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [s390x] cpu: [s390x]
os: [linux] os: [linux]
'@rolldown/binding-linux-x64-gnu@1.0.0': '@rolldown/binding-linux-x64-gnu@1.0.1':
resolution: {integrity: sha512-Dfn7iak9BcMMePxcoJfpSbWqnEyrp/dRF63/8qW/eHBdOZov6x5aShLLEYGYdIeSJ6vMLK/XCVB+lGIxm41bQA==} resolution: {integrity: sha512-XABVmGp9Tg0WspTVvwduTc4fpqy6JnAUrSQe6OuyqD/03nI7r0O9OWUkMIwFrjKAIqolvqoA4ZrJppgwE0Gxmw==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64] cpu: [x64]
os: [linux] os: [linux]
'@rolldown/binding-linux-x64-musl@1.0.0': '@rolldown/binding-linux-x64-musl@1.0.1':
resolution: {integrity: sha512-5/utzzDmD/pD/bmuaUcbTf/sZYy0aztwIVlfpoW1fTjCZ0BaPOMVWGZL1zvgxyi7ZIVYWlxKONHmSbHuiOh8Jw==} resolution: {integrity: sha512-bV4fzswuzVcKD90o/VM6QqKxnxlDq0g2BISDLNVmxrnhpv1DDbyPhCIjYfvzYLV+MvkKKnQt2Q6AO86SEBULUQ==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64] cpu: [x64]
os: [linux] os: [linux]
'@rolldown/binding-openharmony-arm64@1.0.0': '@rolldown/binding-openharmony-arm64@1.0.1':
resolution: {integrity: sha512-ouJs8VcUomfLfpbUECqFMRqdV4x6aeAK3MA4m6vTrJJjKyWTV5KnxZx7Jd9G+GlDaQQxubcba00x16OyJ1meig==} resolution: {integrity: sha512-/Mh0Zhq3OP7fVs0kcQHZP6lZEthMGTaSf8UBQYSFEZDWGXXlEC+nJ6EqenaK2t4LBXMe3A+K/G2BVXXdtOr4PQ==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64] cpu: [arm64]
os: [openharmony] os: [openharmony]
'@rolldown/binding-wasm32-wasi@1.0.0': '@rolldown/binding-wasm32-wasi@1.0.1':
resolution: {integrity: sha512-E+oHKGiDA+lsKMmFtffDDw91EryDT7uJocrIuCHqhm6bCTM6xFK+3gaCkYOHfPwQr0cCNarSM2xaELoQDz9jJg==} resolution: {integrity: sha512-+1xc9X45l8ufsBAm6Gjvx2qDRIY9lTVt0cgWNcJ+1gdhXvkbxePA60yRTwSTuXL09CMhyJmjpV7E3NoyxbqFQQ==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [wasm32] cpu: [wasm32]
'@rolldown/binding-win32-arm64-msvc@1.0.0': '@rolldown/binding-win32-arm64-msvc@1.0.1':
resolution: {integrity: sha512-yYK02n8Rngo+gbm1y6G0+7jk1sJ/2Wt7K0me0Y7k/ErBpyf+LJ2gFpqWVTcRV1rUepBlQRmpgWkTQCiiwrK0Ow==} resolution: {integrity: sha512-1D+UqZdfnuR+Jy1GgMJwi85bD40H21uNmOPRWQhw4oRSuolZ/B5rixZ45DK2KXOTCvmVCecauWgEhbw8bI7tOw==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64] cpu: [arm64]
os: [win32] os: [win32]
'@rolldown/binding-win32-x64-msvc@1.0.0': '@rolldown/binding-win32-x64-msvc@1.0.1':
resolution: {integrity: sha512-14bpChMahXRRXiTwahSl+zzHPW6qQTXtkMuJBFlbo+pqSAews2d4BdCSHfrJ/MBsCZtpmTafsY+1QhBzitcmdg==} resolution: {integrity: sha512-INAycaWuhlOK3wk4mRHGsdgwYWmd9cChdPdE9bwWmy6rn9VqVNYNFGhOdXrofXUxwHIncSiPNb8tNm8knDVIeQ==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64] cpu: [x64]
os: [win32] os: [win32]
@@ -788,6 +806,9 @@ packages:
'@rolldown/pluginutils@1.0.0': '@rolldown/pluginutils@1.0.0':
resolution: {integrity: sha512-aKs/3GSWyV0mrhNmt/96/Z3yczC3yvrzYATCiCXQebBsGyYzjNdUphRVLeJQ67ySKVXRfMxt2lm12pmXvbPFQQ==} resolution: {integrity: sha512-aKs/3GSWyV0mrhNmt/96/Z3yczC3yvrzYATCiCXQebBsGyYzjNdUphRVLeJQ67ySKVXRfMxt2lm12pmXvbPFQQ==}
'@rolldown/pluginutils@1.0.0-rc.7':
resolution: {integrity: sha512-qujRfC8sFVInYSPPMLQByRh7zhwkGFS4+tyMQ83srV1qrxL4g8E2tyxVVyxd0+8QeBM1mIk9KbWxkegRr76XzA==}
'@sapphire/async-queue@1.5.5': '@sapphire/async-queue@1.5.5':
resolution: {integrity: sha512-cvGzxbba6sav2zZkH8GPf2oGk9yYoD5qrNWdu9fRehifgnFZJMV+nuy2nON2roRO4yQQ+v7MK/Pktl/HgfsUXg==} resolution: {integrity: sha512-cvGzxbba6sav2zZkH8GPf2oGk9yYoD5qrNWdu9fRehifgnFZJMV+nuy2nON2roRO4yQQ+v7MK/Pktl/HgfsUXg==}
engines: {node: '>=v14.0.0', npm: '>=7.0.0'} engines: {node: '>=v14.0.0', npm: '>=7.0.0'}
@@ -934,6 +955,14 @@ packages:
'@types/range-parser@1.2.7': '@types/range-parser@1.2.7':
resolution: {integrity: sha512-hKormJbkJqzQGhziax5PItDUTMAM9uE2XXQmM37dyd4hVM+5aVl7oVxMVUiVQn2oCQFN/LKCZdvSM0pFRqbSmQ==} resolution: {integrity: sha512-hKormJbkJqzQGhziax5PItDUTMAM9uE2XXQmM37dyd4hVM+5aVl7oVxMVUiVQn2oCQFN/LKCZdvSM0pFRqbSmQ==}
'@types/react-dom@19.2.3':
resolution: {integrity: sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ==}
peerDependencies:
'@types/react': ^19.2.0
'@types/react@19.2.14':
resolution: {integrity: sha512-ilcTH/UniCkMdtexkoCN0bI7pMcJDvmQFPvuPvmEaYA/NSfFTAgdUSLAoVjaRJm7+6PvcM+q1zYOwS4wTYMF9w==}
'@types/retry@0.12.2': '@types/retry@0.12.2':
resolution: {integrity: sha512-XISRgDJ2Tc5q4TRqvgJtzsRkFYNJzZrhTdtMoGVBttwzzQJkPnS3WWTFc7kuDRoPtPakl+T+OfdEUjYJj7Jbow==} resolution: {integrity: sha512-XISRgDJ2Tc5q4TRqvgJtzsRkFYNJzZrhTdtMoGVBttwzzQJkPnS3WWTFc7kuDRoPtPakl+T+OfdEUjYJj7Jbow==}
@@ -949,6 +978,19 @@ packages:
'@types/ws@8.18.1': '@types/ws@8.18.1':
resolution: {integrity: sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg==} resolution: {integrity: sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg==}
'@vitejs/plugin-react@6.0.1':
resolution: {integrity: sha512-l9X/E3cDb+xY3SWzlG1MOGt2usfEHGMNIaegaUGFsLkb3RCn/k8/TOXBcab+OndDI4TBtktT8/9BwwW8Vi9KUQ==}
engines: {node: ^20.19.0 || >=22.12.0}
peerDependencies:
'@rolldown/plugin-babel': ^0.1.7 || ^0.2.0
babel-plugin-react-compiler: ^1.0.0
vite: ^8.0.0
peerDependenciesMeta:
'@rolldown/plugin-babel':
optional: true
babel-plugin-react-compiler:
optional: true
'@vitest/expect@4.1.6': '@vitest/expect@4.1.6':
resolution: {integrity: sha512-7EHDquPthALSV0jhhjgEW8FXaviMx7rSqu8W6oqCoAuOhKov814P99QDV1pxMA3QPv21YudvJngIhjrNI4opLg==} resolution: {integrity: sha512-7EHDquPthALSV0jhhjgEW8FXaviMx7rSqu8W6oqCoAuOhKov814P99QDV1pxMA3QPv21YudvJngIhjrNI4opLg==}
@@ -1164,6 +1206,9 @@ packages:
resolution: {integrity: sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w==} resolution: {integrity: sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w==}
engines: {node: '>= 0.6'} engines: {node: '>= 0.6'}
csstype@3.2.3:
resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==}
dateformat@4.6.3: dateformat@4.6.3:
resolution: {integrity: sha512-2P0p0pFGzHS5EMnhdxQi7aJN+iMheud0UhG4dlE1DLAlvL8JHjJJTX/CSm4JXwV0Ka5nGk3zC5mcb5bUQUxxMA==} resolution: {integrity: sha512-2P0p0pFGzHS5EMnhdxQi7aJN+iMheud0UhG4dlE1DLAlvL8JHjJJTX/CSm4JXwV0Ka5nGk3zC5mcb5bUQUxxMA==}
@@ -1967,6 +2012,15 @@ packages:
resolution: {integrity: sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==} resolution: {integrity: sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==}
hasBin: true hasBin: true
react-dom@19.2.6:
resolution: {integrity: sha512-0prMI+hvBbPjsWnxDLxlCGyM8PN6UuWjEUCYmZhO67xIV9Xasa/r/vDnq+Xyq4Lo27g8QSbO5YzARu0D1Sps3g==}
peerDependencies:
react: ^19.2.6
react@19.2.6:
resolution: {integrity: sha512-sfWGGfavi0xr8Pg0sVsyHMAOziVYKgPLNrS7ig+ivMNb3wbCBw3KxtflsGBAwD3gYQlE/AEZsTLgToRrSCjb0Q==}
engines: {node: '>=0.10.0'}
readable-stream@3.6.2: readable-stream@3.6.2:
resolution: {integrity: sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==} resolution: {integrity: sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==}
engines: {node: '>= 6'} engines: {node: '>= 6'}
@@ -2005,8 +2059,8 @@ packages:
deprecated: Rimraf versions prior to v4 are no longer supported deprecated: Rimraf versions prior to v4 are no longer supported
hasBin: true hasBin: true
rolldown@1.0.0: rolldown@1.0.1:
resolution: {integrity: sha512-yD986aXDESFGS95spT1LAv0jssywP4npMEjmMHyN2/5+eE8qQJUype2AaKkRiLgBgyD0LFlubwAht7VmY8rGoA==} resolution: {integrity: sha512-X0KQHljNnEkWNqqiz9zJrGunh1B0HgOxLXvnFpCOcadzcy5qohZ3tqMEUg00vncoRovXuK3ZqCT9KnnKzoInFQ==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
hasBin: true hasBin: true
@@ -2024,6 +2078,9 @@ packages:
safer-buffer@2.1.2: safer-buffer@2.1.2:
resolution: {integrity: sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==} resolution: {integrity: sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==}
scheduler@0.27.0:
resolution: {integrity: sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==}
secure-json-parse@2.7.0: secure-json-parse@2.7.0:
resolution: {integrity: sha512-6aU+Rwsezw7VR8/nyvKTx8QpWH9FrcYiXXlqC4z5d5XQBDRqtbfsRjnwGyqbi3gddNtWHuEk9OANUotL26qKUw==} resolution: {integrity: sha512-6aU+Rwsezw7VR8/nyvKTx8QpWH9FrcYiXXlqC4z5d5XQBDRqtbfsRjnwGyqbi3gddNtWHuEk9OANUotL26qKUw==}
@@ -2252,8 +2309,8 @@ packages:
resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==} resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==}
engines: {node: '>= 0.8'} engines: {node: '>= 0.8'}
vite@8.0.12: vite@8.0.13:
resolution: {integrity: sha512-w2dDofOWv2QB09ZITZBsvKTVAlYvPR4IAmrY/v0ir9KvLs0xybR7i48wxhM1/oyBWO34wPns+bPGw5ZrZqDpZg==} resolution: {integrity: sha512-MFtjBYgzmSxmgA4RAfjIyXWpGe1oALnjgUTzzV7QLx/TKxCzjtMH6Fd9/eVK+5Fg1qNoz5VAwsmMs/NofrmJvw==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
hasBin: true hasBin: true
peerDependencies: peerDependencies:
@@ -2785,61 +2842,63 @@ snapshots:
'@otplib/plugin-crypto': 12.0.1 '@otplib/plugin-crypto': 12.0.1
'@otplib/plugin-thirty-two': 12.0.1 '@otplib/plugin-thirty-two': 12.0.1
'@oxc-project/types@0.129.0': {} '@oxc-project/types@0.130.0': {}
'@pinojs/redact@0.4.0': {} '@pinojs/redact@0.4.0': {}
'@rolldown/binding-android-arm64@1.0.0': '@rolldown/binding-android-arm64@1.0.1':
optional: true optional: true
'@rolldown/binding-darwin-arm64@1.0.0': '@rolldown/binding-darwin-arm64@1.0.1':
optional: true optional: true
'@rolldown/binding-darwin-x64@1.0.0': '@rolldown/binding-darwin-x64@1.0.1':
optional: true optional: true
'@rolldown/binding-freebsd-x64@1.0.0': '@rolldown/binding-freebsd-x64@1.0.1':
optional: true optional: true
'@rolldown/binding-linux-arm-gnueabihf@1.0.0': '@rolldown/binding-linux-arm-gnueabihf@1.0.1':
optional: true optional: true
'@rolldown/binding-linux-arm64-gnu@1.0.0': '@rolldown/binding-linux-arm64-gnu@1.0.1':
optional: true optional: true
'@rolldown/binding-linux-arm64-musl@1.0.0': '@rolldown/binding-linux-arm64-musl@1.0.1':
optional: true optional: true
'@rolldown/binding-linux-ppc64-gnu@1.0.0': '@rolldown/binding-linux-ppc64-gnu@1.0.1':
optional: true optional: true
'@rolldown/binding-linux-s390x-gnu@1.0.0': '@rolldown/binding-linux-s390x-gnu@1.0.1':
optional: true optional: true
'@rolldown/binding-linux-x64-gnu@1.0.0': '@rolldown/binding-linux-x64-gnu@1.0.1':
optional: true optional: true
'@rolldown/binding-linux-x64-musl@1.0.0': '@rolldown/binding-linux-x64-musl@1.0.1':
optional: true optional: true
'@rolldown/binding-openharmony-arm64@1.0.0': '@rolldown/binding-openharmony-arm64@1.0.1':
optional: true optional: true
'@rolldown/binding-wasm32-wasi@1.0.0': '@rolldown/binding-wasm32-wasi@1.0.1':
dependencies: dependencies:
'@emnapi/core': 1.10.0 '@emnapi/core': 1.10.0
'@emnapi/runtime': 1.10.0 '@emnapi/runtime': 1.10.0
'@napi-rs/wasm-runtime': 1.1.4(@emnapi/core@1.10.0)(@emnapi/runtime@1.10.0) '@napi-rs/wasm-runtime': 1.1.4(@emnapi/core@1.10.0)(@emnapi/runtime@1.10.0)
optional: true optional: true
'@rolldown/binding-win32-arm64-msvc@1.0.0': '@rolldown/binding-win32-arm64-msvc@1.0.1':
optional: true optional: true
'@rolldown/binding-win32-x64-msvc@1.0.0': '@rolldown/binding-win32-x64-msvc@1.0.1':
optional: true optional: true
'@rolldown/pluginutils@1.0.0': {} '@rolldown/pluginutils@1.0.0': {}
'@rolldown/pluginutils@1.0.0-rc.7': {}
'@sapphire/async-queue@1.5.5': {} '@sapphire/async-queue@1.5.5': {}
'@sapphire/shapeshift@4.0.0': '@sapphire/shapeshift@4.0.0':
@@ -2978,6 +3037,14 @@ snapshots:
'@types/range-parser@1.2.7': {} '@types/range-parser@1.2.7': {}
'@types/react-dom@19.2.3(@types/react@19.2.14)':
dependencies:
'@types/react': 19.2.14
'@types/react@19.2.14':
dependencies:
csstype: 3.2.3
'@types/retry@0.12.2': {} '@types/retry@0.12.2': {}
'@types/send@1.2.1': '@types/send@1.2.1':
@@ -2995,6 +3062,11 @@ snapshots:
dependencies: dependencies:
'@types/node': 24.12.4 '@types/node': 24.12.4
'@vitejs/plugin-react@6.0.1(vite@8.0.13(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0))':
dependencies:
'@rolldown/pluginutils': 1.0.0-rc.7
vite: 8.0.13(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0)
'@vitest/expect@4.1.6': '@vitest/expect@4.1.6':
dependencies: dependencies:
'@standard-schema/spec': 1.1.0 '@standard-schema/spec': 1.1.0
@@ -3004,13 +3076,13 @@ snapshots:
chai: 6.2.2 chai: 6.2.2
tinyrainbow: 3.1.0 tinyrainbow: 3.1.0
'@vitest/mocker@4.1.6(vite@8.0.12(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0))': '@vitest/mocker@4.1.6(vite@8.0.13(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0))':
dependencies: dependencies:
'@vitest/spy': 4.1.6 '@vitest/spy': 4.1.6
estree-walker: 3.0.3 estree-walker: 3.0.3
magic-string: 0.30.21 magic-string: 0.30.21
optionalDependencies: optionalDependencies:
vite: 8.0.12(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0) vite: 8.0.13(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0)
'@vitest/pretty-format@4.1.6': '@vitest/pretty-format@4.1.6':
dependencies: dependencies:
@@ -3204,6 +3276,8 @@ snapshots:
cookie@0.7.2: {} cookie@0.7.2: {}
csstype@3.2.3: {}
dateformat@4.6.3: {} dateformat@4.6.3: {}
debug@4.4.3: debug@4.4.3:
@@ -3977,6 +4051,13 @@ snapshots:
minimist: 1.2.8 minimist: 1.2.8
strip-json-comments: 2.0.1 strip-json-comments: 2.0.1
react-dom@19.2.6(react@19.2.6):
dependencies:
react: 19.2.6
scheduler: 0.27.0
react@19.2.6: {}
readable-stream@3.6.2: readable-stream@3.6.2:
dependencies: dependencies:
inherits: 2.0.4 inherits: 2.0.4
@@ -4013,26 +4094,26 @@ snapshots:
dependencies: dependencies:
glob: 7.2.3 glob: 7.2.3
rolldown@1.0.0: rolldown@1.0.1:
dependencies: dependencies:
'@oxc-project/types': 0.129.0 '@oxc-project/types': 0.130.0
'@rolldown/pluginutils': 1.0.0 '@rolldown/pluginutils': 1.0.0
optionalDependencies: optionalDependencies:
'@rolldown/binding-android-arm64': 1.0.0 '@rolldown/binding-android-arm64': 1.0.1
'@rolldown/binding-darwin-arm64': 1.0.0 '@rolldown/binding-darwin-arm64': 1.0.1
'@rolldown/binding-darwin-x64': 1.0.0 '@rolldown/binding-darwin-x64': 1.0.1
'@rolldown/binding-freebsd-x64': 1.0.0 '@rolldown/binding-freebsd-x64': 1.0.1
'@rolldown/binding-linux-arm-gnueabihf': 1.0.0 '@rolldown/binding-linux-arm-gnueabihf': 1.0.1
'@rolldown/binding-linux-arm64-gnu': 1.0.0 '@rolldown/binding-linux-arm64-gnu': 1.0.1
'@rolldown/binding-linux-arm64-musl': 1.0.0 '@rolldown/binding-linux-arm64-musl': 1.0.1
'@rolldown/binding-linux-ppc64-gnu': 1.0.0 '@rolldown/binding-linux-ppc64-gnu': 1.0.1
'@rolldown/binding-linux-s390x-gnu': 1.0.0 '@rolldown/binding-linux-s390x-gnu': 1.0.1
'@rolldown/binding-linux-x64-gnu': 1.0.0 '@rolldown/binding-linux-x64-gnu': 1.0.1
'@rolldown/binding-linux-x64-musl': 1.0.0 '@rolldown/binding-linux-x64-musl': 1.0.1
'@rolldown/binding-openharmony-arm64': 1.0.0 '@rolldown/binding-openharmony-arm64': 1.0.1
'@rolldown/binding-wasm32-wasi': 1.0.0 '@rolldown/binding-wasm32-wasi': 1.0.1
'@rolldown/binding-win32-arm64-msvc': 1.0.0 '@rolldown/binding-win32-arm64-msvc': 1.0.1
'@rolldown/binding-win32-x64-msvc': 1.0.0 '@rolldown/binding-win32-x64-msvc': 1.0.1
router@2.2.0: router@2.2.0:
dependencies: dependencies:
@@ -4050,6 +4131,8 @@ snapshots:
safer-buffer@2.1.2: {} safer-buffer@2.1.2: {}
scheduler@0.27.0: {}
secure-json-parse@2.7.0: {} secure-json-parse@2.7.0: {}
semver@6.3.1: {} semver@6.3.1: {}
@@ -4288,12 +4371,12 @@ snapshots:
vary@1.1.2: {} vary@1.1.2: {}
vite@8.0.12(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0): vite@8.0.13(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0):
dependencies: dependencies:
lightningcss: 1.32.0 lightningcss: 1.32.0
picomatch: 4.0.4 picomatch: 4.0.4
postcss: 8.5.14 postcss: 8.5.14
rolldown: 1.0.0 rolldown: 1.0.1
tinyglobby: 0.2.16 tinyglobby: 0.2.16
optionalDependencies: optionalDependencies:
'@types/node': 24.12.4 '@types/node': 24.12.4
@@ -4301,10 +4384,10 @@ snapshots:
fsevents: 2.3.3 fsevents: 2.3.3
tsx: 4.21.0 tsx: 4.21.0
vitest@4.1.6(@opentelemetry/api@1.9.1)(@types/node@24.12.4)(vite@8.0.12(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0)): vitest@4.1.6(@opentelemetry/api@1.9.1)(@types/node@24.12.4)(vite@8.0.13(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0)):
dependencies: dependencies:
'@vitest/expect': 4.1.6 '@vitest/expect': 4.1.6
'@vitest/mocker': 4.1.6(vite@8.0.12(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0)) '@vitest/mocker': 4.1.6(vite@8.0.13(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0))
'@vitest/pretty-format': 4.1.6 '@vitest/pretty-format': 4.1.6
'@vitest/runner': 4.1.6 '@vitest/runner': 4.1.6
'@vitest/snapshot': 4.1.6 '@vitest/snapshot': 4.1.6
@@ -4321,7 +4404,7 @@ snapshots:
tinyexec: 1.1.2 tinyexec: 1.1.2
tinyglobby: 0.2.16 tinyglobby: 0.2.16
tinyrainbow: 3.1.0 tinyrainbow: 3.1.0
vite: 8.0.12(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0) vite: 8.0.13(@types/node@24.12.4)(esbuild@0.27.7)(tsx@4.21.0)
why-is-node-running: 2.3.0 why-is-node-running: 2.3.0
optionalDependencies: optionalDependencies:
'@opentelemetry/api': 1.9.1 '@opentelemetry/api': 1.9.1

View File

@@ -1,10 +1,10 @@
import "dotenv/config"; import "dotenv/config";
import { migrate } from "drizzle-orm/node-postgres/migrator"; import Database from "better-sqlite3";
import { migrate as migrateSqlite } from "drizzle-orm/better-sqlite3/migrator"; import { migrate as migrateSqlite } from "drizzle-orm/better-sqlite3/migrator";
import { initializeDatabase } from "./drizzle"; import { migrate } from "drizzle-orm/node-postgres/migrator";
import { config } from "../config"; import { config } from "../config";
import { createChildLogger } from "../logger"; import { createChildLogger } from "../logger";
import Database from "better-sqlite3"; import { initializeDatabase } from "./drizzle";
const logger = createChildLogger("migrate"); const logger = createChildLogger("migrate");
@@ -28,7 +28,7 @@ export async function runMigrations(): Promise<void> {
} catch (error) { } catch (error) {
logger.error( logger.error(
{ error: error instanceof Error ? error.message : String(error) }, { error: error instanceof Error ? error.message : String(error) },
"Migration failed" "Migration failed",
); );
throw error; throw error;
} }

View File

@@ -85,6 +85,27 @@ export const pgMessagesTable = pgTable(
userIdx: pgIndex("idx_messages_user").on(table.user_id), userIdx: pgIndex("idx_messages_user").on(table.user_id),
createdIdx: pgIndex("idx_messages_created").on(table.created_at), createdIdx: pgIndex("idx_messages_created").on(table.created_at),
threadIdx: pgIndex("idx_messages_thread").on(table.thread_id), threadIdx: pgIndex("idx_messages_thread").on(table.thread_id),
channelCreatedIdx: pgIndex("idx_messages_channel_created").on(
table.channel_id,
table.created_at,
table.id,
),
threadCreatedIdx: pgIndex("idx_messages_thread_created").on(
table.thread_id,
table.created_at,
table.id,
),
aiStatusCreatedIdx: pgIndex("idx_messages_ai_status_created").on(
table.ai_status,
table.created_at,
table.id,
),
guildAiStatusCreatedIdx: pgIndex("idx_messages_guild_ai_status_created").on(
table.guild_id,
table.ai_status,
table.created_at,
table.id,
),
}), }),
); );
@@ -119,6 +140,16 @@ export const pgAttachmentsTable = pgTable(
channelIdx: pgIndex("idx_attachments_channel").on(table.channel_id), channelIdx: pgIndex("idx_attachments_channel").on(table.channel_id),
messageIdx: pgIndex("idx_attachments_message").on(table.message_id), messageIdx: pgIndex("idx_attachments_message").on(table.message_id),
statusIdx: pgIndex("idx_attachments_status").on(table.upload_status), statusIdx: pgIndex("idx_attachments_status").on(table.upload_status),
channelCreatedIdx: pgIndex("idx_attachments_channel_created").on(
table.channel_id,
table.created_at,
table.id,
),
threadCreatedIdx: pgIndex("idx_attachments_thread_created").on(
table.thread_id,
table.created_at,
table.id,
),
messageFk: pgForeignKey({ messageFk: pgForeignKey({
columns: [table.message_id], columns: [table.message_id],
foreignColumns: [pgMessagesTable.id], foreignColumns: [pgMessagesTable.id],
@@ -137,6 +168,39 @@ export const pgUIStateTable = pgTable("ui_state", {
updated_at: pgBigint("updated_at", { mode: "number" }).notNull(), updated_at: pgBigint("updated_at", { mode: "number" }).notNull(),
}); });
/**
* AI Analysis Runs Table (PostgreSQL)
* Tracks AI analysis batch runs for conversation-level moderation
*/
export const pgAIAnalysisRunsTable = pgTable(
"ai_analysis_runs",
{
id: pgText("id").primaryKey(),
conversation_key: pgText("conversation_key").notNull(),
target_message_ids: pgText("target_message_ids").notNull(), // JSON array
model: pgText("model").notNull(),
request_tokens_estimate: pgInteger("request_tokens_estimate"),
response_raw: pgText("response_raw"),
status: pgText("status", {
enum: ["pending", "processing", "completed", "failed"],
})
.notNull()
.default("pending"),
error: pgText("error"),
created_at: pgBigint("created_at", { mode: "number" }).notNull(),
completed_at: pgBigint("completed_at", { mode: "number" }),
},
(table) => ({
conversationKeyIdx: pgIndex("idx_ai_analysis_runs_conversation_key").on(
table.conversation_key,
),
statusIdx: pgIndex("idx_ai_analysis_runs_status").on(table.status),
createdAtIdx: pgIndex("idx_ai_analysis_runs_created_at").on(
table.created_at,
),
}),
);
// SQLite Schema // SQLite Schema
// ============= // =============
@@ -206,6 +270,24 @@ export const sqliteMessagesTable = sqliteTable(
userIdx: sqliteIndex("idx_messages_user").on(table.user_id), userIdx: sqliteIndex("idx_messages_user").on(table.user_id),
createdIdx: sqliteIndex("idx_messages_created").on(table.created_at), createdIdx: sqliteIndex("idx_messages_created").on(table.created_at),
threadIdx: sqliteIndex("idx_messages_thread").on(table.thread_id), threadIdx: sqliteIndex("idx_messages_thread").on(table.thread_id),
channelCreatedIdx: sqliteIndex("idx_messages_channel_created").on(
table.channel_id,
table.created_at,
table.id,
),
threadCreatedIdx: sqliteIndex("idx_messages_thread_created").on(
table.thread_id,
table.created_at,
table.id,
),
aiStatusCreatedIdx: sqliteIndex("idx_messages_ai_status_created").on(
table.ai_status,
table.created_at,
table.id,
),
guildAiStatusCreatedIdx: sqliteIndex(
"idx_messages_guild_ai_status_created",
).on(table.guild_id, table.ai_status, table.created_at, table.id),
}), }),
); );
@@ -240,6 +322,16 @@ export const sqliteAttachmentsTable = sqliteTable(
channelIdx: sqliteIndex("idx_attachments_channel").on(table.channel_id), channelIdx: sqliteIndex("idx_attachments_channel").on(table.channel_id),
messageIdx: sqliteIndex("idx_attachments_message").on(table.message_id), messageIdx: sqliteIndex("idx_attachments_message").on(table.message_id),
statusIdx: sqliteIndex("idx_attachments_status").on(table.upload_status), statusIdx: sqliteIndex("idx_attachments_status").on(table.upload_status),
channelCreatedIdx: sqliteIndex("idx_attachments_channel_created").on(
table.channel_id,
table.created_at,
table.id,
),
threadCreatedIdx: sqliteIndex("idx_attachments_thread_created").on(
table.thread_id,
table.created_at,
table.id,
),
}), }),
); );
@@ -253,6 +345,39 @@ export const sqliteUIStateTable = sqliteTable("ui_state", {
updated_at: sqliteInteger("updated_at").notNull(), updated_at: sqliteInteger("updated_at").notNull(),
}); });
/**
* AI Analysis Runs Table (SQLite)
* Tracks AI analysis batch runs for conversation-level moderation
*/
export const sqliteAIAnalysisRunsTable = sqliteTable(
"ai_analysis_runs",
{
id: sqliteText("id").primaryKey(),
conversation_key: sqliteText("conversation_key").notNull(),
target_message_ids: sqliteText("target_message_ids").notNull(), // JSON array
model: sqliteText("model").notNull(),
request_tokens_estimate: sqliteInteger("request_tokens_estimate"),
response_raw: sqliteText("response_raw"),
status: sqliteText("status", {
enum: ["pending", "processing", "completed", "failed"],
})
.notNull()
.default("pending"),
error: sqliteText("error"),
created_at: sqliteInteger("created_at").notNull(),
completed_at: sqliteInteger("completed_at"),
},
(table) => ({
conversationKeyIdx: sqliteIndex("idx_ai_analysis_runs_conversation_key").on(
table.conversation_key,
),
statusIdx: sqliteIndex("idx_ai_analysis_runs_status").on(table.status),
createdAtIdx: sqliteIndex("idx_ai_analysis_runs_created_at").on(
table.created_at,
),
}),
);
// Runtime table selection based on config // Runtime table selection based on config
// ======================================== // ========================================
@@ -270,6 +395,11 @@ export const attachmentsTable =
export const uiStateTable = export const uiStateTable =
config.DATABASE_TYPE === "postgres" ? pgUIStateTable : sqliteUIStateTable; config.DATABASE_TYPE === "postgres" ? pgUIStateTable : sqliteUIStateTable;
export const aiAnalysisRunsTable =
config.DATABASE_TYPE === "postgres"
? pgAIAnalysisRunsTable
: sqliteAIAnalysisRunsTable;
// Export table types for use in queries // Export table types for use in queries
export type MuxerJob = typeof muxerJobsTable.$inferSelect; export type MuxerJob = typeof muxerJobsTable.$inferSelect;
export type MuxerJobInsert = typeof muxerJobsTable.$inferInsert; export type MuxerJobInsert = typeof muxerJobsTable.$inferInsert;
@@ -282,3 +412,6 @@ export type AttachmentInsert = typeof attachmentsTable.$inferInsert;
export type UIState = typeof uiStateTable.$inferSelect; export type UIState = typeof uiStateTable.$inferSelect;
export type UIStateInsert = typeof uiStateTable.$inferInsert; export type UIStateInsert = typeof uiStateTable.$inferInsert;
export type AIAnalysisRun = typeof aiAnalysisRunsTable.$inferSelect;
export type AIAnalysisRunInsert = typeof aiAnalysisRunsTable.$inferInsert;

View File

@@ -1,373 +1,280 @@
import { config } from "../config"; import { config } from "../config";
import { createChildLogger } from "../logger"; import { createChildLogger } from "../logger";
import type { SqliteDatabase } from "../muxer-queue"; import { buildConversationPromptMessages } from "./conversationContext";
import { retryWithBackoff } from "../retry"; import { runModerationAnalysis } from "./llmModerationClient";
import { import {
getConversationContextBefore,
getMessageById, getMessageById,
getPendingAIAnalysisMessages, getPendingConversationKeys,
getPendingMessagesByConversation,
updateMessageAIAnalysis, updateMessageAIAnalysis,
} from "./messageStore"; } from "./messageStore";
import type { MessageRecord } from "./types"; import type { AnalysisQueueStatus, MessageRecord } from "./types";
const logger = createChildLogger("ai-analyzer"); const logger = createChildLogger("ai-analyzer");
const queuedMessageIds = new Set<string>();
let isProcessing = false; // Debounce state per conversation key
const conversationDebounceTimers = new Map<string, NodeJS.Timeout>();
// Track conversations currently being processed
const conversationProcessing = new Set<string>();
// Track conversations in error cooldown (failed recently)
const conversationErrorCooldown = new Map<string, number>();
let activeRequests = 0; let activeRequests = 0;
const MAX_CONCURRENT_REQUESTS = 1; let lastError: string | null = null;
const MAX_AI_REQUEST_TOKENS = 12_000; const MAX_ACTIVE_REQUESTS = 1;
const AI_PROMPT_TOKEN_RESERVE = 3_000; const DEBOUNCE_MS = 1500;
const MAX_AI_BATCH_MESSAGES = 80; const RECOVERY_INTERVAL_MS = 15000;
const ERROR_COOLDOWN_MS = 30000;
const MAX_CONTEXT_TOKENS = 8000;
const MAX_BATCH_SIZE = 25;
interface ChatCompletionResponse { /**
choices?: Array<{ * Gets the conversation key for a message (thread_id or channel_id)
message?: { */
content?: string; export function getConversationKey(message: MessageRecord): string {
}; return message.thread_id || message.channel_id;
}>;
} }
interface LLMAnalysis { /**
status: "clean" | "warn" | "flagged"; * Picks a batch of messages within token budget
flags: string[]; */
score: number; export function pickBatchWithinBudget(
analysis: string;
}
function getAnalysisText(message: MessageRecord): string {
return (message.edited_content || message.content || "").trim();
}
function estimateTokens(text: string): number {
return Math.ceil(text.length / 4);
}
function formatMessageForAnalysis(
message: MessageRecord,
index: number,
): string {
const text = getAnalysisText(message);
const time = new Date(message.created_at).toISOString();
return `${index + 1}. id=${message.id} time=${time} user=${message.username}: ${text}`;
}
function estimateMessageTokens(message: MessageRecord): number {
return estimateTokens(formatMessageForAnalysis(message, 0)) + 16;
}
async function fetchJson(url: string, init: RequestInit): Promise<unknown> {
const controller = new AbortController();
const timeout = setTimeout(
() => controller.abort(),
config.AI_ANALYSIS_TIMEOUT_MS,
);
try {
const response = await fetch(url, { ...init, signal: controller.signal });
const text = await response.text();
if (!response.ok) {
const message = text.includes("{")
? JSON.stringify(JSON.parse(text.substring(text.indexOf("{"))))
: text;
throw new Error(`AI request failed (${response.status}): ${message}`);
}
// Handle streaming response: extract JSON from response text
const jsonStart = text.indexOf("{");
const jsonEnd = text.lastIndexOf("}");
if (jsonStart >= 0 && jsonEnd > jsonStart) {
try {
return JSON.parse(text.substring(jsonStart, jsonEnd + 1));
} catch {
// Fall through to parse full text
}
}
return JSON.parse(text);
} finally {
clearTimeout(timeout);
}
}
function parseLLMAnalysis(content: string): LLMAnalysis {
const jsonStart = content.indexOf("{");
const jsonEnd = content.lastIndexOf("}");
if (jsonStart >= 0 && jsonEnd > jsonStart) {
try {
const parsed = JSON.parse(content.slice(jsonStart, jsonEnd + 1));
const status =
parsed.status === "flagged"
? "flagged"
: parsed.status === "warn"
? "warn"
: "clean";
const flags = Array.isArray(parsed.flags) ? parsed.flags.map(String) : [];
const score = Math.max(0, Math.min(1, Number(parsed.score) || 0));
const analysis =
typeof parsed.analysis === "string" ? parsed.analysis : content;
return { status, flags, score, analysis };
} catch {
// Fall through to text-only parsing.
}
}
return {
status:
/flagged|bahaya|berisiko|toxic|hate|harassment|violence|sexual|self-harm|illegal|scam|hacking/i.test(
content,
)
? "flagged"
: /warn|provokasi|hinaan|menyerang/i.test(content)
? "warn"
: "clean",
flags: [],
score: 0,
analysis: content.trim() || "Tidak ada analisis dari LLM.",
};
}
async function runLLMAnalysis(
messages: MessageRecord[], messages: MessageRecord[],
): Promise<{ results: LLMAnalysis[]; raw: unknown }> { maxTokens: number,
const response = (await retryWithBackoff( tokensPerMessage: number,
() => ): MessageRecord[] {
fetchJson(`${config.AI_LLM_BASE_URL}/chat/completions`, { const batch: MessageRecord[] = [];
method: "POST", let usedTokens = 0;
headers: {
Authorization: `Bearer ${config.AI_LLM_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model: config.AI_LLM_MODEL,
messages: [
{
role: "system",
content: `Kamu moderator Discord komunitas. Analisis setiap pesan dengan 3 kategori:
- CLEAN: Pesan normal, tidak melanggar aturan
- WARN: Melanggar aturan minor yang menarget orang lain (tone menyerang, hinaan ringan, konflik kecil) - butuh peringatan tapi tidak dihapus
- FLAGGED: Melanggar aturan berat (NSFW, ilegal, hacking, scam, harassment, violence, SARA, gore, spam, promosi judi) - butuh review moderator untuk penghapusan
ATURAN KOMUNITAS LENGKAP: for (const msg of messages) {
// Estimate tokens based on actual content length
const content = msg.edited_content ?? msg.content;
const contentTokens = Math.ceil(content.length / 4);
const msgTokens = contentTokens + tokensPerMessage;
1. JAGA SIKAP DAN HORMATI SESAMA if (usedTokens + msgTokens <= maxTokens) {
- Gunakan bahasa yang sopan dan menghormati semua anggota batch.push(msg);
- Tanpa memandang latar belakang, usia, gender, atau pandangan usedTokens += msgTokens;
- Dilarang keras: pelecehan, rasisme, seksisme, diskriminasi
2. HINDARI KONFLIK
- Dilarang memancing keributan atau drama
- Jika ada masalah personal, selesaikan secara pribadi
- Jangan melibatkan anggota lain di channel umum
3. KONTEN EKSPLISIT DILARANG
- Dilarang keras: NSFW, ilegal, pornografi, kekerasan (gore), SARA
- Tidak ada tempat untuk penyimpangan atau LGBT
- Tidak ada promosi aktivitas atau ideologi LGBT
4. JAGA PRIVASI
- Dilarang menyebarkan informasi pribadi milik anggota lain tanpa izin
5. PROFIL YANG SOPAN
- Username, foto profil, dan server tag harus pantas
- Jangan gunakan unsur ofensif atau vulgar
6. DILARANG SPAM DAN PENIPUAN
- Dilarang: hoaks, link berbahaya (phishing/scam), spam
- Dilarang: promosi, judi, link referral
7. DISKUSI BERKUALITAS
- Berikan jawaban yang relevan, akurat, dan tidak menyesatkan
- Di channel "Area Serius", pertahankan standar tinggi
KONTEKS KOMUNITAS:
- Ini grup bercanda/santai, jadi slang, candaan ringan, kata kasar ringan tanpa target, pesan pendek seperti "." atau "P", dan pertanyaan tidak jelas tetap CLEAN
- Jangan beri WARN hanya karena pesan singkat, informal, ambigu, low-quality, atau kurang konteks
- Pahami alur pembahasan antar pesan: pesan yang sendiri terlihat normal bisa WARN/FLAGGED jika dalam konteks percakapan sedang memancing konflik, menormalisasi pelanggaran, atau melanjutkan provokasi
- Jangan menghukum orang yang sedang menasehati, menjelaskan bahaya, mengutip, atau menolak tindakan buruk; nilai maksud dan konteksnya
- WARN hanya jika ada orang/kelompok yang diserang, dihina, diprovokasi, atau konflik mulai dipancing
PENENTUAN STATUS:
- WARN jika: hinaan ringan yang menarget orang/kelompok, provokasi konflik kecil, username/profil kurang pantas
- FLAGGED jika: profanity berat, harassment, threats, violence, illegal activity, hacking, scam, NSFW, SARA, gore, spam, judi, LGBT content
Balas JSON array dengan schema: [{"status":"clean|warn|flagged","flags":["..."],"score":0..1,"analysis":"ringkasan Bahasa Indonesia + alasan + aksi disarankan"}]
Satu JSON object per pesan dalam array.`,
},
{
role: "user",
content: `Analisis ${messages.length} pesan berikut sebagai satu alur percakapan. Tetap kembalikan satu hasil per pesan dengan urutan yang sama:\n${messages.map(formatMessageForAnalysis).join("\n")}`,
},
],
temperature: 0.2,
}),
signal: AbortSignal.timeout(config.AI_ANALYSIS_TIMEOUT_MS),
}),
{ retries: 2, logger },
)) as ChatCompletionResponse;
const content = response.choices?.[0]?.message?.content?.trim() || "";
// Extract JSON array from response
const jsonStart = content.indexOf("[");
const jsonEnd = content.lastIndexOf("]");
let results: LLMAnalysis[] = [];
if (jsonStart >= 0 && jsonEnd > jsonStart) {
try {
const parsed = JSON.parse(content.substring(jsonStart, jsonEnd + 1));
if (Array.isArray(parsed)) {
results = parsed.map((item: any) => {
const status =
item.status === "flagged"
? "flagged"
: item.status === "warn"
? "warn"
: "clean";
return {
status,
flags: Array.isArray(item.flags) ? item.flags.map(String) : [],
score: Math.max(0, Math.min(1, Number(item.score) || 0)),
analysis:
typeof item.analysis === "string" ? item.analysis : content,
};
});
}
} catch {
// Fall through to individual parsing
} }
} }
// If batch parsing failed, parse as individual responses return batch;
if (results.length === 0) {
results = messages.map(() => parseLLMAnalysis(content));
}
return { results, raw: response };
} }
async function analyzeAndStoreBatch(messages: MessageRecord[]): Promise<void> { /**
* Processes a batch of messages for a conversation
*/
async function processBatch(
conversationKey: string,
messages: MessageRecord[],
): Promise<void> {
if (messages.length === 0) return; if (messages.length === 0) return;
const analyzableMessages = messages.filter(
(message) => getAnalysisText(message).length > 0,
);
if (analyzableMessages.length === 0) return;
activeRequests++; activeRequests++;
conversationProcessing.add(conversationKey);
try { try {
const { results, raw } = await runLLMAnalysis(analyzableMessages); // Get context before the first message
const firstMessage = messages[0];
const contextBefore = await getConversationContextBefore({
channelId: firstMessage.channel_id,
threadId: firstMessage.thread_id,
beforeCreatedAt: firstMessage.created_at,
limit: 20,
});
for (let i = 0; i < analyzableMessages.length; i++) { // Build prompt with context
const message = analyzableMessages[i]; const promptMessages = buildConversationPromptMessages({
const result = results[i] || parseLLMAnalysis(""); contextBefore,
targets: messages,
maxTokens: MAX_CONTEXT_TOKENS,
});
const row = await updateMessageAIAnalysis(message.id, { const contextText = promptMessages.join("\n");
status: result.status as
| "pending" // Run moderation analysis
| "clean" const result = await runModerationAnalysis({
| "warn" targets: messages,
| "flagged" contextText,
| "error", });
flags: JSON.stringify(result.flags),
score: result.score, // Store results
raw: JSON.stringify(raw), const analyzedRows: MessageRecord[] = [];
analysis: result.analysis, for (const analysisResult of result.results) {
const row = await updateMessageAIAnalysis(analysisResult.messageId, {
status: analysisResult.status,
flags: JSON.stringify(analysisResult.flags),
score: analysisResult.score,
raw: JSON.stringify(result.raw),
analysis: analysisResult.analysis,
analyzedAt: Date.now(), analyzedAt: Date.now(),
error: null, error: null,
}); });
if (row) (globalThis as any).broadcastMessageAnalyzed?.(row); if (row) {
} analyzedRows.push(row);
} catch (error) { }
if (analyzableMessages.length > 1) {
const midpoint = Math.ceil(analyzableMessages.length / 2);
logger.warn(
{
count: analyzableMessages.length,
nextBatchSizes: [midpoint, analyzableMessages.length - midpoint],
error,
},
"AI batch failed, splitting into smaller batches",
);
await analyzeAndStoreBatch(analyzableMessages.slice(0, midpoint));
await analyzeAndStoreBatch(analyzableMessages.slice(midpoint));
return;
} }
const errorMsg = error instanceof Error ? error.message : String(error); // Broadcast analyzed messages
for (const message of analyzableMessages) { for (const row of analyzedRows) {
const row = await updateMessageAIAnalysis(message.id, { (globalThis as any).moderationBroadcaster?.messageAnalyzed(row);
}
// Clear error cooldown on success
conversationErrorCooldown.delete(conversationKey);
logger.info(
{ conversationKey, count: messages.length },
"Batch analysis complete",
);
} catch (error) {
lastError = error instanceof Error ? error.message : String(error);
logger.error(
{ conversationKey, error: lastError },
"Batch analysis failed",
);
// Mark all messages in batch as error
for (const msg of messages) {
const row = await updateMessageAIAnalysis(msg.id, {
status: "error", status: "error",
flags: null, flags: null,
score: null, score: null,
raw: null, raw: null,
analysis: null, analysis: null,
analyzedAt: Date.now(), analyzedAt: Date.now(),
error: errorMsg, error: lastError,
}); });
if (row) (globalThis as any).broadcastMessageAnalyzed?.(row); if (row) {
(globalThis as any).moderationBroadcaster?.messageAnalyzed(row);
}
} }
logger.warn({ count: messages.length, error }, "AI batch analysis failed");
// Set error cooldown for this conversation
conversationErrorCooldown.set(
conversationKey,
Date.now() + ERROR_COOLDOWN_MS,
);
} finally { } finally {
activeRequests--; activeRequests--;
conversationProcessing.delete(conversationKey);
} }
} }
async function drainQueue(): Promise<void> { /**
if (isProcessing) return; * Debounced analysis trigger for a conversation
isProcessing = true; */
try { function scheduleConversationAnalysis(conversationKey: string): void {
const batchTokenLimit = MAX_AI_REQUEST_TOKENS - AI_PROMPT_TOKEN_RESERVE; // Skip if already processing
if (conversationProcessing.has(conversationKey)) {
while (queuedMessageIds.size > 0) { logger.debug(
while (activeRequests >= MAX_CONCURRENT_REQUESTS) { { conversationKey },
await new Promise((resolve) => setTimeout(resolve, 100)); "Conversation already processing, skipping schedule",
}
const batch: MessageRecord[] = [];
let tokenEstimate = 0;
for (const messageId of Array.from(queuedMessageIds)) {
const message = await getMessageById(messageId);
queuedMessageIds.delete(messageId);
if (!message) continue;
const messageTokens = estimateMessageTokens(message);
if (
batch.length > 0 &&
(batch.length >= MAX_AI_BATCH_MESSAGES ||
tokenEstimate + messageTokens > batchTokenLimit)
) {
queuedMessageIds.add(messageId);
break;
}
batch.push(message);
tokenEstimate += messageTokens;
}
if (batch.length > 0) {
logger.info(
{ count: batch.length, tokenEstimate },
"Processing AI analysis batch",
);
await analyzeAndStoreBatch(batch);
}
}
} finally {
isProcessing = false;
}
}
export function queueMessageAnalysis(messageId: string): void {
if (!config.AI_ANALYSIS_ENABLED) return;
logger.debug({ messageId }, "Queueing AI analysis");
queuedMessageIds.add(messageId);
setImmediate(() => {
drainQueue().catch((error) =>
logger.error({ error }, "AI analysis queue failed"),
); );
}); return;
}
// Skip if in error cooldown
const cooldownUntil = conversationErrorCooldown.get(conversationKey);
if (cooldownUntil && Date.now() < cooldownUntil) {
logger.debug(
{ conversationKey, cooldownMs: cooldownUntil - Date.now() },
"Conversation in error cooldown, skipping schedule",
);
return;
}
// Clear existing timer
const existingTimer = conversationDebounceTimers.get(conversationKey);
if (existingTimer) {
clearTimeout(existingTimer);
}
// Set new debounced timer
const timer = setTimeout(async () => {
conversationDebounceTimers.delete(conversationKey);
// If activeRequests >= MAX_ACTIVE_REQUESTS, requeue instead of waiting
if (activeRequests >= MAX_ACTIVE_REQUESTS) {
logger.debug(
{ conversationKey, activeRequests },
"Max active requests reached, requeuing conversation",
);
scheduleConversationAnalysis(conversationKey);
return;
}
// Get pending messages for this conversation
const messages = await getPendingMessagesByConversation(
conversationKey,
MAX_BATCH_SIZE,
);
if (messages.length > 0) {
await processBatch(conversationKey, messages);
}
}, DEBOUNCE_MS);
conversationDebounceTimers.set(conversationKey, timer);
} }
/**
* Queues a message for analysis (debounced by conversation)
*/
export async function queueMessageAnalysis(messageId: string): Promise<void> {
if (!config.AI_ANALYSIS_ENABLED) return;
logger.debug({ messageId }, "Queueing message for analysis");
try {
// Look up the message to get its conversation key
const message = await getMessageById(messageId);
if (!message) {
logger.warn({ messageId }, "Message not found for analysis queue");
return;
}
// Schedule its conversation for analysis
const conversationKey = getConversationKey(message);
queueConversationAnalysis(conversationKey);
} catch (error) {
logger.error(
{
messageId,
error: error instanceof Error ? error.message : String(error),
},
"Failed to queue message for analysis",
);
}
}
/**
* Queues a conversation for analysis (debounced)
*/
export function queueConversationAnalysis(conversationKey: string): void {
if (!config.AI_ANALYSIS_ENABLED) return;
logger.debug({ conversationKey }, "Queueing conversation for analysis");
// Schedule debounced analysis
scheduleConversationAnalysis(conversationKey);
}
/**
* Gets current analysis queue status
*/
export function getAnalysisQueueStatus(): AnalysisQueueStatus {
return {
queuedConversations: conversationDebounceTimers.size,
activeRequests,
lastError,
};
}
/**
* Starts the pending AI analysis recovery worker
*/
export function startPendingAIAnalysisWorker(): void { export function startPendingAIAnalysisWorker(): void {
if (!config.AI_ANALYSIS_ENABLED) { if (!config.AI_ANALYSIS_ENABLED) {
logger.info("AI analysis disabled"); logger.info("AI analysis disabled");
@@ -375,19 +282,37 @@ export function startPendingAIAnalysisWorker(): void {
} }
logger.info("AI analysis worker started"); logger.info("AI analysis worker started");
setInterval(async () => { setInterval(async () => {
if (isProcessing) return; try {
const pendingMessages = await getPendingAIAnalysisMessages(500); // Get pending conversation keys
if (pendingMessages.length === 0) return; const conversationKeys = await getPendingConversationKeys(100);
logger.info(
{ count: pendingMessages.length }, for (const key of conversationKeys) {
"Queueing pending AI analysis messages", // Skip if already scheduled
); if (conversationDebounceTimers.has(key)) {
for (const message of pendingMessages) { continue;
queuedMessageIds.add(message.id); }
// Skip if currently processing
if (conversationProcessing.has(key)) {
continue;
}
// Skip if in error cooldown
const cooldownUntil = conversationErrorCooldown.get(key);
if (cooldownUntil && Date.now() < cooldownUntil) {
continue;
}
logger.debug(
{ conversationKey: key },
"Recovering pending conversation",
);
scheduleConversationAnalysis(key);
}
} catch (error) {
logger.error({ error }, "Pending AI analysis recovery worker failed");
} }
drainQueue().catch((error) => }, RECOVERY_INTERVAL_MS);
logger.error({ error }, "Pending AI analysis worker failed"),
);
}, 15000);
} }

View File

@@ -0,0 +1,75 @@
import type { WebSocket } from "ws";
import { createChildLogger } from "../logger";
import type {
AnalysisQueueStatus,
AttachmentRecord,
MessageRecord,
ModerationWsEvent,
} from "./types";
type ClientLike = Pick<WebSocket, "readyState" | "send">;
const log = createChildLogger("broadcaster");
function sendJson(clients: Set<ClientLike>, event: ModerationWsEvent): void {
const payload = JSON.stringify({ ...event, timestamp: Date.now() });
for (const client of clients) {
if (client.readyState === 1) {
try {
client.send(payload);
} catch (error) {
log.warn(
{ error, eventType: event.type },
"Failed to send event to client",
);
}
}
}
}
export function createBroadcaster() {
const clients = new Set<ClientLike>();
return {
addClient(client: ClientLike) {
clients.add(client);
log.debug({ clientCount: clients.size }, "Client added");
},
removeClient(client: ClientLike) {
clients.delete(client);
log.debug({ clientCount: clients.size }, "Client removed");
},
clientCount() {
return clients.size;
},
getClients() {
return Array.from(clients);
},
uiState(state: unknown) {
sendJson(clients, { type: "ui_state", state });
},
userState(users: unknown[]) {
sendJson(clients, { type: "user_state", users });
},
messageCreated(data: MessageRecord) {
sendJson(clients, { type: "message_created", data });
},
messageUpdated(data: Partial<MessageRecord> & { id: string }) {
sendJson(clients, { type: "message_updated", data });
},
messageDeleted(data: { id: string; deleted_at: number }) {
sendJson(clients, { type: "message_deleted", data });
},
messageAnalyzed(data: MessageRecord) {
sendJson(clients, { type: "message_analyzed", data });
},
attachmentCreated(data: AttachmentRecord) {
sendJson(clients, { type: "attachment_created", data });
},
analysisQueueStatus(data: AnalysisQueueStatus) {
sendJson(clients, { type: "analysis_queue_status", data });
},
};
}
export type ModerationBroadcaster = ReturnType<typeof createBroadcaster>;

View File

@@ -0,0 +1,76 @@
import type { MessageRecord } from "./types";
export interface ConversationContextInput {
contextBefore: MessageRecord[];
targets: MessageRecord[];
maxTokens: number;
}
/**
* Formats a timestamp to ISO 8601 string
*/
function formatTimestamp(ms: number): string {
return new Date(ms).toISOString();
}
/**
* Estimates token count for a string (rough approximation: ~4 chars per token)
*/
function estimateTokens(text: string): number {
return Math.ceil(text.length / 4);
}
/**
* Builds conversation prompt messages with context and targets
* - Marks target messages with [target], prior context with [context]
* - Uses edited_content when present, otherwise content
* - Maintains chronological order
* - Respects maxTokens budget, prioritizing targets and most recent context
*/
export function buildConversationPromptMessages(
input: ConversationContextInput,
): string[] {
const { contextBefore, targets, maxTokens } = input;
// Format all messages
const formatMessage = (msg: MessageRecord, label: string): string => {
const content = msg.edited_content ?? msg.content;
const timestamp = formatTimestamp(msg.created_at);
return `[${label}] id=${msg.id} time=${timestamp} user=${msg.username}: ${content}`;
};
const targetLines = targets.map((msg) => formatMessage(msg, "target"));
const contextLines = contextBefore.map((msg) =>
formatMessage(msg, "context"),
);
// Calculate tokens for targets (always include)
let usedTokens = targetLines.reduce(
(sum, line) => sum + estimateTokens(line),
0,
);
// Add context lines in reverse chronological order (most recent first)
// until we hit the token budget
const selectedContextLines: string[] = [];
for (let i = contextLines.length - 1; i >= 0; i--) {
const line = contextLines[i];
const lineTokens = estimateTokens(line);
if (usedTokens + lineTokens <= maxTokens) {
selectedContextLines.unshift(line); // prepend to maintain chronological order
usedTokens += lineTokens;
}
}
// Combine: context (chronological) + targets (chronological)
const allMessages = [...selectedContextLines, ...targetLines];
// Sort by timestamp to ensure chronological order
allMessages.sort((a, b) => {
const timeA = a.match(/time=([^\s]+)/)?.[1] ?? "";
const timeB = b.match(/time=([^\s]+)/)?.[1] ?? "";
return timeA.localeCompare(timeB);
});
return allMessages;
}

View File

@@ -0,0 +1,260 @@
import { config } from "../config";
import { createChildLogger } from "../logger";
import { retryWithBackoff } from "../retry";
import type { AnalysisResult, MessageRecord } from "./types";
const log = createChildLogger("llmModerationClient");
interface RawModerationResult {
message_id: string;
status: string;
flags: unknown;
score: number;
analysis: string;
}
interface RawModerationResponse {
results: RawModerationResult[];
}
/**
* Parses LLM moderation response and validates against target IDs.
* Extracts JSON from surrounding text, validates structure, and transforms to AnalysisResult[].
* Scans from first '{' and attempts JSON.parse at each candidate closing brace.
*/
export function parseModerationResponse(
content: string,
targetIds: string[],
): AnalysisResult[] {
// Find first opening brace
const startIdx = content.indexOf("{");
if (startIdx === -1) {
throw new Error("No JSON object found in response");
}
// Scan from start and try parsing at each closing brace
let parsed: unknown;
let lastError: Error | null = null;
for (let i = startIdx + 1; i < content.length; i++) {
if (content[i] === "}") {
const candidate = content.substring(startIdx, i + 1);
try {
parsed = JSON.parse(candidate);
// Successfully parsed, break out
break;
} catch (error) {
// Store error and continue scanning
lastError = error instanceof Error ? error : new Error(String(error));
continue;
}
}
}
if (!parsed) {
throw new Error(
`Failed to parse JSON: ${lastError?.message || "No valid JSON object found"}`,
);
}
// Validate structure
if (!parsed || typeof parsed !== "object" || !("results" in parsed)) {
throw new Error("Response missing 'results' array");
}
const response = parsed as RawModerationResponse;
if (!Array.isArray(response.results)) {
throw new Error("'results' must be an array");
}
// Track which target IDs were found
const foundIds = new Set<string>();
const targetIdSet = new Set(targetIds);
// Parse and validate each result
const results: AnalysisResult[] = response.results.map((result) => {
const { message_id, status, flags, score, analysis } = result;
// Validate message_id exists and is in target list
if (!message_id) {
throw new Error("Result missing 'message_id'");
}
if (!targetIdSet.has(message_id)) {
throw new Error(`Unknown message_id: ${message_id}`);
}
if (foundIds.has(message_id)) {
throw new Error(`Duplicate message_id in results: ${message_id}`);
}
foundIds.add(message_id);
// Validate status
const validStatuses = ["clean", "warn", "flagged"] as const;
if (!validStatuses.includes(status as (typeof validStatuses)[number])) {
throw new Error(
`Invalid status: ${status}. Must be one of: ${validStatuses.join(", ")}`,
);
}
// Validate score: reject null/undefined/non-finite before coercion
if (score === null || score === undefined) {
throw new Error("Invalid score: must not be null or undefined");
}
let numScore = Number(score);
if (!Number.isFinite(numScore)) {
throw new Error(`Invalid score: ${score}. Must be a finite number`);
}
numScore = Math.max(0, Math.min(1, numScore));
// Coerce flags to string array
let flagsArray: string[] = [];
if (Array.isArray(flags)) {
flagsArray = flags.map((f) => String(f));
} else if (flags) {
flagsArray = [String(flags)];
}
// Fallback analysis
const analysisStr = analysis ? String(analysis) : "";
return {
messageId: message_id,
status: status as "clean" | "warn" | "flagged",
flags: flagsArray,
score: numScore,
analysis: analysisStr,
};
});
// Check that all target IDs were found
const missingIds = targetIds.filter((id) => !foundIds.has(id));
if (missingIds.length > 0) {
throw new Error(`Missing target ids in response: ${missingIds.join(", ")}`);
}
return results;
}
interface ModerationInput {
targets: MessageRecord[];
contextText: string;
}
interface ModerationOutput {
results: AnalysisResult[];
raw: unknown;
}
/**
* Runs LLM-based moderation analysis on messages.
* POSTs to AI_LLM_BASE_URL with auth bearer token.
*/
export async function runModerationAnalysis(
input: ModerationInput,
): Promise<ModerationOutput> {
const { targets, contextText } = input;
if (!targets.length) {
throw new Error("No targets provided for analysis");
}
const targetIds = targets.map((t) => t.id);
// Build prompt
const messagesText = targets
.map((msg) => `[${msg.id}] ${msg.username}: ${msg.content}`)
.join("\n");
const prompt = `You are a content moderation assistant. Analyze the following messages for policy violations.
Context: ${contextText}
Messages to analyze:
${messagesText}
For each message, respond with a JSON object containing a "results" array. Each result must have:
- message_id: the message ID
- status: "clean", "warn", or "flagged"
- flags: array of violation flags (e.g., ["spam", "hate_speech"])
- score: confidence score from 0 to 1
- analysis: brief explanation
Return ONLY valid JSON, no other text.`;
const result = await retryWithBackoff(
async () => {
const controller = new AbortController();
const timeoutId = setTimeout(
() => controller.abort(),
config.AI_ANALYSIS_TIMEOUT_MS,
);
try {
const response = await fetch(
`${config.AI_LLM_BASE_URL}/chat/completions`,
{
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${config.AI_LLM_API_KEY}`,
},
signal: controller.signal,
body: JSON.stringify({
model: config.AI_LLM_MODEL,
messages: [
{
role: "user",
content: prompt,
},
],
temperature: 0.3,
}),
},
);
if (!response.ok) {
const text = await response.text();
throw new Error(`LLM API error ${response.status}: ${text}`);
}
return response.json();
} finally {
clearTimeout(timeoutId);
}
},
{
retries: 3,
minTimeout: 1000,
maxTimeout: 10000,
logger: log,
},
);
// Extract content from response
if (!result.choices || !Array.isArray(result.choices) || !result.choices[0]) {
throw new Error("Invalid LLM response structure");
}
const content = result.choices[0].message?.content;
if (!content) {
throw new Error("No content in LLM response");
}
// Parse and validate
const parsed = parseModerationResponse(content, targetIds);
log.info(
{
targetCount: targets.length,
resultCount: parsed.length,
},
"Moderation analysis complete",
);
return {
results: parsed,
raw: result,
};
}

View File

@@ -1,8 +1,5 @@
import type { Client, Message } from "discord.js-selfbot-v13"; import type { Client, Message } from "discord.js-selfbot-v13";
import { eq } from "drizzle-orm";
import { config } from "../config"; import { config } from "../config";
import { getDatabase } from "../database/drizzle";
import { messagesTable } from "../database/schema";
import { createChildLogger } from "../logger"; import { createChildLogger } from "../logger";
import { queueMessageAnalysis } from "./aiAnalyzer"; import { queueMessageAnalysis } from "./aiAnalyzer";
import { import {
@@ -10,8 +7,18 @@ import {
getMessageLocation, getMessageLocation,
getMessageMetadata, getMessageMetadata,
} from "./messageMetadata"; } from "./messageMetadata";
import { insertAttachment, insertMessage } from "./messageStore"; import {
import type { AttachmentRecord, MessageRecord } from "./types"; getMessageById,
insertAttachment,
updateMessageAsDeleted,
updateMessageAsEdited,
upsertMessageForCapture,
} from "./messageStore";
import type {
AttachmentRecord,
MessageRecord,
ModerationBroadcaster,
} from "./types";
const logger = createChildLogger("message-capture"); const logger = createChildLogger("message-capture");
@@ -39,12 +46,14 @@ export async function captureMessage(
metadata: JSON.stringify(metadata), metadata: JSON.stringify(metadata),
}; };
await insertMessage(messageRecord); await upsertMessageForCapture(messageRecord);
queueMessageAnalysis(message.id); queueMessageAnalysis(message.id);
const broadcaster = globalThis as any; const broadcaster = (globalThis as any).moderationBroadcaster as
if (broadcaster.broadcastMessageCreated) { | ModerationBroadcaster
broadcaster.broadcastMessageCreated({ | undefined;
if (broadcaster) {
broadcaster.messageCreated({
...messageRecord, ...messageRecord,
type: "text", type: "text",
}); });
@@ -72,14 +81,8 @@ export async function captureMessage(
await insertAttachment(attachmentRecord); await insertAttachment(attachmentRecord);
if (broadcaster.broadcastAttachmentUploaded) { if (broadcaster) {
broadcaster.broadcastAttachmentUploaded({ broadcaster.attachmentCreated(attachmentRecord);
id: attachment.id,
message_id: message.id,
filename: attachment.name || "unknown",
channel_id: location.channelId,
created_at: Date.now(),
});
} }
} }
} }
@@ -118,16 +121,9 @@ export function registerMessageCapture(client: Client): void {
if (newMessage.author?.bot) return; if (newMessage.author?.bot) return;
try { try {
const { updateMessageAsEdited } = await import("./messageStore"); const existing = await getMessageById(newMessage.id);
const db = getDatabase() as any;
const existing = await db if (existing) {
.select()
.from(messagesTable)
.where(eq(messagesTable.id, newMessage.id))
.limit(1);
if (existing.length > 0) {
const editedAt = Date.now(); const editedAt = Date.now();
await updateMessageAsEdited( await updateMessageAsEdited(
newMessage.id, newMessage.id,
@@ -136,9 +132,11 @@ export function registerMessageCapture(client: Client): void {
); );
queueMessageAnalysis(newMessage.id); queueMessageAnalysis(newMessage.id);
const broadcaster = globalThis as any; const broadcaster = (globalThis as any).moderationBroadcaster as
if (broadcaster.broadcastMessageUpdated) { | ModerationBroadcaster
broadcaster.broadcastMessageUpdated({ | undefined;
if (broadcaster) {
broadcaster.messageUpdated({
id: newMessage.id, id: newMessage.id,
edited_content: getDisplayContent(newMessage as Message), edited_content: getDisplayContent(newMessage as Message),
edited_at: editedAt, edited_at: editedAt,
@@ -163,13 +161,14 @@ export function registerMessageCapture(client: Client): void {
if (!message.author) return; if (!message.author) return;
try { try {
const { updateMessageAsDeleted } = await import("./messageStore");
const deletedAt = Date.now(); const deletedAt = Date.now();
await updateMessageAsDeleted(message.id, deletedAt); await updateMessageAsDeleted(message.id, deletedAt);
const broadcaster = globalThis as any; const broadcaster = (globalThis as any).moderationBroadcaster as
if (broadcaster.broadcastMessageDeleted) { | ModerationBroadcaster
broadcaster.broadcastMessageDeleted({ | undefined;
if (broadcaster) {
broadcaster.messageDeleted({
id: message.id, id: message.id,
deleted_at: deletedAt, deleted_at: deletedAt,
}); });

View File

@@ -1,11 +1,39 @@
import { and, asc, desc, eq, isNull, or } from "drizzle-orm"; import { and, asc, desc, eq, isNull, or, sql } from "drizzle-orm";
import { getDatabase } from "../database/drizzle"; import { getDatabase } from "../database/drizzle";
import { attachmentsTable, messagesTable } from "../database/schema"; import { attachmentsTable, messagesTable } from "../database/schema";
import { createChildLogger } from "../logger"; import { createChildLogger } from "../logger";
import type { AttachmentRecord, MessageRecord } from "./types"; import type {
AttachmentRecord,
MessageQuery,
MessageRecord,
PageResult,
} from "./types";
const logger = createChildLogger("message-store"); const logger = createChildLogger("message-store");
// Cursor helpers for pagination
interface CursorData {
created_at: number;
id: string;
}
export function encodeCursor(data: CursorData): string {
return Buffer.from(JSON.stringify(data)).toString("base64");
}
export function decodeCursor(cursor?: string): CursorData | null {
if (!cursor) return null;
try {
const data = JSON.parse(Buffer.from(cursor, "base64").toString("utf-8"));
if (typeof data.created_at === "number" && typeof data.id === "string") {
return data;
}
return null;
} catch {
return null;
}
}
export async function insertMessage(message: MessageRecord): Promise<void> { export async function insertMessage(message: MessageRecord): Promise<void> {
try { try {
const db = getDatabase() as any; const db = getDatabase() as any;
@@ -27,6 +55,40 @@ export async function insertMessage(message: MessageRecord): Promise<void> {
} }
} }
export async function upsertMessageForCapture(
message: MessageRecord,
): Promise<void> {
try {
const db = getDatabase() as any;
// Set ai_status to pending for new or recaptured/edited text
const messageWithAIStatus = {
...message,
ai_status: "pending" as const,
};
// Try insert first (fast path for new messages)
await db
.insert(messagesTable)
.values(messageWithAIStatus)
.onConflictDoNothing();
logger.debug(
{ messageId: message.id, channelId: message.channel_id },
"Message upserted for capture",
);
} catch (error) {
logger.error(
{
messageId: message.id,
error: error instanceof Error ? error.message : String(error),
},
"Failed to upsert message for capture",
);
throw error;
}
}
export async function updateMessageAsEdited( export async function updateMessageAsEdited(
messageId: string, messageId: string,
editedContent: string, editedContent: string,
@@ -40,6 +102,13 @@ export async function updateMessageAsEdited(
edited_content: editedContent, edited_content: editedContent,
edited_at: editedAt, edited_at: editedAt,
type: "edited", type: "edited",
ai_status: "pending",
ai_moderation_flags: null,
ai_moderation_score: null,
ai_moderation_raw: null,
ai_analysis: null,
ai_analyzed_at: null,
ai_error: null,
}) })
.where(eq(messagesTable.id, messageId)); .where(eq(messagesTable.id, messageId));
@@ -327,3 +396,225 @@ export async function getMessageById(
throw error; throw error;
} }
} }
export async function listMessages(
query: MessageQuery,
): Promise<PageResult<MessageRecord>> {
try {
const db = getDatabase() as any;
const conditions: any[] = [];
// Apply filters
if (query.guildId) {
conditions.push(eq(messagesTable.guild_id, query.guildId));
}
if (query.channelId) {
conditions.push(
or(
eq(messagesTable.channel_id, query.channelId),
eq(messagesTable.thread_id, query.channelId),
),
);
}
if (query.threadId) {
conditions.push(eq(messagesTable.thread_id, query.threadId));
}
if (query.userId) {
conditions.push(eq(messagesTable.user_id, query.userId));
}
if (query.status && query.status.length > 0) {
conditions.push(
or(
...query.status.map((status) => eq(messagesTable.ai_status, status)),
),
);
}
// Text search
if (query.q) {
const pattern = `%${query.q.toLowerCase()}%`;
conditions.push(sql`lower(${messagesTable.content}) like ${pattern}`);
}
// Cursor-based pagination (newest first)
if (query.cursor) {
const cursorData = decodeCursor(query.cursor);
if (cursorData) {
conditions.push(
or(
sql`${messagesTable.created_at} < ${cursorData.created_at}`,
and(
eq(messagesTable.created_at, cursorData.created_at),
sql`${messagesTable.id} < ${cursorData.id}`,
),
),
);
}
}
// Fetch limit + 1 to determine if there's a next page
const fetchLimit = query.limit + 1;
const rows = await db
.select()
.from(messagesTable)
.where(conditions.length > 0 ? and(...conditions) : undefined)
.orderBy(desc(messagesTable.created_at), desc(messagesTable.id))
.limit(fetchLimit);
const hasMore = rows.length > query.limit;
const data = rows.slice(0, query.limit) as MessageRecord[];
let nextCursor: string | null = null;
if (hasMore && data.length > 0) {
const lastItem = data[data.length - 1];
nextCursor = encodeCursor({
created_at: lastItem.created_at,
id: lastItem.id,
});
}
return { data, nextCursor };
} catch (error) {
logger.error(
{
query,
error: error instanceof Error ? error.message : String(error),
},
"Failed to list messages",
);
throw error;
}
}
export async function listReviewMessages(
query: Omit<MessageQuery, "status">,
): Promise<PageResult<MessageRecord>> {
return listMessages({
...query,
status: ["warn", "flagged", "error"],
});
}
export async function getConversationContextBefore(input: {
channelId: string;
threadId: string | null;
beforeCreatedAt: number;
limit: number;
}): Promise<MessageRecord[]> {
try {
const db = getDatabase() as any;
const { channelId, threadId, beforeCreatedAt, limit } = input;
// Query same thread if threadId exists, otherwise channelId
const locationCondition = threadId
? eq(messagesTable.thread_id, threadId)
: eq(messagesTable.channel_id, channelId);
const rows = await db
.select()
.from(messagesTable)
.where(
and(
locationCondition,
sql`${messagesTable.created_at} < ${beforeCreatedAt}`,
isNull(messagesTable.deleted_at),
),
)
.orderBy(desc(messagesTable.created_at))
.limit(limit);
// Return in chronological order (oldest first)
return (rows as MessageRecord[]).reverse();
} catch (error) {
logger.error(
{
channelId: input.channelId,
threadId: input.threadId,
error: error instanceof Error ? error.message : String(error),
},
"Failed to get conversation context before",
);
throw error;
}
}
export async function getPendingMessagesByConversation(
conversationKey: string,
limit: number = 25,
): Promise<MessageRecord[]> {
try {
const db = getDatabase() as any;
// conversationKey is either thread_id or channel_id
// Query both to safely handle the key
const rows = await db
.select()
.from(messagesTable)
.where(
and(
or(
eq(messagesTable.thread_id, conversationKey),
eq(messagesTable.channel_id, conversationKey),
),
eq(messagesTable.ai_status, "pending"),
isNull(messagesTable.deleted_at),
),
)
.orderBy(asc(messagesTable.created_at))
.limit(limit);
return rows as MessageRecord[];
} catch (error) {
logger.error(
{
conversationKey,
error: error instanceof Error ? error.message : String(error),
},
"Failed to get pending messages by conversation",
);
throw error;
}
}
export async function getPendingConversationKeys(
limit: number = 100,
): Promise<string[]> {
try {
const db = getDatabase() as any;
// Get distinct conversation keys (thread_id or channel_id) for pending messages
const rows = await db
.selectDistinct({
thread_id: messagesTable.thread_id,
channel_id: messagesTable.channel_id,
})
.from(messagesTable)
.where(
and(
eq(messagesTable.ai_status, "pending"),
isNull(messagesTable.deleted_at),
),
)
.limit(limit);
const keys: string[] = [];
for (const row of rows as any[]) {
const key = row.thread_id || row.channel_id;
if (key && !keys.includes(key)) {
keys.push(key);
}
}
return keys;
} catch (error) {
logger.error(
{ error: error instanceof Error ? error.message : String(error) },
"Failed to get pending conversation keys",
);
throw error;
}
}

View File

@@ -1,3 +1,9 @@
import type { ModerationBroadcaster } from "./broadcaster";
export type AIStatus = "pending" | "clean" | "warn" | "flagged" | "error";
export type { ModerationBroadcaster };
export interface MessageRecord { export interface MessageRecord {
id: string; id: string;
guild_id: string; guild_id: string;
@@ -13,7 +19,7 @@ export interface MessageRecord {
deleted_at: number | null; deleted_at: number | null;
type: "text" | "edited" | "deleted"; type: "text" | "edited" | "deleted";
metadata: string | null; metadata: string | null;
ai_status?: "pending" | "clean" | "warn" | "flagged" | "error" | null; ai_status?: AIStatus | null;
ai_moderation_flags?: string | null; ai_moderation_flags?: string | null;
ai_moderation_score?: number | null; ai_moderation_score?: number | null;
ai_moderation_raw?: string | null; ai_moderation_raw?: string | null;
@@ -61,3 +67,43 @@ export interface DashboardMessage {
created_at: number; created_at: number;
type: "text" | "image" | "voice"; type: "text" | "image" | "voice";
} }
export interface MessageQuery {
guildId?: string;
channelId?: string;
threadId?: string;
status?: AIStatus[];
userId?: string;
q?: string;
cursor?: string;
limit: number;
}
export interface PageResult<T> {
data: T[];
nextCursor: string | null;
}
export interface AnalysisResult {
messageId: string;
status: Exclude<AIStatus, "pending" | "error">;
flags: string[];
score: number;
analysis: string;
}
export type ModerationWsEvent =
| { type: "ui_state"; state: unknown }
| { type: "user_state"; users: unknown[] }
| { type: "message_created"; data: MessageRecord }
| { type: "message_updated"; data: Partial<MessageRecord> & { id: string } }
| { type: "message_deleted"; data: { id: string; deleted_at: number } }
| { type: "message_analyzed"; data: MessageRecord }
| { type: "attachment_created"; data: AttachmentRecord }
| { type: "analysis_queue_status"; data: AnalysisQueueStatus };
export interface AnalysisQueueStatus {
queuedConversations: number;
activeRequests: number;
lastError: string | null;
}

View File

@@ -0,0 +1,57 @@
import type { Router } from "express";
import express from "express";
import { AppError } from "../errors";
import { createChildLogger } from "../logger";
import {
getAnalysisQueueStatus,
queueMessageAnalysis,
} from "../moderation/aiAnalyzer";
import { getMessageById } from "../moderation/messageStore";
const logger = createChildLogger("analysis-routes");
export function createAnalysisRoutes(): Router {
const router = express.Router();
// GET /api/analysis/status - Get current analysis queue status
router.get("/analysis/status", (_req, res, next) => {
try {
const status = getAnalysisQueueStatus();
res.json(status);
} catch (error) {
next(error);
}
});
// POST /api/messages/:id/reanalyze - Queue a message for re-analysis
router.post("/messages/:id/reanalyze", async (req, res, next) => {
try {
const { id } = req.params;
if (!id) {
throw new AppError("Message ID is required", "MISSING_MESSAGE_ID", 400);
}
// Verify message exists
const message = await getMessageById(id);
if (!message) {
throw new AppError("Message not found", "MESSAGE_NOT_FOUND", 404);
}
// Queue for analysis
await queueMessageAnalysis(id);
logger.info({ messageId: id }, "Message queued for re-analysis");
res.json({
success: true,
messageId: id,
queued: true,
});
} catch (error) {
next(error);
}
});
return router;
}

222
src/routes/messageRoutes.ts Normal file
View File

@@ -0,0 +1,222 @@
import type { Router } from "express";
import express from "express";
import { AppError } from "../errors";
import {
getAttachmentsByChannel,
getMessageById,
getMessagesByChannel,
listMessages,
listReviewMessages,
} from "../moderation/messageStore";
import type { AIStatus, MessageQuery } from "../moderation/types";
const aiStatuses = new Set<AIStatus>([
"pending",
"clean",
"warn",
"flagged",
"error",
]);
function parseStatuses(value?: string): AIStatus[] | undefined {
if (!value) return undefined;
return value
.split(",")
.filter((item): item is AIStatus => aiStatuses.has(item as AIStatus));
}
export function createMessageRoutes(): Router {
const router = express.Router();
// GET /api/messages - List messages by channel (backward compatible)
// Query params: channel (required), type (text|image), limit, offset
// Also supports new params: channelId, status, cursor, limit
router.get("/messages", async (req, res, next) => {
try {
const {
channel,
channelId,
type,
limit = "50",
offset = "0",
status,
cursor,
} = req.query as {
channel?: string;
channelId?: string;
type?: string;
limit?: string;
offset?: string;
status?: string;
cursor?: string;
};
const targetChannel = channelId || channel;
const limitNum = Math.min(parseInt(limit) || 50, 100);
const offsetNum = parseInt(offset) || 0;
if (type === "image") {
if (!targetChannel) {
throw new AppError(
"channel or channelId query parameter is required",
"MISSING_CHANNEL",
400,
);
}
const attachments = await getAttachmentsByChannel(
targetChannel,
limitNum,
offsetNum,
);
res.json({
type: "image",
data: attachments,
count: attachments.length,
nextCursor: null,
});
} else if (channelId || cursor || status) {
const result = await listMessages({
channelId: targetChannel,
cursor,
limit: limitNum,
status: parseStatuses(status),
});
res.json({
type: "text",
data: result.data,
count: result.data.length,
nextCursor: result.nextCursor,
});
} else {
if (!targetChannel) {
throw new AppError(
"channel or channelId query parameter is required",
"MISSING_CHANNEL",
400,
);
}
const messages = await getMessagesByChannel(
targetChannel,
limitNum,
offsetNum,
);
res.json({
type: "text",
data: messages,
count: messages.length,
nextCursor: null,
});
}
} catch (error) {
next(error);
}
});
// GET /api/messages/:id - Get a specific message
router.get("/messages/:id", async (req, res, next) => {
try {
const { id } = req.params;
if (!id) {
throw new AppError("Message ID is required", "MISSING_MESSAGE_ID", 400);
}
const message = await getMessageById(id);
if (!message) {
throw new AppError("Message not found", "MESSAGE_NOT_FOUND", 404);
}
res.json(message);
} catch (error) {
next(error);
}
});
// GET /api/review - List messages flagged for review
// Query params: guildId, channelId, threadId, userId, cursor, limit
router.get("/review", async (req, res, next) => {
try {
const {
guildId,
channelId,
threadId,
userId,
cursor,
limit = "50",
} = req.query as {
guildId?: string;
channelId?: string;
threadId?: string;
userId?: string;
cursor?: string;
limit?: string;
};
const limitNum = Math.min(parseInt(limit) || 50, 100);
const query: Omit<MessageQuery, "status"> = {
guildId,
channelId,
threadId,
userId,
cursor,
limit: limitNum,
};
const result = await listReviewMessages(query);
res.json({
data: result.data,
nextCursor: result.nextCursor,
count: result.data.length,
});
} catch (error) {
next(error);
}
});
// GET /api/attachments - List attachments by channel
// Query params: channel (required), limit, offset
router.get("/attachments", async (req, res, next) => {
try {
const {
channel,
limit = "50",
offset = "0",
} = req.query as {
channel?: string;
limit?: string;
offset?: string;
};
if (!channel) {
throw new AppError(
"channel query parameter is required",
"MISSING_CHANNEL",
400,
);
}
const limitNum = Math.min(parseInt(limit) || 50, 100);
const offsetNum = parseInt(offset) || 0;
const attachments = await getAttachmentsByChannel(
channel,
limitNum,
offsetNum,
);
res.json({
data: attachments,
count: attachments.length,
});
} catch (error) {
next(error);
}
});
return router;
}

53
src/routes/syncRoutes.ts Normal file
View File

@@ -0,0 +1,53 @@
import type { Client } from "discord.js-selfbot-v13";
import type { Router } from "express";
import express from "express";
import { AppError } from "../errors";
import { createChildLogger } from "../logger";
import { syncSelectedChannelBacklog } from "../moderation/backlogSync";
const logger = createChildLogger("sync-routes");
export function createSyncRoutes(client: Client): Router {
const router = express.Router();
// POST /api/backlog-sync - Sync message backlog for a channel
router.post("/backlog-sync", async (req, res, next) => {
try {
const { guildId, channelId } = req.body as {
guildId?: string;
channelId?: string;
};
if (!guildId || !channelId) {
throw new AppError(
"guildId and channelId are required",
"MISSING_BACKLOG_PARAMS",
400,
);
}
logger.info({ guildId, channelId }, "Starting backlog sync");
const count = await syncSelectedChannelBacklog(
client,
guildId,
channelId,
);
logger.info(
{ guildId, channelId, messagesSync: count },
"Backlog sync complete",
);
res.json({
success: true,
channelId,
messagesSync: count,
});
} catch (error) {
next(error);
}
});
return router;
}

View File

@@ -0,0 +1,47 @@
import type { Router } from "express";
import express from "express";
import { createChildLogger } from "../logger";
const logger = createChildLogger("ui-state-routes");
export interface SharedUIState {
selectedGuild: string;
selectedVoiceChannel: string;
selectedTextChannel: string;
activeTab: "voice" | "text";
isListening: boolean;
isStreaming: boolean;
}
export interface UIStateRouteOptions {
getSharedUIState: () => SharedUIState;
patchSharedUIState: (patch: Partial<SharedUIState>) => SharedUIState;
}
export function createUIStateRoutes(options: UIStateRouteOptions): Router {
const router = express.Router();
const { getSharedUIState, patchSharedUIState } = options;
// GET /api/ui-state - Get current UI state
router.get("/ui-state", (_req, res, next) => {
try {
const state = getSharedUIState();
res.json(state);
} catch (error) {
next(error);
}
});
// POST /api/ui-state - Update UI state
router.post("/ui-state", (req, res, next) => {
try {
const patch = req.body as Partial<SharedUIState>;
const updated = patchSharedUIState(patch);
res.json(updated);
} catch (error) {
next(error);
}
});
return router;
}

166
src/routes/voiceRoutes.ts Normal file
View File

@@ -0,0 +1,166 @@
import type { Router } from "express";
import express from "express";
import { AppError } from "../errors";
import { createChildLogger } from "../logger";
import type { ModerationBroadcaster } from "../moderation/broadcaster";
import type { VoiceController } from "../voiceController";
import type { SharedUIState } from "./uiStateRoutes";
const logger = createChildLogger("voice-routes");
export interface VoiceRouteOptions {
voiceController: VoiceController;
patchSharedUIState: (patch: Partial<SharedUIState>) => SharedUIState;
broadcaster: ModerationBroadcaster;
}
export function createVoiceRoutes(
options: VoiceRouteOptions | VoiceController,
): Router {
const router = express.Router();
// Support both old signature (VoiceController) and new signature (options object)
let voiceController: VoiceController;
let patchSharedUIState:
| ((patch: Partial<SharedUIState>) => SharedUIState)
| undefined;
let broadcaster: ModerationBroadcaster | undefined;
if ("connect" in options && "disconnect" in options) {
// Old signature: just VoiceController
voiceController = options as VoiceController;
} else {
// New signature: options object
const opts = options as VoiceRouteOptions;
voiceController = opts.voiceController;
patchSharedUIState = opts.patchSharedUIState;
broadcaster = opts.broadcaster;
}
// GET /api/status - Get voice connection status
router.get("/status", (_req, res, next) => {
try {
const status = voiceController.getStatus();
res.json(status);
} catch (error) {
next(error);
}
});
// GET /api/guilds - List available guilds
router.get("/guilds", (_req, res, next) => {
try {
const guilds = voiceController.listGuilds();
res.json(guilds);
} catch (error) {
next(error);
}
});
// GET /api/guilds/:guildId/voice-channels - List voice channels in a guild
router.get("/guilds/:guildId/voice-channels", async (req, res, next) => {
try {
const { guildId } = req.params;
if (!guildId) {
throw new AppError("Guild ID is required", "MISSING_GUILD_ID", 400);
}
const channels = await voiceController.listVoiceChannels(guildId);
res.json(channels);
} catch (error) {
next(error);
}
});
// GET /api/guilds/:guildId/channels - List text channels in a guild
router.get("/guilds/:guildId/channels", async (req, res, next) => {
try {
const { guildId } = req.params;
if (!guildId) {
throw new AppError("Guild ID is required", "MISSING_GUILD_ID", 400);
}
const channels = await voiceController.listWatchableChannels(guildId);
res.json(channels);
} catch (error) {
next(error);
}
});
// GET /api/guilds/:guildId/threads - List threads in a guild
router.get("/guilds/:guildId/threads", async (req, res, next) => {
try {
const { guildId } = req.params;
if (!guildId) {
throw new AppError("Guild ID is required", "MISSING_GUILD_ID", 400);
}
const threads = await voiceController.listThreads(guildId);
res.json(threads);
} catch (error) {
next(error);
}
});
// POST /api/connect - Connect to a voice channel
router.post("/connect", async (req, res, next) => {
try {
const { guildId, channelId } = req.body as {
guildId?: string;
channelId?: string;
};
if (!guildId || !channelId) {
throw new AppError(
"guildId and channelId are required",
"MISSING_CONNECT_FIELDS",
400,
);
}
logger.info({ guildId, channelId }, "Connecting to voice channel");
const status = await voiceController.connect(guildId, channelId);
// Update UI state and broadcast to connected clients
if (patchSharedUIState && broadcaster) {
const updatedState = patchSharedUIState({
selectedGuild: guildId,
selectedVoiceChannel: channelId,
});
broadcaster.uiState(updatedState);
}
res.json(status);
} catch (error) {
next(error);
}
});
// POST /api/disconnect - Disconnect from voice channel
router.post("/disconnect", async (_req, res, next) => {
try {
logger.info("Disconnecting from voice channel");
const status = await voiceController.disconnect();
// Update UI state and broadcast to connected clients
if (patchSharedUIState && broadcaster) {
const updatedState = patchSharedUIState({
selectedGuild: "",
selectedVoiceChannel: "",
});
broadcaster.uiState(updatedState);
}
res.json(status);
} catch (error) {
next(error);
}
});
return router;
}

View File

@@ -1,25 +1,22 @@
import type { Client } from "discord.js-selfbot-v13"; import type { Client } from "discord.js-selfbot-v13";
import express from "express"; import express from "express";
import fs from "fs";
import helmet from "helmet"; import helmet from "helmet";
import http from "http"; import http from "http";
import path from "path"; import path from "path";
import * as prism from "prism-media"; import * as prism from "prism-media";
import { WebSocketServer } from "ws"; import { WebSocketServer } from "ws";
import { getDatabase } from "./database/drizzle";
import { AppError } from "./errors"; import { AppError } from "./errors";
import { createChildLogger, logger } from "./logger"; import { createChildLogger, logger } from "./logger";
import { getMetrics, uptimeGauge } from "./metrics"; import { getMetrics, uptimeGauge } from "./metrics";
import { syncSelectedChannelBacklog } from "./moderation/backlogSync"; import { createBroadcaster } from "./moderation/broadcaster";
import { import { getPersistedValue, setPersistedValue } from "./muxer-queue";
getAttachmentsByChannel,
getMessagesByChannel,
} from "./moderation/messageStore";
import {
getDatabase as getMuxerDatabase,
getPersistedValue,
setPersistedValue,
} from "./muxer-queue";
import { discordPlayer } from "./player"; import { discordPlayer } from "./player";
import { createAnalysisRoutes } from "./routes/analysisRoutes";
import { createMessageRoutes } from "./routes/messageRoutes";
import { createSyncRoutes } from "./routes/syncRoutes";
import { createUIStateRoutes } from "./routes/uiStateRoutes";
import { createVoiceRoutes } from "./routes/voiceRoutes";
import type { VoiceController } from "./voiceController"; import type { VoiceController } from "./voiceController";
const wsLogger = createChildLogger("webserver"); const wsLogger = createChildLogger("webserver");
@@ -28,7 +25,6 @@ const activeUsers = new Map<
string, string,
{ username: string; avatar: string; speaking: boolean } { username: string; avatar: string; speaking: boolean }
>(); >();
let wsClients = new Set<any>();
interface SharedUIState { interface SharedUIState {
selectedGuild: string; selectedGuild: string;
@@ -58,16 +54,6 @@ function getSharedUIState(): SharedUIState {
return { ...sharedUIState }; return { ...sharedUIState };
} }
function broadcastUIState() {
const payload = JSON.stringify({
type: "ui_state",
state: getSharedUIState(),
});
wsClients.forEach((client) => {
if (client.readyState === 1) client.send(payload);
});
}
function patchSharedUIState(patch: Partial<SharedUIState>) { function patchSharedUIState(patch: Partial<SharedUIState>) {
if (typeof patch.selectedGuild === "string") { if (typeof patch.selectedGuild === "string") {
sharedUIState.selectedGuild = patch.selectedGuild; sharedUIState.selectedGuild = patch.selectedGuild;
@@ -88,7 +74,6 @@ function patchSharedUIState(patch: Partial<SharedUIState>) {
sharedUIState.isStreaming = patch.isStreaming; sharedUIState.isStreaming = patch.isStreaming;
} }
setPersistedValue("web-ui-state", sharedUIState); setPersistedValue("web-ui-state", sharedUIState);
broadcastUIState();
return getSharedUIState(); return getSharedUIState();
} }
@@ -131,6 +116,10 @@ export async function startWebserver(
const wss = new WebSocketServer({ server, path: wsPath }); const wss = new WebSocketServer({ server, path: wsPath });
wsLogger.info({ port, wsPath }, "WebSocket server listening"); wsLogger.info({ port, wsPath }, "WebSocket server listening");
// Create broadcaster instance
const broadcaster = createBroadcaster();
(globalThis as any).moderationBroadcaster = broadcaster;
// Security headers. CSP disabled because the current static UI uses inline scripts/styles. // Security headers. CSP disabled because the current static UI uses inline scripts/styles.
app.use( app.use(
helmet({ helmet({
@@ -163,7 +152,12 @@ export async function startWebserver(
app.use(express.static(path.join(__dirname, "../public"))); app.use(express.static(path.join(__dirname, "../public")));
app.get("/", (_req, res) => { app.get("/", (_req, res) => {
res.sendFile(path.join(__dirname, "../public/index.html")); const reactIndex = path.join(__dirname, "../public/app/index.html");
if (fs.existsSync(reactIndex)) {
res.sendFile(reactIndex);
} else {
res.sendFile(path.join(__dirname, "../public/index.html"));
}
}); });
// Health check endpoint // Health check endpoint
@@ -173,7 +167,7 @@ export async function startWebserver(
timestamp: new Date().toISOString(), timestamp: new Date().toISOString(),
uptime: process.uptime(), uptime: process.uptime(),
activeUsers: activeUsers.size, activeUsers: activeUsers.size,
wsClients: wsClients.size, wsClients: broadcaster.clientCount(),
}); });
}); });
@@ -184,163 +178,22 @@ export async function startWebserver(
res.send(await getMetrics()); res.send(await getMetrics());
}); });
app.get("/api/status", (_req, res) => { // Register route modules
res.json(voiceController.getStatus()); app.use(
}); "/api",
createUIStateRoutes({ getSharedUIState, patchSharedUIState }),
app.get("/api/ui-state", (_req, res) => { );
res.json(getSharedUIState()); app.use(
}); "/api",
createVoiceRoutes({
app.post("/api/ui-state", (req, res) => { voiceController,
res.json(patchSharedUIState(req.body as Partial<SharedUIState>)); patchSharedUIState,
}); broadcaster,
}),
app.get("/api/guilds", (_req, res) => { );
res.json(voiceController.listGuilds()); app.use("/api", createMessageRoutes());
}); app.use("/api", createAnalysisRoutes());
app.use("/api", createSyncRoutes(_client));
app.get("/api/guilds/:guildId/voice-channels", async (req, res, next) => {
try {
res.json(await voiceController.listVoiceChannels(req.params.guildId));
} catch (error) {
next(error);
}
});
app.get("/api/guilds/:guildId/channels", async (req, res, next) => {
try {
res.json(await voiceController.listWatchableChannels(req.params.guildId));
} catch (error) {
next(error);
}
});
app.get("/api/guilds/:guildId/threads", async (req, res, next) => {
try {
res.json(await voiceController.listThreads(req.params.guildId));
} catch (error) {
next(error);
}
});
app.post("/api/connect", async (req, res, next) => {
try {
const { guildId, channelId } = req.body as {
guildId?: string;
channelId?: string;
};
if (!guildId || !channelId) {
throw new AppError(
"guildId and channelId are required",
"MISSING_CONNECT_FIELDS",
400,
);
}
const status = await voiceController.connect(guildId, channelId);
patchSharedUIState({
selectedGuild: guildId,
selectedVoiceChannel: channelId,
});
res.json(status);
} catch (error) {
next(error);
}
});
app.post("/api/disconnect", async (_req, res, next) => {
try {
res.json(await voiceController.disconnect());
} catch (error) {
next(error);
}
});
// Moderation API endpoints
app.get("/api/messages", async (req, res, next) => {
try {
const {
channel,
type,
limit = "50",
offset = "0",
} = req.query as {
channel?: string;
type?: string;
limit?: string;
offset?: string;
};
if (!channel) {
throw new AppError(
"channel query parameter is required",
"MISSING_CHANNEL",
400,
);
}
const limitNum = Math.min(parseInt(limit) || 50, 100);
const offsetNum = parseInt(offset) || 0;
if (type === "image") {
const attachments = await getAttachmentsByChannel(
channel,
limitNum,
offsetNum,
);
res.json({
type: "image",
data: attachments,
count: attachments.length,
});
} else {
const messages = await getMessagesByChannel(
channel,
limitNum,
offsetNum,
);
res.json({
type: "text",
data: messages,
count: messages.length,
});
}
} catch (error) {
next(error);
}
});
app.post("/api/backlog-sync", async (req, res, next) => {
try {
const { guildId, channelId } = req.body as {
guildId?: string;
channelId?: string;
};
if (!guildId || !channelId) {
throw new AppError(
"guildId and channelId are required",
"MISSING_BACKLOG_PARAMS",
400,
);
}
const count = await syncSelectedChannelBacklog(
_client,
guildId,
channelId,
);
res.json({
success: true,
channelId,
messagesSync: count,
});
} catch (error) {
next(error);
}
});
// Inbound: Discord PCM → tagged chunks → browser // Inbound: Discord PCM → tagged chunks → browser
(global as any).broadcastPcmToWeb = (chunk: Buffer, userId: string) => { (global as any).broadcastPcmToWeb = (chunk: Buffer, userId: string) => {
@@ -352,9 +205,9 @@ export async function startWebserver(
const header = Buffer.alloc(4); const header = Buffer.alloc(4);
header.writeInt32LE(hash, 0); header.writeInt32LE(hash, 0);
const packet = Buffer.concat([header, chunk]); const packet = Buffer.concat([header, chunk]);
wsClients.forEach((client) => { for (const client of broadcaster.getClients()) {
if (client.readyState === 1) client.send(packet); if (client.readyState === 1) client.send(packet);
}); }
}; };
(global as any).updateActiveUser = ( (global as any).updateActiveUser = (
@@ -366,49 +219,13 @@ export async function startWebserver(
}; };
function broadcastUserState() { function broadcastUserState() {
const payload = JSON.stringify({ const users = Array.from(activeUsers.entries()).map(([id, data]) => ({
type: "user_state", id,
users: Array.from(activeUsers.entries()).map(([id, data]) => ({ ...data,
id, }));
...data, broadcaster.userState(users);
})),
});
wsClients.forEach((client) => {
if (client.readyState === 1) client.send(payload);
});
} }
function broadcastMessageEvent(type: string, data: any) {
const payload = JSON.stringify({
type,
data,
timestamp: Date.now(),
});
wsClients.forEach((client) => {
if (client.readyState === 1) client.send(payload);
});
}
(global as any).broadcastMessageCreated = (data: any) => {
broadcastMessageEvent("message_created", data);
};
(global as any).broadcastMessageUpdated = (data: any) => {
broadcastMessageEvent("message_updated", data);
};
(global as any).broadcastMessageDeleted = (data: any) => {
broadcastMessageEvent("message_deleted", data);
};
(global as any).broadcastAttachmentUploaded = (data: any) => {
broadcastMessageEvent("attachment_uploaded", data);
};
(global as any).broadcastMessageAnalyzed = (data: any) => {
broadcastMessageEvent("message_analyzed", data);
};
// --- Outbound: browser PCM (24kHz mono) → Opus → Discord --- // --- Outbound: browser PCM (24kHz mono) → Opus → Discord ---
const RATE = 48000; const RATE = 48000;
const CHANNELS = 2; const CHANNELS = 2;
@@ -497,7 +314,7 @@ export async function startWebserver(
wss.on("connection", (ws) => { wss.on("connection", (ws) => {
wsLogger.info({ port, wsPath }, "New WebSocket connection"); wsLogger.info({ port, wsPath }, "New WebSocket connection");
wsClients.add(ws); broadcaster.addClient(ws);
ws.send( ws.send(
JSON.stringify({ JSON.stringify({
@@ -524,10 +341,10 @@ export async function startWebserver(
}); });
ws.on("close", () => { ws.on("close", () => {
wsClients.delete(ws); broadcaster.removeClient(ws);
}); });
ws.on("error", () => { ws.on("error", () => {
wsClients.delete(ws); broadcaster.removeClient(ws);
}); });
}); });

View File

@@ -0,0 +1,46 @@
import { describe, expect, it } from "vitest";
import {
getConversationKey,
pickBatchWithinBudget,
} from "../../src/moderation/aiAnalyzer";
import type { MessageRecord } from "../../src/moderation/types";
function message(
id: string,
content: string,
thread_id: string | null = null,
): MessageRecord {
return {
id,
guild_id: "g1",
channel_id: "c1",
thread_id,
user_id: "u1",
username: "u1",
avatar_url: null,
content,
edited_content: null,
created_at: Number(id.replace("m", "")) || 1,
edited_at: null,
deleted_at: null,
type: "text",
metadata: null,
ai_status: "pending",
};
}
describe("analysis queue helpers", () => {
it("uses thread id before channel id", () => {
expect(getConversationKey(message("m1", "hello", "t1"))).toBe("t1");
expect(getConversationKey(message("m1", "hello", null))).toBe("c1");
});
it("picks batch within budget", () => {
const batch = pickBatchWithinBudget(
[message("m1", "a"), message("m2", "x".repeat(1000))],
50,
10,
);
expect(batch.map((item) => item.id)).toEqual(["m1"]);
});
});

View File

@@ -0,0 +1,116 @@
import { describe, expect, it, vi } from "vitest";
import { createBroadcaster } from "../../src/moderation/broadcaster";
function client() {
return { readyState: 1, send: vi.fn() };
}
describe("createBroadcaster", () => {
it("sends JSON events to open clients", () => {
const ws = client();
const broadcaster = createBroadcaster();
broadcaster.addClient(ws as any);
broadcaster.messageAnalyzed({ id: "m1", ai_status: "clean" } as any);
expect(ws.send).toHaveBeenCalledTimes(1);
expect(JSON.parse(ws.send.mock.calls[0][0])).toMatchObject({
type: "message_analyzed",
data: { id: "m1", ai_status: "clean" },
});
});
it("skips closed clients", () => {
const ws = { readyState: 3, send: vi.fn() };
const broadcaster = createBroadcaster();
broadcaster.addClient(ws as any);
broadcaster.messageDeleted({ id: "m1", deleted_at: 123 });
expect(ws.send).not.toHaveBeenCalled();
});
it("broadcasts to multiple open clients", () => {
const ws1 = client();
const ws2 = client();
const ws3 = client();
const broadcaster = createBroadcaster();
broadcaster.addClient(ws1 as any);
broadcaster.addClient(ws2 as any);
broadcaster.addClient(ws3 as any);
broadcaster.messageCreated({
id: "m1",
content: "test",
} as any);
expect(ws1.send).toHaveBeenCalledTimes(1);
expect(ws2.send).toHaveBeenCalledTimes(1);
expect(ws3.send).toHaveBeenCalledTimes(1);
});
it("failed send on one client does not prevent another client from receiving event", () => {
const ws1 = client();
const ws2 = client();
const ws3 = client();
const broadcaster = createBroadcaster();
// ws1 throws on send
ws1.send.mockImplementation(() => {
throw new Error("Send failed");
});
broadcaster.addClient(ws1 as any);
broadcaster.addClient(ws2 as any);
broadcaster.addClient(ws3 as any);
broadcaster.messageUpdated({
id: "m1",
content: "updated",
} as any);
// ws1 attempted send (threw)
expect(ws1.send).toHaveBeenCalledTimes(1);
// ws2 and ws3 should still receive the event
expect(ws2.send).toHaveBeenCalledTimes(1);
expect(ws3.send).toHaveBeenCalledTimes(1);
});
it("clientCount tracks add/remove", () => {
const ws1 = client();
const ws2 = client();
const broadcaster = createBroadcaster();
expect(broadcaster.clientCount()).toBe(0);
broadcaster.addClient(ws1 as any);
expect(broadcaster.clientCount()).toBe(1);
broadcaster.addClient(ws2 as any);
expect(broadcaster.clientCount()).toBe(2);
broadcaster.removeClient(ws1 as any);
expect(broadcaster.clientCount()).toBe(1);
broadcaster.removeClient(ws2 as any);
expect(broadcaster.clientCount()).toBe(0);
});
it("payload includes numeric timestamp", () => {
const ws = client();
const broadcaster = createBroadcaster();
broadcaster.addClient(ws as any);
broadcaster.attachmentCreated({
id: "a1",
message_id: "m1",
} as any);
expect(ws.send).toHaveBeenCalledTimes(1);
const payload = JSON.parse(ws.send.mock.calls[0][0]);
expect(payload.timestamp).toBeDefined();
expect(typeof payload.timestamp).toBe("number");
expect(payload.timestamp).toBeGreaterThan(0);
});
});

View File

@@ -0,0 +1,146 @@
import { describe, expect, it } from "vitest";
import { buildConversationPromptMessages } from "../../src/moderation/conversationContext";
import type { MessageRecord } from "../../src/moderation/types";
function message(
id: string,
content: string,
created_at: number,
): MessageRecord {
return {
id,
guild_id: "g1",
channel_id: "c1",
thread_id: null,
user_id: `u-${id}`,
username: `user-${id}`,
avatar_url: null,
content,
edited_content: null,
created_at,
edited_at: null,
deleted_at: null,
type: "text",
metadata: null,
ai_status: "pending",
};
}
describe("buildConversationPromptMessages", () => {
it("marks target messages and keeps chronological order", () => {
const lines = buildConversationPromptMessages({
contextBefore: [message("a", "hello", 1)],
targets: [message("b", "bad?", 2)],
maxTokens: 1000,
});
expect(lines).toContain(
"[context] id=a time=1970-01-01T00:00:00.001Z user=user-a: hello",
);
expect(lines).toContain(
"[target] id=b time=1970-01-01T00:00:00.002Z user=user-b: bad?",
);
const indexA = lines.findIndex((line) => line.includes("id=a"));
const indexB = lines.findIndex((line) => line.includes("id=b"));
expect(indexA).toBeLessThan(indexB);
});
it("uses edited content when present", () => {
const target = message("b", "original", 2);
target.edited_content = "edited";
const lines = buildConversationPromptMessages({
contextBefore: [],
targets: [target],
maxTokens: 1000,
});
expect(lines.some((line) => line.includes("edited"))).toBe(true);
expect(lines.some((line) => line.includes("original"))).toBe(false);
});
it("empty targets returns only fitting context or empty string if no context", () => {
// Case 1: No context, no targets
const lines1 = buildConversationPromptMessages({
contextBefore: [],
targets: [],
maxTokens: 1000,
});
expect(lines1).toEqual([]);
// Case 2: Context but no targets
const lines2 = buildConversationPromptMessages({
contextBefore: [message("a", "hello", 1)],
targets: [],
maxTokens: 1000,
});
expect(lines2).toHaveLength(1);
expect(lines2[0]).toContain("[context]");
});
it("maxTokens budget includes target lines even when targets exceed budget", () => {
// Create targets that exceed budget
const longContent = "x".repeat(500); // ~125 tokens
const targets = [
message("t1", longContent, 1),
message("t2", longContent, 2),
message("t3", longContent, 3),
];
const lines = buildConversationPromptMessages({
contextBefore: [message("c1", "context", 0)],
targets,
maxTokens: 200, // Only 200 tokens, but targets alone are ~375
});
// All targets should be included
expect(lines.some((line) => line.includes("id=t1"))).toBe(true);
expect(lines.some((line) => line.includes("id=t2"))).toBe(true);
expect(lines.some((line) => line.includes("id=t3"))).toBe(true);
// Context should be excluded due to budget
expect(lines.some((line) => line.includes("id=c1"))).toBe(false);
});
it("most recent context is kept when context budget is tight", () => {
// Create multiple context messages with different timestamps
// Use longer content to ensure they consume meaningful tokens
const contextBefore = [
message(
"c1",
"oldest context with some extra words to make it longer",
1000,
),
message(
"c2",
"middle context with some extra words to make it longer",
2000,
),
message(
"c3",
"newest context with some extra words to make it longer",
3000,
),
];
const lines = buildConversationPromptMessages({
contextBefore,
targets: [message("t1", "target message", 4000)],
maxTokens: 90, // Very tight budget: target ~35 tokens, room for ~55 tokens of context (fits only c3)
});
// Should include target
expect(lines.some((line) => line.includes("id=t1"))).toBe(true);
// Should include newest context (c3) but not oldest (c1)
// With tight budget, only the most recent context should fit
expect(lines.some((line) => line.includes("id=c3"))).toBe(true);
expect(lines.some((line) => line.includes("id=c1"))).toBe(false);
// Verify chronological order is maintained
const indexT1 = lines.findIndex((line) => line.includes("id=t1"));
const indexC3 = lines.findIndex((line) => line.includes("id=c3"));
expect(indexC3).toBeLessThan(indexT1); // context before target
});
});

View File

@@ -0,0 +1,339 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import {
parseModerationResponse,
runModerationAnalysis,
} from "../../src/moderation/llmModerationClient";
import type { MessageRecord } from "../../src/moderation/types";
vi.mock("../../src/retry", () => ({
retryWithBackoff: vi.fn((fn) => fn()),
}));
/**
* Helper to create a full MessageRecord fixture with sensible defaults.
* Only override fields that differ from defaults.
*/
function createMessageRecord(
overrides: Partial<MessageRecord> = {},
): MessageRecord {
const now = Date.now();
return {
id: "m1",
guild_id: "guild123",
channel_id: "channel123",
thread_id: null,
user_id: "user123",
username: "user1",
avatar_url: null,
content: "hello",
edited_content: null,
created_at: now,
edited_at: null,
deleted_at: null,
type: "text",
metadata: null,
...overrides,
};
}
describe("parseModerationResponse", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("parses valid keyed results", () => {
const result = parseModerationResponse(
JSON.stringify({
results: [
{
message_id: "m1",
status: "warn",
flags: ["provokasi"],
score: 0.7,
analysis: "Perlu peringatan.",
},
],
}),
["m1"],
);
expect(result).toEqual([
{
messageId: "m1",
status: "warn",
flags: ["provokasi"],
score: 0.7,
analysis: "Perlu peringatan.",
},
]);
});
it("rejects missing target ids", () => {
expect(() =>
parseModerationResponse(JSON.stringify({ results: [] }), ["m1"]),
).toThrow(/missing/i);
});
it("rejects unknown ids", () => {
expect(() =>
parseModerationResponse(
JSON.stringify({
results: [
{
message_id: "m2",
status: "clean",
flags: [],
score: 0,
analysis: "OK",
},
],
}),
["m1"],
),
).toThrow(/unknown/i);
});
it("handles surrounding text around JSON", () => {
const content = `Some preamble text here.
{
"results": [
{
"message_id": "m1",
"status": "clean",
"flags": [],
"score": 0.1,
"analysis": "OK"
}
]
}
Some trailing text here.`;
const result = parseModerationResponse(content, ["m1"]);
expect(result).toHaveLength(1);
expect(result[0].messageId).toBe("m1");
});
it("handles nested fields in results", () => {
const content = JSON.stringify({
results: [
{
message_id: "m1",
status: "warn",
flags: ["spam", "abuse"],
score: 0.85,
analysis: "Multiple violations detected",
metadata: {
nested: "field",
count: 5,
},
},
],
});
const result = parseModerationResponse(content, ["m1"]);
expect(result).toHaveLength(1);
expect(result[0].score).toBe(0.85);
});
it("rejects null score", () => {
expect(() =>
parseModerationResponse(
JSON.stringify({
results: [
{
message_id: "m1",
status: "clean",
flags: [],
score: null,
analysis: "OK",
},
],
}),
["m1"],
),
).toThrow(/null or undefined/i);
});
it("rejects undefined score", () => {
expect(() =>
parseModerationResponse(
JSON.stringify({
results: [
{
message_id: "m1",
status: "clean",
flags: [],
analysis: "OK",
},
],
}),
["m1"],
),
).toThrow(/null or undefined/i);
});
it("rejects duplicate message_id", () => {
expect(() =>
parseModerationResponse(
JSON.stringify({
results: [
{
message_id: "m1",
status: "clean",
flags: [],
score: 0.1,
analysis: "OK",
},
{
message_id: "m1",
status: "warn",
flags: ["spam"],
score: 0.5,
analysis: "Duplicate",
},
],
}),
["m1"],
),
).toThrow(/duplicate/i);
});
it("rejects invalid status", () => {
expect(() =>
parseModerationResponse(
JSON.stringify({
results: [
{
message_id: "m1",
status: "invalid_status",
flags: [],
score: 0.5,
analysis: "OK",
},
],
}),
["m1"],
),
).toThrow(/invalid status/i);
});
it("clamps score to 0-1 range", () => {
const result = parseModerationResponse(
JSON.stringify({
results: [
{
message_id: "m1",
status: "clean",
flags: [],
score: 1.5,
analysis: "OK",
},
],
}),
["m1"],
);
expect(result[0].score).toBe(1);
});
it("clamps negative score to 0", () => {
const result = parseModerationResponse(
JSON.stringify({
results: [
{
message_id: "m1",
status: "clean",
flags: [],
score: -0.5,
analysis: "OK",
},
],
}),
["m1"],
);
expect(result[0].score).toBe(0);
});
});
describe("runModerationAnalysis", () => {
it("parses successful response from LLM", async () => {
const mockResponse = {
choices: [
{
message: {
content: JSON.stringify({
results: [
{
message_id: "m1",
status: "clean",
flags: [],
score: 0.1,
analysis: "OK",
},
],
}),
},
},
],
};
global.fetch = vi.fn().mockResolvedValue({
ok: true,
json: async () => mockResponse,
});
const result = await runModerationAnalysis({
targets: [createMessageRecord()],
contextText: "test context",
});
expect(result.results).toHaveLength(1);
expect(result.results[0].messageId).toBe("m1");
expect(result.raw).toEqual(mockResponse);
});
it("throws on non-ok HTTP response", async () => {
global.fetch = vi.fn().mockResolvedValue({
ok: false,
status: 500,
text: async () => "Internal Server Error",
});
await expect(
runModerationAnalysis({
targets: [createMessageRecord()],
contextText: "test context",
}),
).rejects.toThrow(/LLM API error 500/);
});
it("throws on missing choices in response", async () => {
global.fetch = vi.fn().mockResolvedValue({
ok: true,
json: async () => ({}),
});
await expect(
runModerationAnalysis({
targets: [createMessageRecord()],
contextText: "test context",
}),
).rejects.toThrow(/Invalid LLM response structure/);
});
it("throws on missing content in message", async () => {
global.fetch = vi.fn().mockResolvedValue({
ok: true,
json: async () => ({
choices: [{ message: {} }],
}),
});
await expect(
runModerationAnalysis({
targets: [createMessageRecord()],
contextText: "test context",
}),
).rejects.toThrow(/No content in LLM response/);
});
});

View File

@@ -0,0 +1,574 @@
import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest";
import {
closeDatabase,
getDatabase,
initializeDatabase,
} from "../../src/database/drizzle";
import { createChildLogger } from "../../src/logger";
import {
decodeCursor,
encodeCursor,
getMessageById,
insertMessage,
listMessages,
listReviewMessages,
updateMessageAsEdited,
} from "../../src/moderation/messageStore";
import type { MessageRecord } from "../../src/moderation/types";
const logger = createChildLogger("messageStoreQueries.test");
describe("message cursor helpers", () => {
it("round-trips created_at and id", () => {
const cursor = encodeCursor({ created_at: 1710000000000, id: "abc" });
expect(decodeCursor(cursor)).toEqual({
created_at: 1710000000000,
id: "abc",
});
});
it("returns null for invalid cursor", () => {
expect(decodeCursor("not-base64-json")).toBeNull();
});
});
describe("message query integration tests", () => {
beforeAll(async () => {
await initializeDatabase();
// Create tables using Drizzle schema (SQLite doesn't support migrations with PostgreSQL syntax)
const db = getDatabase() as any;
try {
// Create messages table
await db.run(`
CREATE TABLE IF NOT EXISTS "messages" (
"id" text PRIMARY KEY NOT NULL,
"guild_id" text NOT NULL,
"channel_id" text NOT NULL,
"thread_id" text,
"user_id" text NOT NULL,
"username" text NOT NULL,
"avatar_url" text,
"content" text NOT NULL,
"edited_content" text,
"created_at" integer NOT NULL,
"edited_at" integer,
"deleted_at" integer,
"type" text DEFAULT 'text' NOT NULL,
"metadata" text,
"ai_status" text DEFAULT 'pending' NOT NULL,
"ai_moderation_flags" text,
"ai_moderation_score" real,
"ai_moderation_raw" text,
"ai_analysis" text,
"ai_analyzed_at" integer,
"ai_error" text
)
`);
} catch (error) {
logger.debug(
{ error },
"Messages table already exists or error creating it",
);
}
});
beforeEach(async () => {
// Clear messages table before each test
try {
const db = getDatabase() as any;
await db.run(`DELETE FROM "messages"`);
} catch (error) {
logger.debug({ error }, "Could not clear messages table");
}
});
afterAll(async () => {
try {
await closeDatabase();
} catch (error) {
logger.error(
{ error: error instanceof Error ? error.message : String(error) },
"Error closing database in afterAll",
);
}
});
describe("listMessages", () => {
const createTestMessage = (
overrides: Partial<MessageRecord> = {},
): MessageRecord => ({
id: `msg-${Date.now()}-${Math.random().toString(36).slice(2)}`,
guild_id: "guild-123",
channel_id: "channel-456",
thread_id: null,
user_id: "user-789",
username: "testuser",
avatar_url: null,
content: "Test message",
edited_content: null,
created_at: Date.now(),
edited_at: null,
deleted_at: null,
type: "text",
metadata: null,
ai_status: "pending",
...overrides,
});
it("returns messages in newest-first order", async () => {
const now = Date.now();
const msg1 = createTestMessage({
id: "msg-1",
created_at: now - 3000,
content: "oldest",
});
const msg2 = createTestMessage({
id: "msg-2",
created_at: now - 2000,
content: "middle",
});
const msg3 = createTestMessage({
id: "msg-3",
created_at: now - 1000,
content: "newest",
});
await insertMessage(msg1);
await insertMessage(msg2);
await insertMessage(msg3);
const result = await listMessages({
channelId: "channel-456",
limit: 10,
});
expect(result.data).toHaveLength(3);
expect(result.data[0].id).toBe("msg-3");
expect(result.data[1].id).toBe("msg-2");
expect(result.data[2].id).toBe("msg-1");
});
it("returns nextCursor when more results exist than limit", async () => {
const now = Date.now();
const channelId = `channel-${Math.random().toString(36).slice(2)}`;
// Insert 5 messages
for (let i = 0; i < 5; i++) {
await insertMessage(
createTestMessage({
id: `msg-limit-${i}`,
channel_id: channelId,
created_at: now - i * 1000,
}),
);
}
const result = await listMessages({
channelId,
limit: 3,
});
expect(result.data).toHaveLength(3);
expect(result.nextCursor).not.toBeNull();
});
it("returns null nextCursor when all results fit within limit", async () => {
const now = Date.now();
const channelId = `channel-${Math.random().toString(36).slice(2)}`;
// Insert 2 messages
for (let i = 0; i < 2; i++) {
await insertMessage(
createTestMessage({
id: `msg-nomore-${i}`,
channel_id: channelId,
created_at: now - i * 1000,
}),
);
}
const result = await listMessages({
channelId,
limit: 10,
});
expect(result.data).toHaveLength(2);
expect(result.nextCursor).toBeNull();
});
it("second page using nextCursor does not duplicate first page", async () => {
const now = Date.now();
const channelId = `channel-${Math.random().toString(36).slice(2)}`;
// Insert 6 messages
const messageIds: string[] = [];
for (let i = 0; i < 6; i++) {
const id = `msg-dup-${i}`;
messageIds.push(id);
await insertMessage(
createTestMessage({
id,
channel_id: channelId,
created_at: now - i * 1000,
}),
);
}
// Get first page
const page1 = await listMessages({
channelId,
limit: 3,
});
expect(page1.data).toHaveLength(3);
expect(page1.nextCursor).not.toBeNull();
const page1Ids = page1.data.map((m) => m.id);
// Get second page using cursor
const page2 = await listMessages({
channelId,
limit: 3,
cursor: page1.nextCursor!,
});
expect(page2.data).toHaveLength(3);
const page2Ids = page2.data.map((m) => m.id);
// Verify no overlap
const overlap = page1Ids.filter((id) => page2Ids.includes(id));
expect(overlap).toHaveLength(0);
// Verify all messages are accounted for
const allIds = [...page1Ids, ...page2Ids];
expect(allIds.sort()).toEqual(messageIds.sort());
});
it("filters by channelId correctly", async () => {
const now = Date.now();
const channel1 = `channel-${Math.random().toString(36).slice(2)}`;
const channel2 = `channel-${Math.random().toString(36).slice(2)}`;
// Insert messages in two channels
await insertMessage(
createTestMessage({
id: "msg-ch1-1",
channel_id: channel1,
created_at: now,
}),
);
await insertMessage(
createTestMessage({
id: "msg-ch2-1",
channel_id: channel2,
created_at: now,
}),
);
const result = await listMessages({
channelId: channel1,
limit: 10,
});
expect(result.data).toHaveLength(1);
expect(result.data[0].channel_id).toBe(channel1);
});
it("filters by status correctly", async () => {
const now = Date.now();
const channelId = `channel-${Math.random().toString(36).slice(2)}`;
// Insert messages with different statuses
const msg1 = createTestMessage({
id: "msg-status-1",
channel_id: channelId,
created_at: now,
ai_status: "clean",
});
const msg2 = createTestMessage({
id: "msg-status-2",
channel_id: channelId,
created_at: now - 1000,
ai_status: "warn",
});
const msg3 = createTestMessage({
id: "msg-status-3",
channel_id: channelId,
created_at: now - 2000,
ai_status: "flagged",
});
await insertMessage(msg1);
await insertMessage(msg2);
await insertMessage(msg3);
// Query for warn and flagged only
const result = await listMessages({
channelId,
status: ["warn", "flagged"],
limit: 10,
});
expect(result.data).toHaveLength(2);
expect(result.data.map((m) => m.id).sort()).toEqual(
["msg-status-2", "msg-status-3"].sort(),
);
});
it("filters by channelId and status together", async () => {
const now = Date.now();
const channel1 = `channel-${Math.random().toString(36).slice(2)}`;
const channel2 = `channel-${Math.random().toString(36).slice(2)}`;
// Insert messages in two channels with different statuses
await insertMessage(
createTestMessage({
id: "msg-combo-1",
channel_id: channel1,
created_at: now,
ai_status: "warn",
}),
);
await insertMessage(
createTestMessage({
id: "msg-combo-2",
channel_id: channel1,
created_at: now - 1000,
ai_status: "clean",
}),
);
await insertMessage(
createTestMessage({
id: "msg-combo-3",
channel_id: channel2,
created_at: now - 2000,
ai_status: "warn",
}),
);
// Query for warn status in channel1 only
const result = await listMessages({
channelId: channel1,
status: ["warn"],
limit: 10,
});
expect(result.data).toHaveLength(1);
expect(result.data[0].id).toBe("msg-combo-1");
});
});
describe("listReviewMessages", () => {
const createTestMessage = (
overrides: Partial<MessageRecord> = {},
): MessageRecord => ({
id: `msg-${Date.now()}-${Math.random().toString(36).slice(2)}`,
guild_id: "guild-123",
channel_id: "channel-456",
thread_id: null,
user_id: "user-789",
username: "testuser",
avatar_url: null,
content: "Test message",
edited_content: null,
created_at: Date.now(),
edited_at: null,
deleted_at: null,
type: "text",
metadata: null,
ai_status: "pending",
...overrides,
});
it("defaults to warn, flagged, and error statuses", async () => {
const now = Date.now();
const channelId = `channel-${Math.random().toString(36).slice(2)}`;
// Insert messages with all statuses
await insertMessage(
createTestMessage({
id: "msg-review-1",
channel_id: channelId,
created_at: now,
ai_status: "clean",
}),
);
await insertMessage(
createTestMessage({
id: "msg-review-2",
channel_id: channelId,
created_at: now - 1000,
ai_status: "warn",
}),
);
await insertMessage(
createTestMessage({
id: "msg-review-3",
channel_id: channelId,
created_at: now - 2000,
ai_status: "flagged",
}),
);
await insertMessage(
createTestMessage({
id: "msg-review-4",
channel_id: channelId,
created_at: now - 3000,
ai_status: "error",
}),
);
await insertMessage(
createTestMessage({
id: "msg-review-5",
channel_id: channelId,
created_at: now - 4000,
ai_status: "pending",
}),
);
const result = await listReviewMessages({
channelId,
limit: 10,
});
expect(result.data).toHaveLength(3);
const ids = result.data.map((m) => m.id).sort();
expect(ids).toEqual(
["msg-review-2", "msg-review-3", "msg-review-4"].sort(),
);
});
it("excludes clean status messages", async () => {
const now = Date.now();
const channelId = `channel-${Math.random().toString(36).slice(2)}`;
await insertMessage(
createTestMessage({
id: "msg-clean-1",
channel_id: channelId,
created_at: now,
ai_status: "clean",
}),
);
await insertMessage(
createTestMessage({
id: "msg-clean-2",
channel_id: channelId,
created_at: now - 1000,
ai_status: "warn",
}),
);
const result = await listReviewMessages({
channelId,
limit: 10,
});
expect(result.data).toHaveLength(1);
expect(result.data[0].id).toBe("msg-clean-2");
});
it("respects pagination with review messages", async () => {
const now = Date.now();
const channelId = `channel-${Math.random().toString(36).slice(2)}`;
// Insert 5 review-worthy messages
for (let i = 0; i < 5; i++) {
await insertMessage(
createTestMessage({
id: `msg-review-page-${i}`,
channel_id: channelId,
created_at: now - i * 1000,
ai_status: i % 2 === 0 ? "warn" : "flagged",
}),
);
}
const page1 = await listReviewMessages({
channelId,
limit: 2,
});
expect(page1.data).toHaveLength(2);
expect(page1.nextCursor).not.toBeNull();
const page2 = await listReviewMessages({
channelId,
limit: 2,
cursor: page1.nextCursor!,
});
expect(page2.data).toHaveLength(2);
// Verify no overlap
const page1Ids = page1.data.map((m) => m.id);
const page2Ids = page2.data.map((m) => m.id);
const overlap = page1Ids.filter((id) => page2Ids.includes(id));
expect(overlap).toHaveLength(0);
});
});
describe("updateMessageAsEdited", () => {
const createTestMessage = (
overrides: Partial<MessageRecord> = {},
): MessageRecord => ({
id: `msg-${Date.now()}-${Math.random().toString(36).slice(2)}`,
guild_id: "guild-123",
channel_id: "channel-456",
thread_id: null,
user_id: "user-789",
username: "testuser",
avatar_url: null,
content: "Test message",
edited_content: null,
created_at: Date.now(),
edited_at: null,
deleted_at: null,
type: "text",
metadata: null,
ai_status: "pending",
...overrides,
});
it("resets ai_status to pending and clears AI fields when message is edited", async () => {
const messageId = `msg-edit-test-${Date.now()}`;
const msg = createTestMessage({
id: messageId,
content: "original content",
ai_status: "clean",
ai_moderation_flags: "test_flag",
ai_moderation_score: 0.5,
ai_moderation_raw: '{"test": "data"}',
ai_analysis: "This is clean",
ai_analyzed_at: Date.now() - 10000,
ai_error: null,
});
// Insert message with AI analysis already done
await insertMessage(msg);
// Verify initial state
let retrieved = await getMessageById(messageId);
expect(retrieved?.ai_status).toBe("clean");
expect(retrieved?.ai_moderation_flags).toBe("test_flag");
expect(retrieved?.ai_moderation_score).toBe(0.5);
expect(retrieved?.ai_analysis).toBe("This is clean");
// Edit the message
await updateMessageAsEdited(messageId, "edited content", Date.now());
// Verify AI fields are reset
retrieved = await getMessageById(messageId);
expect(retrieved?.edited_content).toBe("edited content");
expect(retrieved?.type).toBe("edited");
expect(retrieved?.ai_status).toBe("pending");
expect(retrieved?.ai_moderation_flags).toBeNull();
expect(retrieved?.ai_moderation_score).toBeNull();
expect(retrieved?.ai_moderation_raw).toBeNull();
expect(retrieved?.ai_analysis).toBeNull();
expect(retrieved?.ai_analyzed_at).toBeNull();
expect(retrieved?.ai_error).toBeNull();
});
});
});

View File

@@ -1,5 +1,8 @@
import { config } from "dotenv";
import { defineConfig } from "vitest/config"; import { defineConfig } from "vitest/config";
config({ path: ".env.test" });
export default defineConfig({ export default defineConfig({
test: { test: {
environment: "node", environment: "node",