refactor: remove outdated frontend and services specifications
- Deleted the frontend refactor design document to streamline project scope. - Removed the services refactoring design document to eliminate redundancy. - Eliminated the visual redesign document as part of the cleanup process. - Purged the Discord Automod redesign document to focus on current objectives.
This commit is contained in:
Developer
@@ -1,695 +0,0 @@
|
|||||||
# CLAUDE.md
|
|
||||||
|
|
||||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
||||||
|
|
||||||
## Project Overview
|
|
||||||
|
|
||||||
**Bete (Discord Moderation Watcher)** — A comprehensive microservice-based Discord monitoring and moderation bot. Captures text messages, images, voice audio, and screenshares from Discord servers. Features AI-powered content moderation with auto-delete, voice recording with real-time streaming, music playback, and a React dashboard.
|
|
||||||
|
|
||||||
Built with **pnpm workspace monorepo** with 3 services and 1 shared library:
|
|
||||||
|
|
||||||
| Package | Path | Description |
|
|
||||||
|---------|------|-------------|
|
|
||||||
| `discord-moderation-backend` | `services/backend` | Express HTTP/WS server, REST API, Redis bridge |
|
|
||||||
| `@bete/discord-gateway` | `services/discord-gateway` | Discord client, voice recording, message capture, AI moderation |
|
|
||||||
| `frontend` | `services/frontend` | Next.js 16 (React 19) static dashboard, Tailwind v4, shadcn/ui |
|
|
||||||
| `@bete/shared` | `packages/shared` | Shared types, errors, logger, utilities |
|
|
||||||
|
|
||||||
**Database:** PostgreSQL (Drizzle ORM) — NOT SQLite.
|
|
||||||
|
|
||||||
**Inter-service communication:** Redis pub/sub.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
### High-Level Flow
|
|
||||||
|
|
||||||
```
|
|
||||||
Discord
|
|
||||||
|
|
|
||||||
v
|
|
||||||
discord-gateway ---- Redis ---- backend ---- WebSocket ---- frontend
|
|
||||||
| pub/sub (broadcast) (Next.js static)
|
|
||||||
| |
|
|
||||||
| |
|
|
||||||
<------------------+
|
|
||||||
(command channel)
|
|
||||||
```
|
|
||||||
|
|
||||||
1. **discord-gateway** connects to Discord via `discord.js-selfbot-v13`, captures events (messages, voice, attachments), stores in PostgreSQL, and publishes events to Redis channels (e.g., `discord:message:created`, `discord:voice:pcm`).
|
|
||||||
|
|
||||||
2. **backend** subscribes to Redis channels, broadcasts events to WebSocket clients, and serves REST API endpoints.
|
|
||||||
|
|
||||||
3. **frontend** connects via WebSocket and HTTP to the backend, provides a dashboard for live monitoring (text, voice, media) and AI moderation oversight.
|
|
||||||
|
|
||||||
4. **Command flow (reverse):** Frontend -> Backend HTTP/WS -> Redis (`backend:command`) -> discord-gateway (command handler) - for actions like connect voice, play media, moderate message.
|
|
||||||
|
|
||||||
### Data Flow
|
|
||||||
|
|
||||||
```
|
|
||||||
Message Capture:
|
|
||||||
Discord -> messageCapture.ts -> messageStore.ts (PostgreSQL)
|
|
||||||
|
|
|
||||||
+> eventBroadcaster -> Redis -> backend -> WS clients
|
|
||||||
|
|
||||||
Voice Recording:
|
|
||||||
Discord -> voiceController.ts -> recorder.ts -> OGG files on disk
|
|
||||||
| |
|
|
||||||
+> eventBroadcaster +> decoder.ts -> PCM -> Redis -> WS clients
|
|
||||||
|
|
||||||
AI Moderation:
|
|
||||||
messageStore -> aiAnalyzer.ts -> LLM API -> moderation result
|
|
||||||
| |
|
|
||||||
+> eventBroadcaster +> update message in DB
|
|
||||||
```
|
|
||||||
|
|
||||||
## Service Breakdown
|
|
||||||
|
|
||||||
### backend (`services/backend`)
|
|
||||||
|
|
||||||
Express 5 + Helmet HTTP server with WebSocket (ws) on port 3001 (default).
|
|
||||||
|
|
||||||
**REST API endpoints (all public):**
|
|
||||||
- `GET /api/health` — Health check with optional `?verbose=true`
|
|
||||||
- `GET /api/config` — App configuration
|
|
||||||
- `GET /api/messages` — List messages (cursor pagination)
|
|
||||||
- `GET /api/messages/:channelId` — Messages by channel
|
|
||||||
- `GET /api/messages/:channelId/attachments` — Attachments by channel
|
|
||||||
- `GET /api/messages/detail/:id` — Single message
|
|
||||||
- `POST /api/messages/reanalyze-batch` — Bulk retry AI analysis
|
|
||||||
- `POST /api/messages/:id/reanalyze` — Retry single message
|
|
||||||
- `POST /api/messages/:id/moderate` — Dispatch moderation action
|
|
||||||
- `GET /api/review` — Flagged/warned messages
|
|
||||||
- `GET /api/analysis/search` — Full-text search with `?q=`
|
|
||||||
- `POST /api/chat` — AI chatbot chat
|
|
||||||
- `GET /api/chat/history` — Chat history
|
|
||||||
- `POST /api/chat/clear` — Clear chat history
|
|
||||||
- `POST /api/voice/command` — Send voice transmit commands
|
|
||||||
- `GET /api/status` — Voice connection status
|
|
||||||
- `POST /api/connect` — Connect to voice channel
|
|
||||||
- `POST /api/disconnect` — Disconnect from voice
|
|
||||||
- `GET /api/guilds` — List guilds
|
|
||||||
- `GET /api/guilds/:guildId/channels` — Text channels
|
|
||||||
- `GET /api/guilds/:guildId/voice-channels` — Voice channels
|
|
||||||
- `GET /api/media/status` — Media player status
|
|
||||||
- `POST /api/media/queue` — Queue media (music/screen)
|
|
||||||
- `POST /api/media/skip` — Skip current track
|
|
||||||
- `POST /api/media/stop` — Stop playback
|
|
||||||
- `POST /api/media/volume` — Set volume
|
|
||||||
- `GET /api/recordings` — Voice recordings list
|
|
||||||
- `GET /api/ui-state` — Get persistent UI state
|
|
||||||
- `POST /api/ui-state` — Save UI state
|
|
||||||
|
|
||||||
**Modules (feature-based, under `src/modules/`):**
|
|
||||||
- `health/` — Database connectivity check
|
|
||||||
- `messages/` — Message + attachment CRUD, review, reanalyze
|
|
||||||
- `voice/` — Voice connection, guilds, channels
|
|
||||||
- `media/` — Music/screenshare player control
|
|
||||||
- `analysis/` — Full-text search across analyzed messages
|
|
||||||
- `chatbot/` — AI chatbot with server context
|
|
||||||
- `recordings/` — Voice recording listing
|
|
||||||
- `ui-state/` — Persistent UI state for dashboard
|
|
||||||
- `config/` — App config endpoint
|
|
||||||
|
|
||||||
**WebSocket events (outbound to frontend):**
|
|
||||||
- `message_created`, `message_updated`, `message_deleted`, `message_analyzed`
|
|
||||||
- `attachment_created`, `attachment_uploaded`
|
|
||||||
- `voice_recording_started`, `voice_recording_stopped`, `voice_recording_uploaded`
|
|
||||||
- `voice_active_user`, `voice_pcm_data`
|
|
||||||
- `analysis_queue_status`
|
|
||||||
- `user_state`, `ui_state`, `media_state`
|
|
||||||
- `heartbeat` (every 30s)
|
|
||||||
|
|
||||||
**WebSocket inbound (from frontend):**
|
|
||||||
- JSON `{ type: "voice_transmit", buffer: "<base64 PCM>" }` — forwarded to Redis
|
|
||||||
- JSON `{ type: "voice_command", command: "..." }` — forwarded to discord-gateway
|
|
||||||
|
|
||||||
### discord-gateway (`services/discord-gateway`)
|
|
||||||
|
|
||||||
The core service that connects to Discord using `discord.js-selfbot-v13`.
|
|
||||||
|
|
||||||
**Modules:**
|
|
||||||
|
|
||||||
- **`message-capture/`** — Listens to `messageCreate`, `messageUpdate`, `messageDelete` events. Stores messages in PostgreSQL. Handles edits, deletes, and backlog sync.
|
|
||||||
- `messageCapture.ts` — Event listeners
|
|
||||||
- `messageStore.ts` — Database operations (upsert, update, delete)
|
|
||||||
- `messageMetadata.ts` — User/channel metadata extraction
|
|
||||||
- `broadcaster.ts` — Internal event dispatch
|
|
||||||
- `pagination.ts` — Backlog sync for historical messages
|
|
||||||
- `analyticsStore.ts` — Per-channel analytics tracking
|
|
||||||
|
|
||||||
- **`voice-recording/`** — Voice channel connection, recording, and real-time PCM streaming.
|
|
||||||
- `voiceController.ts` — Connection lifecycle (connect/disconnect per guild+channel)
|
|
||||||
- `recorder.ts` — Manages speaking users, subscribes to audio streams
|
|
||||||
- `recorder/audioStream.ts` — Opus packet subscription per user
|
|
||||||
- `recorder/decoder.ts` — Opus to PCM decoding with rotation/cooldown
|
|
||||||
- `recorder/segment.ts` — OGG file segment rotation (default 5s)
|
|
||||||
- `recorder/metadata.ts` — User metadata JSON for each segment
|
|
||||||
- `recorder/sessionRecording.ts` — Session-scoped recording management
|
|
||||||
- `recorder/uploader.ts` — Upload completed segments
|
|
||||||
- `player.ts` — Discord player (music/screenshare playback)
|
|
||||||
- `transmitter.ts` — Browser-to-Discord audio transmission (Redis -> Opus -> Discord)
|
|
||||||
- `muxer.ts` — Audio muxing logic
|
|
||||||
- `packetFilter.ts` — Opus packet filtering
|
|
||||||
- `ffmpegProcess.ts` — FFmpeg-based processing
|
|
||||||
- `mediaTypes.ts` — Audio/video format definitions
|
|
||||||
- `teleUpload.ts` — Upload to tele/picser
|
|
||||||
|
|
||||||
- **`attachment-upload/`** — Downloads Discord attachments, uploads to external service.
|
|
||||||
- `attachmentUploader.ts` — Download + upload with retry
|
|
||||||
- `imageResizer.ts` — Resize images before upload
|
|
||||||
- `teleUpload.ts` — Upload to tele/picser API
|
|
||||||
|
|
||||||
- **`ai-moderation/`** — AI-powered content moderation pipeline.
|
|
||||||
- `aiAnalyzer.ts` — Analysis worker (batch + individual fallback)
|
|
||||||
- `aiAnalysisWorker.ts` — Piscina worker thread for batch processing
|
|
||||||
- `llmClient.ts` — Generic LLM API client
|
|
||||||
- `llmModerationClient.ts` — Moderation-specific LLM client
|
|
||||||
- `moderationPrompt.ts` — System prompt builder with few-shot
|
|
||||||
- `autoDeleteManager.ts` — Auto-delete flagged messages
|
|
||||||
- `conversationContext.ts` — Conversation window builder
|
|
||||||
- `concurrencyLimiter.ts` — Rate limiter for LLM calls
|
|
||||||
- `channelCultureStore.ts` — Channel norms/slang context
|
|
||||||
- `cultureLearner.ts` — Learn channel culture over time
|
|
||||||
- `userReputationStore.ts` — User trust scores
|
|
||||||
- `textCacheStore.ts` — Deduplicate repeated text analysis
|
|
||||||
- `stickerCache.ts` — Upload and cache sticker images
|
|
||||||
- `stickerPrompt.ts` — Sticker analysis prompt
|
|
||||||
- `urlFetcher.ts` — Fetch URL content for analysis
|
|
||||||
- `responseLogger.ts` — Log moderation responses
|
|
||||||
|
|
||||||
- **`event-broadcaster/`** — Redis pub/sub publisher for all events.
|
|
||||||
- `eventBroadcaster.ts` — `EventBroadcaster` class with typed methods
|
|
||||||
- `eventTypes.ts` — Channel constants and event interfaces
|
|
||||||
|
|
||||||
- **`command-handler/`** — Listens on `backend:command` Redis channel for backend requests.
|
|
||||||
- `commandHandler.ts` — Handles voice connect/disconnect, guilds, channels, media, transmit
|
|
||||||
|
|
||||||
**Infrastructure:**
|
|
||||||
- `src/shared/config/config.ts` — Zod-validated env config (DISCORD_TOKEN, REDIS_URL, AI_LLM_*, etc.)
|
|
||||||
- `src/shared/database/schema.ts` — Full PostgreSQL schema definition
|
|
||||||
- `src/shared/database/drizzle.ts` — Drizzle + pg pool initialization
|
|
||||||
- `src/shared/database/migrate.ts` — Migration runner with advisory locking
|
|
||||||
- `src/shared/database/voiceRecordingRepo.ts` — Voice recording queries
|
|
||||||
- `src/shared/discord/clientOptions.ts` — Discord client configuration
|
|
||||||
|
|
||||||
### frontend (`services/frontend`)
|
|
||||||
|
|
||||||
Next.js 16 (React 19) static export dashboard, built with TypeScript + Tailwind v4 + shadcn/ui + base-ui.
|
|
||||||
|
|
||||||
**Tech stack:**
|
|
||||||
- Next.js 16 (App Router, static export)
|
|
||||||
- React 19 with React Compiler
|
|
||||||
- TypeScript strict
|
|
||||||
- Tailwind v4 + shadcn/ui + base-ui components
|
|
||||||
- lucide-react icons
|
|
||||||
|
|
||||||
**Feature structure:**
|
|
||||||
- `src/app/` — App Router pages (login, dashboard with tabs)
|
|
||||||
- `src/features/` — Feature components (dashboard, messages, live, chatbot)
|
|
||||||
- `src/lib/` — Shared utilities (types, API client, WebSocket, hooks)
|
|
||||||
- `src/components/` — Shared UI components (layout, ui)
|
|
||||||
|
|
||||||
### shared (`packages/shared`)
|
|
||||||
|
|
||||||
Shared library used by both backend and discord-gateway.
|
|
||||||
|
|
||||||
**Exports:**
|
|
||||||
- `@bete/shared` — Everything below
|
|
||||||
- `@bete/shared/types` — AppConfig, MessageRecord, AttachmentRecord, VoiceSegment, etc.
|
|
||||||
- `@bete/shared/errors` — AppError, ValidationError, NotFoundError, UnauthorizedError, DatabaseError, ConfigError, DiscordError, TimeoutError, etc.
|
|
||||||
- `@bete/shared/logger` — Pino-based `createChildLogger(context)`
|
|
||||||
- `@bete/shared/utils` — Shared utilities
|
|
||||||
|
|
||||||
## Database Schema (PostgreSQL)
|
|
||||||
|
|
||||||
All tables defined in `services/discord-gateway/src/shared/database/schema.ts`.
|
|
||||||
|
|
||||||
### messages
|
|
||||||
Stores text messages with AI moderation results.
|
|
||||||
- `id` (text PK), `guild_id`, `channel_id`, `thread_id`
|
|
||||||
- `user_id`, `username`, `avatar_url`
|
|
||||||
- `content`, `edited_content`, `type` (text|edited|deleted)
|
|
||||||
- `created_at`, `edited_at`, `deleted_at`
|
|
||||||
- `ai_status` (pending|processing|clean|warn|flagged|error)
|
|
||||||
- `ai_moderation_flags`, `ai_moderation_score`, `ai_analysis`, `ai_categories`
|
|
||||||
- `ai_severity` (none|low|medium|high|critical), `ai_confidence`
|
|
||||||
- `ai_recommended_action` (none|monitor|warn|review|delete|escalate)
|
|
||||||
- `ai_analyzed_at`, `ai_error`, `metadata`
|
|
||||||
- Indexes: channel, user, created_at, thread, channel+created, thread+created, ai_status+created, guild+ai_status+created, guild+created+deleted, channel+ai_status+created, thread+ai_status+created
|
|
||||||
|
|
||||||
### attachments
|
|
||||||
Discord attachment metadata with upload tracking.
|
|
||||||
- `id` (text PK), `message_id` (FK -> messages cascade), `guild_id`, `channel_id`
|
|
||||||
- `filename`, `size`, `type` (MIME), `discord_url`, `uploaded_url`
|
|
||||||
- `upload_status` (pending|uploaded|failed), `upload_error`
|
|
||||||
- `created_at`, `uploaded_at`
|
|
||||||
- Indexes: channel, message, upload_status, channel+created, thread+created
|
|
||||||
|
|
||||||
### voice_recordings
|
|
||||||
Voice segment metadata.
|
|
||||||
- `id` (text PK), `user_id`, `username`, `avatar_url`
|
|
||||||
- `guild_id`, `channel_id`, `channel_name`
|
|
||||||
- `filename`, `size_bytes`, `download_url`
|
|
||||||
- `upload_status` (pending|uploaded|failed), `upload_error`
|
|
||||||
- `created_at`, `uploaded_at`
|
|
||||||
- Indexes: user_id, channel_id, created_at
|
|
||||||
|
|
||||||
### ui_state
|
|
||||||
Persistent dashboard UI state (key-value).
|
|
||||||
- `key` (text PK), `value` (text), `updated_at`
|
|
||||||
|
|
||||||
### ai_analysis_runs
|
|
||||||
Tracks AI analysis batch runs.
|
|
||||||
- `id` (text PK), `conversation_key`, `target_message_ids` (JSON)
|
|
||||||
- `model`, `request_tokens_estimate`, `response_raw`
|
|
||||||
- `status` (pending|processing|completed|failed), `error`
|
|
||||||
- `created_at`, `completed_at`
|
|
||||||
- Indexes: conversation_key, status, created_at
|
|
||||||
|
|
||||||
### user_reputations
|
|
||||||
User trust scores for AI context.
|
|
||||||
- `user_id` (text PK), `guild_id`, `trust_score`, `clean_message_streak`
|
|
||||||
- `total_infractions`, `last_infraction_at`, `created_at`, `updated_at`
|
|
||||||
- Indexes: guild_id, trust_score
|
|
||||||
|
|
||||||
### channel_cultures
|
|
||||||
AI-generated channel norms and slang summaries.
|
|
||||||
- `channel_id` (text PK), `guild_id`, `culture_summary`, `last_analyzed_at`
|
|
||||||
- Index: guild_id
|
|
||||||
|
|
||||||
### message_reviews
|
|
||||||
Manual review tracking for flagged messages.
|
|
||||||
- `id` (text PK), `message_id`, `guild_id`, `channel_id`
|
|
||||||
- `reviewer_id`, `status` (pending|approved|rejected|escalated)
|
|
||||||
- `notes`, `created_at`, `reviewed_at`
|
|
||||||
- Indexes: message_id, status, created_at, guild+status+created
|
|
||||||
|
|
||||||
### moderation_actions
|
|
||||||
Action audit log (delete/mute/warn/kick/ban).
|
|
||||||
- `id` (text PK), `message_id`, `user_id`, `guild_id`
|
|
||||||
- `action_type` (delete_message|mute_user|warn_user|kick_user|ban_user)
|
|
||||||
- `reason`, `executed_by`, `status` (pending|executed|failed)
|
|
||||||
- `error`, `created_at`, `executed_at`
|
|
||||||
- Indexes: message_id, user_id, status, guild+status+created
|
|
||||||
|
|
||||||
### retention_policies
|
|
||||||
Data retention rules per guild/channel.
|
|
||||||
- `id` (text PK), `guild_id`, `channel_id`
|
|
||||||
- `retention_days`, `apply_to_media`, `apply_to_voice`, `enabled`
|
|
||||||
- `created_at`, `updated_at`
|
|
||||||
- Indexes: guild_id, enabled
|
|
||||||
|
|
||||||
### text_analysis_cache
|
|
||||||
Caches normalized-text moderation results to avoid redundant LLM calls.
|
|
||||||
- `text` (text PK), `flags` (JSON array), `source` (local|primary_ai|vision_llm)
|
|
||||||
- `analyzed_at`, `expires_at`, `hit_count`
|
|
||||||
- Indexes: expires_at, source
|
|
||||||
|
|
||||||
### sticker_cache
|
|
||||||
Uploaded sticker image URLs for vision analysis.
|
|
||||||
- `name` (text PK), `image_url`, `mime_type`, `fetched_at`
|
|
||||||
- Index: fetched_at
|
|
||||||
|
|
||||||
### corrected_moderations
|
|
||||||
Manual corrections (false positives) for few-shot injection.
|
|
||||||
- `id` (text PK), `message_id`, `original_flags`, `corrected_flags`
|
|
||||||
- `correction_notes`, `content_snippet`, `created_at`
|
|
||||||
- Indexes: created_at, message_id
|
|
||||||
|
|
||||||
### muxer_jobs
|
|
||||||
Audio post-processing job queue.
|
|
||||||
- `id` (text PK), `data` (JSON), `status` (pending|processing|completed|failed)
|
|
||||||
- `attempts`, `maxAttempts`, `created_at`, `updated_at`, `error`
|
|
||||||
- Indexes: status, created_at
|
|
||||||
|
|
||||||
## Redis Communication
|
|
||||||
|
|
||||||
### discord-gateway publishes (event channels):
|
|
||||||
| Channel | Event type | When |
|
|
||||||
|---------|-----------|------|
|
|
||||||
| `discord:message:created` | `message_created` | New message |
|
|
||||||
| `discord:message:updated` | `message_updated` | Message edited |
|
|
||||||
| `discord:message:deleted` | `message_deleted` | Message deleted |
|
|
||||||
| `discord:message:analyzed` | `message_analyzed` | AI analysis complete |
|
|
||||||
| `discord:attachment:created` | `attachment_created` | New attachment |
|
|
||||||
| `discord:attachment:uploaded` | `attachment_uploaded` | Upload complete |
|
|
||||||
| `discord:voice:started` | `voice_recording_started` | Recording started |
|
|
||||||
| `discord:voice:stopped` | `voice_recording_stopped` | Recording stopped |
|
|
||||||
| `discord:voice:uploaded` | `voice_recording_uploaded` | Upload complete |
|
|
||||||
| `discord:voice:active_user` | `voice_active_user` | Speaker state change |
|
|
||||||
| `discord:voice:pcm` | `voice_pcm_data` | Live PCM audio chunk |
|
|
||||||
| `discord:analysis:queue_status` | `analysis_queue_status` | Queue stats |
|
|
||||||
|
|
||||||
### backend publishes (command channel):
|
|
||||||
| Channel | Command type | Description |
|
|
||||||
|---------|-------------|-------------|
|
|
||||||
| `backend:command` | `voice:connect` | Connect to voice |
|
|
||||||
| `backend:command` | `voice:disconnect` | Disconnect voice |
|
|
||||||
| `backend:command` | `voice:channels` | List voice channels |
|
|
||||||
| `backend:command` | `voice:transmit:start/stop` | Audio transmit |
|
|
||||||
| `backend:command` | `guilds:list` | List guilds |
|
|
||||||
| `backend:command` | `guilds:text-channels` | List text channels |
|
|
||||||
| `backend:command` | `media:queue/skip/stop/volume` | Media control |
|
|
||||||
| `backend:command` | `moderation:action` | Execute moderation action |
|
|
||||||
|
|
||||||
Envelope format: `{ id, type, payload, replyChannel }`.
|
|
||||||
|
|
||||||
Status keys: `voice:status`, `media:status` (set by discord-gateway, read by backend).
|
|
||||||
|
|
||||||
## Development Commands
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Install all dependencies
|
|
||||||
pnpm install
|
|
||||||
|
|
||||||
# Run each service in development mode (separate terminal each)
|
|
||||||
pnpm run dev:backend # Backend on port 3001
|
|
||||||
pnpm run dev:discord-gateway # Discord client + all features
|
|
||||||
pnpm run dev:web # Frontend via next dev (port 3000)
|
|
||||||
|
|
||||||
# Build
|
|
||||||
pnpm run build:backend
|
|
||||||
pnpm run build:discord-gateway
|
|
||||||
pnpm run build:web # next build (static export)
|
|
||||||
|
|
||||||
# Type checking
|
|
||||||
pnpm run typecheck # Node services (pnpm -r)
|
|
||||||
pnpm run typecheck:web # Frontend typecheck (next build)
|
|
||||||
|
|
||||||
# Lint (Biome)
|
|
||||||
pnpm run lint
|
|
||||||
|
|
||||||
# Format (Biome)
|
|
||||||
pnpm run format
|
|
||||||
|
|
||||||
# Run tests across all packages
|
|
||||||
pnpm run test
|
|
||||||
|
|
||||||
# Database migrations (Drizzle)
|
|
||||||
pnpm run db:generate # Generate new migration
|
|
||||||
pnpm run db:migrate # Apply pending migrations
|
|
||||||
pnpm run db:studio # Open Drizzle Studio
|
|
||||||
|
|
||||||
# Install yt-dlp for media download
|
|
||||||
pnpm run install:yt-dlp
|
|
||||||
|
|
||||||
# Deploy to VPS (build + hot-patch running containers)
|
|
||||||
./deploy.sh # Build + deploy all services
|
|
||||||
./deploy.sh --frontend # Frontend (Next.js) only
|
|
||||||
./deploy.sh --backend # Backend TypeScript only
|
|
||||||
./deploy.sh --no-build # Skip build, just copy files
|
|
||||||
```
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
Configuration via `.env` (see `.env.example`). Managed by Zod schemas:
|
|
||||||
- discord-gateway: `services/discord-gateway/src/shared/config/config.ts`
|
|
||||||
- backend: `services/backend/src/shared/config/index.ts`
|
|
||||||
|
|
||||||
### Core (both services)
|
|
||||||
- `DISCORD_TOKEN` — Discord user token (required)
|
|
||||||
- `MONITOR_GUILD_ID` — Target guild for text monitoring
|
|
||||||
- `NODE_ENV` — development|production|test
|
|
||||||
- `LOG_LEVEL` — Pino log level (default: info)
|
|
||||||
- `VERBOSE` — Enable debug logging (default: false)
|
|
||||||
|
|
||||||
### Database (PostgreSQL)
|
|
||||||
- `DATABASE_URL` — Connection string (overrides individual params)
|
|
||||||
- `POSTGRES_HOST`, `POSTGRES_PORT` (5432), `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB`
|
|
||||||
- `POSTGRES_POOL_MIN` (2), `POSTGRES_POOL_MAX` (10)
|
|
||||||
- `AUTO_MIGRATE_ON_STARTUP` (default: true)
|
|
||||||
|
|
||||||
### Redis
|
|
||||||
- `REDIS_URL` — Connection string (default: redis://localhost:6379)
|
|
||||||
|
|
||||||
### Voice Recording (discord-gateway)
|
|
||||||
- `RECORDINGS_DIR` — Audio file output (default: ./recordings)
|
|
||||||
- `RECORDING_SEGMENT_MS` — OGG segment duration (default: 5000)
|
|
||||||
- `DECODER_ROTATE_MS` — Opus decoder rotation (default: 5000)
|
|
||||||
- `DECODER_COOLDOWN_MS` — Decoder error cooldown (default: 30000)
|
|
||||||
- `AUDIO_STREAM_SILENCE_DURATION_MS` — Silence threshold (default: 3000)
|
|
||||||
- `VOICE_CONNECTION_TIMEOUT_MS` — Connection timeout (default: 15000)
|
|
||||||
- `RECONNECT_TIMEOUT_MS` — Reconnect timeout (default: 5000)
|
|
||||||
- `PACKET_FILTER_MIN_SIZE` — Minimum Opus packet size (default: 8)
|
|
||||||
- `OPUS_FRAME_SIZE` (960), `AUDIO_SAMPLE_RATE` (48000), `AUDIO_CHANNELS` (2)
|
|
||||||
- `VOICE_GUILD_ID`, `VOICE_CHANNEL_ID`
|
|
||||||
|
|
||||||
### Attachments
|
|
||||||
- `TELE_UPLOAD_URL` — Upload endpoint (default: https://upload.asepharyana.my.id/api/upload)
|
|
||||||
- `ATTACHMENT_UPLOAD_TIMEOUT_MS` (30000), `ATTACHMENT_MAX_SIZE_MB` (100), `ATTACHMENT_RETRY_ATTEMPTS` (3)
|
|
||||||
|
|
||||||
### AI Moderation (discord-gateway)
|
|
||||||
- `AI_ANALYSIS_ENABLED` — Enable AI analysis (default: false)
|
|
||||||
- `AI_LLM_API_KEY` — LLM API key (required if enabled)
|
|
||||||
- `AI_LLM_BASE_URL` — LLM endpoint (default: https://9router.asepharyana.my.id/v1)
|
|
||||||
- `AI_LLM_MODEL` — Text model (default: text)
|
|
||||||
- `AI_LLM_VISION_MODEL` — Vision model (optional fallback)
|
|
||||||
- `AI_LLM_MAX_CONCURRENT` (5), `AI_LLM_TEXT_BATCH_SIZE` (20)
|
|
||||||
- `AI_LLM_MEDIA_ANALYSIS_TIMEOUT_MS` (60000), `AI_LLM_IMAGE_MAX_DIMENSION` (1024)
|
|
||||||
- `AI_ANALYSIS_DEBOUNCE_MS` (500), `AI_ANALYSIS_MAX_BATCH_SIZE` (200)
|
|
||||||
- `AI_ANALYSIS_PROCESSING_TIMEOUT_MS` (120000)
|
|
||||||
- `PISCINA_MAX_THREADS` — Worker pool size (optional)
|
|
||||||
|
|
||||||
### Auto-Delete
|
|
||||||
- `AUTO_DELETE_FLAGGED_ENABLED` (true), `AUTO_DELETE_FLAGGED_DRY_RUN` (true)
|
|
||||||
- `AUTO_DELETE_FLAGGED_DELAY_MS` (0), `AUTO_DELETE_MIN_CONFIDENCE` (0.5)
|
|
||||||
- `AUTO_DELETE_ALLOWED_SEVERITIES`, `AUTO_DELETE_ALLOWED_CATEGORIES`
|
|
||||||
- `AUTO_DELETE_EXCLUDED_CHANNEL_IDS`, `AUTO_DELETE_EXCLUDED_USER_IDS`
|
|
||||||
- `AUTO_DELETE_NOTIFY_USER`, `AUTO_DELETE_LOG_CHANNEL_ID`
|
|
||||||
|
|
||||||
### OpenAI Moderation (optional separate endpoint)
|
|
||||||
- `OPENAI_MODERATION_API_KEY`, `OPENAI_MODERATION_BASE_URL`, `OPENAI_MODERATION_MODEL`
|
|
||||||
|
|
||||||
### Backend
|
|
||||||
- `WEBSERVER_PORT` (3001)
|
|
||||||
- `BACKLOG_SYNC_HOURS` (24), `BACKLOG_SYNC_BATCH_SIZE` (100)
|
|
||||||
|
|
||||||
### Retention
|
|
||||||
- `RETENTION_MESSAGES_DAYS` (0=off), `RETENTION_ATTACHMENTS_DAYS`, `RETENTION_VOICE_DAYS`
|
|
||||||
- `RETENTION_CLEANUP_INTERVAL_MS` (86400000), `RETENTION_DRY_RUN` (true)
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
Tests use **Vitest**. Currently minimal test coverage. Test directories should be created per service:
|
|
||||||
|
|
||||||
```
|
|
||||||
services/backend/tests/
|
|
||||||
services/discord-gateway/tests/
|
|
||||||
services/frontend/tests/
|
|
||||||
```
|
|
||||||
|
|
||||||
Run tests: `pnpm run test` (runs `vitest run` in each package).
|
|
||||||
|
|
||||||
## Code Style
|
|
||||||
|
|
||||||
- **Formatter**: Biome (2-space indent)
|
|
||||||
- **Linter**: Biome with strict rules
|
|
||||||
- **Language**: TypeScript with strict mode
|
|
||||||
- **Logging**: Use `createChildLogger(context)` from `@bete/shared/logger`
|
|
||||||
- **Errors**: Throw custom `AppError` subclasses with `code` + `statusCode`
|
|
||||||
- **Database**: Use Drizzle ORM or raw parameterized queries (never string interpolation)
|
|
||||||
- **Imports**: Use `.js` extensions in source files (ESM convention)
|
|
||||||
|
|
||||||
## Key Patterns
|
|
||||||
|
|
||||||
### Event-Driven Architecture
|
|
||||||
|
|
||||||
All inter-service communication happens through Redis pub/sub. The discord-gateway publishes events on typed channels, the backend subscribes and broadcasts to WebSocket clients. The backend publishes commands on `backend:command` with reply channels for request-response patterns.
|
|
||||||
|
|
||||||
### Message Capture Lifecycle
|
|
||||||
|
|
||||||
1. Discord event fires (`messageCreate`, `messageUpdate`, `messageDelete`)
|
|
||||||
2. Check guild matches MONITOR_GUILD_ID
|
|
||||||
3. Extract message metadata (user, channel, content, timestamp)
|
|
||||||
4. Upsert into `messages` table in PostgreSQL
|
|
||||||
5. Publish event to Redis (`discord:message:*`)
|
|
||||||
6. If attachments exist, insert into `attachments` table with `status='pending'`
|
|
||||||
7. Start async upload to tele/picser (non-blocking)
|
|
||||||
8. On success: update `uploaded_url`, `status='uploaded'`
|
|
||||||
9. On failure: store error, `status='failed'`
|
|
||||||
|
|
||||||
### AI Moderation Pipeline
|
|
||||||
|
|
||||||
1. Messages with `ai_status='pending'` are picked up by `aiAnalyzer.ts`
|
|
||||||
2. Batches messages by conversation (thread/channel proximity)
|
|
||||||
3. Builds context window (recent messages + channel culture + user reputation)
|
|
||||||
4. Calls LLM via `llmModerationClient.ts` with moderation prompt
|
|
||||||
5. Updates message with `ai_status`, `ai_moderation_flags`, `ai_severity`, `ai_confidence`, `ai_recommended_action`
|
|
||||||
6. If `AUTO_DELETE_FLAGGED_ENABLED` and confidence meets threshold, triggers auto-delete
|
|
||||||
7. Falls back to individual analysis for messages that could not be batched
|
|
||||||
8. Caches normalized text results in `text_analysis_cache` to avoid repeat calls
|
|
||||||
|
|
||||||
### Voice Recording Lifecycle
|
|
||||||
|
|
||||||
1. `VoiceController.connect(guildId, channelId)` via Redis command
|
|
||||||
2. Joins Discord voice channel, sets up audio receiver
|
|
||||||
3. On user start speaking: create per-user stream, OGG segment manager, Opus decoder
|
|
||||||
4. Opus packets -> OGG segments on disk + PCM decode for WebSocket broadcast
|
|
||||||
5. PCM data published to Redis (`discord:voice:pcm`) -> backend -> WS clients
|
|
||||||
6. On silence (3s timeout): close stream, finalize segment
|
|
||||||
7. After segment complete: upload to external storage, update database
|
|
||||||
8. `VoiceController.disconnect()` stops all recording
|
|
||||||
|
|
||||||
### WebSocket Protocol (frontend)
|
|
||||||
|
|
||||||
**Outbound (backend -> frontend):**
|
|
||||||
- Binary: PCM audio (24kHz mono s16le), prefixed with 4-byte user hash
|
|
||||||
- JSON events: all typed in `WSEventMap` — `message_*`, `voice_*`, `attachment_*`, `user_state`, `ui_state`, `media_state`
|
|
||||||
|
|
||||||
**Inbound (frontend -> backend):**
|
|
||||||
- JSON `{ type: "voice_transmit", buffer: "<base64 PCM>" }` for mic-to-Discord
|
|
||||||
- JSON `{ type: "voice_command", command: "..." }` for voice control
|
|
||||||
|
|
||||||
### Graceful Shutdown
|
|
||||||
|
|
||||||
discord-gateway handles SIGINT/SIGTERM/uncaughtException/unhandledRejection:
|
|
||||||
1. Close database pool
|
|
||||||
2. Disconnect voice controller
|
|
||||||
3. Close event broadcaster (Redis)
|
|
||||||
4. Close command handler (Redis)
|
|
||||||
5. Destroy Discord client
|
|
||||||
6. Exit process
|
|
||||||
|
|
||||||
### Public API
|
|
||||||
|
|
||||||
All backend endpoints are publicly accessible — no authentication required.
|
|
||||||
|
|
||||||
## Recording Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
recordings/
|
|
||||||
+-- <user-id>/
|
|
||||||
| +-- <user-id>-<session-start>-0.ogg
|
|
||||||
| +-- <user-id>-<session-start>-0.json
|
|
||||||
| +-- <user-id>-<session-start>-1.ogg
|
|
||||||
| +-- ...
|
|
||||||
```
|
|
||||||
|
|
||||||
Each segment is 5s (configurable via `RECORDING_SEGMENT_MS`). Metadata JSON includes user info, roles, timestamps, duration.
|
|
||||||
|
|
||||||
## Vendor Packages
|
|
||||||
|
|
||||||
### discord.js-selfbot-v13 (`vendor/discord.js-selfbot-v13`)
|
|
||||||
Fork of discord.js-selfbot-v13 (git submodule). Provides Discord API access via user account.
|
|
||||||
|
|
||||||
### discord-video-stream (`vendor/discord-video-stream`)
|
|
||||||
Go Live / video streaming support library. Includes:
|
|
||||||
- H264 encoding (NVENC, VAAPI, software)
|
|
||||||
- WebRTC wrapper for Discord voice/video connections
|
|
||||||
- Stream connection management
|
|
||||||
|
|
||||||
## Dependencies
|
|
||||||
|
|
||||||
**Shared (`@bete/shared`):**
|
|
||||||
- pino — Structured logging
|
|
||||||
- zod — Schema validation
|
|
||||||
|
|
||||||
**Backend:**
|
|
||||||
- express 5 — HTTP server
|
|
||||||
- ws — WebSocket server
|
|
||||||
- helmet — Security headers
|
|
||||||
- @discordjs/voice — Voice state querying (minimal)
|
|
||||||
- drizzle-orm + pg — PostgreSQL ORM
|
|
||||||
- ioredis — Redis client
|
|
||||||
- pino, pino-http — Logging
|
|
||||||
- prom-client — Prometheus metrics
|
|
||||||
- axios — HTTP client
|
|
||||||
- zod — Config validation
|
|
||||||
|
|
||||||
**discord-gateway:**
|
|
||||||
- discord.js-selfbot-v13 — Discord client (user account)
|
|
||||||
- @discordjs/voice — Voice connection
|
|
||||||
- @discordjs/opus — Native Opus codec
|
|
||||||
- prism-media — Audio encode/decode
|
|
||||||
- @snazzah/davey — DA-VEY (Discord Audio Video End-to-end encryption)
|
|
||||||
- ioredis — Redis client
|
|
||||||
- drizzle-orm + pg — PostgreSQL ORM
|
|
||||||
- sharp — Image processing
|
|
||||||
- openai — OpenAI API client
|
|
||||||
- piscina — Worker threads for AI analysis
|
|
||||||
- tiktoken — Token counting
|
|
||||||
- p-retry, p-limit — Async utilities
|
|
||||||
- lru-cache — In-memory caching
|
|
||||||
- libsodium-wrappers — Encryption
|
|
||||||
- node-crc — CRC checksums
|
|
||||||
- imghash — Image hashing
|
|
||||||
- ws — WebSocket (internal)
|
|
||||||
- zod — Config validation
|
|
||||||
|
|
||||||
**Frontend:**
|
|
||||||
- react 19, react-dom 19
|
|
||||||
- @tanstack/react-query — Data fetching
|
|
||||||
- three, @react-three/fiber, @react-three/drei — 3D
|
|
||||||
- gsap, framer-motion — Animations
|
|
||||||
- @radix-ui/* — Accessible UI primitives
|
|
||||||
- tailwindcss 4, @tailwindcss/postcss — Styling
|
|
||||||
- lucide-react — Icons
|
|
||||||
- clsx, tailwind-merge — Class management
|
|
||||||
- vite 8 — Bundler
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- Bot uses selfbot variant (user account) — check Discord ToS
|
|
||||||
- Opus decoding requires native `@discordjs/opus` or `opusscript` under Node.js
|
|
||||||
- OGG segments include metadata JSON for each segment (user info, timestamps, duration)
|
|
||||||
- WebSocket broadcasts PCM in real-time; browser can transmit audio back to Discord
|
|
||||||
- Graceful shutdown ensures clean disconnection and resource cleanup
|
|
||||||
- All database operations use parameterized queries to prevent SQL injection
|
|
||||||
- Attachment uploads are non-blocking (async) to avoid blocking message capture
|
|
||||||
- Message capture continues even if AI analysis or attachment upload fails
|
|
||||||
|
|
||||||
## Common Tasks
|
|
||||||
|
|
||||||
### Add a new config variable
|
|
||||||
1. Add to config schema in both `services/backend/src/shared/config/index.ts` and `services/discord-gateway/src/shared/config/config.ts` with Zod validation
|
|
||||||
2. Add to `.env.example` with description
|
|
||||||
3. Use via `config.VARIABLE_NAME`
|
|
||||||
|
|
||||||
### Add a new REST endpoint
|
|
||||||
1. Create route handler in `services/backend/src/modules/<module>/<name>.routes.ts`
|
|
||||||
2. Register in `services/backend/src/http/app.ts`
|
|
||||||
3. Use `asyncHandler` wrapper for error handling
|
|
||||||
4. Return JSON response
|
|
||||||
|
|
||||||
### Add a new WebSocket event
|
|
||||||
1. Add to `eventTypes.ts` in discord-gateway
|
|
||||||
2. Add publish method to `EventBroadcaster` in discord-gateway
|
|
||||||
3. Add subscription + broadcast mapping in `services/backend/src/ws/redis-bridge.ts`
|
|
||||||
4. Add event type to `WSEventMap` in frontend `events.ts`
|
|
||||||
5. Add handler to `WsHandlers` in frontend `socket.ts`
|
|
||||||
|
|
||||||
### Add a new database table
|
|
||||||
1. Add table definition in `services/discord-gateway/src/shared/database/schema.ts`
|
|
||||||
2. Generate migration: `pnpm run db:generate`
|
|
||||||
3. Check migration file in `drizzle/migrations/`
|
|
||||||
4. Apply: `pnpm run db:migrate`
|
|
||||||
|
|
||||||
### Add a new Redis command
|
|
||||||
1. Add handler case in `commandHandler.ts` switch statement
|
|
||||||
2. Add publish call on backend side (see `voice.service.ts` or `media.service.ts`)
|
|
||||||
3. Update frontend API client if needed
|
|
||||||
|
|
||||||
### Debug AI moderation
|
|
||||||
- Set `AI_ANALYSIS_ENABLED=true` and `VERBOSE=true`
|
|
||||||
- Check `ai_status`, `ai_error` fields in messages table
|
|
||||||
- Monitor `/api/analysis/search?q=<text>` for analysis results
|
|
||||||
- Check `ai_analysis_runs` table for batch run status
|
|
||||||
- Adjust `AI_ANALYSIS_*` tuning variables
|
|
||||||
|
|
||||||
### Debug voice recording
|
|
||||||
- Set `VERBOSE=true`
|
|
||||||
- Check `/api/status` for active connection
|
|
||||||
- Monitor segment files in `recordings/<user-id>/`
|
|
||||||
- Check `voice_recordings` table for upload status
|
|
||||||
|
|
||||||
## CodeGraph Usage (Required)
|
|
||||||
|
|
||||||
- Use CodeGraph first for repo-level questions: architecture, dependencies, references, callers/callees, impact, flow, routes, components.
|
|
||||||
- If graph is missing or stale, run scan first to refresh `.codegraph/graph.json`.
|
|
||||||
- Prefer graph-backed flow:
|
|
||||||
1. scan-codegraph (build/refresh graph)
|
|
||||||
2. query-codegraph (find definitions/references/callers/dependencies)
|
|
||||||
3. analyze-codegraph (architecture, impact, risk, cycles, orphans, hotspots)
|
|
||||||
4. export-codegraph (json/mermaid/dot/markdown/html when needed)
|
|
||||||
5. open-codegraph-ui (interactive visualization when requested)
|
|
||||||
- Avoid broad grep/find or repeated wide file reads before graph lookup, except for exact literal search or known single-file edits.
|
|
||||||
@@ -1,542 +0,0 @@
|
|||||||
# CI/CD Overhaul Implementation Plan
|
|
||||||
|
|
||||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
||||||
|
|
||||||
**Goal:** Migrate from hybrid CI/CD (GitHub Actions + GitLab CI + hot-deploy) to single Gitea CI pipeline with container registry — VPS pulls only.
|
|
||||||
|
|
||||||
**Architecture:** Three Docker images (backend, discord-gateway, proxy) built in Gitea CI, pushed to `git.imrnes.team/MythEclipse/GMW/*`, VPS pulls and restarts via SSH. No more hot-deploy bind-mounts.
|
|
||||||
|
|
||||||
**Tech Stack:** Gitea CI (Act Runner, GitHub Actions-compatible syntax), Docker Buildx, Gitea Container Registry, appleboy/ssh-action
|
|
||||||
|
|
||||||
## Global Constraints
|
|
||||||
|
|
||||||
- Docker images must be self-contained (no bind-mount overlay at runtime)
|
|
||||||
- All three images must be built from monorepo root using `infra/docker/Dockerfile.*`
|
|
||||||
- Frontend static export built inside proxy Dockerfile (multi-stage, Next.js → Nginx)
|
|
||||||
- Gitea CI variables: GITEA_REGISTRY_TOKEN (secret), VPS_HOST (secret), VPS_USER (secret), VPS_SSH_KEY (secret), ENV_FILE (secret), GITEA_REGISTRY (variable)
|
|
||||||
- Registry URL: `git.imrnes.team/MythEclipse/GMW/`
|
|
||||||
- Work on `main` branch only
|
|
||||||
- Must preserve voice recordings volume persistence across container restarts
|
|
||||||
|
|
||||||
---
|
|
||||||
### Task 1: Create Gitea CI workflow
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `.gitea/workflows/deploy.yml`
|
|
||||||
|
|
||||||
**Interfaces:**
|
|
||||||
- Consumes: Dockerfiles at `infra/docker/Dockerfile.{backend,discord-gateway,proxy}`
|
|
||||||
- Produces: Docker images pushed to `git.imrnes.team/MythEclipse/GMW/bete-*:latest` and `:{sha}`
|
|
||||||
- Depends on: Task 2 (proxy Dockerfile), Task 3 (backend Dockerfile) — but workflow can reference files that are being written in the same commit
|
|
||||||
|
|
||||||
- [ ] **Step 1: Create `.gitea/workflows/` directory and `deploy.yml`**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p .gitea/workflows
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Write the workflow file**
|
|
||||||
|
|
||||||
Create `.gitea/workflows/deploy.yml`:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
name: Build & Deploy
|
|
||||||
run-name: "Build & Deploy ${{ gitea.sha }}"
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [main]
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
build-and-push:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
strategy:
|
|
||||||
fail-fast: false
|
|
||||||
max-parallel: 2
|
|
||||||
matrix:
|
|
||||||
service: [backend, discord-gateway, proxy]
|
|
||||||
steps:
|
|
||||||
- name: Checkout
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
with:
|
|
||||||
submodules: recursive
|
|
||||||
|
|
||||||
- name: Set up Docker Buildx
|
|
||||||
uses: docker/setup-buildx-action@v4
|
|
||||||
|
|
||||||
- name: Login to Gitea Registry
|
|
||||||
uses: docker/login-action@v4
|
|
||||||
with:
|
|
||||||
registry: ${{ vars.GITEA_REGISTRY }}
|
|
||||||
username: ${{ gitea.actor }}
|
|
||||||
password: ${{ secrets.GITEA_REGISTRY_TOKEN }}
|
|
||||||
|
|
||||||
- name: Build & Push ${{ matrix.service }}
|
|
||||||
uses: docker/build-push-action@v7
|
|
||||||
with:
|
|
||||||
context: .
|
|
||||||
file: infra/docker/Dockerfile.${{ matrix.service }}
|
|
||||||
push: true
|
|
||||||
tags: |
|
|
||||||
${{ vars.GITEA_REGISTRY }}/MythEclipse/GMW/bete-${{ matrix.service }}:${{ gitea.sha }}
|
|
||||||
${{ vars.GITEA_REGISTRY }}/MythEclipse/GMW/bete-${{ matrix.service }}:latest
|
|
||||||
cache-from: type=gha,scope=bete-${{ matrix.service }}
|
|
||||||
cache-to: type=gha,mode=max,scope=bete-${{ matrix.service }}
|
|
||||||
build-args: |
|
|
||||||
VITE_BE_API_URL=https://imphnen.asepharyana.my.id
|
|
||||||
VITE_BE_WS_URL=wss://imphnen.asepharyana.my.id
|
|
||||||
|
|
||||||
deploy:
|
|
||||||
needs: build-and-push
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
if: gitea.ref == 'refs/heads/main'
|
|
||||||
steps:
|
|
||||||
- name: Deploy to VPS
|
|
||||||
uses: appleboy/ssh-action@v1.2.5
|
|
||||||
env:
|
|
||||||
ENV_FILE: ${{ secrets.ENV_FILE }}
|
|
||||||
with:
|
|
||||||
host: ${{ secrets.VPS_HOST }}
|
|
||||||
username: ${{ secrets.VPS_USER }}
|
|
||||||
key: ${{ secrets.VPS_SSH_KEY }}
|
|
||||||
envs: ENV_FILE
|
|
||||||
script: |
|
|
||||||
set -eu
|
|
||||||
APP_DIR=/opt/imphenbot
|
|
||||||
cd "$APP_DIR/infra/docker"
|
|
||||||
printf '%s\n' "$ENV_FILE" | tr -d '\r' > .env
|
|
||||||
docker compose pull
|
|
||||||
docker compose up -d --remove-orphans
|
|
||||||
docker image prune -f
|
|
||||||
```
|
|
||||||
|
|
||||||
Note: Gitea's Act Runner supports `gitea.*` context variables (`gitea.sha`, `gitea.actor`, `gitea.ref`). If `gitea.*` vars don't resolve, fall back to `github.*` equivalents (Act Runner emulates GitHub context).
|
|
||||||
|
|
||||||
- [ ] **Step 3: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .gitea/workflows/deploy.yml
|
|
||||||
git commit -m "ci: add Gitea CI workflow for build & deploy
|
|
||||||
|
|
||||||
Gitea CI builds three Docker images (backend, discord-gateway, proxy),
|
|
||||||
pushes to Gitea Container Registry, then deploys to VPS via SSH pull.
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
### Task 2: Rewrite Dockerfile.proxy for Next.js static export
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Rewrite: `infra/docker/Dockerfile.proxy`
|
|
||||||
|
|
||||||
**Interfaces:**
|
|
||||||
- Consumes: `services/frontend/` (Next.js app), `packages/shared/` (workspace dep), `infra/docker/nginx/nginx.conf`
|
|
||||||
- Produces: Nginx image serving Next.js static export at `/usr/share/nginx/html/`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Rewrite Dockerfile.proxy**
|
|
||||||
|
|
||||||
Replace entire content with:
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
# ---- Stage 1: Build Next.js static export ----
|
|
||||||
FROM node:22-slim AS frontend-builder
|
|
||||||
|
|
||||||
WORKDIR /app
|
|
||||||
|
|
||||||
# Install pnpm
|
|
||||||
RUN corepack enable
|
|
||||||
|
|
||||||
# Install build essentials for native deps
|
|
||||||
RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends \
|
|
||||||
python3 make g++ && rm -rf /var/lib/apt/lists/*
|
|
||||||
|
|
||||||
# Copy dependency manifests first for layer caching
|
|
||||||
COPY pnpm-lock.yaml pnpm-workspace.yaml package.json ./
|
|
||||||
COPY packages/shared/package.json ./packages/shared/package.json
|
|
||||||
COPY services/frontend/package.json ./services/frontend/package.json
|
|
||||||
COPY services/frontend/tsconfig.json ./services/frontend/tsconfig.json
|
|
||||||
|
|
||||||
# Install dependencies (frontend + shared)
|
|
||||||
RUN --mount=type=cache,id=pnpm-store,target=/root/.local/share/pnpm/store \
|
|
||||||
pnpm install --frozen-lockfile --filter './packages/shared' --filter './services/frontend'
|
|
||||||
|
|
||||||
# Copy source code
|
|
||||||
COPY packages/shared/ ./packages/shared/
|
|
||||||
COPY services/frontend/ ./services/frontend/
|
|
||||||
|
|
||||||
# Pass API/WS URLs as build args for the frontend
|
|
||||||
ARG VITE_BE_API_URL
|
|
||||||
ARG VITE_BE_WS_URL
|
|
||||||
ENV VITE_BE_API_URL=${VITE_BE_API_URL}
|
|
||||||
ENV VITE_BE_WS_URL=${VITE_BE_WS_URL}
|
|
||||||
|
|
||||||
# Build shared lib first, then frontend static export
|
|
||||||
RUN --mount=type=cache,id=pnpm-store,target=/root/.local/share/pnpm/store \
|
|
||||||
pnpm --filter './packages/shared' run build
|
|
||||||
RUN --mount=type=cache,id=pnpm-store,target=/root/.local/share/pnpm/store \
|
|
||||||
pnpm --filter frontend run build
|
|
||||||
|
|
||||||
# ---- Stage 2: Nginx ----
|
|
||||||
FROM nginx:alpine
|
|
||||||
|
|
||||||
# Nginx config (API/WS proxy + static file serving)
|
|
||||||
COPY infra/docker/nginx/nginx.conf /etc/nginx/conf.d/default.conf
|
|
||||||
|
|
||||||
# Static export from frontend builder
|
|
||||||
COPY --from=frontend-builder /app/services/frontend/out/ /usr/share/nginx/html/
|
|
||||||
|
|
||||||
EXPOSE 80
|
|
||||||
|
|
||||||
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
|
||||||
CMD wget -qO- http://localhost:80/ || exit 1
|
|
||||||
|
|
||||||
CMD ["nginx", "-g", "daemon off;"]
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Validate nginx.conf handles static files correctly**
|
|
||||||
|
|
||||||
Read and confirm `infra/docker/nginx/nginx.conf`.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cat infra/docker/nginx/nginx.conf
|
|
||||||
```
|
|
||||||
|
|
||||||
Verify it has:
|
|
||||||
- Static file location with `try_files $uri /index.html` (SPA fallback)
|
|
||||||
- `/api` and `/ws` proxied to `http://backend:3000`
|
|
||||||
|
|
||||||
- [ ] **Step 3: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add infra/docker/Dockerfile.proxy
|
|
||||||
git commit -m "docker(proxy): rewrite for Next.js static export
|
|
||||||
|
|
||||||
Replaced stale Rust WASM build with multi-stage Docker build:
|
|
||||||
stage 1 builds Next.js static export, stage 2 serves via Nginx.
|
|
||||||
Includes VITE_BE_API_URL/VITE_BE_WS_URL build args.
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
### Task 3: Add build args to Dockerfile.backend
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `infra/docker/Dockerfile.backend`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Add VITE build args to Dockerfile.backend**
|
|
||||||
|
|
||||||
Insert after `WORKDIR /app`:
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
# Build args for frontend API URLs (passed through for future use)
|
|
||||||
ARG VITE_BE_API_URL
|
|
||||||
ARG VITE_BE_WS_URL
|
|
||||||
ENV VITE_BE_API_URL=${VITE_BE_API_URL}
|
|
||||||
ENV VITE_BE_WS_URL=${VITE_BE_WS_URL}
|
|
||||||
```
|
|
||||||
|
|
||||||
Note: These are consumed by the proxy Dockerfile (Task 2), not needed by backend itself but passed through the CI workflow to all three images for consistency.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add infra/docker/Dockerfile.backend
|
|
||||||
git commit -m "docker(backend): add VITE_BE_API_URL and VITE_BE_WS_URL build args
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
### Task 4: Rewrite docker-compose.yml for Gitea registry + no bind-mounts
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Rewrite: `infra/docker/docker-compose.yml`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write new docker-compose.yml**
|
|
||||||
|
|
||||||
Replace entire content:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
version: '3.8'
|
|
||||||
|
|
||||||
services:
|
|
||||||
proxy:
|
|
||||||
image: ${GITEA_REGISTRY}/MythEclipse/GMW/bete-proxy:${IMAGE_TAG:-latest}
|
|
||||||
container_name: imphenbot-proxy
|
|
||||||
restart: unless-stopped
|
|
||||||
labels:
|
|
||||||
- "traefik.enable=true"
|
|
||||||
- "traefik.http.routers.imphenbot.rule=Host(`imphnen.asepharyana.my.id`)"
|
|
||||||
- "traefik.http.routers.imphenbot.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.imphenbot.tls=true"
|
|
||||||
- "traefik.http.services.imphenbot.loadbalancer.server.port=80"
|
|
||||||
depends_on:
|
|
||||||
- backend
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1/"]
|
|
||||||
interval: 30s
|
|
||||||
timeout: 5s
|
|
||||||
retries: 3
|
|
||||||
deploy:
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
memory: 64M
|
|
||||||
networks:
|
|
||||||
- app-shared-net
|
|
||||||
|
|
||||||
backend:
|
|
||||||
image: ${GITEA_REGISTRY}/MythEclipse/GMW/bete-backend:${IMAGE_TAG:-latest}
|
|
||||||
container_name: imphenbot-backend
|
|
||||||
restart: unless-stopped
|
|
||||||
env_file:
|
|
||||||
- .env
|
|
||||||
environment:
|
|
||||||
NODE_ENV: production
|
|
||||||
WEBSERVER_PORT: 3000
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/health"]
|
|
||||||
interval: 30s
|
|
||||||
timeout: 10s
|
|
||||||
start_period: 15s
|
|
||||||
retries: 3
|
|
||||||
deploy:
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
memory: 256M
|
|
||||||
networks:
|
|
||||||
- app-shared-net
|
|
||||||
|
|
||||||
discord-gateway:
|
|
||||||
image: ${GITEA_REGISTRY}/MythEclipse/GMW/bete-discord-gateway:${IMAGE_TAG:-latest}
|
|
||||||
container_name: imphenbot-discord-gateway
|
|
||||||
restart: unless-stopped
|
|
||||||
env_file:
|
|
||||||
- .env
|
|
||||||
environment:
|
|
||||||
NODE_ENV: production
|
|
||||||
volumes:
|
|
||||||
- recordings:/app/recordings
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD-SHELL", "kill -0 1 || exit 1"]
|
|
||||||
interval: 30s
|
|
||||||
timeout: 5s
|
|
||||||
start_period: 30s
|
|
||||||
retries: 3
|
|
||||||
deploy:
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
memory: 512M
|
|
||||||
networks:
|
|
||||||
- app-shared-net
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
recordings:
|
|
||||||
|
|
||||||
networks:
|
|
||||||
app-shared-net:
|
|
||||||
name: app-shared-net
|
|
||||||
external: true
|
|
||||||
```
|
|
||||||
|
|
||||||
Key changes:
|
|
||||||
- Image refs: `registry.gitlab.com/...` → `${GITEA_REGISTRY}/MythEclipse/GMW/...`
|
|
||||||
- Removed all bind-mount volumes: `./backend-dist`, `./gateway-dist`, `./frontend-dist`, `./shared-dist`
|
|
||||||
- Changed `./recordings` bind-mount → named volume `recordings:` (persists across restarts)
|
|
||||||
- Added `depends_on: backend` to proxy (proxy needs backend for API/WS, though Nginx handles startup gracefully)
|
|
||||||
|
|
||||||
- [ ] **Step 2: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add infra/docker/docker-compose.yml
|
|
||||||
git commit -m "docker(compose): switch to Gitea registry, remove bind-mounts
|
|
||||||
|
|
||||||
Images now come from git.imrnes.team/MythEclipse/GMW. All hot-deploy
|
|
||||||
bind-mounts removed — containers are fully self-contained. Voice
|
|
||||||
recordings use a named volume instead of bind-mount.
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
### Task 5: Create lightweight deploy.sh
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `deploy.sh`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write deploy.sh**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
#!/bin/bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
||||||
INFRA_DIR="$SCRIPT_DIR/infra/docker"
|
|
||||||
|
|
||||||
: "${VPS_HOST:?required}"
|
|
||||||
: "${VPS_USER:?required}"
|
|
||||||
: "${VPS_SSH_KEY:?required}"
|
|
||||||
|
|
||||||
echo "=== Deploy to $VPS_HOST ==="
|
|
||||||
|
|
||||||
# Copy local .env if it exists (overrides CI env)
|
|
||||||
if [ -f "$INFRA_DIR/.env" ]; then
|
|
||||||
scp -i "$VPS_SSH_KEY" "$INFRA_DIR/.env" "$VPS_USER@$VPS_HOST:/opt/imphenbot/infra/docker/.env"
|
|
||||||
fi
|
|
||||||
|
|
||||||
ssh -i "$VPS_SSH_KEY" "$VPS_USER@$VPS_HOST" << 'REMOTESCRIPT'
|
|
||||||
set -eu
|
|
||||||
cd /opt/imphenbot/infra/docker
|
|
||||||
echo "=== Pulling images ==="
|
|
||||||
docker compose pull
|
|
||||||
echo "=== Restarting containers ==="
|
|
||||||
docker compose up -d --remove-orphans
|
|
||||||
echo "=== Cleaning up ==="
|
|
||||||
docker image prune -f
|
|
||||||
echo "=== Active containers ==="
|
|
||||||
docker ps --filter "name=imphenbot" --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
|
|
||||||
REMOTESCRIPT
|
|
||||||
|
|
||||||
echo "=== Deploy complete ==="
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Make executable**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
chmod +x deploy.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 3: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add deploy.sh
|
|
||||||
git commit -m "chore: rewrite deploy.sh as lightweight SSH pull script
|
|
||||||
|
|
||||||
Replaced hot-deploy tar-pipe script with simple SSH-based deploy
|
|
||||||
that pulls latest images from Gitea registry and restarts containers.
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
### Task 6: Disable old CI files
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Disable: `.github/workflows/deploy-docker.yml`
|
|
||||||
- Keep: `.gitlab-ci.yml` if exists (already may have been removed)
|
|
||||||
|
|
||||||
- [ ] **Step 1: Rename GitHub Actions workflow to .disabled**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mv .github/workflows/deploy-docker.yml .github/workflows/deploy-docker.yml.disabled
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Remove docker compose file's old frontend-dist directory from git** (if tracked)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check if frontend-dist is tracked (it should be gitignored, but check)
|
|
||||||
git ls-files infra/docker/frontend-dist 2>/dev/null || echo "Not tracked — OK"
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 3: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .github/workflows/deploy-docker.yml.disabled
|
|
||||||
git rm --cached .github/workflows/deploy-docker.yml 2>/dev/null || true
|
|
||||||
git commit -m "ci: disable GitHub Actions workflow
|
|
||||||
|
|
||||||
Renamed to .disabled. All CI now goes through Gitea CI (.gitea/workflows/).
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
### Task 7: Update .gitignore
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `.gitignore`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Add .gitea exclusion note and any missing entries**
|
|
||||||
|
|
||||||
Read current `.gitignore`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cat .gitignore
|
|
||||||
```
|
|
||||||
|
|
||||||
Then append (only if not already present):
|
|
||||||
|
|
||||||
```
|
|
||||||
# Gitea workflow logs (local runners)
|
|
||||||
.gitea/workflows/*.log
|
|
||||||
```
|
|
||||||
|
|
||||||
The `.gitea/workflows/` YAML files themselves should be tracked in git.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .gitignore
|
|
||||||
git commit -m "chore: update gitignore for Gitea CI artifacts
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
### Task 8: Push and verify CI pipeline
|
|
||||||
|
|
||||||
- [ ] **Step 1: Verify all changes**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git status
|
|
||||||
git log --oneline -10
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: clean working tree, all 7 commits ready to push.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Push to main**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git push origin main
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 3: Monitor CI run**
|
|
||||||
|
|
||||||
Watch Gitea CI at `https://git.imrnes.team/MythEclipse/GMW/actions`.
|
|
||||||
|
|
||||||
Expected outcome:
|
|
||||||
1. `build-and-push` job runs 3 matrix builds (backend, discord-gateway, proxy) in parallel (max 2)
|
|
||||||
2. Each image is pushed to `git.imrnes.team/MythEclipse/GMW/bete-*` with both `:latest` and `:{sha}` tags
|
|
||||||
3. `deploy` job SSHes into VPS, pulls images, restarts containers
|
|
||||||
4. All 3 containers `imphenbot-proxy`, `imphenbot-backend`, `imphenbot-discord-gateway` are running
|
|
||||||
|
|
||||||
- [ ] **Step 4: Verify containers on VPS**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# SSH into VPS and check
|
|
||||||
ssh -i "$VPS_SSH_KEY" "$VPS_USER@$VPS_HOST" "
|
|
||||||
docker ps --filter 'name=imphenbot' --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}'
|
|
||||||
docker compose -f /opt/imphenbot/infra/docker/docker-compose.yml ps
|
|
||||||
"
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 5: Verify no hot-deploy artifacts remain**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ssh -i "$VPS_SSH_KEY" "$VPS_USER@$VPS_HOST" "
|
|
||||||
ls -la /opt/imphenbot/infra/docker/ | grep -E 'dist$' || echo 'No dist dirs — clean'
|
|
||||||
"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
## Rollback
|
|
||||||
|
|
||||||
If the pipeline fails at any point:
|
|
||||||
|
|
||||||
1. **Fix and re-push**: Edit the broken file, commit, push to main — CI re-runs automatically
|
|
||||||
2. **Emergency rollback**: SSH to VPS, run `docker compose up -d` with a known-good IMAGE_TAG:
|
|
||||||
```bash
|
|
||||||
IMAGE_TAG=<last-working-sha> docker compose up -d
|
|
||||||
```
|
|
||||||
3. **Restore old CI**: Move `.github/workflows/deploy-docker.yml.disabled` back and push
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,736 +0,0 @@
|
|||||||
# Backend & Gateway Refactoring — Phase 1 Implementation Plan
|
|
||||||
|
|
||||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
||||||
|
|
||||||
**Goal:** Clean up ~130 lines of dead/duplicate code, consolidate duplicated database initialization, and simplify the MessageStore layering in discord-gateway.
|
|
||||||
|
|
||||||
**Architecture:** Three independent tasks that can be done in any order. Task 1 consolidates database pool/drizzle init into `@bete/shared` so both services use one canonical pattern. Task 2 removes backward-compat function wrappers from `messageStore.ts`. Task 3 deletes dead files and functions.
|
|
||||||
|
|
||||||
**Tech Stack:** TypeScript, Node.js, Drizzle ORM, PostgreSQL, pnpm workspace
|
|
||||||
|
|
||||||
## Global Constraints
|
|
||||||
|
|
||||||
- All imports use `.js` extensions (ESM convention)
|
|
||||||
- Follow existing code style (Biome, 2-space indent)
|
|
||||||
- Keep `@bete/shared` as the single source of truth for shared infrastructure
|
|
||||||
- No package.json changes needed — `@bete/shared` already has `drizzle-orm` and `pg` as dependencies
|
|
||||||
- Do not change any business logic — only structural refactoring
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 1: Consolidate Database Initialization into `@bete/shared`
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `packages/shared/src/database/init.ts`
|
|
||||||
- Modify: `packages/shared/src/database/pool.ts` — add `getPool()` export
|
|
||||||
- Modify: `packages/shared/src/index.ts` — export new `./database/init.js`
|
|
||||||
- Modify: `packages/shared/package.json` — add `"./database/init"` export entry
|
|
||||||
- Modify: `services/backend/src/shared/database/index.ts` — re-export from shared
|
|
||||||
- Modify: `services/discord-gateway/src/shared/database/drizzle.ts` — re-export from shared
|
|
||||||
- Delete: (functions migrate, no file deletion here — both local files stay as thin wrappers)
|
|
||||||
|
|
||||||
**Interfaces:**
|
|
||||||
- Produces:
|
|
||||||
- `@bete/shared/database/init` exports:
|
|
||||||
- `let db: ReturnType<typeof drizzle> | null` (module-level, for getDatabase())
|
|
||||||
- `let rawPool: Pool | null` (module-level, for getPool())
|
|
||||||
- `initializeDatabase(schema?: Record<string, unknown>): Promise<ReturnType<typeof drizzle>>` — creates pool via `createPoolFromConfig`, wraps with `drizzle()`. Accepts optional schema object (gateway needs it, backend doesn't). Reads config from env/config module internally.
|
|
||||||
- `getDatabase(): ReturnType<typeof drizzle>` — throws if not initialized
|
|
||||||
- `getPool(): Pool` — returns raw pool for raw SQL queries, throws if not initialized
|
|
||||||
- `closeDatabase(): Promise<void>` — closes pool and nullifies references
|
|
||||||
- `executeAll(sql: string, params?: unknown[]): Promise<unknown[]>` — raw SQL query, returns all rows
|
|
||||||
- `executeGet(sql: string, params?: unknown[]): Promise<unknown>` — raw SQL query, returns first row or null
|
|
||||||
- `withDatabaseClient<T>(callback: (client: PoolClient) => Promise<T>): Promise<T>`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Create `packages/shared/src/database/init.ts`**
|
|
||||||
|
|
||||||
This is the canonical database initialization module. It merges what both services currently do:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
import { createChildLogger } from "@bete/shared/logger";
|
|
||||||
import { closePool, createPoolFromConfig } from "@bete/shared/database/pool";
|
|
||||||
import { drizzle } from "drizzle-orm/node-postgres";
|
|
||||||
import type { Pool, PoolClient } from "pg";
|
|
||||||
import { config } from "../config/index.js";
|
|
||||||
|
|
||||||
const logger = createChildLogger("database.init");
|
|
||||||
|
|
||||||
let db: ReturnType<typeof drizzle> | null = null;
|
|
||||||
let rawPool: Pool | null = null;
|
|
||||||
|
|
||||||
export async function initializeDatabase(schema?: Record<string, unknown>) {
|
|
||||||
if (db !== null) return db;
|
|
||||||
|
|
||||||
const pool = config.DATABASE_URL
|
|
||||||
? createPoolFromConfig({
|
|
||||||
url: config.DATABASE_URL,
|
|
||||||
min: config.POSTGRES_POOL_MIN,
|
|
||||||
max: config.POSTGRES_POOL_MAX,
|
|
||||||
})
|
|
||||||
: createPoolFromConfig({
|
|
||||||
host: config.POSTGRES_HOST,
|
|
||||||
port: config.POSTGRES_PORT,
|
|
||||||
user: config.POSTGRES_USER,
|
|
||||||
password: config.POSTGRES_PASSWORD,
|
|
||||||
database: config.POSTGRES_DB,
|
|
||||||
min: config.POSTGRES_POOL_MIN,
|
|
||||||
max: config.POSTGRES_POOL_MAX,
|
|
||||||
});
|
|
||||||
|
|
||||||
rawPool = pool;
|
|
||||||
db = drizzle(pool, schema ? { schema } : undefined);
|
|
||||||
|
|
||||||
// Test connection
|
|
||||||
try {
|
|
||||||
const client = await pool.connect();
|
|
||||||
client.release();
|
|
||||||
logger.info("Database connection successful");
|
|
||||||
} catch (err) {
|
|
||||||
logger.error({ err }, "Failed to connect to database");
|
|
||||||
throw err;
|
|
||||||
}
|
|
||||||
|
|
||||||
return db;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function getDatabase() {
|
|
||||||
if (db === null) {
|
|
||||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
|
||||||
}
|
|
||||||
return db;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function getPool() {
|
|
||||||
if (!rawPool) {
|
|
||||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
|
||||||
}
|
|
||||||
return rawPool;
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function closeDatabase() {
|
|
||||||
if (rawPool !== null) {
|
|
||||||
await closePool(rawPool);
|
|
||||||
}
|
|
||||||
rawPool = null;
|
|
||||||
db = null;
|
|
||||||
logger.info("Database connection closed");
|
|
||||||
}
|
|
||||||
|
|
||||||
function convertPlaceholdersForPostgres(sql: string) {
|
|
||||||
let i = 0;
|
|
||||||
return sql.replace(/\?/g, () => `$${++i}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function executeAll(sql: string, params?: unknown[]) {
|
|
||||||
if (!rawPool) {
|
|
||||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
|
||||||
}
|
|
||||||
const query = convertPlaceholdersForPostgres(sql);
|
|
||||||
const result = await rawPool.query(query, params || []);
|
|
||||||
return result.rows;
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function executeGet(sql: string, params?: unknown[]) {
|
|
||||||
if (!rawPool) {
|
|
||||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
|
||||||
}
|
|
||||||
const query = convertPlaceholdersForPostgres(sql);
|
|
||||||
const result = await rawPool.query(query, params || []);
|
|
||||||
return result.rows[0] ?? null;
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function withDatabaseClient<T>(
|
|
||||||
callback: (client: PoolClient) => Promise<T>,
|
|
||||||
): Promise<T> {
|
|
||||||
if (!rawPool) {
|
|
||||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
|
||||||
}
|
|
||||||
const client = await rawPool.connect();
|
|
||||||
try {
|
|
||||||
return await callback(client);
|
|
||||||
} finally {
|
|
||||||
client.release();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Note:** This uses `config` from `@bete/shared/config`. The backend's config proxies to that already (`services/backend/src/shared/config/index.ts` re-exports from `@bete/shared/config`). The gateway's config at `services/discord-gateway/src/shared/config/config.ts` has the same field names but is its own Zod schema. Since `@bete/shared/config` doesn't have the PostgreSQL pool config fields currently, we need to check what it exports.
|
|
||||||
|
|
||||||
Actually — `@bete/shared/config` may not have `POSTGRES_HOST` etc. Let me adjust: the `initializeDatabase` function should accept config values as parameters instead of reading from a shared config.
|
|
||||||
|
|
||||||
Revised approach for `packages/shared/src/database/init.ts`:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
import { createChildLogger } from "@bete/shared/logger";
|
|
||||||
import { closePool, createPoolFromConfig } from "./pool.js";
|
|
||||||
import { drizzle } from "drizzle-orm/node-postgres";
|
|
||||||
import type { Pool, PoolClient } from "pg";
|
|
||||||
|
|
||||||
const logger = createChildLogger("database.init");
|
|
||||||
|
|
||||||
let db: ReturnType<typeof drizzle> | null = null;
|
|
||||||
let rawPool: Pool | null = null;
|
|
||||||
|
|
||||||
export interface DatabaseConfig {
|
|
||||||
DATABASE_URL?: string;
|
|
||||||
POSTGRES_HOST?: string;
|
|
||||||
POSTGRES_PORT?: number;
|
|
||||||
POSTGRES_USER?: string;
|
|
||||||
POSTGRES_PASSWORD?: string;
|
|
||||||
POSTGRES_DB?: string;
|
|
||||||
POSTGRES_POOL_MIN?: number;
|
|
||||||
POSTGRES_POOL_MAX?: number;
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function initializeDatabase(
|
|
||||||
cfg: DatabaseConfig,
|
|
||||||
schema?: Record<string, unknown>,
|
|
||||||
) {
|
|
||||||
if (db !== null) return db;
|
|
||||||
|
|
||||||
const pool = cfg.DATABASE_URL
|
|
||||||
? createPoolFromConfig({
|
|
||||||
url: cfg.DATABASE_URL,
|
|
||||||
min: cfg.POSTGRES_POOL_MIN,
|
|
||||||
max: cfg.POSTGRES_POOL_MAX,
|
|
||||||
})
|
|
||||||
: createPoolFromConfig({
|
|
||||||
host: cfg.POSTGRES_HOST,
|
|
||||||
port: cfg.POSTGRES_PORT,
|
|
||||||
user: cfg.POSTGRES_USER,
|
|
||||||
password: cfg.POSTGRES_PASSWORD,
|
|
||||||
database: cfg.POSTGRES_DB,
|
|
||||||
min: cfg.POSTGRES_POOL_MIN,
|
|
||||||
max: cfg.POSTGRES_POOL_MAX,
|
|
||||||
});
|
|
||||||
|
|
||||||
rawPool = pool;
|
|
||||||
db = drizzle(pool, schema ? { schema } : undefined);
|
|
||||||
|
|
||||||
try {
|
|
||||||
const client = await pool.connect();
|
|
||||||
client.release();
|
|
||||||
logger.info("Database connection successful");
|
|
||||||
} catch (err) {
|
|
||||||
logger.error({ err }, "Failed to connect to database");
|
|
||||||
throw err;
|
|
||||||
}
|
|
||||||
|
|
||||||
return db;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function getDatabase() {
|
|
||||||
if (db === null) {
|
|
||||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
|
||||||
}
|
|
||||||
return db;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function getPool() {
|
|
||||||
if (!rawPool) {
|
|
||||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
|
||||||
}
|
|
||||||
return rawPool;
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function closeDatabase() {
|
|
||||||
if (rawPool !== null) {
|
|
||||||
await closePool(rawPool);
|
|
||||||
}
|
|
||||||
rawPool = null;
|
|
||||||
db = null;
|
|
||||||
logger.info("Database connection closed");
|
|
||||||
}
|
|
||||||
|
|
||||||
function convertPlaceholdersForPostgres(sql: string) {
|
|
||||||
let i = 0;
|
|
||||||
return sql.replace(/\?/g, () => `$${++i}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function executeAll(sql: string, params?: unknown[]) {
|
|
||||||
if (!rawPool) {
|
|
||||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
|
||||||
}
|
|
||||||
const query = convertPlaceholdersForPostgres(sql);
|
|
||||||
const result = await rawPool.query(query, params || []);
|
|
||||||
return result.rows;
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function executeGet(sql: string, params?: unknown[]) {
|
|
||||||
if (!rawPool) {
|
|
||||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
|
||||||
}
|
|
||||||
const query = convertPlaceholdersForPostgres(sql);
|
|
||||||
const result = await rawPool.query(query, params || []);
|
|
||||||
return result.rows[0] ?? null;
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function withDatabaseClient<T>(
|
|
||||||
callback: (client: PoolClient) => Promise<T>,
|
|
||||||
): Promise<T> {
|
|
||||||
if (!rawPool) {
|
|
||||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
|
||||||
}
|
|
||||||
const client = await rawPool.connect();
|
|
||||||
try {
|
|
||||||
return await callback(client);
|
|
||||||
} finally {
|
|
||||||
client.release();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Add export to `packages/shared/src/index.ts`**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
export * from "./database/init.js";
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 3: Add export to `packages/shared/package.json`**
|
|
||||||
|
|
||||||
```json
|
|
||||||
"./database/init": "./dist/database/init.js",
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 4: Build the shared package to verify it compiles**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /home/code/GMW/packages/shared
|
|
||||||
pnpm run build
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 5: Rewrite `services/backend/src/shared/database/index.ts`**
|
|
||||||
|
|
||||||
Change to a thin wrapper that imports from `@bete/shared/database/init` and passes the backend's config:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
import { createChildLogger } from "@bete/shared/logger";
|
|
||||||
import { initializeDatabase as sharedInit, getDatabase as sharedGetDb, getPool as sharedGetPool, closeDatabase as sharedCloseDb } from "@bete/shared/database/init";
|
|
||||||
import { config } from "../config/index.js";
|
|
||||||
|
|
||||||
const logger = createChildLogger("database");
|
|
||||||
|
|
||||||
const dbConfig = {
|
|
||||||
DATABASE_URL: config.DATABASE_URL,
|
|
||||||
POSTGRES_HOST: config.POSTGRES_HOST,
|
|
||||||
POSTGRES_PORT: config.POSTGRES_PORT,
|
|
||||||
POSTGRES_USER: config.POSTGRES_USER,
|
|
||||||
POSTGRES_PASSWORD: config.POSTGRES_PASSWORD,
|
|
||||||
POSTGRES_DB: config.POSTGRES_DB,
|
|
||||||
POSTGRES_POOL_MIN: config.POSTGRES_POOL_MIN,
|
|
||||||
POSTGRES_POOL_MAX: config.POSTGRES_POOL_MAX,
|
|
||||||
};
|
|
||||||
|
|
||||||
export async function initializeDatabase() {
|
|
||||||
logger.info("Initializing database");
|
|
||||||
return sharedInit(dbConfig);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function getDatabase() {
|
|
||||||
return sharedGetDb();
|
|
||||||
}
|
|
||||||
|
|
||||||
export function getPool() {
|
|
||||||
return sharedGetPool();
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function closeDatabase() {
|
|
||||||
logger.info("Closing database");
|
|
||||||
return sharedCloseDb();
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 6: Rewrite `services/discord-gateway/src/shared/database/drizzle.ts`**
|
|
||||||
|
|
||||||
Change to a thin wrapper:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
import { createChildLogger } from "@bete/shared/logger";
|
|
||||||
import { initializeDatabase as sharedInit, getDatabase as sharedGetDb, closeDatabase as sharedCloseDb, executeAll as sharedExecAll, executeGet as sharedExecGet, withDatabaseClient as sharedWithClient } from "@bete/shared/database/init";
|
|
||||||
import { config } from "../../shared/config/config.js";
|
|
||||||
import * as schema from "./schema.js";
|
|
||||||
|
|
||||||
const logger = createChildLogger("drizzle");
|
|
||||||
|
|
||||||
const dbConfig = {
|
|
||||||
DATABASE_URL: config.DATABASE_URL,
|
|
||||||
POSTGRES_HOST: config.POSTGRES_HOST,
|
|
||||||
POSTGRES_PORT: config.POSTGRES_PORT,
|
|
||||||
POSTGRES_USER: config.POSTGRES_USER,
|
|
||||||
POSTGRES_PASSWORD: config.POSTGRES_PASSWORD,
|
|
||||||
POSTGRES_DB: config.POSTGRES_DB,
|
|
||||||
POSTGRES_POOL_MIN: config.POSTGRES_POOL_MIN,
|
|
||||||
POSTGRES_POOL_MAX: config.POSTGRES_POOL_MAX,
|
|
||||||
};
|
|
||||||
|
|
||||||
export async function initializeDatabase() {
|
|
||||||
return sharedInit(dbConfig, schema);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function getDatabase() {
|
|
||||||
return sharedGetDb();
|
|
||||||
}
|
|
||||||
|
|
||||||
export { sharedCloseDb as closeDatabase };
|
|
||||||
export { sharedExecAll as executeAll, sharedExecGet as executeGet, sharedWithClient as withDatabaseClient };
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 7: Run typecheck on all packages to verify**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /home/code/GMW
|
|
||||||
pnpm run typecheck
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 8: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add packages/shared/src/database/init.ts packages/shared/src/index.ts packages/shared/package.json
|
|
||||||
git add services/backend/src/shared/database/index.ts services/discord-gateway/src/shared/database/drizzle.ts
|
|
||||||
git commit -m "refactor: consolidate database initialization into @bete/shared/database/init"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 2: Remove Backward-Compat Function Wrappers from MessageStore
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `services/discord-gateway/src/modules/message-capture/messageStore.ts` — remove lines 310-398 (backward-compat wrappers), export singleton directly
|
|
||||||
- Modify: `services/discord-gateway/src/modules/message-capture/index.ts` — update re-exports to use `messageStore` singleton
|
|
||||||
- Modify: `services/discord-gateway/src/modules/message-capture/messageCapture.ts` — update imports to use `messageStore.methodName()`
|
|
||||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/batchProcessor.ts` — update imports
|
|
||||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/batchScheduler.ts` — update imports
|
|
||||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/individualFallbackProcessor.ts` — update imports
|
|
||||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/moderationBuilders.ts` — update imports
|
|
||||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/aiAnalysisWorker.ts` — update imports (uses `getConversationContextBefore` and `updateMessagesAIAnalysisBulk`)
|
|
||||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/aiAnalyzer.ts` — update imports (uses many functions)
|
|
||||||
- Possibly modify: other files that import the wrapper functions
|
|
||||||
|
|
||||||
**Interfaces:**
|
|
||||||
- Consumes: Existing `MessageStore` class methods (unchanged signatures)
|
|
||||||
- Produces: Singleton `messageStore` instance as the single export point
|
|
||||||
|
|
||||||
The key insight: the backward-compat wrappers at lines 310-398 of `messageStore.ts` are function-level exports that delegate to `getInstance()`. Every importer can instead import the singleton `messageStore` instance and call methods on it directly.
|
|
||||||
|
|
||||||
Current importers of wrapper functions:
|
|
||||||
|
|
||||||
| File | Functions Used |
|
|
||||||
|------|---------------|
|
|
||||||
| `messageCapture.ts` | `getMessageById`, `insertMessageEdit`, `updateMessageAsEdited`, `updateMessageAsDeleted`, `upsertMessageForCapture` |
|
|
||||||
| `batchProcessor.ts` | `updateMessagesAIAnalysisBulk` |
|
|
||||||
| `batchScheduler.ts` | `getPendingMessagesByConversation` |
|
|
||||||
| `individualFallbackProcessor.ts` | `updateMessagesAIAnalysisBulk` |
|
|
||||||
| `moderationBuilders.ts` | `getMessageById` |
|
|
||||||
| `aiAnalysisWorker.ts` | `getConversationContextBefore`, `updateMessagesAIAnalysisBulk` |
|
|
||||||
| `aiAnalyzer.ts` | `getConversationKeysWithIncompleteAnalysis`, `getIncompleteMessagesByConversation`, `getMessageById`, `getPendingConversationKeys`, `updateMessageAIAnalysis` |
|
|
||||||
|
|
||||||
- [ ] **Step 1: Modify `messageStore.ts`** — replace backward-compat wrappers with a singleton export
|
|
||||||
|
|
||||||
Replace lines 22-33 (lazy singleton pattern) and lines 310-398 (wrapper functions) with:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// ─── Singleton instance ─────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
const logger = createChildLogger("message-store");
|
|
||||||
const database = getDatabase() as unknown as NodePgDatabase<typeof schema>;
|
|
||||||
export const messageStore = new MessageStore(database, logger);
|
|
||||||
```
|
|
||||||
|
|
||||||
Then remove everything from line 310 onward (the backward-compat function wrappers section).
|
|
||||||
|
|
||||||
- [ ] **Step 2: Update `message-capture/index.ts`**
|
|
||||||
|
|
||||||
Change the re-exports from individual functions to the `messageStore` singleton:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
export { messageStore } from "../message-capture/messageStore.js";
|
|
||||||
export {
|
|
||||||
getDisplayContent,
|
|
||||||
getMessageLocation,
|
|
||||||
getMessageMetadata,
|
|
||||||
} from "../message-capture/messageMetadata.js";
|
|
||||||
// ... rest unchanged
|
|
||||||
```
|
|
||||||
|
|
||||||
Also remove the individual function re-exports since they no longer exist.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Update `messageCapture.ts`**
|
|
||||||
|
|
||||||
Change imports from:
|
|
||||||
```typescript
|
|
||||||
import {
|
|
||||||
getMessageById,
|
|
||||||
insertMessageEdit,
|
|
||||||
upsertMessageForCapture,
|
|
||||||
updateMessageAsDeleted,
|
|
||||||
updateMessageAsEdited,
|
|
||||||
} from "./messageStore.js";
|
|
||||||
```
|
|
||||||
To:
|
|
||||||
```typescript
|
|
||||||
import { messageStore } from "./messageStore.js";
|
|
||||||
```
|
|
||||||
|
|
||||||
Then update every call site:
|
|
||||||
- `upsertMessageForCapture(messageRecord)` → `messageStore.upsertMessageForCapture(messageRecord)`
|
|
||||||
- `insertMessageEdit(...)` → `messageStore.insertMessageEdit(...)`
|
|
||||||
- `updateMessageAsEdited(...)` → `messageStore.updateMessageAsEdited(...)`
|
|
||||||
- `updateMessageAsDeleted(...)` → `messageStore.updateMessageAsDeleted(...)`
|
|
||||||
- `getMessageById(...)` → `messageStore.getMessageById(...)`
|
|
||||||
|
|
||||||
- [ ] **Step 4: Update `batchProcessor.ts`**
|
|
||||||
|
|
||||||
Change from:
|
|
||||||
```typescript
|
|
||||||
import { updateMessagesAIAnalysisBulk } from "../message-capture/messageStore.js";
|
|
||||||
```
|
|
||||||
To:
|
|
||||||
```typescript
|
|
||||||
import { messageStore } from "../message-capture/messageStore.js";
|
|
||||||
```
|
|
||||||
|
|
||||||
Then update call sites:
|
|
||||||
- `updateMessagesAIAnalysisBulk(updates)` → `messageStore.messages.updateMessagesAIAnalysisBulk(updates)`
|
|
||||||
|
|
||||||
Wait — `updateMessagesAIAnalysisBulk` is actually defined in `MessagesAnalysis` class, which is called via `MessageStore` → `MessagesDb` → `MessagesAnalysis`. Let me check the actual delegation chain.
|
|
||||||
|
|
||||||
Looking at the wrapper functions:
|
|
||||||
```typescript
|
|
||||||
export const updateMessagesAIAnalysisBulk = (
|
|
||||||
updates: Array<{ messageId: string; result: AIAnalysisUpdate }>,
|
|
||||||
): Promise<MessageRecord[]> =>
|
|
||||||
getInstance().updateMessagesAIAnalysisBulk(updates);
|
|
||||||
```
|
|
||||||
|
|
||||||
And in the class:
|
|
||||||
```typescript
|
|
||||||
class MessageStore {
|
|
||||||
readonly messages: MessagesDb;
|
|
||||||
// ...
|
|
||||||
}
|
|
||||||
|
|
||||||
class MessagesDb {
|
|
||||||
readonly analysis: MessagesAnalysis;
|
|
||||||
// ...
|
|
||||||
updateMessagesAIAnalysisBulk(...) {
|
|
||||||
return this.analysis.updateMessagesAIAnalysisBulk(...)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
So the call chain is: `messageStore.messages.updateMessagesAIAnalysisBulk()`. But actually, looking at `MessagesDb`, it might have its own `updateMessagesAIAnalysisBulk` that delegates to `this.analysis.updateMessagesAIAnalysisBulk()`. Let me verify...
|
|
||||||
|
|
||||||
Actually, for simplicity and to minimize changes, let me look at whether `MessagesDb` has `updateMessagesAIAnalysisBulk` or if only the wrapper has it.
|
|
||||||
|
|
||||||
Let me check:
|
|
||||||
|
|
||||||
Actually I already read that `MessagesDb` has methods. Let me look at what methods `MessagesDb` exposes vs the wrapper functions.
|
|
||||||
|
|
||||||
Instead of guessing, the safe approach is to keep the thin function wrappers but simplify them. Actually, a better approach for this task:
|
|
||||||
|
|
||||||
**Revised approach:** Instead of making all importers use `messageStore.messages.analysis.methodName()`, add all the forwarded methods directly to the `MessageStore` class (which it already does for most), and just have external files import the singleton and call `messageStore.methodName()`.
|
|
||||||
|
|
||||||
Let me check what methods `MessageStore` already has vs what's only available as backward-compat wrappers:
|
|
||||||
|
|
||||||
Looking at the code:
|
|
||||||
- `insertMessageEdit` — EXISTS in MessageStore class (line 54)
|
|
||||||
- `upsertMessageForCapture` — EXISTS in MessageStore class
|
|
||||||
- `updateMessageAsEdited` — EXISTS in MessageStore class
|
|
||||||
- `updateMessageAsDeleted` — EXISTS in MessageStore class
|
|
||||||
- `getMessagesByChannel` — EXISTS in MessageStore class
|
|
||||||
- `updateMessageAIAnalysis` — EXISTS in MessageStore class
|
|
||||||
- `updateMessagesAIAnalysisBulk` — EXISTS in MessageStore class
|
|
||||||
- `getPendingAIAnalysisMessages` — EXISTS in MessageStore class
|
|
||||||
- `getMessageById` — EXISTS in MessageStore class
|
|
||||||
- `listMessages` — EXISTS in MessageStore class (delegates to MessagesPagination)
|
|
||||||
- `listReviewMessages` — EXISTS in MessageStore class (delegates to MessagesPagination)
|
|
||||||
- `getConversationContextBefore` — EXISTS in MessageStore class
|
|
||||||
- `getPendingMessagesByConversation` — EXISTS in MessageStore class
|
|
||||||
- `getPendingConversationKeys` — EXISTS in MessageStore class
|
|
||||||
- `getConversationKeysWithIncompleteAnalysis` — EXISTS in MessageStore class
|
|
||||||
- `getIncompleteMessagesByConversation` — EXISTS in MessageStore class
|
|
||||||
|
|
||||||
So every function wrapper has a corresponding method on `MessageStore` class. The change is straightforward.
|
|
||||||
|
|
||||||
Now, after creating the singleton `messageStore`, all importers just do `messageStore.updateMessagesAIAnalysisBulk(...)` instead of calling the bare function.
|
|
||||||
|
|
||||||
But there's one complication: `MessagesDb.updateMessagesAIAnalysisBulk` is actually calling `this.analysis.updateMessagesAIAnalysisBulk()`. Does the `MessageStore` class have its own direct `updateMessagesAIAnalysisBulk`? Let me check the class definition...
|
|
||||||
|
|
||||||
Actually, I already saw from the grep output that `MessageStore` class has `updateMessagesAIAnalysisBulk` — the wrapper says `getInstance().updateMessagesAIAnalysisBulk(updates)`, and the class has that method.
|
|
||||||
|
|
||||||
OK so the mapping is 1:1 between wrapper functions and MessageStore class methods. This is safe.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Update `batchScheduler.ts`**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Before:
|
|
||||||
import { getPendingMessagesByConversation } from "../message-capture/messageStore.js";
|
|
||||||
// After:
|
|
||||||
import { messageStore } from "../message-capture/messageStore.js";
|
|
||||||
```
|
|
||||||
And call: `messageStore.getPendingMessagesByConversation(...)`
|
|
||||||
|
|
||||||
- [ ] **Step 6: Update `individualFallbackProcessor.ts`**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Before:
|
|
||||||
import { updateMessagesAIAnalysisBulk } from "../message-capture/messageStore.js";
|
|
||||||
// After:
|
|
||||||
import { messageStore } from "../message-capture/messageStore.js";
|
|
||||||
```
|
|
||||||
And call: `messageStore.updateMessagesAIAnalysisBulk(...)`
|
|
||||||
|
|
||||||
- [ ] **Step 7: Update `moderationBuilders.ts`**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Before:
|
|
||||||
import { getMessageById } from "../message-capture/messageStore.js";
|
|
||||||
// After:
|
|
||||||
import { messageStore } from "../message-capture/messageStore.js";
|
|
||||||
```
|
|
||||||
And call: `messageStore.getMessageById(...)`
|
|
||||||
|
|
||||||
- [ ] **Step 8: Update `aiAnalysisWorker.ts`**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Before:
|
|
||||||
import { getConversationContextBefore, updateMessagesAIAnalysisBulk } from "../message-capture/messageStore.js";
|
|
||||||
// After:
|
|
||||||
import { messageStore } from "../message-capture/messageStore.js";
|
|
||||||
```
|
|
||||||
And update all call sites.
|
|
||||||
|
|
||||||
- [ ] **Step 9: Update `aiAnalyzer.ts`**
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Before:
|
|
||||||
import {
|
|
||||||
getConversationKeysWithIncompleteAnalysis,
|
|
||||||
getIncompleteMessagesByConversation,
|
|
||||||
getMessageById,
|
|
||||||
getPendingConversationKeys,
|
|
||||||
updateMessageAIAnalysis,
|
|
||||||
} from "../message-capture/messageStore.js";
|
|
||||||
// After:
|
|
||||||
import { messageStore } from "../message-capture/messageStore.js";
|
|
||||||
```
|
|
||||||
And update all call sites.
|
|
||||||
|
|
||||||
- [ ] **Step 10: Update `message-capture/index.ts`**
|
|
||||||
|
|
||||||
Remove individual function re-exports, replace with `messageStore`:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
export { messageStore } from "./messageStore.js";
|
|
||||||
export {
|
|
||||||
getDisplayContent,
|
|
||||||
getMessageLocation,
|
|
||||||
getMessageMetadata,
|
|
||||||
} from "./messageMetadata.js";
|
|
||||||
export type {
|
|
||||||
AIRecommendedAction,
|
|
||||||
AISeverity,
|
|
||||||
AIStatus,
|
|
||||||
AttachmentRecord,
|
|
||||||
MessageRecord,
|
|
||||||
VoiceSegmentRecord,
|
|
||||||
} from "./types.js";
|
|
||||||
export type { TextCaptureTarget } from "./messageCapture.js";
|
|
||||||
export {
|
|
||||||
captureMessage,
|
|
||||||
registerMessageCapture,
|
|
||||||
setEventBroadcaster,
|
|
||||||
} from "./messageCapture.js";
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 11: Run typecheck**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /home/code/GMW
|
|
||||||
pnpm run typecheck
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 12: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add services/discord-gateway/src/modules/message-capture/
|
|
||||||
git add services/discord-gateway/src/modules/ai-moderation/
|
|
||||||
git commit -m "refactor: remove backward-compat function wrappers from messageStore"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 3: Remove Dead Code
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Delete: `services/backend/src/modules/response.ts` — empty deprecated file
|
|
||||||
- Delete: `services/discord-gateway/src/modules/webhook-notifications/webhookNotifier.ts`
|
|
||||||
- Delete: `services/discord-gateway/src/modules/webhook-notifications/index.ts`
|
|
||||||
- Delete: `services/discord-gateway/src/modules/webhook-notifications/` (directory)
|
|
||||||
- Modify: `services/backend/src/ws/server.ts` — remove duplicate `broadcastBinaryToFrontend()` function, keep only `broadcastBinary()`
|
|
||||||
|
|
||||||
**Interfaces:**
|
|
||||||
- None — these are deletions only, no consumer impact
|
|
||||||
|
|
||||||
- [ ] **Step 1: Delete `modules/response.ts`**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
rm /home/code/GMW/services/backend/src/modules/response.ts
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Fix `ws/server.ts`** — remove duplicate `broadcastBinaryToFrontend`
|
|
||||||
|
|
||||||
In `ws/server.ts`, `broadcastBinaryToFrontend` (line 238) and `broadcastBinary` (line 267) do exactly the same thing. Replace the `broadcastBinaryToFrontend(data)` call on line 147 with a call to `broadcastBinary(data)`, then delete the `broadcastBinaryToFrontend` function.
|
|
||||||
|
|
||||||
Edit line 147:
|
|
||||||
```typescript
|
|
||||||
// Before:
|
|
||||||
broadcastBinaryToFrontend(data);
|
|
||||||
// After:
|
|
||||||
broadcastBinary(data);
|
|
||||||
```
|
|
||||||
|
|
||||||
Remove the `broadcastBinaryToFrontend` function (lines 238-248):
|
|
||||||
```typescript
|
|
||||||
// Remove this entire function:
|
|
||||||
function broadcastBinaryToFrontend(data: Buffer) {
|
|
||||||
for (const client of frontendClients) {
|
|
||||||
if (client.readyState === WebSocket.OPEN) {
|
|
||||||
try {
|
|
||||||
client.send(data);
|
|
||||||
} catch (err) {
|
|
||||||
logger.error({ err }, "Failed to send binary to frontend client");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 3: Check if anything imports `webhook-notifications`**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
grep -rn "webhook-notifications\|webhookNotifier\|triggerWebhook" /home/code/GMW/services/ --include='*.ts' | grep -v "node_modules" | grep -v "services/discord-gateway/src/modules/webhook-notifications/"
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected: empty (confirmed earlier)
|
|
||||||
|
|
||||||
- [ ] **Step 4: Delete webhook-notifications module**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
rm -rf /home/code/GMW/services/discord-gateway/src/modules/webhook-notifications/
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 5: Run typecheck to verify no broken imports**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /home/code/GMW
|
|
||||||
pnpm run typecheck
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 6: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add services/backend/src/modules/response.ts services/backend/src/ws/server.ts
|
|
||||||
git add services/discord-gateway/src/modules/webhook-notifications/
|
|
||||||
git commit -m "chore: remove dead code (response.ts, broadcastBinaryToFrontend, webhook-notifications)"
|
|
||||||
```
|
|
||||||
@@ -1,579 +0,0 @@
|
|||||||
# Services Refactoring Implementation Plan
|
|
||||||
|
|
||||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
||||||
|
|
||||||
**Goal:** Refactor backend (4.2k lines) and discord-gateway (17.9k lines) for consistency, reduced file sizes, deduplication, and pattern uniformity across 11 phases.
|
|
||||||
|
|
||||||
**Architecture:** Phase1-3 target backend unchanged; Phase4-8 split large gateway files; Phase9 deduplicates shared database init; Phase10-11 are minor consolidation. Each phase is independently testable by verifying the service still compiles and runs.
|
|
||||||
|
|
||||||
**Tech Stack:** TypeScript (ESM), Express 5, ws, Discord.js selfbot, Drizzle ORM, Redis (ioredis), pino logger, Biome (formatter)
|
|
||||||
|
|
||||||
## Global Constraints
|
|
||||||
|
|
||||||
- All files use ESM (`.js` extensions in imports)
|
|
||||||
- Biome formatter handles formatting — run `pnpm run format` after each phase
|
|
||||||
- TypeScript strict mode — run `pnpm run typecheck` after each phase (for node services)
|
|
||||||
- Logging uses `createChildLogger(context)` from `@bete/shared/logger`
|
|
||||||
- Import via barrel files where available
|
|
||||||
- No logic changes — pure refactoring
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 1: Fix messages.controller.ts pattern (Phase 1)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `services/backend/src/modules/messages/messages.controller.ts`
|
|
||||||
|
|
||||||
**Interfaces:**
|
|
||||||
- Consumes: `asyncHandler` from `../../shared/middlewares/index.js`
|
|
||||||
- Produces: Same exported handler functions, but using decorator pattern
|
|
||||||
|
|
||||||
- [ ] **Step 1: Read current messages.controller.ts**
|
|
||||||
|
|
||||||
The file currently uses the convoluted pattern:
|
|
||||||
```ts
|
|
||||||
export function handleListMessages(req, res, next) {
|
|
||||||
return asyncHandler(async (req, res) => {
|
|
||||||
// ...
|
|
||||||
})(req, res, next);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Rewrite all handlers to decorator pattern**
|
|
||||||
|
|
||||||
Replace every handler to use the clean decorator pattern:
|
|
||||||
|
|
||||||
```ts
|
|
||||||
import { createChildLogger } from "@bete/shared/logger";
|
|
||||||
import type { Request, Response } from "express";
|
|
||||||
import { asyncHandler } from "../../shared/middlewares/index.js";
|
|
||||||
import { messageQuerySchema } from "./messages.schema.js";
|
|
||||||
import { messagesService } from "./messages.service.js";
|
|
||||||
|
|
||||||
const logger = createChildLogger("messages.controller");
|
|
||||||
|
|
||||||
export const handleListMessages = asyncHandler(async (req: Request, res: Response) => {
|
|
||||||
const query = messageQuerySchema.parse(req.query);
|
|
||||||
logger.debug({ query }, "Handling list messages request");
|
|
||||||
const result = await messagesService.listMessages(query);
|
|
||||||
res.json(result);
|
|
||||||
});
|
|
||||||
|
|
||||||
export const handleGetMessagesByChannel = asyncHandler(async (req: Request, res: Response) => {
|
|
||||||
const channelId = String(req.params.channelId ?? "");
|
|
||||||
if (!channelId) {
|
|
||||||
res.status(400).json({ error: "MISSING_CHANNEL_ID" });
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const query = messageQuerySchema.parse(req.query);
|
|
||||||
logger.debug({ channelId, query }, "Handling get messages by channel");
|
|
||||||
const result = await messagesService.getMessagesByChannel(channelId, query);
|
|
||||||
res.json(result);
|
|
||||||
});
|
|
||||||
|
|
||||||
export const handleGetMessageById = asyncHandler(async (req: Request, res: Response) => {
|
|
||||||
const id = String(req.params.id ?? "");
|
|
||||||
if (!id) {
|
|
||||||
res.status(400).json({ error: "MISSING_ID" });
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
logger.debug({ id }, "Handling get message by ID");
|
|
||||||
const result = await messagesService.getMessageById(id);
|
|
||||||
res.json(result);
|
|
||||||
});
|
|
||||||
|
|
||||||
export const handleGetImageMessages = asyncHandler(async (req: Request, res: Response) => {
|
|
||||||
const guildId = String(req.query.guildId ?? "");
|
|
||||||
if (!guildId) {
|
|
||||||
res.status(400).json({ error: "MISSING_GUILD_ID" });
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const limit = Number(req.query.limit) || 50;
|
|
||||||
logger.debug({ guildId, limit }, "Handling get image messages");
|
|
||||||
const result = await messagesService.getImageMessages(guildId, limit);
|
|
||||||
res.json(result);
|
|
||||||
});
|
|
||||||
|
|
||||||
export const handleGetAttachmentsByChannel = asyncHandler(async (req: Request, res: Response) => {
|
|
||||||
const channelId = String(req.params.channelId ?? "");
|
|
||||||
if (!channelId) {
|
|
||||||
res.status(400).json({ error: "MISSING_CHANNEL_ID" });
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const query = messageQuerySchema.parse(req.query);
|
|
||||||
logger.debug({ channelId, query }, "Handling get attachments by channel");
|
|
||||||
const result = await messagesService.getAttachmentsByChannel(channelId, query);
|
|
||||||
res.json(result);
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
NOTE: The old pattern used `requireParam` from middlewares to validate params. The new pattern uses simple string checks with early returns. This is equivalent since `requireParam` threw `ValidationError` which the errorHandler middleware catches — but for these handlers the decorator pattern can't throw synchronously in the handler wrapper; the `asyncHandler` catches async rejects. Early return with explicit error response is cleaner.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Verify the module still compiles**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
|
||||||
Expected: No TypeScript errors
|
|
||||||
|
|
||||||
- [ ] **Step 4: Run biome format**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run format`
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add services/backend/src/modules/messages/messages.controller.ts
|
|
||||||
git commit -m "refactor(backend): fix messages.controller.ts to use decorator pattern
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 2: Clean up response.ts usage (Phase 2)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `services/backend/src/modules/health/health.controller.ts` (remove `success()` usage, use plain `res.json()`)
|
|
||||||
- Modify: `services/backend/src/modules/response.ts` (deprecate/remove)
|
|
||||||
|
|
||||||
**Interfaces:**
|
|
||||||
- Consumes: all response-producing route files
|
|
||||||
- Produces: consistent plain `res.json()` pattern everywhere
|
|
||||||
|
|
||||||
- [ ] **Step 1: Check all places that import from response.ts**
|
|
||||||
|
|
||||||
Run: `grep -r 'from.*response\.js' services/backend/src/`
|
|
||||||
|
|
||||||
- [ ] **Step 2: Remove `success()` usage from health.controller.ts**
|
|
||||||
|
|
||||||
Replace:
|
|
||||||
```ts
|
|
||||||
import { success } from "../response.js";
|
|
||||||
// ...
|
|
||||||
res.status(status).json(success(result));
|
|
||||||
```
|
|
||||||
With:
|
|
||||||
```ts
|
|
||||||
res.status(status).json({ success: true, data: result });
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 3: Run biome format + typecheck**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run format && pnpm run typecheck`
|
|
||||||
|
|
||||||
- [ ] **Step 4: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add services/backend/src/modules/health/health.controller.ts services/backend/src/modules/response.ts
|
|
||||||
git commit -m "refactor(backend): remove response.ts helpers, inline health response
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 3: Add ws/ barrel (Phase 3)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `services/backend/src/ws/index.ts`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Create barrel file**
|
|
||||||
|
|
||||||
```ts
|
|
||||||
export { setBroadcastFunctions, clearBroadcastFunctions, broadcastEvent, broadcastBinary } from "./broadcast.js";
|
|
||||||
export { startRedisBridge, stopRedisBridge } from "./redis-bridge.js";
|
|
||||||
export { createWebSocketServer, closeWebSocketServer } from "./server.js";
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Run typecheck**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
|
||||||
|
|
||||||
- [ ] **Step 3: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add services/backend/src/ws/index.ts
|
|
||||||
git commit -m "refactor(backend): add ws barrel index
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 4: Split moderationPrompt.ts (Phase 4)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `services/discord-gateway/src/modules/ai-moderation/prompts/text-analysis.ts`
|
|
||||||
- Create: `services/discord-gateway/src/modules/ai-moderation/prompts/media-analysis.ts`
|
|
||||||
- Create: `services/discord-gateway/src/modules/ai-moderation/prompts/stickers.ts`
|
|
||||||
- Create: `services/discord-gateway/src/modules/ai-moderation/prompts/emojis.ts`
|
|
||||||
- Create: `services/discord-gateway/src/modules/ai-moderation/prompts/system.ts`
|
|
||||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/moderationPrompt.ts` (become barrel re-export)
|
|
||||||
|
|
||||||
- [ ] **Step 1: Read the full moderationPrompt.ts**
|
|
||||||
|
|
||||||
Read the file to identify all exports and their dependencies.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Create `prompts/system.ts` — system prompt builder + shared helpers**
|
|
||||||
|
|
||||||
Move: `buildSystemPrompt` function, `sanitizeAiContent`, `escapeXml`, `buildCustomEmojiVisionPrompt`, any shared helper functions.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Create `prompts/text-analysis.ts` — text moderation prompts**
|
|
||||||
|
|
||||||
Move: All text-specific prompt strings and builders.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Create `prompts/media-analysis.ts` — image/video prompts**
|
|
||||||
|
|
||||||
Move: `buildGeneralImageVisionPrompt` and related media prompt builders.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Create `prompts/stickers.ts` — sticker prompts**
|
|
||||||
|
|
||||||
Move: `buildStickerVisionPrompt`, `buildStickerTextOnlyWarning`.
|
|
||||||
|
|
||||||
- [ ] **Step 6: Create `prompts/emojis.ts` — emoji prompts**
|
|
||||||
|
|
||||||
Move: `buildCustomEmojiVisionPrompt` if it exists separately.
|
|
||||||
|
|
||||||
- [ ] **Step 7: Replace moderationPrompt.ts with barrel re-exports**
|
|
||||||
|
|
||||||
```ts
|
|
||||||
export { buildSystemPrompt, sanitizeAiContent } from "./prompts/system.js";
|
|
||||||
export { buildGeneralImageVisionPrompt } from "./prompts/media-analysis.js";
|
|
||||||
export { buildStickerVisionPrompt, buildStickerTextOnlyWarning } from "./prompts/stickers.js";
|
|
||||||
export { buildCustomEmojiVisionPrompt } from "./prompts/emojis.js";
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 8: Run typecheck**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
|
||||||
Expected: No errors. Existing importers continue to work via the barrel.
|
|
||||||
|
|
||||||
- [ ] **Step 9: Run biome format**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run format`
|
|
||||||
|
|
||||||
- [ ] **Step 10: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add services/discord-gateway/src/modules/ai-moderation/prompts/ services/discord-gateway/src/modules/ai-moderation/moderationPrompt.ts
|
|
||||||
git commit -m "refactor(gateway): split moderationPrompt.ts into domain-specific prompt files
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 5: Split moderationOrchestrator.ts (Phase 5)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `services/discord-gateway/src/modules/ai-moderation/textBatchProcessor.ts`
|
|
||||||
- Create: `services/discord-gateway/src/modules/ai-moderation/mediaBatchProcessor.ts`
|
|
||||||
- Create: `services/discord-gateway/src/modules/ai-moderation/simpleFallback.ts`
|
|
||||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/moderationOrchestrator.ts` (extract & re-export)
|
|
||||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/index.ts` (update exports if needed)
|
|
||||||
|
|
||||||
- [ ] **Step 1: Read full moderationOrchestrator.ts**
|
|
||||||
|
|
||||||
Map all exports and dependencies.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Extract `runTextOnlyBatch` into `textBatchProcessor.ts`**
|
|
||||||
|
|
||||||
Move the function and its helper `buildCorrectedFewShotExamples`. Export it.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Extract `runMediaBatch` into `mediaBatchProcessor.ts`**
|
|
||||||
|
|
||||||
Move the function and all its dependencies. Export it.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Extract `runSimpleTextFallback` into `simpleFallback.ts`**
|
|
||||||
|
|
||||||
Move the function. Export it.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Update moderationOrchestrator.ts**
|
|
||||||
|
|
||||||
Replace extracted functions with imports:
|
|
||||||
```ts
|
|
||||||
export { runTextOnlyBatch } from "./textBatchProcessor.js";
|
|
||||||
export { runMediaBatch } from "./mediaBatchProcessor.js";
|
|
||||||
export { runSimpleTextFallback } from "./simpleFallback.js";
|
|
||||||
```
|
|
||||||
Keep the `runModerationAnalysis` entry point function which orchestrates text + media + caching.
|
|
||||||
|
|
||||||
- [ ] **Step 6: Run typecheck**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
|
||||||
|
|
||||||
- [ ] **Step 7: Run biome format**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run format`
|
|
||||||
|
|
||||||
- [ ] **Step 8: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add services/discord-gateway/src/modules/ai-moderation/textBatchProcessor.ts services/discord-gateway/src/modules/ai-moderation/mediaBatchProcessor.ts services/discord-gateway/src/modules/ai-moderation/simpleFallback.ts services/discord-gateway/src/modules/ai-moderation/moderationOrchestrator.ts
|
|
||||||
git commit -m "refactor(gateway): split moderationOrchestrator into dedicated processors
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 6: Split mediaAnalysisClient.ts (Phase 6)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `services/discord-gateway/src/modules/ai-moderation/mediaCache.ts`
|
|
||||||
- Create: `services/discord-gateway/src/modules/ai-moderation/mediaDownloader.ts`
|
|
||||||
- Create: `services/discord-gateway/src/modules/ai-moderation/visionAnalyzer.ts`
|
|
||||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/mediaAnalysisClient.ts` (become barrel)
|
|
||||||
|
|
||||||
- [ ] **Step 1: Read full mediaAnalysisClient.ts**
|
|
||||||
|
|
||||||
Map all exports and dependencies across the 826 lines.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Extract all cache logic into `mediaCache.ts`**
|
|
||||||
|
|
||||||
Move: LRU cache, phash dedup, `getCachedMediaAnalysis`, `setCachedMediaAnalysis`, `computeImagePhash`, `deleteCachedMediaAnalysis`, `acquireMediaAnalysisLock`.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Extract all download logic into `mediaDownloader.ts`**
|
|
||||||
|
|
||||||
Move: Image download, video download, ffmpeg frame extraction, temporary file handling.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Extract vision LLM logic into `visionAnalyzer.ts`**
|
|
||||||
|
|
||||||
Move: Vision LLM calls, message preparation for vision, `prepareMediaMessage`.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Update mediaAnalysisClient.ts to re-export**
|
|
||||||
|
|
||||||
```ts
|
|
||||||
export { getCachedMediaAnalysis, setCachedMediaAnalysis, computeImagePhash } from "./mediaCache.js";
|
|
||||||
export { downloadAndExtractFrame } from "./mediaDownloader.js";
|
|
||||||
export { prepareMediaMessage, hasMediaContent } from "./visionAnalyzer.js";
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 6: Run typecheck**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
|
||||||
|
|
||||||
- [ ] **Step 7: Run biome format**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run format`
|
|
||||||
|
|
||||||
- [ ] **Step 8: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add services/discord-gateway/src/modules/ai-moderation/mediaCache.ts services/discord-gateway/src/modules/ai-moderation/mediaDownloader.ts services/discord-gateway/src/modules/ai-moderation/visionAnalyzer.ts services/discord-gateway/src/modules/ai-moderation/mediaAnalysisClient.ts
|
|
||||||
git commit -m "refactor(gateway): split mediaAnalysisClient into cache, downloader, and vision analyzer
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 7: Extract retention cleanup from bootstrap.ts (Phase 7)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `services/discord-gateway/src/app/retention.ts`
|
|
||||||
- Modify: `services/discord-gateway/src/app/bootstrap.ts`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Create `app/retention.ts`**
|
|
||||||
|
|
||||||
Move `deleteExpiredRecords` and `startRetentionCleanup` from `bootstrap.ts`:
|
|
||||||
```ts
|
|
||||||
import { createChildLogger } from "@bete/shared/logger";
|
|
||||||
import { lt, inArray } from "drizzle-orm";
|
|
||||||
import type { NodePgDatabase } from "drizzle-orm/node-postgres";
|
|
||||||
import { config } from "../shared/config/config.js";
|
|
||||||
import { getDatabase } from "../shared/database/drizzle.js";
|
|
||||||
import * as schema from "../shared/database/schema.js";
|
|
||||||
import { messagesTable, attachmentsTable, voiceRecordingsTable } from "../shared/database/schema.js";
|
|
||||||
|
|
||||||
const log = createChildLogger("retention");
|
|
||||||
|
|
||||||
// ... move deleteExpiredRecords here ...
|
|
||||||
|
|
||||||
// ... move startRetentionCleanup here ...
|
|
||||||
|
|
||||||
export { startRetentionCleanup };
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Remove inline retention code from bootstrap.ts**
|
|
||||||
|
|
||||||
- Remove the `deleteExpiredRecords` function
|
|
||||||
- Remove the `startRetentionCleanup` function
|
|
||||||
- Add: `import { startRetentionCleanup } from "./retention.js";`
|
|
||||||
- Replace the call: call `startRetentionCleanup()` directly
|
|
||||||
|
|
||||||
- [ ] **Step 3: Run typecheck**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
|
||||||
|
|
||||||
- [ ] **Step 4: Run biome format**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run format`
|
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add services/discord-gateway/src/app/retention.ts services/discord-gateway/src/app/bootstrap.ts
|
|
||||||
git commit -m "refactor(gateway): extract retention cleanup from bootstrap into dedicated module
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 8: Consolidate EventBroadcaster (Phase 8)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `services/discord-gateway/src/modules/event-broadcaster/eventBroadcaster.ts`
|
|
||||||
- Modify: `services/discord-gateway/src/modules/event-broadcaster/index.ts`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Read current eventBroadcaster.ts**
|
|
||||||
|
|
||||||
Identify `RedisEventPublisher` and `EventBroadcaster` classes.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Merge RedisEventPublisher into EventBroadcaster**
|
|
||||||
|
|
||||||
Inline `RedisEventPublisher` as a private detail inside `EventBroadcaster`. Keep the public API unchanged.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Update index.ts if needed**
|
|
||||||
|
|
||||||
Ensure the barrel still exports `EventBroadcaster`.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Run typecheck**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
|
||||||
|
|
||||||
- [ ] **Step 5: Run biome format**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run format`
|
|
||||||
|
|
||||||
- [ ] **Step 6: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add services/discord-gateway/src/modules/event-broadcaster/eventBroadcaster.ts services/discord-gateway/src/modules/event-broadcaster/index.ts
|
|
||||||
git commit -m "refactor(gateway): merge RedisEventPublisher into EventBroadcaster
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 9: Cross-cutting database initialization dedup (Phase 9)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `packages/shared/src/database/schema.ts` — add database lifecycle helpers
|
|
||||||
- Modify: `services/backend/src/shared/database/index.ts` — use shared helpers
|
|
||||||
- Modify: `services/discord-gateway/src/shared/database/drizzle.ts` — use shared helpers
|
|
||||||
|
|
||||||
- [ ] **Step 1: Check current shared database setup**
|
|
||||||
|
|
||||||
Read `packages/shared/` structure to see if there's already a database module.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Add pool creation helper in @bete/shared**
|
|
||||||
|
|
||||||
In `packages/shared/src/database/schema.ts` or create `packages/shared/src/database/pool.ts`:
|
|
||||||
|
|
||||||
```ts
|
|
||||||
import { Pool } from "pg";
|
|
||||||
|
|
||||||
export function createPostgresPool(url: string, opts?: { min?: number; max?: number }): Pool {
|
|
||||||
return new Pool({
|
|
||||||
connectionString: url,
|
|
||||||
min: opts?.min ?? 2,
|
|
||||||
max: opts?.max ?? 10,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface PoolConfig {
|
|
||||||
host?: string;
|
|
||||||
port?: number;
|
|
||||||
user?: string;
|
|
||||||
password?: string;
|
|
||||||
database?: string;
|
|
||||||
url?: string;
|
|
||||||
min?: number;
|
|
||||||
max?: number;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function createPoolFromConfig(cfg: PoolConfig): Pool {
|
|
||||||
if (cfg.url) return createPostgresPool(cfg.url, { min: cfg.min, max: cfg.max });
|
|
||||||
return new Pool({
|
|
||||||
host: cfg.host,
|
|
||||||
port: cfg.port,
|
|
||||||
user: cfg.user,
|
|
||||||
password: cfg.password,
|
|
||||||
database: cfg.database,
|
|
||||||
min: cfg.min ?? 2,
|
|
||||||
max: cfg.max ?? 10,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Export from `packages/shared/src/database/schema.ts` or create a barrel.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Update backend's shared/database/index.ts**
|
|
||||||
|
|
||||||
Replace inline Pool creation with `createPoolFromConfig` from `@bete/shared`.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Update gateway's shared/database/drizzle.ts**
|
|
||||||
|
|
||||||
Replace inline Pool creation with `createPoolFromConfig` from `@bete/shared`.
|
|
||||||
|
|
||||||
- [ ] **Step 5: Run typecheck across all services**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
|
||||||
|
|
||||||
- [ ] **Step 6: Run biome format**
|
|
||||||
|
|
||||||
Run: `cd /home/code/GMW && pnpm run format`
|
|
||||||
|
|
||||||
- [ ] **Step 7: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add packages/shared/src/database/ services/backend/src/shared/database/index.ts services/discord-gateway/src/shared/database/drizzle.ts
|
|
||||||
git commit -m "refactor: extract shared database pool creation into @bete/shared
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 10: Audit moderationState vs conversationState overlap (Phase 10)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Read: `services/discord-gateway/src/modules/ai-moderation/moderationState.ts`
|
|
||||||
- Read: `services/discord-gateway/src/modules/ai-moderation/conversationState.ts`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Read both files and identify overlap**
|
|
||||||
|
|
||||||
Look for duplicated state management (maps, sets, timers).
|
|
||||||
|
|
||||||
- [ ] **Step 2: If overlap found, merge into one file**
|
|
||||||
|
|
||||||
Otherwise, just add comments documenting the boundary.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Commit**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add services/discord-gateway/src/modules/ai-moderation/
|
|
||||||
git commit -m "refactor(gateway): consolidate conversattion/moderation state management
|
|
||||||
|
|
||||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 11: Redis connection audit (Phase 11)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Read: all Redis connection sites in gateway
|
|
||||||
|
|
||||||
- [ ] **Step 1: Identify all Redis connections**
|
|
||||||
|
|
||||||
Search for `new Redis(` patterns in gateway.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Verify each has a valid reason for a separate connection**
|
|
||||||
|
|
||||||
Document with comments if needed.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Commit (if any changes made)**
|
|
||||||
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,455 +0,0 @@
|
|||||||
# CI/CD Overhaul: Gitea CI + Container Registry Design
|
|
||||||
|
|
||||||
**Status:** Draft
|
|
||||||
**Last updated:** 2026-07-27
|
|
||||||
|
|
||||||
## 1. Problem Statement
|
|
||||||
|
|
||||||
The current CI/CD pipeline has multiple issues:
|
|
||||||
|
|
||||||
1. **Split across 3 CI systems**: GitHub Actions (build + deploy), GitLab CI (build only, no deploy), and `deploy.sh` (hot-deploy bind-mounts)
|
|
||||||
2. **Registry mismatch**: GitHub Actions pushes to `ghcr.io` but `docker-compose.yml` references `registry.gitlab.com` — the deploy route is unclear
|
|
||||||
3. **Hot-deploy complexity**: `deploy.sh` builds locally, tars dist files, SSH pipes, and binds into containers at runtime. Fragile and not reproducible
|
|
||||||
4. **No frontend in Docker**: Frontend is never built into an image — only hot-deployed via bind-mounts
|
|
||||||
5. **Stale Dockerfile**: `Dockerfile.proxy` builds a Rust WASM frontend that no longer exists
|
|
||||||
6. **Dockerfile.frontend is missing**: Frontend image doesn't exist at all
|
|
||||||
7. **Shared package fragility**: The previous refactor added `@bete/shared/database/init` export, but Docker images built from `master` don't have it — containers crash
|
|
||||||
|
|
||||||
## 2. Goal
|
|
||||||
|
|
||||||
Single CI/CD pipeline that:
|
|
||||||
|
|
||||||
- Builds Docker images for all 3 services (backend, discord-gateway, proxy-serving-frontend)
|
|
||||||
- Pushes them to Gitea's built-in Container Registry
|
|
||||||
- On the VPS, only pulls images and restarts containers — no more hot-deploy bind-mounts
|
|
||||||
- All 3 services built in one pipeline, deployed together atomically
|
|
||||||
|
|
||||||
## 3. Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
Developer pushes to main
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌────────────────────────────┐
|
|
||||||
│ Gitea Runner (server X) │
|
|
||||||
│ │
|
|
||||||
│ Job 1: build-and-push │
|
|
||||||
│ ├── bete-backend:latest │──────────▶ Gitea Container Registry
|
|
||||||
│ ├── bete-discord-gateway │──────────▶ git.imrnes.team/MythEclipse/GMW/
|
|
||||||
│ │ :latest │ bete-backend:{sha,latest}
|
|
||||||
│ └── bete-proxy:latest │──────────▶ bete-discord-gateway:{sha,latest}
|
|
||||||
│ │──────────▶ bete-proxy:{sha,latest}
|
|
||||||
│ Job 2: deploy (SSH) │
|
|
||||||
│ └─── SSH ke VPS ──────────┤
|
|
||||||
└────────────────────────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌────────────────────────────┐
|
|
||||||
│ VPS Production │
|
|
||||||
│ /opt/imphenbot/infra/ │
|
|
||||||
│ docker/ │
|
|
||||||
│ │
|
|
||||||
│ docker compose pull │
|
|
||||||
│ docker compose up -d │
|
|
||||||
│ docker image prune -f │
|
|
||||||
│ │
|
|
||||||
│ 3 containers: │
|
|
||||||
│ ┌────────┐ ┌──────────┐ │
|
|
||||||
│ │ proxy │ │ backend │ │
|
|
||||||
│ │ :80 │ │ :3000 │ │
|
|
||||||
│ └───┬────┘ └──────────┘ │
|
|
||||||
│ │ ┌─────────────┐ │
|
|
||||||
│ └────┤discord- │ │
|
|
||||||
│ │gateway │ │
|
|
||||||
│ └─────────────┘ │
|
|
||||||
└────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3.1 Service Images
|
|
||||||
|
|
||||||
| Image | From | Runs |
|
|
||||||
|-------|------|------|
|
|
||||||
| `bete-backend` | `Dockerfile.backend` | Express HTTP/WS on port 3000 |
|
|
||||||
| `bete-discord-gateway` | `Dockerfile.discord-gateway` | Discord client, internal only |
|
|
||||||
| `bete-proxy` | `Dockerfile.proxy` (rewritten) | Nginx serving frontend + proxying `/api` and `/ws` to backend |
|
|
||||||
|
|
||||||
### 3.2 Registry
|
|
||||||
|
|
||||||
Gitea provides a built-in container registry per repository at:
|
|
||||||
```
|
|
||||||
git.imrnes.team/MythEclipse/GMW/<image-name>:<tag>
|
|
||||||
```
|
|
||||||
|
|
||||||
Images are tagged with both `latest` and the commit SHA for traceability.
|
|
||||||
|
|
||||||
## 4. Files to Create / Modify
|
|
||||||
|
|
||||||
### 4.1 Create: `.gitea/workflows/deploy.yml`
|
|
||||||
|
|
||||||
One workflow, two jobs:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
name: Build & Deploy
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [main]
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
build-and-push:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
strategy:
|
|
||||||
matrix:
|
|
||||||
service: [backend, discord-gateway, proxy]
|
|
||||||
max-parallel: 2
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
with:
|
|
||||||
submodules: recursive
|
|
||||||
- name: Set up Docker Buildx
|
|
||||||
uses: docker/setup-buildx-action@v3
|
|
||||||
- name: Login to Gitea Registry
|
|
||||||
uses: docker/login-action@v3
|
|
||||||
with:
|
|
||||||
registry: ${{ vars.GITEA_REGISTRY }}
|
|
||||||
username: ${{ github.actor }}
|
|
||||||
password: ${{ secrets.GITEA_REGISTRY_TOKEN }}
|
|
||||||
- name: Build & Push
|
|
||||||
uses: docker/build-push-action@v6
|
|
||||||
with:
|
|
||||||
context: .
|
|
||||||
file: infra/docker/Dockerfile.${{ matrix.service }}
|
|
||||||
push: true
|
|
||||||
tags: |
|
|
||||||
${{ vars.GITEA_REGISTRY }}/${{ github.repository }}/bete-${{ matrix.service }}:${{ github.sha }}
|
|
||||||
${{ vars.GITEA_REGISTRY }}/${{ github.repository }}/bete-${{ matrix.service }}:latest
|
|
||||||
cache-from: type=gha
|
|
||||||
cache-to: type=gha,mode=max
|
|
||||||
|
|
||||||
deploy:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
needs: build-and-push
|
|
||||||
if: github.ref == 'refs/heads/main'
|
|
||||||
steps:
|
|
||||||
- name: SSH & Deploy
|
|
||||||
uses: appleboy/ssh-action@v1.2.5
|
|
||||||
with:
|
|
||||||
host: ${{ secrets.VPS_HOST }}
|
|
||||||
username: ${{ secrets.VPS_USER }}
|
|
||||||
key: ${{ secrets.VPS_SSH_KEY }}
|
|
||||||
script: |
|
|
||||||
cd /opt/imphenbot/infra/docker
|
|
||||||
echo "${{ secrets.ENV_FILE }}" > .env
|
|
||||||
docker compose pull
|
|
||||||
docker compose up -d --remove-orphans
|
|
||||||
docker image prune -f
|
|
||||||
```
|
|
||||||
|
|
||||||
Note: Gitea CI uses GitHub Actions-compatible syntax (Act Runner). The above uses the standard `actions/*` actions and `docker/*` actions that work with both GitHub and Gitea. If Gitea's runner doesn't fully support `docker/build-push-action`, fallback to inline `docker build` and `docker push` commands.
|
|
||||||
|
|
||||||
Sensitive variables: `GITEA_REGISTRY_TOKEN`, `VPS_HOST`, `VPS_USER`, `VPS_SSH_KEY`, `ENV_FILE` set in Gitea repo Settings → Actions → Secrets. Non-sensitive: `GITEA_REGISTRY` as a Variable.
|
|
||||||
|
|
||||||
### 4.2 Rewrite: `Dockerfile.proxy`
|
|
||||||
|
|
||||||
Current proxy Dockerfile builds a Rust WASM frontend (stale — no longer exists in codebase). Replace with multi-stage build:
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
# Stage 1: Build frontend (Next.js 16 static export)
|
|
||||||
FROM node:22-slim AS frontend-builder
|
|
||||||
|
|
||||||
WORKDIR /app
|
|
||||||
|
|
||||||
# Install pnpm
|
|
||||||
RUN corepack enable
|
|
||||||
|
|
||||||
# Copy dependency manifests
|
|
||||||
COPY pnpm-lock.yaml pnpm-workspace.yaml package.json ./
|
|
||||||
COPY packages/shared/package.json ./packages/shared/package.json
|
|
||||||
COPY services/frontend/package.json ./services/frontend/package.json
|
|
||||||
|
|
||||||
# Install dependencies
|
|
||||||
RUN pnpm install --frozen-lockfile --filter './services/frontend' --filter '@bete/shared'
|
|
||||||
|
|
||||||
# Copy source code
|
|
||||||
COPY packages/shared/ ./packages/shared/
|
|
||||||
COPY services/frontend/ ./services/frontend/
|
|
||||||
|
|
||||||
# Build Next.js static export
|
|
||||||
RUN pnpm --filter frontend run build
|
|
||||||
# Result in services/frontend/out/
|
|
||||||
|
|
||||||
# Stage 2: Nginx
|
|
||||||
FROM nginx:alpine
|
|
||||||
|
|
||||||
# Nginx config
|
|
||||||
COPY infra/docker/nginx/nginx.conf /etc/nginx/conf.d/default.conf
|
|
||||||
|
|
||||||
# Static frontend files
|
|
||||||
COPY --from=frontend-builder /app/services/frontend/out/ /usr/share/nginx/html/
|
|
||||||
|
|
||||||
EXPOSE 80
|
|
||||||
|
|
||||||
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
|
||||||
CMD wget -qO- http://localhost:80/ || exit 1
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4.3 Modify: `Dockerfile.backend`
|
|
||||||
|
|
||||||
Add `VITE_BE_API_URL` and `VITE_BE_WS_URL` build args (already listed in GitHub Actions but not in Dockerfile):
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
# Add to existing Dockerfile.backend — after FROM, before WORKDIR
|
|
||||||
ARG VITE_BE_API_URL
|
|
||||||
ARG VITE_BE_WS_URL
|
|
||||||
ENV VITE_BE_API_URL=${VITE_BE_API_URL}
|
|
||||||
ENV VITE_BE_WS_URL=${VITE_BE_WS_URL}
|
|
||||||
```
|
|
||||||
|
|
||||||
These build args are now consumed at build time for future-proofing even though they were previously only needed for frontend builds (which now lives in the proxy Dockerfile).
|
|
||||||
|
|
||||||
### 4.4 Modify: `Dockerfile.discord-gateway`
|
|
||||||
|
|
||||||
No structural changes needed — verify Drizzle migrations path:
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
# COPY drizzle, line in existing Dockerfile.discord-gateway:
|
|
||||||
COPY services/discord-gateway/drizzle/ ./services/discord-gateway/drizzle/
|
|
||||||
# This should work as-is since workspace is copied at /app
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4.5 Rewrite: `deploy.sh`
|
|
||||||
|
|
||||||
From hot-deploy tar-pipe SSH to lightweight SSH exec:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
#!/bin/bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
||||||
INFRA_DIR="$SCRIPT_DIR/infra/docker"
|
|
||||||
|
|
||||||
: "${VPS_HOST:?required}"
|
|
||||||
: "${VPS_USER:?required}"
|
|
||||||
: "${VPS_SSH_KEY:?required}"
|
|
||||||
|
|
||||||
echo "=== Deploy to $VPS_HOST ==="
|
|
||||||
|
|
||||||
# Copy .env if it exists locally
|
|
||||||
if [ -f "$INFRA_DIR/.env" ]; then
|
|
||||||
scp -i "$VPS_SSH_KEY" "$INFRA_DIR/.env" "$VPS_USER@$VPS_HOST:/opt/imphenbot/infra/docker/.env"
|
|
||||||
fi
|
|
||||||
|
|
||||||
ssh -i "$VPS_SSH_KEY" "$VPS_USER@$VPS_HOST" << 'REMOTESCRIPT'
|
|
||||||
set -e
|
|
||||||
cd /opt/imphenbot/infra/docker
|
|
||||||
echo "=== Pulling images ==="
|
|
||||||
docker compose pull
|
|
||||||
echo "=== Restarting containers ==="
|
|
||||||
docker compose up -d --remove-orphans
|
|
||||||
echo "=== Cleaning up ==="
|
|
||||||
docker image prune -f
|
|
||||||
echo "=== Verify ==="
|
|
||||||
docker ps --filter "name=imphenbot" --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
|
|
||||||
REMOTESCRIPT
|
|
||||||
|
|
||||||
echo "=== Deploy complete ==="
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4.6 Rewrite: `infra/docker/docker-compose.yml`
|
|
||||||
|
|
||||||
Replace all GitLab registry image references with Gitea registry. Remove bind-mounts. Add recordings named volume.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
version: "3.8"
|
|
||||||
|
|
||||||
services:
|
|
||||||
proxy:
|
|
||||||
image: ${GITEA_REGISTRY}/${GITEA_REPO}/bete-proxy:${IMAGE_TAG:-latest}
|
|
||||||
container_name: imphenbot-proxy
|
|
||||||
restart: unless-stopped
|
|
||||||
ports:
|
|
||||||
- "127.0.0.1:8080:80"
|
|
||||||
networks:
|
|
||||||
- app-shared-net
|
|
||||||
healthcheck:
|
|
||||||
test: wget -qO- http://localhost:80/ || exit 1
|
|
||||||
interval: 30s
|
|
||||||
timeout: 3s
|
|
||||||
start_period: 10s
|
|
||||||
retries: 3
|
|
||||||
deploy:
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
memory: 64M
|
|
||||||
labels:
|
|
||||||
traefik.enable: "true"
|
|
||||||
traefik.http.routers.imphenbot.rule: "Host(`imphnen.asepharyana.my.id`)"
|
|
||||||
traefik.http.routers.imphenbot.entrypoints: websecure
|
|
||||||
traefik.http.routers.imphenbot.tls: "true"
|
|
||||||
traefik.http.services.imphenbot.loadbalancer.server.port: "80"
|
|
||||||
|
|
||||||
backend:
|
|
||||||
image: ${GITEA_REGISTRY}/${GITEA_REPO}/bete-backend:${IMAGE_TAG:-latest}
|
|
||||||
container_name: imphenbot-backend
|
|
||||||
restart: unless-stopped
|
|
||||||
env_file:
|
|
||||||
- .env
|
|
||||||
environment:
|
|
||||||
NODE_ENV: production
|
|
||||||
WEBSERVER_PORT: 3000
|
|
||||||
networks:
|
|
||||||
- app-shared-net
|
|
||||||
healthcheck:
|
|
||||||
test: wget -qO- http://localhost:3000/api/health || exit 1
|
|
||||||
interval: 30s
|
|
||||||
timeout: 5s
|
|
||||||
start_period: 20s
|
|
||||||
retries: 3
|
|
||||||
deploy:
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
memory: 256M
|
|
||||||
depends_on:
|
|
||||||
- proxy
|
|
||||||
|
|
||||||
discord-gateway:
|
|
||||||
image: ${GITEA_REGISTRY}/${GITEA_REPO}/bete-discord-gateway:${IMAGE_TAG:-latest}
|
|
||||||
container_name: imphenbot-discord-gateway
|
|
||||||
restart: unless-stopped
|
|
||||||
env_file:
|
|
||||||
- .env
|
|
||||||
environment:
|
|
||||||
NODE_ENV: production
|
|
||||||
volumes:
|
|
||||||
- recordings:/app/recordings
|
|
||||||
networks:
|
|
||||||
- app-shared-net
|
|
||||||
healthcheck:
|
|
||||||
test: sh -c "kill -0 1"
|
|
||||||
interval: 30s
|
|
||||||
timeout: 5s
|
|
||||||
start_period: 20s
|
|
||||||
retries: 3
|
|
||||||
deploy:
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
memory: 512M
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
recordings:
|
|
||||||
|
|
||||||
networks:
|
|
||||||
app-shared-net:
|
|
||||||
external: true
|
|
||||||
```
|
|
||||||
|
|
||||||
Key changes:
|
|
||||||
- Image refs: `registry.gitlab.com/mytheclipse-group/gmw/...` → `${GITEA_REGISTRY}/${GITEA_REPO}/...`
|
|
||||||
- **All bind-mounts removed** (`./backend-dist`, `./gateway-dist`, `./frontend-dist`, `./shared-dist`)
|
|
||||||
- `recordings` → named volume (persists across container restarts/recreates)
|
|
||||||
- `proxy` binds to `127.0.0.1:8080` instead of host port 80 (Traefik handles external routing)
|
|
||||||
- Added `depends_on: proxy` to backend for startup ordering
|
|
||||||
|
|
||||||
### 4.7 Remove: GitHub Actions & GitLab CI files
|
|
||||||
|
|
||||||
After Gitea CI is verified working:
|
|
||||||
- Delete `.github/workflows/deploy-docker.yml` (or rename to `.github/workflows/deploy-docker.yml.disabled`)
|
|
||||||
- Delete `.gitlab-ci.yml` (or rename to `.gitlab-ci.yml.disabled`)
|
|
||||||
|
|
||||||
### 4.8 Ensure: `.gitea/workflows/` directory
|
|
||||||
|
|
||||||
The directory must exist in git. Some setups ignore `.gitea/` — verify `.gitignore` does not exclude it.
|
|
||||||
|
|
||||||
## 5. Gitea Registry Integration
|
|
||||||
|
|
||||||
### 5.1 Enable Container Registry in Gitea
|
|
||||||
|
|
||||||
In Gitea Admin Settings:
|
|
||||||
- Go to Settings → Repository → Enable "Container Registry"
|
|
||||||
- Default registry URL format: `gitea.<domain>/<owner>/<repo>`
|
|
||||||
|
|
||||||
### 5.2 Registry Token
|
|
||||||
|
|
||||||
Create a Gitea access token with `read` and `write` access to packages:
|
|
||||||
- Settings → Applications → Generate Token → `registry-token` → scope: `write:packages`
|
|
||||||
|
|
||||||
### 5.3 CI Variables
|
|
||||||
|
|
||||||
Set these in Gitea repo → Settings → Actions → Secrets:
|
|
||||||
|
|
||||||
| Name | Example Value | Notes |
|
|
||||||
|------|---------------|-------|
|
|
||||||
| `GITEA_REGISTRY_TOKEN` | `gitea_token_abc123` | Docker login password |
|
|
||||||
| `VPS_HOST` | `123.123.123.123` | VPS IP/hostname |
|
|
||||||
| `VPS_USER` | `root` | SSH user |
|
|
||||||
| `VPS_SSH_KEY` | `-----BEGIN OPENSSH PRIVATE KEY-----...` | Private key |
|
|
||||||
| `ENV_FILE` | full .env content | Written to VPS before compose |
|
|
||||||
|
|
||||||
As Variables (not secrets, visible but non-sensitive):
|
|
||||||
|
|
||||||
| Name | Example Value | Notes |
|
|
||||||
|------|---------------|-------|
|
|
||||||
| `GITEA_REGISTRY` | `git.imrnes.team` | Registry hostname — no protocol prefix |
|
|
||||||
|
|
||||||
### 5.4 VPS Setup (one-time)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Docker login to Gitea registry
|
|
||||||
docker login git.imrnes.team
|
|
||||||
# Use Gitea username + access token (with write:packages scope)
|
|
||||||
|
|
||||||
# 2. Create recordings named volume
|
|
||||||
docker volume create imphenbot_recordings
|
|
||||||
|
|
||||||
# 3. Remove old bind-mount directories (after verifying old containers stopped)
|
|
||||||
rm -rf /opt/imphenbot/infra/docker/backend-dist
|
|
||||||
rm -rf /opt/imphenbot/infra/docker/gateway-dist
|
|
||||||
rm -rf /opt/imphenbot/infra/docker/shared-dist
|
|
||||||
rm -rf /opt/imphenbot/infra/docker/frontend-dist
|
|
||||||
|
|
||||||
# 4. Ensure compose file is updated (via git pull)
|
|
||||||
cd /opt/imphenbot && git pull origin main
|
|
||||||
```
|
|
||||||
|
|
||||||
## 6. Migration Plan
|
|
||||||
|
|
||||||
### Phase 1: Prepare (this session)
|
|
||||||
|
|
||||||
1. Write `.gitea/workflows/deploy.yml`
|
|
||||||
2. Rewrite `Dockerfile.proxy` for Next.js
|
|
||||||
3. Modify `infra/docker/docker-compose.yml` for Gitea registry + named volumes
|
|
||||||
4. Rewrite `deploy.sh` to SSH-only
|
|
||||||
5. Mark old CI files as disabled (rename, not delete yet)
|
|
||||||
6. Add VITE_BE_API_URL/VITE_BE_WS_URL build args to backend Dockerfile
|
|
||||||
|
|
||||||
### Phase 2: VPS Preparation (one-time SSH)
|
|
||||||
|
|
||||||
7. User runs `docker login` to Gitea registry on VPS
|
|
||||||
8. User sets CI secrets in Gitea UI
|
|
||||||
9. User creates `imphenbot_recordings` named volume
|
|
||||||
|
|
||||||
### Phase 3: Deploy
|
|
||||||
|
|
||||||
10. Commit and push to `main`
|
|
||||||
11. Gitea CI triggers — builds 3 images, pushes to registry
|
|
||||||
12. Deploy job SSHes into VPS, pulls images, restarts containers
|
|
||||||
13. Verify with `docker ps` and health checks
|
|
||||||
|
|
||||||
### Phase 4: Cleanup
|
|
||||||
|
|
||||||
14. After all services running stably for 1-2 pushes: delete old CI files
|
|
||||||
15. Remove old Dockerfiles if no longer referenced
|
|
||||||
|
|
||||||
## 7. Rollback Plan
|
|
||||||
|
|
||||||
If something goes wrong:
|
|
||||||
|
|
||||||
1. **Quick rollback**: `docker compose up -d` with previous `IMAGE_TAG` (pin to last working SHA)
|
|
||||||
2. **Full rollback**: Revert git changes, push to `main` — Gitea CI will rebuild with old config
|
|
||||||
3. **Emergency**: SSH to VPS, use `docker compose` commands to restart specific containers
|
|
||||||
|
|
||||||
## 8. Future Considerations
|
|
||||||
|
|
||||||
- **Auto-deploy on tag**: Optionally trigger CI only on version tags (`v*`) instead of every `main` push
|
|
||||||
- **Health check notifications**: Add webhook notification on deploy failure
|
|
||||||
- **Multi-architecture builds**: Add `--platform linux/amd64,linux/arm64` for future ARM VPS migration
|
|
||||||
- **Secrets management**: Consider HashiCorp Vault or Gitea's built-in encrypted secrets for larger teams
|
|
||||||
@@ -1,154 +0,0 @@
|
|||||||
# Frontend Refactor: Cleanup, API Alignment & Rebrand
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
Refactor the frontend (`services/frontend/`) to be cleaner, more maintainable, properly aligned with backend API, and rebranded from "bete/GMW" to "Discord Automod" and from "chatbot" to "chatbot".
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
### A. Code Quality & Structure
|
|
||||||
1. **Extract inline page components** into dedicated files under `components/<feature>/`
|
|
||||||
2. **Remove dead code** (`live-stats.tsx`, `useSearch`, `Item`, etc.)
|
|
||||||
3. **Remove duplicate code** (merge `extractImage`/`extractFirstImage`, consolidate `WsHook` type, consolidate `isActive` functions)
|
|
||||||
4. **Fix Tailwind v4 dynamic class** (`grid-cols-${columns}`) in `LoadingSkeleton`
|
|
||||||
5. **Fix navigation icon** (Settings should use `Settings`, not `BarChart3`)
|
|
||||||
|
|
||||||
### B. API Layer Separation
|
|
||||||
- Split `voiceApi` into `voiceApi` + `mediaApi`
|
|
||||||
- Keep `chatbot.ts` as is (frontend already uses "chatbot" naming)
|
|
||||||
|
|
||||||
### C. Data Fetching Consistency
|
|
||||||
- `GuildSelector` → use `useGuilds` + `useConfig` React Query hooks
|
|
||||||
- `useVoiceChannels` → convert from manual `useState` to `useQuery`
|
|
||||||
- Chatbot → convert to `useQuery` + `useMutation` (user approved this)
|
|
||||||
|
|
||||||
### D. Rebrand
|
|
||||||
- **bete/GMW → Discord Automod**: page title, sidebar, settings, comments
|
|
||||||
- **chatbot → chatbot**: the frontend already uses "chatbot" naming for the component and API module; backend paths (`/api/chatbot/chat`) stay unchanged on frontend since they reference the actual backend path
|
|
||||||
|
|
||||||
### E. Dead Code Removal
|
|
||||||
- Remove `components/landing/` (including `live-stats.tsx`)
|
|
||||||
- Remove `components/ui/item.tsx` (unused)
|
|
||||||
- Remove `useSearch` from `use-messages.ts`
|
|
||||||
- Remove unused shadcn/ui components (verified by grep)
|
|
||||||
|
|
||||||
## Target Directory Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
src/
|
|
||||||
app/(dashboard)/
|
|
||||||
messages/page.tsx # slim → imports from components/messages/
|
|
||||||
dashboard/page.tsx # slim
|
|
||||||
voice/page.tsx # slim
|
|
||||||
media/page.tsx # slim
|
|
||||||
recordings/page.tsx # slim
|
|
||||||
analysis/page.tsx # slim
|
|
||||||
settings/page.tsx # slim
|
|
||||||
layout.tsx # unchanged
|
|
||||||
app/layout.tsx # update title
|
|
||||||
app/page.tsx # unchanged (redirect)
|
|
||||||
|
|
||||||
components/
|
|
||||||
messages/
|
|
||||||
message-card.tsx # from inline in messages/page.tsx
|
|
||||||
message-detail-view.tsx # from inline DetailView
|
|
||||||
ai-status-badge.tsx # from inline AiStatusBadge
|
|
||||||
images-grid.tsx # images tab content
|
|
||||||
review-list.tsx # review tab content
|
|
||||||
dashboard/
|
|
||||||
stats-section.tsx
|
|
||||||
users-section.tsx
|
|
||||||
user-detail-section.tsx
|
|
||||||
channels-section.tsx
|
|
||||||
channel-detail-section.tsx
|
|
||||||
voice/
|
|
||||||
voice-connection-card.tsx
|
|
||||||
active-speakers-panel.tsx
|
|
||||||
microphone-card.tsx
|
|
||||||
media/
|
|
||||||
music-player.tsx
|
|
||||||
recordings/
|
|
||||||
recording-list.tsx
|
|
||||||
analysis/
|
|
||||||
search-panel.tsx
|
|
||||||
shared/ # existing
|
|
||||||
layout/ # existing
|
|
||||||
chatbot/ # existing
|
|
||||||
ui/ # shadcn — remove unused
|
|
||||||
|
|
||||||
hooks/
|
|
||||||
use-messages.ts # cleaned, use shared WsHook type
|
|
||||||
use-dashboard.ts
|
|
||||||
use-voice.ts # cleaned
|
|
||||||
use-media.ts # cleaned
|
|
||||||
use-recordings.ts # cleaned
|
|
||||||
use-guilds.ts
|
|
||||||
use-config.ts
|
|
||||||
use-mobile.ts
|
|
||||||
index.ts
|
|
||||||
|
|
||||||
lib/
|
|
||||||
ws-hook.ts # NEW: shared WsHook type
|
|
||||||
api/
|
|
||||||
client.ts
|
|
||||||
messages.ts
|
|
||||||
voice.ts # voice-only
|
|
||||||
media.ts # NEW: extracted from voiceApi
|
|
||||||
dashboard.ts
|
|
||||||
recordings.ts
|
|
||||||
config.ts
|
|
||||||
chatbot.ts
|
|
||||||
ui-state.ts
|
|
||||||
index.ts
|
|
||||||
types/ # no structural changes, verify alignment
|
|
||||||
ws/ # no structural changes
|
|
||||||
format.ts
|
|
||||||
navigation.ts
|
|
||||||
utils.ts
|
|
||||||
```
|
|
||||||
|
|
||||||
## Key Changes Detail
|
|
||||||
|
|
||||||
### 1. Component Extraction
|
|
||||||
Each page file that has inline components (messages=689 lines, dashboard=570 lines) will have those components extracted into dedicated files. The page file becomes a thin composition layer.
|
|
||||||
|
|
||||||
### 2. WsHook Type Consolidation
|
|
||||||
Three files define `type WsHook = { on: <E>(eventType: E, handler: ...) => () => void }`. This moves to `lib/ws-hook.ts` and all three hooks import it.
|
|
||||||
|
|
||||||
### 3. LoadingSkeleton Fix
|
|
||||||
Replace dynamic `grid-cols-${columns}` with explicit Tailwind classes or inline style:
|
|
||||||
```tsx
|
|
||||||
const gridCols = columns === 2 ? "grid-cols-1 md:grid-cols-2" : "grid-cols-1";
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. API Separation
|
|
||||||
```typescript
|
|
||||||
// lib/api/voice.ts — voice + guilds only
|
|
||||||
export const voiceApi = {
|
|
||||||
getGuilds, getTextChannels, getVoiceChannels,
|
|
||||||
getStatus, connect, disconnect, sendCommand,
|
|
||||||
};
|
|
||||||
|
|
||||||
// lib/api/media.ts — media player only (NEW)
|
|
||||||
export const mediaApi = {
|
|
||||||
getStatus, queue, skip, stop, volume,
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5. Data Fetching Consistency
|
|
||||||
`GuildSelector` will use `useGuilds()` and `useConfig()` hooks instead of manual fetch in useEffect.
|
|
||||||
`useVoiceChannels` will use `useQuery` with `enabled: !!guildId`.
|
|
||||||
Chatbot will use `useQuery` for history and `useMutation` for send.
|
|
||||||
|
|
||||||
### 6. Rebrand
|
|
||||||
- `app/layout.tsx`: title → "Discord Automod"
|
|
||||||
- Sidebar brand: keep "DC Automod" (already done)
|
|
||||||
- Settings page: keep "DC Automod" reference
|
|
||||||
- Comments referencing "bete" → update
|
|
||||||
- No changes to package names or external references (backend still "bete" internally)
|
|
||||||
|
|
||||||
### Non-Goals
|
|
||||||
- No changes to backend API paths
|
|
||||||
- No changes to package.json names (pnpm workspace naming)
|
|
||||||
- No changes to Router/App Router structure
|
|
||||||
- No changes to CSS/styling system
|
|
||||||
- No functional changes — visual behavior identical
|
|
||||||
@@ -1,134 +0,0 @@
|
|||||||
# Refactoring Backend & Discord-Gateway Services
|
|
||||||
|
|
||||||
**Date:** 2026-07-27
|
|
||||||
**Status:** Draft
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
Comprehensive refactoring of `services/backend` (4.2k lines) and `services/discord-gateway` (17.9k lines) targeting code consistency, file-size reduction, deduplication, and pattern uniformity.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
### Phase 1 — Backend Controller Consistency
|
|
||||||
|
|
||||||
**Problem:** Two competing controller patterns.
|
|
||||||
|
|
||||||
- `messages.controller.ts`, `chatbot-chat.controller.ts` use convoluted `asyncHandler` inside function body (Gaya A)
|
|
||||||
- `voice.controller.ts`, `health.controller.ts` use clean `asyncHandler` decorator (Gaya B)
|
|
||||||
|
|
||||||
**Fix:** Convert all controllers to **Gaya B** (decorator pattern).
|
|
||||||
|
|
||||||
Before (Gaya A):
|
|
||||||
```ts
|
|
||||||
export function handleListMessages(req, res, next) {
|
|
||||||
return asyncHandler(async (req, res) => {
|
|
||||||
// ...
|
|
||||||
})(req, res, next);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
After (Gaya B):
|
|
||||||
```ts
|
|
||||||
export const handleListMessages = asyncHandler(async (req, res) => {
|
|
||||||
// ...
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
**Files affected:**
|
|
||||||
- `modules/messages/messages.controller.ts`
|
|
||||||
- `modules/chatbot-chat/chatbot-chat.controller.ts`
|
|
||||||
|
|
||||||
### Phase 2 — Backend `response.ts` Cleanup
|
|
||||||
|
|
||||||
**Problem:** `success()`/`error()` helpers exist but are unused (except health controller).
|
|
||||||
|
|
||||||
**Fix:** Apply `success()` consistently to all API responses that are successful data returns. Remove `error()` if unused after audit.
|
|
||||||
|
|
||||||
**Files affected:** All route/service files that `res.json()` data.
|
|
||||||
|
|
||||||
### Phase 3 — Backend `ws/` Barrel
|
|
||||||
|
|
||||||
**Problem:** `ws/broadcast.ts`, `ws/redis-bridge.ts`, `ws/server.ts` — no barrel.
|
|
||||||
|
|
||||||
**Fix:** Add `ws/index.ts` barrel.
|
|
||||||
|
|
||||||
### Phase 4 — Gateway: Split `moderationPrompt.ts` (1015 lines)
|
|
||||||
|
|
||||||
**Problem:** Monolithic prompt file mixing all prompt types.
|
|
||||||
|
|
||||||
**Fix:** Split into:
|
|
||||||
- `prompts/text-analysis.ts` — Text moderation prompts
|
|
||||||
- `prompts/media-analysis.ts` — Image/video analysis prompts
|
|
||||||
- `prompts/stickers.ts` — Sticker analysis prompts
|
|
||||||
- `prompts/emojis.ts` — Custom emoji prompts
|
|
||||||
- `prompts/system.ts` — System prompt builder and shared helpers
|
|
||||||
|
|
||||||
### Phase 5 — Gateway: Split `moderationOrchestrator.ts` (955 lines)
|
|
||||||
|
|
||||||
**Problem:** Entry point that also contains inline text-only batch, media batch, and simple fallback.
|
|
||||||
|
|
||||||
**Fix:** Extract into:
|
|
||||||
- `textBatchProcessor.ts` — All text-only batching logic
|
|
||||||
- `mediaBatchProcessor.ts` — All media batching logic
|
|
||||||
- `simpleFallback.ts` — The `runSimpleTextFallback` function
|
|
||||||
|
|
||||||
### Phase 6 — Gateway: Split `mediaAnalysisClient.ts` (826 lines)
|
|
||||||
|
|
||||||
**Problem:** Cache logic (LRU + phash + DB), download logic (image/video + ffmpeg), and vision LLM in one file.
|
|
||||||
|
|
||||||
**Fix:** Extract into:
|
|
||||||
- `mediaCache.ts` — All caching layers (LRU, phash dedup, DB)
|
|
||||||
- `mediaDownloader.ts` — Image/video download, ffmpeg frame extraction
|
|
||||||
- `visionAnalyzer.ts` — Vision LLM orchestration
|
|
||||||
|
|
||||||
### Phase 7 — Gateway: Consolidate `bootstrap.ts`
|
|
||||||
|
|
||||||
**Problem:** 304-line bootstrap that embeds retention cleanup inline.
|
|
||||||
|
|
||||||
**Fix:** Extract `startRetentionCleanup` into `app/retention.ts`. Leave event registrations in bootstrap as they're inherently app-wide wiring.
|
|
||||||
|
|
||||||
### Phase 8 — Gateway: Simplify EventBroadcaster
|
|
||||||
|
|
||||||
**Problem:** `RedisEventPublisher` wrapping is thin — only adds a `publish` wrapper.
|
|
||||||
|
|
||||||
**Fix:** Merge `RedisEventPublisher` into `EventBroadcaster` as a private inner detail.
|
|
||||||
|
|
||||||
### Phase 9 — Cross-cutting: Database initialization dedup
|
|
||||||
|
|
||||||
**Problem:** Backend (`shared/database/index.ts`) and gateway (`shared/database/drizzle.ts`) have near-identical pool creation and lifecycle code.
|
|
||||||
|
|
||||||
**Fix:** Extract common pool/drizzle lifecycle into `@bete/shared`:
|
|
||||||
```ts
|
|
||||||
// packages/shared/src/database/index.ts
|
|
||||||
export function createDatabasePool(url: string, opts?: PoolOpts): Pool
|
|
||||||
export function createDrizzleClient(pool: Pool): DrizzleClient
|
|
||||||
export function closePool(pool: Pool): Promise<void>
|
|
||||||
```
|
|
||||||
Both services keep their own getDatabase/close wrappers but delegate pool creation to shared.
|
|
||||||
|
|
||||||
### Phase 10 — Gateway: Consolidate `moderationState.ts` / `conversationState.ts`
|
|
||||||
|
|
||||||
**Problem:** Two state files with overlapping concerns.
|
|
||||||
|
|
||||||
**Fix:** Audit both for overlap, merge if significant duplication found.
|
|
||||||
|
|
||||||
### Phase 11 — Gateway: Redis connection usage audit
|
|
||||||
|
|
||||||
**Problem:** Multiple independent Redis connections for EventBroadcaster and CommandHandler.
|
|
||||||
|
|
||||||
**Fix:** Both already need separate connections (Redis pub/sub limits). Document the pattern. No structural change.
|
|
||||||
|
|
||||||
## Files Changed
|
|
||||||
|
|
||||||
| Phase | Files | Type |
|
|
||||||
|-------|-------|------|
|
|
||||||
| 1 | 3 | edit |
|
|
||||||
| 2 | ~15 | edit |
|
|
||||||
| 3 | 1 | create |
|
|
||||||
| 4 | ~6 | split |
|
|
||||||
| 5 | ~4 | split |
|
|
||||||
| 6 | ~4 | split |
|
|
||||||
| 7 | 2 | split |
|
|
||||||
| 8 | 2 | refactor |
|
|
||||||
| 9 | 2 | refactor |
|
|
||||||
| 10 | 1-2 | audit+merge |
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
# Visual Redesign: Discord Automod Dashboard
|
|
||||||
|
|
||||||
## Design Direction
|
|
||||||
|
|
||||||
**Vibe:** "Monitoring hub" — deep, technical, trustworthy. Think security operations center meets modern dev tool.
|
|
||||||
|
|
||||||
## Palette
|
|
||||||
|
|
||||||
**Dark (primary):**
|
|
||||||
| Token | Value | Role |
|
|
||||||
|-------|-------|------|
|
|
||||||
| `--bg` | `oklch(0.09 0.015 245)` | Deeper navy canvas |
|
|
||||||
| `--card` | `oklch(0.13 0.02 245)` | Surface with subtle separation |
|
|
||||||
| `--primary` | `oklch(0.62 0.17 215)` | Teal-cyan accent (shift from sky blue) |
|
|
||||||
| `--accent` | `oklch(0.7 0.18 260)` | Electric blue-purple for secondary highlights |
|
|
||||||
| `--warn` | `oklch(0.7 0.17 75)` | Amber-gold for warnings (distinct from red) |
|
|
||||||
| `--border` | `oklch(1 0 0 / 0.06)` | Softer borders |
|
|
||||||
|
|
||||||
## Typography
|
|
||||||
- Geist Sans (body) + Geist Mono (code/data) — already loaded
|
|
||||||
- H1: `text-lg font-semibold tracking-tight`
|
|
||||||
- Card titles: `text-sm font-semibold tracking-tight`
|
|
||||||
- Labels/captions: `text-xs text-muted-foreground tracking-wide uppercase`
|
|
||||||
|
|
||||||
## Layout Changes
|
|
||||||
|
|
||||||
1. **Background**: Subtle dot-grid pattern (`radial-gradient(circle, oklch(1 0 0 / 0.03) 1px, transparent 1px)`) — monitoring station feel
|
|
||||||
2. **Sidebar**: Slightly wider (w-64), active item gets a glow bar + subtle teal tint background, connection dot with breathing animation
|
|
||||||
3. **Cards**: Hover state adds a thin teal border-top glow, softer shadow
|
|
||||||
4. **Stat cards**: Gradient background per stat type (like live-stats had), with icon in colored bubble
|
|
||||||
5. **Severity indicators**: Colored dot + label instead of just colored border
|
|
||||||
6. **Mobile nav**: Tighter spacing, active indicator as dot above icon
|
|
||||||
7. **Header**: Clean, thin bottom border glow, page title larger
|
|
||||||
|
|
||||||
## Signature Element
|
|
||||||
- **Grid background** + **teal glow** on active/interactive elements
|
|
||||||
- **Gradient accent bar** on sidebar active item (wider, glowing)
|
|
||||||
@@ -1,508 +0,0 @@
|
|||||||
---
|
|
||||||
name: "Discord Automod — Neo Surveillance Redesign"
|
|
||||||
version: "1.0.0"
|
|
||||||
date: "2026-07-28"
|
|
||||||
status: "approved"
|
|
||||||
inspiration:
|
|
||||||
- "Summit Cloud Migration Platform (glassmorphic, dark premium)"
|
|
||||||
- "AeroNet Visualization (data panels, modular layout)"
|
|
||||||
colors:
|
|
||||||
canvas: "oklch(0.07 0.015 250)"
|
|
||||||
surface: "oklch(0.11 0.02 245 / 0.6)"
|
|
||||||
surface-hover: "oklch(0.15 0.02 245 / 0.7)"
|
|
||||||
border: "oklch(1 0 0 / 0.06)"
|
|
||||||
border-glow: "oklch(0.62 0.17 215 / 0.3)"
|
|
||||||
primary: "oklch(0.62 0.17 215)"
|
|
||||||
primary-glow: "oklch(0.62 0.17 215 / 0.4)"
|
|
||||||
accent-purple: "oklch(0.65 0.2 280)"
|
|
||||||
accent-amber: "oklch(0.7 0.17 75)"
|
|
||||||
text-primary: "oklch(0.93 0.01 245)"
|
|
||||||
text-secondary: "oklch(0.55 0.02 245)"
|
|
||||||
text-mono: "oklch(0.62 0.17 215)"
|
|
||||||
glass-bg: "oklch(1 0 0 / 0.04)"
|
|
||||||
glass-border: "oklch(1 0 0 / 0.08)"
|
|
||||||
glass-shadow: "0 8px 32px oklch(0 0 0 / 0.4)"
|
|
||||||
typography:
|
|
||||||
display: "Inter 28-48px weight 600"
|
|
||||||
body: "Inter 14-16px weight 400"
|
|
||||||
mono: "JetBrains Mono 11-13px weight 500-600"
|
|
||||||
data: "JetBrains Mono 24-36px weight 600, teal tint"
|
|
||||||
radius:
|
|
||||||
card: "16px"
|
|
||||||
panel: "12px"
|
|
||||||
control: "8px"
|
|
||||||
pill: "9999px"
|
|
||||||
---
|
|
||||||
|
|
||||||
# Discord Automod — Neo Surveillance Redesign
|
|
||||||
|
|
||||||
Full frontend redesign for Discord Automod, a Discord moderation watcher dashboard. Complete rewrite of layout, design system, navigation, and page architecture.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Design Philosophy
|
|
||||||
|
|
||||||
**"Neo Surveillance"** — a Security Operations Center (SOC) inspired dashboard where monitoring feels immersive and powerful. Full-screen glass panels float over a dark animated canvas. No persistent sidebar clutter. The interface disappears into the background, letting live data and alerts take center stage.
|
|
||||||
|
|
||||||
Key pillars:
|
|
||||||
- **Immersion** — Full-viewport canvas with ambient motion, glass panels float over content
|
|
||||||
- **Awareness** — Live data streams, real-time voice waveforms, animated moderation alerts
|
|
||||||
- **Presence** — Live2D vtuber chatbot character as chatbot interface, reacts to server events
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Layout & Navigation System
|
|
||||||
|
|
||||||
### 2.1 Global Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
┌──────────────────────────────────────────────────────┐
|
|
||||||
│ ● Discord Automod Dashboard Msgs Voice … 🟢 ● │ ← Floating Top Bar (~44px)
|
|
||||||
├──────────────────────────────────────────────────────┤
|
|
||||||
│ [Sub-navigation tabs] ← muncul per-page │
|
|
||||||
│━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━│
|
|
||||||
│ │
|
|
||||||
│ ┌─────────┐ ┌──────────────┐ │
|
|
||||||
│ │ Glass │ │ Content │ │
|
|
||||||
│ │ Panels │ │ Area │ │
|
|
||||||
│ │ │ │ (scroll) │ │
|
|
||||||
│ └─────────┘ └──────────────┘ │
|
|
||||||
│ │
|
|
||||||
├──────────────────────────────────────────────────────┤
|
|
||||||
│ 🎵 [Mini-player] ← bottom-left 🎭 [Chatbot] ← BR │
|
|
||||||
└──────────────────────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2.2 Floating Top Navigation Bar
|
|
||||||
|
|
||||||
- **Style:** Glass (`backdrop-blur-xl`), subtle glow border bottom, `h-11` (44px)
|
|
||||||
- **Left:** App logo "Discord Automod" with teal live dot indicator + current page name
|
|
||||||
- **Center:** Horizontal nav links — Dashboard, Messages, Voice, Recordings, Settings
|
|
||||||
- Icon + label, active state with glow underline (`box-shadow` teal)
|
|
||||||
- Hover: text brighter, no background fill
|
|
||||||
- **Search** has no nav link — triggered globally via Cmd+K or `/` shortcut, opens spotlight
|
|
||||||
- **Right:** Connection status dot (pulse when connected) + theme toggle (sun/moon icon)
|
|
||||||
- **Hover sidebar hotspot:** Left edge 4px trigger → slide-in sidebar with guild selector, bookmarks, recent channels (auto-hide 300ms after mouse leave)
|
|
||||||
|
|
||||||
### 2.3 Sub-navigation
|
|
||||||
|
|
||||||
Each page has its own tab bar below the top nav, also glass-styled:
|
|
||||||
- Dashboard: Stats | Live | Activity
|
|
||||||
- Messages: All | Images | Review
|
|
||||||
- Voice: Connection | Activity
|
|
||||||
- Recordings: Library | Stats
|
|
||||||
- Settings: Connection | Appearance | Config | About
|
|
||||||
|
|
||||||
### 2.4 Hidden Sidebar (Hover-activated)
|
|
||||||
|
|
||||||
- Trigger: 4px hotspot at left screen edge
|
|
||||||
- Slide-in animation (150ms, ease-out-expo)
|
|
||||||
- Contains: guild selector dropdown, bookmarked channels, recent activity shortcuts
|
|
||||||
- Auto-hide on mouse leave with 300ms delay
|
|
||||||
|
|
||||||
### 2.5 Floating Media Player
|
|
||||||
|
|
||||||
- No dedicated Media page — persistent floating mini-player at bottom-left
|
|
||||||
- Visible only when a track is active
|
|
||||||
- Click to expand: full queue management overlay
|
|
||||||
- Controls: play/pause, skip, stop, volume slider, progress bar
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Design Tokens
|
|
||||||
|
|
||||||
### 3.1 Color Palette
|
|
||||||
|
|
||||||
| Token | Value | Usage |
|
|
||||||
|-------|-------|-------|
|
|
||||||
| `--canvas` | `oklch(0.07 0.015 250)` | Deep navy background |
|
|
||||||
| `--surface` | `oklch(0.11 0.02 245 / 0.6)` | Glass card base |
|
|
||||||
| `--surface-hover` | `oklch(0.15 0.02 245 / 0.7)` | Card hover state |
|
|
||||||
| `--border` | `oklch(1 0 0 / 0.06)` | Subtle border |
|
|
||||||
| `--border-glow` | `oklch(0.62 0.17 215 / 0.3)` | Active card border glow |
|
|
||||||
| `--primary` | `oklch(0.62 0.17 215)` | Teal-cyan accent, buttons |
|
|
||||||
| `--primary-glow` | `oklch(0.62 0.17 215 / 0.4)` | Active state glow |
|
|
||||||
| `--accent-purple` | `oklch(0.65 0.2 280)` | Moderation flagged items |
|
|
||||||
| `--accent-amber` | `oklch(0.7 0.17 75)` | Warnings |
|
|
||||||
| `--text-primary` | `oklch(0.93 0.01 245)` | Body text |
|
|
||||||
| `--text-secondary` | `oklch(0.55 0.02 245)` | Secondary labels |
|
|
||||||
| `--text-mono` | `oklch(0.62 0.17 215)` | Data metrics (teal tint) |
|
|
||||||
| `--glass-bg` | `oklch(1 0 0 / 0.04)` | Glass base |
|
|
||||||
| `--glass-border` | `oklch(1 0 0 / 0.08)` | Glass border |
|
|
||||||
| `--glass-shadow` | `0 8px 32px oklch(0 0 0 / 0.4)` | Glass shadow |
|
|
||||||
|
|
||||||
### 3.2 Typography
|
|
||||||
|
|
||||||
| Role | Font | Size / Weight |
|
|
||||||
|------|------|--------------|
|
|
||||||
| Display | Inter | 28-48px, weight 600 |
|
|
||||||
| Body | Inter | 14-16px, weight 400 |
|
|
||||||
| Label / Mono | JetBrains Mono | 11-13px, weight 500-600 |
|
|
||||||
| Data metrics | JetBrains Mono | 24-36px, weight 600, teal tint |
|
|
||||||
|
|
||||||
### 3.3 Radius System
|
|
||||||
|
|
||||||
| Token | Value |
|
|
||||||
|-------|-------|
|
|
||||||
| Card | 16px |
|
|
||||||
| Panel | 12px |
|
|
||||||
| Button / Control | 8px |
|
|
||||||
| Pill | 9999px |
|
|
||||||
|
|
||||||
### 3.4 Motion Tokens
|
|
||||||
|
|
||||||
```css
|
|
||||||
--ease-out-expo: cubic-bezier(0.19, 1, 0.22, 1);
|
|
||||||
--ease-smooth: cubic-bezier(0.4, 0, 0.2, 1);
|
|
||||||
--duration-fast: 150ms;
|
|
||||||
--duration-normal: 250ms;
|
|
||||||
--duration-slow: 400ms;
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Component System
|
|
||||||
|
|
||||||
### 4.1 Glass Card System
|
|
||||||
|
|
||||||
- **Base:** `glass-bg` + `glass-border` + border-radius `16px`
|
|
||||||
- **Elevated:** Deeper shadow + subtle primary glow border
|
|
||||||
- **Interactive:** Hover `scale(1.01)` + border glow intensify
|
|
||||||
- **Danger:** Red-tinted border for critical items
|
|
||||||
- Inner padding: `20px` (card), `16px` (panel), `12px` (dense)
|
|
||||||
|
|
||||||
### 4.2 Button Variants
|
|
||||||
|
|
||||||
| Variant | Style |
|
|
||||||
|---------|-------|
|
|
||||||
| Primary | `bg-primary` + `shadow-[0_0_12px] shadow-primary/40` (glow) |
|
|
||||||
| Secondary | `glass-bg` + `border` |
|
|
||||||
| Ghost | Transparent, hover → subtle glass bg |
|
|
||||||
| Icon | Size 32px, rounded 8px |
|
|
||||||
| Danger | Red-tinted variant for destructive actions |
|
|
||||||
|
|
||||||
### 4.3 Status Indicators
|
|
||||||
|
|
||||||
- **Live dot:** Pulsing ring animation (`pulse-ring` 1.5s)
|
|
||||||
- **AI Badge:** Teal pill with sparkle icon, mono font
|
|
||||||
- **Severity badges:** Clean (green), Warn (amber), Flagged (purple), Critical (red)
|
|
||||||
- **Connection:** Green (connected), Yellow (connecting), Red (disconnected)
|
|
||||||
|
|
||||||
### 4.4 Charts (Recharts)
|
|
||||||
|
|
||||||
Custom theme matching design tokens:
|
|
||||||
- Line/Area: gradient fill (primary → transparent)
|
|
||||||
- Bar: rounded bars, teal-cyan gradient
|
|
||||||
- Heatmap: activity by hour × weekday
|
|
||||||
- Radar: multi-axis for moderation categories
|
|
||||||
|
|
||||||
### 4.5 Live2D Chatbot / Chatbot
|
|
||||||
|
|
||||||
- Replaces the existing `Chatbot` component entirely — chatbot panel is the new chat interface
|
|
||||||
- **Location:** Floating panel, bottom-right corner, draggable
|
|
||||||
- **Default size:** Compact — upper body visible (~200×280px)
|
|
||||||
- **Click character:** Expand with full chat panel
|
|
||||||
- **Dynamic expressions:**
|
|
||||||
- Idle: subtle breathing, blink every 4s
|
|
||||||
- New message: head tilt "listening"
|
|
||||||
- Flagged detected: eyes widen, ! bubble
|
|
||||||
- User click: happy wave
|
|
||||||
- Voice active: ear/head tilt toward audio
|
|
||||||
- Chat reply: mouth sync animation
|
|
||||||
- Disconnect: sad expression
|
|
||||||
- **Technology:** Live2D Cubism SDK (WebGL via pixi.js wrapper), `.model3.json` + `.moc3` format
|
|
||||||
- **Chat panel:** Glass-styled input + message history, context-aware (server context)
|
|
||||||
|
|
||||||
### 4.6 Loading States
|
|
||||||
|
|
||||||
- **Skeleton:** Glass card shape with shimmer gradient (teal → transparent → teal)
|
|
||||||
- **Button loading:** Spinner within button
|
|
||||||
- **Full page:** Glass skeleton grid matching target layout
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. Page Layouts
|
|
||||||
|
|
||||||
### 5.1 Dashboard — "Ops Center"
|
|
||||||
|
|
||||||
Full-viewport command center:
|
|
||||||
- **Stat cards row:** Total Messages, Today, Users, Active 24h, Flagged, Clean — each with micro sparkline chart (Recharts mini area) behind the number
|
|
||||||
- **Live Message Stream:** Auto-scrolling glass panel showing recent messages, fade-in animation, click for detail
|
|
||||||
- **Mod Queue:** Flagged messages with quick action buttons (approve/delete/escalate)
|
|
||||||
- **Message Trend Chart:** 7-day area chart
|
|
||||||
- **Activity Heatmap:** Hour × day-of-week, moderation event density
|
|
||||||
- **Top Channels:** Bar chart with channel names
|
|
||||||
- **Chatbot visible** floating bottom-right
|
|
||||||
|
|
||||||
### 5.2 Messages — Split Pane
|
|
||||||
|
|
||||||
- **Left pane:** Scrollable message list, glass cards with severity badge, channel tag, timestamp
|
|
||||||
- **Right pane:** Detail/preview — full message content, attachments gallery, AI analysis breakdown (severity, flags, confidence, categories)
|
|
||||||
- **Global search bar** in top area: spotlight-style overlay (Cmd+K)
|
|
||||||
- **State in URL params:** `?guild=xxx&channel=yyy&selected=msg123&tab=all`
|
|
||||||
- **Tabs:** All | Images (grid view) | Review (flagged queue)
|
|
||||||
- **Actions:** Reanalyze, Moderate (dropdown: delete/warn/escalate)
|
|
||||||
|
|
||||||
### 5.3 Voice — Connection Center
|
|
||||||
|
|
||||||
- **Connection card:** Guild/channel selectors, status with live dot + duration
|
|
||||||
- **Active Speakers:** Per-user waveform visualization (canvas-based, 100ms update)
|
|
||||||
- **Microphone/Transmit:** Toggle mic, volume slider
|
|
||||||
- **Voice Activity Timeline:** Bar chart showing who spoke and total duration
|
|
||||||
- **Recordings quick link** to Recordings page
|
|
||||||
|
|
||||||
### 5.4 Recordings — Voice Library
|
|
||||||
|
|
||||||
- **Search + filter bar:** By user, channel, date range
|
|
||||||
- **Recording cards:** Glass card, waveform preview (canvas), duration, timestamp
|
|
||||||
- **Inline playback:** Play button, audio player without leaving page
|
|
||||||
- **Actions:** Download, Copy Link
|
|
||||||
|
|
||||||
### 5.5 Settings
|
|
||||||
|
|
||||||
- **Sections:** Connection (WebSocket status, guild info), Appearance (theme toggle), Server Config (read-only), About
|
|
||||||
- All glass cards, mono font for config values
|
|
||||||
- Toggle switches with glass styling
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. Animations & Micro-interactions
|
|
||||||
|
|
||||||
### 6.1 Ambient Background
|
|
||||||
- Gradient mesh with slow-shift (30s cycle)
|
|
||||||
- 2-3 soft color blobs (teal, purple, amber), opacity 0.03-0.06
|
|
||||||
- Grid dot pattern: `radial-gradient(circle, oklch(1 0 0 / 0.025) 1px, transparent 1px)`, 24px spacing
|
|
||||||
|
|
||||||
### 6.2 Page Transitions
|
|
||||||
- Route change: `fade-in-up` 200ms ease-out
|
|
||||||
- Content section: `scale(0.98→1)` + `opacity(0.6→1)`
|
|
||||||
|
|
||||||
### 6.3 Card Interactions
|
|
||||||
- Hover: `scale(1.01)` + border glow intensify + shadow lift
|
|
||||||
- Click: `scale(0.98)` brief (100ms)
|
|
||||||
- Panel enter: `translateY(-4px)` + `opacity` fade-in
|
|
||||||
- Stat counter: count-up animation (JS tween, 400ms)
|
|
||||||
|
|
||||||
### 6.4 Live Data
|
|
||||||
- Message stream: fade-in from top, slide down as new arrive
|
|
||||||
- Voice waveform: real-time canvas draw, 100ms interval
|
|
||||||
- Recording: pulsing dot + ring expansion (1.5s loop)
|
|
||||||
- Connection: slow pulse when connected
|
|
||||||
- Flagged: brief red/purple border flash on new flagged message
|
|
||||||
|
|
||||||
### 6.5 Micro-interactions
|
|
||||||
- Toggle: slide with glow
|
|
||||||
- Scrollbar: custom thin (6px), auto-hide, rounded
|
|
||||||
- Drag handle: subtle dot grip for split pane
|
|
||||||
- Copy: brief "Copied!" toast
|
|
||||||
- Reanalyze: 360° icon rotation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7. Data Flow & State Management
|
|
||||||
|
|
||||||
### 7.1 Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
WS Provider (auto-reconnect, typed events, event buffer)
|
|
||||||
↓
|
|
||||||
TanStack Query (fetches + cache)
|
|
||||||
↓
|
|
||||||
Query invalidation on WS events
|
|
||||||
↓
|
|
||||||
Optimistic cache updates for real-time data
|
|
||||||
```
|
|
||||||
|
|
||||||
### 7.2 WS → Cache Strategy
|
|
||||||
|
|
||||||
| WS Event | Action |
|
|
||||||
|----------|--------|
|
|
||||||
| `message_created` | Optimistic insert to message list + dashboard stats |
|
|
||||||
| `message_analyzed` | Update AI fields in message cache |
|
|
||||||
| `message_deleted` | Remove from cache + update counters |
|
|
||||||
| `voice_recording_started` | Update voice status |
|
|
||||||
| `voice_pcm_data` | Buffer to waveform canvas (bypass React) |
|
|
||||||
| `voice_active_user` | Update speakers cache |
|
|
||||||
| `analysis_queue_status` | Update queue progress |
|
|
||||||
|
|
||||||
### 7.3 Query Config
|
|
||||||
|
|
||||||
- `staleTime: 10_000` (10s)
|
|
||||||
- `gcTime: 5 * 60 * 1000` (5 min)
|
|
||||||
- `refetchOnWindowFocus: false`
|
|
||||||
|
|
||||||
### 7.4 Global State (React Context)
|
|
||||||
|
|
||||||
- `useMediaPlayer()` — current track, queue, play/skip/stop/volume
|
|
||||||
- `useChatbot()` — expression, minimized, chatHistory, setExpression
|
|
||||||
- Externally triggerable: `chatbot.setExpression("surprise")` on flagged message, `("listening")` on voice activity
|
|
||||||
|
|
||||||
### 7.5 URL State
|
|
||||||
|
|
||||||
Persistent page state via search params (not React state):
|
|
||||||
```
|
|
||||||
/messages?guild=xxx&channel=yyy&selected=msg123&tab=all
|
|
||||||
```
|
|
||||||
|
|
||||||
### 7.6 Error Boundaries
|
|
||||||
|
|
||||||
Each page has its own error boundary. One page failure doesn't affect others.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 8. Technology Stack
|
|
||||||
|
|
||||||
- **Framework:** Next.js 16 (App Router, static export)
|
|
||||||
- **Language:** TypeScript strict
|
|
||||||
- **Styling:** Tailwind v4 + CSS custom properties
|
|
||||||
- **UI Base:** shadcn/ui components (adapted for glass theme)
|
|
||||||
- **Icons:** lucide-react
|
|
||||||
- **State/data:** @tanstack/react-query v5
|
|
||||||
- **Charts:** Recharts 3.8 (with custom theme)
|
|
||||||
- **3D/Chatbot:** Live2D Cubism SDK WebGL (pixi.js wrapper)
|
|
||||||
- **Audio:** Web Audio API for waveform visualization
|
|
||||||
- **Animation:** CSS animations + transitions (no GSAP/framer-motion dependency unless specifically needed)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 9. File Structure (New)
|
|
||||||
|
|
||||||
```
|
|
||||||
src/
|
|
||||||
├── app/
|
|
||||||
│ ├── layout.tsx # Root layout (fonts, theme script, Toaster)
|
|
||||||
│ ├── page.tsx # Redirect → /dashboard
|
|
||||||
│ ├── globals.css # Complete redesign CSS (tokens, glass, animations)
|
|
||||||
│ └── (dashboard)/
|
|
||||||
│ ├── layout.tsx # Dashboard layout (top nav, QueryClient, WS, chatbot)
|
|
||||||
│ ├── dashboard/
|
|
||||||
│ │ └── page.tsx # Ops Center
|
|
||||||
│ ├── messages/
|
|
||||||
│ │ └── page.tsx # Split pane messages
|
|
||||||
│ ├── voice/
|
|
||||||
│ │ └── page.tsx # Voice connection center
|
|
||||||
│ ├── recordings/
|
|
||||||
│ │ └── page.tsx # Recording library
|
|
||||||
│ └── settings/
|
|
||||||
│ └── page.tsx # Settings page
|
|
||||||
│
|
|
||||||
├── components/
|
|
||||||
│ ├── layout/
|
|
||||||
│ │ ├── top-nav.tsx # Floating top navigation bar
|
|
||||||
│ │ ├── sub-nav.tsx # Per-page sub-navigation tabs
|
|
||||||
│ │ ├── hidden-sidebar.tsx # Hover-activated guild sidebar
|
|
||||||
│ │ └── mobile-nav.tsx # Mobile bottom nav (updated design)
|
|
||||||
│ │
|
|
||||||
│ ├── glass/
|
|
||||||
│ │ ├── card.tsx # Glass card component (base, elevated, interactive)
|
|
||||||
│ │ ├── panel.tsx # Glass panel wrapper
|
|
||||||
│ │ └── divider.tsx # Glass-styled separator
|
|
||||||
│ │
|
|
||||||
│ ├── dashboard/
|
|
||||||
│ │ ├── stat-card.tsx # Stat card with micro sparkline
|
|
||||||
│ │ ├── live-stream.tsx # Auto-scrolling message stream
|
|
||||||
│ │ ├── mod-queue.tsx # Moderation queue with quick actions
|
|
||||||
│ │ ├── message-trend-chart.tsx # 7-day area chart
|
|
||||||
│ │ ├── activity-heatmap.tsx # Hour × day heatmap
|
|
||||||
│ │ └── top-channels-chart.tsx # Top channels bar chart
|
|
||||||
│ │
|
|
||||||
│ ├── messages/
|
|
||||||
│ │ ├── message-list.tsx # Left pane — scrollable message list
|
|
||||||
│ │ ├── message-card.tsx # Individual message card (redesigned)
|
|
||||||
│ │ ├── message-detail.tsx # Right pane — full detail
|
|
||||||
│ │ ├── attachments-grid.tsx # Attachments gallery
|
|
||||||
│ │ ├── ai-analysis-panel.tsx # AI analysis breakdown
|
|
||||||
│ │ └── search-overlay.tsx # Cmd+K search spotlight
|
|
||||||
│ │
|
|
||||||
│ ├── voice/
|
|
||||||
│ │ ├── connection-card.tsx # Guild/channel selector + status
|
|
||||||
│ │ ├── speaker-waveform.tsx # Canvas waveform per speaker
|
|
||||||
│ │ ├── mic-control.tsx # Mic toggle + volume
|
|
||||||
│ │ └── activity-timeline.tsx # Voice activity bar chart
|
|
||||||
│ │
|
|
||||||
│ ├── recordings/
|
|
||||||
│ │ ├── recording-card.tsx # Glass card with waveform preview
|
|
||||||
│ │ └── recording-player.tsx # Inline audio player
|
|
||||||
│ │
|
|
||||||
│ ├── chatbot/
|
|
||||||
│ │ ├── chatbot-container.tsx # Floating L2D container
|
|
||||||
│ │ ├── chatbot-canvas.tsx # WebGL canvas for L2D rendering
|
|
||||||
│ │ ├── chat-panel.tsx # Chat input + history
|
|
||||||
│ │ └── chatbot-context.tsx # Context provider
|
|
||||||
│ │
|
|
||||||
│ ├── media/
|
|
||||||
│ │ └── mini-player.tsx # Floating mini media player
|
|
||||||
│ │
|
|
||||||
│ ├── shared/
|
|
||||||
│ │ ├── error-state.tsx # Error boundary fallback
|
|
||||||
│ │ ├── loading-skeleton.tsx # Glass shimmer skeleton
|
|
||||||
│ │ └── empty-state.tsx # Empty state illustration
|
|
||||||
│ │
|
|
||||||
│ └── ui/ # shadcn/ui components (adapted to glass)
|
|
||||||
│ ├── button.tsx, badge.tsx, dialog.tsx, ...
|
|
||||||
│
|
|
||||||
├── lib/
|
|
||||||
│ ├── api/ # Existing API client (unchanged)
|
|
||||||
│ ├── ws/
|
|
||||||
│ │ ├── context.tsx # WS provider (unchanged)
|
|
||||||
│ │ └── types.ts # WS event types
|
|
||||||
│ ├── hooks/ # Existing hooks + new ones
|
|
||||||
│ │ ├── use-media-player.ts # Global media state
|
|
||||||
│ │ ├── use-chatbot.ts # Chatbot context hook
|
|
||||||
│ │ └── use-heatmap.ts # Heatmap data hook
|
|
||||||
│ ├── types/ # Existing types (unchanged)
|
|
||||||
│ ├── navigation.ts # Nav items (updated)
|
|
||||||
│ └── format.ts # Format utilities
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 10. Implementation Order
|
|
||||||
|
|
||||||
### Phase 1 — Foundation
|
|
||||||
1. Update `globals.css` with new design tokens (colors, glass, radius, typography, animations)
|
|
||||||
2. Rewrite root `layout.tsx` with theme system
|
|
||||||
3. Build glass component system (`card.tsx`, `panel.tsx`)
|
|
||||||
4. Build `top-nav.tsx`, `sub-nav.tsx`, `hidden-sidebar.tsx`
|
|
||||||
5. Update dashboard layout with new nav
|
|
||||||
|
|
||||||
### Phase 2 — Dashboard Ops Center
|
|
||||||
6. Build `stat-card.tsx` with micro sparkline
|
|
||||||
7. Build `live-stream.tsx`
|
|
||||||
8. Build `mod-queue.tsx`
|
|
||||||
9. Build charts: `message-trend-chart.tsx`, `activity-heatmap.tsx`, `top-channels-chart.tsx`
|
|
||||||
10. Rewrite dashboard page
|
|
||||||
|
|
||||||
### Phase 3 — Messages (Split Pane)
|
|
||||||
11. Build `message-list.tsx`, `message-card.tsx` (redesigned)
|
|
||||||
12. Build `message-detail.tsx`, `ai-analysis-panel.tsx`, `attachments-grid.tsx`
|
|
||||||
13. Build `search-overlay.tsx`
|
|
||||||
14. Rewrite messages page with split-pane layout
|
|
||||||
|
|
||||||
### Phase 4 — Voice, Recordings, Settings
|
|
||||||
15. Build voice components and rewrite voice page
|
|
||||||
16. Build recording components and rewrite recordings page
|
|
||||||
17. Rewrite settings page
|
|
||||||
|
|
||||||
### Phase 5 — Floating Elements
|
|
||||||
18. Build `mini-player.tsx` for media
|
|
||||||
19. Build chatbot components (L2D integration)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 11. Testing
|
|
||||||
|
|
||||||
- Visual regression checks per component
|
|
||||||
- WS integration tests for cache updates
|
|
||||||
- Responsive breakpoint testing (mobile bottom nav)
|
|
||||||
- L2D chatbot load + expression trigger
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 12. Non-Goals (Out of Scope)
|
|
||||||
|
|
||||||
- Authentication — remains public
|
|
||||||
- Backend API changes — only frontend redesign
|
|
||||||
- Database changes — no schema modifications
|
|
||||||
- New backend WebSocket events — reuse existing
|
|
||||||
- L2D model creation — integration only (model file provided separately)
|
|
||||||
Reference in New Issue
Block a user