Commit Graph
22 Commits
Author SHA1 Message Date
asepharyana 00e8d68ce5 feat(gmw): public features #7-14 — scam domains, top channels, hourly heatmap, category drill-down, coverage stats, channel culture glossary, term KB, edit history
ALSO fixes: dashboard.repository still JOINed dropped user_reputations table (listUsers/getUserDetail crash).
2026-08-18 20:45:12 +07:00
asepharyana 9b3134d767 feat(gmw): public features #2-#6 — live moderation feed, toxic topic trends, channel timeline, CSV export, activity heatmap
- Live Moderation Feed: gateway publishes discord:moderation:action (Redis) → backend WS emits moderation_action → public web shows realtime stream.
- Toxic Topic Trends: backend moderation.trends aggregates categories/severity/action_type (read-only) → SVG bar + donut.
- Channel Timeline: messages view gets Feed/Timeline toggle with date-grouped separators.
- CSV Export: client-side downloadCsv for moderation actions (no backend write scope).
- Activity Heatmap: backend messages.activity (per-hour volume by channel) → pure-SVG grid.

User reputation deliberately excluded — no such feature exists in the codebase.
All read-only / public-facing / fully automatic per project rules.
2026-08-18 17:43:02 +07:00
asepharyana 0cb0b82fb1 feat(messages): stream history one-message-per-WS-frame instead of 50-row batch
- backend: add streamMany generator (paginated, yields one record at a time)
  + messagesService.streamMessages + WS 'stream_messages' handler emitting
  'message_snapshot' per message, 'message_snapshot_end' with nextCursor
- frontend: useMessagesStream hook accumulates snapshots into SWR list,
  SSR getMessages seeds first paint, WsHook gains sendText
- add stream-many.test.ts locking the one-at-a-time + cursor contract
2026-08-18 10:21:08 +07:00
asepharyana 5505983dbd fix(goLive): retry VIDEO(op12)/SPEAKING(op5) opcodes until ws OPEN — broken shared-screen video
Root cause: BaseMediaConnection.sendOpcode is a silent no-op when
ws.readyState !== OPEN. In GoLive, playStream() calls setVideoAttributes(true)
+ setSpeaking(true) the instant createStream() resolves (right after
SELECT_PROTOCOL_ACK), but the StreamConnection WebSocket can still be in
CONNECTING for a few ms — so op 12 (VIDEO, activating the video SSRC) was
silently DROPPED every session. Empirically verified: 0 ops 12/5 ever logged
across the entire journal, yet 10k+ video frames were sent and audio played
(audio SSRC is activated via the VoiceConnection handshake, independent of
GoLive op 12). Discord's media server thus received video RTP on video_ssrc
but was never told to forward it → black/broken shared-screen video with
working voice.

sendOpcodeWhenOpen retries up to ~2s for ws OPEN instead of dropping. Also
emits a=fmtp:101 packetization-mode=1;profile-level-id=42e01f in the answer
SDP (H264 FU-A fragments require packetization-mode=1 to reassemble).

Also removes pre-existing noNonNullAssertion lint (biome 2.5.8 now errors)
that was blocking the deploy CI.
2026-08-13 01:44:40 +07:00
asepharyana f999be4fa0 feat(dashboard): add moderation log page + message edit history
- Backend moderation module: GET /api/moderation/stats (per-status + failed rate)
  + GET /api/moderation/actions (filter by status/actionType, cursor paging),
  joins messages for target username + content
- Message GET /api/messages/detail/:id now returns edit_count + edit_history
  (old_content snapshots from message_edits, newest first)
- FE: new /moderation page — summary cards (total/executed/failed/pending +
  failed-rate), status+type filter chips, timeline rows with action icon,
  target user, reason, status badge, timestamps, error text
- FE: message detail shows 'Riwayat edit' panel with previous versions
2026-08-05 11:11:10 +07:00
asepharyana 2f51f94610 fix(moderation): remove manual reanalyze triggers — auto-recovery only
Manual per-message and batch reanalyze buttons/endpoints let anyone
re-queue arbitrary messages for LLM analysis, burning AI credits on
spam. Removed:
- FE: Reanalyze buttons in message list, search panel, and messages page
- FE: useReanalyze/useReanalyzeBatch hooks + messagesApi methods
- BE: POST /api/messages/:id/reanalyze and /reanalyze-batch endpoints
- BE: markForReanalysis/reanalyzeErrorBatch service+repository methods

Recovery of failed messages is fully automatic: the discord-gateway
startPendingAIAnalysisWorker retries 'pending' (batch path) and
'error/analysis_incomplete' (individual path) messages on
AI_ANALYSIS_RECOVERY_INTERVAL_MS.
2026-08-04 15:37:56 +07:00
asepharyana 6293d588bc chore(lint): biome cleanup across services — format, sort imports, drop unused
- discord-gateway: 74 lint errors -> 0 (format, import sorting, unused
  imports/vars, dead breath var)
- backend: format + sort imports (11 warnings left: noExplicitAny)
- frontend: remove unused imports, drop dead breathing var, fix
  useExhaustiveDependencies (scroll keyed on messages), a11y biome-ignore
  for drag surface + stopPropagation container (mouse-only gestures)
- remaining warnings are false positives: index keys on static lists,
  <img> in static export (next/image unsupported), noExplicitAny

tsc --noEmit clean on all 3 services; vitest green (60+36).
2026-08-01 22:09:02 +07:00
asepharyana 8e7ba67248 fix: message detail shows only its own images, not everyone's
Build & Deploy (Nix) / build-and-deploy (discord-gateway) (push) Successful in 2m47s
Build & Deploy (Nix) / build-and-deploy (backend) (push) Successful in 3m12s
Build & Deploy (Nix) / build-and-deploy (proxy) (push) Successful in 3m12s
useMessageDetail fetch attachments per-channel -> detail view nampilin
10 image terbaru di channel itu (semua orang), bukan image pesan yang
diklik. Root cause: getAttachmentsByChannel cuma filter channel_id.

- backend: messageQuerySchema + messageId optional; repository
  tambah eq(message_id) pas messageId ada. Sudah ada index
  idx_attachments_message.
- frontend: getAttachments(channelId, limit, cursor, messageId);
  useMessageDetail pass id pesan.

Verifikasi: backend tsc PASS, frontend tsc PASS + next build PASS.
E2E local (tsx + env prod, port 3901): tanpa messageId -> 3 row
(termasuk image.png punya message lain); dengan messageId -> tepat
2 row milik pesan itu. psql juga konfirmasi channel-only 3 vs
message-filtered 2.
2026-08-01 10:59:54 +07:00
Developer dcd13482c2 refactor: break monorepo into 3 standalone services (gateway, backend, frontend)
Build & Deploy / build-and-push (backend) (push) Failing after 35s
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 25s
Build & Deploy / build-and-push (proxy) (push) Failing after 25s
- Remove pnpm workspace, moon repo, and all monorepo tooling
- Delete packages/shared/, embed shared code directly into each service
- Copy packages/shared/src/* -> services/backend/src/shared/ and services/discord-gateway/src/shared/
- Replace all @bete/shared imports with @/shared/ path alias
- Remove @bete/shared workspace dependency from both services
- Update root package.json scripts from --filter to --prefix
- Rewrite Dockerfiles to build each service standalone
- Clean up biome.json, .gitignore, remove root drizzle.config.ts
2026-07-30 11:50:48 +07:00
DeveloperandClaude Opus 4.8 5802d02e29 refactor: large codebase cleanup - consolidate schemas, migrate to Drizzle ORM, extract frontend components, modernize Docker builds
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 2m22s
Build & Deploy / build-and-push (backend) (push) Failing after 3m22s
Build & Deploy / build-and-push (proxy) (push) Successful in 1m36s
Build & Deploy / deploy (push) Skipped
- Consolidate all DB schema definitions into packages/shared as single source of truth
- Migrate backend from raw SQL to Drizzle ORM across all modules
- Extract frontend inline UI into separate component files
- Refactor discord-gateway circuitBreaker into conversationState + moderationState
- Convert messageStore to Proxy singleton pattern
- Add validateBody/validateQuery middleware + Zod schemas for API endpoints
- Modernize Docker builds with multi-stage + pnpm deploy
- Migrate CI/CD from deployment to image-based pipeline
- Remove 60+ unused/dead files (~15K lines)
- Update color scheme from sky-blue to teal-cyan
- Move DB connection management to @bete/shared/database

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 21:54:31 +07:00
asepharyana 9621317a3c chore: format files for lint
Deploy to VPS / deploy (push) Failing after 8m8s
2026-07-08 01:37:46 +07:00
asepharyana c0edb0fa40 feat: enhance deployment instructions and exclude spam threads from message capture 2026-07-05 06:58:17 +07:00
asepharyana c3b2b3f334 feat: add /api/messages/images endpoint + ImageGrid fetch
Backend:
- New GET /api/messages/images?guildId=&limit= endpoint
- Queries attachments table for image/* MIME type, returns distinct messages
- Repository uses Drizzle subquery + join pattern

Frontend:
- New get_images() API client function
- ImageGrid fetches from /api/messages/images when Images tab selected
- Separate image_messages signal, fetched on tab switch

Deploy: hot-patched JS/WASM files to running containers
2026-07-05 04:48:29 +07:00
MythEclipse f502918b27 feat: multimodal video detection + reply/forward/crosspost + batch optimization
- Video frame extraction via ffmpeg (4 key frames per video → vision LLM)
- Video display in FE MessageCard with HTML5 <video> player
- Reply/forward/crosspost indicator in FE + pipeline in DG/BE
- Fix: missing sanitizeAiContent + escapeXml in media path (prompt injection)
- Optimize: text-only batch results saved to DB immediately, no longer wait for media analysis
- BE mapper/schema/repo: add reference fields (is_reply, is_forward, etc.)
2026-06-22 09:46:06 +07:00
MythEclipseandClaude Opus 4.8 07032ab521 refactor: atomic, DRY, and logging improvements across codebase
- Split llmModerationClient.ts (2170 lines) into 5 focused sub-modules
- Split aiAnalyzer.ts (1282 lines) into 4 modular pipelines
- Split messages.db.ts (826 lines) into 5 domain-specific modules
- Moved shared schema to @bete/shared, eliminated backend duplication
- Added createChildLogger to all voice-recording and AI moderation modules
- Extracted tryCommandThenFallback, normalizeMediaState, DEFAULT_VOICE_STATUS
- Created shared pagination.ts utility, eliminated 5+ cursor-pagination duplications
- Created shared messageMapper.ts for row mapping
- Standardized backend error handling with asyncHandler
- Added frontend createLogger utility and useAsyncAction hook
- Added structured logging to frontend hooks, socket, and API client

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 19:46:08 +07:00
MythEclipseandClaude Opus 4.8 b68789fffc refactor: atomic, DRY, and logging improvements
- Shared Redis channel constants as single source of truth (redis-channels.ts)
- commandHandler.ts split into VoiceHandler, MediaHandler, GuildHandler,
  ModerationHandler with handler-registry.ts dispatch
- messageStore.ts (1322 lines) split into domain-specific DB files:
  messages.db.ts, attachments.db.ts, reviews.db.ts,
  moderation-actions.db.ts, retention.db.ts
- recorder.ts startSpeaking callback extracted into speakingHandler.ts,
  streamSetup.ts, segmentFinalizer.ts
- autoDeleteManager.ts split into autoDeleteEligibility.ts,
  autoDeleteNotify.ts, autoDeleteLogger.ts
- Added createChildLogger() logging across 8 service files
- Backend messages.repository.ts migrated from raw SQL to Drizzle ORM
- Fixed biome.json to exclude packages/**/dist/* from lint
- Fixed config.ts GUILD_ID pre-existing type error

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 17:34:18 +07:00
MythEclipse 7108f6bb47 feat(system): implement graceful shutdown and expand websocket events
Improve system reliability and real-time capabilities by implementing a robust lifecycle management system and adding new broadcast events for attachments and voice recordings.

- Implement asynchronous graceful shutdown in backend to close HTTP, WebSocket, Redis, and database connections.
- Add new WebSocket broadcast events: `attachment_created`, `voice_recording_started`, `voice_recording_stopped`, `voice_recording_uploaded`, and `analysis_queue_status`.
- Refactor media status handling to use boolean `playing` state instead of string-based status.
- Centralize `PageResult` and `VoiceRecording` types to improve consistency between frontend and backend.
- Update frontend API client to include `listRecordings` and handle new WebSocket event types.
- Fix type mismatches in voice command handling and media status reporting.
2026-06-09 16:56:44 +07:00
MythEclipseandClaude Opus 4.8 4becf0d6f1 refactor: comprehensive codebase cleanup and architecture hardening
- Sprint 1 (Quick Wins): Remove dead analytics modules, fix 4 unresolved
  imports, replace 3 console.warn with logger, remove mock-crc import
- Sprint 2 (Architecture): Create MascotChatRepository, AnalysisRepository,
  3 Zod schemas (mascot-chat, analysis, voice), deduplicate error classes,
  move 3 SQL queries from routes to repository
- Sprint 3 (Complexity): Replace 7 any types with proper interfaces,
  extract 6 helpers from prepareMediaMessage (CC 85 -> ~15)
- Sprint 4 (Config): Remove 22 dead env vars from .env, add 30 missing
  vars to .env.example, standardize naming

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 10:16:04 +07:00
MythEclipseandClaude Opus 4.8 6da8a32c9b feat: add "Retry All Errors" batch reanalyze button
Backend:
- POST /api/messages/reanalyze-batch — bulk reset error messages to pending
- messages.repository.reanalyzeErrorBatch() — scoped by guildId/channelId/messageIds
- messages.service.reanalyzeErrorBatch() — validation layer

Frontend:
- reanalyzeErrorBatch() API client function
- useMessages.reanalyzeAllErrors() — optimistic state + batch call
- MessagesPanel: destructive button visible when error count > 0
- Shows confirmation text with queued count

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 00:17:51 +07:00
MythEclipse 2d79d8aefd feat(shared): reorder AppError constructor to (message, code, status), add singleton logger, retry and TTL cache utilities 2026-06-02 21:06:42 +07:00
MythEclipseandClaude Opus 4.8 3b2709455e feat: implement all real backend endpoints, Redis pub/sub bridge, and gateway command handler
Backend (real implementations, no more stubs):
- Redis pub/sub bridge: subscribes to discord-gateway events (message/attachment/voice) and broadcasts to WS clients
- Redis command channel: backend publishes voice/media commands, discord-gateway executes and replies
- Voice service: connectVoice/disconnectVoice/getVoiceStatus via Redis commands with graceful fallback
- Media service: queue/skip/stop/volume via Redis commands, reads status from Redis cache
- Messages repository: ALL 7 methods now use real PostgreSQL queries (findMany, findById, findByChannel, create, update, delete, getAttachmentsByChannel)
- Analytics: period returns {start,end} epoch millis, overview includes hourly/topics/top_users, worst_flags as string[]
- Health check: actually queries SELECT 1 against database
- VoiceStatus type fixed: {connected, activeGuildId, activeChannelId, activeChannelName}
- Guild type: includes icon: string | null
- asyncHandler: accepts Promise<unknown> instead of Promise<void>

Discord Gateway:
- CommandHandler: subscribes to 'backend:command' Redis channel, executes voice/media commands, publishes replies
- Publishes voice:status and media:status to Redis for backend caching
- Shutdown handler updated to close command handler

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 00:32:38 +07:00
MythEclipseandClaude Opus 4.8 c48a0c5e3b refactor: split monolith into 3 microservices (frontend, backend, discord-gateway)
- Extract services into services/{frontend,backend,discord-gateway}
- Create packages/shared/ for shared logger, errors, utils, types
- Setup Modular MVC pattern in backend (controller→service→repository)
- Setup event-driven architecture in discord-gateway with Redis pub/sub
- Move Docker files to infra/docker/ with per-service Dockerfiles
- Update docker-compose.yml to use Traefik-only routing (no port exposes)
- Update GitHub Actions deploy workflow for multi-service matrix build
- Fix all import paths and resolve type errors across all services
- All 3 services pass tsc --noEmit clean

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 21:44:29 +07:00