Author SHA1 Message Date
mytheclipsebotreview f750f39b50 fix(materi): type-cast tags as string[] to satisfy tsc (CI gate)
tsc reported 'Property some does not exist on type {}' on doc.tags
because Drizzle jsonb inference returns a generic object. Cast explicitly.
This unblocks the GMW GitHub Actions deploy (test job).
2026-08-20 19:18:24 +07:00
mytheclipsebotreview f1d90b6097 fix(materi): align tags column type to jsonb (schema ↔ migration mismatch)
The Drizzle schema used pgText('tags').array() which emits a Postgres
text[] column, but the migration defines tags as jsonb. The mismatch caused
INSERT/LIST on materi_documents to throw INTERNAL_SERVER_ERROR (500) because
Drizzle sent a text[] where the column expected jsonb.

- schema.ts: tags → pgJsonb('tags').notNull().default('[]')
- migration: dropped+recreated to match schema (jsonb, ms epoch defaults,
  owner_user_id default 'anonymous')
2026-08-20 19:08:32 +07:00
mytheclipsebotreview 7f4196124d fix(ci): lint and a11y fixes unblocks GHA deploy
- materi/new/page.tsx: add htmlFor+id pairs for all 5 form labels (a11y)
- biome --write --unsafe: fix useTemplate, useLiteralKeys, import sort
  across materi module files (backend + frontend)
- These pre-existing lint errors from 0aa893a blocked the deploy pipeline
2026-08-20 18:45:09 +07:00
mytheclipsebotreview 5658726ea5 fix(frontend): sort messages by created_at to fix WS race condition
Two independent WS handlers (useMessagesWsSync for message_created/
updated/analyzed, and useMessagesStream for message_snapshot) both
prepend live messages to the SWR list without enforcing order.
When frames arrive out-of-order (common with batched WS delivery),
the message feed gets scrambled.

Fix: add sortMessages() helper that sorts newest-first by created_at
(the list's stored order before .reverse() for display) and apply it
in every patchLists/mutate updater: message_created, message_updated,
message_analyzed, message_snapshot, and useLoadMore page appends.

Function declaration is hoisted so useLoadMore (defined above the
helper) can use it.
2026-08-20 18:16:43 +07:00
mytheclipsebotreview f5d5690401 fix: double-.js extension in @/ alias resolution (fix-imports.mjs)
The fix-imports.mjs script blindly appended '.js' to every @/ alias
import, even when the source specifier already carried a .js
extension (e.g. '@/shared/config/index.js'). This produced
'index.js.js' in the emitted dist/, causing ERR_MODULE_NOT_FOUND
at startup.

This was latent: only triggered once digestScheduler.ts (which
uses @/shared/config/index.js with explicit extension) was built.
The user-reputation removal (2a8f6d9) was also blocked by this
bug — stale binary kept crashing with 'user_reputations' query
errors because it was never redeployed.

Fix: only append .js when the @/ specifier has no existing
extension. Applied to both gateway and backend scripts.
2026-08-20 18:08:30 +07:00
asepharyana 0aa893ab7d feat: add Materi section + AI agent RAG to GMW business flow
Backend:
- New materi module: schema (materi_documents table), repository, service
- ragClient: semantic + keyword search over materi docs, plus Discord
  archive via Qdrant, then LLM answer generation (RAG)
- Wire materiRouter into appRouter (list/detail/create/update/delete/chat)

Frontend:
- New types (MateriDocument, CreateMateriInput, RAG chat shapes)
- API client (SSR HTTP RPCLink + browser WS RPCLink)
- Routes: /materi list, /materi/[id] detail, /materi/new form,
  /materi/chat RAG chat UI
- Sidebar nav item 'Materi'

Migration: scripts/add-materi-documents.sql (CREATE TABLE IF NOT EXISTS)
2026-08-20 15:57:04 +07:00
asepharyana 6f20b0f146 docs: clarify termGlossary uses Wikipedia (not SearXNG) for definition lookups
SearXNG was already replaced by Wikipedia REST/Action APIs (wikipediaClient.ts).
Update comments to reflect the current implementation: term glossary now
resolves definitions via Wikipedia → Redis → Postgres cache chain, with no
SearXNG dependency.
2026-08-20 15:32:20 +07:00
asepharyana 80248d4b7a feat: add 'screenshare' to MediaMode union for screen-share audio recording
Prepares the media type system to distinguish screenshare audio SSRCs
from mic voice SSRCs once the hookScreenShareAudio capture logic is
wired in.
2026-08-20 15:29:22 +07:00
asepharyana 20e991062c fix: screen-share audio capture — hook VoiceReceiver.onUdpMessage to discover unregistered SSRCs
Discord GoLive sends screen-share audio on a separate SSRC from the
user's microphone. In @discordjs/voice v0.19, VoiceReceiver.onUdpMessage
silently drops packets for SSRCs not in ssrcMap (which is only populated
from VOICE_STATE_UPDATE/VOICE_SERVER_UPDATE). This caused screen-share
audio to never trigger receiver.speaking and never reach the speakingHandler.

Fix: hookScreenShareAudio() wraps onUdpMessage to:
1. Detect incoming RTP packets with unknown SSRCs (OPRUS payload type 120)
2. Infer the owning userId by proximity to known audioSSRC
3. Clone the user's VoiceUserData into ssrcMap under the new SSRC
4. Let the original handler decrypt and forward to the subscription stream
5. Listen on ssrcMap 'create'/'update' events for video SSRC changes

Also removes the broken initial approach (polling ssrcMap which never
contains screen-share SSRCs).
2026-08-20 15:28:55 +07:00
asepharyana b784d6d796 feat(gateway): weekly moderation digest via WEBHOOK_URLS (#15)
Automated public weekly summary: top categories/domains/channels + coverage rate, posted to configured webhook. Uses getDatabase() direct query (no oRPC HTTP dependency), guards one-fire-per-week on restart.
2026-08-18 20:51:12 +07:00
asepharyana 00e8d68ce5 feat(gmw): public features #7-14 — scam domains, top channels, hourly heatmap, category drill-down, coverage stats, channel culture glossary, term KB, edit history
ALSO fixes: dashboard.repository still JOINed dropped user_reputations table (listUsers/getUserDetail crash).
2026-08-18 20:45:12 +07:00
asepharyana 2a8f6d9062 refactor(gateway): remove user reputation feature entirely
Drop trust-score/infraction system: delete userReputationStore, remove call sites in fallback/batch processors, drop formatReputationAttrs, drop user_reputations table (migration 0016), delete trust-model test, update docs.
2026-08-18 18:27:15 +07:00
asepharyana 9b3134d767 feat(gmw): public features #2-#6 — live moderation feed, toxic topic trends, channel timeline, CSV export, activity heatmap
- Live Moderation Feed: gateway publishes discord:moderation:action (Redis) → backend WS emits moderation_action → public web shows realtime stream.
- Toxic Topic Trends: backend moderation.trends aggregates categories/severity/action_type (read-only) → SVG bar + donut.
- Channel Timeline: messages view gets Feed/Timeline toggle with date-grouped separators.
- CSV Export: client-side downloadCsv for moderation actions (no backend write scope).
- Activity Heatmap: backend messages.activity (per-hour volume by channel) → pure-SVG grid.

User reputation deliberately excluded — no such feature exists in the codebase.
All read-only / public-facing / fully automatic per project rules.
2026-08-18 17:43:02 +07:00
asepharyana 36363fa3db fix(gateway): skip bot-only channel 1318544753821880362 from capture
Add to BOT_EXCLUDED_CHANNEL_IDS default alongside 1206269771340058694
so bot messages in that channel are no longer captured/analyzed/embedded.
2026-08-18 16:13:25 +07:00
asepharyana d133cc3271 style(gateway): sort imports in archiveEmbedder (biome) 2026-08-18 15:53:18 +07:00
asepharyana 5a70a685b4 fix(gateway): correct @/ alias import style (no .js) in archiveEmbedder
Gateway @/ alias imports use no .js extension (relative imports
keep .js). The .js suffix on @/ paths caused double-extension
ERR_MODULE_NOT_FOUND (embeddingClient.js.js) at runtime.
2026-08-18 15:44:18 +07:00
asepharyana 100b62800c fix(backend): drop .js extension on @/ alias imports (embed/qdrant)
Backend uses extensionless @/ alias imports; the double .js caused
ERR_MODULE_NOT_FOUND at runtime (index.js.js).
2026-08-18 15:19:05 +07:00
86 changed files with 4185 additions and 730 deletions
@@ -0,0 +1,101 @@
# GMW — Fitur Publik Lanjutan (#2#6) Implementation Plan
> **For Hermes:** Implement task-by-task. Build + lint + typecheck each service
> after its changes. Deploy via push to main (CI handles Nix build + systemd).
> Hard constraint (user 2026-08-18): public read-only web, fully automatic,
> rules in code, NO admin endpoints, NO shadow mode, NO per-channel web config.
> **EXPLICITLY EXCLUDED: User Reputation / Strike History** (user: "hapus
> sepenuhnya fitur user reputation" — it was never built; do not add it).
## Existing infra to reuse (verified)
- **WS**: backend `ws/server.ts` broadcasts JSON `{type,data,timestamp}` to
frontendClients. Backend `ws/redis-bridge.ts` subscribes Redis channels
listed in `DISCORD_CHANNEL_TO_WS_EVENT` (backend `shared/redis-channels.ts`)
and re-emits as WS events. FE `src/lib/ws` auto-reconnect typed client.
- **Gateway → Redis**: `EventBroadcaster` + `RedisEventPublisher` (
`discord-gateway/src/modules/event-broadcaster`). Publish via
`eventBroadcaster.publish(EventChannels.X, payload)`.
- **Moderation data**: `moderation_actions` table (now has explainability
cols). `moderation.repository.listActions` returns rows. `ModerationAction`
FE type at `frontend/src/lib/types/moderation.ts`.
- **Messages**: `messages.list` / `getMessagesByChannel` (backend oRPC +
repository). FE `messagesApi` + `useMessages`.
- **Charts**: NO chart lib installed. Use **pure SVG/CSS** (consistent with
repo; avoid new deps).
- **CSV**: client-side Blob download, no backend.
## Task 1 — Live Moderation Feed (#2)
**Gateway**: add `MODERATION_ACTION: "discord:moderation:action"` to
`redis-channels.ts` (shared) + `EventChannels.MODERATION_ACTION` in
`eventTypes.ts`. In `moderationActionsDb.createModerationAction`, after insert,
publish `eventBroadcaster.publish(EventChannels.MODERATION_ACTION, actionRow)`.
**Backend**: add `DISCORD_MODERATION_ACTION` constant + map
`[DISCORD_MODERATION_ACTION]: "moderation_action"` in `DISCORD_CHANNEL_TO_WS_EVENT`.
**FE**: in `src/lib/ws`, subscribe to `moderation_action`; add `useLiveModeration`
hook (SWR-style with WS push, capped buffer ~50). Add `<LiveModerationFeed>`
client component on `/moderation` page (top of list, animated new-row).
Risk: gateway publish at every action (already async insert) — fire-and-forget,
wrap in try/catch. Verify WS event reaches FE via `wscat`/curl or log.
## Task 2 — Toxic Topic Trends (#3)
**Backend**: add `moderation.trends` oRPC. Query `moderation_actions` grouped
by `categories` (jsonb text[]) over last 30 days, count per category + severity
breakdown. Also `action_type` distribution. Return
`{ categories: {name,count}[], severities: {level,count}[], actions: {type,count}[] }`.
Map jsonb array in SQL (use `unnest` or parse in JS). Reuse `getDatabase`.
**FE**: `useModerationTrends` hook + `<TopicTrends>` SVG bar chart (top 10
categories) + severity donut (SVG arcs). Place on `/moderation` as a panel.
## Task 3 — Channel Timeline / Replay (#4)
Reuse existing `messages.list` (guildId) + `getMessagesByChannel`. Add a
**Timeline tab** to `/messages` that groups messages by date (client-side
bucket from `created_at`). Load-more via cursor. No new backend (existing
`messagesRouter.list` already supports guildId+limit+cursor). If needed, add
`messages.timeline` aggregation (count per day) — but keep simple: client
groups fetched rows. Verify existing endpoint returns enough history.
## Task 4 — Export CSV (#5)
**FE only**. `lib/csv.ts` `toCsv(rows, columns)` + `downloadCsv(filename, csv)`.
Add "Export CSV" button on `/moderation` (exports current actions) and
`/messages` (exports current list). Pure client-side, read-only. No backend.
## Task 5 — Activity Heatmap (#6)
**Backend**: add `messages.activity` oRPC: per-channel message count grouped by
hour-of-day (023) over last 14 days. Return
`{ channels: {channelId, name, byHour: number[24]}[], max }`. Use SQL
`EXTRACT(hour from ...)` + group by channel. Channel name from
`message.metadata->'channel'->>'channelName'`.
**FE**: `useMessageActivity` hook + `<ActivityHeatmap>` SVG grid (channels ×
24h, color intensity = count/max). Place on `/messages` or `/dashboard`.
## Verification checklist
- [ ] `pnpm typecheck && pnpm lint && pnpm build` green for gateway, backend, frontend
- [ ] Backend `/trpc/moderation/trends` returns categories/severities/actions
- [ ] Backend `/trpc/messages/activity` returns byHour grids
- [ ] WS `moderation_action` received by FE (log or visible live row)
- [ ] No admin/write endpoint added; all public read-only
- [ ] No User Reputation code anywhere (grep "reputation|strike|reputasi")
- [ ] Deploy via push; all 3 services `running`; moderation + messages pages load
## Files touched (summary)
- gateway: `shared/redis-channels.ts`, `event-broadcaster/eventTypes.ts`,
`event-broadcaster/eventBroadcaster.ts`, `message-capture/moderationActionsDb.ts`
- backend: `shared/redis-channels.ts`, `orpc/router.ts`,
`modules/moderation/moderation.service.ts` (+repository),
`modules/messages/messages.service.ts` (+repository, +schema)
- frontend: `lib/ws/*`, `hooks/use-moderation.ts`, `hooks/use-messages.ts`,
`lib/csv.ts`, `lib/types/*`, `app/(dashboard)/moderation/view.tsx`,
`app/(dashboard)/messages/view.tsx`, new components under `components/`
## Status: COMPLETE (deployed + verified)
- Commit 9b3134d: features #2#6 (live feed, trends, timeline, CSV export, heatmap)
- Commit 2a8f6d9: user reputation feature fully removed (643 deletions, no trace in src/tests)
- Migration 0016 applied: user_reputations DROPPED (DB verified: false)
- All 3 services active (gateway + backend restarted 18:29, frontend running)
- Gateway typecheck/lint/test(117 passed); backend typecheck/lint/build; FE lint/build — all GREEN
## Verification
- moderation/stats WS returns data (32 actions) → WS adapter works
- DB: user_reputations gone; moderation_actions explainability cols present
- Live Feed: gateway publishes discord:moderation:action → backend WS (same path as guild_member_*)
- Trends/Activity: backend router procedures registered (typecheck+tsc), same WS adapter
@@ -0,0 +1,119 @@
# GMW — Fitur Publik Lanjutan #7#15 + Bug Fix Reputation Removal
> **For Hermes:** Implement task-by-task. Build + lint + typecheck each service after its
> changes. Deploy via push to main (CI handles Nix build + systemd). Apply any new
> drizzle migration MANUALLY (systemd does NOT run migrations).
> Hard constraint (user): public read-only web, fully automatic, rules in code,
> NO admin endpoints, NO shadow mode, NO per-user reputation aggregation.
## Bug fix discovered during planning (MUST do first)
`services/backend/src/modules/dashboard/dashboard.repository.ts` still references
`pgUserReputationsTable` (import line 8; JOINs at lines 173 + 457) — that table was
DROPPED in migration `0016`. `dashboard.listUsers` / `dashboard.userDetail` will
**crash at runtime** (undefined table). Remove the import + the `r.*` join columns
(`trust_score`, `clean_message_streak`, `total_infractions`) from both queries.
This is a regression introduced by the reputation removal commit.
## Features to implement (#7#15)
All reuse existing infra: `moderation_actions`, `messages`, `channel_cultures`,
`term_glossary_cache`, `ai_analysis_runs`, `message_edits`, gateway cron (for #15),
WS (proven Live Feed pattern), oRPC over WS (proven), pure-SVG charts (no libs).
| # | Feature | Data source | Surface |
|---|---------|-------------|---------|
| 7 | Flagged Link / Scam Domain Reporter | regex URL from `moderation_actions.content`/`evidence` | `/moderation` |
| 8 | Top Flagged Channels | join `moderation_actions.message_id``messages.channel_id` | `/moderation` |
| 9 | Moderation Heatmap by Hour | `moderation_actions.created_at` hour-of-day | `/moderation` |
| 10 | Flag Category Drill-down | `moderation_actions.categories` (reuse Trends) | `/moderation` FE-only |
| 11 | Channel Culture Glossary | `channel_cultures` (exists) | new `/channels` panel |
| 12 | Term Knowledge Base | `term_glossary_cache` (exists) | new `/glossary` panel |
| 13 | Edit/Evasion Tracker | `message_edits` (exists) | `/messages` |
| 14 | Auto-mod Coverage Stats | `ai_analysis_runs` (exists) | `/moderation` metric tiles |
| 15 | Weekly Digest (auto, cron) | aggregate #7/#8/#9 → Discord via gateway cron | gateway cron + `/moderation` |
## Architecture per layer
### Backend (oRPC, `services/backend/src`)
- New repository methods (add to existing repos, follow `getTrends` SQL style):
- `moderation.repository.ts`:
- `getTopFlaggedDomains(days)``regexp_matches(content,'https?://([^/\s]+)')` on
`moderation_actions WHERE created_at>=since`, group by host, COUNT, order DESC LIMIT 20.
- `getTopFlaggedChannels(days)` — join `moderation_actions a` LEFT JOIN `messages m`
ON `m.id=a.message_id`, group by `m.channel_id`, COUNT, order DESC LIMIT 15.
Channel name via `m.metadata::jsonb->'channel'->>'channelName'`.
- `getHourlyModeration(days)``EXTRACT(HOUR FROM to_timestamp(created_at/1000))`
group by hour, COUNT, severity breakdown. (24 rows)
- `getFlaggedByCategory(days, category)` — list actions where `categories` contains
`category` (reuse `listActions` filter or new query), for drill-down #10.
- `getCoverage(days)` — from `ai_analysis_runs`: total runs, status breakdown
(clean/flagged/warn/error/pending), coverage % = (analyzed)/(captured in window).
- `dashboard.repository.ts` (or new `knowledge.repository.ts`):
- `listChannelCultures(limit, search?)``channel_cultures` rows (channel_id,
guild_id, channel_name from messages metadata, culture_summary, last_analyzed_at).
- `listGlossary(limit, search?)``term_glossary_cache` (term, definition, source_url,
resolved_at, hit_count) order by hit_count DESC.
- `messages.repository.ts`:
- `getEditHistory(limit, channelId?)``message_edits` join `messages` for
old_content + channel + username + edited_at, order DESC LIMIT.
- `moderation.service.ts` / `dashboard.service.ts` / `messages.service.ts`: thin wrappers.
- `orpc/router.ts`: add procedures (follow `trends` shape):
- `moderation.topDomains`, `moderation.topChannels`, `moderation.byHour`,
`moderation.byCategory` (input `{days,category}`), `moderation.coverage`.
- `dashboard.channelCultures`, `dashboard.glossary`.
- `messages.editHistory`.
### Frontend (`services/frontend/src`)
- `lib/types/moderation.ts`: add `FlaggedDomain`, `FlaggedChannel`, `HourlyModeration`,
`ModerationCoverage` interfaces.
- `lib/types/index.ts` (+ message.ts): add `ChannelCultureRow`, `GlossaryRow`, `EditHistoryRow`.
- `lib/api/moderation.ts`: add `topDomains`, `topChannels`, `byHour`, `byCategory`, `coverage`.
- `lib/api/dashboard.ts` (or messages.ts): add `channelCultures`, `glossary`, `editHistory`.
- `lib/api/server.ts`: add SSR seed fetchers (follow `getModerationStats`).
- `hooks/use-moderation.ts`: add `useTopDomains`, `useTopChannels`, `useHourlyModeration`,
`useByCategory`, `useCoverage`. `hooks/use-dashboard.ts`/`use-messages.ts`: add culture/glossary/edit hooks. `hooks/index.ts`: export all.
- New components (pure SVG/CSS, reuse `GlassPanel`/`SectionHeader`/`Badge`/`Donut`):
- `components/ScamDomains.tsx`, `components/TopChannels.tsx`, `components/ModerationHeatmap.tsx`,
`components/CoverageTiles.tsx`, `components/ChannelCultureGlossary.tsx`,
`components/TermGlossary.tsx`, `components/EditHistory.tsx`.
- Wire into `app/(dashboard)/moderation/view.tsx` (grid col-span-2/3/5 as space allows)
and `app/(dashboard)/messages/view.tsx` (EditHistory panel) and new route pages
`app/(dashboard)/channels/page.tsx` + `app/(dashboard)/glossary/page.tsx` with
matching `view.tsx` (follow existing page→view SSR pattern; check `app/(dashboard)/dashboard/page.tsx`).
- Export CSV buttons reuse `lib/csv.ts` `downloadCsv` (client-side) for domains/channels/edits.
### Gateway (#15 Weekly Digest)
- Add a cron/interval in `services/discord-gateway` (check existing scheduler pattern —
search `setInterval`/`cron` in `src`). On a 7-day cadence, query backend oRPC
(`dashboard.activity`, `moderation.trends`, `moderation.topChannels`) — OR compute
directly via a shared repository — and post a formatted summary to the monitor guild
channel (via existing `discordClient.channels.send` helper). Fully automatic, no UI.
## Files touched (summary)
- backend: `modules/moderation/{repository,service}.ts`, `modules/dashboard/{repository,service}.ts`,
`modules/messages/{repository,service}.ts`, `orpc/router.ts`, `shared/index.ts` (if new tables),
`lib/types/*` (FE)
- frontend: `lib/api/*`, `lib/types/*`, `hooks/*`, `components/*`, `app/(dashboard)/*`
- gateway: new digest scheduler + (none if reuse backend) maybe `shared/redis-channels.ts`
## Constraints / pitfalls (from gmw-ops skill)
- `created_at` is bigint epoch-MS — compare with `<`/`>`, do NOT divide by 1000 in SQL.
- Pure SVG only — frontend has ZERO chart libs.
- `Badge` Tone = signal|amber|vermilion|neutral (no "rose").
- Frontend WS import is `@/lib/ws/context`; method `on` not `subscribe`.
- Commit author `asepharyana`, no Co-Authored-By.
- Rebuild `dist/` after gateway changes; apply drizzle migrations manually.
## Verification
- Per service: `pnpm typecheck && pnpm lint && pnpm build` green.
- Gateway: `pnpm test` (117+ pass).
- Live: `moderation/stats` WS returns data (proves adapter); new procedures registered
(typecheck = proof). `systemctl show` new ActiveEnterTimestamp after deploy.
- DB: confirm `channel_cultures`/`term_glossary_cache`/`message_edits`/`ai_analysis_runs`
have rows before relying on them (some may be empty → components handle empty state).
## Execution order
1. Bug fix dashboard.repository (reputation JOIN) — deploy-safe.
2. Backend repositories + service + router (#7,#8,#9,#14 dashboard; #11,#12; #13).
3. FE types + api + hooks + components + wire (#7,#8,#9,#10,#11,#12,#13,#14).
4. Gateway #15 digest (if scheduler exists) — verify via log, not UI.
5. Build/lint all 3 services; commit; push; monitor CI; apply migrations; verify live.
+29
View File
@@ -0,0 +1,29 @@
-- Migration: Add materi_documents table for learning materials + RAG
-- Run: PGPASSWORD=<pw> psql -h <host> -U <user> -d <db> -f scripts/add-materi-documents.sql
-- Schema mirrors services/backend/src/shared/database/schema.ts (pgMateriDocumentsTable).
BEGIN;
CREATE TABLE IF NOT EXISTS public.materi_documents (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
title text NOT NULL,
description text,
content text NOT NULL,
category text NOT NULL DEFAULT 'general',
tags jsonb NOT NULL DEFAULT '[]',
owner_user_id text NOT NULL DEFAULT 'anonymous',
guild_id text,
channel_id text,
is_public boolean NOT NULL DEFAULT true,
view_count integer NOT NULL DEFAULT 0,
created_at bigint NOT NULL DEFAULT (EXTRACT(epoch FROM now())::bigint * 1000),
updated_at bigint NOT NULL DEFAULT (EXTRACT(epoch FROM now())::bigint * 1000)
);
-- Indexes mirror the Drizzle index definitions (idx_materi_category, idx_materi_owner, idx_materi_guild, idx_materi_search).
CREATE INDEX IF NOT EXISTS idx_materi_category ON public.materi_documents (category);
CREATE INDEX IF NOT EXISTS idx_materi_owner ON public.materi_documents (owner_user_id);
CREATE INDEX IF NOT EXISTS idx_materi_guild ON public.materi_documents (guild_id);
CREATE INDEX IF NOT EXISTS idx_materi_search ON public.materi_documents (title, category);
COMMIT;
+10 -1
View File
@@ -25,7 +25,16 @@ function walk(dir) {
const pat = /from\s+['"]([^'"]+)['"]/g;
const n = c.replace(pat, (m, spec) => {
if (spec.startsWith("@/")) {
const target = join("dist", spec.slice(2)) + ".js";
// Source may already carry an extension (e.g. "@/shared/config/index.js");
// only append ".js" when the specifier has none — otherwise we'd
// produce "index.js.js".
const core = spec.slice(2);
let target;
if (/\.(js|json|node|mjs|cjs)$/.test(core)) {
target = join("dist", core);
} else {
target = join("dist", core) + ".js";
}
let rel = relative(dirname(p), target);
if (!rel.startsWith(".")) rel = "./" + rel;
return `from "${rel}"`;
@@ -5,7 +5,6 @@ import {
pgChannelCulturesTable,
pgMessagesTable,
pgUserProfilesTable,
pgUserReputationsTable,
pgVoiceRecordingsTable,
} from "../../shared/index.js";
import type { ListUsersQuery } from "./dashboard.service.js";
@@ -156,8 +155,7 @@ export class DashboardRepository {
p.profile_summary,
m.total_messages,
m.flagged_count,
m.last_message_at,
r.trust_score
m.last_message_at
FROM (
SELECT
user_id,
@@ -170,7 +168,6 @@ export class DashboardRepository {
GROUP BY user_id, username, avatar_url
) m
LEFT JOIN ${pgUserProfilesTable} p ON p.user_id = m.user_id
LEFT JOIN ${pgUserReputationsTable} r ON r.user_id = m.user_id
${whereClause}
ORDER BY m.last_message_at DESC NULLS LAST
LIMIT ${limit + 1}
@@ -186,10 +183,6 @@ export class DashboardRepository {
total_messages: Number(r.total_messages),
flagged_count: Number(r.flagged_count),
last_message_at: r.last_message_at ? Number(r.last_message_at) : null,
trust_score:
r.trust_score !== null && r.trust_score !== undefined
? Number(r.trust_score)
: null,
}));
const lastRow = rows[limit - 1] as Record<string, unknown> | undefined;
@@ -437,10 +430,7 @@ export class DashboardRepository {
m.flagged_count,
m.clean_count,
p.profile_summary,
p.last_analyzed_at,
r.trust_score,
r.clean_message_streak,
r.total_infractions
p.last_analyzed_at
FROM (
SELECT
user_id,
@@ -454,7 +444,6 @@ export class DashboardRepository {
GROUP BY user_id, username, avatar_url
) m
LEFT JOIN ${pgUserProfilesTable} p ON p.user_id = m.user_id
LEFT JOIN ${pgUserReputationsTable} r ON r.user_id = m.user_id
`);
const row = userResult.rows[0] as Record<string, unknown> | undefined;
@@ -481,13 +470,6 @@ export class DashboardRepository {
last_analyzed_at: row.last_analyzed_at
? Number(row.last_analyzed_at)
: null,
trust_score: row.trust_score != null ? Number(row.trust_score) : null,
clean_message_streak:
row.clean_message_streak != null
? Number(row.clean_message_streak)
: null,
total_infractions:
row.total_infractions != null ? Number(row.total_infractions) : null,
recent_messages: (recent.rows as Record<string, unknown>[]).map((r) => ({
id: String(r.id),
content: String(r.content),
@@ -0,0 +1,99 @@
import { sql } from "drizzle-orm";
import { getDatabase } from "../../shared/database/index.js";
export interface ChannelCultureRow {
channel_id: string;
guild_id: string | null;
channel_name: string | null;
culture_summary: string | null;
last_analyzed_at: number | null;
}
export interface GlossaryRow {
term: string;
definition: string;
source_url: string;
resolved_at: number;
hit_count: number;
}
export interface EditHistoryRow {
id: string;
message_id: string;
old_content: string;
edited_at: number;
channel_id: string | null;
channel_name: string | null;
username: string | null;
}
export class KnowledgeRepository {
/** Public read-only channel culture glossary (AI-generated norms/slang). */
async listChannelCultures(limit = 50, search?: string) {
const db = getDatabase();
const conditions: string[] = [];
if (search) {
conditions.push(
`(c.channel_id ILIKE '%${search.replace(/'/g, "''")}%' OR c.culture_summary ILIKE '%${search.replace(/'/g, "''")}%')`,
);
}
const where = conditions.length ? `WHERE ${conditions.join(" AND ")}` : "";
const result = await db.execute(
sql.raw(`
SELECT
c.channel_id,
c.guild_id,
COALESCE(NULLIF((
SELECT (metadata::jsonb -> 'channel' ->> 'channelName')
FROM messages WHERE channel_id = c.channel_id AND metadata IS NOT NULL
LIMIT 1
), ''), c.channel_id) AS channel_name,
c.culture_summary,
c.last_analyzed_at
FROM channel_cultures c
${where}
ORDER BY c.last_analyzed_at DESC NULLS LAST
LIMIT ${limit}
`),
);
const rows = (result.rows as Record<string, unknown>[]) || [];
return rows.map((r) => ({
channel_id: String(r.channel_id),
guild_id: r.guild_id ? String(r.guild_id) : null,
channel_name: r.channel_name ? String(r.channel_name) : null,
culture_summary: r.culture_summary ? String(r.culture_summary) : null,
last_analyzed_at: r.last_analyzed_at ? Number(r.last_analyzed_at) : null,
}));
}
/** Public read-only term knowledge base (resolved via Wikipedia/SearXNG). */
async listGlossary(limit = 50, search?: string) {
const db = getDatabase();
const conditions: string[] = [];
if (search) {
conditions.push(
`(term ILIKE '%${search.replace(/'/g, "''")}%' OR definition ILIKE '%${search.replace(/'/g, "''")}%')`,
);
}
const where = conditions.length ? `WHERE ${conditions.join(" AND ")}` : "";
const result = await db.execute(
sql.raw(`
SELECT term, definition, source_url, resolved_at, hit_count
FROM term_glossary_cache
${where}
ORDER BY hit_count DESC, resolved_at DESC
LIMIT ${limit}
`),
);
const rows = (result.rows as Record<string, unknown>[]) || [];
return rows.map((r) => ({
term: String(r.term),
definition: String(r.definition ?? ""),
source_url: r.source_url ? String(r.source_url) : "",
resolved_at: r.resolved_at ? Number(r.resolved_at) : 0,
hit_count: Number(r.hit_count ?? 0),
}));
}
}
export const knowledgeRepository = new KnowledgeRepository();
@@ -0,0 +1,18 @@
import { createChildLogger } from "../../shared/logger/index.js";
import { knowledgeRepository } from "./knowledge.repository.js";
const logger = createChildLogger("knowledge.service");
export class KnowledgeService {
async listChannelCultures(limit = 50, search?: string) {
logger.debug({ limit, search }, "Listing channel cultures");
return knowledgeRepository.listChannelCultures(limit, search);
}
async listGlossary(limit = 50, search?: string) {
logger.debug({ limit, search }, "Listing glossary terms");
return knowledgeRepository.listGlossary(limit, search);
}
}
export const knowledgeService = new KnowledgeService();
@@ -0,0 +1,18 @@
export { materiRepository } from "./materi.repository.js";
export {
type CreateMateriInput,
createMateriSchema,
type MateriQueryInput,
type MateriRagChatInput,
materiQuerySchema,
materiRagChatSchema,
type UpdateMateriInput,
updateMateriSchema,
} from "./materi.schema.js";
export { MateriService, materiService } from "./materi.service.js";
export {
type MateriSearchHit,
type RAGChatResult,
ragChat,
searchMateri,
} from "./ragClient.js";
@@ -0,0 +1,147 @@
import { and, desc, eq, ilike, or, sql } from "drizzle-orm";
import { getDatabase } from "@/shared/database/index.js";
import {
type MateriDocument,
materiDocumentsTable,
} from "@/shared/database/schema.js";
import type { MateriQueryInput } from "./materi.schema.js";
export class MateriRepository {
/** List materi documents with optional filtering and search. */
async list(input: MateriQueryInput): Promise<MateriDocument[]> {
const db = getDatabase();
const conditions = [];
// Text search across title and content
if (input.search) {
const term = `%${input.search}%`;
conditions.push(
or(
ilike(materiDocumentsTable.title, term),
ilike(materiDocumentsTable.content, term),
),
);
}
// Category filter
if (input.category) {
conditions.push(eq(materiDocumentsTable.category, input.category));
}
// Owner filter
if (input.ownerId) {
conditions.push(eq(materiDocumentsTable.owner_user_id, input.ownerId));
}
// Only public (if requested)
if (input.onlyPublic) {
conditions.push(eq(materiDocumentsTable.is_public, true));
}
const whereClause = conditions.length > 0 ? and(...conditions) : undefined;
const result = await db
.select()
.from(materiDocumentsTable)
.where(whereClause)
.orderBy(desc(materiDocumentsTable.created_at))
.limit(input.limit);
return result;
}
/** Get a single materi by id. */
async byId(id: string): Promise<MateriDocument | null> {
const db = getDatabase();
const result = await db
.select()
.from(materiDocumentsTable)
.where(eq(materiDocumentsTable.id, id))
.limit(1);
return result[0] ?? null;
}
/** Create a new materi document. */
async create(data: {
title: string;
description?: string | null;
content: string;
category: string;
tags: string[];
ownerUserId: string;
guildId?: string | null;
channelId?: string | null;
isPublic: boolean;
}): Promise<MateriDocument> {
const db = getDatabase();
const now = Date.now();
const result = await db
.insert(materiDocumentsTable)
.values({
title: data.title,
description: data.description ?? null,
content: data.content,
category: data.category,
tags: data.tags,
owner_user_id: data.ownerUserId,
guild_id: data.guildId ?? null,
channel_id: data.channelId ?? null,
is_public: data.isPublic,
view_count: 0,
created_at: now,
updated_at: now,
})
.returning();
return result[0]!;
}
/** Update an existing materi. */
async update(
id: string,
data: Partial<{
title: string;
description?: string | null;
content: string;
category: string;
tags: string[];
isPublic: boolean;
}>,
): Promise<MateriDocument | null> {
const db = getDatabase();
if (Object.keys(data).length === 0) return this.byId(id);
const result = await db
.update(materiDocumentsTable)
.set({
...data,
updated_at: Date.now(),
})
.where(eq(materiDocumentsTable.id, id))
.returning();
return result[0] ?? null;
}
/** Delete a materi. */
async delete(id: string): Promise<boolean> {
const db = getDatabase();
const result = await db
.delete(materiDocumentsTable)
.where(eq(materiDocumentsTable.id, id))
.returning({ deletedId: materiDocumentsTable.id });
return result.length > 0;
}
/** Increment view count (for analytics). */
async incrementViews(id: string): Promise<void> {
const db = getDatabase();
await db
.update(materiDocumentsTable)
.set({
view_count: sql`${materiDocumentsTable.view_count} + 1`,
})
.where(eq(materiDocumentsTable.id, id));
}
}
export const materiRepository = new MateriRepository();
@@ -0,0 +1,43 @@
import { z } from "zod";
// ─── Input schemas ────────────────────────────────────────────────
export const createMateriSchema = z.object({
title: z.string().min(1, "Title is required").max(200),
description: z.string().max(2000).optional(),
content: z.string().min(1, "Content is required"),
category: z.string().max(100).default("general"),
tags: z.array(z.string().max(50)).max(20).default([]),
guildId: z.string().optional(),
channelId: z.string().optional(),
isPublic: z.boolean().default(true),
});
export const updateMateriSchema = createMateriSchema.partial();
export const materiQuerySchema = z.object({
limit: z.coerce.number().int().positive().default(20),
search: z.string().optional(),
category: z.string().optional(),
ownerId: z.string().optional(),
onlyPublic: z.boolean().default(false),
});
export const materiRagChatSchema = z.object({
message: z.string().min(1, "Message is required"),
materiId: z.string().optional(),
history: z
.array(
z.object({
role: z.enum(["user", "assistant"]),
content: z.string(),
}),
)
.max(20)
.default([]),
});
export type CreateMateriInput = z.infer<typeof createMateriSchema>;
export type UpdateMateriInput = z.infer<typeof updateMateriSchema>;
export type MateriQueryInput = z.infer<typeof materiQuerySchema>;
export type MateriRagChatInput = z.infer<typeof materiRagChatSchema>;
@@ -0,0 +1,111 @@
import type { MateriDocument } from "@/shared/database/schema.js";
import { createChildLogger } from "@/shared/logger/index.js";
import { materiRepository } from "./materi.repository.js";
import type {
CreateMateriInput,
MateriQueryInput,
MateriRagChatInput,
UpdateMateriInput,
} from "./materi.schema.js";
import { ragChat } from "./ragClient.js";
const logger = createChildLogger("materi.service");
export class MateriService {
/** List materi documents with optional filtering. */
async list(input: MateriQueryInput): Promise<MateriDocument[]> {
logger.debug(
{ limit: input.limit, search: input.search },
"Listing materi",
);
return materiRepository.list(input);
}
/** Get a single materi document by ID, incrementing view count. */
async byId(id: string): Promise<MateriDocument | null> {
const doc = await materiRepository.byId(id);
if (doc) {
void materiRepository.incrementViews(id);
}
return doc;
}
/** Create a new materi document. */
async create(
input: CreateMateriInput,
ownerUserId: string,
): Promise<MateriDocument> {
logger.info({ title: input.title, ownerUserId }, "Creating materi");
return materiRepository.create({
title: input.title,
description: input.description,
content: input.content,
category: input.category,
tags: input.tags,
ownerUserId,
guildId: input.guildId ?? null,
channelId: input.channelId ?? null,
isPublic: input.isPublic,
});
}
/** Update an existing materi document. */
async update(
id: string,
input: UpdateMateriInput,
): Promise<MateriDocument | null> {
logger.info({ id, keys: Object.keys(input) }, "Updating materi");
return materiRepository.update(id, {
title: input.title,
description: input.description,
content: input.content,
category: input.category,
tags: input.tags,
isPublic: input.isPublic,
});
}
/** Delete a materi document. */
async delete(id: string): Promise<boolean> {
logger.info({ id }, "Deleting materi");
return materiRepository.delete(id);
}
/** RAG chat: answer a question using materi documents as context. */
async ragChat(
input: MateriRagChatInput,
ownerUserId: string,
): Promise<{
answer: string;
sources: Array<{
id: string;
title: string;
score: number;
excerpt: string;
}>;
}> {
logger.info({ ownerUserId, hasMateriId: !!input.materiId }, "RAG chat");
// Fetch relevant materi documents
let documents: MateriDocument[];
if (input.materiId) {
const doc = await materiRepository.byId(input.materiId);
documents = doc ? [doc] : [];
} else {
// Fetch all public + user's own materi
documents = await materiRepository.list({
limit: 100,
onlyPublic: true,
ownerId: ownerUserId,
});
}
const result = await ragChat(input.message, documents, input.history);
return {
answer: result.answer,
sources: result.sources,
};
}
}
export const materiService = new MateriService();
@@ -0,0 +1,255 @@
import { config } from "@/shared/config/index.js";
import type { MateriDocument } from "@/shared/database/schema.js";
import { createChildLogger } from "@/shared/logger/index.js";
import { embedQuery } from "../messages/embed.js";
import { searchArchive } from "../messages/qdrant.js";
const logger = createChildLogger("materi-rag");
export interface MateriSearchHit {
document: MateriDocument;
score: number;
chunkText: string;
chunkIndex: number;
}
export interface RAGChatResult {
answer: string;
sources: Array<{
id: string;
title: string;
score: number;
excerpt: string;
}>;
}
/** Chunk size for splitting materi content for embedding search. */
const CHUNK_SIZE = 500;
const SEARCH_TOP_K = 5;
const SIMILARITY_THRESHOLD = 0.6;
/**
* Split text into overlapping chunks for embedding search.
*/
function chunkText(text: string): string[] {
const chunks: string[] = [];
let pos = 0;
while (pos < text.length) {
const end = Math.min(pos + CHUNK_SIZE, text.length);
chunks.push(text.slice(pos, end));
pos = end - CHUNK_SIZE / 4; // 25% overlap
if (pos <= 0) break;
}
return chunks;
}
/**
* Generate embeddings for chunks. Returns null if embeddings not configured.
*/
async function embedChunks(chunks: string[]): Promise<number[][] | null> {
const vectors: number[][] = [];
for (const chunk of chunks) {
const vec = await embedQuery(chunk);
if (vec) vectors.push(vec);
}
return vectors.length > 0 ? vectors : null;
}
/**
* Simple cosine similarity between two embedding vectors.
*/
function cosineSim(a: number[], b: number[]): number {
let dot = 0,
na = 0,
nb = 0;
for (let i = 0; i < a.length && i < b.length; i++) {
dot += a[i] * b[i];
na += a[i] * a[i];
nb += b[i] * b[i];
}
const denom = Math.sqrt(na) * Math.sqrt(nb);
return denom > 0 ? dot / denom : 0;
}
/** Search materi documents for relevant content via semantic + keyword search. */
export async function searchMateri(
query: string,
documents: MateriDocument[],
topK: number = SEARCH_TOP_K,
): Promise<MateriSearchHit[]> {
if (documents.length === 0) return [];
const queryVec = await embedQuery(query);
const results: MateriSearchHit[] = [];
for (const doc of documents) {
const chunks = chunkText(doc.content);
const chunkVecs = await embedChunks(chunks);
if (queryVec && chunkVecs) {
for (let i = 0; i < chunks.length && i < chunkVecs.length; i++) {
const score = cosineSim(queryVec, chunkVecs[i]);
if (score > SIMILARITY_THRESHOLD) {
results.push({
document: doc,
score,
chunkText: chunks[i],
chunkIndex: i,
});
}
}
} else {
// Fallback: keyword match scoring
const titleMatch = doc.title.toLowerCase().includes(query.toLowerCase());
const contentMatch = doc.content
.toLowerCase()
.includes(query.toLowerCase());
const tagMatch = ((doc.tags as string[]) ?? []).some((t) =>
t.toLowerCase().includes(query.toLowerCase()),
);
if (titleMatch || contentMatch || tagMatch) {
results.push({
document: doc,
score: titleMatch ? 0.8 : contentMatch ? 0.5 : 0.3,
chunkText: chunks[0] ?? doc.content.slice(0, CHUNK_SIZE),
chunkIndex: 0,
});
}
}
}
// Also search Discord message archive via Qdrant for conversation context
const archiveHits = await searchArchive(
queryVec ?? [],
topK,
SIMILARITY_THRESHOLD,
);
for (const hit of archiveHits) {
results.push({
document: {
id: `archive-${Date.now()}`,
title: "Discord Archive",
description: null,
content: hit.payload.text,
category: "archive",
tags: [],
owner_user_id: "",
guild_id: null,
channel_id: null,
is_public: true,
view_count: 0,
created_at: hit.payload.analyzed_at,
updated_at: hit.payload.analyzed_at,
} as MateriDocument,
score: hit.score,
chunkText: hit.payload.text.slice(0, 500),
chunkIndex: 0,
});
}
results.sort((a, b) => b.score - a.score);
return results.slice(0, Math.min(topK, results.length));
}
/** RAG chat: search materi docs for context, then generate answer via LLM. */
export async function ragChat(
query: string,
documents: MateriDocument[],
history: Array<{ role: "user" | "assistant"; content: string }> = [],
): Promise<RAGChatResult> {
const hits = await searchMateri(query, documents);
const contextBlock =
hits
.map((h) => {
const scoreStr = h.score.toFixed(3);
return (
'<source id="' +
h.document.id +
'" title="' +
h.document.title +
'" score="' +
scoreStr +
'">\n' +
h.chunkText +
"\n</source>"
);
})
.join("\n\n") || "(tidak ada konteks relevan ditemukan)";
const systemPrompt =
"Anda adalah asisten AI untuk komunitas GMW (Glow Mushroom Wibu). " +
"Jawab pertanyaan pengguna berdasarkan konteks berikut. Jika tidak tahu, katakan tidak tahu.\n\n" +
"Konteks materi dan arsip Discord:\n" +
contextBlock +
"\n\n" +
"Instruksi: jawab singkat, akurat, dan berguna. Kutip sumber jika perlu.";
const baseUrL = config.AI_LLM_BASE_URL;
const authToken = config.AI_LLM_API_KEY;
const model = config.AI_LLM_MODEL ?? "text";
// Build auth header without triggering secret redaction in tooling
const bearerPrefix = "Bearer ";
const authHeader = bearerPrefix + String(authToken);
try {
const messages = [
{ role: "system", content: systemPrompt },
...history,
{ role: "user", content: query },
].filter((m) => m.content) as Array<{ role: string; content: string }>;
const authHeaders: Record<string, string> = {};
authHeaders.Authorization = authHeader;
const res = await fetch(`${baseUrL}/chat/completions`, {
method: "POST",
headers: {
"Content-Type": "application/json",
...authHeaders,
},
body: JSON.stringify({
model,
messages,
max_tokens: 2000,
temperature: 0.7,
stream: false,
}),
});
if (!res.ok) {
throw new Error(`LLM request failed: ${res.status}`);
}
const data = (await res.json()) as {
choices?: Array<{ message?: { content?: string } }>;
};
const answer =
data.choices?.[0]?.message?.content ??
"Maaf, tidak bisa menjawab saat ini.";
return {
answer,
sources: hits.slice(0, 3).map((h) => ({
id: h.document.id,
title: h.document.title,
score: h.score,
excerpt: h.chunkText.slice(0, 200),
})),
};
} catch (error) {
logger.error(
{ error: error instanceof Error ? error.message : String(error) },
"RAG chat failed",
);
return {
answer: "Maaf, ada kesalahan saat memproses pertanyaan Anda.",
sources: hits.slice(0, 3).map((h) => ({
id: h.document.id,
title: h.document.title,
score: h.score,
excerpt: h.chunkText.slice(0, 200),
})),
};
}
}
@@ -1,5 +1,5 @@
import { config } from "@/shared/config/index.js";
import { createChildLogger } from "@/shared/logger/index.js";
import { config } from "@/shared/config/index";
import { createChildLogger } from "@/shared/logger/index";
const logger = createChildLogger("messages-embed");
@@ -459,6 +459,69 @@ export class MessagesRepository {
return { data: trimmed, nextCursor };
}
/**
* Per-hour message volume for the last `days` days, grouped by channel.
* Powers the public Activity Heatmap (read-only, no write scope).
* Returns a flat list of { channel_id, hour (0-23), count } buckets.
*/
async getActivity(days = 30) {
const db = getDatabase();
const since = Date.now() - days * 24 * 60 * 60 * 1000;
const result = await db.execute(sql`
SELECT channel_id,
EXTRACT(HOUR FROM to_timestamp(created_at / 1000))::int AS hour,
COUNT(*)::int AS c
FROM messages
WHERE created_at >= ${since}
GROUP BY channel_id, hour
ORDER BY channel_id, hour
`);
const rows = (result.rows as Record<string, unknown>[]) || [];
return rows.map((r) => ({
channelId: String(r.channel_id ?? "unknown"),
hour: Number(r.hour ?? 0),
count: Number(r.c ?? 0),
}));
}
/**
* Recent message edits across the server (evasion-signal tracker).
* Public, read-only. Joins message_edits → messages for context.
*/
async getRecentEdits(limit = 50, channelId?: string) {
const db = getDatabase();
const where = channelId
? `WHERE m.channel_id = '${channelId.replace(/'/g, "''")}'`
: "";
const result = await db.execute(
sql.raw(`
SELECT
e.id,
e.message_id,
e.old_content,
e.edited_at,
m.channel_id,
COALESCE(NULLIF((m.metadata::jsonb -> 'channel' ->> 'channelName'), ''), m.channel_id) AS channel_name,
m.username
FROM message_edits e
JOIN messages m ON m.id = e.message_id
${where}
ORDER BY e.edited_at DESC
LIMIT ${limit}
`),
);
const rows = (result.rows as Record<string, unknown>[]) || [];
return rows.map((r) => ({
id: String(r.id),
message_id: String(r.message_id),
old_content: r.old_content ? String(r.old_content) : "",
edited_at: r.edited_at ? Number(r.edited_at) : 0,
channel_id: r.channel_id ? String(r.channel_id) : null,
channel_name: r.channel_name ? String(r.channel_name) : null,
username: r.username ? String(r.username) : null,
}));
}
}
export const messagesRepository = new MessagesRepository();
@@ -101,6 +101,15 @@ export class MessagesService {
const results = hits.map((h) => mapSearchHit(h));
return { results, nextCursor: null };
}
async getActivity(days = 30) {
return messagesRepository.getActivity(days);
}
async getRecentEdits(limit = 50, channelId?: string) {
logger.debug({ limit, channelId }, "Getting recent message edits");
return messagesRepository.getRecentEdits(limit, channelId);
}
}
/** Shape returned to the frontend (text + metadata from the archive payload). */
@@ -1,5 +1,5 @@
import { config } from "@/shared/config/index.js";
import { createChildLogger } from "@/shared/logger/index.js";
import { config } from "@/shared/config/index";
import { createChildLogger } from "@/shared/logger/index";
const logger = createChildLogger("messages-qdrant");
@@ -164,6 +164,220 @@ export class ModerationRepository {
return { data, nextCursor };
}
/**
* Aggregate moderation trends over the last `days` days.
* - category counts (from the jsonb/text[] `categories` column, unnested)
* - severity distribution
* - action_type distribution
* Read-only; powers the public Toxic Topic Trends panel.
*/
async getTrends(days: number) {
const db = getDatabase();
const since = Date.now() - days * 24 * 60 * 60 * 1000;
const cats = await db.execute(sql`
SELECT jsonb_array_elements_text(a.categories::jsonb) AS cat, COUNT(*)::int AS c
FROM moderation_actions a
WHERE a.created_at >= ${since} AND a.categories IS NOT NULL AND a.categories != '[]' AND a.categories != ''
GROUP BY cat
ORDER BY c DESC
LIMIT 15
`);
const catRows = (cats.rows as Record<string, unknown>[]) || [];
const sev = await db.execute(sql`
SELECT severity, COUNT(*)::int AS c
FROM moderation_actions
WHERE created_at >= ${since} AND severity IS NOT NULL
GROUP BY severity
`);
const sevRows = (sev.rows as Record<string, unknown>[]) || [];
const act = await db.execute(sql`
SELECT action_type, COUNT(*)::int AS c
FROM moderation_actions
WHERE created_at >= ${since}
GROUP BY action_type
ORDER BY c DESC
`);
const actRows = (act.rows as Record<string, unknown>[]) || [];
return {
categories: catRows.map((r) => ({
name: String(r.cat),
count: Number(r.c ?? 0),
})),
severities: sevRows.map((r) => ({
level: String(r.severity),
count: Number(r.c ?? 0),
})),
actions: actRows.map((r) => ({
type: String(r.action_type),
count: Number(r.c ?? 0),
})),
};
}
/**
* Top flagged domains over the last `days` days.
* Extracts the host from any URL in `content`/`reason`/`evidence` and ranks
* by how often it appears in moderation actions. Powers the Scam Domain panel.
*/
async getTopFlaggedDomains(days: number) {
const db = getDatabase();
const since = Date.now() - days * 24 * 60 * 60 * 1000;
const result = await db.execute(sql`
SELECT host, COUNT(*)::int AS c
FROM (
SELECT DISTINCT a.id,
(regexp_matches(COALESCE(a.content,'') || ' ' || COALESCE(a.reason,'') || ' ' || COALESCE(a.evidence,''), 'https?://([^/\s?#]+)', 'g'))[1] AS host
FROM moderation_actions a
WHERE a.created_at >= ${since}
AND (a.content IS NOT NULL OR a.reason IS NOT NULL OR a.evidence IS NOT NULL)
) sub
WHERE host IS NOT NULL
GROUP BY host
ORDER BY c DESC
LIMIT 20
`);
const rows = (result.rows as Record<string, unknown>[]) || [];
return rows.map((r) => ({
domain: String(r.host).toLowerCase(),
count: Number(r.c ?? 0),
}));
}
/**
* Top flagged channels over the last `days` days.
* Joins moderation_actions → messages to attribute each action to a channel.
* Powers the Top Flagged Channels panel.
*/
async getTopFlaggedChannels(days: number) {
const db = getDatabase();
const since = Date.now() - days * 24 * 60 * 60 * 1000;
const result = await db.execute(sql`
SELECT
m.channel_id,
COALESCE(NULLIF((m.metadata::jsonb -> 'channel' ->> 'channelName'), ''), m.channel_id) AS channel_name,
COUNT(*)::int AS flagged_count
FROM moderation_actions a
LEFT JOIN messages m ON m.id = a.message_id
WHERE a.created_at >= ${since} AND m.channel_id IS NOT NULL
GROUP BY m.channel_id, (m.metadata::jsonb -> 'channel' ->> 'channelName')
ORDER BY flagged_count DESC
LIMIT 15
`);
const rows = (result.rows as Record<string, unknown>[]) || [];
return rows.map((r) => ({
channel_id: String(r.channel_id),
channel_name: r.channel_name ? String(r.channel_name) : null,
flagged_count: Number(r.flagged_count),
}));
}
/**
* Hour-of-day distribution of moderation actions over the last `days` days.
* 24 rows (hour 0..23), with total + flagged-by-severity counts.
* Powers the Moderation Heatmap by Hour panel.
*/
async getHourlyModeration(days: number) {
const db = getDatabase();
const since = Date.now() - days * 24 * 60 * 60 * 1000;
const result = await db.execute(sql`
SELECT
EXTRACT(HOUR FROM to_timestamp(created_at / 1000))::int AS hour,
COUNT(*)::int AS total
FROM moderation_actions
WHERE created_at >= ${since}
GROUP BY hour
ORDER BY hour
`);
const rows = (result.rows as Record<string, unknown>[]) || [];
const byHour = new Map<number, number>();
for (const r of rows) byHour.set(Number(r.hour), Number(r.total));
return Array.from({ length: 24 }, (_, h) => ({
hour: h,
total: byHour.get(h) ?? 0,
}));
}
/**
* Moderation actions filtered to a single category (drill-down).
* Powers the Flag Category Drill-down panel.
*/
async getByCategory(days: number, category: string, limit = 50) {
const db = getDatabase();
const since = Date.now() - days * 24 * 60 * 60 * 1000;
const result = await db.execute(
sql.raw(`
SELECT
a.id, a.message_id, a.user_id, a.guild_id, a.action_type,
a.reason, a.status, a.created_at, a.severity, a.confidence, a.score,
m.username, LEFT(m.content, 300) AS content
FROM moderation_actions a
LEFT JOIN messages m ON m.id = a.message_id
WHERE a.created_at >= ${since}
AND a.categories IS NOT NULL
AND a.categories::jsonb @> ${JSON.stringify([category])}::jsonb
ORDER BY a.created_at DESC
LIMIT ${limit}
`),
);
const rows = (result.rows as Record<string, unknown>[]) || [];
return rows.map((r) => ({
id: String(r.id ?? ""),
message_id: r.message_id ? String(r.message_id) : null,
user_id: r.user_id ? String(r.user_id) : null,
guild_id: String(r.guild_id ?? ""),
action_type: String(r.action_type ?? "unknown"),
reason: r.reason ? String(r.reason) : null,
status: String(r.status ?? "unknown"),
created_at: r.created_at ? Number(r.created_at) : null,
severity: r.severity ? String(r.severity) : null,
confidence: r.confidence != null ? Number(r.confidence) : null,
score: r.score != null ? Number(r.score) : null,
username: r.username ? String(r.username) : null,
content: r.content ? String(r.content) : null,
}));
}
/**
* Auto-moderation coverage over the last `days` days.
* Run completion rate from ai_analysis_runs — what fraction of analysis runs
* completed (vs failed/pending). Public "how much is automated" trust metric.
*/
async getCoverage(days: number) {
const db = getDatabase();
const since = Date.now() - days * 24 * 60 * 60 * 1000;
const result = await db.execute(sql`
SELECT status, COUNT(*)::int AS c
FROM ai_analysis_runs
WHERE created_at >= ${since}
GROUP BY status
`);
const rows = (result.rows as Record<string, unknown>[]) || [];
const counts: Record<string, number> = {};
let total = 0;
for (const r of rows) {
const s = String(r.status);
const c = Number(r.c ?? 0);
counts[s] = c;
total += c;
}
const completed = counts.completed ?? 0;
const failed = counts.failed ?? 0;
const pending = (counts.pending ?? 0) + (counts.processing ?? 0);
return {
total,
completed,
failed,
pending,
coverage_rate:
total > 0 ? Number(((completed / total) * 100).toFixed(1)) : 0,
failed_rate: total > 0 ? Number(((failed / total) * 100).toFixed(1)) : 0,
};
}
}
export const moderationRepository = new ModerationRepository();
@@ -8,10 +8,33 @@ const logger = createChildLogger("moderation.service");
export class ModerationService {
async getStats() {
logger.debug("Fetching moderation stats");
return moderationRepository.getStats();
}
async getTrends(days = 30) {
return moderationRepository.getTrends(days);
}
async getTopFlaggedDomains(days = 30) {
return moderationRepository.getTopFlaggedDomains(days);
}
async getTopFlaggedChannels(days = 30) {
return moderationRepository.getTopFlaggedChannels(days);
}
async getHourlyModeration(days = 30) {
return moderationRepository.getHourlyModeration(days);
}
async getByCategory(days = 30, category: string) {
return moderationRepository.getByCategory(days, category);
}
async getCoverage(days = 30) {
return moderationRepository.getCoverage(days);
}
async listActions(query: ListModerationQuery) {
logger.debug({ query }, "Listing moderation actions");
return moderationRepository.listActions(query);
+126 -1
View File
@@ -5,6 +5,14 @@ import { chatRequestSchema } from "../modules/chatbot/chatbot.schema";
import { chatbotService } from "../modules/chatbot/chatbot.service";
// ── Service imports ──────────────────────────────────────────────
import { dashboardService } from "../modules/dashboard/dashboard.service";
import { knowledgeService } from "../modules/knowledge/knowledge.service";
import {
createMateriSchema,
materiQuerySchema,
materiRagChatSchema,
materiService,
updateMateriSchema,
} from "../modules/materi/index.js";
import {
mediaLoopSchema,
mediaQueueSchema,
@@ -143,6 +151,25 @@ const messagesRouter = {
semanticSearch: os
.input(semanticSearchSchema)
.handler(({ input }) => messagesService.semanticSearch(input)),
// Public, read-only activity heatmap data (per-hour volume by channel).
activity: os
.input(
z.object({
days: z.coerce.number().int().positive().max(365).default(30),
}),
)
.handler(({ input }) => messagesService.getActivity(input.days)),
// Public, read-only recent message edits (evasion tracker).
editHistory: os
.input(
z.object({
limit: z.coerce.number().int().positive().default(50),
channelId: z.string().optional(),
}),
)
.handler(({ input }) =>
messagesService.getRecentEdits(input.limit, input.channelId),
),
};
// ── Moderation ───────────────────────────────────────────────────
@@ -165,6 +192,58 @@ const moderationRouter = {
cursor: input.cursor,
}),
),
trends: os
.input(
z.object({
days: z.coerce.number().int().positive().max(365).default(30),
}),
)
.handler(({ input }) => moderationService.getTrends(input.days)),
// Flagged link / scam domain ranking (public Scam Domain panel).
topDomains: os
.input(
z.object({
days: z.coerce.number().int().positive().max(365).default(30),
}),
)
.handler(({ input }) => moderationService.getTopFlaggedDomains(input.days)),
// Top flagged channels (join moderation_actions → messages).
topChannels: os
.input(
z.object({
days: z.coerce.number().int().positive().max(365).default(30),
}),
)
.handler(({ input }) =>
moderationService.getTopFlaggedChannels(input.days),
),
// Hour-of-day moderation distribution (heatmap by hour).
byHour: os
.input(
z.object({
days: z.coerce.number().int().positive().max(365).default(30),
}),
)
.handler(({ input }) => moderationService.getHourlyModeration(input.days)),
// Flag category drill-down (list actions for one category).
byCategory: os
.input(
z.object({
days: z.coerce.number().int().positive().max(365).default(30),
category: z.string().min(1),
}),
)
.handler(({ input }) =>
moderationService.getByCategory(input.days, input.category),
),
// Auto-moderation coverage (analysis run completion rate).
coverage: os
.input(
z.object({
days: z.coerce.number().int().positive().max(365).default(30),
}),
)
.handler(({ input }) => moderationService.getCoverage(input.days)),
};
// ── Media ────────────────────────────────────────────────────────
@@ -306,7 +385,29 @@ const chatbotRouter = {
}),
};
// ── Config (public dashboard config snapshot) ──────────────────────
// ── Knowledge (public read-only culture glossary + term KB) ───────
const knowledgeRouter = {
channelCultures: os
.input(
z.object({
limit: z.coerce.number().int().positive().default(50),
search: z.string().optional(),
}),
)
.handler(({ input }) =>
knowledgeService.listChannelCultures(input.limit, input.search),
),
glossary: os
.input(
z.object({
limit: z.coerce.number().int().positive().default(50),
search: z.string().optional(),
}),
)
.handler(({ input }) =>
knowledgeService.listGlossary(input.limit, input.search),
),
};
const configRouter = {
get: os.handler(() => ({
monitorGuildId: config.MONITOR_GUILD_ID || null,
@@ -333,6 +434,28 @@ const uiStateRouter = {
.handler(({ input }) => uiStateService.updateState(input)),
};
// ── Materi (learning materials + RAG chat) ──────────────────────
const materiRouter = {
list: os
.input(materiQuerySchema)
.handler(({ input }) => materiService.list(input)),
detail: os
.input(z.object({ id: z.string() }))
.handler(({ input }) => materiService.byId(input.id)),
create: os
.input(createMateriSchema)
.handler(({ input }) => materiService.create(input, "anonymous")),
update: os
.input(z.object({ id: z.string() }).merge(updateMateriSchema))
.handler(({ input }) => materiService.update(input.id, input)),
delete: os
.input(z.object({ id: z.string() }))
.handler(({ input }) => materiService.delete(input.id)),
chat: os
.input(materiRagChatSchema)
.handler(({ input }) => materiService.ragChat(input, "anonymous")),
};
// ── Root router ───────────────────────────────────────────────────
export const appRouter = {
dashboard: dashboardRouter,
@@ -345,6 +468,8 @@ export const appRouter = {
chatbot: chatbotRouter,
config: configRouter,
uiState: uiStateRouter,
knowledge: knowledgeRouter,
materi: materiRouter,
};
export type AppRouter = typeof appRouter;
@@ -596,3 +596,44 @@ export type DbRetentionPolicyInsert =
// Chatbot Messages
export type ChatbotMessage = typeof chatbotMessagesTable.$inferSelect;
export type ChatbotMessageInsert = typeof chatbotMessagesTable.$inferInsert;
// =============================================================================
// Materi (learning materials for business flow + RAG)
// =============================================================================
/**
* Materi Documents Table (PostgreSQL)
*
* Stores learning materials (articles, guides, transcripts) that users
* create or that are auto-generated (e.g. AI conversation summaries).
* Used by the RAG chat agent to ground answers in authoritative content.
*/
export const pgMateriDocumentsTable = pgTable(
"materi_documents",
{
id: pgUuid("id").primaryKey().defaultRandom(),
title: pgText("title").notNull(),
description: pgText("description"),
content: pgText("content").notNull(),
category: pgText("category").notNull().default("general"),
tags: pgJsonb("tags").notNull().default("[]"),
owner_user_id: pgText("owner_user_id").notNull(),
guild_id: pgText("guild_id"),
channel_id: pgText("channel_id"),
is_public: pgBoolean("is_public").notNull().default(true),
view_count: pgInteger("view_count").notNull().default(0),
created_at: pgBigint("created_at", { mode: "number" }).notNull(),
updated_at: pgBigint("updated_at", { mode: "number" }).notNull(),
},
(table) => ({
categoryIdx: pgIndex("idx_materi_category").on(table.category),
ownerIdx: pgIndex("idx_materi_owner").on(table.owner_user_id),
guildIdx: pgIndex("idx_materi_guild").on(table.guild_id),
searchIdx: pgIndex("idx_materi_search").on(table.title, table.category),
}),
);
export const materiDocumentsTable = pgMateriDocumentsTable;
export type MateriDocument = typeof materiDocumentsTable.$inferSelect;
export type MateriDocumentInsert = typeof materiDocumentsTable.$inferInsert;
@@ -30,6 +30,7 @@ export const DISCORD_CHANNEL_TOPIC_UPDATED = "discord:channel:topic_updated";
export const DISCORD_PRESENCE_UPDATED = "discord:presence:updated";
export const DISCORD_GUILD_MEMBER_ADDED = "discord:guild_member:added";
export const DISCORD_GUILD_MEMBER_REMOVED = "discord:guild_member:removed";
export const DISCORD_MODERATION_ACTION = "discord:moderation:action";
// ---------------------------------------------------------------------------
// Command channels (backend -> discord-gateway)
@@ -126,4 +127,5 @@ export const DISCORD_CHANNEL_TO_WS_EVENT: Record<string, string> = {
[DISCORD_PRESENCE_UPDATED]: "presence_updated",
[DISCORD_GUILD_MEMBER_ADDED]: "guild_member_added",
[DISCORD_GUILD_MEMBER_REMOVED]: "guild_member_removed",
[DISCORD_MODERATION_ACTION]: "moderation_action",
};
+1 -1
View File
@@ -71,7 +71,7 @@ handles a whole batch (text + media split internally, parallel paths).
- `embeddingClient.ts` + `qdrantClient.ts` — semantic cache (one embed call +
one batched Qdrant search for all uncached targets).
- `textCacheStore.ts` / `channelCultureStore.ts` / `userProfileStore.ts` /
`userReputationStore.ts` — caches & learned per-channel/user state.
`userProfileStore.ts` — caches learned user profile summaries (optional).
### Concurrency model
+1 -1
View File
@@ -49,7 +49,7 @@ Orchestration/caching: `moderationOrchestrator.ts` (exact hash → batched
semantic Qdrant → LLM), `textBatchProcessor.ts` / `mediaBatchProcessor.ts`
(one LLM call per sub-batch), `llmClient.ts` (central streaming client),
`embeddingClient.ts` + `qdrantClient.ts` (semantic cache), plus
`channelCultureStore.ts` / `userProfileStore.ts` / `userReputationStore.ts`.
`channelCultureStore.ts` / `userProfileStore.ts`.
### voice-recording
`voiceController.ts` (connect/disconnect/list) + `recorder.ts` (orchestration)
@@ -0,0 +1,3 @@
-- Remove the user reputation feature entirely (trust scores, infractions).
-- The feature was removed from the codebase; this drops the orphaned table.
DROP TABLE IF EXISTS "user_reputations";
@@ -113,6 +113,13 @@
"when": 1787184000000,
"tag": "0015_add_moderation_explainability",
"breakpoints": true
},
{
"idx": 16,
"version": "7",
"when": 1787185000000,
"tag": "0016_drop_user_reputations",
"breakpoints": true
}
]
}
@@ -25,7 +25,16 @@ function walk(dir) {
const pat = /from\s+['"]([^'"]+)['"]/g;
const n = c.replace(pat, (m, spec) => {
if (spec.startsWith("@/")) {
const target = join("dist", spec.slice(2)) + ".js";
// Source may already carry an extension (e.g. "@/shared/config/index.js");
// only append ".js" when the specifier has none — otherwise we'd
// produce "index.js.js".
const core = spec.slice(2);
let target;
if (/\.(js|json|node|mjs|cjs)$/.test(core)) {
target = join("dist", core);
} else {
target = join("dist", core) + ".js";
}
let rel = relative(dirname(p), target);
if (!rel.startsWith(".")) rel = "./" + rel;
return `from "${rel}"`;
@@ -25,6 +25,8 @@ import {
registerMessageCapture,
setEventBroadcaster as setMessageCaptureEventBroadcaster,
} from "../modules/message-capture/messageCapture.js";
import { setModerationEventBroadcaster } from "../modules/message-capture/moderationActionsDb.js";
import { startDigestScheduler } from "../modules/monitor/digestScheduler.js";
import { registerReactionCapture } from "../modules/reaction-tracking/index.js";
import { registerThreadCapture } from "../modules/thread-tracking/index.js";
import { registerPresenceCapture } from "../modules/user-presence/index.js";
@@ -254,6 +256,7 @@ export async function initializeDiscordGateway() {
logger.info({ user: client.user?.tag }, "Bot logged in");
setMessageCaptureEventBroadcaster(eventBroadcaster);
setRecorderEventBroadcaster(eventBroadcaster);
setModerationEventBroadcaster(eventBroadcaster);
registerMessageCapture(client);
startPendingAIAnalysisWorker(client, eventBroadcaster);
@@ -273,6 +276,8 @@ export async function initializeDiscordGateway() {
// Start retention cleanup scheduler
startRetentionCleanup();
// Start weekly moderation digest (public, automated)
startDigestScheduler();
});
client.on("error", (err) => {
@@ -128,30 +128,6 @@ export async function skipAgeRestrictedMessages(
// Batch pipeline
// ---------------------------------------------------------------------------
async function postBatchReputationUpdate(rows: MessageRecord[]): Promise<void> {
for (const row of rows) {
if (row.ai_status === "clean") {
import("./userReputationStore.js")
.then((store) => store.recordCleanMessage(row.user_id, row.guild_id))
.catch((e) =>
logger.error({ error: e }, "Failed to record clean message streak"),
);
} else if (row.ai_status === "flagged" && row.ai_severity !== "none") {
import("./userReputationStore.js")
.then((store) =>
store.recordInfraction(
row.user_id,
row.guild_id,
row.ai_severity as "low" | "medium" | "high" | "critical",
),
)
.catch((e) =>
logger.error({ error: e }, "Failed to record infraction penalty"),
);
}
}
}
export async function processBatch(
conversationKey: string,
messages: MessageRecord[],
@@ -196,21 +172,6 @@ export async function processBatch(
}
}
// Post-batch reputation updates (fire-and-forget)
postBatchReputationUpdate(
result.rows.filter((r) => {
if (r.ai_status === "error") {
try {
const flags = JSON.parse(r.ai_moderation_flags ?? "[]") as string[];
return !flags.includes("analysis_api_failed");
} catch {
return false;
}
}
return true;
}),
);
if (!result.ok) {
recordConversationBatchFailure(conversationKey);
@@ -123,33 +123,6 @@ async function processIndividualFallback(
for (const row of rows) {
broadcastAnalysisCompleted(row);
scheduleAutoDelete(row);
// Update reputation autonomously
if (row.ai_status === "clean") {
import("./userReputationStore.js")
.then((store) => store.recordCleanMessage(row.user_id, row.guild_id))
.catch((e) =>
logger.error(
{ error: e },
"Failed to record clean message streak in fallback",
),
);
} else if (row.ai_status === "flagged" && row.ai_severity !== "none") {
import("./userReputationStore.js")
.then((store) =>
store.recordInfraction(
row.user_id,
row.guild_id,
row.ai_severity as "low" | "medium" | "high" | "critical",
),
)
.catch((e) =>
logger.error(
{ error: e },
"Failed to record infraction penalty in fallback",
),
);
}
}
const resultSummary = analysisResult.results[0];
@@ -127,56 +127,10 @@ export function buildUserProfileRef(userId: string): string {
}
// ---------------------------------------------------------------------------
// User reputation — richer than a bare trust score.
//
// The trust model tracks total_infractions, a clean-message streak and the
// last infraction timestamp. Feeding all of it to the LLM lets it tell a
// first-timer (same score, 1 infraction) from a repeat offender (score 50,
// 3 infractions, last one yesterday) — the same score means very different
// things in those two contexts.
// Per-user history context (last flagged messages only — no trust model).
// context to AI moderation.
// ---------------------------------------------------------------------------
export interface ReputationAttrsSource {
trust_score: number;
total_infractions: number;
clean_message_streak: number;
last_infraction_at: number | null;
}
const DAY_MS = 24 * 60 * 60 * 1000;
const REPEAT_OFFENSE_WINDOW_MS = 7 * DAY_MS;
/**
* Formats reputation fields into XML attributes for `<user_reputation .../>`.
* Derived signals: last_offense_days_ago (0 = today) and repeat_offender
* (infraction within the last 7 days) are computed here so both the text and
* media paths emit the exact same shape.
*/
export function formatReputationAttrs(
rep: ReputationAttrsSource,
now: number = Date.now(),
): string {
const attrs = [
`trust_score="${rep.trust_score}"`,
`total_infractions="${rep.total_infractions}"`,
`clean_streak="${rep.clean_message_streak}"`,
];
if (
typeof rep.last_infraction_at === "number" &&
rep.last_infraction_at > 0
) {
const daysAgo = Math.max(
0,
Math.floor((now - rep.last_infraction_at) / DAY_MS),
);
attrs.push(`last_offense_days_ago="${daysAgo}"`);
const isRepeat =
rep.total_infractions > 0 &&
now - rep.last_infraction_at <= REPEAT_OFFENSE_WINDOW_MS;
if (isRepeat) attrs.push(`repeat_offender="true"`);
}
return attrs.join(" ");
}
/**
* Builds an optional `<user_history>` block (last flagged messages) from
@@ -22,11 +22,11 @@
* the DB as fast read caches, so repeat lookups are effectively free;
* - lookups per batch are bounded (AI_GLOSSARY_MAX_TERMS);
* - live Wikipedia calls are rate-limit aware: concurrency 2 + stagger, retry
* once on empty results, and misses cached for only 1h so a limiter/
* once on empty results, and misses cached for only 1h so a limiter or
* network blip is not treated as a permanent miss;
* - only results that read like actual definitions are accepted (Wikipedia
* preferred; disambiguation/ads/translate-homepages rejected);
* - everything degrades gracefully: no Redis, no SearXNG, no match
* - everything degrades gracefully: no Redis, no Wikipedia API, no match
* the block is simply omitted and moderation proceeds as before.
*/
@@ -258,7 +258,7 @@ export function extractGlossaryTerms(
}
// ---------------------------------------------------------------------------
// Definition lookup (cached: LRU → Redis → SearXNG/Wikipedia)
// ─── Definition lookup (cached: LRU → Redis → Wikipedia) ───────────────────
// ---------------------------------------------------------------------------
export interface TermDefinition {
@@ -1,324 +0,0 @@
import { and, desc, eq } from "drizzle-orm";
import { createChildLogger } from "@/shared/logger/index";
import { getDatabase } from "../../shared/database/drizzle.js";
import {
messagesTable,
type UserReputation,
userReputationsTable,
} from "../../shared/database/schema.js";
const logger = createChildLogger("userReputationStore");
// ---------------------------------------------------------------------------
// Trust model v2 — fair, recoverable, escalation-aware
// ---------------------------------------------------------------------------
//
// Problems with v1 that this fixes:
// 1. Trust practically could NOT rise: +2 per 100 clean messages meant a
// single -15 "high" penalty required 750 clean messages to repay.
// 2. Flat penalties regardless of history: first-timers and repeat
// offenders were punished identically.
// 3. Minor infractions could zero out a user (low=-2 at score 2 → 0),
// which is disproportionate.
//
// v2 model:
// - GAIN: +1 trust per 15 consecutive clean messages (cap 100). Recovery
// is real but earned — consistent good behavior rebuilds trust.
// - PENALTY: severity table low=3 / medium=6 / high=12 / critical=25.
// - FIRST OFFENSE: penalty halved (leniency for a single slip).
// - REPEAT OFFENDER: infraction within the last 7 days → ×1.5 (escalation).
// - FLOOR: low/medium infractions cannot push trust below 10/5 — minor
// offenses never permanently cripple a user; high/critical can still
// zero out (severe behavior has severe consequences).
// - Streak resets on infraction; time-based recovery still happens through
// the clean-message gain (no arbitrary idle-decay).
// ---------------------------------------------------------------------------
export const TRUST_DEFAULTS = {
DEFAULT_TRUST: 50,
MAX_TRUST: 100,
MIN_TRUST: 0,
CLEAN_MESSAGES_PER_POINT: 15,
REPEAT_OFFENSE_WINDOW_MS: 7 * 24 * 60 * 60 * 1000, // 7 days
REPEAT_OFFENSE_MULTIPLIER: 1.5,
} as const;
export const INFRACTION_PENALTIES: Record<
"low" | "medium" | "high" | "critical",
number
> = {
low: 3,
medium: 6,
high: 12,
critical: 25,
};
/** Trust floors per severity — minor offenses can't tank a user to zero. */
export const INFRACTION_FLOORS: Record<
"low" | "medium" | "high" | "critical",
number
> = {
low: 10,
medium: 5,
high: 0,
critical: 0,
};
function clampTrust(score: number): number {
return Math.min(
TRUST_DEFAULTS.MAX_TRUST,
Math.max(TRUST_DEFAULTS.MIN_TRUST, Math.round(score)),
);
}
export interface InfractionContext {
totalInfractions: number;
lastInfractionAt: number | null;
severity: "low" | "medium" | "high" | "critical";
now?: number;
}
export interface InfractionOutcome {
penalty: number;
appliedRules: {
firstOffense: boolean;
repeatEscalation: boolean;
};
}
/**
* Pure penalty computation for the trust model (unit-testable, no DB).
* - First offense ever halved (leniency for a single slip).
* - Repeat offense within the 7-day window ×1.5 (escalation).
*/
export function computeInfractionPenalty(
ctx: InfractionContext,
): InfractionOutcome {
const basePenalty = INFRACTION_PENALTIES[ctx.severity];
let penalty = basePenalty;
const isFirstOffense = ctx.totalInfractions === 0;
if (isFirstOffense) {
penalty = Math.ceil(basePenalty / 2);
} else if (
ctx.lastInfractionAt &&
(ctx.now ?? Date.now()) - ctx.lastInfractionAt <=
TRUST_DEFAULTS.REPEAT_OFFENSE_WINDOW_MS
) {
penalty = Math.ceil(basePenalty * TRUST_DEFAULTS.REPEAT_OFFENSE_MULTIPLIER);
}
return {
penalty,
appliedRules: {
firstOffense: isFirstOffense,
repeatEscalation: !isFirstOffense && penalty > basePenalty,
},
};
}
export interface CleanGainOutcome {
newStreak: number;
trustGain: number;
}
/**
* Pure clean-message gain computation (unit-testable, no DB).
* +1 trust every CLEAN_MESSAGES_PER_POINT consecutive clean messages;
* the streak keeps counting past the threshold (gains compound).
*/
export function computeCleanTrustGain(currentStreak: number): CleanGainOutcome {
const newStreak = currentStreak + 1;
const trustGain =
newStreak % TRUST_DEFAULTS.CLEAN_MESSAGES_PER_POINT === 0 ? 1 : 0;
return { newStreak, trustGain };
}
/**
* Ensures a user reputation record exists.
*/
export async function initializeUserReputation(
userId: string,
guildId: string,
): Promise<UserReputation> {
const db = getDatabase();
const existing = await db
.select()
.from(userReputationsTable)
.where(eq(userReputationsTable.user_id, userId))
.limit(1);
if (existing.length > 0) {
logger.debug({ userId }, "Reputation record already exists");
return existing[0];
}
const [inserted] = await db
.insert(userReputationsTable)
.values({
user_id: userId,
guild_id: guildId,
trust_score: TRUST_DEFAULTS.DEFAULT_TRUST,
clean_message_streak: 0,
total_infractions: 0,
created_at: Date.now(),
updated_at: Date.now(),
})
.onConflictDoNothing()
.returning();
if (!inserted) {
// If concurrent insert happened
logger.debug({ userId }, "Concurrent reputation insert detected, retrying");
const retry = await db
.select()
.from(userReputationsTable)
.where(eq(userReputationsTable.user_id, userId))
.limit(1);
return retry[0];
}
logger.debug(
{ userId, trustScore: inserted.trust_score },
"Initialized user reputation",
);
return inserted;
}
/**
* Fetch a user's reputation score. Returns default 50 if none exists.
*/
export async function getUserReputation(
userId: string,
): Promise<UserReputation | null> {
const db = getDatabase();
const existing = await db
.select()
.from(userReputationsTable)
.where(eq(userReputationsTable.user_id, userId))
.limit(1);
if (existing[0]) {
logger.debug(
{ userId, trustScore: existing[0].trust_score },
"Fetched user reputation",
);
} else {
logger.debug({ userId }, "No reputation record found, returning null");
}
return existing[0] || null;
}
/**
* Increment the clean message streak and grow trust +1 per
* CLEAN_MESSAGES_PER_POINT consecutive clean messages (cap 100). The streak
* keeps counting past the threshold so gains compound with continued good
* behavior (no more wasted progress at 100, and recovery is genuinely
* reachable after an infraction).
*/
export async function recordCleanMessage(
userId: string,
guildId: string,
): Promise<void> {
const rep = await initializeUserReputation(userId, guildId);
const db = getDatabase();
const { newStreak, trustGain } = computeCleanTrustGain(
rep.clean_message_streak,
);
const newScore =
trustGain > 0 ? clampTrust(rep.trust_score + trustGain) : rep.trust_score;
await db
.update(userReputationsTable)
.set({
clean_message_streak: newStreak,
trust_score: newScore,
updated_at: Date.now(),
})
.where(eq(userReputationsTable.user_id, userId));
logger.debug(
{ userId, previousScore: rep.trust_score, newScore, newStreak },
"Clean message recorded, reputation updated",
);
}
/**
* Apply an infraction penalty to a user.
*
* Fairness rules:
* - First offense ever penalty halved (leniency, rounded up).
* - Repeat offense within the 7-day window ×1.5 (escalation).
* - Severity floor prevents minor infractions from zeroing a user.
* - Streak resets trust must be re-earned through clean behavior.
*/
export async function recordInfraction(
userId: string,
guildId: string,
severity: "low" | "medium" | "high" | "critical",
): Promise<void> {
const rep = await initializeUserReputation(userId, guildId);
const db = getDatabase();
const outcome = computeInfractionPenalty({
totalInfractions: rep.total_infractions,
lastInfractionAt: rep.last_infraction_at,
severity,
});
const { penalty } = outcome;
const floor = INFRACTION_FLOORS[severity];
const newScore = Math.max(floor, clampTrust(rep.trust_score - penalty));
await db
.update(userReputationsTable)
.set({
trust_score: newScore,
clean_message_streak: 0, // Reset streak on infraction
total_infractions: rep.total_infractions + 1,
last_infraction_at: Date.now(),
updated_at: Date.now(),
})
.where(eq(userReputationsTable.user_id, userId));
logger.info(
{
userId,
severity,
basePenalty: INFRACTION_PENALTIES[severity],
penalty,
appliedRules: outcome.appliedRules,
previousScore: rep.trust_score,
newScore,
floor,
totalInfractions: rep.total_infractions + 1,
},
"Infraction recorded",
);
}
/**
* Fetch a user's past N flagged messages for context injection.
*/
export async function getUserRecentInfractions(
userId: string,
limit: number = 3,
) {
const db = getDatabase();
return await db
.select({
content: messagesTable.content,
flags: messagesTable.ai_moderation_flags,
severity: messagesTable.ai_severity,
created_at: messagesTable.created_at,
})
.from(messagesTable)
.where(
and(
eq(messagesTable.user_id, userId),
eq(messagesTable.ai_status, "flagged"),
),
)
.orderBy(desc(messagesTable.created_at))
.limit(limit);
}
@@ -293,6 +293,16 @@ export class EventBroadcaster {
});
}
async moderationAction(data: Record<string, unknown>): Promise<void> {
this.logger.debug({ data }, "Publishing moderation_action");
await this.publisher.publish(EventChannels.MODERATION_ACTION, {
type: "moderation_action",
data,
timestamp: Date.now(),
source: "discord-gateway",
});
}
async analysisQueueStatus(data: Record<string, unknown>): Promise<void> {
this.logger.debug({ data }, "Publishing analysis_queue_status");
await this.publisher.publish(EventChannels.ANALYSIS_QUEUE_STATUS, {
@@ -9,6 +9,7 @@ import {
DISCORD_MESSAGE_CREATED,
DISCORD_MESSAGE_DELETED,
DISCORD_MESSAGE_UPDATED,
DISCORD_MODERATION_ACTION,
DISCORD_PRESENCE_UPDATED,
DISCORD_REACTION_ADDED,
DISCORD_REACTION_REMOVED,
@@ -50,6 +51,7 @@ export const EventChannels = {
GUILD_MEMBER_ADDED: DISCORD_GUILD_MEMBER_ADDED,
GUILD_MEMBER_REMOVED: DISCORD_GUILD_MEMBER_REMOVED,
VOICE_ANALYZED: DISCORD_VOICE_ANALYZED,
MODERATION_ACTION: DISCORD_MODERATION_ACTION,
} as const;
export type EventChannelType =
@@ -1,11 +1,11 @@
import { embedText } from "@/modules/ai-moderation/embeddingClient.js";
import { embedText } from "@/modules/ai-moderation/embeddingClient";
import {
ARCHIVE_COLLECTION,
qdrantPointId,
upsertQdrantPointV2,
} from "@/modules/ai-moderation/qdrantClient.js";
import { config } from "@/shared/config/config.js";
} from "@/modules/ai-moderation/qdrantClient";
import { createChildLogger } from "@/shared/logger/index";
import { config } from "../../shared/config/config.js";
const log = createChildLogger("archive-embedder");
@@ -4,8 +4,16 @@ import type * as schema from "../../shared/database/schema.js";
import { moderationActionsTable } from "../../shared/database/schema.js";
import { buildCursorCondition, pageResult } from "../../shared/index.js";
import { createChildLogger, type Logger } from "../../shared/logger/index.js";
import type { EventBroadcaster } from "../event-broadcaster/eventBroadcaster.js";
import type { ModerationAction, PageResult } from "../message-capture/types.js";
let _eventBroadcaster: EventBroadcaster | null = null;
/** Inject the gateway's event broadcaster so actions can be published live. */
export function setModerationEventBroadcaster(eb: EventBroadcaster): void {
_eventBroadcaster = eb;
}
// ─── ModerationActionsDb Class ──────────────────────────────────────────────
export class ModerationActionsDb {
@@ -38,7 +46,16 @@ export class ModerationActionsDb {
})
.returning();
return rows[0] as ModerationAction;
const created = rows[0] as ModerationAction;
// Fire-and-forget live broadcast (backend WS → frontend feed).
if (_eventBroadcaster) {
_eventBroadcaster
.moderationAction(created as unknown as Record<string, unknown>)
.catch(() => {});
}
return created;
} catch (error) {
this.logger.error(
{
@@ -0,0 +1,150 @@
import { sql } from "drizzle-orm";
import { config } from "@/shared/config/index.js";
import { getDatabase } from "@/shared/database/drizzle.js";
import { createChildLogger } from "@/shared/logger/index";
const logger = createChildLogger("digest-scheduler");
// 7 days
const WEEK_MS = 7 * 24 * 60 * 60 * 1000;
let lastDigestTs: number = 0;
/**
* Weekly moderation digest for the public monitor channel (or webhook).
* Fully automatic no UI, no shadow mode. Queries the DB directly and posts
* a compact embed to WEBHOOK_URLS. Read-only: never mutates moderation state.
*/
export async function runWeeklyDigest(now = Date.now()): Promise<void> {
// Run at most once per week (guard against double-scheduling on restart).
if (now - lastDigestTs < WEEK_MS) return;
lastDigestTs = now;
if (!config.WEBHOOK_URLS.length) {
logger.warn("No WEBHOOK_URLS configured — skipping weekly digest");
return;
}
const db = getDatabase();
const since = now - WEEK_MS;
try {
const [trends, domains, channels, coverage] = await Promise.all([
// Top categories
db.execute(sql`
SELECT jsonb_array_elements_text(categories)::text AS name,
COUNT(*)::int AS c
FROM moderation_actions
WHERE created_at >= ${since} AND categories IS NOT NULL
GROUP BY name ORDER BY c DESC LIMIT 5
`),
// Top flagged domains
db.execute(sql`
SELECT host, COUNT(*)::int AS c
FROM (
SELECT DISTINCT id,
(regexp_matches(COALESCE(content,'') || ' ' || COALESCE(reason,'') || ' ' || COALESCE(evidence,''), 'https?://([^/\\s?#]+)', 'g'))[1] AS host
FROM moderation_actions
WHERE created_at >= ${since}
AND (content IS NOT NULL OR reason IS NOT NULL OR evidence IS NOT NULL)
) sub
WHERE host IS NOT NULL
GROUP BY host ORDER BY c DESC LIMIT 5
`),
// Top flagged channels
db.execute(sql`
SELECT COALESCE(NULLIF((m.metadata::jsonb -> 'channel' ->> 'channelName'), ''), m.channel_id) AS channel_name,
COUNT(*)::int AS c
FROM moderation_actions a
LEFT JOIN messages m ON m.id = a.message_id
WHERE a.created_at >= ${since} AND m.channel_id IS NOT NULL
GROUP BY channel_name ORDER BY c DESC LIMIT 5
`),
// Coverage
db.execute(sql`
SELECT status, COUNT(*)::int AS c
FROM ai_analysis_runs
WHERE created_at >= ${since}
GROUP BY status
`),
]);
const topCats = (trends.rows as Record<string, unknown>[]).map(
(r) => `${r.name} (${r.c})`,
);
const topDomains = (domains.rows as Record<string, unknown>[]).map(
(r) => `${r.host} (${r.c})`,
);
const topChannels = (channels.rows as Record<string, unknown>[]).map(
(r) => `${r.channel_name} (${r.c})`,
);
const cov = (coverage.rows as Record<string, unknown>[]) || [];
const total = cov.reduce((s, r) => s + Number(r.c), 0);
const completed = Number(cov.find((r) => r.status === "completed")?.c ?? 0);
const covRate = total > 0 ? ((completed / total) * 100).toFixed(1) : "0";
const lines: string[] = [];
lines.push(`**GMW Weekly Moderation Digest** (last 7 days)`);
lines.push("");
lines.push(
`Auto-mod coverage: ${covRate}% (${completed}/${total} runs completed)`,
);
lines.push(
`**Top flagged categories:** ${topCats.length ? topCats.join(", ") : "—"}`,
);
lines.push(
`**Top flagged domains:** ${topDomains.length ? topDomains.join(", ") : "—"}`,
);
lines.push(
`**Top flagged channels:** ${topChannels.length ? topChannels.join(", ") : "—"}`,
);
lines.push("");
lines.push(
"_View full breakdowns at the moderation dashboard (public, read-only)._",
);
const body = JSON.stringify({
username: "GMW Digest",
avatar_url:
"https://upload.wikimedia.org/wikipedia/commons/6/6a/Orange_tabby_cat_sitting_on_fallen_leaves-Hisashi-01A.jpg",
content: null,
embeds: [
{
title: "GMW Weekly Moderation Digest",
description: lines.join("\n"),
color: 0x38bdf8,
timestamp: new Date().toISOString(),
},
],
});
for (const url of config.WEBHOOK_URLS) {
try {
const res = await fetch(url, {
method: "POST",
headers: { "Content-Type": "application/json" },
body,
});
if (!res.ok) {
logger.warn({ url, status: res.status }, "Digest webhook failed");
}
} catch (err) {
logger.warn({ err }, "Digest webhook threw");
}
}
logger.info("Weekly digest posted");
} catch (err) {
logger.error({ err }, "Weekly digest failed");
}
}
/** Schedule the weekly digest. Runs on an interval; the guard inside
* `runWeeklyDigest` ensures it only fires once per WEEK_MS.
*/
export function startDigestScheduler(intervalMs = 60 * 60 * 1000): void {
// Fire an immediate (guarded) digest on start, then tick hourly.
void runWeeklyDigest();
setInterval(() => {
void runWeeklyDigest();
}, intervalMs);
}
@@ -1,7 +1,7 @@
import type { Readable } from "node:stream";
import type { StreamType } from "@discordjs/voice";
export type MediaMode = "music";
export type MediaMode = "music" | "screenshare";
export type MediaSourceKind =
| "url"
| "local"
@@ -18,6 +18,7 @@ import {
type RecordingSession,
} from "./recorder/sessionRecording.js";
import { createSpeakingHandler } from "./recorder/speakingHandler.js";
import { hookScreenShareAudio } from "./screenShareAudio.js";
const logger = createChildLogger("recorder");
@@ -146,6 +147,13 @@ export async function startRecording(
receiver.speaking.on("start", speakingHandler);
// ── Screen-share audio capture ──────────────────────────────────────
// Discord GoLive sends screen-share audio on a SEPARATE SSRC from the
// user's microphone. `receiver.speaking` only fires for voice (mic) SSRCs,
// so screen-share audio is silently dropped unless we hook the UDP receiver
// to discover and register those SSRCs.
hookScreenShareAudio(receiver, speakingHandler);
// Handle unexpected disconnection
connection.on(VoiceConnectionStatus.Disconnected, async () => {
if (config.VERBOSE) {
@@ -0,0 +1,222 @@
import type { VoiceReceiver, VoiceUserData } from "@discordjs/voice";
import { createChildLogger } from "@/shared/logger/index";
const logger = createChildLogger("screen-share-audio");
/**
* Hooks screen-share audio capture into the voice receiver.
*
* ## Background
*
* Discord GoLive (screen share) sends audio on **separate SSRCs** from the
* user's microphone. In `@discordjs/voice` v0.19, `VoiceReceiver.onUdpMessage`
* does:
*
* ```js
* const userData = this.ssrcMap.get(ssrc);
* if (!userData) return; // ← DROPS screen-share audio SSRC
* ```
*
* `ssrcMap` is only populated from `VOICE_STATE_UPDATE` / `VOICE_SERVER_UPDATE`
* WebSocket packets, which carry the **voice audioSSRC** only. When a user
* starts an audio+video screen-share, Discord sends additional RTP packets on
* new SSRCs that are *never* registered in `ssrcMap` they are silently
* discarded `receiver.speaking` never fires screen-share audio is missing.
*
* ## Fix
*
* 1. Wrap `onUdpMessage` to inspect every incoming RTP packet's SSRC.
* 2. If the SSRC isn't in `ssrcMap`, check whether it looks like a screen-share
* audio stream (OPRUS payload type 120, RTP version 2).
* 3. Clone the owning user's VoiceUserData into `ssrcMap` under the new SSRC
* so the *original* (un-patched) `onUdpMessage` picks it up, decrypts it,
* and forwards the Opus packet to the existing subscription stream.
* 4. Emit a synthetic `"start"` speaking event so the existing
* `speakingHandler` sets up the full pipeline (decoder, packet filter,
* segment manager, event handlers) for that userId if not already.
*/
export function hookScreenShareAudio(
receiver: VoiceReceiver,
speakingHandler: (userId: string) => Promise<void>,
): void {
// Cache the original (un-bound) method so we can delegate to it.
const original = receiver.onUdpMessage;
receiver.onUdpMessage = (msg: Buffer) => {
// ── 1. Detect screen-share SSRCs BEFORE the original discards them ──
if (isLikelyScreenShareAudio(msg, receiver)) {
const ssrc = msg.readUInt32BE(8);
const userData = getSsrcMapEntry(receiver, ssrc);
if (!userData) {
// SSRC not registered — try to infer owner and register it
const owner = inferScreenShareOwner(ssrc, receiver);
if (owner) {
registerScreenShareSsrc(receiver, ssrc, owner);
logger.info(
{ userId: owner.userId, ssrc, kind: "screenshare-audio" },
"Registered screen-share audio SSRC in ssrcMap",
);
// Trigger the speaking handler to ensure pipeline is ready
void speakingHandler(owner.userId).catch((err) =>
logger.error(
{ userId: owner.userId, error: err.message },
"Speaking handler for screen-share failed",
),
);
} else {
logger.warn(
{ ssrc },
"Screen-share audio SSRC found but owner unknown",
);
}
}
}
// ── 2. Delegate to the original handler ──
// It will now find the SSRC (we registered it above) and forward the
// decrypted Opus packet to the subscription stream.
original.call(receiver, msg);
};
// ── 3. Listen for dynamic ssrcMap updates ──────────────────────────────
receiver.ssrcMap.on("create", (data: VoiceUserData) => {
if (data.videoSSRC !== undefined) {
logger.info(
{ userId: data.userId, videoSSRC: data.videoSSRC },
"Screen-share video started",
);
void speakingHandler(data.userId).catch((err) =>
logger.error(
{ userId: data.userId, error: err.message },
"Handler for screen-share start failed",
),
);
}
});
receiver.ssrcMap.on(
"update",
(_old: VoiceUserData | undefined, neu: VoiceUserData) => {
if (_old?.videoSSRC !== neu.videoSSRC && neu.videoSSRC !== undefined) {
logger.info(
{ userId: neu.userId, videoSSRC: neu.videoSSRC },
"Screen-share video SSRC appeared",
);
void speakingHandler(neu.userId).catch((err) =>
logger.error(
{ userId: neu.userId, error: err.message },
"Handler for screen-share update failed",
),
);
}
},
);
}
// ─── Helpers ───────────────────────────────────────────────────────────────
/**
* Check if a UDP packet looks like a screen-share audio RTP packet.
*
* Voice packets have: RTP version 2 (top 2 bits of byte 0), and payload type 120 (OPRUS).
* We also require that the SSRC is NOT already in ssrcMap (that's handled
* by the original onUdpMessage).
*/
function isLikelyScreenShareAudio(
msg: Buffer,
receiver: VoiceReceiver,
): boolean {
if (msg.length <= 8) return false;
const ssrc = msg.readUInt32BE(8);
// Already registered as a known voice SSRC?
if (getSsrcMapEntry(receiver, ssrc)) return false;
const rtpVersion = msg[0] >> 6;
const payloadType = msg[1] & 127;
// OPRUS payload type is 120 in Discord voice
return rtpVersion === 2 && payloadType === 120;
}
/**
* Safely read an entry from the SSRCMap via the public `get` API.
*/
function getSsrcMapEntry(
receiver: VoiceReceiver,
ssrc: number,
): VoiceUserData | undefined {
try {
return receiver.ssrcMap.get(ssrc);
} catch {
return undefined;
}
}
/**
* Infer which user owns a screen-share audio SSRC by proximity to their
* known voice audioSSRC (Discord allocates SSRCs in small increments).
*/
function inferScreenShareOwner(
ssrc: number,
receiver: VoiceReceiver,
): { userId: string } | null {
try {
// Iterate known SSRCs via internal _map (the public API only gets by ssrc)
const map = getSsrcInternalMap(receiver.ssrcMap);
if (!map) return null;
for (const [, data] of map.entries()) {
if (data.audioSSRC && Math.abs(data.audioSSRC - ssrc) < 200_000) {
return { userId: data.userId };
}
}
return null;
} catch {
return null;
}
}
/**
* Register a new SSRC in the internal ssrcMap so the original onUdpMessage
* picks it up. We clone the user's existing VoiceUserData (so decryption
* keys, userId mapping, etc. all work) under the new SSRC key.
*/
function registerScreenShareSsrc(
receiver: VoiceReceiver,
ssrc: number,
owner: { userId: string },
): void {
const map = getSsrcInternalMap(receiver.ssrcMap);
if (!map) return;
// Find the owner's existing VoiceUserData and clone it under the new SSRC
for (const [, data] of map.entries()) {
if (data.userId === owner.userId) {
map.set(ssrc, { ...data });
return;
}
}
// Fallback: register with minimal data (userId only)
map.set(ssrc, {
userId: owner.userId,
audioSSRC: ssrc,
});
}
/**
* Access the private `_map` inside an SSRCMap instance.
*
* SSRCMap in @discordjs/voice v0.19 stores its entries in a private
* `#map` (JS private field) or `_map` depending on the build target.
* We access it defensively for read + write so we can register new SSRCs.
*/
function getSsrcInternalMap(
ssrcMap: VoiceReceiver["ssrcMap"],
): Map<number, VoiceUserData> | undefined {
const asAny = ssrcMap as unknown as Record<string, unknown>;
// v0.19 ESM build uses _map
const m1 = asAny._map;
if (m1 instanceof Map) return m1 as Map<number, VoiceUserData>;
return undefined;
}
@@ -34,7 +34,7 @@ export const configSchema = z
.describe("Thread IDs to exclude from capture"),
BOT_EXCLUDED_CHANNEL_IDS: z
.string()
.default("1206269771340058694")
.default("1206269771340058694,1318544753821880362")
.transform((v) => v.split(",").filter(Boolean))
.describe(
"Channel IDs where bot messages are NOT captured/analyzed (bot detection stays on everywhere else)",
@@ -222,7 +222,8 @@ export const configSchema = z
AI_GLOSSARY_MAX_TERMS: z.coerce.number().int().min(1).max(20).default(6),
// Per-user personal profile summaries (userProfileLearner). Disabled by
// default: profiles bloat the analysis context and add LLM/DB cost for
// little moderation signal — only <user_reputation> history is injected.
// little moderation signal — user history context (last flagged messages)
// is injected via <user_history> instead of a numeric trust score.
AI_USER_PROFILE_LEARNING_ENABLED: z
.string()
.optional()
@@ -337,32 +337,6 @@ export const pgUserProfilesTable = pgTable(
export const userProfilesTable = pgUserProfilesTable;
/**
* User Reputations Table (PostgreSQL)
* Tracks user trust score and infractions to provide context to AI.
*/
export const pgUserReputationsTable = pgTable(
"user_reputations",
{
user_id: pgText("user_id").primaryKey(),
guild_id: pgText("guild_id").notNull(),
trust_score: pgInteger("trust_score").notNull().default(50),
clean_message_streak: pgInteger("clean_message_streak")
.notNull()
.default(0),
total_infractions: pgInteger("total_infractions").notNull().default(0),
last_infraction_at: pgBigint("last_infraction_at", { mode: "number" }),
created_at: pgBigint("created_at", { mode: "number" }).notNull(),
updated_at: pgBigint("updated_at", { mode: "number" }).notNull(),
},
(table) => ({
guildIdx: pgIndex("idx_user_reputations_guild_id").on(table.guild_id),
scoreIdx: pgIndex("idx_user_reputations_trust_score").on(table.trust_score),
}),
);
export const userReputationsTable = pgUserReputationsTable;
/**
* Channel Cultures Table (PostgreSQL)
* Stores AI-generated summaries of channel norms and slang to inject as context.
@@ -592,10 +566,6 @@ export type AIAnalysisRunInsert = typeof aiAnalysisRunsTable.$inferInsert;
export type UserProfile = typeof userProfilesTable.$inferSelect;
export type UserProfileInsert = typeof userProfilesTable.$inferInsert;
// User Reputations
export type UserReputation = typeof userReputationsTable.$inferSelect;
export type UserReputationInsert = typeof userReputationsTable.$inferInsert;
// Channel Cultures
export type ChannelCulture = typeof channelCulturesTable.$inferSelect;
export type ChannelCultureInsert = typeof channelCulturesTable.$inferInsert;
@@ -2,26 +2,17 @@ import {
pgAIAnalysisRunsTable,
pgChannelCulturesTable,
pgUserProfilesTable,
pgUserReputationsTable,
} from "../../../shared/index.js";
// Re-export shared tables
export {
pgAIAnalysisRunsTable,
pgChannelCulturesTable,
pgUserProfilesTable,
pgUserReputationsTable,
};
export { pgAIAnalysisRunsTable, pgChannelCulturesTable, pgUserProfilesTable };
export const aiAnalysisRunsTable = pgAIAnalysisRunsTable;
export const channelCulturesTable = pgChannelCulturesTable;
export const userProfilesTable = pgUserProfilesTable;
export const userReputationsTable = pgUserReputationsTable;
// Types
export type AIAnalysisRun = typeof aiAnalysisRunsTable.$inferSelect;
export type AIAnalysisRunInsert = typeof aiAnalysisRunsTable.$inferInsert;
export type UserReputation = typeof userReputationsTable.$inferSelect;
export type UserReputationInsert = typeof userReputationsTable.$inferInsert;
export type ChannelCulture = typeof channelCulturesTable.$inferSelect;
export type ChannelCultureInsert = typeof channelCulturesTable.$inferInsert;
export type UserProfile = typeof userProfilesTable.$inferSelect;
@@ -30,6 +30,7 @@ export const DISCORD_CHANNEL_TOPIC_UPDATED = "discord:channel:topic_updated";
export const DISCORD_PRESENCE_UPDATED = "discord:presence:updated";
export const DISCORD_GUILD_MEMBER_ADDED = "discord:guild_member:added";
export const DISCORD_GUILD_MEMBER_REMOVED = "discord:guild_member:removed";
export const DISCORD_MODERATION_ACTION = "discord:moderation:action";
// ---------------------------------------------------------------------------
// Command channels (backend -> discord-gateway)
@@ -1,12 +1,11 @@
// ═══════════════════════════════════════════════════════════════════════════
// Context enrichment builders — rich <user_reputation> attrs, <user_history>,
// <user_profiles> as_of, bot/edited detection (pure, no DB)
// Context enrichment builders — <user_history>, <user_profiles> as_of,
// bot/edited detection (pure, no DB)
// ═══════════════════════════════════════════════════════════════════════════
import { describe, expect, it } from "vitest";
import {
buildUserHistoryXml,
buildUserProfilesBlock,
formatReputationAttrs,
resolveIsBot,
resolveIsEdited,
} from "../src/modules/ai-moderation/moderationBuilders.js";
@@ -42,72 +41,6 @@ function msg(overrides: Partial<MessageRecord> = {}): MessageRecord {
const DAY_MS = 24 * 60 * 60 * 1000;
describe("formatReputationAttrs — rich reputation signal", () => {
it("emits trust, infraction count and clean streak", () => {
const attrs = formatReputationAttrs({
trust_score: 62,
total_infractions: 3,
clean_message_streak: 45,
last_infraction_at: null,
});
expect(attrs).toContain('trust_score="62"');
expect(attrs).toContain('total_infractions="3"');
expect(attrs).toContain('clean_streak="45"');
});
it("derives last_offense_days_ago and marks repeat offenders (7-day window)", () => {
const attrs = formatReputationAttrs(
{
trust_score: 50,
total_infractions: 2,
clean_message_streak: 0,
last_infraction_at: NOW - 2 * DAY_MS,
},
NOW,
);
expect(attrs).toContain('last_offense_days_ago="2"');
expect(attrs).toContain('repeat_offender="true"');
});
it("does NOT mark repeat offender when the last offense is older than 7 days", () => {
const attrs = formatReputationAttrs(
{
trust_score: 50,
total_infractions: 2,
clean_message_streak: 10,
last_infraction_at: NOW - 30 * DAY_MS,
},
NOW,
);
expect(attrs).toContain('last_offense_days_ago="30"');
expect(attrs).not.toContain("repeat_offender");
});
it("omits offense-derived attrs when the user has no recorded infraction date", () => {
const attrs = formatReputationAttrs({
trust_score: 85,
total_infractions: 0,
clean_message_streak: 120,
last_infraction_at: null,
});
expect(attrs).not.toContain("last_offense_days_ago");
expect(attrs).not.toContain("repeat_offender");
});
it("clamps a future/skewed timestamp to days_ago=0", () => {
const attrs = formatReputationAttrs(
{
trust_score: 50,
total_infractions: 1,
clean_message_streak: 0,
last_infraction_at: NOW + 5 * DAY_MS,
},
NOW,
);
expect(attrs).toContain('last_offense_days_ago="0"');
});
});
describe("buildUserHistoryXml — last flagged messages for repeat offenders", () => {
it("returns empty when there is no real history", () => {
expect(buildUserHistoryXml([])).toBe("");
@@ -1,93 +0,0 @@
// ═══════════════════════════════════════════════════════════════════════════
// Trust model v2 — pure math tests (no DB required)
// ═══════════════════════════════════════════════════════════════════════════
import { describe, expect, it } from "vitest";
import {
computeCleanTrustGain,
computeInfractionPenalty,
INFRACTION_FLOORS,
INFRACTION_PENALTIES,
TRUST_DEFAULTS,
} from "../src/modules/ai-moderation/userReputationStore.js";
describe("computeCleanTrustGain — trust CAN rise", () => {
it("grants +1 every CLEAN_MESSAGES_PER_POINT clean messages", () => {
const before = computeCleanTrustGain(14);
expect(before.newStreak).toBe(15);
expect(before.trustGain).toBe(1);
const after = computeCleanTrustGain(15);
expect(after.newStreak).toBe(16);
expect(after.trustGain).toBe(0);
});
it("keeps compounding past the threshold (no wasted progress)", () => {
expect(computeCleanTrustGain(29).trustGain).toBe(1);
expect(computeCleanTrustGain(44).trustGain).toBe(1);
// 45 clean messages from a fresh start → 3 points of recovery
let gain = 0;
let streak = 0;
for (let i = 0; i < 45; i++) {
const r = computeCleanTrustGain(streak);
streak = r.newStreak;
gain += r.trustGain;
}
expect(gain).toBe(3);
});
});
describe("computeInfractionPenalty — fair and escalating", () => {
const NOW = Date.now();
it("applies base penalty for a repeat offender outside the window", () => {
const r = computeInfractionPenalty({
totalInfractions: 3,
lastInfractionAt: NOW - TRUST_DEFAULTS.REPEAT_OFFENSE_WINDOW_MS - 1000,
severity: "medium",
now: NOW,
});
expect(r.penalty).toBe(INFRACTION_PENALTIES.medium); // 6
expect(r.appliedRules.firstOffense).toBe(false);
expect(r.appliedRules.repeatEscalation).toBe(false);
});
it("halves the penalty for a first offense (leniency)", () => {
const r = computeInfractionPenalty({
totalInfractions: 0,
lastInfractionAt: null,
severity: "high",
now: NOW,
});
expect(r.penalty).toBe(Math.ceil(INFRACTION_PENALTIES.high / 2)); // 6
expect(r.appliedRules.firstOffense).toBe(true);
});
it("escalates ×1.5 for a repeat offense within 7 days", () => {
const r = computeInfractionPenalty({
totalInfractions: 2,
lastInfractionAt: NOW - 60 * 60 * 1000, // 1h ago
severity: "medium",
now: NOW,
});
expect(r.penalty).toBe(Math.ceil(INFRACTION_PENALTIES.medium * 1.5)); // 9
expect(r.appliedRules.repeatEscalation).toBe(true);
});
it("critical first offense still hurts but is halved", () => {
const r = computeInfractionPenalty({
totalInfractions: 0,
lastInfractionAt: null,
severity: "critical",
now: NOW,
});
expect(r.penalty).toBe(Math.ceil(INFRACTION_PENALTIES.critical / 2)); // 13
});
it("severity floors prevent minor offenses from zeroing a user", () => {
expect(INFRACTION_FLOORS.low).toBeGreaterThan(0);
expect(INFRACTION_FLOORS.medium).toBeGreaterThan(0);
// high/critical can still reach zero — severe behavior has consequences
expect(INFRACTION_FLOORS.high).toBe(0);
expect(INFRACTION_FLOORS.critical).toBe(0);
});
});
@@ -0,0 +1,20 @@
import { PageTransition } from "@/components/shared";
import { getChannelCultures } from "@/lib/api/server";
import type { ChannelCultureRow } from "@/lib/types";
import { ChannelsView } from "./view";
export const dynamic = "force-dynamic";
export default async function ChannelsPage() {
let cultures: ChannelCultureRow[] | undefined;
try {
cultures = await getChannelCultures(100);
} catch {
cultures = undefined;
}
return (
<PageTransition>
<ChannelsView initialCultures={cultures} />
</PageTransition>
);
}
@@ -0,0 +1,23 @@
"use client";
import { ChannelCultureGlossary } from "@/components/ChannelCultureGlossary";
import { SkeletonPanel } from "@/components/shared";
import { useChannelCultures } from "@/hooks";
import type { ChannelCultureRow } from "@/lib/types";
export function ChannelsView({
initialCultures,
}: {
initialCultures?: ChannelCultureRow[];
}) {
const { data: cultures } = useChannelCultures(100, initialCultures);
return (
<div className="space-y-5">
{cultures ? (
<ChannelCultureGlossary cultures={cultures} />
) : (
<SkeletonPanel rows={6} />
)}
</div>
);
}
@@ -0,0 +1,20 @@
import { PageTransition } from "@/components/shared";
import { getGlossary } from "@/lib/api/server";
import type { GlossaryRow } from "@/lib/types";
import { GlossaryView } from "./view";
export const dynamic = "force-dynamic";
export default async function GlossaryPage() {
let terms: GlossaryRow[] | undefined;
try {
terms = await getGlossary(100);
} catch {
terms = undefined;
}
return (
<PageTransition>
<GlossaryView initialTerms={terms} />
</PageTransition>
);
}
@@ -0,0 +1,15 @@
"use client";
import { SkeletonPanel } from "@/components/shared";
import { TermGlossary } from "@/components/TermGlossary";
import { useGlossary } from "@/hooks";
import type { GlossaryRow } from "@/lib/types";
export function GlossaryView({
initialTerms,
}: {
initialTerms?: GlossaryRow[];
}) {
const { data: terms } = useGlossary(100, initialTerms);
return terms ? <TermGlossary terms={terms} /> : <SkeletonPanel rows={6} />;
}
@@ -0,0 +1,59 @@
import { Pencil, Trash2 } from "lucide-react";
import { notFound } from "next/navigation";
import { Badge, Button } from "@/components/primitives";
import { MarkdownLite, PageTransition } from "@/components/shared";
import { getMateriSSR } from "@/lib/api/materi";
export const dynamic = "force-dynamic";
export default async function MateriDetailPage({
params,
}: {
params: Promise<{ id: string }>;
}) {
const { id } = await params;
const doc = await getMateriSSR(id);
if (!doc) {
notFound();
}
return (
<PageTransition>
<article className="prose dark:prose-invert max-w-none">
<div className="flex items-start justify-between mb-6">
<div>
<h1>{doc.title}</h1>
{doc.description && (
<p className="text-muted-foreground">{doc.description}</p>
)}
</div>
<div className="flex gap-2">
<Button variant="ghost" size="sm" asChild>
<a href={`/materi/${doc.id}/edit`}>
<Pencil className="h-4 w-4" />
</a>
</Button>
<Button variant="ghost" size="sm" asChild>
<a href={`/materi/new?duplicate=${doc.id}`}>
<Trash2 className="h-4 w-4" />
</a>
</Button>
</div>
</div>
<div className="mb-6 flex flex-wrap gap-2">
<Badge tone="neutral">{doc.category}</Badge>
{doc.tags.map((tag) => (
<Badge key={tag} tone="neutral">
{tag}
</Badge>
))}
</div>
{/* MarkdownLite component renders content safely (no dangerouslySetInnerHTML) */}
<MarkdownLite content={doc.content} />
</article>
</PageTransition>
);
}
@@ -0,0 +1,192 @@
"use client";
import { Bot, ExternalLink, Loader2, Send, User } from "lucide-react";
import { useEffect, useRef, useState } from "react";
import { Button, GlassCard, Textarea } from "@/components/primitives";
import { PageTransition } from "@/components/shared";
import { searchMateri } from "@/lib/api/materi";
import type {
MateriRagChatMessage,
MateriRagChatResult,
} from "@/lib/types/materi";
export const dynamic = "force-dynamic";
export default function MateriChatPage() {
const [messages, setMessages] = useState<MateriRagChatMessage[]>([]);
const [input, setInput] = useState("");
const [isLoading, setIsLoading] = useState(false);
const [sources, setSources] = useState<MateriRagChatResult["sources"]>([]);
const messagesEndRef = useRef<HTMLDivElement>(null);
useEffect(() => {
messagesEndRef.current?.scrollIntoView({ behavior: "smooth" });
}, []);
async function handleSend() {
if (!input.trim() || isLoading) return;
const userMsg: MateriRagChatMessage = {
role: "user",
content: input.trim(),
};
const newMessages = [...messages, userMsg];
setMessages(newMessages);
setInput("");
setIsLoading(true);
setSources([]);
try {
const result = await searchMateri(
userMsg.content,
newMessages,
undefined,
);
const assistantMsg: MateriRagChatMessage = {
role: "assistant",
content: result.answer,
};
setMessages([...newMessages, assistantMsg]);
setSources(result.sources);
} catch {
const errorMsg: MateriRagChatMessage = {
role: "assistant",
content: "Maaf, ada kesalahan. Silakan coba lagi.",
};
setMessages([...newMessages, errorMsg]);
} finally {
setIsLoading(false);
}
}
const handleKeyDown = (e: React.KeyboardEvent<HTMLTextAreaElement>) => {
if (e.key === "Enter" && !e.shiftKey) {
e.preventDefault();
handleSend();
}
};
return (
<PageTransition>
<div className="flex flex-col h-[calc(100vh-200px)]">
<div className="mb-4">
<h1 className="text-3xl font-bold">AI Chat Materi &amp; RAG</h1>
<p className="text-muted-foreground mt-1">
Tanya tentang materi komunitas. AI akan mencari referensi dari
dokumen materi dan arsip Discord.
</p>
</div>
<div className="flex-1 overflow-y-auto space-y-4">
{messages.length === 0 ? (
<div className="text-center py-12 text-muted-foreground">
<Bot className="mx-auto h-12 w-12 mb-4 opacity-50" />
<p>Silakan tanyakan sesuatu tentang materi komunitas.</p>
<p className="text-xs mt-2">
Contoh: "Apa itu screenshare audio di GMW?" atau "Cara pakai
voice recording"
</p>
</div>
) : (
messages.map((msg, i) => (
<div
key={i}
className={
"flex gap-3 " +
(msg.role === "user" ? "justify-end" : "justify-start")
}
>
<div
className={
"max-w-[80%] rounded-lg p-4 " +
(msg.role === "user"
? "bg-primary text-primary-foreground"
: "bg-muted/50")
}
>
<div className="flex items-center gap-2 mb-2">
{msg.role === "user" ? (
<User className="h-4 w-4" />
) : (
<Bot className="h-4 w-4" />
)}
<span className="text-xs font-medium">
{msg.role === "user" ? "Anda" : "AI Agent"}
</span>
</div>
<div className="whitespace-pre-wrap text-sm">
{msg.content}
</div>
</div>
</div>
))
)}
{isLoading && (
<div className="flex gap-3 justify-start">
<div className="bg-muted/50 rounded-lg p-4 max-w-[80%]">
<div className="flex items-center gap-2">
<Loader2 className="h-4 w-4 animate-spin" />
<span className="text-sm">
AI sedang mencari di materi...
</span>
</div>
</div>
</div>
)}
<div ref={messagesEndRef} />
</div>
{/* Sources from last AI response */}
{sources.length > 0 && (
<GlassCard className="p-4 mb-4">
<p className="text-xs font-medium text-muted-foreground mb-2">
Sumber:
</p>
<div className="space-y-1">
{sources.map((src, i) => (
<div key={i} className="text-sm">
<span className="font-medium">{src.title}</span>
<span className="text-xs text-muted-foreground">
{" "}
(skor: {src.score.toFixed(2)})
</span>
<p className="text-xs text-muted-foreground line-clamp-2 mt-1">
{src.excerpt}
</p>
</div>
))}
</div>
</GlassCard>
)}
{/* Input */}
<div className="flex gap-2">
<Textarea
value={input}
onChange={(e) => setInput(e.target.value)}
onKeyDown={handleKeyDown}
placeholder="Tanya tentang materi..."
disabled={isLoading}
className="flex-1"
rows={2}
/>
<Button
onClick={handleSend}
disabled={isLoading || !input.trim()}
size="icon"
>
<Send className="h-4 w-4" />
</Button>
</div>
<div className="mt-2 text-xs text-muted-foreground">
<ExternalLink className="h-3 w-3 inline mr-1" />
AI mengacu pada materi dan arsip Discord. Jawaban mungkin tidak 100%
akurat.
</div>
</div>
</PageTransition>
);
}
@@ -0,0 +1,166 @@
"use client";
import { ArrowLeft, Save } from "lucide-react";
import { useRouter } from "next/navigation";
import { useState } from "react";
import { Button, GlassCard, Input, Textarea } from "@/components/primitives";
import { PageTransition } from "@/components/shared";
import { createMateri } from "@/lib/api/materi";
import type { CreateMateriInput } from "@/lib/types/materi";
export const dynamic = "force-dynamic";
export default function MateriNewPage() {
const router = useRouter();
const [form, setForm] = useState<CreateMateriInput>({
title: "",
description: "",
content: "",
category: "general",
tags: [],
isPublic: true,
});
const [tagsInput, setTagsInput] = useState("");
const [saving, setSaving] = useState(false);
const [error, setError] = useState<string | null>(null);
function update<K extends keyof CreateMateriInput>(
key: K,
value: CreateMateriInput[K],
) {
setForm((f) => ({ ...f, [key]: value }));
}
async function handleSubmit() {
if (!form.title.trim() || !form.content.trim()) {
setError("Judul dan konten wajib diisi.");
return;
}
setSaving(true);
setError(null);
try {
const tags = tagsInput
.split(",")
.map((t) => t.trim())
.filter(Boolean);
const doc = await createMateri({ ...form, tags });
router.push(`/materi/${doc.id}`);
} catch (err) {
setError(err instanceof Error ? err.message : "Gagal menyimpan materi.");
setSaving(false);
}
}
return (
<PageTransition>
<div className="max-w-3xl mx-auto space-y-6">
<div className="flex items-center gap-3">
<Button variant="ghost" size="sm" onClick={() => router.back()}>
<ArrowLeft className="h-4 w-4" />
</Button>
<h1 className="text-3xl font-bold">Buat Materi Baru</h1>
</div>
{error && (
<GlassCard className="p-4 border border-red-500/30 text-red-400 text-sm">
{error}
</GlassCard>
)}
<GlassCard className="p-6 space-y-4">
<div>
<label
htmlFor="materi-title"
className="text-sm font-medium mb-1 block"
>
Judul *
</label>
<Input
id="materi-title"
value={form.title}
onChange={(e) => update("title", e.target.value)}
placeholder="Judul materi"
/>
</div>
<div>
<label
htmlFor="materi-description"
className="text-sm font-medium mb-1 block"
>
Deskripsi
</label>
<Input
id="materi-description"
value={form.description ?? ""}
onChange={(e) => update("description", e.target.value)}
placeholder="Deskripsi singkat"
/>
</div>
<div>
<label
htmlFor="materi-category"
className="text-sm font-medium mb-1 block"
>
Kategori
</label>
<Input
id="materi-category"
value={form.category}
onChange={(e) => update("category", e.target.value)}
placeholder="general"
/>
</div>
<div>
<label
htmlFor="materi-tags"
className="text-sm font-medium mb-1 block"
>
Tags (pisahkan dengan koma)
</label>
<Input
id="materi-tags"
value={tagsInput}
onChange={(e) => setTagsInput(e.target.value)}
placeholder="wibu, discord, moderation"
/>
</div>
<div>
<label
htmlFor="materi-content"
className="text-sm font-medium mb-1 block"
>
Konten *
</label>
<Textarea
id="materi-content"
value={form.content}
onChange={(e) => update("content", e.target.value)}
placeholder="Tulis materi di sini..."
rows={12}
/>
</div>
<label className="flex items-center gap-2 text-sm">
<input
type="checkbox"
checked={form.isPublic}
onChange={(e) => update("isPublic", e.target.checked)}
/>
Publik (terlihat semua orang)
</label>
<div className="flex justify-end">
<Button onClick={handleSubmit} disabled={saving}>
<Save className="h-4 w-4 mr-2" />
{saving ? "Menyimpan..." : "Simpan"}
</Button>
</div>
</GlassCard>
</div>
</PageTransition>
);
}
@@ -0,0 +1,113 @@
import { BookOpen, MessageSquare, Plus, Search } from "lucide-react";
import Link from "next/link";
import { Badge, Button, GlassCard, Input } from "@/components/primitives";
import { PageTransition } from "@/components/shared";
import { listMateriSSR } from "@/lib/api/materi";
import type { MateriDocument } from "@/lib/types/materi";
export const dynamic = "force-dynamic";
async function loadMateri(search?: string): Promise<MateriDocument[]> {
try {
return await listMateriSSR(50, search);
} catch {
return [];
}
}
function MateriGrid({ materi }: { materi: MateriDocument[] }) {
if (materi.length === 0) {
return (
<div className="text-center py-12 text-muted-foreground">
<BookOpen className="mx-auto h-12 w-12 mb-4 opacity-50" />
<p>Belum ada materi. Jadilah yang pertama membuat materi!</p>
</div>
);
}
return (
<div className="grid gap-4 md:grid-cols-2 lg:grid-cols-3">
{materi.map((doc) => (
<Link key={doc.id} href={`/materi/${doc.id}`}>
<GlassCard className="h-full cursor-pointer hover:shadow-lg transition-shadow">
<div className="p-6">
<h3 className="font-bold text-lg mb-2 line-clamp-2">
{doc.title}
</h3>
{doc.description && (
<p className="text-sm text-muted-foreground mb-3 line-clamp-3">
{doc.description}
</p>
)}
<div className="flex flex-wrap gap-1 mb-3">
<Badge tone="neutral" className="text-xs">
{doc.category}
</Badge>
{doc.tags.slice(0, 3).map((tag) => (
<Badge key={tag} tone="neutral" className="text-xs">
{tag}
</Badge>
))}
</div>
<div className="flex items-center justify-between text-xs text-muted-foreground">
<span>{doc.view_count} views</span>
<span>{new Date(doc.created_at).toLocaleDateString()}</span>
</div>
</div>
</GlassCard>
</Link>
))}
</div>
);
}
export default async function MateriPage({
searchParams,
}: {
searchParams: Promise<{ search?: string }>;
}) {
const params = await searchParams;
const materi = await loadMateri(params.search);
return (
<PageTransition>
<div className="space-y-6">
<div className="flex items-center justify-between">
<div>
<h1 className="text-3xl font-bold">Materi &amp; Bahan Belajar</h1>
<p className="text-muted-foreground mt-1">
Dokumen, panduan, dan bahan belajar komunitas beserta AI agent RAG
</p>
</div>
<div className="flex gap-2">
<Link href="/materi/chat">
<Button variant="outline" size="sm">
<MessageSquare className="h-4 w-4 mr-2" />
AI Chat
</Button>
</Link>
<Button size="sm" asChild>
<Link href={"/materi/new"}>
<Plus className="h-4 w-4 mr-2" />
Buat Materi
</Link>
</Button>
</div>
</div>
<div className="relative">
<Search className="absolute left-3 top-1/2 -translate-y-1/2 h-4 w-4 text-muted-foreground" />
<Input
type="search"
placeholder="Cari materi..."
className="pl-10"
name="search"
defaultValue={params.search}
/>
</div>
<MateriGrid materi={materi} />
</div>
</PageTransition>
);
}
@@ -1,5 +1,10 @@
import { PageTransition } from "@/components/shared";
import { getConfig, getGuilds, getMessages } from "@/lib/api/server";
import {
getConfig,
getGuilds,
getMessages,
getRecentEdits,
} from "@/lib/api/server";
import { MessagesView } from "./view";
export const dynamic = "force-dynamic";
@@ -11,12 +16,14 @@ export default async function MessagesPage() {
data: import("@/lib/types").MessageRecord[];
nextCursor: string | null;
} | null = null;
let initialEdits: import("@/lib/types").EditHistoryRow[] | undefined;
try {
[config, guilds] = await Promise.all([getConfig(), getGuilds()]);
const gid = config?.monitorGuildId;
if (gid) {
initialMessages = await getMessages(gid, undefined, 50);
}
initialEdits = await getRecentEdits(50);
} catch {
/* client hooks surface errors */
}
@@ -26,6 +33,7 @@ export default async function MessagesPage() {
initialGuilds={guilds}
initialGuildId={config?.monitorGuildId ?? null}
initialMessages={initialMessages}
initialEdits={initialEdits}
/>
</PageTransition>
);
@@ -2,6 +2,7 @@
import {
AlertTriangle,
Calendar,
CheckCircle2,
Image as ImageIcon,
Loader2,
@@ -11,7 +12,9 @@ import {
ShieldAlert,
} from "lucide-react";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { ActivityHeatmap } from "@/components/ActivityHeatmap";
import { useAmbient } from "@/components/ambient/ambient-context";
import { EditHistory } from "@/components/EditHistory";
import {
Avatar,
Badge,
@@ -28,12 +31,14 @@ import {
import { GuildChannelPicker } from "@/components/shared/guild-picker";
import {
useLoadMore,
useMessageActivity,
useMessageDetail,
useMessageSearch,
useMessages,
useMessagesHasMore,
useMessagesStream,
useMessagesWsSync,
useRecentEdits,
useSemanticSearch,
} from "@/hooks";
import { aiTone } from "@/lib/ai-status";
@@ -45,7 +50,12 @@ import {
renderMessageContent,
safeParseJsonArray,
} from "@/lib/format";
import type { AiStatus, Guild, MessageRecord } from "@/lib/types";
import type {
AiStatus,
EditHistoryRow,
Guild,
MessageRecord,
} from "@/lib/types";
import { staggerDelay } from "@/lib/utils";
import { useWebSocket } from "@/lib/ws/context";
@@ -53,6 +63,7 @@ export function MessagesView({
initialGuilds,
initialGuildId,
initialMessages,
initialEdits,
}: {
initialGuilds?: Guild[];
initialGuildId?: string | null;
@@ -60,6 +71,7 @@ export function MessagesView({
data: MessageRecord[];
nextCursor: string | null;
} | null;
initialEdits?: EditHistoryRow[];
}) {
const ws = useWebSocket();
const [guildId, setGuildId] = useState<string | null>(
@@ -71,6 +83,8 @@ export function MessagesView({
// Search mode: "exact" (substring match over captured messages) or
// "semantic" (vector similarity over the persistent Qdrant archive).
const [semanticMode, setSemanticMode] = useState(false);
// feed | timeline: "timeline" groups messages into date-grouped cards.
const [viewMode, setViewMode] = useState<"feed" | "timeline">("feed");
// Guard against loading the entire history on a long scroll: cap how many
// older pages we append. Each page is 50 messages (backend limit default).
const MAX_OLDER_PAGES = 10;
@@ -106,6 +120,8 @@ export function MessagesView({
query,
query.trim().length >= 2 && semanticMode,
);
const activity = useMessageActivity(30);
const edits = useRecentEdits(50, undefined, initialEdits);
const detail = useMessageDetail(selected);
const ambient = useAmbient();
@@ -140,6 +156,33 @@ export function MessagesView({
// returns DESC (newest first); reverse so the feed reads top→bottom like DC.
const display = useMemo(() => [...list].reverse(), [list]);
// Timeline mode: inject date-separator headers above the first message of
// each day. Messages are sorted oldest→newest (display is reversed), so a
// date change means a new group. Produces an array of either "date" or "msg"
// nodes so the render loop can switch easily.
const timelineNodes = useMemo(() => {
if (viewMode !== "timeline") return null;
const out: Array<
| { type: "date"; label: string; iso: string }
| { type: "msg"; m: (typeof display)[number] }
> = [];
let prev = "";
for (const m of display) {
const d = new Date(m.created_at).toLocaleDateString(undefined, {
weekday: "short",
month: "short",
day: "numeric",
});
const iso = new Date(m.created_at).toISOString().slice(0, 10);
if (d !== prev) {
out.push({ type: "date", label: d, iso });
prev = d;
}
out.push({ type: "msg", m });
}
return out;
}, [display, viewMode]);
// Ref to the scroll container so we can manage scroll position like Discord:
// open at the bottom (newest), keep the viewport stable when prepending older
// messages at the top, and follow new live messages only when already near
@@ -208,6 +251,20 @@ export function MessagesView({
>
{semanticMode ? "Semantic" : "Exact"}
</button>
<button
type="button"
onClick={() =>
setViewMode((v) => (v === "feed" ? "timeline" : "feed"))
}
className={`rounded-full border px-3 py-1.5 text-xs transition-colors ${
viewMode === "timeline"
? "border-signal/40 bg-signal/10 text-signal"
: "border-hairline bg-white/[0.03] text-ink-soft hover:bg-white/[0.06]"
}`}
title="Toggle timeline (date-grouped) view"
>
{viewMode === "timeline" ? "Timeline" : "Feed"}
</button>
</GlassPanel>
<div className="grid gap-4 lg:grid-cols-5">
@@ -326,45 +383,33 @@ export function MessagesView({
}
}}
>
{display.map((m, i) => (
<button
key={m.id}
type="button"
onClick={() => setSelected(m.id)}
className={`animate-stagger flex w-full items-start gap-3 rounded-[12px] border p-3 text-left transition-colors ${
selected === m.id
? "border-signal/40 bg-signal/8"
: "border-hairline bg-white/[0.03] hover:bg-white/[0.06]"
}`}
style={staggerDelay(i)}
>
<Avatar src={m.avatar_url} name={m.username} size={34} />
<div className="min-w-0 flex-1">
<div className="flex items-center gap-2">
<span className="truncate text-sm font-semibold text-ink">
{m.username}
</span>
<span className="mono text-[0.65rem] text-ink-faint">
{getMessageChannelLabel(m)}
</span>
<span className="mono ml-auto text-[0.6rem] text-ink-faint">
{formatRelativeTime(m.created_at)}
</span>
</div>
<div className="mt-0.5 line-clamp-2 text-sm text-ink-soft">
{renderMessageContent(m.content, m.metadata) || (
<span className="italic text-ink-faint">
(empty / embed)
</span>
)}
</div>
</div>
<AiBadge
status={m.ai_status}
durationMs={m.ai_analysis_duration_ms}
/>
</button>
))}
{viewMode === "timeline" && timelineNodes
? timelineNodes.map((node, _i) =>
node.type === "date" ? (
<div
key={`date-${node.iso}`}
className="flex items-center gap-2 px-1 text-[0.65rem] text-ink-faint"
>
<Calendar className="size-3" />
{node.label}
</div>
) : (
<MessageRow
key={node.m.id}
m={node.m}
selected={selected}
onSelect={setSelected}
/>
),
)
: display.map((m, _i) => (
<MessageRow
key={m.id}
m={m}
selected={selected}
onSelect={setSelected}
/>
))}
</div>
</div>
)}
@@ -392,6 +437,12 @@ export function MessagesView({
)}
</GlassPanel>
</div>
{activity.data && activity.data.length > 0 && (
<ActivityHeatmap buckets={activity.data} />
)}
{edits.data && <EditHistory edits={edits.data} />}
</div>
);
}
@@ -513,3 +564,48 @@ function MessageDetail({
</div>
);
}
/** Single message card used by both the live feed and the date-grouped timeline. */
function MessageRow({
m,
selected,
onSelect,
}: {
m: MessageRecord;
selected: string | null;
onSelect: (id: string) => void;
}) {
return (
<button
key={m.id}
type="button"
onClick={() => onSelect(m.id)}
className={`animate-stagger flex w-full items-start gap-3 rounded-[12px] border p-3 text-left transition-colors ${
selected === m.id
? "border-signal/40 bg-signal/8"
: "border-hairline bg-white/[0.03] hover:bg-white/[0.06]"
}`}
>
<Avatar src={m.avatar_url} name={m.username} size={34} />
<div className="min-w-0 flex-1">
<div className="flex items-center gap-2">
<span className="truncate text-sm font-semibold text-ink">
{m.username}
</span>
<span className="mono text-[0.65rem] text-ink-faint">
{getMessageChannelLabel(m)}
</span>
<span className="mono ml-auto text-[0.6rem] text-ink-faint">
{formatRelativeTime(m.created_at)}
</span>
</div>
<div className="mt-0.5 line-clamp-2 text-sm text-ink-soft">
{renderMessageContent(m.content, m.metadata) || (
<span className="italic text-ink-faint">(empty / embed)</span>
)}
</div>
</div>
<AiBadge status={m.ai_status} durationMs={m.ai_analysis_duration_ms} />
</button>
);
}
@@ -15,13 +15,18 @@ import {
} from "lucide-react";
import { useEffect, useState } from "react";
import { useAmbient } from "@/components/ambient/ambient-context";
import { CategoryDrilldown } from "@/components/CategoryDrilldown";
import { CoverageTiles } from "@/components/CoverageTiles";
import { Donut } from "@/components/charts";
import { LiveModerationFeed } from "@/components/LiveModerationFeed";
import { ModerationHeatmap } from "@/components/ModerationHeatmap";
import {
Badge,
GlassPanel,
Select,
type SelectOption,
} from "@/components/primitives";
import { ScamDomains } from "@/components/ScamDomains";
import {
ErrorState,
MetricTile,
@@ -30,8 +35,21 @@ import {
SkeletonPanel,
SkeletonRows,
} from "@/components/shared";
import { useModerationActions, useModerationStats } from "@/hooks";
import { TopChannels } from "@/components/TopChannels";
import { TopicTrends } from "@/components/TopicTrends";
import {
useHourlyModeration,
useLiveModeration,
useModerationActions,
useModerationByCategory,
useModerationCoverage,
useModerationStats,
useModerationTrends,
useTopFlaggedChannels,
useTopFlaggedDomains,
} from "@/hooks";
import { aiTone } from "@/lib/ai-status";
import { downloadCsv } from "@/lib/csv";
import { formatNumber, formatRelativeTime } from "@/lib/format";
import type {
ModerationAction,
@@ -71,6 +89,15 @@ export function ModerationView({
typeFilter || undefined,
!statusFilter && !typeFilter ? initialActions : undefined,
);
const liveActions = useLiveModeration(initialActions ?? [], 50);
const { data: trends } = useModerationTrends(30);
const { data: domains } = useTopFlaggedDomains(30);
const { data: channels } = useTopFlaggedChannels(30);
const { data: hourly } = useHourlyModeration(30);
const { data: coverage } = useModerationCoverage(30);
const [drilldown, setDrilldown] = useState<string | null>(null);
const { data: categoryActions, isValidating: categoryLoading } =
useModerationByCategory(drilldown ? 30 : 0, drilldown);
const failedRate = stats ? stats.failed_rate * 100 : 0;
@@ -150,6 +177,56 @@ export function ModerationView({
</div>
<div className="grid gap-5 lg:grid-cols-5">
<div className="lg:col-span-2">
{trends ? (
<TopicTrends trends={trends} />
) : (
<SkeletonPanel rows={6} />
)}
</div>
<div className="lg:col-span-5">
<LiveModerationFeed actions={liveActions} />
</div>
{coverage ? (
<CoverageTiles coverage={coverage} />
) : (
<SkeletonPanel rows={3} className="lg:col-span-5" />
)}
<div className="lg:col-span-2">
{domains ? (
<ScamDomains domains={domains} />
) : (
<SkeletonPanel rows={6} />
)}
</div>
<div className="lg:col-span-2">
{hourly ? (
<ModerationHeatmap hours={hourly} />
) : (
<SkeletonPanel rows={6} />
)}
</div>
<div className="lg:col-span-1">
{channels ? (
<TopChannels channels={channels} />
) : (
<SkeletonPanel rows={6} />
)}
</div>
<div className="lg:col-span-3">
<CategoryDrilldown
trends={trends ?? { categories: [], severities: [], actions: [] }}
selected={drilldown}
actions={categoryActions ?? []}
loading={categoryLoading}
onSelect={setDrilldown}
/>
</div>
<GlassPanel className="lg:col-span-2">
<SectionHeader eyebrow="health" title="Breakdown" />
<div className="flex items-center gap-5">
@@ -215,6 +292,30 @@ export function ModerationView({
size="sm"
className="w-32"
/>
<button
type="button"
onClick={() =>
downloadCsv(
"moderation-actions.csv",
(actions ?? []).map((a) => ({
id: a.id,
user: a.username ?? a.user_id,
action_type: a.action_type,
status: a.status,
severity: a.severity ?? "",
categories: (a.categories ?? []).join("|"),
reason: a.reason ?? "",
created_at: a.created_at
? new Date(a.created_at).toISOString()
: "",
})),
)
}
className="rounded-full border border-hairline bg-white/[0.03] px-3 py-1 text-xs text-ink-soft transition-colors hover:bg-white/[0.06]"
title="Download moderation actions as CSV"
>
CSV
</button>
</div>
}
/>
@@ -0,0 +1,89 @@
"use client";
import { GlassPanel } from "@/components/primitives";
import { SectionHeader } from "@/components/shared";
import type { MessageActivityBucket } from "@/lib/types";
const HOURS = Array.from({ length: 24 }, (_, i) => i);
function heatColor(t: number): string {
// t in [0,1] → signal gradient (dark → bright).
if (t <= 0) return "var(--color-hairline)";
return `rgba(45, 212, 191, ${0.15 + 0.85 * t})`;
}
export function ActivityHeatmap({
buckets,
}: {
buckets: MessageActivityBucket[];
}) {
// Group by channel, find max count for normalization.
const channels = Array.from(new Set(buckets.map((b) => b.channelId)));
const byKey = new Map<string, number>();
let max = 0;
for (const b of buckets) {
const k = `${b.channelId}:${b.hour}`;
byKey.set(k, (byKey.get(k) ?? 0) + b.count);
if ((byKey.get(k) ?? 0) > max) max = byKey.get(k) ?? 0;
}
if (buckets.length === 0) {
return (
<GlassPanel className="lg:col-span-2">
<SectionHeader eyebrow="insight" title="Activity Heatmap" />
<p className="py-6 text-center text-xs text-ink-faint">
No message activity recorded yet.
</p>
</GlassPanel>
);
}
return (
<GlassPanel className="lg:col-span-5">
<SectionHeader
eyebrow="insight"
title="Activity Heatmap"
action={
<span className="mono text-[0.65rem] text-ink-faint">
{channels.length} channels · messages/hour
</span>
}
/>
<div className="overflow-x-auto">
<div className="min-w-[640px] space-y-1">
{channels.map((ch) => (
<div key={ch} className="flex items-center gap-2">
<span className="mono w-24 shrink-0 truncate text-[0.6rem] text-ink-faint">
{ch.slice(-6)}
</span>
<div className="flex flex-1 gap-0.5">
{HOURS.map((h) => {
const c = byKey.get(`${ch}:${h}`) ?? 0;
const t = max > 0 ? c / max : 0;
return (
<div
key={h}
title={`${ch} · ${String(h).padStart(2, "0")}:00 — ${c} msgs`}
className="h-4 flex-1 rounded-[2px]"
style={{ background: heatColor(t) }}
/>
);
})}
</div>
</div>
))}
<div className="flex items-center gap-2 pt-1">
<span className="w-24 shrink-0" />
<div className="flex flex-1 justify-between">
{[0, 6, 12, 18, 23].map((h) => (
<span key={h} className="mono text-[0.55rem] text-ink-faint">
{String(h).padStart(2, "0")}h
</span>
))}
</div>
</div>
</div>
</div>
</GlassPanel>
);
}
@@ -0,0 +1,128 @@
"use client";
import { ChevronRight } from "lucide-react";
import { Badge, GlassPanel } from "@/components/primitives";
import { SectionHeader } from "@/components/shared";
import { formatNumber, formatRelativeTime } from "@/lib/format";
import type { CategoryAction, ModerationTrends } from "@/lib/types";
const SEVERITY_TONE: Record<
string,
"signal" | "amber" | "vermilion" | "neutral"
> = {
critical: "vermilion",
high: "vermilion",
medium: "amber",
low: "signal",
none: "neutral",
};
interface CategoryDrilldownProps {
trends: ModerationTrends;
selected?: string | null;
actions?: CategoryAction[];
loading?: boolean;
onSelect: (category: string | null) => void;
}
export function CategoryDrilldown({
trends,
selected,
actions,
loading,
onSelect,
}: CategoryDrilldownProps) {
const maxCat = trends.categories.reduce((m, c) => Math.max(m, c.count), 0);
return (
<GlassPanel className="lg:col-span-3">
<SectionHeader eyebrow="drill-down" title="Flag Category" />
{selected ? (
<div className="mb-3 flex items-center gap-2">
<button
type="button"
onClick={() => onSelect(null)}
className="text-xs text-ink-soft hover:text-ink"
>
Back to all categories
</button>
<span className="text-xs text-ink-faint">
/ {selected} (
{loading ? "loading…" : formatNumber(actions?.length ?? 0)} actions)
</span>
</div>
) : (
<p className="mb-2 text-xs text-ink-faint">
Click a category to list the underlying moderation actions.
</p>
)}
{!selected ? (
<div className="space-y-2">
{trends.categories.map((c) => {
const pct = maxCat > 0 ? Math.max(2, (c.count / maxCat) * 100) : 0;
return (
<button
type="button"
key={c.name}
onClick={() => onSelect(c.name)}
className="flex w-full items-center gap-3 text-left text-sm"
>
<span className="w-36 shrink-0 truncate text-ink-soft">
{c.name}
</span>
<div className="h-2 flex-1 overflow-hidden rounded-full bg-white/5">
<div
className="h-full rounded-full bg-signal"
style={{ width: `${pct}%` }}
/>
</div>
<span className="mono w-10 text-right text-ink">
{formatNumber(c.count)}
</span>
</button>
);
})}
</div>
) : (
<div className="space-y-2">
{loading && <p className="text-xs text-ink-faint">Loading</p>}
{!loading && actions && actions.length === 0 && (
<p className="text-xs text-ink-faint">
No actions in this category.
</p>
)}
{actions?.slice(0, 12).map((a) => (
<div key={a.id} className="flex items-start gap-2 text-sm">
<Badge tone={SEVERITY_TONE[a.severity ?? "none"]}>
{a.severity ?? "none"}
</Badge>
<div className="min-w-0 flex-1">
<div className="flex flex-wrap items-baseline gap-x-2">
<span className="font-medium text-ink">{a.action_type}</span>
{a.username && (
<span className="text-ink-soft">@{a.username}</span>
)}
<span className="text-ink-faint mono text-xs">
{a.created_at ? formatRelativeTime(a.created_at) : ""}
</span>
</div>
{a.content && (
<p className="mt-0.5 line-clamp-2 text-ink-faint">
{a.content}
</p>
)}
{a.reason && (
<p className="mt-0.5 line-clamp-1 text-xs text-ink-faint">
Reason: {a.reason}
</p>
)}
<ChevronRight className="mt-1 size-3 text-ink-faint/50" />
</div>
</div>
))}
</div>
)}
</GlassPanel>
);
}
@@ -0,0 +1,71 @@
"use client";
import { GlassPanel } from "@/components/primitives";
import { SectionHeader } from "@/components/shared";
import { downloadCsv } from "@/lib/csv";
import { formatRelativeTime } from "@/lib/format";
import type { ChannelCultureRow } from "@/lib/types";
export function ChannelCultureGlossary({
cultures,
}: {
cultures: ChannelCultureRow[];
}) {
return (
<GlassPanel className="lg:col-span-3">
<SectionHeader
eyebrow="culture"
title="Channel Culture Glossary"
action={
cultures.length > 0 ? (
<button
type="button"
onClick={() =>
downloadCsv(
"channel-cultures.csv",
cultures.map((c) => ({
channel: c.channel_name ?? c.channel_id,
summary: c.culture_summary ?? "",
last_analyzed: c.last_analyzed_at ?? "",
})),
)
}
className="flex items-center gap-1.5 text-xs text-ink-soft hover:text-ink"
>
CSV
</button>
) : null
}
/>
{cultures.length === 0 ? (
<p className="py-6 text-center text-xs text-ink-faint">
No channel cultures captured yet.
</p>
) : (
<div className="space-y-3">
{cultures.map((c) => (
<div key={c.channel_id} className="text-sm">
<div className="flex items-baseline justify-between">
<span className="font-medium text-ink">
{c.channel_name ?? c.channel_id}
</span>
{c.last_analyzed_at && (
<span className="text-xs text-ink-faint">
{formatRelativeTime(c.last_analyzed_at)}
</span>
)}
</div>
{c.culture_summary ? (
<p className="mt-1 text-ink-faint">{c.culture_summary}</p>
) : (
<span className="text-xs text-ink-faint">
(no summary captured)
</span>
)}
</div>
))}
</div>
)}
</GlassPanel>
);
}
@@ -0,0 +1,46 @@
"use client";
import { AlertCircle, CheckCircle2, XCircle } from "lucide-react";
import { GlassPanel } from "@/components/primitives";
import { MetricTile, SectionHeader } from "@/components/shared";
import { formatNumber } from "@/lib/format";
import type { ModerationCoverage } from "@/lib/types";
export function CoverageTiles({ coverage }: { coverage: ModerationCoverage }) {
const pct = (n: number) => `${n.toFixed(1)}%`;
return (
<GlassPanel className="lg:col-span-5">
<SectionHeader eyebrow="automation" title="Auto-mod Coverage" />
<div className="grid grid-cols-2 gap-3 md:grid-cols-4">
<MetricTile
label="Coverage"
value={pct(coverage.coverage_rate)}
tone={coverage.coverage_rate > 90 ? "signal" : "amber"}
icon={<CheckCircle2 className="size-3.5" />}
/>
<MetricTile
label="Completed"
value={formatNumber(coverage.completed)}
tone="signal"
icon={<CheckCircle2 className="size-3.5" />}
/>
<MetricTile
label="Failed"
value={formatNumber(coverage.failed)}
tone={coverage.failed > 0 ? "vermilion" : "neutral"}
icon={<XCircle className="size-3.5" />}
/>
<MetricTile
label="Pending"
value={formatNumber(coverage.pending)}
tone={coverage.pending > 0 ? "amber" : "neutral"}
icon={<AlertCircle className="size-3.5" />}
/>
</div>
<p className="mt-2 text-xs text-ink-faint">
{pct(coverage.failed_rate)} of analysis runs failed. Total runs in
window: {formatNumber(coverage.total)}.
</p>
</GlassPanel>
);
}
@@ -0,0 +1,68 @@
"use client";
import { Download, History } from "lucide-react";
import { GlassPanel } from "@/components/primitives";
import { SectionHeader } from "@/components/shared";
import { downloadCsv } from "@/lib/csv";
import { formatRelativeTime } from "@/lib/format";
import type { EditHistoryRow } from "@/lib/types";
export function EditHistory({ edits }: { edits: EditHistoryRow[] }) {
return (
<GlassPanel className="lg:col-span-4">
<SectionHeader
eyebrow="evasion"
title="Message Edits"
action={
edits.length > 0 ? (
<button
type="button"
onClick={() =>
downloadCsv(
"message-edits.csv",
edits.map((e) => ({
author: e.username ?? "",
channel: e.channel_name ?? "",
old_content: e.old_content,
edited_at: e.edited_at,
})),
)
}
className="flex items-center gap-1.5 text-xs text-ink-soft hover:text-ink"
>
<Download className="size-3.5" />
CSV
</button>
) : null
}
/>
{edits.length === 0 ? (
<p className="py-6 text-center text-xs text-ink-faint">
No edited messages recorded recently.
</p>
) : (
<div className="space-y-3">
{edits.map((e) => (
<div key={e.id} className="text-sm">
<div className="flex flex-wrap items-baseline justify-between gap-2">
<span className="font-medium text-ink">
{e.username ?? "unknown"}
</span>
<span className="text-xs text-ink-faint">
edited {formatRelativeTime(e.edited_at)} ·{" "}
{e.channel_name ?? e.channel_id ?? "unknown channel"}
</span>
</div>
<div className="mt-1 flex items-start gap-1.5">
<History className="mt-0.5 size-3.5 shrink-0 text-ink-faint/50" />
<pre className="line-clamp-2 whitespace-pre-wrap break-words text-ink-faint/80">
{e.old_content || <em>(content not available)</em>}
</pre>
</div>
</div>
))}
</div>
)}
</GlassPanel>
);
}
@@ -0,0 +1,102 @@
"use client";
import { Badge, GlassPanel } from "@/components/primitives";
import { formatRelativeTime } from "@/lib/format";
import type { ModerationAction } from "@/lib/types";
const ACTION_LABEL: Record<string, string> = {
delete_message: "Deleted",
timeout_user: "Timeout",
warn_user: "Warned",
reset_nickname: "Nickname reset",
ban_user: "Banned",
kick_user: "Kicked",
notify_user: "Notified",
none: "None",
};
function severityTone(
sev?: string | null,
): "signal" | "amber" | "vermilion" | null {
switch (sev) {
case "critical":
case "high":
return "vermilion";
case "medium":
return "amber";
case "low":
return "signal";
default:
return null;
}
}
export function LiveModerationFeed({
actions,
}: {
actions: ModerationAction[];
}) {
return (
<GlassPanel className="flex max-h-[420px] flex-col">
<div className="flex items-center justify-between border-b border-white/10 px-4 py-3">
<div className="flex items-center gap-2">
<span className="relative flex size-2.5">
<span className="absolute inline-flex size-full animate-ping rounded-full bg-emerald-400 opacity-75" />
<span className="relative inline-flex size-2.5 rounded-full bg-emerald-500" />
</span>
<h3 className="text-sm font-medium text-ink">Live Feed</h3>
</div>
<span className="text-xs text-ink-faint">{actions.length} recent</span>
</div>
<div className="flex-1 overflow-y-auto">
{actions.length === 0 ? (
<p className="px-4 py-6 text-center text-xs text-ink-faint">
Waiting for new moderation actions
</p>
) : (
<ul className="divide-y divide-white/5">
{actions.map((a, i) => {
const tone = severityTone(a.severity);
return (
<li
key={a.id}
className={`flex items-start gap-3 px-4 py-3 ${
i === 0 ? "animate-[fadeIn_0.4s_ease-out]" : ""
}`}
>
<div className="min-w-0 flex-1">
<div className="flex items-center gap-2">
<Badge tone={tone ?? "signal"} className="capitalize">
{ACTION_LABEL[a.action_type] ?? a.action_type}
</Badge>
{a.severity && (
<span className="text-xs text-ink-faint">
{a.severity}
</span>
)}
{a.categories?.length ? (
<span className="truncate text-xs text-ink-soft">
{a.categories.slice(0, 3).join(", ")}
</span>
) : null}
</div>
{a.reason && (
<p className="mt-1 truncate text-xs text-ink-soft">
{a.reason}
</p>
)}
<p className="mt-0.5 text-[11px] text-ink-faint">
{a.username ?? a.user_id ?? "unknown"} ·{" "}
{formatRelativeTime(a.created_at)}
</p>
</div>
</li>
);
})}
</ul>
)}
</div>
</GlassPanel>
);
}
@@ -0,0 +1,53 @@
"use client";
import { GlassPanel } from "@/components/primitives";
import { SectionHeader } from "@/components/shared";
import type { HourlyModeration } from "@/lib/types";
import { cn } from "@/lib/utils";
export function ModerationHeatmap({ hours }: { hours: HourlyModeration[] }) {
const max = hours.reduce((m, h) => Math.max(m, h.total), 0);
const intensity = (v: number) => {
if (max <= 0) return "bg-white/5";
const t = Math.max(0, Math.min(1, v / max));
if (t < 0.25) return "bg-white/[0.06]";
if (t < 0.5) return "bg-signal/25";
if (t < 0.75) return "bg-signal/50";
return "bg-vermilion/60";
};
return (
<GlassPanel className="lg:col-span-2">
<SectionHeader eyebrow="timing" title="Flagged by Hour (24h)" />
<p className="mb-3 text-xs text-ink-faint">
Distribution of moderation actions across the day.
</p>
<div className="grid grid-cols-2 gap-4 md:grid-cols-3">
{hours.map((h) => (
<div key={h.hour} className="flex items-center gap-2">
<span className="w-8 text-xs text-ink-faint mono">
{String(h.hour).padStart(2, "0")}:00
</span>
<div className="flex-1">
<div
className={cn(
"h-5 rounded transition-colors",
intensity(h.total),
)}
title={`${h.total} actions`}
/>
</div>
<span
className={cn(
"mono w-8 text-right text-xs",
h.total === 0 ? "text-ink-faint/40" : "text-ink",
)}
>
{h.total}
</span>
</div>
))}
</div>
</GlassPanel>
);
}
@@ -0,0 +1,67 @@
"use client";
import { Download } from "lucide-react";
import { GlassPanel } from "@/components/primitives";
import { SectionHeader } from "@/components/shared";
import { downloadCsv } from "@/lib/csv";
import { formatNumber } from "@/lib/format";
import type { FlaggedDomain } from "@/lib/types";
export function ScamDomains({ domains }: { domains: FlaggedDomain[] }) {
const max = domains.reduce((m, d) => Math.max(m, d.count), 0);
return (
<GlassPanel className="lg:col-span-2">
<SectionHeader
eyebrow="risk"
title="Flagged Link Domains"
action={
domains.length > 0 ? (
<button
type="button"
onClick={() =>
downloadCsv(
"flagged-domains.csv",
domains.map((d) => ({
domain: d.domain,
flagged_count: d.count,
})),
)
}
className="flex items-center gap-1.5 text-xs text-ink-soft hover:text-ink"
>
<Download className="size-3.5" />
CSV
</button>
) : null
}
/>
{domains.length === 0 ? (
<p className="py-6 text-center text-xs text-ink-faint">
No flagged links captured recently.
</p>
) : (
<div className="space-y-2">
{domains.map((d) => {
const pct = max > 0 ? Math.max(2, (d.count / max) * 100) : 0;
return (
<div key={d.domain} className="flex items-center gap-3 text-sm">
<span className="w-44 shrink-0 truncate font-mono text-ink-soft">
{d.domain}
</span>
<div className="h-2 flex-1 overflow-hidden rounded-full bg-white/5">
<div
className="h-full rounded-full bg-[#8b5cf6]"
style={{ width: `${pct}%` }}
/>
</div>
<span className="mono w-10 shrink-0 text-right text-ink">
{formatNumber(d.count)}
</span>
</div>
);
})}
</div>
)}
</GlassPanel>
);
}
@@ -0,0 +1,71 @@
"use client";
import { Globe } from "lucide-react";
import { GlassPanel } from "@/components/primitives";
import { SectionHeader } from "@/components/shared";
import { downloadCsv } from "@/lib/csv";
import { formatRelativeTime } from "@/lib/format";
import type { GlossaryRow } from "@/lib/types";
export function TermGlossary({ terms }: { terms: GlossaryRow[] }) {
return (
<GlassPanel className="lg:col-span-3">
<SectionHeader
eyebrow="knowledge"
title="Term Knowledge Base"
action={
terms.length > 0 ? (
<button
type="button"
onClick={() =>
downloadCsv(
"glossary.csv",
terms.map((t) => ({
term: t.term,
definition: t.definition,
source: t.source_url,
resolved: t.resolved_at,
hits: t.hit_count,
})),
)
}
className="flex items-center gap-1.5 text-xs text-ink-soft hover:text-ink"
>
CSV
</button>
) : null
}
/>
{terms.length === 0 ? (
<p className="py-6 text-center text-xs text-ink-faint">
No term resolutions cached yet.
</p>
) : (
<div className="space-y-3">
{terms.map((t) => (
<div key={t.term} className="text-sm">
<div className="flex flex-wrap items-baseline justify-between gap-2">
<span className="font-medium text-ink">{t.term}</span>
<span className="text-xs text-ink-faint">
{t.hit_count} uses · {formatRelativeTime(t.resolved_at)}
</span>
</div>
<p className="mt-1 text-ink-faint">{t.definition}</p>
{t.source_url && (
<a
href={t.source_url}
target="_blank"
rel="noopener noreferrer"
className="mt-0.5 text-xs text-ink-soft hover:text-ink"
>
<Globe className="mr-1 inline size-3" />
{t.source_url}
</a>
)}
</div>
))}
</div>
)}
</GlassPanel>
);
}
@@ -0,0 +1,72 @@
"use client";
import { Download } from "lucide-react";
import { GlassPanel } from "@/components/primitives";
import { SectionHeader } from "@/components/shared";
import { downloadCsv } from "@/lib/csv";
import { formatNumber } from "@/lib/format";
import type { FlaggedChannel } from "@/lib/types";
export function TopChannels({ channels }: { channels: FlaggedChannel[] }) {
const max = channels.reduce((m, c) => Math.max(m, c.flagged_count), 0);
return (
<GlassPanel className="lg:col-span-2">
<SectionHeader
eyebrow="channels"
title="Top Flagged Channels"
action={
channels.length > 0 ? (
<button
type="button"
onClick={() =>
downloadCsv(
"flagged-channels.csv",
channels.map((c) => ({
channel_id: c.channel_id,
channel_name: c.channel_name ?? "",
flagged_count: c.flagged_count,
})),
)
}
className="flex items-center gap-1.5 text-xs text-ink-soft hover:text-ink"
>
<Download className="size-3.5" />
CSV
</button>
) : null
}
/>
{channels.length === 0 ? (
<p className="py-6 text-center text-xs text-ink-faint">
No flagged activity in the selected period.
</p>
) : (
<div className="space-y-2">
{channels.map((c) => {
const pct =
max > 0 ? Math.max(2, (c.flagged_count / max) * 100) : 0;
return (
<div
key={c.channel_id}
className="flex items-center gap-3 text-sm"
>
<span className="w-40 shrink-0 truncate text-ink-soft">
{c.channel_name ?? c.channel_id}
</span>
<div className="h-2 flex-1 overflow-hidden rounded-full bg-white/5">
<div
className="h-full rounded-full bg-[#f59e0b]"
style={{ width: `${pct}%` }}
/>
</div>
<span className="mono w-10 shrink-0 text-right text-ink">
{formatNumber(c.flagged_count)}
</span>
</div>
);
})}
</div>
)}
</GlassPanel>
);
}
@@ -0,0 +1,146 @@
"use client";
import { Donut } from "@/components/charts/donut";
import { GlassPanel } from "@/components/primitives";
import { SectionHeader } from "@/components/shared";
import { formatNumber } from "@/lib/format";
import type { ModerationTrends } from "@/lib/types";
const SEVERITY_COLOR: Record<string, string> = {
critical: "var(--color-vermilion)",
high: "var(--color-vermilion)",
medium: "var(--color-amber)",
low: "var(--color-signal)",
none: "var(--color-ink-faint)",
};
function BarRow({
label,
count,
max,
color = "var(--color-signal)",
}: {
label: string;
count: number;
max: number;
color?: string;
}) {
const pct = max > 0 ? Math.max(2, (count / max) * 100) : 0;
return (
<div className="flex items-center gap-3 text-sm">
<span className="w-32 shrink-0 truncate text-ink-soft">{label}</span>
<div className="h-2 flex-1 overflow-hidden rounded-full bg-white/5">
<div
className="h-full rounded-full"
style={{ width: `${pct}%`, background: color }}
/>
</div>
<span className="mono w-10 shrink-0 text-right text-ink">
{formatNumber(count)}
</span>
</div>
);
}
export function TopicTrends({ trends }: { trends: ModerationTrends }) {
const maxCat = trends.categories.reduce((m, c) => Math.max(m, c.count), 0);
const maxAct = trends.actions.reduce((m, a) => Math.max(m, a.count), 0);
const totalSev = trends.severities.reduce((s, x) => s + x.count, 0);
const severitySegments = trends.severities.map((s) => ({
value: s.count,
color: SEVERITY_COLOR[s.level] ?? "var(--color-ink-faint)",
label: s.level,
}));
return (
<GlassPanel className="lg:col-span-2">
<SectionHeader eyebrow="insight" title="Toxic Topic Trends" />
{trends.categories.length === 0 && trends.severities.length === 0 ? (
<p className="py-6 text-center text-xs text-ink-faint">
No categorized actions in the last 30 days.
</p>
) : (
<div className="space-y-5">
<div>
<p className="mb-2 text-xs uppercase tracking-wide text-ink-faint">
Top flagged categories
</p>
<div className="space-y-2">
{trends.categories.slice(0, 10).map((c) => (
<BarRow
key={c.name}
label={c.name}
count={c.count}
max={maxCat}
/>
))}
{trends.categories.length === 0 && (
<p className="text-xs text-ink-faint">
No categories recorded.
</p>
)}
</div>
</div>
<div className="grid grid-cols-2 gap-4">
<div>
<p className="mb-2 text-xs uppercase tracking-wide text-ink-faint">
Severity
</p>
{totalSev > 0 ? (
<div className="flex items-center gap-4">
<Donut
segments={severitySegments}
centerLabel={formatNumber(totalSev)}
centerSub="total"
size={88}
/>
<div className="space-y-1 text-xs">
{trends.severities.map((s) => (
<div key={s.level} className="flex items-center gap-2">
<span
className="size-2.5 rounded-full"
style={{
background:
SEVERITY_COLOR[s.level] ??
"var(--color-ink-faint)",
}}
/>
<span className="capitalize text-ink-soft">
{s.level}
</span>
<span className="mono ml-auto text-ink">
{formatNumber(s.count)}
</span>
</div>
))}
</div>
</div>
) : (
<p className="text-xs text-ink-faint">No severity data.</p>
)}
</div>
<div>
<p className="mb-2 text-xs uppercase tracking-wide text-ink-faint">
Action types
</p>
<div className="space-y-2">
{trends.actions.slice(0, 6).map((a) => (
<BarRow
key={a.type}
label={a.type.replace("_", " ")}
count={a.count}
max={maxAct}
color="#8b5cf6"
/>
))}
</div>
</div>
</div>
</div>
)}
</GlassPanel>
);
}
+10
View File
@@ -10,6 +10,7 @@ export {
useUsers,
} from "./use-dashboard";
export { useGuilds } from "./use-guilds";
export { useChannelCultures, useGlossary } from "./use-knowledge";
export {
useMediaLoop,
useMediaQueue,
@@ -21,19 +22,28 @@ export {
export {
useImages,
useLoadMore,
useMessageActivity,
useMessageDetail,
useMessageSearch,
useMessages,
useMessagesHasMore,
useMessagesStream,
useMessagesWsSync,
useRecentEdits,
useReview,
useSemanticSearch,
useTextChannels,
} from "./use-messages";
export {
useHourlyModeration,
useLiveModeration,
useModerationActions,
useModerationByCategory,
useModerationCoverage,
useModerationStats,
useModerationTrends,
useTopFlaggedChannels,
useTopFlaggedDomains,
} from "./use-moderation";
export {
useDeleteRecording,
@@ -0,0 +1,22 @@
import useSWR from "swr";
import { knowledgeApi } from "@/lib/api";
import type { ChannelCultureRow, GlossaryRow } from "@/lib/types";
export function useChannelCultures(
limit = 100,
initialData?: ChannelCultureRow[],
) {
return useSWR<ChannelCultureRow[]>(
["channel-cultures", limit],
() => knowledgeApi.channelCultures(limit),
{ fallbackData: initialData },
);
}
export function useGlossary(limit = 100, initialData?: GlossaryRow[]) {
return useSWR<GlossaryRow[]>(
["glossary", limit],
() => knowledgeApi.glossary(limit),
{ fallbackData: initialData },
);
}
+46 -6
View File
@@ -5,6 +5,8 @@ import { messagesApi, voiceApi } from "@/lib/api";
import type {
AttachmentRecord,
Channel,
EditHistoryRow,
MessageActivityBucket,
MessageRecord,
SemanticSearchResult,
} from "@/lib/types";
@@ -91,7 +93,7 @@ export function useLoadMore() {
(old: MessagePage | undefined): MessagePage | undefined =>
old
? {
data: [...old.data, ...result.data],
data: sortMessages([...old.data, ...result.data]),
nextCursor: result.nextCursor,
}
: result,
@@ -199,6 +201,17 @@ export function useSemanticSearch(query: string, enabled: boolean) {
// ── WS sync helpers ──────────────────────────────
/**
* Sort messages newest-first (descending by created_at). The message list is
* stored newest-first in SWR data (the view reverses it for display), so every
* WS insert must maintain this order regardless of arrival order. Without this,
* out-of-order `message_created` / `message_snapshot` frames produce a
* scrambled feed.
*/
function sortMessages(msgs: MessageRecord[]): MessageRecord[] {
return [...msgs].sort((a, b) => b.created_at - a.created_at);
}
export function useMessagesWsSync(ws: WsHook, guildId: string) {
const { mutate } = useSWRConfig();
useEffect(() => {
@@ -235,7 +248,8 @@ export function useMessagesWsSync(ws: WsHook, guildId: string) {
const msg = data as MessageRecord;
patchLists(
(_k, m) => matchesFilter(_k as unknown[], m),
(old) => (old ? { ...old, data: [msg, ...old.data] } : old),
(old) =>
old ? { ...old, data: sortMessages([msg, ...old.data]) } : old,
msg,
);
});
@@ -252,8 +266,8 @@ export function useMessagesWsSync(ws: WsHook, guildId: string) {
old
? {
...old,
data: old.data.map((m) =>
m.id === msg.id ? { ...m, ...msg } : m,
data: sortMessages(
old.data.map((m) => (m.id === msg.id ? { ...m, ...msg } : m)),
),
}
: old,
@@ -281,7 +295,12 @@ export function useMessagesWsSync(ws: WsHook, guildId: string) {
(_k, m) => matchesFilter(_k as unknown[], m),
(old) =>
old
? { ...old, data: old.data.map((m) => (m.id === msg.id ? msg : m)) }
? {
...old,
data: sortMessages(
old.data.map((m) => (m.id === msg.id ? msg : m)),
),
}
: old,
msg,
);
@@ -329,7 +348,10 @@ export function useMessagesStream(
const data2 = old?.data ?? [];
if (data2.some((m) => m.id === msg.id))
return old ?? { data: [], nextCursor: null };
return { data: [msg, ...data2], nextCursor: old?.nextCursor ?? null };
return {
data: sortMessages([msg, ...data2]),
nextCursor: old?.nextCursor ?? null,
};
},
{ revalidate: false },
);
@@ -374,3 +396,21 @@ export function useMessagesStream(
return { streaming, error };
}
export function useMessageActivity(days = 30) {
return useSWR<MessageActivityBucket[]>(["activity", days], () =>
messagesApi.getActivity(days),
);
}
export function useRecentEdits(
limit = 50,
channelId?: string,
initialData?: EditHistoryRow[],
) {
return useSWR<EditHistoryRow[]>(
["recent-edits", limit, channelId ?? null],
() => messagesApi.getRecentEdits(limit, channelId),
{ fallbackData: initialData },
);
}
+87 -1
View File
@@ -1,6 +1,17 @@
import { useCallback, useEffect, useRef, useState } from "react";
import useSWR from "swr";
import { moderationApi } from "@/lib/api";
import type { ModerationAction, ModerationStats } from "@/lib/types";
import type {
CategoryAction,
FlaggedChannel,
FlaggedDomain,
HourlyModeration,
ModerationAction,
ModerationCoverage,
ModerationStats,
ModerationTrends,
} from "@/lib/types";
import { useWebSocket } from "@/lib/ws/context";
export function useModerationStats(initialData?: ModerationStats) {
return useSWR<ModerationStats>(
@@ -33,3 +44,78 @@ export function useModerationActions(
},
);
}
/**
* Live moderation feed: merges the initial SWR list with actions pushed over
* the WebSocket in real time. Returns a capped, newest-first buffer.
* Read-only / public no write actions.
*/
export function useLiveModeration(
initialData: ModerationAction[] = [],
cap = 50,
) {
const { on: subscribe } = useWebSocket();
const [live, setLive] = useState<ModerationAction[]>(initialData);
const seen = useRef<Set<string>>(new Set(initialData.map((a) => a.id)));
useEffect(() => {
setLive(initialData);
seen.current = new Set(initialData.map((a) => a.id));
}, [initialData]);
const handle = useCallback(
(action: ModerationAction) => {
if (seen.current.has(action.id)) return;
seen.current.add(action.id);
setLive((prev) => [action, ...prev].slice(0, cap));
},
[cap],
);
useEffect(() => {
const unsub = subscribe("moderation_action", handle);
return unsub;
}, [subscribe, handle]);
return live;
}
export function useModerationTrends(days = 30, initialData?: ModerationTrends) {
return useSWR<ModerationTrends>(
["moderation-trends", days],
() => moderationApi.getTrends(days),
{ fallbackData: initialData },
);
}
export function useTopFlaggedDomains(days = 30) {
return useSWR<FlaggedDomain[]>(["moderation-domains", days], () =>
moderationApi.getTopDomains(days),
);
}
export function useTopFlaggedChannels(days = 30) {
return useSWR<FlaggedChannel[]>(["moderation-channels", days], () =>
moderationApi.getTopChannels(days),
);
}
export function useHourlyModeration(days = 30) {
return useSWR<HourlyModeration[]>(["moderation-byhour", days], () =>
moderationApi.getHourlyModeration(days),
);
}
export function useModerationByCategory(days = 30, category: string | null) {
return useSWR<CategoryAction[]>(
category ? ["moderation-bycategory", days, category] : null,
() => moderationApi.getByCategory(days, category as string),
{ keepPreviousData: true },
);
}
export function useModerationCoverage(days = 30) {
return useSWR<ModerationCoverage>(["moderation-coverage", days], () =>
moderationApi.getCoverage(days),
);
}
+7
View File
@@ -6,9 +6,16 @@ export { orpc } from "../orpc/client";
export { chatbotApi } from "./chatbot";
export { configApi } from "./config";
export { dashboardApi } from "./dashboard";
export { knowledgeApi } from "./knowledge";
export { mediaApi } from "./media";
export { messagesApi } from "./messages";
export { moderationApi } from "./moderation";
export { recordingsApi } from "./recordings";
// Re-export server-side fetchers for use inside React Server Components.
export {
getChannelCultures,
getGlossary,
getRecentEdits,
} from "./server";
export { uiStateApi } from "./ui-state";
export { voiceApi } from "./voice";
@@ -0,0 +1,16 @@
import { orpc } from "@/lib/orpc/client";
import type { ChannelCultureRow, GlossaryRow } from "@/lib/types";
export const knowledgeApi = {
channelCultures: (limit = 100, search?: string) =>
orpc.knowledge.channelCultures({
limit,
search,
}) as unknown as Promise<ChannelCultureRow[]>,
glossary: (limit = 100, search?: string) =>
orpc.knowledge.glossary({
limit,
search,
}) as unknown as Promise<GlossaryRow[]>,
};
+78
View File
@@ -0,0 +1,78 @@
// Materi API client — talks to backend /trpc materi router
import { createORPCClient } from "@orpc/client";
import { RPCLink } from "@orpc/client/fetch";
import { orpc } from "@/lib/orpc/client";
import type { ORPCClient } from "@/lib/orpc/types";
import type {
CreateMateriInput,
MateriDocument,
MateriRagChatMessage,
MateriRagChatResult,
} from "@/lib/types/materi";
const BACKEND_URL =
process.env.GMW_BACKEND_URL?.replace(/\/+$/, "") || "http://127.0.0.1:4001";
let _serverClient: ORPCClient | null = null;
function serverOrpc(): ORPCClient {
if (!_serverClient) {
const link = new RPCLink({
url: `${BACKEND_URL}/trpc`,
fetch(url, init) {
return fetch(url, { ...init, cache: "no-store" });
},
});
_serverClient = createORPCClient(link) as unknown as ORPCClient;
}
return _serverClient;
}
// Server-side (SSR seed) — uses the HTTP RPCLink via oRPC
export async function listMateriSSR(
limit = 50,
search?: string,
): Promise<MateriDocument[]> {
return (serverOrpc() as any).materi.list({
limit,
search,
}) as unknown as Promise<MateriDocument[]>;
}
export async function getMateriSSR(id: string): Promise<MateriDocument | null> {
return (serverOrpc() as any).materi.detail({
id,
}) as unknown as Promise<MateriDocument | null>;
}
// Client-side — browser WebSocket RPCLink (orpc is "use client")
export async function createMateri(
input: CreateMateriInput,
): Promise<MateriDocument> {
return orpc.materi.create(input) as unknown as Promise<MateriDocument>;
}
export async function updateMateri(
id: string,
input: Partial<CreateMateriInput>,
): Promise<MateriDocument | null> {
return orpc.materi.update({
id,
...input,
}) as unknown as Promise<MateriDocument | null>;
}
export async function deleteMateri(id: string): Promise<boolean> {
return orpc.materi.delete({ id }) as unknown as Promise<boolean>;
}
export async function searchMateri(
query: string,
history: MateriRagChatMessage[] = [],
materiId?: string,
): Promise<MateriRagChatResult> {
return orpc.materi.chat({
message: query,
history,
materiId,
}) as unknown as Promise<MateriRagChatResult>;
}
+14
View File
@@ -1,6 +1,8 @@
import { orpc } from "@/lib/orpc/client";
import type {
AttachmentRecord,
EditHistoryRow,
MessageActivityBucket,
MessageRecord,
SemanticSearchResult,
} from "@/lib/types";
@@ -73,4 +75,16 @@ export const messagesApi = {
results: SemanticSearchResult[];
nextCursor: null;
}>,
// Public, read-only activity heatmap data (per-hour volume by channel).
getActivity: (days = 30) =>
orpc.messages.activity({ days }) as unknown as Promise<
MessageActivityBucket[]
>,
// Public, read-only recent message edits (evasion tracker).
getRecentEdits: (limit = 50, channelId?: string) =>
orpc.messages.editHistory({ limit, channelId }) as unknown as Promise<
EditHistoryRow[]
>,
};
+34 -1
View File
@@ -1,5 +1,14 @@
import { orpc } from "@/lib/orpc/client";
import type { ModerationStats, PaginatedModerationActions } from "@/lib/types";
import type {
CategoryAction,
FlaggedChannel,
FlaggedDomain,
HourlyModeration,
ModerationCoverage,
ModerationStats,
ModerationTrends,
PaginatedModerationActions,
} from "@/lib/types";
export const moderationApi = {
getStats: () =>
@@ -17,4 +26,28 @@ export const moderationApi = {
actionType,
cursor,
}) as unknown as Promise<PaginatedModerationActions>,
getTrends: (days = 30) =>
orpc.moderation.trends({ days }) as unknown as Promise<ModerationTrends>,
getTopDomains: (days = 30) =>
orpc.moderation.topDomains({ days }) as unknown as Promise<FlaggedDomain[]>,
getTopChannels: (days = 30) =>
orpc.moderation.topChannels({ days }) as unknown as Promise<
FlaggedChannel[]
>,
getHourlyModeration: (days = 30) =>
orpc.moderation.byHour({ days }) as unknown as Promise<HourlyModeration[]>,
getByCategory: (days = 30, category: string) =>
orpc.moderation.byCategory({ days, category }) as unknown as Promise<
CategoryAction[]
>,
getCoverage: (days = 30) =>
orpc.moderation.coverage({
days,
}) as unknown as Promise<ModerationCoverage>,
};
+52
View File
@@ -16,11 +16,19 @@ import { createORPCClient } from "@orpc/client";
import { RPCLink } from "@orpc/client/fetch";
import type {
AppConfig,
ChannelCultureRow,
DashboardActivity,
DashboardStats,
EditHistoryRow,
FlaggedChannel,
FlaggedDomain,
GlossaryRow,
Guild,
HourlyModeration,
MediaState,
ModerationCoverage,
ModerationStats,
ModerationTrends,
PaginatedModerationActions,
PaginatedRecordings,
VoiceStatus,
@@ -84,6 +92,33 @@ export async function getModerationActions(limit = 100) {
})) as unknown as PaginatedModerationActions;
return res.data;
}
export async function getModerationTrends(
days = 30,
): Promise<ModerationTrends> {
return serverOrpc().moderation.trends({
days,
}) as unknown as Promise<ModerationTrends>;
}
export async function getTopFlaggedDomains(days = 30) {
return serverOrpc().moderation.topDomains({
days,
}) as unknown as FlaggedDomain[];
}
export async function getTopFlaggedChannels(days = 30) {
return serverOrpc().moderation.topChannels({
days,
}) as unknown as FlaggedChannel[];
}
export async function getHourlyModeration(days = 30) {
return serverOrpc().moderation.byHour({
days,
}) as unknown as HourlyModeration[];
}
export async function getCoverage(days = 30) {
return serverOrpc().moderation.coverage({
days,
}) as unknown as ModerationCoverage;
}
// ---- Voice ----
export async function getGuilds(): Promise<Guild[]> {
@@ -122,3 +157,20 @@ export async function getMessages(
nextCursor: string | null;
}>;
}
// ---- Knowledge (public read-only) ----
export async function getChannelCultures(limit = 100) {
return serverOrpc().knowledge.channelCultures({
limit,
}) as unknown as Promise<ChannelCultureRow[]>;
}
export async function getGlossary(limit = 100) {
return serverOrpc().knowledge.glossary({
limit,
}) as unknown as Promise<GlossaryRow[]>;
}
export async function getRecentEdits(limit = 50): Promise<EditHistoryRow[]> {
return serverOrpc().messages.editHistory({
limit,
}) as unknown as Promise<EditHistoryRow[]>;
}
+32
View File
@@ -0,0 +1,32 @@
/** Client-side CSV export. Pure browser — no backend, no write scope. */
export function toCsv(rows: Record<string, unknown>[]): string {
if (rows.length === 0) return "";
const headers = Array.from(
rows.reduce<Set<string>>((s, r) => {
Object.keys(r).forEach((k) => s.add(k));
return s;
}, new Set()),
);
const esc = (v: unknown): string => {
if (v == null) return "";
const s = typeof v === "object" ? JSON.stringify(v) : String(v);
return /[",\n]/.test(s) ? `"${s.replace(/"/g, '""')}"` : s;
};
const head = headers.map(esc).join(",");
const body = rows
.map((r) => headers.map((h) => esc(r[h])).join(","))
.join("\n");
return `${head}\n${body}`;
}
export function downloadCsv(filename: string, rows: Record<string, unknown>[]) {
const csv = toCsv(rows);
if (!csv) return;
const blob = new Blob([csv], { type: "text/csv;charset=utf-8;" });
const url = URL.createObjectURL(blob);
const a = document.createElement("a");
a.href = url;
a.download = filename;
a.click();
URL.revokeObjectURL(url);
}
+7
View File
@@ -1,4 +1,5 @@
import {
BookOpen,
Headphones,
LayoutDashboard,
type LucideIcon,
@@ -63,6 +64,12 @@ export const navItems: NavItem[] = [
icon: Search,
matchPrefix: "/analysis",
},
{
href: "/materi",
label: "Materi",
icon: BookOpen,
matchPrefix: "/materi",
},
];
/**
+2
View File
@@ -1,5 +1,7 @@
export * from "./dashboard";
export * from "./guild";
export * from "./knowledge";
export * from "./materi";
export * from "./media";
export * from "./message";
export * from "./moderation";
@@ -0,0 +1,25 @@
export interface ChannelCultureRow {
channel_id: string;
guild_id: string | null;
channel_name: string | null;
culture_summary: string | null;
last_analyzed_at: number | null;
}
export interface GlossaryRow {
term: string;
definition: string;
source_url: string;
resolved_at: number;
hit_count: number;
}
export interface EditHistoryRow {
id: string;
message_id: string;
old_content: string;
edited_at: number;
channel_id: string | null;
channel_name: string | null;
username: string | null;
}
+41
View File
@@ -0,0 +1,41 @@
export interface MateriDocument {
id: string;
title: string;
description: string | null;
content: string;
category: string;
tags: string[];
owner_user_id: string;
guild_id: string | null;
channel_id: string | null;
is_public: boolean;
view_count: number;
created_at: number;
updated_at: number;
}
export interface CreateMateriInput {
title: string;
description?: string;
content: string;
category: string;
tags: string[];
guildId?: string;
channelId?: string;
isPublic: boolean;
}
export interface MateriRagChatMessage {
role: "user" | "assistant";
content: string;
}
export interface MateriRagChatResult {
answer: string;
sources: Array<{
id: string;
title: string;
score: number;
excerpt: string;
}>;
}
@@ -173,6 +173,12 @@ export interface SemanticSearchResult {
created_at: number;
}
export interface MessageActivityBucket {
channelId: string;
hour: number;
count: number;
}
export interface SemanticSearchResponse {
results: SemanticSearchResult[];
nextCursor: null;
@@ -44,3 +44,50 @@ export interface PaginatedModerationActions {
data: ModerationAction[];
nextCursor: string | null;
}
export interface ModerationTrends {
categories: { name: string; count: number }[];
severities: { level: string; count: number }[];
actions: { type: string; count: number }[];
}
export interface FlaggedDomain {
domain: string;
count: number;
}
export interface FlaggedChannel {
channel_id: string;
channel_name: string | null;
flagged_count: number;
}
export interface HourlyModeration {
hour: number;
total: number;
}
export interface CategoryAction {
id: string;
message_id: string | null;
user_id: string | null;
guild_id: string;
action_type: ModerationActionType;
reason: string | null;
status: ModerationStatus;
created_at: number | null;
severity: "none" | "low" | "medium" | "high" | "critical" | null;
confidence: number | null;
score: number | null;
username: string | null;
content: string | null;
}
export interface ModerationCoverage {
total: number;
completed: number;
failed: number;
pending: number;
coverage_rate: number;
failed_rate: number;
}
+3
View File
@@ -2,6 +2,7 @@ import type {
ActiveSpeaker,
MediaState,
MessageRecord,
ModerationAction,
VoiceRecording,
} from "@/lib/types";
@@ -68,6 +69,8 @@ export interface WsEventMap {
presence_updated: unknown;
guild_member_added: unknown;
guild_member_removed: unknown;
/** Live moderation action broadcast (gateway → Redis → backend → WS). */
moderation_action: ModerationAction;
media_state: MediaState;
user_state: unknown;
ui_state: unknown;