perf(ai-moderation): drop personal user-profile descriptions from context
User insight: personal profile summaries bloat the prompt (less room per request) and add a per-user DB/Redis round-trip for little moderation signal. Only the behavioural <user_reputation> history is kept. - textBatchProcessor: stop fetching getUserProfile; remove <user_profiles> block + <user_profile_ref> from message tags. Keep <user_reputation>. - mediaBatchProcessor + visionAnalyzer: same removal (profile fetch + ref). - prompts/system.ts + prompts/output.ts: drop stale <user_profiles>/ <user_profile_ref> instructions; point LLM at <user_reputation> instead. - aiAnalyzer: gate userProfileLearner behind AI_USER_PROFILE_LEARNING_ENABLED (default false) — generates profiles nobody reads, pure LLM/DB waste. - Add AI_USER_PROFILE_LEARNING_ENABLED config knob. Net: smaller prompts (more messages fit per request), fewer DB round-trips per sub-batch, and no background LLM calls learning unused profiles. tsc, biome, vitest (129) all clean.
This commit is contained in:
asepharyana
@@ -136,9 +136,11 @@ export function startPendingAIAnalysisWorker(
|
||||
import("./cultureLearner.js")
|
||||
.then((m) => m.startCultureLearnerWorker())
|
||||
.catch(console.error);
|
||||
if (config.AI_USER_PROFILE_LEARNING_ENABLED) {
|
||||
import("./userProfileLearner.js")
|
||||
.then((m) => m.startUserProfileLearnerWorker())
|
||||
.catch(console.error);
|
||||
}
|
||||
|
||||
setInterval(() => {
|
||||
// [D] Periodic cache hygiene: purge expired moderation verdicts from
|
||||
|
||||
@@ -16,10 +16,8 @@ import { getChannelCulture } from "./channelCultureStore.js";
|
||||
import type { RetryState } from "./llmCaller.js";
|
||||
import { callModerationLLM } from "./llmCaller.js";
|
||||
import { prepareMediaMessage } from "./mediaAnalysisClient.js";
|
||||
import { buildUserProfilesBlock } from "./moderationBuilders.js";
|
||||
import { buildSystemPrompt as buildSystemPromptModular } from "./moderationPrompt.js";
|
||||
import { buildCorrectedFewShotExamples } from "./textBatchProcessor.js";
|
||||
import { getUserProfile } from "./userProfileStore.js";
|
||||
|
||||
const log = createChildLogger("mediaBatchProcessor");
|
||||
|
||||
@@ -65,32 +63,15 @@ export async function runMediaBatch(
|
||||
channelCulture,
|
||||
});
|
||||
|
||||
// Gather user profiles ONCE for the whole batch and emit a deduplicated
|
||||
// <user_profiles> map (with last-generated timestamp); per-message blocks
|
||||
// (from prepareMediaMessage) reference it via <user_profile_ref>.
|
||||
const profileByUser = new Map<
|
||||
string,
|
||||
{
|
||||
text: string;
|
||||
asOf?: number | null;
|
||||
}
|
||||
>();
|
||||
for (const t of targets) {
|
||||
if (profileByUser.has(t.user_id)) continue;
|
||||
const profile = await getUserProfile(t.user_id);
|
||||
profileByUser.set(t.user_id, {
|
||||
text: profile?.profile_summary ?? "",
|
||||
asOf: profile?.last_analyzed_at ?? null,
|
||||
});
|
||||
}
|
||||
const userProfilesBlock = buildUserProfilesBlock(profileByUser);
|
||||
|
||||
// Per-message blocks (from prepareMediaMessage) carry their own
|
||||
// <user_reputation> history; personal profile descriptions are omitted
|
||||
// (they bloat the prompt and add a per-user DB round-trip for little
|
||||
// moderation signal — see textBatchProcessor).
|
||||
const messagesBlock = prepared.map((p) => p.messageBlock).join("\n");
|
||||
// Data/instruction separation: the system prompt is stable per mode — all
|
||||
// per-batch context (profiles, conversation) lives in the USER payload,
|
||||
// ordered oldest-first so targets come last.
|
||||
// per-batch context (conversation) lives in the USER payload, ordered
|
||||
// oldest-first so targets come last.
|
||||
const userBlocks = [
|
||||
userProfilesBlock?.trimEnd() ?? "",
|
||||
contextBlock?.trimEnd() ?? "",
|
||||
`<messages_to_analyze>\n${messagesBlock}\n</messages_to_analyze>`,
|
||||
].filter((b) => b.trim().length > 0);
|
||||
|
||||
@@ -35,8 +35,8 @@ Instruksi per field:
|
||||
- "message_id": WAJIB sama persis dengan id di input. Setiap <message> di <messages_to_analyze> menghasilkan SATU hasil. Jangan gabungkan beberapa pesan, jangan lewati, jangan karang id.
|
||||
- "evidence": kutipan PERSIS frasa yang melanggar (maks 1 baris). Pelanggaran di gambar/sticker → kutip deskripsi Media analysis. Pelanggaran lewat balasan/referensi → sebut konteks pesan yang dibalas. Boleh tambah label sumber, mis. [media analysis] / [web_search] / [reply]. Kosong jika clean.
|
||||
|
||||
## PERSONALITY & MEMORI — Profil Pengguna dan Kultur Channel
|
||||
Data konteks tersedia: <user_profiles> (peta ringkasan kepribadian, di pesan USER), <user_reputation> (skor trust), dan <channel_culture> (topik/vibe channel). Setiap <message> dapat memuat <user_profile_ref user_id=".../> yang menunjuk ke entri di peta <user_profiles>.
|
||||
## PERSONALITY & MEMORI — Reputasi Pengguna dan Kultur Channel
|
||||
Data konteks tersedia: <user_reputation> (skor trust + histori infraction + repeat_offender), dan <channel_culture> (topik/vibe channel).
|
||||
Gunakan untuk personalisasi analysis, tapi:
|
||||
- Profil/history adalah KONTEKS, bukan bukti. Profil mencurigakan ≠ flag; profil bersih ≠ loloskan pelanggaran. <user_history> (kutipan pesan pernah di-flag) = cari POLA berulang (spam link SAMA, provokasi berulang konten SAMA); JANGAN gunakan untuk "menginterpretasi ulang" pesan bersih yang terpisah. Pesan baru tanpa pola pengulangan jelas → CLEAN.
|
||||
- Perubahan perilaku mencolok (mis. teknis tiba-tiba provokatif) layak dicatat. JANGAN paksa referensi profil jika tidak relevan — analysis natural lebih baik.
|
||||
@@ -70,8 +70,8 @@ CRITICAL:
|
||||
- Jika pesan adalah BALASAN (reply) ke pesan lain, jelaskan konteks balasannya: apa yang sedang dibicarakan, siapa yang dibalas (tanpa nama, cukup peran/isi pesan yang dibalas), dan bagaimana tanggapan pengirim terhadapnya.
|
||||
- Gunakan informasi dari Media analysis untuk mendeskripsikan gambar.
|
||||
- Analisis harus MEMBERI KONTEKS, bukan hanya menyatakan status.
|
||||
- GUNAKAN <user_profile_ref>/<user_profiles> untuk personalisasi analysis — jadikan analysis terasa seperti sistem "mengenal" pengguna.
|
||||
- Jika perilaku pesan menyimpang dari profil yang diketahui, CATAT dalam analysis sebagai informasi kontekstual yang relevan.
|
||||
- Gunakan <user_reputation> (repeat_offender, last_offense_days_ago) untuk memberi konteks histori — analisis terasa seperti sistem "mengenal" histori pengguna tanpa deskripsi profil pribadi.
|
||||
- Jika perilaku pesan menyimpang dari pola histori yang diketahui, CATAT dalam analysis sebagai informasi kontekstual yang relevan.
|
||||
- JANGAN paksa referensi profil jika tidak relevan — analysis natural lebih baik dari yang dipaksakan.`;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
@@ -107,7 +107,7 @@ export function buildSystemPrompt(options: BuildSystemPromptOptions): string {
|
||||
`System prompt ini TIDAK berisi data batch — semua data per-batch ada di pesan USER:\n` +
|
||||
`- <location_context .../>: metadata channel/thread (channel_name, thread_name, topic, nsfw, age_restricted). topic = tujuan resmi channel; gunakan menilai kesesuaian pesan.\n` +
|
||||
`- <conversation_context>: obrolan SEBELUM target. Baris pertama "[conversation_flow] status=... context_msgs=... dropped=..." = metadata sistem (ongoing/sparse/cold_start), BUKAN pesan dinilai. Baris "[context] id=... time=... user=...: isi" = konteks, BUKAN target.\n` +
|
||||
`- <user_profiles>/<user_profile_ref>: peta ringkasan kepribadian per user_id (as_of = kapan dibuat; profil lama mungkin usang). <user_reputation trust_score total_infractions clean_streak last_offense_days_ago repeat_offender>: histori moderasi (repeat_offender=true = pelanggaran ≤7 hari). <user_history>: kutipan pesan pernah di-flag — cari POLA berulang (spam link sama), BUKAN bukti pesan bersih.\n` +
|
||||
`- <user_reputation trust_score total_infractions clean_streak last_offense_days_ago repeat_offender>: histori moderasi (repeat_offender=true = pelanggaran ≤7 hari). <user_history>: kutipan pesan pernah di-flag — cari POLA berulang (spam link sama), BUKAN bukti pesan bersih.\n` +
|
||||
`- <web_searches>/<web_content>: bukti web (prioritas tertinggi). <term_glossary>: definisi kata/slang/jargon (SearXNG) — pakai pahami kata asing, JANGAN tebak arti.\n` +
|
||||
`- <messages_to_analyze>: pesan TARGET yang WAJIB dinilai. Atribut <message>: id, user, time (ISO), repetitions (N = teks sama muncul N× di batch → sinyal spam), bot (true = bot), edited (true = hasil edit setelah posting → evasi potensial).`,
|
||||
);
|
||||
|
||||
@@ -19,8 +19,6 @@ import { callModerationLLM } from "./llmCaller.js";
|
||||
import { analyzeSingleMediaImage } from "./mediaAnalysisClient.js";
|
||||
import {
|
||||
buildReferenceXml,
|
||||
buildUserProfileRef,
|
||||
buildUserProfilesBlock,
|
||||
escapeXml,
|
||||
formatReputationAttrs,
|
||||
getAnalysisContent,
|
||||
@@ -39,7 +37,6 @@ import {
|
||||
import { buildTermGlossaryBlock } from "./termGlossary.js";
|
||||
import { getRecentCorrectedModerations } from "./textCacheStore.js";
|
||||
import { extractUrlsFromText, fetchUrlSafely } from "./urlFetcher.js";
|
||||
import { getUserProfile } from "./userProfileStore.js";
|
||||
import { initializeUserReputation } from "./userReputationStore.js";
|
||||
import type { MessageImagePart } from "./visionAnalyzer.js";
|
||||
|
||||
@@ -204,40 +201,24 @@ export async function runTextOnlyBatch(
|
||||
const batch = subBatches[i];
|
||||
const targetIds = batch.map((t) => t.id);
|
||||
|
||||
// User reputation + profiles (raw summary text — deduplicated into a
|
||||
// single <user_profiles> map per batch; messages only reference it).
|
||||
const userContexts = new Map<string, string>();
|
||||
const userProfiles = new Map<
|
||||
string,
|
||||
{
|
||||
text: string;
|
||||
asOf?: number | null;
|
||||
}
|
||||
>();
|
||||
// ── Per-user reputation + profile context (fetched ONCE per unique user,
|
||||
// in parallel — was a serial per-message loop that cost ~2N sequential
|
||||
// DB/Redis round-trips per sub-batch and dominated latency on small
|
||||
// batches). ─────────────────────────────────────────────────────────
|
||||
// ── Per-user reputation context (fetched ONCE per unique user, in
|
||||
// parallel). Personal profile descriptions are intentionally NOT
|
||||
// injected — they bloat the prompt (less room per request) and add a
|
||||
// per-user DB/Redis round-trip for little moderation signal. Only the
|
||||
// behavioural <user_reputation> history is sent. ─────────────────────
|
||||
const uniqueUserIds = [...new Set(batch.map((m) => m.user_id))];
|
||||
const batchGuildId = batch[0]?.guild_id ?? "";
|
||||
const userFetches = await Promise.all(
|
||||
uniqueUserIds.map(async (uid) => {
|
||||
const [rep, profile] = await Promise.all([
|
||||
initializeUserReputation(uid, batchGuildId),
|
||||
getUserProfile(uid),
|
||||
]);
|
||||
return { uid, rep, profile };
|
||||
const rep = await initializeUserReputation(uid, batchGuildId);
|
||||
return { uid, rep };
|
||||
}),
|
||||
);
|
||||
for (const { uid, rep, profile } of userFetches) {
|
||||
const userContexts = new Map<string, string>();
|
||||
for (const { uid, rep } of userFetches) {
|
||||
const repAttrs = formatReputationAttrs(rep);
|
||||
userContexts.set(uid, `<user_reputation ${repAttrs}/>`);
|
||||
userProfiles.set(uid, {
|
||||
text: profile?.profile_summary ?? "",
|
||||
asOf: profile?.last_analyzed_at ?? null,
|
||||
});
|
||||
}
|
||||
const userProfilesBlock = buildUserProfilesBlock(userProfiles);
|
||||
|
||||
// ── URL images → multimodal vision evidence ─────────────────────────
|
||||
// The text batch fetches inline URLs; whenever one resolved to an image
|
||||
@@ -335,16 +316,11 @@ export async function runTextOnlyBatch(
|
||||
.map((line) => `\n${line}`)
|
||||
.join("");
|
||||
const userCtx = userContexts.get(msg.user_id) ?? "";
|
||||
const userProfileRef = (
|
||||
userProfiles.get(msg.user_id)?.text ?? ""
|
||||
).trim()
|
||||
? buildUserProfileRef(msg.user_id)
|
||||
: "";
|
||||
const refXml = await buildReferenceXml(msg);
|
||||
const repetitionCount = groupMapping.get(msg.id)?.length ?? 1;
|
||||
const isBot = resolveIsBot(msg);
|
||||
const isEdited = resolveIsEdited(msg);
|
||||
return `<message id="${escapeXml(msg.id)}" user="${escapeXml(resolveDisplayName(msg))}" time="${new Date(msg.created_at).toISOString()}"${repetitionCount > 1 ? ` repetitions="${repetitionCount}"` : ""}${isBot ? ` bot="true"` : ""}${isEdited ? ` edited="true"` : ""}>\n ${userCtx}${userProfileRef ? `\n ${userProfileRef}` : ""}${refXml ? `\n ${refXml}` : ""}\n <content>${escapeXml(content)}</content>${webContext}${mediaEvidenceCtx}\n</message>`;
|
||||
return `<message id="${escapeXml(msg.id)}" user="${escapeXml(resolveDisplayName(msg))}" time="${new Date(msg.created_at).toISOString()}"${repetitionCount > 1 ? ` repetitions="${repetitionCount}"` : ""}${isBot ? ` bot="true"` : ""}${isEdited ? ` edited="true"` : ""}>\n ${userCtx}${refXml ? `\n ${refXml}` : ""}\n <content>${escapeXml(content)}</content>${webContext}${mediaEvidenceCtx}\n</message>`;
|
||||
}),
|
||||
)
|
||||
).join("\n");
|
||||
@@ -359,10 +335,10 @@ export async function runTextOnlyBatch(
|
||||
.join("\n")}\n</web_searches>`
|
||||
: "";
|
||||
// Data/instruction separation: the system prompt is stable per mode —
|
||||
// all per-batch context (profiles, conversation, web evidence) lives in
|
||||
// the USER payload, ordered oldest-first so targets come last.
|
||||
// all per-batch context (conversation, web evidence) lives in the USER
|
||||
// payload, ordered oldest-first so targets come last. Personal user
|
||||
// profile descriptions are intentionally omitted (see above).
|
||||
const userBlocks = [
|
||||
userProfilesBlock?.trimEnd() ?? "",
|
||||
contextBlock?.trimEnd() ?? "",
|
||||
searxngBlock,
|
||||
glossaryBlock,
|
||||
|
||||
@@ -63,7 +63,6 @@ import {
|
||||
} from "./mediaDownloader.js";
|
||||
import {
|
||||
buildReferenceXml,
|
||||
buildUserProfileRef,
|
||||
escapeXml,
|
||||
formatReputationAttrs,
|
||||
getAnalysisContent,
|
||||
@@ -85,7 +84,6 @@ import {
|
||||
} from "./searxngSearch.js";
|
||||
import { buildTermGlossaryBlock } from "./termGlossary.js";
|
||||
import { extractUrlsFromText } from "./urlFetcher.js";
|
||||
import { getUserProfile } from "./userProfileStore.js";
|
||||
import { initializeUserReputation } from "./userReputationStore.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -401,21 +399,14 @@ export async function prepareMediaMessage(
|
||||
.join(" ");
|
||||
|
||||
const rep = await initializeUserReputation(target.user_id, target.guild_id);
|
||||
const profile = await getUserProfile(target.user_id);
|
||||
const refXml = await buildReferenceXml(target);
|
||||
// Profile is emitted ONCE per batch in a <user_profiles> map (see
|
||||
// mediaBatchProcessor); here we only reference it to avoid repeating the
|
||||
// full summary on every message of the same user.
|
||||
const profileRef = profile?.profile_summary?.trim()
|
||||
? buildUserProfileRef(target.user_id)
|
||||
: "";
|
||||
|
||||
// Rich reputation — attrs only, no user history injection (per channel context preference)
|
||||
// Only the behavioural <user_reputation> history is injected; personal profile descriptions are omitted (see textBatchProcessor).
|
||||
const repAttrs = formatReputationAttrs(rep);
|
||||
const repXml = `<user_reputation ${repAttrs}/>`;
|
||||
|
||||
const isBot = resolveIsBot(target);
|
||||
const isEdited = resolveIsEdited(target);
|
||||
const messageBlock = `<message id="${escapeXml(target.id)}" user="${escapeXml(resolveDisplayName(target))}" time="${new Date(target.created_at).toISOString()}"${isBot ? ` bot="true"` : ""}${isEdited ? ` edited="true"` : ""}>\n ${repXml}${profileRef ? `\n ${profileRef}` : ""}${refXml ? `\n ${refXml}` : ""}\n <content>${escapeXml(truncateForAi(content))}</content>${mediaContext ? ` ${escapeXml(mediaContext)}` : ""}${webContext}${mediaAnalysisContext}${searxngXml}${glossaryCtx}\n</message>`;
|
||||
const messageBlock = `<message id="${escapeXml(target.id)}" user="${escapeXml(resolveDisplayName(target))}" time="${new Date(target.created_at).toISOString()}"${isBot ? ` bot="true"` : ""}${isEdited ? ` edited="true"` : ""}>\n ${repXml}${refXml ? `\n ${refXml}` : ""}\n <content>${escapeXml(truncateForAi(content))}</content>${mediaContext ? ` ${escapeXml(mediaContext)}` : ""}${webContext}${mediaAnalysisContext}${searxngXml}${glossaryCtx}\n</message>`;
|
||||
return { targetId, messageBlock };
|
||||
}
|
||||
|
||||
@@ -217,6 +217,14 @@ export const configSchema = z
|
||||
.default(true),
|
||||
// Max glossary terms looked up per analysis batch (keeps latency bounded).
|
||||
AI_GLOSSARY_MAX_TERMS: z.coerce.number().int().min(1).max(20).default(6),
|
||||
// Per-user personal profile summaries (userProfileLearner). Disabled by
|
||||
// default: profiles bloat the analysis context and add LLM/DB cost for
|
||||
// little moderation signal — only <user_reputation> history is injected.
|
||||
AI_USER_PROFILE_LEARNING_ENABLED: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v === "true")
|
||||
.default(false),
|
||||
// Min word length for a term to be considered glossary-worthy.
|
||||
AI_GLOSSARY_MIN_WORD_LENGTH: z.coerce
|
||||
.number()
|
||||
|
||||
Reference in New Issue
Block a user