feat(ai-moderation): introduce user reputation and channel culture context

Implements a context-aware moderation system by tracking user behavior
and channel-specific norms to improve AI decision-making accuracy.

- Adds `user_reputations` table to track trust scores, clean streaks,
  and infraction history.
- Adds `channel_cultures` table to store AI-generated summaries of
  channel-specific norms and slang.
- Implements `userReputationStore` to autonomously update user scores
  based on moderation outcomes (clean vs. flagged).
- Implements `cultureLearner` and `channelCultureStore` to manage
  evolving channel contexts.
- Enhances LLM prompts to inject user reputation (trust scores,
  history) and channel culture summaries, enabling "wisdom-based"
  moderation (e.g., giving benefit of the doubt to high-trust users).
- Integrates reputation and culture updates into the existing
  `aiAnalyzer` pipeline.
This commit is contained in:
MythEclipse
2026-06-05 18:04:57 +07:00
parent f057bf1f0b
commit 2f3d7e1d61
7 changed files with 445 additions and 2 deletions
@@ -395,6 +395,10 @@ export interface BuildSystemPromptOptions {
* examples. Injected between static examples and output instructions.
*/
correctedExamples?: string;
/**
* Formatted XML block containing the AI-generated channel culture summary.
*/
channelCulture?: string;
}
export function buildSystemPrompt(options: BuildSystemPromptOptions): string {
@@ -404,6 +408,7 @@ export function buildSystemPrompt(options: BuildSystemPromptOptions): string {
includeMediaInstructions,
correction,
correctedExamples,
channelCulture,
} = options;
// Backward compatibility: if mode is not set but includeMediaInstructions is,
@@ -433,6 +438,13 @@ export function buildSystemPrompt(options: BuildSystemPromptOptions): string {
parts.push(correctedExamples);
}
// Channel Culture Injection (Learning)
if (channelCulture) {
parts.push(`## Kultur Channel (Pembelajaran AI)\n${channelCulture}`);
}
parts.push(`## Konteks Pengguna (Ingatan & Kebijaksanaan)\nSetiap pesan mungkin memiliki tag <user_reputation> dan <user_history>. *Gunakan Kebijaksanaan: Jika trust_score tinggi, beri benefit of the doubt pada ambiguitas. Jika trust_score rendah dan memiliki riwayat pelanggaran serupa, jadilah lebih tegas.*`);
parts.push(OUTPUT_INSTRUCTIONS);
// XML-delimited context — prevents prompt injection