fix(chatbot): proper system role, tool catalog prompt, more rounds/history, tool failure observability
- Send system prompt as role:system instead of merging into the user message (models treat system messages as authoritative -> better tool selection) - Add full 14-tool catalog + multi-hop instruction to the system prompt - Raise MAX_TOOL_ROUNDS 4->6 so multi-hop queries complete - Raise conversation history 3->8 exchanges (less repeat questioning) - Append guild/channel scope hint to each user turn - Add saturating toolErrors counter + structured warn log per tool failure so chatbot tool failures are observable instead of silently absorbed
This commit is contained in:
@@ -64,7 +64,7 @@ class ChatbotService {
|
|||||||
private async getRecentConversationContext(
|
private async getRecentConversationContext(
|
||||||
userId: string,
|
userId: string,
|
||||||
): Promise<string[]> {
|
): Promise<string[]> {
|
||||||
const history = await chatbotRepository.getChatHistory(userId, 3);
|
const history = await chatbotRepository.getChatHistory(userId, 8);
|
||||||
return history.flatMap((row) => [
|
return history.flatMap((row) => [
|
||||||
`User: ${row.user_message}`,
|
`User: ${row.user_message}`,
|
||||||
`Bot: ${row.bot_response}`,
|
`Bot: ${row.bot_response}`,
|
||||||
@@ -82,11 +82,27 @@ class ChatbotService {
|
|||||||
|
|
||||||
${scopeLine}
|
${scopeLine}
|
||||||
|
|
||||||
|
TOOLS YANG TERSEDIA (panggil saat user tanya soal data server):
|
||||||
|
- get_server_stats → statistik server: total pesan, user aktif, flagged/warn/clean
|
||||||
|
- get_top_channels → channel paling aktif (jumlah pesan terbanyak)
|
||||||
|
- get_recent_activity → pesan terbaru (siapa, channel mana, jam berapa, isinya)
|
||||||
|
- get_top_flagged → pesan yang di-flag AI (alasan, severity, analysis)
|
||||||
|
- search_messages → cari pesan berdasarkan kata kunci (LIKE search)
|
||||||
|
- get_user_messages → pesan terbaru dari satu user tertentu
|
||||||
|
- get_user_profile → profil AI dari seorang user (pola perilaku, gaya bicara)
|
||||||
|
- get_channel_culture → norma/slang dari suatu channel
|
||||||
|
- get_message_detail → 1 pesan lengkap beserta hasil analisis AI-nya
|
||||||
|
- get_message_reviews → antrean review moderasi manual
|
||||||
|
- get_voice_recordings → rekaman suara terbaru
|
||||||
|
- get_moderation_timeline → tren harian: total vs flagged vs warn vs clean
|
||||||
|
- get_corrections → riwayat koreksi false-positive AI
|
||||||
|
|
||||||
ATURAN PENTING — JANGAN PAKAI KONTEKS STATIS:
|
ATURAN PENTING — JANGAN PAKAI KONTEKS STATIS:
|
||||||
- Kamu TIDAK punya hafalan soal angka server (jumlah pesan, user aktif, flagged, dll). JANGAN tebak atau karang angka.
|
- Kamu TIDAK punya hafalan soal angka server. JANGAN tebak atau karang angka.
|
||||||
- Untuk SEMUA pertanyaan soal data server (jumlah pesan, user aktif, channel ramai, aktivitas terbaru, pesan di-flag), WAJIB panggil tool yang sesuai (get_server_stats, get_top_channels, get_recent_activity, get_top_flagged). Jawab HANYA dari hasil tool.
|
- Untuk SEMUA pertanyaan soal data server, WAJIB panggil tool yang sesuai. Jawab HANYA dari hasil tool.
|
||||||
- Tool otomatis di-scope ke guild/channel di atas — kalau argumen guildId/channelId kosong, biarkan kosong (sudah otomatis ter-isi). Jangan isi ID yang kamu tebak.
|
- Tool otomatis di-scope ke guild/channel — kalau argumen kosong, biarkan kosong. Jangan isi ID tebakan.
|
||||||
- Kalau tool balas error atau kosong, bilang aja data lagi ga ketemu, jangan karang.
|
- Kalau tool balik error atau kosong, bilang aja data lagi ga ketemu, jangan karang.
|
||||||
|
- Boleh panggil banyak tool dalam satu jawaban kalau pertanyaan butuh beberapa data (multi-hop).
|
||||||
|
|
||||||
Gaya ngobrol:
|
Gaya ngobrol:
|
||||||
- Santai, hangat, kayak ngobrol sama temen
|
- Santai, hangat, kayak ngobrol sama temen
|
||||||
@@ -127,27 +143,31 @@ Gaya ngobrol:
|
|||||||
try {
|
try {
|
||||||
const { default: axios } = await import("axios");
|
const { default: axios } = await import("axios");
|
||||||
|
|
||||||
// Gateway tidak handle role system — gabung konteks ke user message.
|
const scopeHint = scope.guildId
|
||||||
// The system section stays visible to the model as the first user turn.
|
? ` (scope: guild ${scope.guildId}${scope.channelId ? `, channel ${scope.channelId}` : ""})`
|
||||||
const contextPrefixed = `${systemPrompt}\n\nPertanyaan user: ${userMessage}`;
|
: "";
|
||||||
|
// Local message type that models the shapes the agent loop emits and
|
||||||
// Seed conversation: prior turns + current question.
|
// accepts: plain system/user/assistant turns and assistant tool-calls +
|
||||||
const messages: Array<
|
// tool results. role is a union string so history entries typed as
|
||||||
| { role: "user" | "assistant"; content: string }
|
// `"user" | "assistant"` don't break the discriminant.
|
||||||
| {
|
type ChatMsg = {
|
||||||
role: "assistant";
|
role: "system" | "user" | "assistant" | "tool";
|
||||||
content: string | null;
|
content?: string | null;
|
||||||
tool_calls: Array<{
|
tool_call_id?: string;
|
||||||
id: string;
|
tool_calls?: Array<{
|
||||||
type: "function";
|
id: string;
|
||||||
function: { name: string; arguments: string };
|
type: "function";
|
||||||
}>;
|
function: { name: string; arguments: string };
|
||||||
}
|
}>;
|
||||||
| { role: "tool"; tool_call_id: string; content: string }
|
};
|
||||||
> = [...history, { role: "user", content: contextPrefixed }];
|
const messages: ChatMsg[] = [
|
||||||
|
{ role: "system", content: systemPrompt },
|
||||||
|
...history,
|
||||||
|
{ role: "user", content: `${userMessage}${scopeHint}` },
|
||||||
|
];
|
||||||
|
|
||||||
// ── Agentic tool loop ─────────────────────────────────────────
|
// ── Agentic tool loop ─────────────────────────────────────────
|
||||||
const MAX_TOOL_ROUNDS = 4;
|
const MAX_TOOL_ROUNDS = 6;
|
||||||
for (let round = 0; round <= MAX_TOOL_ROUNDS; round += 1) {
|
for (let round = 0; round <= MAX_TOOL_ROUNDS; round += 1) {
|
||||||
const response = await axios.post(
|
const response = await axios.post(
|
||||||
`${baseUrl}/chat/completions`,
|
`${baseUrl}/chat/completions`,
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { and, desc, eq, like, sql } from "drizzle-orm";
|
import { and, desc, eq, like, sql } from "drizzle-orm";
|
||||||
|
import { createChildLogger } from "@/shared/logger/index";
|
||||||
import { getDatabase } from "../../shared/database/index.js";
|
import { getDatabase } from "../../shared/database/index.js";
|
||||||
import {
|
import {
|
||||||
pgChannelCulturesTable,
|
pgChannelCulturesTable,
|
||||||
@@ -19,6 +20,20 @@ import {
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
export type ToolResult = string;
|
export type ToolResult = string;
|
||||||
|
|
||||||
|
const logger = createChildLogger("chatbot.tools");
|
||||||
|
|
||||||
|
/** Saturating counter of chatbot tool execution errors (for observability). */
|
||||||
|
export let toolErrors = 0;
|
||||||
|
const TOOL_ERROR_CAP = 1000;
|
||||||
|
|
||||||
|
function toolExecError(err: unknown, name: string): string {
|
||||||
|
if (toolErrors < TOOL_ERROR_CAP) toolErrors++;
|
||||||
|
const detail = (err as Error)?.message ?? "unknown";
|
||||||
|
logger.warn({ tool: name, error: detail }, "Chatbot tool execution failed");
|
||||||
|
return `Tool ${name} gagal: ${detail}`;
|
||||||
|
}
|
||||||
|
|
||||||
/** Executes a tool call against the real DB and returns a readable result. */
|
/** Executes a tool call against the real DB and returns a readable result. */
|
||||||
export async function executeTool(
|
export async function executeTool(
|
||||||
name: string,
|
name: string,
|
||||||
@@ -90,7 +105,7 @@ export async function executeTool(
|
|||||||
}
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
// Best-effort: if a tool fails, return readable error instead of crashing
|
// Best-effort: if a tool fails, return readable error instead of crashing
|
||||||
return `Terjadi kesalahan saat ambil data: ${(error as Error).message ?? "unknown"}`;
|
return toolExecError(error, name);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user