feat(ai): scope persistence, parallel tools, Prometheus token/cache counters

C2 full scope persistence:
- getRecentConversationContext now returns per-turn guildId/channelId
- processMessage merges historical scope when current request is unscoped
- chatbot remembers server context across all 8 history exchanges (not just 3)

A4+Metrics:
- Add incrementCounterBy(name, delta, labels) to gateway-metrics
- Token-usage counters: llm_tokens_total{model, type, label} per batch
- Cache hit counters: moderation_cache_hits{type} exact/semantic-qdrant/semantic-pg
- Cache miss counters: moderation_cache_misses per batch
- Prometheus /metrics now exposes cost + cache hit-rate for dashboards

Performance:
- Parallel tool execution within each chatbot round (Promise.all)
- All tool results collected before sending to model (ordering preserved)
- Tool failure now logged with structured warning (chatbot.tools.ts)

Verified: backend tsc+biome 37/37, discord-gateway tsc+biome 210/210
This commit is contained in:
asepharyana authored and asepharyana committed 2026-09-04 15:32:03 +07:00
1 parent 9682abef82
commit 276062fdd4
5 files changed
+112 -40

No files matched your search

@@ -14,6 +14,7 @@ import type { ChatCompletion } from "openai/resources/chat/completions";
import { createChildLogger } from "@/shared/logger/index";
import { delay, retryWithBackoff } from "@/shared/utils/index";
import { config } from "../../shared/config/config.js";
import { incrementCounterBy } from "../gateway-metrics/index.js";
import type { AnalysisResult } from "../message-capture/types.js";
import { llmChat } from "./llmClient.js";
import { parseModerationResponse } from "./moderationResponseParser.js";
@@ -170,6 +171,23 @@ export async function callModerationLLM(
// so cost per channel/guild can be tracked (routers bill per token).
const usage = result?.usage;
if (usage && (usage.prompt_tokens || usage.completion_tokens)) {
// Prometheus token-usage counters for cost tracking per model + phase.
// Counters accumulate across scrapes so total spend per model/label is
// queryable (rate(...) dashboards).
if (usage.prompt_tokens) {
incrementCounterBy("llm_tokens_total", usage.prompt_tokens, {
model: config.AI_LLM_MODEL,
type: "prompt",
label,
});
}
if (usage.completion_tokens) {
incrementCounterBy("llm_tokens_total", usage.completion_tokens, {
model: config.AI_LLM_MODEL,
type: "completion",
label,
});
}
log.info(
{
label,