perf+fix(ai-moderation): 11 pipeline optimizations from audit
Audit of the full AI analysis flow found 14 issues; 11 fixed, 3 deferred:
Fixed:
1. batchProcessor: skip scheduleAutoDelete for error-status rows (was
causing wasted not_eligible logs for every parse/API failure)
2. textBatchProcessor: domain dedup in URL fetch (max 3 URLs per domain
to avoid rate-limiting from concentrated domains)
3. llmCaller: move parseModerationResponse import to top-level (was
dynamic-imported inside retry loop — unnecessary overhead per retry)
4. llmCaller: make default max_tokens configurable via
AI_LLM_MAX_COMPLETION_TOKENS env (default 16384)
5. moderationOrchestrator: log cache write errors instead of silent
.catch(() => {}) — surface intermittent Redis failures
6. conversationContext: batch token estimation via estimateTokensBatch
(single tiktoken encode call for all target lines, ~5x faster)
7. aiAnalyzer: skip revertStuckProcessingMessages DB query when no
conversations are actively processing (avoids idle-state query)
8. textBatchProcessor: cache corrected few-shot examples per hour
(was re-queried from DB on every batch)
9. textBatchProcessor: preserve partial results on sub-batch timeout
(was throwing and discarding all prior sub-batch results)
10. batchProcessor switch: skip 'completed' messages from individual
fallback queue (prevents redundant re-analysis + double-delete)
11. autoDeleteManager: expand isAlreadyDeletedError to catch Discord
codes 10003/50001 + text fallback matching
Deferred (not regressions, larger refactors):
- #8 batchScheduler debounce race: not actually a race (JS single-threaded)
- #11 initCacheStore: already has idempotency guard
- #13 individual fallback batching: requires worker pool refactor
7 files changed, 73 insertions(+), 32 deletions(-)
This commit is contained in:
1 parent
5fc3cf86d5
commit
9f02edd646
7 files changed
+73
-32
No files matched your search
@@ -16,6 +16,7 @@ import { delay, retryWithBackoff } from "@/shared/utils/index";
|
||||
import { config } from "../../shared/config/config.js";
|
||||
import type { AnalysisResult } from "../message-capture/types.js";
|
||||
import { llmChat } from "./llmClient.js";
|
||||
import { parseModerationResponse } from "./moderationResponseParser.js";
|
||||
import { logModerationError } from "./responseLogger.js";
|
||||
|
||||
const log = createChildLogger("llm-caller");
|
||||
@@ -79,7 +80,7 @@ export async function callModerationLLM(
|
||||
];
|
||||
const completion = await llmChat({
|
||||
messages,
|
||||
max_tokens: maxTokens ?? 16384,
|
||||
max_tokens: maxTokens ?? config.AI_LLM_MAX_COMPLETION_TOKENS ?? 16384,
|
||||
jsonResponse: { type: "json_object" },
|
||||
retries: 0,
|
||||
signal,
|
||||
@@ -106,9 +107,6 @@ export async function callModerationLLM(
|
||||
if (!rawContent) throw new Error("No content in LLM response");
|
||||
|
||||
try {
|
||||
const { parseModerationResponse } = await import(
|
||||
"./moderationResponseParser.js"
|
||||
);
|
||||
return {
|
||||
parsed: parseModerationResponse(rawContent, targetIds),
|
||||
result: completion,
|
||||
|
||||
Reference in new issue
Block a user