perf(ai-moderation): compact system prompt + memoize build + hoist vision pass
- Memoize buildSystemPrompt by (mode|channelCulture); identical signatures now reuse the ~5k-token core instead of rebuilding per sub-batch call (textBatchProcessor rebuilt it inside the loop; a 200-msg batch re-sent the full system prompt ~4x). Correction tail stays per-attempt (uncached). - Hoist URL-image -> vision evidence out of the per-sub-batch loop in textBatchProcessor: it depends only on fetched images + full target set, so compute once per whole batch, not per sub-batch. - Compact system instructions: collapse 3x-duplicated 'evaluate by content alone' statements into one standalone rule; trim output.ts channel-culture + context framing already covered by rules.ts/system.ts; drop duplicate programming-error-log few-shot (id 17, covered by rules AMAN list). - Fix misleading config default: AI_LLM_BASE_URL default -> omniroute (gateway already runs omniroute via BWS; 9router was dead/misleading). typecheck + lint + build green.
This commit is contained in:
@@ -149,7 +149,7 @@ export const configSchema = z
|
||||
AI_LLM_BASE_URL: z
|
||||
.string()
|
||||
.url()
|
||||
.default("https://9router.asepharyana.my.id/v1"),
|
||||
.default("https://omniroute.imrnes.team/v1"),
|
||||
AI_LLM_MODEL: z.string().default("text"),
|
||||
// Vision uses the SAME router/base URL as text moderation
|
||||
// (AI_LLM_BASE_URL) but a different model alias. The dedicated NVIDIA
|
||||
|
||||
Reference in New Issue
Block a user