fix(gateway): separate text and media lanes in AI analysis queue (#86)

Image messages previously blocked the whole analysis pipeline:
- conversationProcessing was a single lock per conversation; processBatch
  awaited BOTH text and media worker jobs before releasing it, so a fast
  text verdict sat unused until the slow vision/media batch finished
- one global LLM semaphore (AI_LLM_MAX_CONCURRENT) was shared by text and
  media, so a vision backlog could starve text inference
- recovery worker gated on conversationProcessing.size

Now the queue is split into independent text/media lanes:
- conversationProcessing maps key -> Partial<Record<lane, startedAt>>;
  each lane holds its own lock and frees it the moment ITS worker job
  resolves (ownership-guarded clear prevents stale timers clearing newer
  slots)
- two LLM semaphores: AI_LLM_MAX_CONCURRENT (text, default 8) and
  AI_LLM_MEDIA_MAX_CONCURRENT (media, default 4) via
  withLlmConcurrency(fn, { lane })
- batchScheduler schedules per conversation+lane (timer keys
  '<key>::<lane>'); splitMessagesByLane/laneOfMessage moved to pure
  analysisLanes.ts (unit-testable without Piscina)
- ai-analysis-worker batch jobs carry a lane field; per-lane active
  request gauges (active_text_requests / active_media_requests)
- added tests/analysisLaneLock.test.ts (7 tests: independent lane locks,
  preserving other-lane lock, clear-all, ownership guard, lane split)

Docs: ARCHITECTURE.md + AGENTS.md concurrency model updated.
typecheck/lint/test(138)/build all green.
This commit is contained in:
asepharyana authored and GitHub committed 2026-09-24 14:09:53 +07:00
1 parent f9fecfc144
commit e34dcd6bc6
15 files changed
+692 -205

No files matched your search

@@ -194,6 +194,12 @@ export const configSchema = z
QDRANT_ARCHIVE_COLLECTION: z.string().default("gmw_message_archive"),
QDRANT_API_KEY: z.string().optional(),
AI_LLM_MAX_CONCURRENT: z.coerce.number().int().positive().default(8),
// Media-lane LLM concurrency cap (2026-09-24): vision + media-batch calls
// use their OWN semaphore instead of sharing AI_LLM_MAX_CONCURRENT, so a
// slow image backlog can never consume the text lane's concurrency slots.
// Default 4 keeps media churn from saturating the router; text inference
// keeps its full AI_LLM_MAX_CONCURRENT (default 8) regardless.
AI_LLM_MEDIA_MAX_CONCURRENT: z.coerce.number().int().positive().default(4),
AI_LLM_IMAGE_MAX_DIMENSION: z.coerce
.number()
.int()