fix(gateway): separate text and media lanes in AI analysis queue
Image messages previously blocked the whole analysis pipeline:
- conversationProcessing was a single lock per conversation; processBatch
awaited BOTH text and media worker jobs before releasing it, so a fast
text verdict sat unused until the slow vision/media batch finished
- one global LLM semaphore (AI_LLM_MAX_CONCURRENT) was shared by text and
media, so a vision backlog could starve text inference
- recovery worker gated on conversationProcessing.size
Now the queue is split into independent text/media lanes:
- conversationProcessing maps key -> Partial<Record<lane, startedAt>>;
each lane holds its own lock and frees it the moment ITS worker job
resolves (ownership-guarded clear prevents stale timers clearing newer
slots)
- two LLM semaphores: AI_LLM_MAX_CONCURRENT (text, default 8) and
AI_LLM_MEDIA_MAX_CONCURRENT (media, default 4) via
withLlmConcurrency(fn, { lane })
- batchScheduler schedules per conversation+lane (timer keys
'<key>::<lane>'); splitMessagesByLane/laneOfMessage moved to pure
analysisLanes.ts (unit-testable without Piscina)
- ai-analysis-worker batch jobs carry a lane field; per-lane active
request gauges (active_text_requests / active_media_requests)
- added tests/analysisLaneLock.test.ts (7 tests: independent lane locks,
preserving other-lane lock, clear-all, ownership guard, lane split)
Docs: ARCHITECTURE.md + AGENTS.md concurrency model updated.
typecheck/lint/test(138)/build all green.
This commit is contained in:
1 parent
f9fecfc144
commit
6d0d7b34a3
15 files changed
+692
-205
No files matched your search
@@ -69,8 +69,14 @@ textBatchProcessor.ts mediaBatchProcessor.ts llmClient.ts
|
||||
```
|
||||
|
||||
- Entry: `aiAnalyzer.ts` (`queueMessageAnalysis`, `startPendingAIAnalysisWorker`)
|
||||
- Concurrency: LLM semaphore (`AI_LLM_MAX_CONCURRENT`, default 5)
|
||||
- Concurrency: **two per-lane LLM semaphores** (2026-09-24) — text
|
||||
(`AI_LLM_MAX_CONCURRENT`, default 8) and media/vision
|
||||
(`AI_LLM_MEDIA_MAX_CONCURRENT`, default 4); a media backlog can never
|
||||
consume text slots
|
||||
- Piscina: text pool (4 threads) + media pool (2 threads)
|
||||
- Locks are **per conversation per lane** (`conversationProcessing` maps key →
|
||||
lane → startedAt): the text lane of a conversation never waits on that
|
||||
conversation's media lane (this was the "image blocks the queue" bug)
|
||||
- **Each worker thread has its own pg Pool** (min 0, grows to `POSTGRES_POOL_MAX`)
|
||||
|
||||
## Module: message-capture
|
||||
|
||||
Reference in new issue
Block a user