fix(gateway): separate text and media lanes in AI analysis queue (#86)

Image messages previously blocked the whole analysis pipeline:
- conversationProcessing was a single lock per conversation; processBatch
  awaited BOTH text and media worker jobs before releasing it, so a fast
  text verdict sat unused until the slow vision/media batch finished
- one global LLM semaphore (AI_LLM_MAX_CONCURRENT) was shared by text and
  media, so a vision backlog could starve text inference
- recovery worker gated on conversationProcessing.size

Now the queue is split into independent text/media lanes:
- conversationProcessing maps key -> Partial<Record<lane, startedAt>>;
  each lane holds its own lock and frees it the moment ITS worker job
  resolves (ownership-guarded clear prevents stale timers clearing newer
  slots)
- two LLM semaphores: AI_LLM_MAX_CONCURRENT (text, default 8) and
  AI_LLM_MEDIA_MAX_CONCURRENT (media, default 4) via
  withLlmConcurrency(fn, { lane })
- batchScheduler schedules per conversation+lane (timer keys
  '<key>::<lane>'); splitMessagesByLane/laneOfMessage moved to pure
  analysisLanes.ts (unit-testable without Piscina)
- ai-analysis-worker batch jobs carry a lane field; per-lane active
  request gauges (active_text_requests / active_media_requests)
- added tests/analysisLaneLock.test.ts (7 tests: independent lane locks,
  preserving other-lane lock, clear-all, ownership guard, lane split)

Docs: ARCHITECTURE.md + AGENTS.md concurrency model updated.
typecheck/lint/test(138)/build all green.
This commit is contained in:
asepharyana authored and GitHub committed 2026-09-24 14:09:53 +07:00
1 parent f9fecfc144
commit e34dcd6bc6
15 files changed
+692 -205

No files matched your search

@@ -5,7 +5,9 @@ import type { EventBroadcaster } from "../event-broadcaster/index.js";
import { messageStore } from "../message-capture/messageStore.js";
import type { AnalysisQueueStatus } from "../message-capture/types.js";
import {
activeMediaRequests,
activeRequests,
activeTextRequests,
buildAgeRestrictedSkipResult,
buildSkipAnalysisUserResult,
isAgeRestrictedMessage,
@@ -16,6 +18,9 @@ import {
import { scheduleConversationAnalysis } from "./batchScheduler.js";
import { getConversationKey } from "./circuitBreaker.js";
import {
ANALYSIS_LANES,
type AnalysisLane,
clearConversationProcessing,
conversationConsecutiveErrors,
conversationDebounceTimers,
conversationErrorCooldown,
@@ -129,6 +134,8 @@ export function getAnalysisQueueStatus(): AnalysisQueueStatus {
return {
queuedConversations: conversationDebounceTimers.size,
activeRequests,
activeTextRequests,
activeMediaRequests,
activeIndividualRequests,
individualInFlightCount: individualInFlight.size,
individualCircuitBreakerActive: Date.now() < individualCooldownUntil,
@@ -204,9 +211,18 @@ export function startPendingAIAnalysisWorker(
for (const [key, expiry] of conversationErrorCooldown) {
if (now >= expiry) conversationErrorCooldown.delete(key);
}
for (const [key, startedAt] of conversationProcessing) {
if (now - startedAt >= config.AI_ANALYSIS_PROCESSING_TIMEOUT_MS) {
conversationProcessing.delete(key);
// conversationProcessing is now a Partial<Record<lane, startedAt>>.
// Prune stale lane slots individually so one stale lane never clears
// the other lane's healthy lock.
for (const [key, record] of conversationProcessing) {
for (const lane of ANALYSIS_LANES as readonly AnalysisLane[]) {
const startedAt = record?.[lane];
if (
startedAt &&
now - startedAt >= config.AI_ANALYSIS_PROCESSING_TIMEOUT_MS
) {
clearConversationProcessing(key, lane);
}
}
}
@@ -235,12 +251,23 @@ export function startPendingAIAnalysisWorker(
// --- Batch recovery for pending messages ---
for (const key of pendingKeys) {
if (conversationDebounceTimers.has(key)) continue;
if (
ANALYSIS_LANES.some((lane) =>
conversationDebounceTimers.has(`${key}::${lane}`),
)
) {
continue;
}
// Batch recovery must not race ANY in-flight batch lane, so the
// lock check is lane-agnostic here (individual fallback handles
// error rows separately).
if (isConversationProcessingLocked(key)) continue;
if (individualInFlightByConversation.has(key)) continue;
if (incompleteKeySet.has(key)) continue;
const cooldownUntil = conversationErrorCooldown.get(key);
if (cooldownUntil && now < cooldownUntil) continue;
// No lane specified → schedule BOTH lanes; each fetches its own
// pending subset from the DB.
scheduleConversationAnalysis(key);
}