perf(ai): kontiguitas batch budget + max_tokens dinamis + urutan kronologis RETURNING

- pickBatchWithinBudget: stop di overflow pertama (break), bukan skip —
  batch tetap prefix kronologis tanpa gap analisis di tengah timeline.
  Diekstrak ke batchBudget.ts (pure, estimator di-inject) + regression test.
- callModerationLLM: param opsional maxTokens; text/media caller menghitung
  ceiling dari estimasi prompt (floor 2048, cap 16384) — batch kecil tak
  lagi reserve window completion 16k.
- getPending/IncompleteMessagesByConversation: sort hasil UPDATE..RETURNING
  by created_at ASC — Postgres tak menjamin urutan, konsumen (anchor konteks
  messages[0], prefix batch) bergantung pada urutan kronologis.
This commit is contained in:
asepharyana
2026-08-22 17:19:41 +07:00
parent 1397380fe9
commit 16becd5340
8 changed files with 230 additions and 19 deletions
@@ -49,6 +49,10 @@ export async function callModerationLLM(
targetIds: string[],
label: string,
signal?: AbortSignal,
// Output-side token cap. Defaults to the previous hard-coded value; batch
// callers pass a prompt-derived ceiling so small batches don't reserve a
// 16k completion budget (some routers pre-allocate KV cache per max_tokens).
maxTokens?: number,
): Promise<{
results: AnalysisResult[];
raw: ChatCompletion | null;
@@ -75,7 +79,7 @@ export async function callModerationLLM(
];
const completion = await llmChat({
messages,
max_tokens: 16384,
max_tokens: maxTokens ?? 16384,
jsonResponse: { type: "json_object" },
retries: 0,
signal,