feat(gmw): route all LLM traffic through 9router
GMW moves off omniroute (100.121.180.82:20128) and off the direct NVIDIA vision endpoint onto 9router, which runs on the same host as both services (127.0.0.1:4014) — loopback avoids the TLS/proxy hop and localhost calls bypass 9router's remote-key guard. - gateway + backend: AI_LLM_BASE_URL default -> http://127.0.0.1:4014/v1 - drop stale 'omniroute' router references from comments/docs now that the active router is 9router (llmClient, llmCaller, ARCHITECTURE, AGENTS) Verified against 9router before wiring: model 'text' -> gemini-3.5-flash-lite (SSE, as the pipeline expects), 'multimodal' -> nemotron-3-nano-omni answers image input, and gemini/gemini-embedding-001 returns 3072 dims — matching the existing Qdrant collections (no reindex needed). The GMW key is already registered in 9router's apiKeys table. typecheck + lint + tests green (gateway 138, backend 37 excluding e2e).
This commit is contained in:
1 parent
750f3aa598
commit
ef7708bf7d
6 files changed
+15
-13
No files matched your search
@@ -131,7 +131,7 @@ type LLMResponseChunk = {
|
||||
* `delta.content`; falls back to reasoning fields so reasoning-only models
|
||||
* still produce usable aggregated text. Providers differ in the field name:
|
||||
* - DeepSeek-style / Cloudflare gemma → `delta.reasoning_content`
|
||||
* - mimo (via omniroute) streams reasoning in `delta.reasoning` +
|
||||
* - mimo (via the router) streams reasoning in `delta.reasoning` +
|
||||
* `delta.reasoning_details[].text` (content:"") — without these fallbacks
|
||||
* vision aggregation came back empty ("Vision API null response").
|
||||
* Exported for unit tests.
|
||||
@@ -267,7 +267,7 @@ export function buildLlmParams(
|
||||
reasoning: { enabled: false },
|
||||
// vLLM / Qwen / litellm
|
||||
chat_template_kwargs: { enable_thinking: false },
|
||||
// Anthropic / Claude-format (omniroute exposes thinkingFormat
|
||||
// Anthropic / Claude-format (router exposes thinkingFormat
|
||||
// "claude-adaptive" / "claude-budget" on its reasoning models)
|
||||
thinking: { type: "disabled" },
|
||||
} as Record<string, unknown>);
|
||||
|
||||
Reference in new issue
Block a user