refactor(chat): unify generation flow and fix tool/streaming bugs

- Unified synchronous generate() core with callback; both streaming and
  non-streaming paths run it via spawn_blocking (context: std Mutex).
- build_prompt now passes tool definitions to the template (was dead) and
  embeds assistant tool-call history as XML matching the parser format;
  fixes double <tool_response> wrap and template set-scoping bug.
- Tokenize with AddBos::Never (template owns <s>) to remove double BOS.
- Streaming: preserve inter-word spaces (per-chunk trim removed), add
  [DONE] + usage chunk, emit error events, single-shot tool_calls delta.
- Strict model validation (400 on unknown model); health/UI/README aligned
  to minicpm5-1b-fable5-v2-thinking; auth returns JSON errors; n_ctx/
  n_batch/n_threads env-configurable.
- Added 18 unit tests; cargo check/clippy/fmt clean.
- scripts/smoke-test.sh for post-deploy verification on the VPS.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
asepharyana
2026-08-03 08:58:13 +07:00
co-authored by Claude Opus 5
parent 344bc195fa
commit b636496497
14 changed files with 883 additions and 445 deletions
+2 -1
View File
@@ -8,6 +8,7 @@ use crate::domain::entity::HealthResponse;
pub async fn health_check() -> Json<HealthResponse> {
Json(HealthResponse {
status: "ok".into(),
model: format!("{MODEL_ID}-q8_0"),
// Report the exact model id served by /v1/models (no extra suffix).
model: MODEL_ID.into(),
})
}