refactor(chat): unify generation flow and fix tool/streaming bugs

- Unified synchronous generate() core with callback; both streaming and
  non-streaming paths run it via spawn_blocking (context: std Mutex).
- build_prompt now passes tool definitions to the template (was dead) and
  embeds assistant tool-call history as XML matching the parser format;
  fixes double <tool_response> wrap and template set-scoping bug.
- Tokenize with AddBos::Never (template owns <s>) to remove double BOS.
- Streaming: preserve inter-word spaces (per-chunk trim removed), add
  [DONE] + usage chunk, emit error events, single-shot tool_calls delta.
- Strict model validation (400 on unknown model); health/UI/README aligned
  to minicpm5-1b-fable5-v2-thinking; auth returns JSON errors; n_ctx/
  n_batch/n_threads env-configurable.
- Added 18 unit tests; cargo check/clippy/fmt clean.
- scripts/smoke-test.sh for post-deploy verification on the VPS.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
asepharyanaandClaude Opus 5 committed 2026-08-03 08:58:13 +07:00
1 parent 344bc195fa
commit b636496497
14 files changed
+883 -445

No files matched your search

@@ -6,7 +6,7 @@
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- '\n</tools>\n\nTool usage guidelines:\n- You may call zero or more functions. If no function calls are needed, just answer normally.\n- When calling a function, use: <function name="name"><param name="key">value</param></function>' }}
{{- '\n</tools>\n\nTool usage guidelines:\n- You may call zero or more functions. If no function calls are needed, just answer normally.\n- When calling a function, wrap each call in <tool_call> tags:\n<tool_call>\n<function=name>\n<parameter=key>value</parameter>\n</function>\n</tool_call>' }}
{%- endset %}
{{- '<|im_start|>system\n' }}
@@ -26,17 +26,8 @@
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- if message.tool_calls %}
{%- set content = message.content %}
{%- for tool_call in message.tool_calls %}
{%- if tool_call.type == "function" %}
{%- set content = content + "\n<tool_call>\n<function=" + tool_call.function.name + ">\n" + (tool_call.function.arguments | tojson) + "\n</function>\n</tool_call>" %}
{%- endif %}
{%- endfor %}
{{- '<|im_start|>assistant\n' + content + '<|im_end|>\n' }}
{%- else %}
{{- '<|im_start|>assistant\n' + content + '<|im_end|>\n' }}
{%- endif %}
{#- Tool-call XML from history is pre-embedded in content by build_prompt. #}
{{- '<|im_start|>assistant\n' + content + '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{{- '<|im_start|>user\n<tool_response>\n' + content + '\n</tool_response><|im_end|>\n' }}
{%- endif %}