MythEclipse/llm-api
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

feat: render GGUF Jinja template via minijinja crate

Browse Source 
- Replaced manual prompt building with GGUF's chat template rendered through minijinja (Rust Jinja2 engine)
- Simplified template: removed multi-step tool detection, reasoning extraction (not needed at template level)
- Added reasoning_content separation: clean_text() returns (reasoning, answer) tuple
- Added reasoning_content field to ResponseMessage and SseDelta for OpenAI-compatible output
- Embedded template at build time via include_str! from templates/chat_template.jinja
- Built-in support for enable_thinking, tool_definitions, tool_calls, tool_response
This commit is contained in:
Asep Haryana
2026-07-26 18:21:35 +07:00
parent 9a63ff1601
commit 6ff31b5f62
6 changed files with 183 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/application/chat/templates/chat_template.jinja
+53
View File
@@ -0,0 +1,53 @@
{{- bos_token }}
{%- if tools %}
{%- set tool_definitions %}
{{- "# Tools\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- '\n</tools>\n\nTool usage guidelines:\n- You may call zero or more functions. If no function calls are needed, just answer normally.\n- When calling a function, use: <function name="name"><param name="key">value</param></function>' }}
{%- endset %}
{{- '<|im_start|>system\n' }}
{%- if messages[0].role == 'system' %}
{{- messages[0].content + '\n\n' + tool_definitions }}
{%- else %}
{{- tool_definitions }}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- else %}
{%- if messages[0].role == 'system' %}
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- for message in messages %}
{%- set content = message.content %}
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{%- if message.tool_calls %}
{%- set content = message.content %}
{%- for tool_call in message.tool_calls %}
{%- if tool_call.type == "function" %}
{%- set content = content + "\n<tool_call>\n<function=" + tool_call.function.name + ">\n" + (tool_call.function.arguments | tojson) + "\n</function>\n</tool_call>" %}
{%- endif %}
{%- endfor %}
{{- '<|im_start|>assistant\n' + content + '<|im_end|>\n' }}
{%- else %}
{{- '<|im_start|>assistant\n' + content + '<|im_end|>\n' }}
{%- endif %}
{%- elif message.role == "tool" %}
{{- '<|im_start|>user\n<tool_response>\n' + content + '\n</tool_response><|im_end|>\n' }}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined %}
{%- if enable_thinking is false %}
{{- '<think>\n\n</think>\n\n' }}
{%- elif enable_thinking is true %}
{{- '<think>\n' }}
{%- endif %}
{%- endif %}
{%- endif %}