xbruce22 commited on
Commit
ee5e8e4
·
verified ·
1 Parent(s): a513388

Corrected jinja template with tool Support works with PR llama.cpp/pull/15186

Browse files

I have added tool support to llama.cpp [here](https://github.com/ggml-org/llama.cpp/pull/15186)
But parsing had issue with Zai's provided Jinja template.
I have added corrected template here.

Files changed (1) hide show
  1. chat_template.jinja +116 -0
chat_template.jinja ADDED
@@ -0,0 +1,116 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [gMASK]<sop>
2
+ {%- if tools -%}
3
+ <|system|>
4
+ # Tools
5
+
6
+ You may call one or more functions to assist with the user query.
7
+
8
+ You are provided with function signatures within <tools></tools> XML tags:
9
+ <tools>
10
+ {% for tool in tools %}
11
+ {{ tool | tojson }}
12
+ {% endfor %}
13
+ </tools>
14
+
15
+ For each function call, output the function name and arguments within the following XML format:
16
+ <tool_call>{function-name}
17
+ <arg_key>{arg-key-1}</arg_key>
18
+ <arg_value>{arg-value-1}</arg_value>
19
+ <arg_key>{arg-key-2}</arg_key>
20
+ <arg_value>{arg-value-2}</arg_value>
21
+ ...
22
+ </tool_call>{%- endif -%}
23
+ {%- macro visible_text(content) -%}
24
+ {%- if content is string -%}
25
+ {{- content }}
26
+ {%- elif content is iterable and content is not mapping -%}
27
+ {%- for item in content -%}
28
+ {%- if item is mapping and item.type == 'text' -%}
29
+ {{- item.text }}
30
+ {%- elif item is string -%}
31
+ {{- item }}
32
+ {%- endif -%}
33
+ {%- endfor -%}
34
+ {%- else -%}
35
+ {{- content }}
36
+ {%- endif -%}
37
+ {%- endmacro -%}
38
+ {%- set ns = namespace(last_user_index=-1) %}
39
+ {%- for m in messages %}
40
+ {%- if m.role == 'user' %}
41
+ {% set ns.last_user_index = loop.index0 -%}
42
+ {%- endif %}
43
+ {%- endfor %}
44
+ {% for m in messages %}
45
+ {%- if m.role == 'user' -%}<|user|>
46
+ {%- set user_content = visible_text(m.content) -%}
47
+ {{ user_content }}
48
+ {%- if enable_thinking is defined and not enable_thinking -%}
49
+ {%- if not user_content.endswith("/nothink") -%}
50
+ {{- '/nothink' -}}
51
+ {%- endif -%}
52
+ {%- endif -%}
53
+ {%- elif m.role == 'assistant' -%}
54
+ <|assistant|>
55
+ {%- set reasoning_content = '' %}
56
+ {%- set content = visible_text(m.content) %}
57
+ {%- if m.reasoning_content is string %}
58
+ {%- set reasoning_content = m.reasoning_content %}
59
+ {%- else %}
60
+ {%- if '</think>' in content %}
61
+ {%- set think_parts = content.split('</think>') %}
62
+ {%- if think_parts|length > 1 %}
63
+ {%- set before_end_think = think_parts[0] %}
64
+ {%- set after_end_think = think_parts[1] %}
65
+ {%- set think_start_parts = before_end_think.split('<think>') %}
66
+ {%- if think_start_parts|length > 1 %}
67
+ {%- set reasoning_content = think_start_parts[-1].lstrip('\n') %}
68
+ {%- endif %}
69
+ {%- set content = after_end_think.lstrip('\n') %}
70
+ {%- endif %}
71
+ {%- endif %}
72
+ {%- endif %}
73
+ {%- if loop.index0 > ns.last_user_index and reasoning_content -%}
74
+ {{ '\n<think>' + reasoning_content.strip() + '</think>'}}
75
+ {%- else -%}
76
+ {{ '\n<think></think>' }}
77
+ {%- endif -%}
78
+ {%- if content.strip() -%}
79
+ {{ '\n' + content.strip() }}
80
+ {%- endif -%}
81
+ {% if m.tool_calls %}
82
+ {% for tc in m.tool_calls %}
83
+ {%- if tc.function %}
84
+ {%- set tc = tc.function %}
85
+ {%- endif %}
86
+ {{ '\n<tool_call>' + tc.name }}
87
+ {% set _args = tc.arguments %}
88
+ {% for k, v in _args.items() %}
89
+ <arg_key>{{ k }}</arg_key>
90
+ <arg_value>{{ v | tojson if v is not string else v }}</arg_value>
91
+ {% endfor %}
92
+ </tool_call>{% endfor %}
93
+ {% endif %}
94
+ {%- elif m.role == 'tool' -%}
95
+ {%- if m.content is string -%}
96
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
97
+ {{- '<|observation|>' }}
98
+ {%- endif %}
99
+ {{- '\n<tool_response>\n' }}
100
+ {{- m.content }}
101
+ {{- '\n</tool_response>' }}
102
+ {%- else -%}
103
+ <|observation|>{% for tr in m.content %}
104
+
105
+ <tool_response>
106
+ {{ tr.output if tr.output is defined else tr }}
107
+ </tool_response>{% endfor -%}
108
+ {% endif -%}
109
+ {%- elif m.role == 'system' -%}
110
+ <|system|>
111
+ {{ visible_text(m.content) }}
112
+ {%- endif -%}
113
+ {%- endfor -%}
114
+ {%- if add_generation_prompt -%}
115
+ <|assistant|>{{- '\n<think></think>' if (enable_thinking is defined and not enable_thinking) else '' -}}
116
+ {%- endif -%}