From 152e080b6ab5e0fd2ef4484c131bbf9897366918 Mon Sep 17 00:00:00 2001
From: Julien BODIN <137780842+JulienJBO@users.noreply.github.com>
Date: Wed, 5 Aug 2026 12:51:55 +0200
Subject: [PATCH] Add Mistral [THINK]/[/THINK] thinking format (mistral3 arch)
 (#2380)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The reasoning budget derived from reasoning_effort never applied to Mistral
models. gpttype_adapter.cpp picks the think delimiters from a switch on the
model architecture, and mistral3 has no case, so it falls back to <think> /
</think>. Those are not vocabulary tokens for Ministral-3, so TokenizeString
returns more than one token each, the expected_start/end_tokens guard clears
all three vectors, and apply_reasoning_budget() returns at its first if.
The parameter is accepted, converted and passed down to the sampler, then
dropped on a size check, with nothing logged.

Adding the mistral3 case arms the budget. [THINK] and [/THINK] are single
vocabulary tokens (ids 34 and 35 on Ministral-3), so the size guard passes.

The thinkformats entry is a separate fix for a separate defect: without it the
thinking block was never split out, so it leaked into content with its [THINK]
marker still in it, instead of going to reasoning_content.

Measured on Ministral-3-14B-Reasoning-2512 (IQ4_XS, ctx 8192, --jinja), 5 real
prompts x 3 samples per cell, max_tokens 3000 (so a 750-token budget at "low"):

  reasoning_effort   thinking words before      thinking words after
  none               311 - 2314                 7 (the forced-close phrase)
  low                340 - 2255                 521 - 574

Forced closes: 0/15 before, 14/15 after at "low" and 15/15 at "none". Three
samples per cell because this model's variance at temperature 0.7 spans a
factor of 4 on an identical payload — a single sample per cell cannot tell an
effect from noise.

No regression on a non-reasoning mistral3 model: Ministral-3-8B-Instruct with
reasoning_effort "low" returns finish_reason "stop", a normal answer and zero
forced closes, since apply_reasoning_budget() bails out when the start marker
never appears.
---
 gpttype_adapter.cpp | 5 +++++
 koboldcpp.py        | 3 ++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/gpttype_adapter.cpp b/gpttype_adapter.cpp
index 9840e5c4a..6098f6eab 100644
--- a/gpttype_adapter.cpp
+++ b/gpttype_adapter.cpp
@@ -5859,6 +5859,11 @@ generation_outputs gpttype_generate(const generation_inputs inputs)
                 end = "<|END_THINKING|>";
                  budget_exceeded = "\n(Reasoning budget exceeded)\nTime to respond now.\n<|END_THINKING|>";
                 break;
+            case llm_arch::LLM_ARCH_MISTRAL3:
+                start = "[THINK]";
+                end = "[/THINK]";
+                budget_exceeded = "\n(Reasoning budget exceeded)\nTime to respond now.\n[/THINK]";
+                break;
             default:
                 break;
         }
diff --git a/koboldcpp.py b/koboldcpp.py
index 9ac7aba1c..fd3f967a1 100644
--- a/koboldcpp.py
+++ b/koboldcpp.py
@@ -186,7 +186,8 @@ thinkformats = [{"start":"<|channel|>analysis<|message|>","end":"<|start|>assist
                 {"start":"<think>","end":"</think>"},
                 {"start":"<seed:think>","end":"</seed:think>"},
                 {"start":"<|START_THINKING|>","end":"<|END_THINKING|>"},
-                {"start":"<|channel>thought","end":"<channel|>"}]
+                {"start":"<|channel>thought","end":"<channel|>"},
+                {"start":"[THINK]","end":"[/THINK]"}]
 tool_call_pairs = [ #third element is optional str to match in chat template before we use this pair, fourth element is whether its stream-handleable
     ("<tool_call>", "</tool_call>", None, True), #qwen, glm
     ("<seed:tool_call>", "</seed:tool_call>", None, True), #seed oss