mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2026-09-18 16:55:14 +02:00
FIX: mtmd_tokenize: error (#2360)
DEEPSEEK4 SLOP Explanation. (Tested and is working correctly after build)
fix mtmd: missing text_len in gpttype_adapter.cpp aggregate init
Commit 4114ba18b ("mtmd: fix silent prompt truncation on embedded NUL
(#25548)") added a `text_len` field to `mtmd_input_text`, changing the
struct layout from:
{ text, add_special, parse_special }
to:
{ text, text_len, add_special, parse_special }
The aggregate initialization in `gpttype_adapter.cpp` was never updated
to account for the new field. With C++ aggregate init, the positional
arguments shifted:
mtmd_input_text inp_txt = {
mtmd_default_marker(), // text = "<__media__>"
/* add_special */ false, // text_len = 0 <-- BUG
/* parse_special */ true, // add_special = true
};
// parse_special is zero-initialized
Because text_len was 0, `input_text.assign(text->text, text->text_len)`
in the mtmd_tokenizer constructor produced an empty string.
`split_text()` on an empty string returned an empty vector, yielding
zero media markers in the text -- but one bitmap was still provided,
triggering the error:
mtmd_tokenize: error: number of media markers in text (0) does not
match number of bitmaps (1)
Fix by explicitly capturing the marker pointer and passing its length.
This commit is contained in:
+3
-1
@@ -5334,8 +5334,10 @@ static void PrepareMediaEmbds(const int nctx, const std::vector<int> & media_int
|
||||
printf("\nError: MTMD media %d failed to load!",i);
|
||||
continue;
|
||||
}
|
||||
const auto * marker = mtmd_default_marker();
|
||||
mtmd_input_text inp_txt = {
|
||||
mtmd_default_marker(),
|
||||
marker,
|
||||
strlen(marker),
|
||||
/* add_special */ false,
|
||||
/* parse_special */ true,
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user