Public/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-18 16:55:05 +02:00
Code Issues Packages Projects Releases Wiki Activity

server: fix speculation after an image (#28715)

Browse Source 
* server: fix speculation after an image

Pass the actual position to the drafter after an image, instead of the
token count. Affects every drafter, not just DFlash.

* rename draft n_past to pos0

n_past is used to denote number of tokens and this parameter is meant to be a position
This commit is contained in:
Jesus Gulfo
2026-09-11 03:33:26 -05:00
committed by GitHub
parent 16378d93f9
commit b0dcb8192b
4 changed files with 13 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/server/server-context.cpp
+1 -1
View File
@@ -3028,7 +3028,7 @@ private:
common_speculative_get_draft_params(spec.get(), slot.id) = {
/* .drafting = */ true,
/* .n_max = */ n_draft_max,
/* .n_past = */ slot.prompt.n_tokens(),
/* .pos0 = */ slot.prompt.tokens.pos_next(),
/* .id_last = */ slot.sampled,
/* .prompt = */ &slot.spec_prompt,
/* .result = */ &slot.spec_draft,