Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-03 11:35:48 +02:00
Code Issues Packages Projects Releases Wiki Activity

mtmd: add --mmproj-device argument (#23255)

Browse Source 
* feat: add --mmproj-device arg & backwards compatible MTMD_BACKEND_DEVICE env var

* feat: load mmproj device backend immediately, add -mmdev shortflag

* fix: its a pointer now get the name

* clean up

* gen docs

* nits

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
This commit is contained in:
John-Henry Lim
2026-08-21 00:45:37 +08:00
committed by GitHub
parent 07822bddf8
commit 681c29d36a
12 changed files with 41 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/server/server-context.cpp
+1
View File
@@ -998,6 +998,7 @@ private:
mtmd_context_params mparams = mtmd_context_params_default();
if (has_mmproj) {
mparams.use_gpu = params_base.mmproj_use_gpu;
mparams.device = params_base.mmproj_device;
mparams.print_timings = false;
mparams.n_threads = params_base.cpuparams.n_threads;
mparams.flash_attn_type = params_base.flash_attn_type;