Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-19 09:15:18 +02:00
Code Issues Packages Projects Releases Wiki Activity

Pass img_min_params and img_max_params to ctx_clip_params (#2133)

Browse Source 
* Pass img_min_params and img_max_params to ctx_clip_params

These values determine the minimum and maximum size (in
tokens) of vision embeddings. The default value of -1
uses a model-dependent default size, for example for
Gemma 4 the default is a 280 token embedding. For higher
quality results (at the cost of using more memory and
slower speed) you can increase the size of the embedding
to 1120 tokens.

* Change dict to mydict to match change to method
This commit is contained in:
Llama
2026-04-15 21:27:06 -07:00
committed by GitHub
parent a9f9e9a38b
commit c592bd01da
4 changed files with 22 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
gpttype_adapter.cpp
+4 -2
View File
@@ -2159,6 +2159,8 @@ ModelLoadResult gpttype_load_model(const load_model_inputs inputs, FileFormat in
kcpp_pipeline_parallelism = inputs.pipelineparallel;
kcpp_data->n_batch = GetBatchSize(inputs.batchsize, in_file_format);
kcpp_data->n_ubatch = kcpp_data->n_batch;
kcpp_data->image_min_tokens = inputs.image_min_tokens;
kcpp_data->image_max_tokens = inputs.image_max_tokens;
if(isGguf && kcpp_pipeline_parallelism)
{
//double the logical batch, while keeping the physical batch the same, pipeline parallel set GGML_SCHED_MAX_COPIES to 2
@@ -2748,8 +2750,8 @@ ModelLoadResult gpttype_load_model(const load_model_inputs inputs, FileFormat in
clip_context_params ctx_clip_params {
/* use_gpu */ true,
/* flash_attn_type */ clip_fa,
/* image_min_tokens */ -1,
/* image_max_tokens */ -1,
/* image_min_tokens */ kcpp_data->image_min_tokens,
/* image_max_tokens */ kcpp_data->image_max_tokens,
};
clip_init_result cres = clip_init(mmproj_filename.c_str(), ctx_clip_params);
clp_ctx_v = cres.ctx_v;