Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-04 12:01:24 +02:00
Code Issues Packages Projects Releases Wiki Activity

llama : load MTP tensors only if they are really used (#26296)

Browse Source 
* llama : load MTP tensors only if they are really used

* llama : skip loading MTP (if not used) in remaining models that support MTP

---------

Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
This commit is contained in:
fairydreaming
2026-07-31 14:57:02 +02:00
committed by GitHub
parent 6f3c0a790b
commit 82dbc4f017
13 changed files with 65 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/llama-model-loader.cpp
+2
View File
@@ -526,6 +526,7 @@ llama_model_loader::llama_model_loader(
llama_load_mode load_mode,
bool check_tensors,
bool no_alloc,
bool load_mtp,
const llama_model_kv_override * param_overrides_p,
const llama_model_tensor_buft_override * param_tensor_buft_overrides_p)
: metadata(meta), set_tensor_data(set_tensor_data), set_tensor_data_ud(set_tensor_data_ud) {
@@ -812,6 +813,7 @@ llama_model_loader::llama_model_loader(
this->check_tensors = check_tensors;
this->no_alloc = no_alloc;
this->load_mtp = load_mtp;
}
std::string llama_model_loader::get_arch_name() const {