Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-11 15:29:12 +02:00
Code Issues Packages Projects Releases Wiki Activity

model : fix SWA not being enabled for EXAONE 4.5 (#26848)

Browse Source 
* model : fix SWA not being enabled for EXAONE 4.5

load_arch_hparams tests `hparams.n_layer() == 64` before
LLM_KV_NEXTN_PREDICT_LAYERS has been read. n_layer() returns
n_layer_all - n_layer_nextn and n_layer_nextn defaults to 0, so a GGUF
carrying the MTP head (block_count=65, nextn=1) evaluates to 65 and the
whole SWA block is skipped. The model type switch further down in the
same function reads 64, because by then the key has been loaded.

n_swa is still filled in by the unconditional get_key below the block, so
llama_model_n_swa() reports 4096 and the logs look correct while only
swa_type stays LLAMA_SWA_TYPE_NONE.

This affects the official LGAI-EXAONE GGUF release as well. EXAONE 4.0 has
no MTP head, so block_count is 64 there and the check matches.

* model-loader : skip TENSOR_SKIP tensors in the metadata-only path

create_tensor asserts on a null buffer type when building from metadata
alone, but buft_for_tensor returns null by design for tensors marked
TENSOR_SKIP, which is how architectures with nextn/MTP layers mark theirs.
Those models cannot be constructed by llama_model_init_from_user at all.

The file-backed path below already returns nullptr for the same tensors, so
callers see the same thing either way.

* tests : cover exaone4 hparams ordering

Builds a synthetic exaone4 model with the layout the shipped EXAONE 4.5
GGUFs use (block_count 65 + nextn 1). The swa_type check is the one that
catches the ordering bug; the n_layer_nextn and n_layer() checks only tell
a broken fixture apart from a real regression.

Fails before the ordering fix with "swa_type is not STANDARD", passes after.

* Revert "tests : cover exaone4 hparams ordering"

This reverts commit d2f3bafeee591ad691396b2708de4baef3aaf602.

* Revert "model-loader : skip TENSOR_SKIP tensors in the metadata-only path"

This reverts commit aecb9bc0c7896b52afbc43921a1f572aa7b5e53c.
This commit is contained in:
Junmo Kim
2026-08-11 13:20:17 +09:00
committed by GitHub
parent 48d22e295e
commit 14e78ddef7
1 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/models/exaone4.cpp
+3 -3
View File
@@ -1,6 +1,9 @@
#include "models.h"
void llama_model_exaone4::load_arch_hparams(llama_model_loader & ml) {
ml.get_key(LLM_KV_NEXTN_PREDICT_LAYERS, hparams.n_layer_nextn, false);
GGML_ASSERT(hparams.n_layer_nextn < hparams.n_layer_all && "n_layer_nextn must be < n_layer");
if (hparams.n_layer() == 64) { // 32B
hparams.swa_type = LLAMA_SWA_TYPE_STANDARD;
hparams.n_swa = 4096;
@@ -15,9 +18,6 @@ void llama_model_exaone4::load_arch_hparams(llama_model_loader & ml) {
ml.get_key(LLM_KV_ATTENTION_SLIDING_WINDOW, hparams.n_swa, false);
ml.get_key(LLM_KV_ATTENTION_LAYERNORM_RMS_EPS, hparams.f_norm_rms_eps);
ml.get_key(LLM_KV_NEXTN_PREDICT_LAYERS, hparams.n_layer_nextn, false);
GGML_ASSERT(hparams.n_layer_nextn < hparams.n_layer_all && "n_layer_nextn must be < n_layer");
switch (hparams.n_layer()) {
case 30: type = LLM_TYPE_1_2B; break;