* tests : run test-save-load-state across all architectures
test-save-load-state previously only ran in ctest against a single
downloaded model (tinyllamas/stories15M), i.e. only the llama arch.
Add a --models DIR mode to test-save-load-state that runs the full
save/load suite over every *.gguf in a directory, reporting a
per-model PASS/FAIL and exiting non-zero if any model fails, and wire
a ctest to run it over all architectures using the existing
generate-models fixture (test-llama-archs). The single-model -m mode
is preserved (still used by ci/run.sh).
Also bump the dummy-model training context in test-llama-archs from
128 to 256 so that the per-sequence context (which is padded up to a
multiple of 256) no longer exceeds n_ctx_train and emits the
"possible training context overflow" warning.
The test is expected to fail until the affected arches are fixed:
deepseek4 (host seq-copy), gemma2/gpt-oss/lfm2 (device seq-copy),
minimax-01 (state load). It aborts at the first arch that crashes.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* tests : match dummy DSA indexer to fused Lightning Indexer kernel
The dummy DSA indexer (deepseek32, glm-dsa, ...) used key_length=64 and head_count=1, so the fused Lightning Indexer op's q tensor was shaped [64, 1, ...]. The Metal fused kernel is fixed to DK=128, NH=64, so it rejected the op and the scheduler fell back to CPU, emitting a 'layer assigned to MTL but Lightning Indexer on CPU' warning. Bump key_length to 128 and the DSA head_count to 64 so the fused op runs on the GPU.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* tests : add --help and document -o in test-llama-archs
Add a --help/-h flag to test-llama-archs and list the existing -o/--out option in the usage text, which was previously missing.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* tests : use 64 indexer heads for deepseek4
deepseek4's indexer head count was set to n_head (8), which does not match the fused Lightning Indexer kernel's fixed NH=64, so the fused op fell back to the CPU backend and emitted a device-mismatch warning. Give it the same fixed 64 as the other indexer archs by dropping it from the n_head ternary (only minimax-m3 keeps n_head, since it does not use the fused Lightning Indexer op).
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* tests : fix dsv4 save-load n_stream mismatch
The dsv4 KV cache keeps per-sequence KV/state streams even in unified mode, so its n_stream equals n_seq_max. The test saved the state in the baseline with n_seq_max=1 but loaded it in the seq-copy tests with n_seq_max=2, so state_read threw an n_stream mismatch. Use n_seq_max=2 in the baseline and state-load tests so the save and load agree.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* context : relax on-device seq-copy chunk alignment
The on-device state seq copy (llama_state_seq_set_data with LLAMA_STATE_SEQ_FLAGS_ON_DEVICE) copied the write-side cpy tensors to the read-side targets 1:1 by index, requiring the writer and reader to emit the same number of chunks in the same order with the same per-chunk sizes. state_write_data chunks per cell-range while state_read_data chunks contiguous-or-per-cell, so the counts diverged for non-contiguous sources (dsv4, SWA) and the copy aborted with "memory buffer mismatch".
All state writers and readers enumerate the same logical data in the same order, differing only in chunking. Copy the flat write-side data into the read-side targets with a byte cursor that walks both tensor lists across their boundaries, so the chunking no longer needs to match. Keep the total-size guard; drop the n_tensors equality check.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* model : fix dangling hparams ref in minimax-01 LA graph input
llm_graph_input_la stored const llama_hparams & hparams, bound to the llm_graph_params temporary in llama_context::process_ubatch. The input object outlives that temporary (it is kept in llm_graph_result::inputs for graph reuse), so set_input() read destroyed stack memory on every graph reuse - test-save-load-state crashed for minimax-01 when the stack region was overwritten (n_layer_all read as 0, abort in llama_hparams::n_head). Store a copy like every other graph input class.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* context : handle "worst case" graph and add TODO
* tests : refactor test-save-load-state to accept token input
- Default prompt is now empty; when not provided, generate n_batch
random tokens (useful for models without a tokenizer)
- Tokenization happens once upfront; pass token vector to test functions
- generate_tokens prints token IDs instead of decoded pieces
- Use llama_model_get_vocab / llama_vocab_n_tokens API
- Upgrade log level from LOG_TRC to LOG_INF for visibility
Assisted-by: llama.cpp:local pi
* cont : use llama_tokens alias
* common : fix state save in common_prompt_batch_decode
This commit addresses a bug in common_prompt_batch_decode that affects
the session state store/restore in completion.cpp and
save-load-state.cpp.
The motivation for this is that currently the code is saving n-1 tokens
in both the session_tokens and in the KV cache. Then when loading the
session tokens, and if the prompt matches, it would replay the last
saved token (n-1) into the next position, effectively replaying the
same token in the wrong position.
The fix is to store all n tokens in session_tokens, while the memory
state only reflects n-1 processed tokens as the saving happens before
the last token is decoded in common_prompt_batch_decode.
I ran both completion.cpp and save-load-state.cpp with a transformer, a
recurrent, and a hybrid model.
Resolves: https://github.com/ggml-org/llama.cpp/issues/23400
Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>
* tests : move save-load-state from examples to tests
- Move examples/save-load-state/ to tests/test-save-load-state.cpp
- Remove subdirectory reference from examples/CMakeLists.txt
- Add test to tests/CMakeLists.txt as a model test
- Remove CODEOWNERS entry for removed example directory
Assisted-by: llama.cpp:local pi
* cont : update ci
Georgi Gerganov
helanfxz
Aman Gupta
Daniel Bevenius