kv-cache: clear the cache once when restoring a whole context

state_read walks the streams of the cache in turn, and for a whole-context restore
each stream went through state_read_meta, which starts by calling clear(). clear()
resets every stream at once, so each stream after the first threw away the streams
already restored, and the K/V buffers with them. A non-unified cache holds one
stream per sequence, so a context saved with N sequences in it came back with only
the sequence in the last stream that carried any cells - the highest sequence id.
A unified cache has one stream and never showed it.

The cache is now emptied once, before the loop, which is what a whole-context
restore means. A blob whose streams are all empty now empties the cache as well,
where before it left the old contents in place.
This commit is contained in:
Daniel Han
2026-08-27 04:13:14 +00:00
parent 8b3ed0a400
commit f43070aa5c
+7 -2
View File
@@ -2071,6 +2071,13 @@ const slot_info_vec_t * sinfos_in) {
throw std::runtime_error("n_stream mismatch");
}
// a whole-context restore replaces every stream, so the cache is emptied once here. clear()
// resets all streams at once, so doing this per stream below would throw away the streams
// already read and leave only the last one
if (seq_id == -1) {
clear(true);
}
for (uint32_t s = 0; s < n_stream; ++s) {
uint32_t cell_count;
io.read(&cell_count, sizeof(cell_count));
@@ -2345,8 +2352,6 @@ bool llama_kv_cache::state_read_meta(llama_io_read_i & io, uint32_t strm, uint32
return false;
}
clear(true);
for (uint32_t i = 0; i < cell_count; ++i) {
llama_pos pos;
uint32_t n_seq_id;