Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-19 17:25:07 +02:00
Code Issues Packages Projects Releases Wiki Activity

reverted sequence mode for rwkv due to multiple issues with speed loss with bigger quantized models

Browse Source
This commit is contained in:
Concedo
2023-06-14 20:03:14 +08:00
parent f83b66606b
commit 3ed3e7b7e2
4 changed files with 5 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
otherarch/rwkv_v3.cpp
+2 -2
View File
@@ -484,8 +484,8 @@ struct rwkv_ggml_context {
return;
}
const size_t memory_required_overhead = size_t(256) * 1024 * 1024;
const size_t memory_required_overhead_sc = size_t(128) * 1024 * 1024;
const size_t memory_required_overhead = size_t(128) * 1024 * 1024;
const size_t memory_required_overhead_sc = size_t(64) * 1024 * 1024;
ctx = ggml_init({ size.objects_count * GGML_OBJECT_SIZE + size.objects_size + memory_required_overhead, NULL, false});