Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-05 12:31:15 +02:00
Code Issues Packages Projects Releases Wiki Activity

allow blasbatchsize -1 which disables blas, but keeps benefits like gpu offloads.

Browse Source
This commit is contained in:
Concedo
2023-06-01 22:33:50 +08:00
parent 49272e3c53
commit 37659d2c4e
2 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
gpttype_adapter.cpp
+1 -1
View File
@@ -791,7 +791,7 @@ generation_outputs gpttype_generate(const generation_inputs inputs, generation_o
file_format == FileFormat::GPTJ_2 ||
file_format == FileFormat::RWKV_1 ||
file_format==FileFormat::RWKV_2);
bool blasmode = (approved_format && embd_inp.size() >= 32 && ggml_cpu_has_blas());
bool blasmode = (approved_format && embd_inp.size() >= 32 && ggml_cpu_has_blas() && blasbatchsize!=-1);
// bool blasmode = false;
int original_batch = params.n_batch;
int original_threads = params.n_threads;