Public/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-20 01:31:31 +02:00
Code Issues Packages Projects Releases Wiki Activity

cuda : add CUDA_USE_TENSOR_CORES and GGML_CUDA_FORCE_MMQ macros

Browse Source
This commit is contained in:
Georgi Gerganov
2023-10-25 18:48:36 +03:00
parent 4c6744b526
commit a4e15a36e4
3 changed files with 104 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
llama.h
+1 -1
View File
@@ -178,7 +178,7 @@ extern "C" {
float rope_freq_scale; // RoPE frequency scaling factor, 0 = from model
// Keep the booleans together to avoid misalignment during copy-by-value.
bool mul_mat_q; // if true, use experimental mul_mat_q kernels
bool mul_mat_q; // if true, use experimental mul_mat_q kernels (DEPRECATED - always true)
bool f16_kv; // use fp16 for KV cache, fp32 otherwise
bool logits_all; // the llama_eval() call computes all logits, not just the last one
bool embedding; // embedding mode only