Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-19 01:05:09 +02:00
Code Issues Packages Projects Releases Wiki Activity

Revert "Revert "set flags to optimize for mmq""

Browse Source 
This reverts commit 7959e937a1.
This commit is contained in:
Concedo
2024-06-26 16:47:12 +08:00
parent 7959e937a1
commit 70000b47e2
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml-cuda/mmq.cuh
+1
View File
@@ -8,6 +8,7 @@
#include <cstdint>
#define MMQ_DP4A_MAX_BATCH_SIZE 64 // Max. batch size to use for dp4a MMQ kernels when FP16 tensor cores are available.
#define GGML_CUDA_FORCE_MMQ
typedef void (*load_tiles_mmq_t)(const char * __restrict__ x, int * x_tile, const int & kbx0, const int & i_max, const int & stride);
typedef void (*vec_dot_mmq_t)(const int * __restrict__ x, const int * __restrict__ y, float * __restrict__ sum, const int & k0);