Vulkan Mixture of Experts (MoE) support (#7628)

Browse Source

* Finish Vulkan mul_mat_id implementation

* Add Vulkan sum_rows and div ops

* Fix MUL_MAT_ID matrix matrix shader

* Fix MUL_MAT_ID matrix vector shader dispatch size

* Fix MUL_MAT_ID matrix vector shader and dispatch code

* Update Vulkan CPU offload for MUL_MAT_ID

* Fix crash when using split mode none and setting a main GPU

...

This commit is contained in:

0cc4m

2024-06-03 10:59:14 +02:00

committed by GitHub

parent a10cda58d3

commit 3d7ebf6312

5 changed files with 73389 additions and 13839 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

ggml-vulkan.cpp

+425 -292

File diff suppressed because it is too large Load Diff