Mistral Small 2506 models using Pixtral vision encoder were running out
of GPU memory when processing images larger than 1024x1024 pixels due to
exponential memory growth from unlimited image size.
This fix applies the same 1024x1024 limit used by Qwen2VL models to
prevent OOM issues while maintaining compatibility with existing models.
* Add support for VK_EXT_debug_utils to add labels to Vulkan objects. In step 1 compute pipelines are getting labeled.
* remove #ifdef for debug utils and add queue marker.
* add backend support for changing the VAE tiling threshold
* trigger VAE tiling by image area instead of dimensions
I've tested with GGML_VULKAN_MEMORY_DEBUG all resolutions with
the same 768x768 area (even extremes like 64x9216), and many
below that: all consistently allocate 6656 bytes per image pixel.
As tiling is primarily useful to avoid excessive memory usage, it
seems reasonable to enable VAE tiling based on area rather than
maximum image side.
However, as there is currently no user interface option to change
it back to a lower value, it's best to maintain the default
behavior for now.
* replace the notile option with a configurable threshold
This allows selecting a lower threshold value, reducing the
peak memory usage.
The legacy sdnotile parameter gets automatically converted to
the new parameter, if it's the only one supplied.
* simplify tiling checks, 768 default visible in launcher
---------
Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
* Add header and namespace to use enqueue_functions extension
* Convert submit and parallel_for to use new extension in convert.cpp
* Convert submit and parallel_for to use extension in ggml-sycl.cpp
* Convert submit and parallel_for to use extension in gla.cpp
* Convert submit and parallel_for in mmq.cpp
* Convert submit and parallel_for in mmvq.cpp
* Convert submit and parallel_for in remaining files
* Convert all simple parallel_for to nd_launch from enqueue_functions
extension
* Wrapping extension in general function
Create a general function that enable the enqueue_functions extension if
it is enable in the compiler, otherwise call the general SYCL function
to launch kernels.
---------
Signed-off-by: nscipione <nicolo.scipione@codeplay.com>
* Add PowerPC feature detection and scoring
* ggml-cpu: Implement GGML_CPU_ALL_VARIANTS for PowerPC
* ggml-cpu: Delay some initializations until function is called
When using GGML_BACKEND_DL=ON, these initializations might use
instructions that are not supported by the current CPU.
---------
Co-authored-by: Diego Devesa <slarengh@gmail.com>
Concedo
uvos
yuiseki
Sigbjørn Skjæret
Aman Gupta
Markus Tavenrath
Georgi Gerganov
Daniel Han
Wagner Bruna
Acly
David Chiu
Ruikai Peng
henk717
Nicolò Scipione
Christian Kastner
Diego Devesa
Charles Xu
Alex Trotta
aa956