d3f9db8d33
fix system32 writability check
2025-09-24 14:43:41 +08:00
4d9ea03d17
codeowners : use slash prefix for root files [no ci] ( #16210 )
...
This commit adds a leading slash to the paths of root-level files
in the CODEOWNERS file.
The motivation for this is that these might otherwise match files
in subdirectories that have other/additional owners will override them.
Refs: https://github.com/ggml-org/llama.cpp/pull/16209#issuecomment-3326434274
2025-09-24 08:10:09 +02:00
8ba548dae2
model-conversion : fix the make targets in the README.md ( #16209 )
...
Fix two incorrect make targets in the readme.
Signed-off-by: Jie Fu <jiefu@tencent.com >
2025-09-24 06:19:23 +02:00
f505bd83ca
ci : disable AMD workflows + update NVIDIA workflows ( #16200 )
...
* ci : disable AMD workflows + update NVIDIA workflows
* cont : fixes
* cont : update nvidia vulkan workflows
2025-09-23 20:41:40 +03:00
84a67d289c
tts show model filename
2025-09-23 22:42:43 +08:00
0889589dbe
ci : enable Vulkan workflow on Mac ( #16194 )
2025-09-23 13:44:25 +03:00
4e29084ba4
ggml-cpu: Respect cpumask settings ( #16164 )
2025-09-23 11:58:12 +03:00
f6b4af3d04
ggml : fix uninitialized is_on_grid in quantize_row_iq3_xxs_impl ( #15928 )
...
* fix uninitialized is_on_grid in quantize_row_iq3_xxs_impl
* change initialization to true
2025-09-23 10:25:20 +02:00
264f1b5187
zdnn: refactor codebase + add docs ( #16178 )
...
* zdnn: initial matmul refactor
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: rm static from funcs
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: update ggml-zdnn.h
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: change header files to hpp
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: switch to common.hpp
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: move mulmat forward around
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: rm inline from utils
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: code cleanup
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* docs: add zDNN docs
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
---------
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
2025-09-23 14:53:05 +08:00
0bc7cc7154
codeowners : add @danbev to model-conversion example [no ci] ( #16190 )
...
This commit adds examples/model-conversion/ to the CODEOWNERS file and
assigns myself (@danbev) as the code owner for this directory.
2025-09-23 09:13:22 +03:00
4b9f4cb0f8
devops: add s390x containers ( #15915 )
...
* devops: add s390x dockerfile
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add missing ninja
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: move s390x docker into cpu docker
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: rework s390x docker
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: copy more tools
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add server build step
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove apt clean steps as distroless misses it
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove apt commands from distroless
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix shared libs in distroless
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: use correct libs path
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix shared libs
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add collector stage
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix missing stage ref
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix permission issue
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix unknown model loading failures
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: attempt at fixing model loading failure
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix missing ggml shared object
failure to load model
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove move shared objects
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: move libggml-cpu and blas into bin
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: finalise hardened server stage
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add cli target
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix typos
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix missing shared libraries in base
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: update debian target
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: formalise llama.cpp loc
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* Revert "devops: formalise llama.cpp loc"
This reverts commit 0a7664af8466a15f318ff209e02ac3c4e551cc18.
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: formalise llama.cpp loc
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
(cherry picked from commit 0a7664af8466a15f318ff209e02ac3c4e551cc18)
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: attempt at fixing missing dir
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: attempt at making it cache the build
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix copying process
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: make build dir an argument
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* Revert "devops: make build dir an argument"
This reverts commit 438698976b8a5181c1e8179600527cfd5a50cc23.
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add build stage for gguf-py
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: move gguf-py installation into build stage
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: break system packages?
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add rust compiler installer
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix rustc not found
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove cache mount to allow rustc to persist
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: move rustc installation to another layer
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: move gguf-py installation to full stage, fix copying
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove rustc installation in build
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: disable full target for now
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: attempting static build
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: merge s390x dockerfile into cpu for now
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: switch to gcc image for build step
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove build essentials
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: install openblas into base target
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: go back to s390x dockerfile
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove libggml and libblas
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add full target
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add break system packages
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add libjpeg
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add missing cmake dep
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: finalise docker images for s390x
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add custom openblas patch
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: use libopenblas-dev instead of libopenblas-openmp-dev
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add s390x docker build
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
---------
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
2025-09-23 13:59:34 +08:00
85e72271ba
ggml-cpu : fix typo in gemm comments [no ci] ( #16189 )
2025-09-23 05:59:03 +02:00
1d0125bcf1
feat: Add conversion support in GraniteHybrid for non-hybrid (all attn) ( #16177 )
...
This is a configuration of the hparams in the GraniteHybrid architecture
that devolves to the Granite (or GraniteMoe) architecture (ie Granite 3.x).
It may be used for some models in the Granite 4 family with the
GraniteHybrid architecture acting as a superset arch. Rather than support
it directly in the c++ graph, we simply coerce the architecture flag back
to the correct "granite" or "granitemoe" architecture.
Branch: gabe-l-hart/GraniteNonHybridConversion
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2025-09-22 20:40:10 +02:00
351f3da39c
clang-tidy : disable warning about performance enum size ( #16127 )
...
Disable 'performance-enum-size' checking:
Enum 'llama_token_type' uses a larger base type ('unsigned int', size: 4 bytes)
than necessary for its value set, consider using 'std::uint8_t' (1 byte) as the
base type to reduce its size.
2025-09-22 19:57:46 +02:00
3ecb2f671a
ggml : implement set_rows with i32 index ( #16159 )
...
* implement set_rows with i32 index
* template fix
* test quantized path
warnings--
* Apply suggestions from code review
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* forgotten name change
* deduplicate cuda/sycl and test-fix
* indent++
* vulkan: support set_rows with i32 index type (#16162 )
* disable i32 index for webgpu for now
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
Co-authored-by: Jeff Bolz <jbolz@nvidia.com >
2025-09-22 19:13:00 +02:00
432cf4304c
codeowners : update + cleanup ( #16174 )
...
---------
Co-authored-by: slaren <slarengh@gmail.com >
2025-09-22 18:20:21 +03:00
174d00bb74
fix aria2c with both download cases
2025-09-22 21:47:08 +08:00
efe546390b
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# CODEOWNERS
# CONTRIBUTING.md
# README.md
# ci/run.sh
# examples/embedding/README.md
# tests/test-backend-ops.cpp
2025-09-22 21:25:29 +08:00
59b6a09ae1
try to fix kokoro alloc again
2025-09-22 21:22:41 +08:00
37a23c17bd
common : enable --offline mode without curl support ( #16137 )
...
* common : use the json parser
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
* common : enable --offline mode without CURL support
This change refactors the download logic to properly support offline mode
even when the project is built without CURL.
Without this commit, using `--offline` would give the following error:
error: built without CURL, cannot download model from the internet
even if all the files are already cached.
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
---------
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2025-09-22 15:13:51 +03:00
138c87ce8b
webui : fix handling incomplete chunks ( #16107 )
2025-09-22 11:53:13 +03:00
c6db9a1027
embedding : fix typos in README ( #16171 )
2025-09-22 11:49:58 +03:00
d05affbab7
common : remove unused local variables ( #16140 )
...
These two local variables 'arg' and 'arg_prefix' have been overriden by:
1. for (const auto & arg : opt.args)
2. for (int i = 1; i < argc; i++) {
const std::string arg_prefix = "--";
std::string arg = argv[i];
2025-09-22 11:48:42 +03:00
4f324a556c
ggml : extend ggml_can_fuse to work with non-sequential nodes ( #16123 )
...
* ggml : extend ggml_can_fuse to work with non-sequential nodes in the graph
* cont : fix wrong bounds check condition
* cont : remove unnecessary overload
2025-09-22 11:12:37 +03:00
a71ae3ba7a
ggml : add ggml_op_is_empty ( #16122 )
...
* ggml : add ggml_op_is_empty
* ggml : move to ggml-impl.h
2025-09-22 11:12:09 +03:00
05a2458121
codeowners : update ownership for @ngxson and @allozuar ( #16128 )
2025-09-22 11:10:58 +03:00
96fdca043b
Vulkan: add conv_transpose_2d operation ( #16022 )
...
* Vulkan: add conv_transpose_2d operation
* Vulkan: fix typo in conv_transpose_2d shader(s0mp, s0L, s1mp, s1L)
* Vulkan: fix incorrect indentation in conv_transpose_2d shader
* Vulkan: add checking the push constants size limit and reuse conv2d_mm.comp for conv_transpose_2d operation
* Vulkan: revert the order of the index calculation and bound check in conv_2d shader
* Vulkan: explicity check push constants limit in supports_op() for conv_transpose_2d operation.
* Vulkan: remove unnecessary lower bound checks for H/W_idx in the conv_2d shader.
2025-09-22 10:04:01 +02:00
b2d980fce0
codeowners : claim responsibility for ci, models, gguf-py and convert ( #16124 )
...
* claim responsibility for ci, gguf-py and convert
* add myself to various src/llama- files
2025-09-22 10:59:05 +03:00
5c6106a696
contrib : update roles ( #16113 )
...
* contrib : update roles
* contrib : merge PR sections + add link to CI instructions
Updated pull request guidelines for contributors and collaborators, and clarified merging practices for maintainers.
2025-09-22 10:58:02 +03:00
ec65fb52f0
ci : remove vulkaninfo calls ( #16169 )
2025-09-22 10:16:05 +03:00
1d660d2fae
ci : use smaller model ( #16168 )
...
* ci : switch from gemma to qwen3 0.6b
* ci : use smaller model for some tests
2025-09-22 09:11:39 +03:00
216b766aee
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-riscv-native.yml
# .github/workflows/build.yml
# ci/README.md
# ci/run.sh
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# tests/test-backend-ops.cpp
2025-09-22 13:56:02 +08:00
b5931c98ba
Revert "Revert "vulkan: use vec dot for matrix matrix multiplications ( #16056 )""
...
This reverts commit 3170fc38cb
2025-09-22 13:49:16 +08:00
08c0246f24
prioritize cwd for downloads if its writable
2025-09-22 13:47:50 +08:00
a20d810d79
vulkan: add RTE variants of exp shader ( #16165 )
...
This fixes some failures on Turing where "round to zero" rounds to the max f16
value but the CPU reference value is infinite.
2025-09-22 07:37:17 +02:00
f50840455b
revert more mistral imposed dependencies
2025-09-22 13:32:51 +08:00
4d0a7cbc61
ci : adjust params for less runtime ( #16167 )
...
* ci : adjust params for less runtime
* ci : gate BF16 on some hardware
* ci : move extra tests to Arm runner
2025-09-22 08:31:40 +03:00
9073a73d82
vulkan: vec dot matrix multiplication fix ( #16151 )
...
* vulkan: fix matrix multiplication index calculation for odd m/n and odd k in combination with batching
* add odd m/n + odd k test with batching
2025-09-22 07:22:43 +02:00
13bee0d39d
some minor fixes
2025-09-22 13:20:06 +08:00
51f5a45fbe
opencl: fix concat crash on win arm64 with Adreno ( #15944 )
2025-09-21 16:42:10 -07:00
c4510dc937
opencl: initial q8_0 mv support ( #15732 )
2025-09-21 14:48:44 -07:00
da30ab5f86
ci : add label for the RISC-V runner ( #16150 )
2025-09-21 19:00:27 +03:00
8018e5222a
try fix compile issues rocm (+4 squashed commit)
...
Squashed commit:
[9e33a33f2] try fix compile issues rocm
[36e5eb56b] try fix compile issues rocm
[6ca18812b] try fix compile issues rocm
[b56ad2e8f] try fix compile issues rocm
2025-09-21 22:31:09 +08:00
28baac9c9f
ci : migrate ggml ci to self-hosted runners ( #16116 )
...
* ci : migrate ggml ci to a self-hosted runners
* ci : add T4 runner
* ci : add instructions for adding self-hosted runners
* ci : disable test-backend-ops from debug builds due to slowness
* ci : add AMD V710 runner (vulkan)
* cont : add ROCM workflow
* ci : switch to qwen3 0.6b model
* cont : fix the context size
2025-09-21 16:50:45 +03:00
3170fc38cb
Revert "vulkan: use vec dot for matrix matrix multiplications ( #16056 )"
...
This reverts commit 803dac2e48
2025-09-21 21:16:43 +08:00
c8686a627e
don't mandate mistral common for other model usage
2025-09-21 21:16:02 +08:00
dd0bf706a3
fix Chroma workaround for flash attention ( #1746 )
...
chroma_use_dit_mask is a context parameter, so changing it
after creating the context has no effect.
2025-09-21 20:53:40 +08:00
9e7661352c
Revert "FA default on"
...
This reverts commit 19c3efb34a
2025-09-21 17:28:49 +08:00
1eeb523c3e
vulkan: optimize UMA buffer operations and fix driver hangs ( #16059 )
...
* vulkan: optimize UMA buffer operations and fix driver hangs
The previous implementation was blocking the GPU for extended periods,
causing the i915 driver to reset the context due to the hangcheck
protection.
[32628.443070] i915 0000:00:02.0: [drm] GPU HANG: ecode 12:1:85dffffb, in llama-server [194114]
[32628.443091] i915 0000:00:02.0: [drm] llama-server[194114] context reset due to GPU hang
* vulkan: implement deferred_memset on UMA
---------
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com >
2025-09-21 08:31:55 +02:00
5bb4a3edec
vulkan: fix validation error about VK_PIPELINE_CREATE_CAPTURE_STATISTICS_BIT_KHR ( #16086 )
2025-09-21 08:23:37 +02:00
Concedo
Daniel Bevenius
Jie Fu (傅杰)
Georgi Gerganov
Xiangyan Sun
Sigbjørn Skjæret
Aaron Teo
Gabe Goodhart
Haiyue Wang
Adrien Gallouët
Quentin Bramas
GideonSerf
Xuan-Son Nguyen
Shin-myoung-serp
Jeff Bolz
Ruben Ortlam
lhez
Wagner Bruna
Giuseppe Scrivano