01c1acc9a2
sdtype_adapter refactor ( #1753 )
...
* reorganize sampler selection in sdtype_adapter
* simplify image metadata generation
Use a stringstream to avoid the explicit conversions to
string. Also, generate from the sd.cpp struct, to reduce
dependencies on the global context fields.
* clean up default parameters in sdtype_adapter
Most image generation parameters are not customizable through
the API, and their values simply repeat the default set by
sd.cpp. So, remove them from the SDParams struct, and place
the needed constant values directly on the code.
Also, print debug information directly from the sd.cpp
parameters. And to help with type mismatches and general
readability, replace the fromatting with stringstreams.
2025-09-24 22:09:17 +08:00
3a59971967
model : add label for LiquidAI LFM2-2.6B model ( #16204 )
...
* model : add label for LiquidAI LFM2-2.6B model
HF link: [LiquidAI/LFM2-2.6B](https://huggingface.co/LiquidAI/LFM2-2.6B ).
Support for GGUF conversion and inference is added in #14620 .
However, due to similar `n_embd`, it identifies as a 1.2B model.
Fix the label by using `n_ff` to identify the model instead.
Output of `llama-bench`:
```
| model | size | params | backend | threads | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: |
| lfm2 1.2B F16 | 2.18 GiB | 1.17 B | CPU | 10 | pp512 | 223.97 ± 5.32 |
| lfm2 2.6B F16 | 4.79 GiB | 2.57 B | CPU | 10 | pp512 | 92.53 ± 4.14 |
| lfm2 350M F16 | 676.25 MiB | 354.48 M | CPU | 10 | pp512 | 725.52 ± 11.70 |
| lfm2 700M F16 | 1.38 GiB | 742.49 M | CPU | 10 | pp512 | 336.22 ± 12.93 |
```
* Update src/llama-model.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2025-09-24 13:42:26 +02:00
5d89a48a50
add more rnn models supported
2025-09-24 18:14:59 +08:00
c7a1eec4e4
try to solve ttscpp oom regression
2025-09-24 17:45:28 +08:00
63b54c81a6
model-conversion : make causal-verify-logits fails with model names containing "." ( #16215 )
...
Signed-off-by: Jie Fu <jiefu@tencent.com >
2025-09-24 10:25:26 +02:00
152729f884
common : add missing chrono header for common.cpp ( #16211 )
...
Signed-off-by: Uilian Ries <uilianries@gmail.com >
2025-09-24 09:53:47 +03:00
c0c59c1157
codeowners : match all requirements files ( #16214 )
2025-09-24 08:53:20 +02:00
7735706b93
model-conversion : run-org-model.py fails to run on mac m1 ( #16213 )
...
Signed-off-by: Jie Fu <jiefu@tencent.com >
2025-09-24 08:46:52 +02:00
d3f9db8d33
fix system32 writability check
2025-09-24 14:43:41 +08:00
4d9ea03d17
codeowners : use slash prefix for root files [no ci] ( #16210 )
...
This commit adds a leading slash to the paths of root-level files
in the CODEOWNERS file.
The motivation for this is that these might otherwise match files
in subdirectories that have other/additional owners will override them.
Refs: https://github.com/ggml-org/llama.cpp/pull/16209#issuecomment-3326434274
2025-09-24 08:10:09 +02:00
8ba548dae2
model-conversion : fix the make targets in the README.md ( #16209 )
...
Fix two incorrect make targets in the readme.
Signed-off-by: Jie Fu <jiefu@tencent.com >
2025-09-24 06:19:23 +02:00
f505bd83ca
ci : disable AMD workflows + update NVIDIA workflows ( #16200 )
...
* ci : disable AMD workflows + update NVIDIA workflows
* cont : fixes
* cont : update nvidia vulkan workflows
2025-09-23 20:41:40 +03:00
84a67d289c
tts show model filename
2025-09-23 22:42:43 +08:00
0889589dbe
ci : enable Vulkan workflow on Mac ( #16194 )
2025-09-23 13:44:25 +03:00
4e29084ba4
ggml-cpu: Respect cpumask settings ( #16164 )
2025-09-23 11:58:12 +03:00
f6b4af3d04
ggml : fix uninitialized is_on_grid in quantize_row_iq3_xxs_impl ( #15928 )
...
* fix uninitialized is_on_grid in quantize_row_iq3_xxs_impl
* change initialization to true
2025-09-23 10:25:20 +02:00
264f1b5187
zdnn: refactor codebase + add docs ( #16178 )
...
* zdnn: initial matmul refactor
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: rm static from funcs
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: update ggml-zdnn.h
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: change header files to hpp
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: switch to common.hpp
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: move mulmat forward around
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: rm inline from utils
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* ggml-zdnn: code cleanup
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* docs: add zDNN docs
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
---------
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
2025-09-23 14:53:05 +08:00
0bc7cc7154
codeowners : add @danbev to model-conversion example [no ci] ( #16190 )
...
This commit adds examples/model-conversion/ to the CODEOWNERS file and
assigns myself (@danbev) as the code owner for this directory.
2025-09-23 09:13:22 +03:00
4b9f4cb0f8
devops: add s390x containers ( #15915 )
...
* devops: add s390x dockerfile
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add missing ninja
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: move s390x docker into cpu docker
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: rework s390x docker
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: copy more tools
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add server build step
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove apt clean steps as distroless misses it
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove apt commands from distroless
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix shared libs in distroless
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: use correct libs path
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix shared libs
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add collector stage
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix missing stage ref
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix permission issue
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix unknown model loading failures
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: attempt at fixing model loading failure
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix missing ggml shared object
failure to load model
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove move shared objects
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: move libggml-cpu and blas into bin
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: finalise hardened server stage
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add cli target
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix typos
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix missing shared libraries in base
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: update debian target
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: formalise llama.cpp loc
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* Revert "devops: formalise llama.cpp loc"
This reverts commit 0a7664af8466a15f318ff209e02ac3c4e551cc18.
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: formalise llama.cpp loc
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
(cherry picked from commit 0a7664af8466a15f318ff209e02ac3c4e551cc18)
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: attempt at fixing missing dir
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: attempt at making it cache the build
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix copying process
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: make build dir an argument
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* Revert "devops: make build dir an argument"
This reverts commit 438698976b8a5181c1e8179600527cfd5a50cc23.
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add build stage for gguf-py
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: move gguf-py installation into build stage
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: break system packages?
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add rust compiler installer
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: fix rustc not found
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove cache mount to allow rustc to persist
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: move rustc installation to another layer
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: move gguf-py installation to full stage, fix copying
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove rustc installation in build
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: disable full target for now
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: attempting static build
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: merge s390x dockerfile into cpu for now
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: switch to gcc image for build step
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove build essentials
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: install openblas into base target
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: go back to s390x dockerfile
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: remove libggml and libblas
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add full target
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add break system packages
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add libjpeg
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add missing cmake dep
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: finalise docker images for s390x
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add custom openblas patch
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: use libopenblas-dev instead of libopenblas-openmp-dev
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* devops: add s390x docker build
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
---------
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
2025-09-23 13:59:34 +08:00
85e72271ba
ggml-cpu : fix typo in gemm comments [no ci] ( #16189 )
2025-09-23 05:59:03 +02:00
1d0125bcf1
feat: Add conversion support in GraniteHybrid for non-hybrid (all attn) ( #16177 )
...
This is a configuration of the hparams in the GraniteHybrid architecture
that devolves to the Granite (or GraniteMoe) architecture (ie Granite 3.x).
It may be used for some models in the Granite 4 family with the
GraniteHybrid architecture acting as a superset arch. Rather than support
it directly in the c++ graph, we simply coerce the architecture flag back
to the correct "granite" or "granitemoe" architecture.
Branch: gabe-l-hart/GraniteNonHybridConversion
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2025-09-22 20:40:10 +02:00
351f3da39c
clang-tidy : disable warning about performance enum size ( #16127 )
...
Disable 'performance-enum-size' checking:
Enum 'llama_token_type' uses a larger base type ('unsigned int', size: 4 bytes)
than necessary for its value set, consider using 'std::uint8_t' (1 byte) as the
base type to reduce its size.
2025-09-22 19:57:46 +02:00
3ecb2f671a
ggml : implement set_rows with i32 index ( #16159 )
...
* implement set_rows with i32 index
* template fix
* test quantized path
warnings--
* Apply suggestions from code review
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* forgotten name change
* deduplicate cuda/sycl and test-fix
* indent++
* vulkan: support set_rows with i32 index type (#16162 )
* disable i32 index for webgpu for now
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
Co-authored-by: Jeff Bolz <jbolz@nvidia.com >
2025-09-22 19:13:00 +02:00
432cf4304c
codeowners : update + cleanup ( #16174 )
...
---------
Co-authored-by: slaren <slarengh@gmail.com >
2025-09-22 18:20:21 +03:00
174d00bb74
fix aria2c with both download cases
2025-09-22 21:47:08 +08:00
efe546390b
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# CODEOWNERS
# CONTRIBUTING.md
# README.md
# ci/run.sh
# examples/embedding/README.md
# tests/test-backend-ops.cpp
2025-09-22 21:25:29 +08:00
59b6a09ae1
try to fix kokoro alloc again
2025-09-22 21:22:41 +08:00
37a23c17bd
common : enable --offline mode without curl support ( #16137 )
...
* common : use the json parser
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
* common : enable --offline mode without CURL support
This change refactors the download logic to properly support offline mode
even when the project is built without CURL.
Without this commit, using `--offline` would give the following error:
error: built without CURL, cannot download model from the internet
even if all the files are already cached.
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
---------
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2025-09-22 15:13:51 +03:00
138c87ce8b
webui : fix handling incomplete chunks ( #16107 )
2025-09-22 11:53:13 +03:00
c6db9a1027
embedding : fix typos in README ( #16171 )
2025-09-22 11:49:58 +03:00
d05affbab7
common : remove unused local variables ( #16140 )
...
These two local variables 'arg' and 'arg_prefix' have been overriden by:
1. for (const auto & arg : opt.args)
2. for (int i = 1; i < argc; i++) {
const std::string arg_prefix = "--";
std::string arg = argv[i];
2025-09-22 11:48:42 +03:00
4f324a556c
ggml : extend ggml_can_fuse to work with non-sequential nodes ( #16123 )
...
* ggml : extend ggml_can_fuse to work with non-sequential nodes in the graph
* cont : fix wrong bounds check condition
* cont : remove unnecessary overload
2025-09-22 11:12:37 +03:00
a71ae3ba7a
ggml : add ggml_op_is_empty ( #16122 )
...
* ggml : add ggml_op_is_empty
* ggml : move to ggml-impl.h
2025-09-22 11:12:09 +03:00
05a2458121
codeowners : update ownership for @ngxson and @allozuar ( #16128 )
2025-09-22 11:10:58 +03:00
96fdca043b
Vulkan: add conv_transpose_2d operation ( #16022 )
...
* Vulkan: add conv_transpose_2d operation
* Vulkan: fix typo in conv_transpose_2d shader(s0mp, s0L, s1mp, s1L)
* Vulkan: fix incorrect indentation in conv_transpose_2d shader
* Vulkan: add checking the push constants size limit and reuse conv2d_mm.comp for conv_transpose_2d operation
* Vulkan: revert the order of the index calculation and bound check in conv_2d shader
* Vulkan: explicity check push constants limit in supports_op() for conv_transpose_2d operation.
* Vulkan: remove unnecessary lower bound checks for H/W_idx in the conv_2d shader.
2025-09-22 10:04:01 +02:00
b2d980fce0
codeowners : claim responsibility for ci, models, gguf-py and convert ( #16124 )
...
* claim responsibility for ci, gguf-py and convert
* add myself to various src/llama- files
2025-09-22 10:59:05 +03:00
5c6106a696
contrib : update roles ( #16113 )
...
* contrib : update roles
* contrib : merge PR sections + add link to CI instructions
Updated pull request guidelines for contributors and collaborators, and clarified merging practices for maintainers.
2025-09-22 10:58:02 +03:00
ec65fb52f0
ci : remove vulkaninfo calls ( #16169 )
2025-09-22 10:16:05 +03:00
1d660d2fae
ci : use smaller model ( #16168 )
...
* ci : switch from gemma to qwen3 0.6b
* ci : use smaller model for some tests
2025-09-22 09:11:39 +03:00
216b766aee
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-riscv-native.yml
# .github/workflows/build.yml
# ci/README.md
# ci/run.sh
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# tests/test-backend-ops.cpp
2025-09-22 13:56:02 +08:00
b5931c98ba
Revert "Revert "vulkan: use vec dot for matrix matrix multiplications ( #16056 )""
...
This reverts commit 3170fc38cb
2025-09-22 13:49:16 +08:00
08c0246f24
prioritize cwd for downloads if its writable
2025-09-22 13:47:50 +08:00
a20d810d79
vulkan: add RTE variants of exp shader ( #16165 )
...
This fixes some failures on Turing where "round to zero" rounds to the max f16
value but the CPU reference value is infinite.
2025-09-22 07:37:17 +02:00
f50840455b
revert more mistral imposed dependencies
2025-09-22 13:32:51 +08:00
4d0a7cbc61
ci : adjust params for less runtime ( #16167 )
...
* ci : adjust params for less runtime
* ci : gate BF16 on some hardware
* ci : move extra tests to Arm runner
2025-09-22 08:31:40 +03:00
9073a73d82
vulkan: vec dot matrix multiplication fix ( #16151 )
...
* vulkan: fix matrix multiplication index calculation for odd m/n and odd k in combination with batching
* add odd m/n + odd k test with batching
2025-09-22 07:22:43 +02:00
13bee0d39d
some minor fixes
2025-09-22 13:20:06 +08:00
51f5a45fbe
opencl: fix concat crash on win arm64 with Adreno ( #15944 )
2025-09-21 16:42:10 -07:00
c4510dc937
opencl: initial q8_0 mv support ( #15732 )
2025-09-21 14:48:44 -07:00
da30ab5f86
ci : add label for the RISC-V runner ( #16150 )
2025-09-21 19:00:27 +03:00
Wagner Bruna
Tarek Dakhran
Concedo
Jie Fu (傅杰)
Uilian Ries
Sigbjørn Skjæret
Daniel Bevenius
Georgi Gerganov
Xiangyan Sun
Aaron Teo
Gabe Goodhart
Haiyue Wang
Adrien Gallouët
Quentin Bramas
GideonSerf
Xuan-Son Nguyen
Shin-myoung-serp
Jeff Bolz
Ruben Ortlam
lhez