067b8d7af3
Revert "vulkan: force full subgroups for flash attention to fix intel subgroup crash ( #17356 )" ( #18831 )
...
This reverts commit 980b7cd17e
2026-01-21 17:13:43 +01:00
50b7f076a5
vulkan: Use mul_mat_vec_id for small values of n ( #18918 )
...
Change ggml_vk_mul_mat_vec_id_q_f16 to loop over the batch dimension and
update the indexing calculations in get_offsets.
Mat-vec is faster than mat-mat for small values of n. We don't get the same
reuse of the weights as in the non-ID path, but with this the cost is linear
in n rather than n>1 being far slower than n==1.
2026-01-21 16:22:02 +01:00
4984c9bc16
Merge commit '12a4a47e6aaf691492644c39da453745aaee1672' into concedo_experimental
...
# Conflicts:
# ci/run.sh
# examples/model-conversion/scripts/causal/run-converted-model-embeddings-logits.sh
# examples/model-conversion/scripts/causal/run-converted-model.sh
# examples/model-conversion/scripts/embedding/run-converted-model.sh
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-zdnn/ggml-zdnn.cpp
# ggml/src/ggml-zendnn/ggml-zendnn.cpp
# tests/CMakeLists.txt
# tests/test-chat-parser.cpp
# tests/test-chat-peg-parser.cpp
# tests/test-chat.cpp
# tools/cli/cli.cpp
2026-01-21 21:00:44 +08:00
715a76ce5f
sd: sync to master-480-b87fe13 ( #1932 )
2026-01-21 20:49:35 +08:00
ad8d85bd94
memory : add llama_memory_hybrid_iswa ( #18601 )
...
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Successful in 32s
Python check requirements.txt / check-requirements (push) Failing after 9s
Python Type-Check / pyright type-check (push) Failing after 37s
* memory : add llama_memory_hybrid_iswa
* Update src/llama-memory-hybrid-iswa.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-01-21 14:30:23 +02:00
12a4a47e6a
Fix GLM 4.7 Lite MoE gating func ( #18980 )
...
* Fix GLM 4.7 MoE gating func
* Update src/models/deepseek2.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update src/llama-model.cpp
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com >
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com >
2026-01-21 12:35:20 +01:00
28091dec43
pipeline parallel default enable
2026-01-21 17:57:41 +08:00
37c35f0e1c
gguf: display strerrno when cant load a model ( #18884 )
...
I've had issues loading models with llama-server:
[44039] E gguf_init_from_file: failed to open GGUF file 'mistral-7b-v0.1.Q8_0.gguf'
and I was sure it could access the file. Seems like --models-dir and
--models-presets dont interact like I thought they would but I salvaged
this snippet that helps troubleshooting
[44039] E gguf_init_from_file: failed to open GGUF file 'mistral-7b-v0.1.Q8_0.gguf' (errno No such file or directory)
2026-01-21 08:52:46 +02:00
5bd341c9a1
CUDA: Fix builds for older CCCL versions by ifdefing strided_iterator ( #18964 )
...
* CUDA: Fix builds for older CCCL versions by ifdefing strided_iterator
Strided iterator was added in [CCCL
3.1](https://github.com/NVIDIA/cccl/releases/tag/v3.1.0 ), which is packaged into
[CTK
13.1](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#id5 )
* Unindent as per code review request
2026-01-21 02:34:29 +01:00
1c7cf94b22
common, server : use the same User-Agent by default ( #18957 )
...
This commit also ensures that if a custom User-Agent is used, it will be
the only one sent.
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-01-20 18:28:43 +01:00
2c1f199653
cli : fix reasoning responses in CLI ( #18961 )
...
* cli : fix reasoning responses in CLI
* fix build
* fix build (2)
2026-01-20 18:23:25 +01:00
cdd6578a9a
esrgan added
2026-01-20 22:10:37 +08:00
d1e3556481
CUDA: Replace init_offsets kernel with iterators in cub-based argsort ( #18930 )
...
* CUDA: Replace `init_offsets` with iterators in argsort
This is a QOL improvement, saving us the cost of materializing the
iterator
* Remove unnecessary include from top-k.cu
2026-01-20 20:11:01 +08:00
08f3f4a8a3
ggml : cleanup path_str() ( #18928 )
...
- Remove pragmas as `std::codecvt_utf8` is not used.
- Avoid implicit `strlen()`.
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-01-20 11:42:49 +01:00
271191906c
metal : enable FA for MLA heads ( #18950 )
2026-01-20 12:21:28 +02:00
7dee9ff59a
convert : use n_groups instead of hardcoded values in reshape ( #18929 )
...
* convert : use n_groups instead of hardcoded values in reshape
This commit modifies the conversion script for NemotronHModel to use
the 'n_groups' hyperparameter, and allow Python to calculate the the
last dimension, using -1, when reshaping the 'mixer.norm.weight' tensor.
* use self.n_group instead of self.hparams["n_groups"]
2026-01-20 06:55:24 +01:00
6df686bee6
server : refactor oai_parser_opt, move it to server_chat_params ( #18937 )
...
* server_chat_params
* move chat format into CLI
* use meta whenever possible
* clean up, no more chatml fallback
2026-01-19 23:28:01 +01:00
1706a6d7c6
convert : support Glm4MoeLite ( #18936 )
...
* initial commit for branch
* add glm-4.7-flash, move tokenizer hash
* use `glm4` pretok
* silence flake8 E302 (CI)
* apply review feedback
* add <|user|> as eog
* also add EOG `<|observation|>`
* revert llama-vocab
* inherit vocab from glm4
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co >
2026-01-19 23:09:20 +01:00
959ecf7f23
jinja : fix undefined keys and attributes and int/float as bool ( #18924 )
...
* fix undefined keys and attributes
* add falsy tests
* as_bool for integers and floats
* more falsy/truthy tests
* --typo
2026-01-19 20:29:43 +01:00
4037093c66
ci : run test-jinja -py on high perf [no ci] ( #18916 )
2026-01-19 20:29:15 +01:00
18361c579c
server: fix memory reservations in populate_token_probs ( #18787 )
2026-01-19 19:13:31 +01:00
365a3e8c31
ggml : add ggml_build_forward_select ( #18550 )
...
* ggml : add ggml_build_forward_select
* cuda : adapt CUDA graph compat to new feature
* vulkan : update logic to handle command buffer closing
* ggml : check compute for fusion
* ggml : add comment
2026-01-19 20:03:19 +02:00
c9c15749e0
wip on adding esrgan upscaling
2026-01-20 00:35:35 +08:00
393791496d
flux 2 taesd (+1 squashed commits)
...
Squashed commits:
[adfc3f3a2] flux 2 taesd
2026-01-19 23:47:16 +08:00
3d55846a5c
model-conversion : add BUILD_DIR variable to run-converted-model scripts ( #18927 )
...
This commit adds a BUILD_DIR variable to the scripts used for running
converted models.
The motivation for this is that currently the `build` directory is
hardcoded and it can be useful to specify a different build directory,
with builds for different configurations.
2026-01-19 13:12:38 +01:00
d827494f17
fix text for vae (+1 squashed commits)
...
Squashed commits:
[793caed19] fix text
2026-01-19 01:50:07 +08:00
70f92b12f8
sdxs clamp steps and cfg
2026-01-19 01:07:27 +08:00
10851f223d
sd: sync to master-473-9565c7f ( #1927 )
...
* sd: sync to master-473-9565c7f
* sd: add support for flux2 klein
2026-01-19 01:04:34 +08:00
287a33017b
llama : Extend fallback, fix fileno for dio file, exclude case that mmap uses dio file ( #18887 )
2026-01-18 18:35:57 +02:00
7f618454ff
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/labeler.yml
# CODEOWNERS
# docs/backend/OPENCL.md
# docs/ops.md
# docs/ops/CANN.csv
# docs/ops/WebGPU.csv
# ggml/src/ggml-blas/CMakeLists.txt
# ggml/src/ggml-opencl/kernels/mul_mv_q6_k.cl
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/cpy.tmpl.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/set_rows.wgsl
# tests/test-backend-ops.cpp
2026-01-18 23:24:29 +08:00
293a1565dc
docs: add linux to index ( #18907 )
2026-01-18 18:03:35 +08:00
3ba3d15fe3
fixed a typo
2026-01-18 16:41:34 +08:00
95ebfdcde8
Add token ids to logprob data returned by the API ( #1928 )
...
Previously, logprobs only contained the token string
and byte data, as well as the log probability itself.
For workflows that require the token id, translating
from the token bytes to the token id is potentially
costly and unreliable. It is simple and inexpensive
to expose the numeric token ids directly instead.
2026-01-18 16:30:46 +08:00
7b4517c2fe
embeddings memory usage regression fix
2026-01-18 16:26:52 +08:00
fe44d35574
tests : add test-jinja -py option for cross-checking ( #18906 )
...
* tests : add test-jinja -py option or cross-checking
* Update tests/test-jinja.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* fix + add source
* SandboxedEnvironment
* fix array.map case
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-01-18 08:14:27 +01:00
3816391a74
increase logprobs returned to 10
2026-01-18 11:13:42 +08:00
22ddad81b9
device override set in gui
2026-01-18 10:54:20 +08:00
bbcdac0189
jinja : fix object item order (and properly implement dictsort) ( #18904 )
...
* fix object item order
* as_ordered_object
* copy whole object
2026-01-18 03:40:06 +01:00
d03c45c9c5
jinja : attribute support for join, map and sort ( #18883 )
...
* support negative array index and default value
* attribute support (int and str) for join, map and sort
* add tests
* update CODEOWNERS
* improve fixme sorting comment
2026-01-18 02:53:01 +01:00
10c98cbdf6
jinja : add missing tojson filter for bool ( #18900 )
...
* add missing tojson for bool
* add more literal tests
2026-01-18 01:05:09 +01:00
420960ab92
jinja : fix lexing of float literals with sign ( #18901 )
...
* fix lexing of float literals with sign
* add test
* consume_numeric
2026-01-18 00:57:51 +01:00
f55b033ae6
jinja: correct member access rule ( #18905 )
2026-01-18 00:48:55 +01:00
d1b4757ded
opencl: fix q6_K mv for m=1 ( #18893 )
2026-01-17 13:50:32 -08:00
57c0beaed0
ci : add label for jinja changes ( #18903 )
2026-01-17 21:52:02 +01:00
2fbde785bc
kv-cache : optimize KQ mask construction ( #18842 )
...
* kv-cache : optimize KQ mask construction
* cont : add explanation + improve
* cont : fix
2026-01-17 15:42:42 +02:00
ac3392d5da
updated lite
2026-01-17 21:40:53 +08:00
89a205ecc7
bump version
2026-01-17 19:09:14 +08:00
62bea5ef4f
allow overriding the devices directly
2026-01-17 19:08:06 +08:00
21e6ccb8cb
updated lite
2026-01-17 15:59:53 +08:00
d2b2224b0d
vulkan env var always take priority
2026-01-17 10:34:45 +08:00
Masato Nakasaka
Jeff Bolz
Concedo
Wagner Bruna
Tarek Dakhran
Piotr Wilkin (ilintar)
Matthieu Coudron
Oliver Simons
Adrien Gallouët
Xuan-Son Nguyen
Georgi Gerganov
Daniel Bevenius
ddh0
Sigbjørn Skjæret
Lennart Austenfeld
Julius Tischbein
Francisco Herrera
Llama
lhez