ee96e71bae
don't resample audio
2026-03-09 22:53:55 +08:00
e22cd0aa15
metal : extend mul_mv_ext to BF16, Q2_K, Q3_K ( #20250 )
...
Enable mul_mv_ext small-batch kernels (BS 2-8) for BF16, Q2_K,
and Q3_K quantization types. These types previously fell through
to the slower single-row mul_mv path.
BF16 uses the float4 dequantize path (like F16). Q2_K and Q3_K
use the float4x4 K-quant path (like Q4_K/Q5_K/Q6_K).
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-09 16:48:12 +02:00
96cfc4992c
server : fix checkpoints n_tokens calculation ( #20287 )
2026-03-09 16:47:06 +02:00
ed0007aa32
metal : add upscale ( #20284 )
2026-03-09 16:45:11 +02:00
344ee2a38a
server : warn swa-full is not supported for non-SWA models ( #20291 )
2026-03-09 16:44:25 +02:00
d6e1556499
server : fix off-by-1 in server_tokens::size_up_to_pos() ( #20279 )
...
* server : fix off-by-1 in server_tokens::size_up_to_pos()
* cont : fix typo [no ci]
2026-03-09 16:43:38 +02:00
f76565db92
common: map developer role to system ( #20215 )
...
* Map developer role to system
* Simplify
2026-03-09 14:25:11 +01:00
43e1cbd6c1
models : fix assert in mamba2 graph ( #20270 )
2026-03-09 13:15:15 +02:00
107d599952
server : add kill switch when server is stuck ( #20277 )
2026-03-09 10:33:12 +02:00
e8bbc736cb
ggml-cuda: disable gdn for musa ( #20278 )
2026-03-09 16:15:36 +08:00
b518195101
llama-quant : left-align tensor names in output ( #20117 )
2026-03-09 09:28:41 +02:00
e2763a6723
contributing: limit open PRs for new contributors to 1 ( #20036 )
2026-03-09 15:05:34 +08:00
57570f9bd6
Read the persisted llama_kv_cell_ext for n_pos_per_embd > 1 when doing state_read for all sequence ids
2026-03-09 06:46:36 +00:00
0beb8db3a0
ggml-vulkan: add SGN operator, auto-generate Vulkan.csv and ops.md ( #20219 )
2026-03-09 07:24:16 +01:00
b2f460bd3c
vulkan: skip zero size tensors in backend copies ( #20233 )
2026-03-09 07:23:45 +01:00
5f4cdac385
cuda : display total and free VRAM capacity during device initialization ( #20185 )
2026-03-09 12:45:43 +08:00
ae87863dc1
llama-bench: introduce -hf and -hff flags & use --mmap 1 by default ( #20211 )
2026-03-09 09:05:44 +08:00
97c64fbdbd
PEG parser for LFM2 ( #20251 )
...
* PEG parser for LFM2
* Simplify using python_value()
2026-03-09 01:11:22 +01:00
d417bc43dd
server : do not create checkpoints right after mtmd chunks ( #20232 )
2026-03-08 22:16:46 +02:00
35bee031e1
graph : remove redundant scale_w parameter ( #20235 )
2026-03-08 18:58:28 +01:00
451ef08432
common : gracefully handle incomplete output ( #20191 )
...
* common : handle incomplete UTF-8 at end of input in PEG parser
* cont : if reached end prematurely, emit needs_more_input to propagate partial output
* cont: refactor peg parse context to add lenient flag
* cont : remove partial flag, keep lenient flag
2026-03-08 17:17:02 +01:00
9b24886f78
Fix compile bug ( #20203 )
...
* Fix compile bug
* Update common/chat-auto-parser-helpers.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-03-08 17:15:49 +01:00
62b8143ad2
Fix structured outputs ( #20223 )
...
* Fix structured outputs
* Update common/chat-auto-parser-generator.cpp
Co-authored-by: Aldehir Rojas <hello@alde.dev >
---------
Co-authored-by: Aldehir Rojas <hello@alde.dev >
2026-03-08 17:14:43 +01:00
45c74da08b
adjust ace step, still wip on caption rework
2026-03-09 00:11:48 +08:00
9ddd74111f
OpenBSD changes for vulkan backend ( #2026 )
...
* OpenBSD also needs alloca.h
* Changes to compile vulkan backend with OpenBSD
* Update README.md
tweak details for OpenBSD vulkan backend
* Update README.md
2026-03-08 20:41:36 +08:00
d088d5b74f
ggml-vulkan: Add ELU op support ( #20183 )
...
* ggml-Vulkan: add ELU support
* ggml-Vulkan: remove extra spaces and variables
* ggml-Vulkan: fix format issue
* ggml-Vulkan: fix format issue
* fix whitespace issue
* Update Vulkan.csv and ops.md
2026-03-08 12:38:17 +01:00
cd18a50ea5
vulkan: Fix data races in coopmat1 mul_mat(_id) ( #20084 )
...
* vulkan: Fix data races in coopmat1 mul_mat(_id)
Add barriers between coopmat store and regular loads. We sort of got away with
this because it was the same subgroup accessing the values, but it's still a
race and may not work.
* switch to subgroup control barriers
2026-03-08 12:33:48 +01:00
a976ff081b
llama: end-to-end tests ( #19802 )
...
* tests: add end-to-end tests per model architecture
* fixup for rebase
* fix use-after-free in llama-model-loader.cpp
* fix CI
* fix WebGPU
* fix CI
* disable CI for macOS-latest-cmake-arm64
* use expert_weights_scale only if != 0.0f
* comments
2026-03-08 12:30:21 +01:00
a95047979a
readme : update infra list ( #20212 )
2026-03-08 12:42:28 +02:00
b283f6d5b3
Revert to OAI-compatible args ( #20213 )
...
* Revert to OAI-compatible args
* Apply workaround::func_args_not_string
2026-03-08 11:33:03 +01:00
ff52ee964d
server : correct index on finish in OAI completion streams ( #20226 )
2026-03-08 10:08:57 +01:00
270d4ad2c1
fixed a typo
2026-03-08 12:56:08 +08:00
73fc5c4767
handle jinja exceptions
2026-03-08 12:12:02 +08:00
213c4a0b81
[SYCL] supprt Flash Attention for fp32/fp16/Q4/Q5/Q8 ( #20190 )
...
* support flash-attention for fp32/fp16/Q4/Q5/Q8
* rm warining
* update for JIT
2026-03-08 12:00:07 +08:00
41df8b09e5
jinjatools now works mostly well
2026-03-08 11:55:22 +08:00
a981d1ece9
updated lite
2026-03-08 02:33:18 +08:00
9158bd8b4d
sd: sync to master-520-d950627 ( #2006 )
...
* sd: sync to master-509-4cdfff5
* sd: Anima support
* sd: sync to master-514-5792c66
* sd: additional workaround for Anima .safetensors model
* sd: sync to master-517-ba35dd7
* sd: sync to master-520-d950627
2026-03-08 01:23:03 +08:00
ebe44e7819
modify q3tts loader
2026-03-08 00:53:33 +08:00
0df18d2ae2
fixed single token bans
2026-03-07 22:50:53 +08:00
a40038d8e6
further reverse the mxfp4 changes
2026-03-07 22:42:22 +08:00
c5a778891b
ggml: add GATED_DELTA_NET op ( #19504 )
...
* ggml: add GATED_DELTA_NET op
* remove the transpose
* add KDA
* add qwen35 dense
* llama : check for fused gated delta net backend support
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-03-07 15:41:10 +08:00
6fce5c6a7d
opencl: add l2_norm ( #20160 )
2026-03-06 18:03:05 -08:00
c024d85908
Autoparser: True streaming ( #20177 )
...
* Relax atomicity constraint for nicer, more pleasent, True Streaming parsing
* Whitespace
* Remove redundant atomics
2026-03-07 01:55:33 +01:00
2f2923f895
Autoparser: add optional argument reshuffle capability ( #20171 )
...
* Allow reshuffled arguments in tagged argument parser format tool calls.
* Remove shuffle just keep the optional parsers in any order
* Remove unnecessary import
2026-03-06 22:34:15 +01:00
649f06481e
quants : Add memsets and other fixes for IQ quants ( #19861 )
...
* Add memsets and other fixes for IQ quants
* Make memset unconditional, change Laux back to L
* Move another memset
2026-03-06 23:06:56 +02:00
7463687161
Add @pwilkin to CODEOWNERS for autoparser code ( #20174 )
2026-03-06 21:25:41 +01:00
566059a26b
Autoparser - complete refactoring of parser architecture ( #18675 )
...
* Autoparser - full single commit squish
* Final pre-merge changes: minor fixes, Kimi 2.5 model parser
2026-03-06 21:01:00 +01:00
34df42f7be
hexagon: add f32 ssm_conv op ( #20122 )
...
* hexagon: add ssm_conv op
* hexagon: hvx kernel is functional
* hexagon: improvements to ssm-conv hvx kernel
* hexagon: added dma to ssm-conv hvx kernel
* hexagon: ssm-conv dynamically compute gather scratchpad
* hex-ssm-conv: add local context and fix various issues (spad indexing, etc)
---------
Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com >
2026-03-06 09:59:26 -08:00
e68f2fb894
server : preserve anthropic thinking blocks in conversion ( #20120 )
...
* server : preserve anthropic thinking blocks in conversion (#20090 )
* server : add tests for anthropic thinking block conversion
---------
Co-authored-by: root <root@llamacpp.home >
2026-03-06 17:41:12 +01:00
ba2fd11cdf
cpu: skip redudant ROPE cache updates ( #20149 )
2026-03-06 08:32:40 -08:00
Concedo
Paul Flynn
Georgi Gerganov
Piotr Wilkin (ilintar)
Aman Gupta
ddh0
sid
Bertay Eren
Ruben Ortlam
Michael Huang
Aaron Teo
Sigbjørn Skjæret
Aldehir Rojas
JustCommitRandomness
GiantPrince
Jeff Bolz
Johannes Gäßler
Christopher Maher
decahedron1
Neo Zhang
Wagner Bruna
lhez
Bartowski
Todor Boinovski
Tom Vaucourt
Max Krasnyansky