4a7d6dd8a0
alias for draft
2026-06-23 18:43:28 +08:00
a3900a6694
model: Granite Speech Plus ( #24818 )
...
* feat: Add conversion support for Granite Speech Plus
Branch: GraniteSpeechPlus
AI-usage: full (Bob, OpenCode + Qwen3.6-35b)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
* feat: Extend granite_speech to support plus multi-layer concatenation
Branch: GraniteSpeechPlus
AI-usage: draft (Bob, OpenCode + Qwen3.6-35b)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
* fix(conversion): Fix plural naming for feature_layers for audio
Branch: GraniteSpeechPlus
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
* fix(mtmd): Align feature_layer usage and naming everywhere
Branch: GraniteSpeechPlus
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
* style: Use fstring for log
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com >
---------
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com >
2026-06-23 12:03:31 +02:00
7c908502ea
ggml-webgpu: improve MTP inference by using mat-vec path for small batches ( #24811 )
...
* ggml-webgpu: improve small batches decoding
* Add barrier to the NUM_COLS loop in mul-mat-vec
2026-06-23 17:13:55 +09:00
035cd8f9a6
codeowners: add yomaytk to ggml-webgpu ( #24930 )
2026-06-23 15:19:34 +09:00
73618f27a8
server: improve user message detection and create checkpoints at every user message ( #24176 )
...
* server : improve message span logic
* cont : cast size_t to int32_t in comparisons
* server : create checkpoints before every user msg
* chat : remove \n in gemma4 delimiters
* chat : merge msg delimiter structs into one
* cont : reword comment
* cont : initialize tokens in delimiter
* cont : add server_tokens::get_raw_tokens() for mtmd
* cont : move message finding to server_tokens and skip mtmd tokens
* cont : update cohere2moe parser
* cont : increase min-step to 8192 and always produce a chkpt for last user message
2026-06-23 08:27:28 +03:00
23ee8797e1
opencl: q8_0 gemv precision improvement ( #24923 )
2026-06-22 22:25:21 -07:00
dec5ca5577
server : Add id to tool call responses api ( #24882 )
2026-06-22 23:03:12 +02:00
9c0ac887f3
ui: Prioritize favorite models in model selection ( #24766 )
...
Updated model selection prioritization to include favorite models.
2026-06-22 21:00:21 +02:00
721354fbdf
server: (router) move model downloading to dedicated process ( #24834 )
...
* server: real-time model load progress tracking via /models/sse
* update docs
* server: move model download to child process
* rm unused
* fix most problems
* clean up
* nit fixes
* fix test case
* do not detact() thread
* shorter MODEL_DOWNLOAD_TIMEOUT in test
* throttle
2026-06-22 18:24:04 +02:00
6ee0f65793
server: refactor/generalize input file schema ( #24299 )
...
* server: refactor/generalize input file schema
* wire up input_video, accept raw base64
* nits
* nits (2)
* fix windows
2026-06-22 16:42:47 +02:00
099b579acb
ui: model status and load progress via /models/sse feed ( #24878 )
...
* ui: model status and load progress via /models/sse feed
* ui: centralize SSE wire-format delimiters into shared constants for the chat and /models/sse parsers
* ui: type /models/sse event names as a ServerModelsSseEventType enum
Address review from allozaur
2026-06-22 15:55:30 +02:00
7fe6fa6fb6
match draft defaults
2026-06-22 21:40:00 +08:00
a0f39fe0f5
allow drafting with vision
2026-06-22 20:50:39 +08:00
f8cc15f163
[SYCL] support bf16 on bin_bcast OP and unary OPs ( #24838 )
...
* support bf16 on bin_bcast OP and unary OPs
* support the older Intel compiler than 2026.0
2026-06-22 14:09:02 +03:00
37957e8531
sampling : remove unconditional softmax+sort in top-n-sigma sampler ( #22645 )
2026-06-22 14:08:32 +03:00
e4771e8e6b
restore draft state before main state, fixes reloading gemma4 assistant
2026-06-22 18:55:26 +08:00
3090ae0bf7
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/s390x.Dockerfile
# .dockerignore
# .github/workflows/docker.yml
# .github/workflows/release.yml
# docs/android.md
# ggml/src/ggml-cpu/amx/mmq.cpp
# ggml/src/ggml-hexagon/htp/ssm-conv.c
# tests/peg-parser/test-gbnf-generation.cpp
# tests/test-arg-parser.cpp
# tests/test-chat.cpp
# tests/test-jinja.cpp
# tests/test-json-schema-to-grammar.cpp
# tools/server/README.md
2026-06-22 18:23:59 +08:00
d0f9d2e5ac
server: fix edit_file crash on append at end of file (line_start -1) ( #24893 )
...
line_start -1 normalized to n+1, so append inserted at lines.begin() + n + 1,
one past end() -> heap-buffer-overflow in vector::_M_range_insert.
Normalize -1 to n (insert at end()), restrict -1 to append mode and reject it
for replace/delete instead of silently clobbering the last line. Parenthesize
the insert offset so empty-file append computes the position as int first,
avoiding a transient begin() - 1 on a null vector data pointer.
2026-06-22 10:55:28 +02:00
0ef6f06d55
docs/android.md: Add dependency libandroid-spawn for building in termux ( #21812 )
...
Fixes https://github.com/ggml-org/llama.cpp/issues/18615
2026-06-22 05:48:31 +02:00
52b3df0023
common/peg : implement ac parser for stricter grammar generation ( #24869 )
...
* common/peg : implement ac parser
* cont : extract functions
* cont : tidy up
* cont : remove a test
* cont : move ac() def
2026-06-21 16:20:58 -05:00
dfa1c573c4
fix router mode on lcpp ui
2026-06-22 00:21:19 +08:00
7c082bc417
server: fix report progress for loading spec models, add "stages" list ( #24870 )
...
* server: fix report progress for loading spec models, add "stages" list
* improve
* nits
* nits 2
2026-06-21 17:36:52 +02:00
a072dd8304
llama-ui update ( #2281 )
2026-06-21 23:02:16 +08:00
08fbef5049
lcpp ui think budget
2026-06-21 22:56:10 +08:00
44bcead521
mcp fixes
2026-06-21 22:28:47 +08:00
1afe5a730a
minor fixes to handler newer lcpp ui
2026-06-21 22:21:10 +08:00
bddfd2b113
server: refactor batch construction ( #24843 )
...
* server: refactor batch construction
* wip
* wip 2
* wip 3
* wip 4
* add abort_all_slots
* handle batch full more carefully
* fix assert
* rm debug log
* small nits
* (debug) add timings
* debug: force llama_synchronize for accurate timings
* address comments
* disable DEBUG_TIMINGS
2026-06-21 14:16:11 +02:00
0d135df48c
mtmd: fix mtmd_get_memory_usage ( #24867 )
2026-06-21 14:12:15 +02:00
bf533823cd
jinja : implement call statement ( #24847 )
...
* implement call statement
* undo unintended change
* de-lambda
* simplify
* move caller context inside function handler
2026-06-21 14:04:52 +02:00
2f89acc2bc
mtmd: add load progress callback ( #24865 )
2026-06-21 13:40:52 +02:00
bfa3219177
server: add "verbose" field to schema ( #24864 )
2026-06-21 13:03:14 +02:00
d6d899580d
server: real-time model load progress tracking via /models/sse ( #24828 )
...
* server: real-time model load progress tracking via /models/sse
* update docs
* add mutex for notify_to_router
* correct docs
2026-06-21 11:58:14 +02:00
8a118ee86c
minor : clean-up whitespaces ( #24862 )
...
[no ci]
2026-06-21 11:37:12 +03:00
d789527482
spec : Support Step3.5/3.7 flash mtp3 ( #24340 )
...
* add mtp_layer_offset + include nextn flags in graph reuse
* add llama_set_mtp_layer_offset + llama_model_n_nextn_layer API
* offset head select + require all MTP blocks
* speculative multi-head process()
* speculative multi-head draft()
* gather outputs via inp_out_ids
* cleanup
* fix core
* minor cleanup
* merged draft_multi_head into draft()
* mtp rename nextn
* Apply suggestions from code review
Co-authored-by: Aman Gupta <amangupta052@gmail.com >
* clean-up comments
* fix for multi seq
* apply suggestions && chain-heads comment
* add a reference for chain_heads discussion
---------
Co-authored-by: Aman Gupta <amangupta052@gmail.com >
2026-06-21 11:33:18 +03:00
f202c0a457
mtp init -2
2026-06-21 10:21:48 +08:00
063d9c156e
common/peg : refactor until gbnf grammar generation ( #24839 )
...
* common/peg : refactor until gbnf grammar into an ac automaton
* cont : add a test with multiple strings
* cont : pad state with 0s so rules line up
* cont : clean up comments
* cont : use set everywhere
* cont : inline state num string padding
* cont : add a ref to PR
* cont : fix regression in server-tools.cpp
2026-06-20 21:15:06 -05:00
c57607016a
common/json-schema-to-grammar : align spacing rules with parsers ( #24835 )
2026-06-20 17:43:04 -05:00
4a80943174
fix(hexagon): use padded stride for ssm-conv weights ( #24470 )
2026-06-20 14:58:49 -07:00
84de01a1f1
llama : use LLM_KV for quantization_version & file_type ( #24802 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-06-20 20:07:01 +02:00
75f460ac28
arg: try fixing test-args-parser randomly fails ( #24826 )
...
* arg: try fixing test-args-parser randomly fails
* return ref
* try triggering the workflow
* exception wrapper
* wip
* test
* test 2
* arg: guard win32 utf8 argv override
make_utf8_argv rebuilds argv from GetCommandLineW to fix utf8 handling of
non ascii arguments on windows. the override runs unconditionally inside
common_params_parse, so it also clobbers a programmatic argv passed by a
caller. test-arg-parser builds a synthetic argv but then sees the real
process command line instead, the model argument is never parsed, and the
assert that expects success aborts via fastfail (0xC0000409). this shows up
as a random failure in the openvino windows workflow.
only override argv when its length matches the caller argc, so the utf8
repair still applies to real binaries while a programmatic argv stays intact.
---------
Co-authored-by: Pascal <admin@serveurperso.com >
2026-06-20 19:45:27 +02:00
8452824611
release: add missing link for win opencl adreno arm64 ( #24809 )
2026-06-20 23:08:59 +08:00
a5019767c3
docs
2026-06-20 23:02:56 +08:00
e27f308597
server: avoid forwarding auth headers in CORS proxy ( #24373 )
...
* server: avoid forwarding auth headers in CORS proxy
* format
* fix test
* fix e2e test
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co >
2026-06-20 15:34:47 +02:00
67e9fd3b74
docker : prebuild web UI for s390x build [no release] ( #24829 )
2026-06-20 05:54:42 -05:00
796f41bedc
model : glm-dsa load DSA indexer tensors as optional ( #24770 )
...
GLM-5.2 ships the DSA "lightning indexer" on only a subset of layers (the
"full" layers; others omit it), but the GLM_DSA loader created the five
indexer tensors on every layer as required, so loading any GLM-5.2 GGUF
failed with e.g. `missing tensor 'blk.3.indexer.k_norm.weight'`.
GLM_DSA's graph is llama_model_deepseek2::graph (plain MLA) and does not use
the indexer tensors (indexer runtime not yet implemented), so they are
loaded-but-unused. Marking them TENSOR_NOT_REQUIRED lets layers without an
indexer load as nullptr and the model runs as full MLA attention.
DeepSeek-V3.2 (uniform indexer on all layers) is unaffected.
2026-06-20 13:48:24 +03:00
37a77fb057
ggml : optimize AMX ( #24806 )
...
Flatten the partition over n_batch * M so every thread participates in
the quantization
| CPU | Model | Test | t/s OLD | t/s NEW | Speedup |
|:--------------------------------|:------------------------------|:-------|----------:|----------:|----------:|
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_NL - 4.5 bpw | pp512 | 730.71 | 779.86 | 1.07 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_NL - 4.5 bpw | tg128 | 87.88 | 86.79 | 0.99 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_XS - 4.25 bpw | pp512 | 725.09 | 1023.31 | 1.41 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_XS - 4.25 bpw | tg128 | 83.64 | 83.62 | 1.00 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_0 | pp512 | 820.51 | 924.05 | 1.13 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_0 | tg128 | 90.59 | 92.46 | 1.02 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_1 | pp512 | 776.88 | 872.79 | 1.12 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_1 | tg128 | 89.39 | 90.94 | 1.02 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_M | pp512 | 719.28 | 1009.27 | 1.40 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_M | tg128 | 80.62 | 80.86 | 1.00 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_S | pp512 | 732.29 | 1077.29 | 1.47 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_S | tg128 | 86.42 | 83.53 | 0.97 |
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-06-20 13:43:06 +03:00
f4043fec01
convert : more consistent handling of rope_parameters ( #24833 )
2026-06-20 13:42:36 +03:00
73cc7d9287
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/openvino.Dockerfile
# .devops/rocm.Dockerfile
# .devops/s390x.Dockerfile
# .devops/vulkan.Dockerfile
# .devops/zendnn.Dockerfile
# .dockerignore
# .pi/gg/SYSTEM.md
# ggml/CMakeLists.txt
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# scripts/sync-ggml.last
# scripts/sync_vendor.py
# tools/cli/README.md
# tools/cli/cli.cpp
# tools/mtmd/clip.cpp
# tools/server/README.md
2026-06-20 17:16:19 +08:00
84b8856295
Merge commit '32eddaf2ea8dd5d499dee9655592a89b91bfde9d' into concedo_experimental
...
# Conflicts:
# docs/multimodal.md
# docs/preset.md
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hex-dma.h
# ggml/src/ggml-hexagon/htp/hex-utils.h
# ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/hmx-queue.c
# ggml/src/ggml-hexagon/htp/hmx-queue.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# scripts/snapdragon/ggml-hexagon-profile.py
# scripts/ui-assets.cmake
# tools/export-lora/README.md
# tools/server/CMakeLists.txt
2026-06-20 11:18:24 +08:00
f449e05537
ggml-webgpu: add adapter toggles for F16 on Vulkan + NVIDIA
2026-06-20 08:12:32 +09:00
Concedo
Gabe Goodhart
Masashi Yoshimura
Aldehir Rojas
Shawn Gu
Matt Thompson
Mahdiou Diallo
Xuan-Son Nguyen
Pascal
Neo Zhang
Tim Neumann
aafsmarak
henk717
Sigbjørn Skjæret
Georgi Gerganov
YiChen Lv
Guanhuai Zhang
Adrien Gallouët
Muhammad Salem
Matti4
davidrhodus