7fe6fa6fb6
match draft defaults
2026-06-22 21:40:00 +08:00
a0f39fe0f5
allow drafting with vision
2026-06-22 20:50:39 +08:00
e4771e8e6b
restore draft state before main state, fixes reloading gemma4 assistant
2026-06-22 18:55:26 +08:00
3090ae0bf7
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/s390x.Dockerfile
# .dockerignore
# .github/workflows/docker.yml
# .github/workflows/release.yml
# docs/android.md
# ggml/src/ggml-cpu/amx/mmq.cpp
# ggml/src/ggml-hexagon/htp/ssm-conv.c
# tests/peg-parser/test-gbnf-generation.cpp
# tests/test-arg-parser.cpp
# tests/test-chat.cpp
# tests/test-jinja.cpp
# tests/test-json-schema-to-grammar.cpp
# tools/server/README.md
2026-06-22 18:23:59 +08:00
d0f9d2e5ac
server: fix edit_file crash on append at end of file (line_start -1) ( #24893 )
...
line_start -1 normalized to n+1, so append inserted at lines.begin() + n + 1,
one past end() -> heap-buffer-overflow in vector::_M_range_insert.
Normalize -1 to n (insert at end()), restrict -1 to append mode and reject it
for replace/delete instead of silently clobbering the last line. Parenthesize
the insert offset so empty-file append computes the position as int first,
avoiding a transient begin() - 1 on a null vector data pointer.
2026-06-22 10:55:28 +02:00
0ef6f06d55
docs/android.md: Add dependency libandroid-spawn for building in termux ( #21812 )
...
Fixes https://github.com/ggml-org/llama.cpp/issues/18615
2026-06-22 05:48:31 +02:00
52b3df0023
common/peg : implement ac parser for stricter grammar generation ( #24869 )
...
* common/peg : implement ac parser
* cont : extract functions
* cont : tidy up
* cont : remove a test
* cont : move ac() def
2026-06-21 16:20:58 -05:00
dfa1c573c4
fix router mode on lcpp ui
2026-06-22 00:21:19 +08:00
7c082bc417
server: fix report progress for loading spec models, add "stages" list ( #24870 )
...
* server: fix report progress for loading spec models, add "stages" list
* improve
* nits
* nits 2
2026-06-21 17:36:52 +02:00
a072dd8304
llama-ui update ( #2281 )
2026-06-21 23:02:16 +08:00
08fbef5049
lcpp ui think budget
2026-06-21 22:56:10 +08:00
44bcead521
mcp fixes
2026-06-21 22:28:47 +08:00
1afe5a730a
minor fixes to handler newer lcpp ui
2026-06-21 22:21:10 +08:00
bddfd2b113
server: refactor batch construction ( #24843 )
...
* server: refactor batch construction
* wip
* wip 2
* wip 3
* wip 4
* add abort_all_slots
* handle batch full more carefully
* fix assert
* rm debug log
* small nits
* (debug) add timings
* debug: force llama_synchronize for accurate timings
* address comments
* disable DEBUG_TIMINGS
2026-06-21 14:16:11 +02:00
0d135df48c
mtmd: fix mtmd_get_memory_usage ( #24867 )
2026-06-21 14:12:15 +02:00
bf533823cd
jinja : implement call statement ( #24847 )
...
* implement call statement
* undo unintended change
* de-lambda
* simplify
* move caller context inside function handler
2026-06-21 14:04:52 +02:00
2f89acc2bc
mtmd: add load progress callback ( #24865 )
2026-06-21 13:40:52 +02:00
bfa3219177
server: add "verbose" field to schema ( #24864 )
2026-06-21 13:03:14 +02:00
d6d899580d
server: real-time model load progress tracking via /models/sse ( #24828 )
...
* server: real-time model load progress tracking via /models/sse
* update docs
* add mutex for notify_to_router
* correct docs
2026-06-21 11:58:14 +02:00
8a118ee86c
minor : clean-up whitespaces ( #24862 )
...
[no ci]
2026-06-21 11:37:12 +03:00
d789527482
spec : Support Step3.5/3.7 flash mtp3 ( #24340 )
...
* add mtp_layer_offset + include nextn flags in graph reuse
* add llama_set_mtp_layer_offset + llama_model_n_nextn_layer API
* offset head select + require all MTP blocks
* speculative multi-head process()
* speculative multi-head draft()
* gather outputs via inp_out_ids
* cleanup
* fix core
* minor cleanup
* merged draft_multi_head into draft()
* mtp rename nextn
* Apply suggestions from code review
Co-authored-by: Aman Gupta <amangupta052@gmail.com >
* clean-up comments
* fix for multi seq
* apply suggestions && chain-heads comment
* add a reference for chain_heads discussion
---------
Co-authored-by: Aman Gupta <amangupta052@gmail.com >
2026-06-21 11:33:18 +03:00
f202c0a457
mtp init -2
2026-06-21 10:21:48 +08:00
063d9c156e
common/peg : refactor until gbnf grammar generation ( #24839 )
...
* common/peg : refactor until gbnf grammar into an ac automaton
* cont : add a test with multiple strings
* cont : pad state with 0s so rules line up
* cont : clean up comments
* cont : use set everywhere
* cont : inline state num string padding
* cont : add a ref to PR
* cont : fix regression in server-tools.cpp
2026-06-20 21:15:06 -05:00
c57607016a
common/json-schema-to-grammar : align spacing rules with parsers ( #24835 )
2026-06-20 17:43:04 -05:00
4a80943174
fix(hexagon): use padded stride for ssm-conv weights ( #24470 )
2026-06-20 14:58:49 -07:00
84de01a1f1
llama : use LLM_KV for quantization_version & file_type ( #24802 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-06-20 20:07:01 +02:00
75f460ac28
arg: try fixing test-args-parser randomly fails ( #24826 )
...
* arg: try fixing test-args-parser randomly fails
* return ref
* try triggering the workflow
* exception wrapper
* wip
* test
* test 2
* arg: guard win32 utf8 argv override
make_utf8_argv rebuilds argv from GetCommandLineW to fix utf8 handling of
non ascii arguments on windows. the override runs unconditionally inside
common_params_parse, so it also clobbers a programmatic argv passed by a
caller. test-arg-parser builds a synthetic argv but then sees the real
process command line instead, the model argument is never parsed, and the
assert that expects success aborts via fastfail (0xC0000409). this shows up
as a random failure in the openvino windows workflow.
only override argv when its length matches the caller argc, so the utf8
repair still applies to real binaries while a programmatic argv stays intact.
---------
Co-authored-by: Pascal <admin@serveurperso.com >
2026-06-20 19:45:27 +02:00
8452824611
release: add missing link for win opencl adreno arm64 ( #24809 )
2026-06-20 23:08:59 +08:00
a5019767c3
docs
2026-06-20 23:02:56 +08:00
e27f308597
server: avoid forwarding auth headers in CORS proxy ( #24373 )
...
* server: avoid forwarding auth headers in CORS proxy
* format
* fix test
* fix e2e test
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co >
2026-06-20 15:34:47 +02:00
67e9fd3b74
docker : prebuild web UI for s390x build [no release] ( #24829 )
2026-06-20 05:54:42 -05:00
796f41bedc
model : glm-dsa load DSA indexer tensors as optional ( #24770 )
...
GLM-5.2 ships the DSA "lightning indexer" on only a subset of layers (the
"full" layers; others omit it), but the GLM_DSA loader created the five
indexer tensors on every layer as required, so loading any GLM-5.2 GGUF
failed with e.g. `missing tensor 'blk.3.indexer.k_norm.weight'`.
GLM_DSA's graph is llama_model_deepseek2::graph (plain MLA) and does not use
the indexer tensors (indexer runtime not yet implemented), so they are
loaded-but-unused. Marking them TENSOR_NOT_REQUIRED lets layers without an
indexer load as nullptr and the model runs as full MLA attention.
DeepSeek-V3.2 (uniform indexer on all layers) is unaffected.
2026-06-20 13:48:24 +03:00
37a77fb057
ggml : optimize AMX ( #24806 )
...
Flatten the partition over n_batch * M so every thread participates in
the quantization
| CPU | Model | Test | t/s OLD | t/s NEW | Speedup |
|:--------------------------------|:------------------------------|:-------|----------:|----------:|----------:|
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_NL - 4.5 bpw | pp512 | 730.71 | 779.86 | 1.07 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_NL - 4.5 bpw | tg128 | 87.88 | 86.79 | 0.99 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_XS - 4.25 bpw | pp512 | 725.09 | 1023.31 | 1.41 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_XS - 4.25 bpw | tg128 | 83.64 | 83.62 | 1.00 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_0 | pp512 | 820.51 | 924.05 | 1.13 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_0 | tg128 | 90.59 | 92.46 | 1.02 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_1 | pp512 | 776.88 | 872.79 | 1.12 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_1 | tg128 | 89.39 | 90.94 | 1.02 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_M | pp512 | 719.28 | 1009.27 | 1.40 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_M | tg128 | 80.62 | 80.86 | 1.00 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_S | pp512 | 732.29 | 1077.29 | 1.47 |
| Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B Q4_K_S | tg128 | 86.42 | 83.53 | 0.97 |
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-06-20 13:43:06 +03:00
f4043fec01
convert : more consistent handling of rope_parameters ( #24833 )
2026-06-20 13:42:36 +03:00
73cc7d9287
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/openvino.Dockerfile
# .devops/rocm.Dockerfile
# .devops/s390x.Dockerfile
# .devops/vulkan.Dockerfile
# .devops/zendnn.Dockerfile
# .dockerignore
# .pi/gg/SYSTEM.md
# ggml/CMakeLists.txt
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# scripts/sync-ggml.last
# scripts/sync_vendor.py
# tools/cli/README.md
# tools/cli/cli.cpp
# tools/mtmd/clip.cpp
# tools/server/README.md
2026-06-20 17:16:19 +08:00
84b8856295
Merge commit '32eddaf2ea8dd5d499dee9655592a89b91bfde9d' into concedo_experimental
...
# Conflicts:
# docs/multimodal.md
# docs/preset.md
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hex-dma.h
# ggml/src/ggml-hexagon/htp/hex-utils.h
# ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/hmx-queue.c
# ggml/src/ggml-hexagon/htp/hmx-queue.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# scripts/snapdragon/ggml-hexagon-profile.py
# scripts/ui-assets.cmake
# tools/export-lora/README.md
# tools/server/CMakeLists.txt
2026-06-20 11:18:24 +08:00
f449e05537
ggml-webgpu: add adapter toggles for F16 on Vulkan + NVIDIA
2026-06-20 08:12:32 +09:00
2b686a9120
server: refactor child --> router communication ( #24821 )
...
* server: refactor child --> router communication
* fix wakeup case
* add docs
* improve update_status()
* nits
2026-06-20 01:02:26 +02:00
4b48a53b6c
server : optimize get_token_probabilities ( #24796 )
...
Use std::partial_sort to order only the requested top-n tokens instead
of the full vocabulary
logprobs sort: vocab=128000 n_top=0 iters=100
full sort: 8555.6 us/op
partial sort: 704.3 us/op
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-06-19 23:26:54 +02:00
e475fa2b5f
mtmd, arg: fix utf8 handling on windows ( #24779 )
...
* mtmd, arg: fix utf8 handling on windows
* also fix ggml_fopen
* fix build fail
* also fix CLI
2026-06-19 22:28:38 +02:00
175147e8f6
server: remove all internal mentions about "webui" ( #24817 )
2026-06-19 22:12:46 +02:00
2fb3406be7
added ideogram 4 support
2026-06-20 00:29:48 +08:00
fabde3bf51
arg: Add comment line support to --api-key-file ( #23168 )
2026-06-19 17:33:54 +02:00
0d2d9ccbf6
vendor : update cpp-httplib to 0.48.0 ( #24787 )
2026-06-19 22:16:35 +08:00
8c2d6f6475
server: add --agent arg, remove redundant webui naming compat ( #24801 )
...
* server: add --agent arg, remove redundant webui naming compat
* corrent env
* fix the test
* llama-gen-docs
* nits: wordings
2026-06-19 16:06:13 +02:00
9b1e2fa8b8
advanced onready bypass
2026-06-19 21:44:37 +08:00
38724ab593
docker : build the UI ( #24794 )
...
* docker : build the UI
* cont : use existing APP_VERSION
2026-06-19 15:32:31 +02:00
e2e7a9b2d0
mtmd: several bug fixes ( #24784 )
...
* mtmd: several bug fixes
* fix build
* fix gemma4ua
* add sanity check in get_u32()
* fix build (2)
* area() avoid overflow
2026-06-19 12:18:36 +02:00
b14e3fb90c
spec: support eagle3 for qwen3.5 & 3.6 ( #24593 )
...
* spec: support qwen3.5 & 3.6 eagle3 draft
* eagle3: Add deferred boundary checkpoints restore support for hybrid models
* apply suggestions
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* spec: adapt to API change
* spec: fix naming
* cont : add TODO
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-06-19 13:08:50 +03:00
159d093a43
server: fix non-bound n_discard value (ctx shifting) ( #24786 )
...
* server: fix non-bound n_discard value
* Update tools/server/server-context.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-06-19 10:53:44 +02:00
Concedo
Pascal
aafsmarak
Aldehir Rojas
Xuan-Son Nguyen
henk717
Sigbjørn Skjæret
Georgi Gerganov
YiChen Lv
Guanhuai Zhang
Adrien Gallouët
Muhammad Salem
Matti4
davidrhodus
Masashi Yoshimura
Mikolaj Kucharski
Alessandro de Oliveira Faria (A.K.A.CABELO)
Ruixiang Wang