79882d669a
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-android.yml
# .github/workflows/build.yml
# .github/workflows/release.yml
# CMakeLists.txt
# CODEOWNERS
# common/CMakeLists.txt
# common/common.h
# docs/ops.md
# docs/ops/Metal.csv
# examples/batched/CMakeLists.txt
# examples/convert-llama2c-to-ggml/CMakeLists.txt
# examples/debug/CMakeLists.txt
# examples/diffusion/CMakeLists.txt
# examples/embedding/CMakeLists.txt
# examples/eval-callback/CMakeLists.txt
# examples/gen-docs/CMakeLists.txt
# examples/idle/CMakeLists.txt
# examples/lookahead/CMakeLists.txt
# examples/lookup/CMakeLists.txt
# examples/parallel/CMakeLists.txt
# examples/passkey/CMakeLists.txt
# examples/retrieval/CMakeLists.txt
# examples/save-load-state/CMakeLists.txt
# examples/speculative-simple/CMakeLists.txt
# examples/speculative/CMakeLists.txt
# examples/sycl/CMakeLists.txt
# examples/training/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# pocs/vdot/CMakeLists.txt
# src/CMakeLists.txt
# tests/CMakeLists.txt
# tests/test-quantize-stats.cpp
# tools/batched-bench/CMakeLists.txt
# tools/cli/CMakeLists.txt
# tools/cli/cli.cpp
# tools/completion/CMakeLists.txt
# tools/cvector-generator/CMakeLists.txt
# tools/cvector-generator/cvector-generator.cpp
# tools/export-lora/CMakeLists.txt
# tools/gguf-split/CMakeLists.txt
# tools/gguf-split/gguf-split.cpp
# tools/imatrix/CMakeLists.txt
# tools/llama-bench/CMakeLists.txt
# tools/llama-bench/llama-bench.cpp
# tools/mtmd/CMakeLists.txt
# tools/perplexity/CMakeLists.txt
# tools/quantize/CMakeLists.txt
# tools/quantize/quantize.cpp
# tools/results/CMakeLists.txt
# tools/server/CMakeLists.txt
# tools/tokenize/CMakeLists.txt
# tools/tts/CMakeLists.txt
2026-04-17 22:37:37 +08:00
768527b031
Merge commit '1e796eb41fb51950ada45811a303e57a5f4ea974' into concedo_experimental
...
# Conflicts:
# .devops/nix/package.nix
# .github/workflows/build-riscv.yml
# .github/workflows/build-vulkan.yml
# .github/workflows/build.yml
# docs/backend/SYCL.md
# docs/build.md
# docs/development/HOWTO-add-model.md
# embd_res/templates/Reka-Edge.jinja
# ggml/CMakeLists.txt
# ggml/src/ggml-rpc/CMakeLists.txt
# ggml/src/ggml-rpc/ggml-rpc.cpp
# ggml/src/ggml-sycl/CMakeLists.txt
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/dequantize.hpp
# ggml/src/ggml-sycl/dmmv.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/get_rows.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_id.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_reg_tile.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_subgroup_matrix.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/unary.wgsl
# tests/test-chat.cpp
# tools/rpc/README.md
2026-04-17 21:47:29 +08:00
6990e2f1f7
libs : rename libcommon -> libllama-common ( #21936 )
...
* cmake : allow libcommon to be shared
* cmake : rename libcommon to libllama-common
* cont : set -fPIC for httplib
* cont : export all symbols
* cont : fix build_info exports
* libs : add libllama-common-base
* log : add common_log_get_verbosity_thold()
2026-04-17 11:11:46 +03:00
e1a9a6dcbe
autoparser: support case of JSON_NATIVE with per-call markers (test case: Reka-Edge) ( #21892 )
2026-04-15 10:51:50 +02:00
236ae27329
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/close-issue.yml
# docs/multimodal.md
# embd_res/templates/deepseek-ai-DeepSeek-V3.2.jinja
# ggml/CMakeLists.txt
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_reg_tile.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_subgroup_matrix.wgsl
# tests/peg-parser/test-gbnf-generation.cpp
# tests/test-chat.cpp
2026-04-14 21:01:41 +08:00
9c0b9b0bb1
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# docs/development/HOWTO-add-model.md
# docs/multimodal.md
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/dequantize.hpp
# ggml/src/ggml-sycl/element_wise.cpp
# ggml/src/ggml-sycl/gated_delta_net.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/upscale.cpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
# tools/mtmd/CMakeLists.txt
2026-04-14 20:06:04 +08:00
56666fa607
common: skip reasoning budget sampler when no budget is requested ( #21870 )
...
* common: skip reasoning budget sampler when no budget is requested
After I added thinking_start_tag / thinking_end_tag for gemma4 in #21697 , the reasoning budget sampler gets unconditionally created even when no budget is configured (the default -1). The same applies to kimi_k2, lfm2, lfm2_5, and ministral_3 which also set these tags. The budget gets converted to INT_MAX, so the sampler never actually forces any tokens but still runs per-token checks (start tag matching in IDLE state, token-to-piece conversion + UTF-8 checks in COUNTING state).
More importantly, the mere existence of the sampler (non-null rbudget) disables backend sampling. Backend sampling lets the GPU select tokens directly, avoiding a full logits transfer from GPU to CPU every token. This could explain the 30% speed regression reported in #21784 (98 t/s to 70 t/s on Vulkan).
So I added a reasoning_budget_tokens >= 0 check to the sampler creation condition. When the budget is unlimited, the sampler is not created, backend sampling stays enabled, and no per-token overhead is added. When a budget is explicitly set (0, 128, 1024, etc.), the sampler is created and works as before.
* common: preserve rbudget when grammar is lazy
Following up on the review feedback on #21870 : keep the reasoning budget sampler when grammar_lazy is true, so the thinking-block grammar suppression from #20970 still works when tools are in use. This way, we only skip the sampler when both no budget is set AND grammar is not lazy.
2026-04-14 12:43:06 +02:00
e21cdc11a0
common/gemma4 : handle parsing edge cases ( #21760 )
2026-04-13 18:18:18 -05:00
1c0d9081fd
chat: dedicated DeepSeek v3.2 parser + "official" template ( #21785 )
2026-04-13 22:23:53 +02:00
aa00911d12
common : add download cancellation and temp file cleanup ( #21813 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-04-13 11:18:23 +02:00
63ce8ae64e
fix tool builds
2026-04-12 02:36:30 +08:00
4c860ae4ae
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# common/download.cpp
# docs/backend/OPENVINO.md
# docs/backend/snapdragon/CMakeUserPresets.json
# docs/backend/snapdragon/README.md
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/act-ops.c
# ggml/src/ggml-hexagon/htp/argsort-ops.c
# ggml/src/ggml-hexagon/htp/binary-ops.c
# ggml/src/ggml-hexagon/htp/cpy-ops.c
# ggml/src/ggml-hexagon/htp/cumsum-ops.c
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/get-rows-ops.c
# ggml/src/ggml-hexagon/htp/hex-utils.h
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/hmx-ops.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/htp_iface.idl
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/repeat-ops.c
# ggml/src/ggml-hexagon/htp/rope-ops.c
# ggml/src/ggml-hexagon/htp/set-rows-ops.c
# ggml/src/ggml-hexagon/htp/softmax-ops.c
# ggml/src/ggml-hexagon/htp/ssm-conv.c
# ggml/src/ggml-hexagon/htp/sum-rows-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/flash_attn.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/get_rows.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/unary.wgsl
# models/templates/google-gemma-4-31B-it-interleaved.jinja
# models/templates/google-gemma-4-31B-it.jinja
# scripts/snapdragon/adb/run-bench.sh
# scripts/snapdragon/adb/run-cli.sh
# scripts/snapdragon/adb/run-completion.sh
# scripts/snapdragon/adb/run-tool.sh
# scripts/snapdragon/windows/run-bench.ps1
# scripts/snapdragon/windows/run-cli.ps1
# scripts/snapdragon/windows/run-mtmd.ps1
# scripts/snapdragon/windows/run-tool.ps1
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tools/llama-bench/llama-bench.cpp
2026-04-11 11:19:32 +08:00
a165a73120
Merge commit 'd6f3030047f85a98b009189e76f441fe818ea44d' into concedo_experimental
...
# Conflicts:
# examples/model-conversion/scripts/causal/run-casual-gen-embeddings-org.py
# examples/model-conversion/scripts/utils/semantic_check.py
# ggml/CMakeLists.txt
# ggml/src/CMakeLists.txt
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-cpu/amx/amx.cpp
# ggml/src/ggml-cuda/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hip/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-openvino/ggml-openvino.cpp
# ggml/src/ggml-rpc/ggml-rpc.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-virtgpu/ggml-backend-buffer.cpp
# ggml/src/ggml-virtgpu/ggml-backend.cpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-zdnn/ggml-zdnn.cpp
# ggml/src/ggml-zendnn/ggml-zendnn.cpp
# pyproject.toml
# requirements/requirements-convert_legacy_llama.txt
# requirements/requirements-tool_bench.txt
# src/llama-model.cpp
# src/llama.cpp
# tests/test-llama-archs.cpp
# tests/test-tokenizer-0.py
# tests/test-tokenizer-random.py
# tools/llama-bench/llama-bench.cpp
# tools/perplexity/perplexity.cpp
2026-04-11 11:10:55 +08:00
8b90bfe094
Merge commit '4ef9301e4de5ea8f5ebf20b107cf30de0070bf2a' into concedo_experimental
...
# Conflicts:
# .github/labeler.yml
# docs/multimodal.md
# embd_res/ggml-vocab-gemma-4.gguf
# embd_res/ggml-vocab-gemma-4.gguf.inp
# embd_res/ggml-vocab-gemma-4.gguf.out
# ggml/src/ggml-sycl/fattn-tile.cpp
# ggml/src/ggml-sycl/fattn-tile.hpp
# ggml/src/ggml-sycl/fattn-vec.hpp
# ggml/src/ggml-sycl/fattn.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-f16-f16.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-f16-q4_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-f16-q4_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-f16-q5_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-f16-q5_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-f16-q8_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_0-f16.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_0-q4_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_0-q4_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_0-q5_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_0-q5_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_0-q8_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_1-f16.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_1-q4_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_1-q4_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_1-q5_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_1-q5_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q4_1-q8_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_0-f16.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_0-q4_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_0-q4_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_0-q5_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_0-q5_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_0-q8_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_1-f16.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_1-q4_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_1-q4_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_1-q5_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_1-q5_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_1-q8_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q8_0-f16.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q8_0-q4_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q8_0-q4_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q8_0-q5_0.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q8_0-q5_1.cpp
# ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q8_0-q8_0.cpp
# tests/CMakeLists.txt
# tests/test-jinja.cpp
# tools/mtmd/CMakeLists.txt
2026-04-11 09:38:50 +08:00
b136b62cf9
fix: Fix broken structured output when using $refs in json_schema ( #21699 )
2026-04-10 18:26:36 -05:00
3fc65063d9
common : better align to the updated official gemma4 template ( #21704 )
2026-04-10 16:12:53 -05:00
05b3caaa48
common : add callback interface for download progress ( #21735 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-04-10 22:17:00 +02:00
fb38d6f278
common : fix when loading a cached HF models with unavailable API ( #21670 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-04-10 16:37:46 +02:00
0893f50f2d
common: mark --split-mode tensor as experimental ( #21684 )
2026-04-10 12:27:27 +02:00
d7ff074c87
common : enable reasoning budget sampler for gemma4 ( #21697 )
...
* fix: enable reasoning budget sampler for gemma4
Add thinking_start_tag and thinking_end_tag to
common_chat_params_init_gemma4(). Without these, the reasoning
budget sampler never activates for gemma4.
Make the newline after "thought" optional in the PEG parser to
handle budget=0 (sampler forces end tag before the newline).
Add test case for empty thinking block.
Fixes #21487
* use p.space() instead of p.optional(p.literal("\n")) in gemma4 thought parser
2026-04-10 11:49:14 +02:00
e095a482a0
common : add fluidity to the progress bar ( #21671 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-04-10 08:24:53 +02:00
d6f3030047
ggml: backend-agnostic tensor parallelism (experimental) ( #19378 )
...
* ggml: backend-agnostic tensor parallelism
* support for GPT-OSS, Qwen 3 MoE
* partial Vulkan fix
* add support for 4/8 GPUs
* unconditional peer access
* re-use buffers + ggml contexts
* fix output pattern
* NCCL support
* GGML: HIP: add RCCL support
* Remove shfl and AllReduce from backend interface
* move allocation workaround out of ggml-alloc.c
* 2d tensor set/get support
* Fix the seg fault without NCCL
* Apply suggestion from JohannesGaessler
* support for tensor dims % n_devs != 0
* fix view_offs scaling
* arbitrary num. of GPUs/tensor split
* fix compilation
* better granularity estimate
* Support device-specific host buffer types if all underlying backends expose the same type. This allows using pinned memory instead of pageable memory for CUDA.
Fix compilation errors.
* partial Qwen 3 Next support
* Fix qwen3 30b (#8 )
* Fix crash with Qwen-30B-A3B Q4_0
Qwen-30B-A3B Q4_0 has an intermediate dimension of 768. Using a granularity of 256 forces an uneven split between GPUs, which is not supported by the current implementation.
* Decide block size based on tensor quantization type
* Fix crashes due to KV cache serialization (#9 )
KV cache serialization requires non-zero offsets on the tensor. Add support in the meta backend to set/get a tensor with a non-zero offset.
* metal : fix build (#7 )
* static memory allocations, fix usage count
* fix tensor granularity
* more even memory distribution
* use BF16 for allreduce
* rebase fixup
* better error message for unsupported architectures
* Fix device mismatch during scatter of allReduce. (#11 )
There is a mismatch between the dst buffer device and the backend device, causing the use of sync copies
* Enable the previous allreduce implementation. It is better in both perf and stability (#12 )
* delay AllReduce for Moe for less I/O
* build : clean-up compile warnings
* backend : move most of the meta backend API to ggml-backend-impl.h
* cont : hide unused public API in the implementation
* llama : use llama_device + remove ggml_backend_dev_is_meta()
* ggml-backend : remove unused alloc include
* minor : remove regex include
* ggml : introduce ggml-ext.h for staging new APIs
* rebase fixup
* fix tests
* llama : more robust logic for determining Meta devices (#16 )
* llama : more robust logic for determining Meta devices
* cont : fix devs size check
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
* cont : fix log type
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
---------
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
* disable roundtrip for meta backend
* fix arch selection
* Qwen 3.5 support
* fix Gemma 4 MoE
* fix OpenVino, SYCL
* fix test-llama-archs for CPU-only builds
* Fix Qwen 3.5 MoE
* disable meta backend tests for WebGPU
* tests : filter CPU-based devices from the Meta backend tests (#17 )
* meta : formatting, naming, indentation (#18 )
* formatting : llama-model.cpp
* formatting : ggml-ext.h
* formatting : ggml-backend-meta.cpp
* meta : add TODO
* add documentation
* better error messages
* fix GPT-OSS
---------
Co-authored-by: Carl Philipp Klemm <carl@uvos.xyz >
Co-authored-by: Gaurav Garg <gaugarg@nvidia.com >
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-04-09 16:42:19 +02:00
ddf03c6d9a
common : fix ambiguous grammar rule in gemma4 ( #21661 )
...
* common : fix ambiguous grammar rule in gemma4
* cont : fix missing comma...
2026-04-09 12:25:07 +02:00
26229755c5
common : simplify autoparser tagged parser rules ( #21216 )
...
* common : simplify autoparser tagged parser rules
* cont : remove upper limit on optional args
* cont : revert changes to parsing at the end
* cont : undo arbitrary ordering of optional args
* cont : fix uninitialized required parameters
* revert to simplify merge
* re-apply patches
* restore flexible optional arg ordering tests
2026-04-09 12:24:20 +02:00
c82c0b463a
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/labeler.yml
# .github/workflows/release.yml
# examples/debug/debug.cpp
# ggml/src/ggml-cuda/common.cuh
# ggml/src/ggml-cuda/mmq.cuh
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# src/llama-vocab.cpp
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tests/test-json-schema-to-grammar.cpp
# tools/mtmd/CMakeLists.txt
2026-04-09 17:45:04 +08:00
243532e556
jinja : support ensure_ascii=true, string repetition and int/float self-filtering ( #21623 )
...
* feat: jinja engine improvements for reka-edge
Port three Jinja engine improvements needed for the reka-edge model:
1. Python-style string repetition ("ab" * 3 → "ababab")
2. ensure_ascii=true support for tojson filter (escapes non-ASCII to \uXXXX)
3. int() builtin on value_int_t (identity, needed for Reka Edge template)
* fix: escape invalid utf8 bytes when ensure_ascii=true
The json_ensure_ascii_preserving_format function does not correctly
handle an edge case where if UTF-8 parsing fails, it adds the non-ascii
character back to the output as a raw byte.
This commit fixes that by adding the unicode standard replacement
character \\ufffd to the output instead. This is the standard behavior
for various programming languages like Python, Rust, Go, etc.
* chore: address PR comments
1. Add todo comment for supporting string repetition for array/tuples
2. Add support for float identity operation
3. Move invalid ascii test case to test_fuzzing
* chore: accept suggestion for common/jinja/value.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-04-09 11:28:33 +02:00
4293919068
common : skip non-primary GGUF split files when selecting model ( #21633 )
...
We should not assume files are listed in order.
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-04-09 07:28:06 +02:00
85d482e6b6
parser: fix MiniMax handling ( #21573 )
2026-04-08 12:47:25 +02:00
15d269197e
Merge commit '506200cf8b5c8419ce97d16dc8c50f4634e21ebe' into concedo_experimental
...
# Conflicts:
# docs/multimodal.md
# scripts/compare-llama-bench.py
# src/llama-vocab.cpp
# tools/llama-bench/README.md
# tools/llama-bench/llama-bench.cpp
2026-04-07 14:58:36 +08:00
506200cf8b
cli: fix stripping of \n in multiline input ( #21485 )
...
* llama-cli: fix stripping of \n in multiline input
* Change & string to string_view
* Apply suggestions from code review
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Fix EditorConfig linter error
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-04-06 20:54:06 +02:00
9b1f1bbf35
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-vulkan.yml
# .github/workflows/docker.yml
# embd_res/templates/google-gemma-4-31B-it-interleaved.jinja
# embd_res/templates/google-gemma-4-31B-it.jinja
# tests/test-chat.cpp
2026-04-05 18:46:23 +08:00
b8635075ff
common : add gemma 4 specialized parser ( #21418 )
...
* common : add gemma4 dedicated parser
* cont : add '<|tool_response>' as eog
* cont : emit JSON from Gemma4 tool call AST
* cont : more fixes
* cont : refactor convert function
* cont : refine rules and mapping
* cont : add more tests
* cont : clean up
* cont : remove autoparser gemma4 implementation
* cont : more cleanup
* cont : rename gemma4.jinja to match the others
* cont : add custom template to support interleaved thinking
* cont : preserve reasoning in model turns
* cont : fix initializer error
* cont : fix unused vars
* cont : fix accidental static
* cont : fix specialized_template signature
* fix extra semicolon
* remove debug line and extra space [no ci]
2026-04-04 20:39:00 +02:00
376aaf258c
Merge branch 'upstream' into concedo_experimental
2026-04-04 23:56:02 +08:00
d01f6274c0
common : respect specified tag, only fallback when tag is empty ( #21413 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-04-04 15:08:03 +02:00
2e4f94822e
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-self-hosted.yml
# .github/workflows/docker.yml
# ci/run.sh
# docs/build.md
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# src/llama-vocab.cpp
# tests/test-chat.cpp
# tests/test-jinja.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/server/README.md
2026-04-04 14:27:23 +08:00
50e0ad08fb
server: save and clear idle slots on new task (--clear-idle) ( #20993 )
...
* server: clear idle slots KV from VRAM (LLAMA_KV_KEEP_ONLY_ACTIVE)
* server: move idle slot KV clearing to slot release
The save "cost" is now paid by the finishing request.
* server: add --kv-clear-idle flag, enable by default
* server: skip clearing last idle slot, clear on launch
* server: test --no-kv-clear-idle flag
* server: simplify on-release clearing loop
* server: remove on-release KV clearing, keep launch-only
* cont : clean-up
* tests: update log strings after --clear-idle rename
* tests: use debug tags instead of log message matching
* test: fix Windows CI by dropping temp log file unlink
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-04-03 19:02:27 +02:00
f1f793ad06
common/parser: fix call ID detection (Mistral parser mostly) + atomicity for tag-json parsers ( #21230 )
...
* Fix call ID detection (Mistral parser mostly) + atomicity for tag-json parsers
* Rename
* Update common/chat-auto-parser-generator.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-04-03 17:51:52 +02:00
af5c13841f
common : fix tool call type detection for nullable and enum schemas ( #21327 )
...
* common : fix tool call type detection for nullable and enum schemas
* common, tests : fix grammar delegation for nullable/enum schemas and add tests
Fix enum type inference to scan all enum values (not just index 0) so
schemas like {"enum": [0, "celsius"]} correctly detect string type.
Fix schema_delegates in peg-parser to handle nullable type arrays
(["string", "null"]) and typeless enum schemas in raw mode, allowing
the tagged parser to use raw text instead of JSON-formatted strings.
Add test cases for Qwen3-Coder (TAG_WITH_TAGGED format):
- nullable string ["string", "null"]
- nullable string with null first ["null", "string"]
- nullable integer ["integer", "null"]
- enum without explicit type key
2026-04-03 17:51:23 +02:00
1f34806c44
jinja: coerce input for string-specific filters ( #21370 )
2026-04-03 15:03:33 +02:00
8fa87621d1
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/labeler.yml
# common/chat.cpp
# ggml/src/ggml-rpc/ggml-rpc.cpp
2026-04-03 16:36:41 +08:00
57ace0d612
chat : avoid including json in chat.h ( #21306 )
2026-04-03 09:07:59 +03:00
34ad53e950
merged support for gemma4. the e2b, e4b and 26b work, the 31b does not
2026-04-03 11:07:46 +08:00
5208e2d5ba
fix: gemma 4 template ( #21326 )
2026-04-02 23:31:02 +02:00
5dee1a1cbb
Merge commit 'fbd441c37933550c1e3365dc84dd73232334c15d' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# AGENTS.md
# ci/run.sh
# docs/build.md
# embd_res/templates/LFM2.5-Instruct.jinja
# ggml/CMakeLists.txt
# ggml/src/ggml-cuda/fattn.cu
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/htp-msg.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-div.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/dequantize.hpp
# ggml/src/ggml-sycl/mmvq.cpp
# ggml/src/ggml-sycl/vecdotq.hpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/flash_attn.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec.wgsl
# scripts/hip/gcn-cdna-vgpr-check.py
# scripts/sync-ggml.last
# tests/test-chat.cpp
2026-04-03 01:06:02 +08:00
5803c8d115
tests: allow exporting graph ops from HF file without downloading weights ( #21182 )
...
* tests: allow exporting graph ops from HF file without downloading weights
* use unique_ptr for llama_context in HF metadata case
* fix missing non-required tensors falling back to type f32
* use unique pointers where possible
* use no_alloc instead of fixing f32 fallback
* fix missing space
2026-04-02 18:19:20 +02:00
63f8fe0ef4
model, mtmd: fix gguf conversion for audio/vision mmproj ( #21309 )
...
* fix gguf conversion for audio/vision mmproj
* fix test
2026-04-02 17:10:32 +02:00
223373742b
common : add commentary rules for gpt-oss-20b ( #21286 )
2026-04-02 08:59:59 -05:00
e15efe007d
Relax prefill parser to allow space. ( #21240 )
...
* Relax prefill parser to allow space.
* Move changes from prefix() to parser generation
* Only allow spaces if we're not having a pure content parser next
2026-04-02 11:29:11 +02:00
1d6d4cf7a5
fix: tool call parsing for LFM2 and LFM2.5 models ( #21242 )
...
* fix: tool call parsing for LFM2 and LFM2.5 models'
* refactor: add test / break out lfm2 and lfm2.5 parsing logic
2026-04-01 16:22:44 +02:00
31aa072da1
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# .github/workflows/release.yml
# .gitignore
# examples/batched/batched.cpp
# examples/debug/debug.cpp
# examples/eval-callback/eval-callback.cpp
# examples/idle/idle.cpp
# examples/lookahead/lookahead.cpp
# examples/lookup/lookup-create.cpp
# examples/lookup/lookup-stats.cpp
# examples/lookup/lookup.cpp
# examples/parallel/parallel.cpp
# examples/passkey/passkey.cpp
# examples/retrieval/retrieval.cpp
# examples/save-load-state/save-load-state.cpp
# examples/speculative-simple/speculative-simple.cpp
# examples/speculative/speculative.cpp
# examples/training/finetune.cpp
# ggml/CMakeLists.txt
# ggml/src/ggml-cann/aclnn_ops.cpp
# ggml/src/ggml-cann/common.h
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-sycl/fattn-tile.hpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/cpy.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/embed_wgsl.py
# ggml/src/ggml-webgpu/wgsl-shaders/rope.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/soft_max.wgsl
# scripts/sync-ggml.last
# tests/export-graph-ops.cpp
# tests/test-chat.cpp
# tests/test-state-restore-fragmented.cpp
# tests/test-thread-safety.cpp
# tools/batched-bench/batched-bench.cpp
# tools/cli/cli.cpp
# tools/cvector-generator/cvector-generator.cpp
# tools/export-lora/export-lora.cpp
# tools/imatrix/imatrix.cpp
# tools/perplexity/perplexity.cpp
# tools/results/results.cpp
# tools/server/CMakeLists.txt
2026-04-01 10:54:13 +08:00
Concedo
Georgi Gerganov
Piotr Wilkin (ilintar)
Berk Idem
Aldehir Rojas
Adrien Gallouët
Galunid
Johannes Gäßler
Kwa Jie Hao
Bipin Yadav
Yes You Can Have Your Own
Samanvya Tripathi
Sigbjørn Skjæret
Ruben Ortlam
Xuan-Son Nguyen
Jonathan