49dbdaaab5
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# AGENTS.md
# CODEOWNERS
# CONTRIBUTING.md
# docs/backend/OPENCL.md
# docs/development/HOWTO-add-model.md
# examples/training/finetune.cpp
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp-drv.cpp
# ggml/src/ggml-hexagon/htp/act-ops.c
# ggml/src/ggml-hexagon/htp/dma-queue.c
# ggml/src/ggml-hexagon/htp/dma-queue.h
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/flash-attn-ops.h
# ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/htp-tensor.c
# ggml/src/ggml-hexagon/htp/htp-tensor.h
# ggml/src/ggml-hexagon/htp/hvx-fa-kernels.h
# ggml/src/ggml-hexagon/htp/hvx-reduce.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/matmul-ops.h
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.h
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# scripts/compare-llama-bench.py
# scripts/snapdragon/ggml-hexagon-profile.py
# scripts/snapdragon/ggml-hexagon-trace.py
# scripts/sync_vendor.py
# tests/test-arg-parser.cpp
# tests/test-chat.cpp
# tests/test-model-load-cancel.cpp
# tests/test-quantize-stats.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/llama-bench/llama-bench.cpp
# tools/server/README.md
# tools/ui/src/lib/constants/settings-registry.ts
2026-07-25 12:20:51 +08:00
eea06b0a89
Merge commit 'f534da26e4ab045b6899adc07cd2b9a065355ce9' into concedo_experimental
...
# Conflicts:
# .github/workflows/release.yml
# docs/ops.md
# docs/ops/WebGPU.csv
# ggml/src/CMakeLists.txt
# ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-openvino/ggml-openvino.cpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# models/templates/deepseek-ai-DeepSeek-V4.jinja
# tests/test-backend-ops.cpp
# tests/test-chat-auto-parser.cpp
# tests/test-chat.cpp
# tests/test-llama-archs.cpp
2026-07-25 12:00:23 +08:00
8f5ab832ca
cohere2 moe template parser: enforce JSON schema for text responses if a response schema is provided ( #26018 )
2026-07-24 12:54:47 +02:00
e6dd0e29a6
args: refactor mlock/mmap/directio into load-mode ( #20834 )
...
* args: overhaul mmap/mlock/dio into single arg
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* docs: update docs with llama-gen-docs
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* chore: satisfy code quality
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* args: make the `+` sign an actual modifier now
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* chore: general code clean up + comments
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* arg: fix deprecated flags support
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* arg: quick sanity check
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* bench: sync llama-bench argument parsing
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* fix: bugfix variable behaviour + llama-bench lm column size
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* arg: inverse commands should do the opposite instead of doing nothing
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* bench: fix incorrect dash
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* bench: fix missing modifiers for deprecated flags
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* llama: switch back to thread_local
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* arg: switch back to single enum
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* docs: update arg docs
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* chore: fix missing `mlock` from llama_load_mode_from_str + cleanup llama-bench
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* llama: fix mlock not activating
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* arg: add deprecation warning when old and new flags are combined
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* arg: cont add comment for todo in the future
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* docs: sync with upstream
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
* docs: re-sync with upstream again
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
---------
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
2026-07-23 20:32:56 +08:00
e8e6c7af24
minor: fix reasoning preserve var for DS4 [no ci] ( #25999 )
2026-07-22 14:32:54 +02:00
6d5a910c50
common: infer the speculative type from the draft repo sidecars ( #25989 )
...
With -hfd pointing to a repo that ships mtp-/dflash-/eagle3- sidecars
and no --spec-type given, the draft resolved to a full model while the
sidecar was the intended draft.
When the speculative types are still at their default, discover the
sidecars of the draft repo, pick the first available following the
existing mtp > dflash > eagle3 priority, and set the corresponding
type, so this now works without any extra flag:
llama-server -hf repo:Q3_K_M -hfd repo:Q8_0
An explicit --spec-type disables the inference, and a draft repo
without sidecars keeps resolving to a full model as before.
2026-07-22 13:06:35 +02:00
f534da26e4
Fix DeepSeek4 crafted template ( #25414 )
...
* chat: fix DS4 template to explicitly follow reference behavior
* Support DeepSeekv4 flag (`drop_reasoning`).
* fix: hook DS3.2 parser for DS4 as well
* fix: add tool result reordering
* fix: post-merge
2026-07-22 12:54:40 +02:00
1f66c3ce1c
Add support for Laguna XS.2 & M.1 ( #25165 )
2026-07-22 09:54:08 +08:00
60f6a17704
common: resolve draft repo to its requested sidecar ( #25955 )
...
With -hfd pointing to a repo shipping speculative sidecars, the draft
resolved to the main model of that repo, since find_best_model()
excludes sidecar files, and the explicit draft plan suppressed the
sidecar discovery on the -hf repo.
The draft plan already discovers its sidecars, they were just never
consumed. Wire them as the draft, following the fallback pattern of
the main plan, so this now works as expected:
llama-server -hf repo -hfd repo --spec-type draft-dflash
2026-07-21 18:03:43 +02:00
0a0b88a5c0
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# docs/backend/OPENCL.md
# ggml/CMakeLists.txt
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-cpu/kleidiai/kernels.cpp
# ggml/src/ggml-cpu/kleidiai/kernels.h
# ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp-drv.cpp
# ggml/src/ggml-hexagon/htp-drv.h
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/act-ops.c
# ggml/src/ggml-hexagon/htp/argsort-ops.c
# ggml/src/ggml-hexagon/htp/binary-ops.c
# ggml/src/ggml-hexagon/htp/cumsum-ops.c
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hex-dma.h
# ggml/src/ggml-hexagon/htp/hex-profile.h
# ggml/src/ggml-hexagon/htp/hex-utils.h
# ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/hmx-queue.c
# ggml/src/ggml-hexagon/htp/hmx-queue.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/matmul-ops.h
# ggml/src/ggml-hexagon/htp/rope-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/gemm_moe_mxfp4_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_q4_0_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_q4_k_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_q6_k_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/gemm_noshuffle_q4_k_f32.cl
# ggml/src/ggml-opencl/kernels/gemm_noshuffle_q4_k_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_k_f32.cl
# ggml/src/ggml-opencl/kernels/mul_mv_q4_k_f32_flat.cl
# ggml/src/ggml-opencl/kernels/mul_mv_q5_k_f32_flat.cl
# ggml/src/ggml-sycl/dmmv.cpp
# scripts/snapdragon/ggml-hexagon-profile.py
# scripts/sync-ggml.last
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
# tests/test-recurrent-state-rollback.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/server/README.md
2026-07-18 12:26:15 +08:00
4c25a3d829
Merge commit '79bba02a6741de194912d370015866414faa83ad' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-self-hosted.yml
# .github/workflows/release.yml
# .github/workflows/server-self-hosted.yml
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/flash_attn_f32_f16.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32_q4_0.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32_q8_0.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_mxfp4_f32_ns.cl
# ggml/src/ggml-opencl/kernels/mul_mv_f16_f32_l4.cl
# ggml/src/ggml-opencl/kernels/mul_mv_q4_k_f32.cl
# tests/test-backend-ops.cpp
# tools/tokenize/tokenize.cpp
# tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionAddToolsSubmenu.svelte
2026-07-18 11:04:46 +08:00
635cdd5fcc
common : auto-download dflash- and eagle3- HF sidecars ( #25811 )
...
* common: auto-download dflash- and eagle3- HF sidecars
Mirror the existing mtp- sidecar logic to support auto-discovery and
download of DFlash (dflash-) and Eagle3 (eagle3-) speculative decoding
sidecars from Hugging Face repos.
Changes:
- Add --dflash and --eagle3 CLI flags to trigger sidecar download
- Add find_best_dflash() and find_best_eagle3() using find_best_sibling
- Exclude dflash- and eagle3- filenames from primary model selection
- Filter dflash- and eagle3- from cached model listings
- Wire download tasks that set speculative.draft.mparams as fallback
Assisted-by: pi:llama.cpp/Qwen3.6-27B
* docs : regen
2026-07-17 12:15:30 +03:00
aff6eb6e75
tokenize : drop --stdin mutual-exclusion check ( #25672 )
...
match cli and completion, which don't enforce it
2026-07-15 18:41:51 +02:00
956973c764
Fix crash with draft-simple ( #25720 )
...
* Fix crash with draft-simple
* Fix tests for spec decoding
2026-07-15 19:51:34 +05:30
9001369da0
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-cpu/kleidiai/kernels.cpp
# ggml/src/ggml-cpu/kleidiai/kernels.h
# ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
# ggml/src/ggml-cpu/ops.cpp
# ggml/src/ggml-cuda/mmq.cuh
# ggml/src/ggml-hexagon/htp/hmx-queue.c
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_iq4_nl_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q1_0_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_0_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_0_f32_spec.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_1_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_k_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q5_0_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q5_1_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q5_k_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q6_k_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q8_0_f32.cl
# ggml/src/ggml-opencl/kernels/mul_mv_f16_f16.cl
# ggml/src/ggml-opencl/kernels/mul_mv_f16_f32.cl
# ggml/src/ggml-opencl/kernels/mul_mv_f16_f32_1row.cl
# ggml/src/ggml-sycl/fattn-vec.hpp
# tests/test-backend-ops.cpp
# tests/test-chat-auto-parser.cpp
# tests/test-export-graph-ops.cpp
# tests/test-jinja.cpp
# tests/test-llama-archs.cpp
# tools/tokenize/tokenize.cpp
2026-07-15 15:39:31 +08:00
6e52db5b72
server: add --cors-* options ( #25655 )
...
* server: add --cors-* options
* add special "localhost" value
* add tests
* fix test
* add link to PR
2026-07-14 17:23:44 +02:00
8ff8c4299d
tokenize : align usage by using common args ( #25516 )
...
Migrate the tokenize tool to common_params_parse, replacing its
hand-rolled argv parsing, Windows UTF-8 handling and file reading
with the shared common helpers.
Expose the model-sourcing flags (-m, -mu, -dr, -hf, -hff, --offline,
HF_TOKEN) to LLAMA_EXAMPLE_TOKENIZE, and register --ids, --stdin,
--no-bos, --no-parse-special and --show-count as common args.
parse_special defaults to true for TOKENIZE to preserve the old
behavior. Errors now go through LOG_ERR instead of fprintf(stderr).
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-07-14 15:20:53 +02:00
ec0dbef816
arg: Flush log before exiting after usage() ( #25504 )
...
Under certain conditions, it's possible for messages emitted via LOG()
to get lost before exit, apparently because they are emitted by another
thread. common_params_print_usage() uses printf directly, and is not
affected.
Flushing the log before exit seems to resolve this.
2026-07-14 12:03:22 +03:00
2969d6d15d
model: add Hy3 (hy_v3) support with MTP speculative decoding ( #25395 )
...
* model: add Hy3 (hy_v3) architecture support
Adds Tencent Hunyuan 3 (HF architecture HYV3ForCausalLM, GGUF arch
hy_v3): a MoE decoder stack with per-head Q/K RMSNorm, a sigmoid
router with expert selection bias, an always-active ungated shared
expert, and leading dense block(s) (first_k_dense_replace).
The base implementation is ported from charlie12345's fork
(https://github.com/charlie12345/ROCmFPX , src/models/hyv3.cpp),
adapted to current mainline APIs (hparams.n_layer(), build_qkv,
build_moe_ffn with fused gate_up + scale tensors, output_s).
Note: blk.N.exp_probs_b is stored without a .bias suffix for
compatibility with existing hy_v3 GGUFs produced by that fork.
Co-Authored-By: charlie12345 <charlie12345@users.noreply.github.com >
Co-authored-by: Piotr Wilkin <ilintar@gmail.com >
Assisted-by: Claude Fable 5
2026-07-14 00:31:04 +02:00
fa21872f97
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# docs/backend/SYCL.md
# ggml/src/ggml-cpu/ggml-cpu.c
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# ggml/src/ggml-opencl/kernels/gemv_moe_mxfp4_f32_ns.cl
# ggml/src/ggml-opencl/kernels/gemv_moe_q4_k_f32_ns.cl
# ggml/src/ggml-sycl/backend.hpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/dmmv.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# scripts/sync_vendor.py
# tests/CMakeLists.txt
# tests/test-alloc.cpp
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tests/test-gguf.cpp
# tests/test-save-load-state.cpp
2026-07-13 20:53:37 +08:00
91b1e1c89a
fix speculative logging
2026-07-13 18:29:21 +08:00
91c631b21d
chat : fix reasoning leak with force-opened bare <think> templates ( #24674 )
...
* chat : fix reasoning leak with force-opened bare <think> templates
The reasoning start tag inferred from prior turns can carry trailing
whitespace (e.g. <think>\n) while a force-open template prefills a bare
<think>. Trim the tag used for the prefix split so the bare prefill is
matched instead of being swallowed into content.
* chat : fix Nemotron Nano v2 regression
---------
Co-authored-by: Alde Rojas <hello@alde.dev >
2026-07-13 09:45:10 +02:00
0c4fa7a989
server : evict checkpoints within min-step of each other ( #25472 )
2026-07-12 15:59:14 -05:00
faabad2fab
up speculative vocab drafting from 128 to 256
2026-07-12 22:55:00 +08:00
f57cd915a9
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/ui-publish.yml
# CODEOWNERS
# docs/ops.md
# ggml/CMakeLists.txt
# ggml/src/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/argsort-ops.c
# scripts/sync-ggml.last
# tools/mtmd/tests/test-deepseek-ocr.py
2026-07-11 11:36:59 +08:00
9d8a50378a
Merge commit '961e4b26a7dd0e01e20599b27d709a74788ecb55' into concedo_experimental
...
# Conflicts:
# .github/workflows/hip-quality-check.yml
# AGENTS.md
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/concat-ops.c
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/flash-attn-ops.h
# ggml/src/ggml-hexagon/htp/hmx-fa-kernels.h
# ggml/src/ggml-hexagon/htp/hmx-queue.c
# ggml/src/ggml-hexagon/htp/hmx-queue.h
# ggml/src/ggml-hexagon/htp/hmx-utils.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/hvx-utils.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/rope-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-hexagon/htp/worker-pool.c
# ggml/src/ggml-hexagon/htp/worker-pool.h
# ggml/src/ggml-hip/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/flash_attn_f32_f16.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32_q4_0.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32_q8_0.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_mxfp4_f32_ns.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_q4_0_f32_ns.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_q4_1_f32_ns.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_q4_k_f32_ns.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_q5_0_f32_ns.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_q5_1_f32_ns.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_q5_k_f32_ns.cl
# ggml/src/ggml-opencl/kernels/gemm_moe_q6_k_f32_ns.cl
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/wgsl-shaders/flash_attn_vec_split.wgsl
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tools/cli/CMakeLists.txt
# tools/cli/cli.cpp
# tools/llama-bench/llama-bench.cpp
2026-07-11 11:13:27 +08:00
0f0245161e
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# ggml/src/ggml-opencl/ggml-opencl.cpp
# tests/test-backend-ops.cpp
# tests/test-quantize-fns.cpp
# tools/server/README.md
# tools/ui/src/lib/constants/settings-registry.ts
2026-07-11 09:20:40 +08:00
fc6e197fdd
Merge commit '33ca0dcb9d78c7c3a3b543db4c5fc9182abfe519' into concedo_experimental
...
# Conflicts:
# docs/backend/SYCL.md
# docs/build.md
# docs/ops.md
# docs/ops/SYCL.csv
# ggml/src/ggml-cuda/ggml-cuda.cu
# ggml/src/ggml-hip/CMakeLists.txt
# ggml/src/ggml-opencl/fa_tune.h
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/flash_attn_f16.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32_f16.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32_q4_0.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32_q8_0.cl
# ggml/src/ggml-opencl/kernels/mul_mv_f16_f32_l4.cl
# ggml/src/ggml-sycl/backend.hpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/cpy.cpp
# ggml/src/ggml-sycl/cpy.hpp
# ggml/src/ggml-sycl/dmmv.cpp
# ggml/src/ggml-sycl/element_wise.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/presets.hpp
# src/llama-graph.cpp
# src/llama-kv-cache.cpp
2026-07-11 08:46:16 +08:00
22b69b6e92
arg: prevent duplicate spec model downloads ( #25527 )
2026-07-10 16:53:26 +02:00
c4ae9a88f8
server: improve tools, remove apply_diff ( #25498 )
...
* server: improve tools, remove apply_diff
* improve edit tool
* add tools_io abstraction
* add tools_io_basic
* fix build
* move utils to class member
* add const
2026-07-10 11:52:59 +02:00
3de7dd4c8f
cli: add --output option ( #25484 )
2026-07-09 19:37:39 +02:00
c264f65ff9
cli : move to HTTP-based implementation ( #24948 )
...
* cli: move to HTTP-based implementation
* wip
* working
* remote server ok
* cli support router mode
Co-authored-by: Piotr Wilkin <ilintar@gmail.com >
* case: router with only one model
* Apply suggestions from code review
Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com >
* remove outdated comment
* use destructor instead
* add ftype
* cli-view --> cli-ui
* pimpl
* no more json in header
* nits fixes
* also show model aliases
---------
Co-authored-by: Piotr Wilkin <ilintar@gmail.com >
Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com >
2026-07-08 14:52:43 +02:00
f296fdfbed
common: auto-create prompts-log-dir at argument parsing, so all tools using the flag benefit ( #25322 )
2026-07-08 09:45:28 +02:00
c198af4dc2
spec : fix naming, spacing ( #25410 )
2026-07-07 18:52:30 +03:00
f5525f7e7a
server : fix draft model fit vs load inconsistency ( #25056 )
...
* fix: draft model fit vs load inconsistency
* refactor(server): unify draft/mtp parameter initialization, model, and context load
- moves speculative init to speculative.cpp
- changes server_context_impl model_dft and ctx_dft to use raw pointers
- fix: don't throttle progress callback when loading draft model
- refactor: rename draft model/ctx load method
* fix: valign
2026-07-07 17:20:42 +03:00
c1a411fb1b
common : add missing <fstream> include in common.h ( #25220 )
...
Signed-off-by: zhangrunda <zhangrunda1234@outlook.com >
2026-07-07 15:23:53 +02:00
defa95c306
speculative : fix out-of-bounds read in ngram-map on prompt shrink ( #23936 )
...
* speculative : fix out-of-bounds read in ngram-map on prompt shrink
* speculative : fix ngram-map cleanup cutoff after prompt shrink
2026-07-07 10:25:04 +03:00
ee445f93d8
common: Set optimal default thread count for ppc ( linux as well as AIX) ( #25237 )
2026-07-07 05:35:20 +08:00
e944cca86f
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# scripts/sync_vendor.py
# src/llama-model-loader.cpp
# tests/test-backend-ops.cpp
# tests/test-chat-auto-parser.cpp
# tests/test-chat.cpp
# tools/cli/cli.cpp
# tools/server/README.md
2026-07-05 11:30:12 +08:00
2d973636e2
chat: trim messages sent to StepFun parser (fixes long reasoning loops) ( #25238 )
...
* chat: trim messages sent to StepFun parser (fixes long reasoning loops)
* add regression test; remove duplicate template
* chat: trim StepFun content parts before rendering
The StepFun trim workaround ran on the already-rendered messages, where
typed content parts have been concatenated into a single string, so the
per-part whitespace could no longer be reached. Move the trim ahead of
rendering and apply it to content_parts text as well as the string
content and reasoning_content. Adds a content-parts regression test.
Co-Authored-By: Piotr Wilkin <ilintar@gmail.com >
Assisted-By: Claude Fable 5 <noreply@anthropic.com >
---------
Co-authored-by: tarruda <tpadilha84@gmail.com >
2026-07-03 23:12:11 +02:00
152d337fad
spec: support spec-draft-p-min in DFlash ( #25246 )
...
* spec: support spec-draft-p-min in DFlash
* dflash: add n_min guard
* dflash: guard both n_min and n_max
2026-07-03 15:40:06 +02:00
56d11ad4e8
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# docs/backend/OPENCL.md
# ggml/src/ggml-hexagon/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hex-dma.h
# ggml/src/ggml-hexagon/htp/hmx-utils.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-base.h
# ggml/src/ggml-hexagon/htp/hvx-exp.h
# ggml/src/ggml-hexagon/htp/hvx-sigmoid.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# scripts/snapdragon/adb/run-completion.sh
# scripts/snapdragon/adb/run-tool.sh
# scripts/snapdragon/ggml-hexagon-profile.py
# tests/test-backend-ops.cpp
2026-07-02 21:42:36 +08:00
a6647b1a32
common : use hf primary split as model path ( #25194 )
...
Fixes #25181
2026-07-01 18:33:00 +02:00
cb36463e4a
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/get_rows.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
2026-06-30 22:40:37 +08:00
799fcc04a5
common,server: handle bracketed IPv6 literals in URL authority ( #25140 )
...
* common,server: handle bracketed IPv6 literals in URL authority
Parse the [host]:port form (RFC 3986) and bracket IPv6 hosts when
formatting a URL authority: listening log, proxy Host header, proxy
log, client rebuild. The per-request remote_addr stays bare.
* common: restore unsupported scheme throw in url parser
Address @ngxson review: keep the explicit reject in port resolution so
the block stays self-contained. Non-http(s) schemes still throw (also
gated at the top of common_http_parse_url).
2026-06-30 16:16:44 +02:00
61ad97cbc1
Merge commit '8c146a8366304c871efc26057cc90370ccf58dad' into concedo_experimental
...
# Conflicts:
# src/CMakeLists.txt
# tests/test-llama-archs.cpp
2026-06-30 22:00:03 +08:00
6f4f53f2b7
common : dedup preset and cached model entries in /v1/models ( #25131 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-06-29 17:37:23 +02:00
16ef2badf6
fix superfluous filename defines
2026-06-29 21:04:09 +08:00
3b867bd4b1
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/release.yml
# SECURITY.md
# common/CMakeLists.txt
# docs/speculative.md
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f16.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32_f16.cl
# ggml/src/ggml-opencl/kernels/set_rows.cl
# ggml/src/ggml-openvino/ggml-openvino.cpp
# ggml/src/ggml-sycl/norm.cpp
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-chat-template.cpp
# tests/test-chat.cpp
# tests/test-export-graph-ops.cpp
# tests/test-jinja.cpp
# tests/test-llama-archs.cpp
# tools/rpc/CMakeLists.txt
# tools/rpc/README.md
2026-06-29 16:43:44 +08:00
277a105dc8
common : remove unused regex-partial ( #25118 )
2026-06-29 08:48:39 +02:00
Concedo
Matt Thompson
Aaron Teo
Piotr Wilkin (ilintar)
Pascal
Joe Rowell
Georgi Gerganov
Adrien Gallouët
Gaurav Garg
Xuan-Son Nguyen
Christian Kastner
Satinder Grewal
Frosty40
Aldehir Rojas
rankaiyx
Alex
zhangrunda
o7si
shalinib-ibm
Ruixiang Wang