c82c0b463a
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/labeler.yml
# .github/workflows/release.yml
# examples/debug/debug.cpp
# ggml/src/ggml-cuda/common.cuh
# ggml/src/ggml-cuda/mmq.cuh
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# src/llama-vocab.cpp
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tests/test-json-schema-to-grammar.cpp
# tools/mtmd/CMakeLists.txt
2026-04-09 17:45:04 +08:00
5529748a01
Merge commit 'de1aa6fa73e135839109e09fec1a0997f4207b2a' into concedo_experimental
...
# Conflicts:
# docs/build.md
# docs/ops.md
# docs/ops/WebGPU.csv
# ggml/src/ggml-sycl/dequantize.hpp
# ggml/src/ggml-sycl/dmmv.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/mmvq.cpp
# ggml/src/ggml-sycl/quants.hpp
# ggml/src/ggml-sycl/vecdotq.hpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# tests/test-backend-ops.cpp
# tests/test-quantize-fns.cpp
2026-04-09 17:16:33 +08:00
660600081f
server: respect the ignore eos flag ( #21203 )
2026-04-08 17:12:15 +02:00
4a05e0c566
webui : send both backend_sampling == false/true ( #18781 )
...
* webui : send both backend_sampling == false/true
* feat: Parameter sync
---------
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com >
2026-04-08 16:35:52 +02:00
e9fd96283d
Propose fix a couple of typos ( #21581 )
...
Signed-off-by: John E <jeis4wpi@outlook.com >
2026-04-08 16:29:03 +02:00
ece522f98c
chore: Remove legacy files ( #21606 )
2026-04-08 09:55:08 +02:00
09343c0198
model : support step3-vl-10b ( #21287 )
...
* feat: support step3-vl-10b
* use fused QKV && mapping tensor in tensor_mapping.py
* guard hardcoded params and drop crop metadata
* get understand_projector_stride from global config
* img_u8_resize_bilinear_to_f32 move in step3vl class
* Apply suggestions from code review
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* fix the \r\n mess
* add width and heads to MmprojModel.set_gguf_parameters
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-04-08 09:51:31 +02:00
97508acb17
webui: fix syntax highlighting lost after streaming for non-common languages ( #21206 )
...
* webui: fix syntax highlighting lost for non-common languages after streaming
rehype-highlight uses lowlight internally, which only bundles 37 "common"
languages. The streaming code path uses highlight.js directly (192 languages),
so languages like Haskell highlight correctly while streaming but lose all
color once the code block closes. Pass the full lowlight language set to
rehype-highlight so both paths support the same languages.
* webui: rebuild static files after rebase
2026-04-08 08:58:08 +02:00
69c28f1547
llama-server: fix model params not propagated ( #21509 )
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com >
2026-04-07 21:39:41 +08:00
e8f5082697
server : fix restore for checkpoints with pos_min == 0 ( #21510 )
2026-04-07 15:29:17 +03:00
482192f12d
webui : store reasoning_content so it is sent back in subsequent requests ( #21249 )
2026-04-07 13:32:44 +02:00
ecce0087da
fix: Detect streaming state in reasoning content blocks ( #21549 )
2026-04-07 12:04:41 +02:00
d1f82e382d
Fix rtl text rendering ( #21382 )
...
* Fix Arabic RTL text rendering in web UI
- Add dir='auto' attributes to markdown containers and blocks
- Implement post-processing to add dir='auto' to all text elements
- Replace directional CSS properties with logical properties for proper RTL list alignment
- Ensure bidirectional text support for mixed Arabic/English content
* Clean up commented duplicate function
Remove the commented-out duplicate transformMdastNode function
that was left over from refactoring.
* Fix Arabic RTL text rendering in web UI
- Add dir='auto' attributes to markdown containers and blocks
- Implement post-processing to add dir='auto' to all text elements
- Replace directional CSS properties with logical properties for proper RTL list alignment
- Minor code formatting improvements
This ensures bidirectional text support for mixed Arabic/English content in the llama.cpp web UI.
* Implement rehype plugin for comprehensive RTL text support
- Add rehypeRtlSupport plugin that applies dir='auto' to all elements with children
- Replace DOMParser-based approach with efficient HAST tree processing
- Remove hardcoded element lists for better maintainability
- Ensure proper bidirectional text rendering for mixed RTL/LTR content
* Fix RTL text rendering with rehype plugin and cleanup
* fix: prettier formatting
2026-04-07 11:37:20 +02:00
15d269197e
Merge commit '506200cf8b5c8419ce97d16dc8c50f4634e21ebe' into concedo_experimental
...
# Conflicts:
# docs/multimodal.md
# scripts/compare-llama-bench.py
# src/llama-vocab.cpp
# tools/llama-bench/README.md
# tools/llama-bench/llama-bench.cpp
2026-04-07 14:58:36 +08:00
2e1f0a889e
ggml: add Q1_0 1-bit quantization support (CPU) ( #21273 )
...
* ggml: add Q1_0 and Q1_0_g128 1-bit quantization support (CPU)
* add generic fallback for x86
* remove Q1_0 (group size 32)
* rename Q1_0_g128 => Q1_0
* fix Q1_0 LlamaFileType Enum
* Fix trailing spaces; add generic fallback for othre backends
* Apply suggestions from code review
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* fix /r/n spacing + arch-fallback
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-04-06 20:55:21 +02:00
94ca829b60
llama-bench: add -fitc and -fitt to arguments ( #21304 )
...
* llama-bench: add `-fitc` and `-fitt` to arguments
* update README.md
* address review comments
* update compare-llama-bench.py
2026-04-06 22:26:02 +08:00
a395af65db
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-riscv.yml
# .github/workflows/build.yml
# ggml/src/ggml-hexagon/htp/argsort-ops.c
# ggml/src/ggml-sycl/fattn-tile.hpp
# tools/mtmd/CMakeLists.txt
2026-04-06 20:56:02 +08:00
482d862bcb
server : handle unsuccessful sink.write in chunked stream provider ( #21478 )
...
Check the return value of sink.write() in the chunked content provider
and return false when the write fails, matching cpp-httplib's own
streaming contract. This prevents logging chunks as sent when the sink
rejected them and properly aborts the stream on connection failure.
2026-04-06 14:03:02 +02:00
3979f2bb08
docs: add hunyuan-ocr gguf, also add test [no ci] ( #21490 )
2026-04-06 14:02:37 +02:00
58190cc84d
llama : correct platform-independent loading of BOOL metadata ( #21428 )
...
* model-loader : fix GGUF bool array conversion
* model-loader : fix remaining GGUF bool pointer uses
2026-04-06 01:40:38 +02:00
af76639f72
model : add HunyuanOCR support ( #21395 )
...
* HunyuanOCR: add support for text and vision models
- Add HunyuanOCR vision projector (perceiver-based) with Conv2d merge
- Add separate HUNYUAN_OCR chat template (content-before-role format)
- Handle HunyuanOCR's invalid pad_token_id=-1 in converter
- Fix EOS/EOT token IDs from generation_config.json
- Support xdrope RoPE scaling type
- Add tensor mappings for perceiver projector (mm.before_rms, mm.after_rms, etc.)
- Register HunYuanVLForConditionalGeneration for both text and mmproj conversion
* fix proper mapping
* Update gguf-py/gguf/tensor_mapping.py
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com >
* Update tools/mtmd/clip.cpp
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com >
* address comments
* update
* Fix typecheck
* Update convert_hf_to_gguf.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update convert_hf_to_gguf.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update convert_hf_to_gguf.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update convert_hf_to_gguf.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
---------
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com >
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-04-05 23:32:14 +02:00
5d3a4a7da5
server : fix logging of build + system info ( #21460 )
...
This PR changes the logging that occurs at startup of llama-server.
Currently, it is redundant (including CPU information twice) and it is
missing the build + commit info.
2026-04-05 16:14:02 +02:00
376aaf258c
Merge branch 'upstream' into concedo_experimental
2026-04-04 23:56:02 +08:00
9c699074c9
server: Fix undefined timing measurement errors in server context ( #21201 )
...
Co-authored-by: Dan Hoffman <dhoffman@cyket.net >
2026-04-04 22:11:19 +08:00
2e4f94822e
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-self-hosted.yml
# .github/workflows/docker.yml
# ci/run.sh
# docs/build.md
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# src/llama-vocab.cpp
# tests/test-chat.cpp
# tests/test-jinja.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/server/README.md
2026-04-04 14:27:23 +08:00
50e0ad08fb
server: save and clear idle slots on new task (--clear-idle) ( #20993 )
...
* server: clear idle slots KV from VRAM (LLAMA_KV_KEEP_ONLY_ACTIVE)
* server: move idle slot KV clearing to slot release
The save "cost" is now paid by the finishing request.
* server: add --kv-clear-idle flag, enable by default
* server: skip clearing last idle slot, clear on launch
* server: test --no-kv-clear-idle flag
* server: simplify on-release clearing loop
* server: remove on-release KV clearing, keep launch-only
* cont : clean-up
* tests: update log strings after --clear-idle rename
* tests: use debug tags instead of log message matching
* test: fix Windows CI by dropping temp log file unlink
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-04-03 19:02:27 +02:00
f1f793ad06
common/parser: fix call ID detection (Mistral parser mostly) + atomicity for tag-json parsers ( #21230 )
...
* Fix call ID detection (Mistral parser mostly) + atomicity for tag-json parsers
* Rename
* Update common/chat-auto-parser-generator.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-04-03 17:51:52 +02:00
e8cffa37c8
fixed gemma4v image crashing on encode, however images are not yet working correctly
2026-04-03 15:56:35 +08:00
17678748ac
fixed mtmd build
2026-04-03 14:41:24 +08:00
34ad53e950
merged support for gemma4. the e2b, e4b and 26b work, the 31b does not
2026-04-03 11:07:46 +08:00
5dee1a1cbb
Merge commit 'fbd441c37933550c1e3365dc84dd73232334c15d' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# AGENTS.md
# ci/run.sh
# docs/build.md
# embd_res/templates/LFM2.5-Instruct.jinja
# ggml/CMakeLists.txt
# ggml/src/ggml-cuda/fattn.cu
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/htp-msg.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-div.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/dequantize.hpp
# ggml/src/ggml-sycl/mmvq.cpp
# ggml/src/ggml-sycl/vecdotq.hpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/flash_attn.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec.wgsl
# scripts/hip/gcn-cdna-vgpr-check.py
# scripts/sync-ggml.last
# tests/test-chat.cpp
2026-04-03 01:06:02 +08:00
63f8fe0ef4
model, mtmd: fix gguf conversion for audio/vision mmproj ( #21309 )
...
* fix gguf conversion for audio/vision mmproj
* fix test
2026-04-02 17:10:32 +02:00
d6dac92bfd
Ignore Transfer-Encoding header. ( #20269 )
2026-04-02 10:41:19 +02:00
12dbf1da95
server: Bypass API Key validation for WebUI static bundle assets ( #21269 )
...
* fix: Bypass API Key validation for static bundle assets
* refactor: All bypassed routes in `public_endpoints`
* test: Update static assets API Key test
2026-04-01 21:32:15 +02:00
4951250235
llama : refactor llama_model_quantize_params to expose a pure C interface ( #20346 )
...
* Refactor llama_model_quantize_params to expose a pure C interface
* Restore comment and cleanup struct def
* Code review refactoring
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* Code review refactoring
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-04-01 08:43:00 +03:00
31aa072da1
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# .github/workflows/release.yml
# .gitignore
# examples/batched/batched.cpp
# examples/debug/debug.cpp
# examples/eval-callback/eval-callback.cpp
# examples/idle/idle.cpp
# examples/lookahead/lookahead.cpp
# examples/lookup/lookup-create.cpp
# examples/lookup/lookup-stats.cpp
# examples/lookup/lookup.cpp
# examples/parallel/parallel.cpp
# examples/passkey/passkey.cpp
# examples/retrieval/retrieval.cpp
# examples/save-load-state/save-load-state.cpp
# examples/speculative-simple/speculative-simple.cpp
# examples/speculative/speculative.cpp
# examples/training/finetune.cpp
# ggml/CMakeLists.txt
# ggml/src/ggml-cann/aclnn_ops.cpp
# ggml/src/ggml-cann/common.h
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-sycl/fattn-tile.hpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/cpy.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/embed_wgsl.py
# ggml/src/ggml-webgpu/wgsl-shaders/rope.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/soft_max.wgsl
# scripts/sync-ggml.last
# tests/export-graph-ops.cpp
# tests/test-chat.cpp
# tests/test-state-restore-fragmented.cpp
# tests/test-thread-safety.cpp
# tools/batched-bench/batched-bench.cpp
# tools/cli/cli.cpp
# tools/cvector-generator/cvector-generator.cpp
# tools/export-lora/export-lora.cpp
# tools/imatrix/imatrix.cpp
# tools/perplexity/perplexity.cpp
# tools/results/results.cpp
# tools/server/CMakeLists.txt
2026-04-01 10:54:13 +08:00
0fcb3760b2
fix: Use lower-case proxy headers naming ( #21235 )
2026-03-31 17:47:46 +02:00
4a00bbfed6
server: (webui) no more gzip compression ( #21073 )
...
* webui: no more gzip
* try changing a small line
* Revert "try changing a small line"
This reverts commit 0d7a3531593d87b724d404c8727a96becab3ab07.
* fix lint
* fix test
* rebuild
* split into html/css/js
* lint
* chore: update webui build output
* chore: Update git hooks script
* server: update webui build output
* chore: Update pre-commit hook
* refactor: Cleanup
---------
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com >
2026-03-31 15:44:26 +02:00
41361c8599
common : move up common_init() and fix Windows UTF-8 logs ( #21176 )
...
The build info is now only for debug, so we avoid the duplicate
with `--version`.
The UTF-8 setup at the beginning is needed to avoid logging
garbage on Windows.
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-03-31 12:53:41 +02:00
90aa83c6bd
common: add bounds check in common_init_result::sampler to prevent segfault on failed model load ( #21082 )
...
* common: add bounds check in common_init_result::sampler to prevent segfault on failed model load
* Revert a308e584ca
2026-03-31 13:04:42 +03:00
fcc2d598c8
fix: include API key in CORS proxy requests for MCP connections ( #21193 )
...
* fix: include API key in CORS proxy requests for MCP connections
When llama-server is started with --api-key-file and --webui-mcp-proxy,
the /cors-proxy endpoint requires authentication. The WebUI was not
including the Authorization header in proxy requests, causing MCP
connections to fail with 401.
Inject getAuthHeaders() into requestInit when useProxy is true so the
proxy request carries the Bearer token alongside the forwarded target
headers.
Fixes #21167
* fix: simplify headers assignment based on reviewer suggestion
Apply buildProxiedHeaders only when useProxy is true, pass headers
directly to the transport otherwise.
2026-03-31 10:52:34 +02:00
4453e77561
server/webui: cleanup dual representation approach, simplify to openai-compat ( #21090 )
...
* server/webui: cleanup dual representation approach, simplify to openai-compat
* feat: Fix regression for Agentic Loop UI
* chore: update webui build output
* refactor: Post-review code improvements
* chore: update webui build output
* refactor: Cleanup
* chore: update webui build output
---------
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com >
2026-03-31 10:42:06 +02:00
a3a5897d93
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/intel.Dockerfile
# .github/workflows/python-type-check.yml
# embd_res/templates/Qwen3.5-4B.jinja
# examples/model-conversion/scripts/causal/compare-logits.py
# examples/model-conversion/scripts/utils/check-nmse.py
# examples/model-conversion/scripts/utils/compare_tokens.py
# examples/model-conversion/scripts/utils/semantic_check.py
# examples/sycl/build.sh
# examples/sycl/run-llama2.sh
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hex-dma.h
# ggml/src/ggml-hexagon/htp/rope-ops.c
# scripts/gen-unicode-data.py
# tests/test-chat.cpp
2026-03-30 21:41:19 +08:00
42ad89cd86
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/llama-cli-cann.Dockerfile
# .devops/nix/package.nix
# .github/workflows/build-android.yml
# .github/workflows/build-cann.yml
# .github/workflows/build-msys.yml
# .github/workflows/docker.yml
# .github/workflows/editorconfig.yml
# .github/workflows/gguf-publish.yml
# .github/workflows/python-lint.yml
# .github/workflows/release.yml
# CMakeLists.txt
# docs/backend/CANN.md
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-rpc/ggml-rpc.cpp
# scripts/sync_vendor.py
# tests/test-chat-auto-parser.cpp
# tests/test-chat.cpp
# tests/test-json-schema-to-grammar.cpp
# tests/test-reasoning-budget.cpp
# tools/cli/cli.cpp
# tools/server/CMakeLists.txt
# tools/server/README.md
2026-03-30 20:45:38 +08:00
389c7d4955
webui: Fix branching logic on edit message ( #21175 )
...
* fix: Branching logic + small refactor
* chore: update webui build output
2026-03-30 14:40:50 +02:00
923d5fc5d0
warning: clip_image_preprocess has been moved, now you must manually copy init_vision from mtmd into clip.cpp's setup_init_vision_shim_kcpp
2026-03-30 20:39:55 +08:00
e2eb39e81c
ci : bump ty to 0.0.26 ( #21156 )
...
* fix incorrect type ignore comments
* bump ty to 0.0.26
2026-03-30 09:29:15 +02:00
abf9a62161
server: wrap headers for mcp proxy ( #21072 )
...
* server: wrap headers for mcp proxy
* Update tools/server/server-cors-proxy.h
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* fix build
* chore: update webui build output
* chore: update webui build output
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com >
2026-03-30 08:59:16 +02:00
4a09f3805b
prepare for breaking merge
2026-03-29 14:09:29 +08:00
aac220f7e3
Merge commit '0fac87b157305eb82a70902327abffbbce25bd3e' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-android.yml
# .github/workflows/hip-quality-check.yml
# docs/multimodal.md
# scripts/hip/gcn-cdna-vgpr-check.py
# scripts/snapdragon/windows/run-bench.ps1
# scripts/snapdragon/windows/run-cli.ps1
# scripts/snapdragon/windows/run-tool.ps1
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
# tools/imatrix/imatrix.cpp
# tools/mtmd/CMakeLists.txt
2026-03-29 01:14:33 +08:00
Concedo
Yuri Khrustalev
Georgi Gerganov
John Eismeier
Aleksander Grygier
forforever73
Hamish M. Blair
Aaron Teo
Aldehir Rojas
Kabir08
Pasha Khosravi
Aman Gupta
lainon1
Xuan-Son Nguyen
anchortense
Richard Davison
ddh0
Dan Hoffman
Yes You Can Have Your Own
Piotr Wilkin (ilintar)
Roger Chen
Ed Addario
Adrien Gallouët
mtmcp
SATISH K C
Sigbjørn Skjæret