9864d46389
add password for musicui
2026-03-30 21:03:12 +08:00
42ad89cd86
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/llama-cli-cann.Dockerfile
# .devops/nix/package.nix
# .github/workflows/build-android.yml
# .github/workflows/build-cann.yml
# .github/workflows/build-msys.yml
# .github/workflows/docker.yml
# .github/workflows/editorconfig.yml
# .github/workflows/gguf-publish.yml
# .github/workflows/python-lint.yml
# .github/workflows/release.yml
# CMakeLists.txt
# docs/backend/CANN.md
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-rpc/ggml-rpc.cpp
# scripts/sync_vendor.py
# tests/test-chat-auto-parser.cpp
# tests/test-chat.cpp
# tests/test-json-schema-to-grammar.cpp
# tests/test-reasoning-budget.cpp
# tools/cli/cli.cpp
# tools/server/CMakeLists.txt
# tools/server/README.md
2026-03-30 20:45:38 +08:00
923d5fc5d0
warning: clip_image_preprocess has been moved, now you must manually copy init_vision from mtmd into clip.cpp's setup_init_vision_shim_kcpp
2026-03-30 20:39:55 +08:00
4fc3c28f1a
reasoning output parsing improvements
2026-03-29 23:35:24 +08:00
4a09f3805b
prepare for breaking merge
2026-03-29 14:09:29 +08:00
df6b7b5fdb
Merge branch 'concedo_experimental' of https://github.com/LostRuins/koboldcpp into concedo_experimental
2026-03-29 01:25:07 +08:00
3eedde8ab5
Merge commit 'ded446b34c0cd803a0122446b848619adbb458cf' into concedo_experimental
...
# Conflicts:
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# tests/test-backend-ops.cpp
2026-03-29 01:24:31 +08:00
9223f41320
sd: call SetCircularAxesAll directly ( #2078 )
2026-03-29 01:17:48 +08:00
8760d22a84
switch back to newly updated jimver github cuda toolkit
2026-03-29 01:17:11 +08:00
aac220f7e3
Merge commit '0fac87b157305eb82a70902327abffbbce25bd3e' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-android.yml
# .github/workflows/hip-quality-check.yml
# docs/multimodal.md
# scripts/hip/gcn-cdna-vgpr-check.py
# scripts/snapdragon/windows/run-bench.ps1
# scripts/snapdragon/windows/run-cli.ps1
# scripts/snapdragon/windows/run-tool.ps1
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
# tools/imatrix/imatrix.cpp
# tools/mtmd/CMakeLists.txt
2026-03-29 01:14:33 +08:00
674b7f5eee
indicate support for claude messages api
2026-03-29 00:57:58 +08:00
e397d3885c
common/json-schema: fix: handle non-capturing groups (?:...) in JSON schema pattern converter ( #21124 )
...
The regex-to-grammar converter in _visit_pattern() crashes with SIGSEGV
when a JSON schema "pattern" field contains a non-capturing group (?:...).
Root cause: when the parser sees '(' followed by '?', it pushes a warning
but does not advance past '?:'. The recursive transform() call then
interprets '?' as a quantifier and calls seq.back() on an empty vector,
causing undefined behavior.
This commonly occurs when serving OpenAI-compatible tool calls from
clients that include complex regex patterns in their JSON schemas (e.g.,
date validation patterns like ^(?:(?:\d\d[2468][048]|...)-02-29|...)$).
The fix:
- Skip '?:' after '(' to treat non-capturing groups as regular groups
- For unsupported syntax (?=, ?!, etc.), skip to matching ')' safely,
handling escaped characters to avoid miscounting parenthesis depth
- Adjust the ')' unbalanced-parentheses check using direct char
comparisons instead of substr
- Add test cases for non-capturing groups (C++ only, as the JS/Python
implementations do not yet support this syntax)
2026-03-28 17:55:38 +01:00
e3b7905e1c
added anthropic messages api support
2026-03-29 00:55:32 +08:00
5ad9e3ee31
crude openai responses streaming
2026-03-29 00:16:30 +08:00
e6f2ec01ff
common : add reasoning_format = none support to gpt-oss ( #21094 )
2026-03-28 09:33:39 -05:00
edfb440a2f
server : fix processing of multiple back-to-back mtmd chunks ( #21107 )
2026-03-28 16:27:36 +02:00
3d66da1809
ci : gracefully shut down the server ( #21110 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-03-28 14:49:57 +01:00
82b703f8bc
Document custom default webui preferences in server README ( #19771 )
2026-03-28 14:19:16 +01:00
94b266a6b0
musicui fix reset defaults
2026-03-28 21:09:40 +08:00
51a84efc53
webui: Conversation forking + branching improvements ( #21021 )
...
* refactor: Make `DialogConfirmation` extensible with children slot
* feat: Add conversation forking logic
* feat: Conversation forking UI
* feat: Update delete/edit dialogs and logic for forks
* refactor: Improve Chat Sidebar UX and add MCP Servers entry
* refactor: Cleanup
* feat: Update message in place when editing leaf nodes
* chore: Cleanup
* chore: Cleanup
* chore: Cleanup
* chore: Cleanup
* chore: Cleanup
* chore: Cleanup
* refactor: Post-review improvements
* chore: update webui build output
* test: Update Storybook test
* chore: update webui build output
* chore: update webui build output
2026-03-28 13:38:15 +01:00
1e787cd03a
improve responses api
2026-03-28 18:42:15 +08:00
f768b2a4bd
whatever, i tried
2026-03-28 17:32:07 +08:00
b0f0dd3e51
vendor : update cpp-httplib to 0.40.0 ( #21100 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-03-28 08:59:44 +01:00
0eb4764182
vulkan: add noncontiguous GLU support ( #21081 )
...
* vulkan: add noncontiguous GLU support
* fix compile issue
2026-03-28 08:44:56 +01:00
1f5d15e665
common/parser: fix reasoning whitespace bugs + extra parser tests ( #21085 )
...
* fix whitespace reasoning issues + add reconstruction tests
* Proper fix
* fix Nemotron autoparser test expectations to include newline in marker
2026-03-28 07:29:26 +01:00
f80fdd4314
updated sdui
2026-03-28 11:24:03 +08:00
547659fdbf
allow planning music with llm (+1 squashed commits)
...
Squashed commits:
[9a3bbf072] allow planning music with llm
2026-03-28 11:19:39 +08:00
c46758d28f
cli : add /glob command ( #21084 )
...
* add /glob command
* output error when max files reached
* support globbing outside curdir
2026-03-28 02:33:04 +01:00
bf934f28db
docker : fix and enable ARM64 image build ( #20929 )
...
* CI: fix ARM64 image build error & enable compilation
* Update .github/workflows/docker.yml
Co-authored-by: Aaron Teo <taronaeo@gmail.com >
* CI: revert ggml/src/ggml-cpu/CMakeLists.txt
* Update .github/workflows/docker.yml
Co-authored-by: Aaron Teo <taronaeo@gmail.com >
* CI: update runs-on to ubuntu24.04, and update ARM64 build image ( ubuntu_version: "24.04")
* CI: change cpu.Dockerfile gcc to 14;
* CI : cpu.Dockerfile , update pip install .
* Update .github/workflows/docker.yml
Co-authored-by: Aaron Teo <taronaeo@gmail.com >
---------
Co-authored-by: Aaron Teo <taronaeo@gmail.com >
2026-03-28 01:45:09 +01:00
5c1a7b8355
server : add custom socket options to disable SO_REUSEPORT ( #21056 )
...
* server : add custom socket options to disable SO_REUSEPORT
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
* Add --reuse-port
$ strace -e trace=setsockopt,bind build/bin/llama-server -lv 2 --reuse-port
setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
setsockopt(3, SOL_SOCKET, SO_REUSEPORT, [1], 4) = 0
bind(3, {sa_family=AF_INET, sin_port=htons(8080), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
$ strace -e trace=setsockopt,bind build/bin/llama-server -lv 2
setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
bind(3, {sa_family=AF_INET, sin_port=htons(8080), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
* Update tools/server/README.md (llama-gen-docs)
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
* Fix windows
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
---------
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-03-28 01:12:43 +01:00
59d840209a
common : inhibit lazy grammar sampler while reasoning is active ( #20970 )
...
* common : inhibit grammar while reasoning budget is active
* cont : update force_pos in accept
* cont : fix tests
* cont : tweak should apply logic
* cont : return early not using grammar sampler
* Add tests
* cont : prevent backend sampling when reasoning budget enabled
* cont : fix typo
---------
Co-authored-by: Piotr Wilkin <piotr.wilkin@syndatis.com >
2026-03-27 18:30:40 +01:00
3ec6381123
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-self-hosted.yml
# .github/workflows/build.yml
# .github/workflows/copilot-setup-steps.yml
# .github/workflows/gguf-publish.yml
# ci/run.sh
# docs/backend/OPENVINO.md
# examples/llama.android/lib/src/main/cpp/ai_chat.cpp
# ggml/src/ggml-sycl/add-id.cpp
# requirements/requirements-pydantic.txt
# tests/test-gguf.cpp
# tests/test-jinja.cpp
# tests/test-llama-archs.cpp
# tools/gguf-split/README.md
# tools/llama-bench/llama-bench.cpp
2026-03-28 01:18:20 +08:00
2cdf02102e
preserve previous filename
2026-03-28 01:13:03 +08:00
e3c6227d46
sd: report back image generation parameters and metadata ( #2062 )
...
* sd: refactor image generation result handling
* sd: report back image generation metadata
2026-03-28 00:49:03 +08:00
ff934e29bc
server: Introduce LLAMA_BUILD_WEBUI build flag to allow disabling the embedded web ui ( #20158 )
...
* introduce LLAMA_SERVER_NO_WEBUI
* LLAMA_SERVER_NO_WEBUI → LLAMA_BUILD_WEBUI
* LLAMA_BUILD_WEBUI ON by default not based on LLAMA_STANDALONE
* MIssed this
* Add useWebUi to package.nix
2026-03-27 17:25:55 +01:00
ee051c1e4e
hexagon: support for IQ4_NL and MXFP4 ( #21018 )
...
* ggml-hexagon: add IQ4_NL and MXFP4 HMX matmul support
- Add IQ4_NL quantization type support to Hexagon backend (buffer
set/get tensor repack, mul_mat, mul_mat_id dispatch)
- Implement HVX IQ4_NL vec_dot kernels (1x1, 2x1, 2x2) with
LUT-based 4-bit index to int8 kvalue dequantization
- Add MXFP4 HMX dequantization path with E8M0 scale conversion,
including batch-4 fast path and single-tile fallback
- Unify quantized row size / scale offset logic to handle Q4_0,
Q8_0, IQ4_NL, and MXFP4 in the DMA fetch path
* ggml-hexagon: fix SKIP_QUANTIZE src1 address mismatch in mixed-quant models
* Fix the pragma indent
2026-03-27 09:22:41 -07:00
e6f6770515
webui: Improve Chat Messages initial scroll + auto-scroll logic + add lazy loading with transitions to content blocks ( #20999 )
...
* refactor: Always use agentic content renderer for Assistant Message
* feat: Improve initial scroll + auto-scroll logic + implement fade in action for content blocks
* chore: update webui build output
2026-03-27 17:01:36 +01:00
0c2b679ea3
support bf16 quantkv cache type
2026-03-28 00:01:17 +08:00
326542f480
rudimentary responses api, not usable yet
2026-03-27 23:38:08 +08:00
81cebb6179
remove unused field
2026-03-27 22:52:36 +08:00
f0818e1eae
Add socket timeout to is_port_in_use() to fix ~280s startup delay on WSL2 ( #2077 )
...
On WSL2 with networkingMode=mirrored, connect_ex() to non-listening ports
gets black-holed through the Windows host networking stack instead of
returning ECONNREFUSED. Without a timeout, TCP SYN retransmits with
exponential backoff (1+2+4+8+16+32+64 ≈ 127s per port), causing Router
Mode's port scan of 15001-15010 to stall for ~280 seconds on startup.
Adding a 1-second timeout makes connect_ex() fail fast, reducing startup
from ~303s to ~23s on affected systems.
Tested on WSL2 Ubuntu 24.04 with mirrored networking, KoboldCpp v1.110,
RTX 3090 Ti, Qwen3.5-27B Q4_K_M.
2026-03-27 22:50:59 +08:00
48cda24c11
server: remove the verbose_prompt parameter ( #21059 )
...
* server: respect the verbose_prompt parameter
* Revert "server: respect the verbose_prompt parameter"
This reverts commit 8ed885cf375b2c8ba641c661f3667df70b9797f4.
* Remove --verbose-prompt parameter from llama-server
* Using set_examples instead of set_excludes
2026-03-27 13:36:13 +02:00
871f1a2d2f
mtmd: add more sanity checks ( #21047 )
2026-03-27 11:00:52 +01:00
20197b6fe3
server: add built-in tools backend support ( #20898 )
...
* wip: server_tools
* refactor
* displayName -> display_name
* snake_case everywhere
* rm redundant field
* change arg to --tools all
* add readme mention
* llama-gen-docs
2026-03-27 10:07:11 +01:00
ba38f3becc
rpc : proper handling of data pointers to CPU buffers ( #21030 )
...
The compute graph may contain tensors pointing to CPU buffers. In these
cases the buffer address is serialized as 0 and sent over the wire.
However, the data pointer is serialized as-is and this prevents proper
validation on the server side. This patches fixes this by serializing
the data pointer as 0 for non-RPC buffers and doing proper validation on
the server side.
closes : #21006
2026-03-27 10:59:35 +02:00
37f230dd7c
completion : session_tokens insert range in completion tool (no-op → correct) ( #20917 )
...
The embd.begin(), embd.begin() range is empty and inserts nothing, so session_tokens never gets updated after
decoding. Should be embd.begin(), embd.end(). Introduced in commit 2b6dfe8
2026-03-27 09:25:58 +01:00
a308e584ca
completion : Fix segfault on model load failure ( #21049 )
2026-03-27 10:01:13 +02:00
d0fa2c9fbb
Send reasoning content back to the model across turns via the reasoning_content API field ( #21036 )
...
* webui: send reasoning_content back to model in context
Preserve assistant reasoning across turns by extracting it from
internal tags and sending it as a separate reasoning_content field
in the API payload. The server and Jinja templates handle native
formatting (e.g. <think> tags for Qwen, GLM, DeepSeek...).
Adds "Exclude reasoning from context" toggle in Settings > Developer
(off by default, so reasoning is preserved). Includes unit tests.
* webui: add syncable parameter for excludeReasoningFromContext
* chore: update webui build output
2026-03-27 08:17:35 +01:00
9bcb4eff4d
metal : Fix dimension constraint violation in matmul2d descriptor ( #21048 )
...
Updates Metal tensor API test probe to fix the dimension constraint violation in the matmul2d descriptor (at least one value must be a multiple of 16).
2026-03-27 09:05:21 +02:00
6861f6509a
CANN: update docker images to 8.5.0 and improve CANN.md ( #20801 )
...
* cann: update docker images to 8.5.0
- bump CANN base image from 8.3.rc2 to 8.5.0
- bump ASCEND_VERSION from 8.1.RC1.alpha001 to 8.5.0
Move to newer stable releases.
* cann: update CANN.md
* Update CANN.md to include BF16 support
Added BF16 support information to the CANN documentation and corrected formatting for the installation instructions.
* Fix formatting issues in CANN.md
Fix 234: Trailing whitespace
2026-03-27 08:53:00 +08:00
Concedo
Wagner Bruna
Adrien
Aldehir Rojas
Georgi Gerganov
Adrien Gallouët
Woof Dog
Aleksander Grygier
Ruben Ortlam
Piotr Wilkin (ilintar)
Sigbjørn Skjæret
Ts-sound
Kusha Gharahi
Yiwei Shao
scottf007
AN Long
Xuan-Son Nguyen
Radoslav Gerganov
mtmcp
Pascal
ren
KokerZhou