08fbef5049
lcpp ui think budget
2026-06-21 22:56:10 +08:00
44bcead521
mcp fixes
2026-06-21 22:28:47 +08:00
1afe5a730a
minor fixes to handler newer lcpp ui
2026-06-21 22:21:10 +08:00
f202c0a457
mtp init -2
2026-06-21 10:21:48 +08:00
a5019767c3
docs
2026-06-20 23:02:56 +08:00
73cc7d9287
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/openvino.Dockerfile
# .devops/rocm.Dockerfile
# .devops/s390x.Dockerfile
# .devops/vulkan.Dockerfile
# .devops/zendnn.Dockerfile
# .dockerignore
# .pi/gg/SYSTEM.md
# ggml/CMakeLists.txt
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# scripts/sync-ggml.last
# scripts/sync_vendor.py
# tools/cli/README.md
# tools/cli/cli.cpp
# tools/mtmd/clip.cpp
# tools/server/README.md
2026-06-20 17:16:19 +08:00
84b8856295
Merge commit '32eddaf2ea8dd5d499dee9655592a89b91bfde9d' into concedo_experimental
...
# Conflicts:
# docs/multimodal.md
# docs/preset.md
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hex-dma.h
# ggml/src/ggml-hexagon/htp/hex-utils.h
# ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/hmx-queue.c
# ggml/src/ggml-hexagon/htp/hmx-queue.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# scripts/snapdragon/ggml-hexagon-profile.py
# scripts/ui-assets.cmake
# tools/export-lora/README.md
# tools/server/CMakeLists.txt
2026-06-20 11:18:24 +08:00
f449e05537
ggml-webgpu: add adapter toggles for F16 on Vulkan + NVIDIA
2026-06-20 08:12:32 +09:00
2b686a9120
server: refactor child --> router communication ( #24821 )
...
* server: refactor child --> router communication
* fix wakeup case
* add docs
* improve update_status()
* nits
2026-06-20 01:02:26 +02:00
4b48a53b6c
server : optimize get_token_probabilities ( #24796 )
...
Use std::partial_sort to order only the requested top-n tokens instead
of the full vocabulary
logprobs sort: vocab=128000 n_top=0 iters=100
full sort: 8555.6 us/op
partial sort: 704.3 us/op
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-06-19 23:26:54 +02:00
e475fa2b5f
mtmd, arg: fix utf8 handling on windows ( #24779 )
...
* mtmd, arg: fix utf8 handling on windows
* also fix ggml_fopen
* fix build fail
* also fix CLI
2026-06-19 22:28:38 +02:00
175147e8f6
server: remove all internal mentions about "webui" ( #24817 )
2026-06-19 22:12:46 +02:00
2fb3406be7
added ideogram 4 support
2026-06-20 00:29:48 +08:00
fabde3bf51
arg: Add comment line support to --api-key-file ( #23168 )
2026-06-19 17:33:54 +02:00
0d2d9ccbf6
vendor : update cpp-httplib to 0.48.0 ( #24787 )
2026-06-19 22:16:35 +08:00
8c2d6f6475
server: add --agent arg, remove redundant webui naming compat ( #24801 )
...
* server: add --agent arg, remove redundant webui naming compat
* corrent env
* fix the test
* llama-gen-docs
* nits: wordings
2026-06-19 16:06:13 +02:00
9b1e2fa8b8
advanced onready bypass
2026-06-19 21:44:37 +08:00
38724ab593
docker : build the UI ( #24794 )
...
* docker : build the UI
* cont : use existing APP_VERSION
2026-06-19 15:32:31 +02:00
e2e7a9b2d0
mtmd: several bug fixes ( #24784 )
...
* mtmd: several bug fixes
* fix build
* fix gemma4ua
* add sanity check in get_u32()
* fix build (2)
* area() avoid overflow
2026-06-19 12:18:36 +02:00
b14e3fb90c
spec: support eagle3 for qwen3.5 & 3.6 ( #24593 )
...
* spec: support qwen3.5 & 3.6 eagle3 draft
* eagle3: Add deferred boundary checkpoints restore support for hybrid models
* apply suggestions
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* spec: adapt to API change
* spec: fix naming
* cont : add TODO
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-06-19 13:08:50 +03:00
159d093a43
server: fix non-bound n_discard value (ctx shifting) ( #24786 )
...
* server: fix non-bound n_discard value
* Update tools/server/server-context.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-06-19 10:53:44 +02:00
5fd2dc2c41
sync : ggml
2026-06-19 10:19:14 +03:00
1868af13ac
ggml : bump version to 0.15.2 (ggml/1548)
2026-06-19 10:19:14 +03:00
df08e951d0
response format returns actual mp3 filename if requested
2026-06-19 15:18:53 +08:00
98236505e5
fix build
2026-06-19 15:10:34 +08:00
5bd21b8555
pi : remove docs from system prompt ( #24791 )
2026-06-19 09:34:00 +03:00
80452d65b9
server : consolidate slot selection into get_available_slot ( #24755 )
...
Absorb get_slot_by_id logic into get_available_slot so slot selection
is handled by a single function call. When a specific slot id is
requested, the LCP similarity check still runs to enable proper
prompt cache updates.
Assisted-by: pi:llama.cpp/Qwen3.6-27B
2026-06-19 09:22:34 +03:00
8141e730f1
ggml-cpu: support K tails in power10 Q8/Q4 MMA matmul ( #24753 )
...
* ggml-cpu: support K tails in Power10 MMA Q8/Q4 matmul
This patch removes the requirement that K be divisible by kc in the tinyBlas_Q0_PPC tiled matmul path. Process the final K panel using its actual depth and pass the reduced panel size through packing and kernel execution. This allows more workloads to use the MMA kernel and reduces fallback to mnpack.
* Apply suggestion from @taronaeo
Co-authored-by: Aaron Teo <taronaeo@gmail.com >
---------
Co-authored-by: Aaron Teo <taronaeo@gmail.com >
2026-06-19 08:55:38 +03:00
7780cf7288
suppress some warnings
2026-06-19 10:30:55 +08:00
6f4325ac87
support mp3 from api
2026-06-19 10:26:34 +08:00
db52540f73
mtmd: add batching support for internvl ( #24775 )
2026-06-19 01:16:16 +02:00
3a3edc9ac6
Ggml/cuda col2im 1d ( #24417 )
...
* cuda: add GGML_OP_COL2IM_1D, follow-up to the CPU op
* cuda: col2im_1d use fast_div_modulo for the index decomposition
* cuda: col2im_1d tighten supports_op, type match and contiguous dst
2026-06-18 22:23:01 +02:00
40f3aafc45
server: add "X-Accel-Buffering": "no" header to streaming endpoints ( #24774 )
...
* server: add "X-Accel-Buffering": "no" header to streaming endpoints
This header tells Nginx (as a reverse proxy) to NOT buffer responses. (only affects streaming endpoints)
Without it, Nginx will break streaming with certain applications (notably the Pi coding harness).
2026-06-18 22:01:24 +02:00
a6b3260a42
mtmd: add batching for mtmd-cli, add video tests ( #24778 )
2026-06-18 21:55:04 +02:00
32eddaf2ea
cmake : fix ui build with read-only source ( #24752 )
2026-06-18 18:59:18 +02:00
060ce1bf72
mtmd: refactor llava-uhd overview image handling (always use ov_img_first) ( #24769 )
...
* add dedicated "overview" for mtmd_image_preproc_out
* corrections
* correct (again)
* nits
* nits (2)
2026-06-18 18:53:49 +02:00
2bc18617ba
move jinja to the quick tab
2026-06-18 23:52:17 +08:00
d2c67959b3
hexagon: support for op-trace (fine-grain tracing of HVX/HMX/DMA events) ( #24592 )
...
* hex-optrace: add support for optrace and instrument matmul and flash-atten code
* hex-trace: improve trace event and prefetto generator
* hex-trace: add new script dedicated to handling traces, specifically perfetto traces
* hex-trace: add --head/--tail options to profile and trace tools
* hex-trace: fix whitespaces
* hex-trace: fix flake8 warnings
* hex-trace: fix flake8 warnings
* hmx-fa: restore q_tiles clearing
* hex-profile: remove circular dep in includes
* hex-trace: simplify trace sizing check
* hex-profile: sort events in the summary by name
2026-06-18 08:35:02 -07:00
7b6c5a2aed
docs: fix export-lora --lora-scaled syntax [no release] ( #24703 )
...
Assisted-by: Codex
2026-06-18 16:46:17 +02:00
45f49f9bd9
fixed tts mp3 saving
2026-06-18 22:33:30 +08:00
1b36e7f606
option to save tts as mp3, currently slightly bugged
2026-06-18 22:16:29 +08:00
2c64520ba6
failsafe target for macos
2026-06-18 21:46:54 +08:00
fe7c8b2414
server: (router) fix stopping_thread potentially hang ( #24728 )
...
* server: (router) fix stopping_thread potentially hang
* fix windows build
2026-06-18 15:41:09 +02:00
e1efd0991d
server: add "schema" and validation ( #24150 )
...
* wip
* working
* correct some limits
* add field name to error message
2026-06-18 15:40:58 +02:00
635c45e1a0
fix incorrect mtp layers setting (+1 squashed commits)
...
Squashed commits:
[8dad1a5c0] fix incorrect mtp layers setting
2026-06-18 21:10:28 +08:00
6591c33667
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/release.yml
# app/llama.cpp
# common/download.cpp
# docs/backend/SYCL.md
# docs/ops.md
# docs/ops/SYCL.csv
# ggml/CMakeLists.txt
# ggml/src/CMakeLists.txt
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-sycl/CMakeLists.txt
# ggml/src/ggml-sycl/backend.hpp
# ggml/src/ggml-sycl/common.cpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/dequantize.hpp
# ggml/src/ggml-sycl/dmmv.cpp
# ggml/src/ggml-sycl/dpct/helper.hpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/mmvq.cpp
# ggml/src/ggml-sycl/outprod.cpp
# ggml/src/ggml-sycl/vecdotq.hpp
# tools/server/README.md
2026-06-18 21:00:52 +08:00
08023072ef
server : add last-5-seconds generation speed display ( #24291 )
...
* server : add last-5-seconds generation speed display
* cont : clean-up
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-06-18 14:02:20 +02:00
20832179e2
ui: provide touch accessible model selection UI ( #24604 )
...
* ui : add model selector storybook stories
Covers list, favorites, single-model, all status states
(loading/loaded/sleeping/failed/idle), and selection states.
* ui : improve model selector mobile UX with hover media queries
Use @media (hover:none) to show action buttons directly on touch
devices and color-code them by model status (amber=sleeping,
green=loaded, muted=idle). Status dots hidden on touch. Desktop
hover behavior unchanged.
2026-06-18 13:14:20 +02:00
10786217e9
server : return HTTP 400 on invalid grammar ( #24144 ) ( #24154 )
...
Throw on grammar parse failure so the server returns HTTP 400
instead of silently dropping the constraint.
Add a regression test for the invalid-grammar response.
Fixes #24144
2026-06-18 12:49:14 +02:00
552258c535
server: (router) rework -hf preset repo ( #24739 )
...
* server: temporary remove HF remote preset
* rework remove preset.ini support
* rm unused get_remote_preset_whitelist()
* print warning
* add docs
* rm stray file
2026-06-18 12:45:23 +02:00
Concedo
Masashi Yoshimura
Xuan-Son Nguyen
Adrien Gallouët
Mikolaj Kucharski
Alessandro de Oliveira Faria (A.K.A.CABELO)
Aldehir Rojas
Ruixiang Wang
Georgi Gerganov
shalinib-ibm
Pascal
Reguna
o7si
Max Krasnyansky
Kangjia Gao
Aarni Koskela
Amos Wong
Anuj Attri