4e43c21e58
Merge commit '9d5d882d8cd0f0a9283d87ed5e6fe3ee0d925fb1' into concedo_experimental
...
# Conflicts:
# .github/labeler.yml
# app/CMakeLists.txt
# app/llama.cpp
# build-xcframework.sh
# common/CMakeLists.txt
# common/download.h
# docs/backend/SYCL.md
# docs/backend/snapdragon/CMakeUserPresets.json
# docs/speculative.md
# ggml/CMakeLists.txt
# ggml/include/ggml-sycl.h
# ggml/src/ggml-hexagon/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/cmake-toolchain.cmake
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hex-dma.h
# ggml/src/ggml-hexagon/htp/hex-utils.h
# ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/htp_iface.idl
# ggml/src/ggml-hexagon/htp/hvx-base.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/libggml-htp.inf
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/norm.cl
# ggml/src/ggml-sycl/conv3d.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# scripts/snapdragon/adb/run-completion.sh
# scripts/snapdragon/adb/run-tool.sh
# scripts/snapdragon/ggml-hexagon-profile.py
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-thread-safety.cpp
# tools/llama-bench/llama-bench.cpp
# tools/mtmd/CMakeLists.txt
# tools/mtmd/tests/test-deepseek-ocr.py
2026-06-27 10:18:52 +08:00
e9d1b76d0a
server: use status code 403 for disabled features ( #24970 )
...
* server: use status code 403 for disabled features
* cont
* fix test case
2026-06-25 16:36:40 +02:00
60bc8866b1
common: refactor model handling ( #24980 )
...
* common: refactor models handling
* remote preset
* cont
* rm skip_download option
* missing header
* fix plan.model_files
* fix --offline case
* move hf_plan to download
* refactor
* rm redundant curr_ex, add comments
* adapt
2026-06-25 15:17:51 +02:00
3e61ea0e2f
ui: fix always-show-sidebar-on-desktop setting after navigation refactor ( #24979 )
2026-06-25 09:45:55 +02:00
e12a0128ab
build: include libmtmd in Apple XCFramework ( #21935 )
...
Adds an opt-in LLAMA_BUILD_MTMD CMake option so build-xcframework.sh
can link libmtmd.a into the framework binary without pulling in the
rest of tools/ (which doesn't cross-build cleanly to iOS/tvOS/visionOS).
- CMakeLists.txt: new option, default OFF. When on with
LLAMA_BUILD_TOOLS=OFF, only the tools/mtmd subdir is added. Useful
for any binding that wants just libmtmd (Apple XCFramework, WASM).
- tools/mtmd/CMakeLists.txt: gate the CLI exe targets on
LLAMA_BUILD_TOOLS. Gating on LLAMA_BUILD_COMMON is not enough — it
defaults ON in standalone builds and visionOS xcodebuild then fails
with "install TARGETS given no BUNDLE DESTINATION for MACOSX_BUNDLE
executable target 'llama-mtmd-cli'".
- build-xcframework.sh: turn the option on, pass -DLLAMA_BUILD_MTMD,
add libmtmd.a to combine_static_libraries, and copy mtmd.h and
mtmd-helper.h into the framework Headers dir. The umbrella module
map then exposes them, so Swift / Obj-C consumers can import the
mtmd C API directly.
After this, nm on ios-arm64/llama.framework/llama shows 52 _mtmd_
symbols. Verified end-to-end: a Swift target links the produced
framework and calls mtmd_default_marker, mtmd_bitmap_init, etc.
without a shim on macos / iphoneos / iphonesimulator / xros slices.
Co-authored-by: Abraham Gonzalez <abraham@theabecaster.com >
2026-06-25 08:37:30 +03:00
09cedfd699
chat: harden caps check ( #24973 )
2026-06-25 02:49:22 +02:00
894bb27af3
mtmd: model: unlimited-ocr: converter + parity test ( #24969 )
2026-06-24 18:20:22 +02:00
579229d157
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# CODEOWNERS
# README.md
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q8_0_f32.cl
# ggml/src/ggml-sycl/binbcast.cpp
# ggml/src/ggml-sycl/element_wise.cpp
# ggml/src/ggml-vulkan/CMakeLists.txt
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_id_vec.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_q_acc.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/quantize_q8.wgsl
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tests/test-sampling.cpp
# tools/server/README.md
2026-06-24 23:28:21 +08:00
00139b660b
ui: loading bar below the model picker ( #24931 )
...
* ui: show model load progress on the selector trigger
Mirror the in-dropdown stage progress as a thin bar on the selector
trigger, so the active model's load percent stays visible when the menu
is closed. Same status gating and composite fraction as the dropdown
row, so both bars track the selected model in sync.
Suggested-by: Julien Chaumond <@julien-c>
* ui: show model load progress bar on the in-conversation model selector
* ui: tune model load indicator to a pulsing highlight (suggested by @ngxson)
Also wire the indicator onto the mobile sheet trigger, which was missing
it since mobile uses the sheet instead of the dropdown.
* ui: thin (@allozaur) pulsating (@ngxson) model load bar
2026-06-24 10:50:44 +02:00
ef9c13d4c2
ui: New Logo + Navigation cleanup & Mobile UI/UX improvements ( #24897 )
...
* chore: `npm audit fix --force`
* feat: Update sidebar toggle to use Logo
* refactor: Clean up favicon SVG
* feat: Refactor logo component and implement theme-aware favicon generation
* feat: Add configurable padding to generated PWA assets
* test: Add unit tests for writeThemeFavicons
* refactor: Componentization
* feat: WIP
* feat: WIP
* feat: WIP
* feat: Mobile UI
* feat: add SEARCH route constant
* feat: create SidebarNavigationSearchResults component
* refactor: use SidebarNavigationSearchResults in conversation list
* feat: enable mobile search navigation in sidebar actions
* feat: add mobile search route and page
* fix: prevent sidebar overflow on mobile viewports
* fix: Mobile sidebar
* feat: Mobile Search WIP
* feat: Mobile WIP
* feat: Add PWA standalone detection and refine mobile UI
* feat: Improve mobile layout, sidebar handling, and chat scrolling
* feat: Improve mobile sidebar visibility and iOS Safari chat spacing
* fix: Disable auto-scroll on mobile
* chore: Linting
* fix: Wrong condition
* feat: Mobile chat scroll
* refactor: WIP
* fix: Desktop initial scroll always working again
* fix: Partial fix for mobile auto-scroll / initial scroll
* fix: Desktop auto-scroll on initial load and during streaming
* fix: Mobile scrolling logic
* refactor: Clean up
* feat: Improve start UI
* feat: Add `delay` to `fadeInView`
* feat: Auto-scroll button
* refactor: Cleanup
* refactor: Extract chat dialogs and alerts into dedicated component
* refactor: Reorganize ChatScreen component structure and initialization
* feat: Improve auto-scroll after sending message
* feat: UI improvements
* fix: Settings link
* feat: UI improvements
* fix: better UI spacing
* fix: Remove unneeded logic
* fix: Chat Processing Info UI rendering
* feat: Improve mobile UI
* feat: UI improvement
* fix: Conditional transition delay for Chat Messages based on route from
* fix: Delay mobile sidebar collapse for smoother transitions
* fix: Mobile scroll down button + sidebar pointer events
* fix: Mobile UI
* fix: Auto scrolling
* fix: Implement dynamic height calculations for chat auto-scroll positioning and UI elements
* fix: Retrieve `autofocus` for Chat Form textarea
* fix: Use proper class
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
* refactor: extract scroll-to-bottom logic and fix message send flow
* fix: update viewport store usage and remove conflicting autofocus
* feat: add accessibility labels to scroll down button
* fix: correct HTML structure in sidebar empty states
* fix: dynamically toggle processing info visibility
* chore: remove commented exports and fix formatting
* fix
* fix: Mobile Chat Form Add Action Sheet interactions
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-06-24 10:21:33 +02:00
be4a6a63eb
server : check draft context creation error ( #24922 )
2026-06-23 16:56:50 +02:00
75ad0b23ed
server: fix remote preset handling, add test ( #24938 )
...
* server: add test for remote preset
* fix remote preset handling
* fix
* fix test
2026-06-23 13:28:34 +02:00
a3900a6694
model: Granite Speech Plus ( #24818 )
...
* feat: Add conversion support for Granite Speech Plus
Branch: GraniteSpeechPlus
AI-usage: full (Bob, OpenCode + Qwen3.6-35b)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
* feat: Extend granite_speech to support plus multi-layer concatenation
Branch: GraniteSpeechPlus
AI-usage: draft (Bob, OpenCode + Qwen3.6-35b)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
* fix(conversion): Fix plural naming for feature_layers for audio
Branch: GraniteSpeechPlus
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
* fix(mtmd): Align feature_layer usage and naming everywhere
Branch: GraniteSpeechPlus
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
* style: Use fstring for log
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com >
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com >
---------
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com >
2026-06-23 12:03:31 +02:00
73618f27a8
server: improve user message detection and create checkpoints at every user message ( #24176 )
...
* server : improve message span logic
* cont : cast size_t to int32_t in comparisons
* server : create checkpoints before every user msg
* chat : remove \n in gemma4 delimiters
* chat : merge msg delimiter structs into one
* cont : reword comment
* cont : initialize tokens in delimiter
* cont : add server_tokens::get_raw_tokens() for mtmd
* cont : move message finding to server_tokens and skip mtmd tokens
* cont : update cohere2moe parser
* cont : increase min-step to 8192 and always produce a chkpt for last user message
2026-06-23 08:27:28 +03:00
dec5ca5577
server : Add id to tool call responses api ( #24882 )
2026-06-22 23:03:12 +02:00
9c0ac887f3
ui: Prioritize favorite models in model selection ( #24766 )
...
Updated model selection prioritization to include favorite models.
2026-06-22 21:00:21 +02:00
721354fbdf
server: (router) move model downloading to dedicated process ( #24834 )
...
* server: real-time model load progress tracking via /models/sse
* update docs
* server: move model download to child process
* rm unused
* fix most problems
* clean up
* nit fixes
* fix test case
* do not detact() thread
* shorter MODEL_DOWNLOAD_TIMEOUT in test
* throttle
2026-06-22 18:24:04 +02:00
6ee0f65793
server: refactor/generalize input file schema ( #24299 )
...
* server: refactor/generalize input file schema
* wire up input_video, accept raw base64
* nits
* nits (2)
* fix windows
2026-06-22 16:42:47 +02:00
099b579acb
ui: model status and load progress via /models/sse feed ( #24878 )
...
* ui: model status and load progress via /models/sse feed
* ui: centralize SSE wire-format delimiters into shared constants for the chat and /models/sse parsers
* ui: type /models/sse event names as a ServerModelsSseEventType enum
Address review from allozaur
2026-06-22 15:55:30 +02:00
3090ae0bf7
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/s390x.Dockerfile
# .dockerignore
# .github/workflows/docker.yml
# .github/workflows/release.yml
# docs/android.md
# ggml/src/ggml-cpu/amx/mmq.cpp
# ggml/src/ggml-hexagon/htp/ssm-conv.c
# tests/peg-parser/test-gbnf-generation.cpp
# tests/test-arg-parser.cpp
# tests/test-chat.cpp
# tests/test-jinja.cpp
# tests/test-json-schema-to-grammar.cpp
# tools/server/README.md
2026-06-22 18:23:59 +08:00
d0f9d2e5ac
server: fix edit_file crash on append at end of file (line_start -1) ( #24893 )
...
line_start -1 normalized to n+1, so append inserted at lines.begin() + n + 1,
one past end() -> heap-buffer-overflow in vector::_M_range_insert.
Normalize -1 to n (insert at end()), restrict -1 to append mode and reject it
for replace/delete instead of silently clobbering the last line. Parenthesize
the insert offset so empty-file append computes the position as int first,
avoiding a transient begin() - 1 on a null vector data pointer.
2026-06-22 10:55:28 +02:00
dfa1c573c4
fix router mode on lcpp ui
2026-06-22 00:21:19 +08:00
7c082bc417
server: fix report progress for loading spec models, add "stages" list ( #24870 )
...
* server: fix report progress for loading spec models, add "stages" list
* improve
* nits
* nits 2
2026-06-21 17:36:52 +02:00
a072dd8304
llama-ui update ( #2281 )
2026-06-21 23:02:16 +08:00
bddfd2b113
server: refactor batch construction ( #24843 )
...
* server: refactor batch construction
* wip
* wip 2
* wip 3
* wip 4
* add abort_all_slots
* handle batch full more carefully
* fix assert
* rm debug log
* small nits
* (debug) add timings
* debug: force llama_synchronize for accurate timings
* address comments
* disable DEBUG_TIMINGS
2026-06-21 14:16:11 +02:00
0d135df48c
mtmd: fix mtmd_get_memory_usage ( #24867 )
2026-06-21 14:12:15 +02:00
2f89acc2bc
mtmd: add load progress callback ( #24865 )
2026-06-21 13:40:52 +02:00
bfa3219177
server: add "verbose" field to schema ( #24864 )
2026-06-21 13:03:14 +02:00
d6d899580d
server: real-time model load progress tracking via /models/sse ( #24828 )
...
* server: real-time model load progress tracking via /models/sse
* update docs
* add mutex for notify_to_router
* correct docs
2026-06-21 11:58:14 +02:00
063d9c156e
common/peg : refactor until gbnf grammar generation ( #24839 )
...
* common/peg : refactor until gbnf grammar into an ac automaton
* cont : add a test with multiple strings
* cont : pad state with 0s so rules line up
* cont : clean up comments
* cont : use set everywhere
* cont : inline state num string padding
* cont : add a ref to PR
* cont : fix regression in server-tools.cpp
2026-06-20 21:15:06 -05:00
e27f308597
server: avoid forwarding auth headers in CORS proxy ( #24373 )
...
* server: avoid forwarding auth headers in CORS proxy
* format
* fix test
* fix e2e test
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co >
2026-06-20 15:34:47 +02:00
73cc7d9287
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/openvino.Dockerfile
# .devops/rocm.Dockerfile
# .devops/s390x.Dockerfile
# .devops/vulkan.Dockerfile
# .devops/zendnn.Dockerfile
# .dockerignore
# .pi/gg/SYSTEM.md
# ggml/CMakeLists.txt
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# scripts/sync-ggml.last
# scripts/sync_vendor.py
# tools/cli/README.md
# tools/cli/cli.cpp
# tools/mtmd/clip.cpp
# tools/server/README.md
2026-06-20 17:16:19 +08:00
84b8856295
Merge commit '32eddaf2ea8dd5d499dee9655592a89b91bfde9d' into concedo_experimental
...
# Conflicts:
# docs/multimodal.md
# docs/preset.md
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hex-dma.h
# ggml/src/ggml-hexagon/htp/hex-utils.h
# ggml/src/ggml-hexagon/htp/hmx-flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/hmx-matmul-ops.c
# ggml/src/ggml-hexagon/htp/hmx-queue.c
# ggml/src/ggml-hexagon/htp/hmx-queue.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# scripts/snapdragon/ggml-hexagon-profile.py
# scripts/ui-assets.cmake
# tools/export-lora/README.md
# tools/server/CMakeLists.txt
2026-06-20 11:18:24 +08:00
2b686a9120
server: refactor child --> router communication ( #24821 )
...
* server: refactor child --> router communication
* fix wakeup case
* add docs
* improve update_status()
* nits
2026-06-20 01:02:26 +02:00
4b48a53b6c
server : optimize get_token_probabilities ( #24796 )
...
Use std::partial_sort to order only the requested top-n tokens instead
of the full vocabulary
logprobs sort: vocab=128000 n_top=0 iters=100
full sort: 8555.6 us/op
partial sort: 704.3 us/op
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-06-19 23:26:54 +02:00
e475fa2b5f
mtmd, arg: fix utf8 handling on windows ( #24779 )
...
* mtmd, arg: fix utf8 handling on windows
* also fix ggml_fopen
* fix build fail
* also fix CLI
2026-06-19 22:28:38 +02:00
175147e8f6
server: remove all internal mentions about "webui" ( #24817 )
2026-06-19 22:12:46 +02:00
fabde3bf51
arg: Add comment line support to --api-key-file ( #23168 )
2026-06-19 17:33:54 +02:00
8c2d6f6475
server: add --agent arg, remove redundant webui naming compat ( #24801 )
...
* server: add --agent arg, remove redundant webui naming compat
* corrent env
* fix the test
* llama-gen-docs
* nits: wordings
2026-06-19 16:06:13 +02:00
e2e7a9b2d0
mtmd: several bug fixes ( #24784 )
...
* mtmd: several bug fixes
* fix build
* fix gemma4ua
* add sanity check in get_u32()
* fix build (2)
* area() avoid overflow
2026-06-19 12:18:36 +02:00
b14e3fb90c
spec: support eagle3 for qwen3.5 & 3.6 ( #24593 )
...
* spec: support qwen3.5 & 3.6 eagle3 draft
* eagle3: Add deferred boundary checkpoints restore support for hybrid models
* apply suggestions
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* spec: adapt to API change
* spec: fix naming
* cont : add TODO
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-06-19 13:08:50 +03:00
159d093a43
server: fix non-bound n_discard value (ctx shifting) ( #24786 )
...
* server: fix non-bound n_discard value
* Update tools/server/server-context.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-06-19 10:53:44 +02:00
80452d65b9
server : consolidate slot selection into get_available_slot ( #24755 )
...
Absorb get_slot_by_id logic into get_available_slot so slot selection
is handled by a single function call. When a specific slot id is
requested, the LCP similarity check still runs to enable proper
prompt cache updates.
Assisted-by: pi:llama.cpp/Qwen3.6-27B
2026-06-19 09:22:34 +03:00
db52540f73
mtmd: add batching support for internvl ( #24775 )
2026-06-19 01:16:16 +02:00
40f3aafc45
server: add "X-Accel-Buffering": "no" header to streaming endpoints ( #24774 )
...
* server: add "X-Accel-Buffering": "no" header to streaming endpoints
This header tells Nginx (as a reverse proxy) to NOT buffer responses. (only affects streaming endpoints)
Without it, Nginx will break streaming with certain applications (notably the Pi coding harness).
2026-06-18 22:01:24 +02:00
a6b3260a42
mtmd: add batching for mtmd-cli, add video tests ( #24778 )
2026-06-18 21:55:04 +02:00
060ce1bf72
mtmd: refactor llava-uhd overview image handling (always use ov_img_first) ( #24769 )
...
* add dedicated "overview" for mtmd_image_preproc_out
* corrections
* correct (again)
* nits
* nits (2)
2026-06-18 18:53:49 +02:00
7b6c5a2aed
docs: fix export-lora --lora-scaled syntax [no release] ( #24703 )
...
Assisted-by: Codex
2026-06-18 16:46:17 +02:00
fe7c8b2414
server: (router) fix stopping_thread potentially hang ( #24728 )
...
* server: (router) fix stopping_thread potentially hang
* fix windows build
2026-06-18 15:41:09 +02:00
e1efd0991d
server: add "schema" and validation ( #24150 )
...
* wip
* working
* correct some limits
* add field name to error message
2026-06-18 15:40:58 +02:00
Concedo
Xuan-Son Nguyen
Pascal
Abraham Gonzalez
Piotr Wilkin (ilintar)
Saba Fallah
Aleksander Grygier
kononnable
Gabe Goodhart
Aldehir Rojas
Matt Thompson
Mahdiou Diallo
henk717
Matti4
Adrien Gallouët
Mikolaj Kucharski
Ruixiang Wang
Georgi Gerganov
Reguna
Kangjia Gao