e8067a8b36
ui: build-time gzip compression ( #24571 )
...
* ui: keep original file name and path
* fix nocache
* ui: build-time gzip compression
2026-06-13 16:57:27 +02:00
341babcf73
jinja : fix split and replace with empty first arg ( #24574 )
...
* fix split and replace with empty first arg
* fix reserve size
2026-06-13 16:56:59 +02:00
1a7718b4c5
vulkan: support non-contig unary/glu ops ( #24215 )
...
* vulkan: support non-contig unary/glu ops
Change unary/glu ops to pass in all strides and use fastdiv for the index
calculation. Put all unary ops in one file, similar to glu, to share the
code. codex went ahead and added expm1 without me asking, but I had to
make it do a real precision analysis rather than just making stuff up.
unary.comp initially couldn't use generic_unary_head because there wasn't
space for xielu's additional constants. Fixing this required packing the
fastdiv 'L' values.
* attempt to workaround compiler bug
* resolve conflict from #23991
* use expm1
2026-06-13 08:44:15 -05:00
597b6672e8
ui: keep original file name and path ( #24568 )
...
* ui: keep original file name and path
* fix nocache
2026-06-13 14:31:41 +02:00
57fe1f07c3
server: clean up static assets handling ( #24550 )
...
* server: clean up static assets handling
* nits
* simplify file name handling, use static file name everywhere
* cmake/ui : bundle UI assets in an archive
* ui : run prettier on post-build.js
---------
Co-authored-by: Alde Rojas <hello@alde.dev >
2026-06-13 11:51:20 +02:00
af96c9001d
temporarily disable stream layers
2026-06-13 15:30:00 +08:00
d8a24ccee2
fit : wrap llama_device_memory_data ( #24522 )
2026-06-13 08:09:52 +03:00
0a0ddc212c
fixed build
2026-06-13 11:27:05 +08:00
fdc9c96124
sd: sync with master-685-19bdfe2 ( #2265 )
...
* sd: sync with master-682-b3d56d0
* sd: sync with master-685-19bdfe2
* sd: enable layer streaming
2026-06-13 11:14:44 +08:00
45ce7950f1
suppress power throttling error
2026-06-13 11:11:28 +08:00
c34b92235b
fix sycl links in release notes ( #24527 )
...
* fix sycl links in release notes
* remove extra line
2026-06-13 08:37:55 +08:00
e37abd6b5f
mtmd: add batching API ( #24384 )
...
* mtmd: add batching API
* wip
* first working version (gemma4v)
* add arg
* nits
* wire up support_batch()
* fix 0.0 output embd
* fix audio
* nits
* refactor a bit
* nits
* fix non-batching case
* fix comment
2026-06-13 00:10:29 +02:00
f58bad4137
ci : unbreak release harder ( #24545 )
...
* unbreak release harder
* missed one
* remove missing test for now
2026-06-12 23:49:36 +02:00
cd5044661c
ci : unbreak release ( #24544 )
2026-06-12 23:29:49 +03:00
ebc10770ac
server : fix reasoning budget WebUI precedence over model.ini ( #24517 )
...
When reasoning-budget is set in model.ini, the per-request
thinking_budget_tokens from the WebUI was ignored because the
model.ini value took unconditional precedence.
Swap the precedence so the WebUI per-request value is checked
first, with the model.ini value serving as a fallback default.
Assisted-by: pi:llama.cpp/Qwen3.6-27B
2026-06-12 17:59:56 +03:00
3e7bd4f39a
vulkan: add pipeline barriers for memcpy read operations ( #23770 )
...
* vulkan: add pipeline barriers for memcpy read/write operations
* remove unnecessary host write pipeline barriers
2026-06-12 16:43:50 +02:00
f7ca93d12c
ui: PWA support ( #23871 )
...
* feat: Add basic PWA support and service worker for offline caching
* feat: Vite PWA implementation WIP
* feat: Improve PWA icons generation
* feat: Add PWA workbox to server routes
* feat: Include `version.json` in static assets
* feat: Add HTTP cache headers for PWA static assets
* feat: Update app name for `apple-mobile-web-app-title`
* feat: Implement PWA versioning and automatic update detection
* chore: Update `.gitignore` files
* feat: Splash Screens
* feat: Add dark mode favicon support
* refactor: Cleanup
* fix: Use dark logo for dark splash screens
* refactor: Simplify favicons SVG code
* fix: Adjust caching and polling for reliable service worker updates
* fix: Add missing favicon entry
* fix: Align PWA service worker configuration with SvelteKit build structure
* fix: Replace hashed bundle paths with versioned static paths
* test: Add PWA tests
* ci: Add build output for unit tests
* refactor: Cleanup
* fix: Server build & release versioning
* chore: Update package-lock.json
* chore: Increase PWA cache size
* chore: Update packages
* feat: Update favicons
* refactor: Post-merge fix
* feat: support explicit build version for PWA cache busting
* fix: CI
* feat: Improve PWA Refresh Alert UI
* feat: Add toggleable build version display
* refactor: Cleanup
* feat: Add version mismatch detection and manual app reload
* refactor: replace dynamic imports with static
* refactor: Cleanup
* feat: Add safe space for `pwa-<size>.png` rendered icons
* fix: use relative paths for PWA assets to support base path deployment
* feat: add PWA mode detection via URL query parameter
* feat: Use ?cache=true for SW-cached PWA assets
* refactor: Build process cleanup
* refactor: Decouple PWA versioning and remove ?cache=true workaround
* chore: Update README logo
* feat: Include PWA Assets generation in build script
* refactor: `usePwa` hook for core layout
* fix: Relativize base vite plugin
* fix: remove unnecessary backslash escapes in test regexes
* test: update static asset paths for API Key test
* refactor: Move SvelteKit PWA Options config to constants
* ui: fix update notification never appearing
Keep the PWA hook object intact instead of destructuring needRefreshByStorage,
which freezes the reactive getter. Also exclude loading.html from PWA
precache to prevent 404 errors and broken SW installation.
2026-06-12 15:53:26 +02:00
02182fc5b9
fit : avoid including llama-ext.h in fit.h ( #24506 )
2026-06-12 15:57:05 +03:00
f532be8fac
sync : ggml
2026-06-12 15:55:35 +03:00
e08c226a2c
ggml : bump version to 0.15.1 (ggml/1541)
2026-06-12 15:55:35 +03:00
70b54e140c
vendor : update cpp-httplib to 0.47.0 ( #24395 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-06-12 11:34:44 +02:00
104b41091b
refactor drafting
2026-06-12 16:48:00 +08:00
6471e3c090
UI/jpeg exif orientation ( #24196 )
...
* ui: bake jpeg exif orientation into uploaded images
stb_image in mtmd ignores exif metadata, so rotated smartphone photos
reach the model with raw pixel orientation. The webui now reads the
exif orientation tag at send time and feeds it into the existing
capImageDataURLSize canvas pass: the browser applies the rotation when
decoding, so capped images come out upright for free, and images under
the cap threshold get a single plain redraw when orientation > 1.
At most one re-encode ever happens per image. Upright jpegs with
capping disabled pass through untouched, bit perfect.
Adds jpeg-orientation.ts with a minimal exif parser working on a
bounded base64 prefix (both endianness, returns 1 on any malformed
input) and unit tests against handcrafted jpeg byte streams.
* ui: move jpeg exif constants into lib/constants
* ui: add browser test for jpeg orientation and capping
Covers capImageDataURLSize end to end in chromium with real Pillow
generated jpeg fixtures across exif orientations 1/3/5/6/8: upright
quadrant colors checked pixel-wise, expected dimensions with and
without capping, no orientation tag left in the output, and strict
passthrough when nothing needs rewriting.
2026-06-12 10:20:27 +02:00
be1750a3b8
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cuda.Dockerfile
# .github/workflows/build-sycl.yml
# .github/workflows/release.yml
# ggml/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/gated-delta-net-ops.c
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# ggml/src/ggml-opencl/kernels/gated_delta_net.cl
# ggml/src/ggml-sycl/gated_delta_net.cpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/gated_delta_net.wgsl
# scripts/sync-ggml.last
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
2026-06-12 15:39:57 +08:00
88a39274ec
spec: add EAGLE3 speculative decoding support ( #18039 )
...
* llama : enable layer input extraction
* spec: support eagle3
* eagle3: fix params bug
* eagle3: support Gemma4 eagle3 from RedHatAI
* eagle3: set sync when get features from target
Co-authored-by: tnhnyzc <115956684+tnhnyzc@users.noreply.github.com >
* eagle3 : fix ubatch handling in embd_layer_inp extraction and encoder
Co-authored-by: Doğaç Eldenk <dogacel@gmail.com >
* eagle3: adapt to upstream changes
* eagle3: fix rebase issues and adapt to upstream changes
* eagle3:exclude the eagle3 arch from test-llama-archs
* eagle3: fix editorconfig check failures
* eagle3: fix multi-seq issue in d2t vocab mapping
* cont : minor style / clean-up
* spec : remove `common_speculative_setup_draft_model()`
* llama : clean-up unused API
* eagle3: set d2t vocab mapping in decode graph
* cont : assert layer inputs are configured
* hparams : use n_embd_inp instead of n_embd_target_features
* eagle3: make output.weight optional and inherit from target model when needed
* haparams : generic norm-before-residual param
* llama-ext : consistent names
* cont : fix
* hparams : remove target_hidden_size
* cparams : rename output_layer_inp -> embeddings_layer_inp
* arch : reuse ATTN_NORM_2 instead of adding new hidden norm
* llama : clean-up names
* cont : add assert + comment
* Update conversion/llama.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
Co-authored-by: tnhnyzc <115956684+tnhnyzc@users.noreply.github.com >
Co-authored-by: Doğaç Eldenk <dogacel@gmail.com >
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-06-12 10:21:06 +03:00
3424543568
skip dot2 vulkan shaders for noext build
2026-06-12 15:20:41 +08:00
85f99dca8b
ggml: support concat for scalar types at cuda backend ( #24011 )
...
* cuda: support concat for scalar types
* Update concat.cu
* fix metal ci issue
2026-06-12 09:32:44 +03:00
099ea76fb4
[SYCL] Fix CI build & release for SYCL backend ( #24387 )
...
* restore SYCL build and release, remove github cache
* modify for test only
* verify the ccache is used
* remove debug code change
* rm duplicate action, update key in ccache
* add action ccache-clear after building in both ubuntu and windows
* set %NUMBER_OF_PROCESSORS% in widnows build
2026-06-12 09:30:24 +03:00
ba1df050f3
opencl: add q5_0/q5_1 gemm and gemv kernels for Adreno ( #24319 )
...
* opencl: add q5_0 adreno support
* opencl: add q5_1 adreno support
* opencl: cosmetic fix
---------
Co-authored-by: Li He <lih@qti.qualcomm.com >
2026-06-11 21:43:09 -07:00
1593d5684d
docker : support specifying the GCC version for CUDA ( #24447 )
2026-06-11 23:12:09 +02:00
4c6595503f
vulkan: ifdef eMesaHoneykrisp (build fix) ( #24479 )
...
Fixes build/CI after #24306 .
2026-06-11 13:22:17 -05:00
263cc04a54
sync : ggml
2026-06-11 19:34:19 +03:00
17e59d6209
ggml : bump version to 0.15.0 (ggml/1539)
2026-06-11 19:34:19 +03:00
60d0e70d76
RNN lifeboat implementation
2026-06-11 23:40:43 +08:00
7d1147bc35
smartcache improvements
2026-06-11 22:48:37 +08:00
79331fd382
fixed mtp for qwen
2026-06-11 22:04:31 +08:00
fdc3db9b65
vulkan: add fast path for contiguous buffer transfers ( #23973 )
2026-06-11 15:46:25 +02:00
1af154a76f
vulkan: use medium matmul tile on Asahi Linux ( #24306 )
...
* vulkan: use medium matmul tile on Asahi Linux
* vulkan: switch Apple detection to Honeykrisp driver id
2026-06-11 15:43:04 +02:00
18ef86ecec
server: skip unused log lines on router mode ( #24463 )
2026-06-11 11:36:35 +02:00
1bfbdb134e
vocab : adopt leading TemplateProcessing special token as BOS ( #24428 )
2026-06-11 10:37:23 +03:00
68f30663cf
vocab : refactor normalizer flags into options struct, add strip_accents ( #24371 )
...
* vocab : refactor normalizer flags into options struct, add strip_accents
* Update src/llama-vocab.h
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update src/llama-vocab.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-06-11 10:36:50 +03:00
40084bb1f8
mtp drafting added (+1 squashed commits)
...
Squashed commits:
[b0fa1a6de] wip 1
2026-06-11 15:25:49 +08:00
db94854ff5
server : skip checkpoints beyond pos_next ( #24411 )
...
* server : skip checkpoints beyond pos_next
* cont : update comment + TODO + ref
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2026-06-11 10:18:12 +03:00
10e4b6d5e3
support gemma assistant as a draft model
2026-06-11 12:16:43 +08:00
ac4cddeb0d
vendor : update LibreSSL to 4.3.2 ( #24397 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co >
2026-06-10 22:28:03 +02:00
e95dae18d6
Remove padding and multiple D2D copies for MTP ( #24086 )
...
* Make ggml_gated_delta_net take only the initial recurrent state (D, 1, n_seqs) and passes the snapshot count K as an op parameter instead of inferring it from state->ne[1].
Remove the padding hack and copy all emitted snapshots into the recurrent cache with a single strided ggml_cpy
* Make GDN changes in all backends. Address review comments.
* Fix CI build errors
2026-06-10 23:21:16 +05:30
a860ec0b37
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/winget.yml
2026-06-10 23:16:06 +08:00
d2462f8f7a
chat: fix LFM2/LFM2.5 ignoring json_schema ( #24377 )
...
The LFM2 specialized template handler only built a grammar for tool-calling,
silently ignoring json_schema from response_format.
2026-06-10 14:41:41 +02:00
fb83cc9a07
CUDA: Fix ssm_scan_f32 data-races ( #24360 )
...
* Add missing syncthreads before resuing cub_temp_storage
__syncthreads() is required before being allowed to resue TempStorage
smem:
https://nvidia.github.io/cccl/unstable/cub/api/classcub_1_1BlockLoad.html#_CPPv4I0EN3cub9BlockLoad4LoadEv20RandomAccessIteratorRA14ItemsPerThread_1Ti
* Add one more missing __syncthreads
Could also double-buffer, but alternative is to simply ensure all
threads have read smem* before writing to it again in the next loop
iteration
* Remove unused smem from ssm_scan_f32
2026-06-10 14:27:08 +02:00
e8ead6efaa
increase ctx and gen amt defaults
2026-06-10 18:00:26 +08:00
Xuan-Son Nguyen
Sigbjørn Skjæret
Jeff Bolz
Concedo
Georgi Gerganov
Wagner Bruna
Muhammad Salem
Ruben Ortlam
Aleksander Grygier
Adrien Gallouët
Pascal
Ruixiang Wang
ZihaoMu
Neo Zhang
shaofeiqi
wencan
Winston Ma
Kevin Liu
o7si
Aldehir Rojas
Gaurav Garg
Tarek Dakhran
Oliver Simons