97340b4c99
Vulkan: Don't default to CPU device (like llvmpipe), even if no other device is available, to allow fallback to CPU backend ( #14099 )
2025-06-10 13:01:33 +01:00
2bb0467043
rpc : nicer error messages for RPC server crash ( #14076 )
2025-06-10 09:41:01 +03:00
b8e2194efc
sync : ggml
...
ggml-ci
2025-06-10 09:21:56 +03:00
1a3b5e80f7
Add in-build ggml::ggml ALIAS library (ggml/1260)
...
Enable uniform linking with subproject and with find_package.
2025-06-10 09:21:56 +03:00
1f63e75f3b
metal : use less stack memory in FA kernel ( #14088 )
...
* metal : use less stack memory in FA kernel
ggml-ci
* cont : fix BF16 variant
2025-06-09 23:05:02 +03:00
40cbf571c9
kv-cache : fix shift and defrag logic ( #14081 )
...
* kv-cache : fix shift
ggml-ci
* cont : reset shift[i]
ggml-ci
* cont : fix defrag erasing cells that didn't move
ggml-ci
2025-06-09 23:04:35 +03:00
7f4fbe5183
llama : allow building all tests on windows when not using shared libs ( #13980 )
...
* llama : allow building all tests on windows when not using shared libraries
* add static windows build to ci
* tests : enable debug logs for test-chat
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2025-06-09 20:03:09 +02:00
28b35ca879
allow wmma flag for rocm
2025-06-10 01:23:48 +08:00
7d8aa31f1f
fixed embeddings, added new parameter to limit max embeddings context
2025-06-10 01:11:55 +08:00
f470bc36be
ggml-cpu : split arch-specific implementations ( #13892 )
...
* move ggml-cpu-aarch64 to repack
* split quantize_row_q8_0/1
* split helper functions
* split ggml_vec_dot_q4_0_q8_0
* split ggml_vec_dot_q4_1_q8_1
* split ggml_vec_dot_q5_0_q8_0
* split ggml_vec_dot_q5_1_q8_1
* split ggml_vec_dot_q8_0_q8_0
* split ggml_vec_dot_tq1_0_q8_K
* split ggml_vec_dot_tq2_0_q8_K
* split ggml_vec_dot_q2_K_q8_K
* split ggml_vec_dot_q3_K_q8_K
* split ggml_vec_dot_q4_K_q8_K
* split ggml_vec_dot_q5_K_q8_K
* split ggml_vec_dot_q6_K_q8_K
* split ggml_vec_dot_iq2_xxs_q8_K
* split ggml_vec_dot_iq2_xs_q8_K
* split ggml_vec_dot_iq2_s_q8_K
* split ggml_vec_dot_iq3_xxs_q8_K
* split ggml_vec_dot_iq3_s_q8_K
* split ggml_vec_dot_iq1_s_q8_K
* split ggml_vec_dot_iq1_m_q8_K
* split ggml_vec_dot_iq4_nl_q8_0
* split ggml_vec_dot_iq4_xs_q8_K
* fix typos
* fix missing prototypes
* rename ggml-cpu-quants.c
* rename ggml-cpu-traits
* rename arm folder
* move cpu-feats-x86.cpp
* rename ggml-cpu-hbm
* update arm detection macro in quants.c
* move iq quant tables
* split ggml_quantize_mat_q8_0/K
* split ggml_gemv_*
* split ggml_gemm_*
* rename namespace aarch64 to repack
* use weak aliases to replace test macros
* rename GGML_CPU_AARCH64 to GGML_CPU_REPACK
* rename more aarch64 to repack
* clean up rebase leftover
* fix compilation errors
* remove trailing spaces
* try to fix clang compilation errors
* try to fix clang compilation errors again
* try to fix clang compilation errors, 3rd attempt
* try to fix clang compilation errors, 4th attempt
* try to fix clang compilation errors, 5th attempt
* try to fix clang compilation errors, 6th attempt
* try to fix clang compilation errors, 7th attempt
* try to fix clang compilation errors, 8th attempt
* try to fix clang compilation errors, 9th attempt
* more cleanup
* fix compilation errors
* fix apple targets
* fix a typo in arm version of ggml_vec_dot_q4_K_q8_K
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2025-06-09 16:47:13 +02:00
8f47e25f56
cuda : fix device sync on buffer clear ( #14033 )
2025-06-09 16:36:26 +02:00
201b31dc2e
graph : fix geglu ( #14077 )
...
ggml-ci
2025-06-09 17:17:31 +03:00
e21d2d4ae2
CANN: Simplify the environment variable setting( #13104 )
...
* Simplify the environment variable setting to specify the memory pool type.
* Adjust the GGML_CANN_ASYNC_MODE setting to accept yes, enable, 1, or on (case-insensitive) as valid options.
* update
* fix CI
* update
* delete whitespace
* fix according to review
* update CANN.md
* update CANN.md
2025-06-09 19:47:39 +08:00
dc0623fddb
webui: fix sidebar being covered by main content ( #14082 )
...
* webui: fix sidebar being covered by main content
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com >
* webui: update index.html.gz
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com >
---------
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com >
2025-06-09 12:01:17 +02:00
87d34b381d
server : fix LRU check ( #14079 )
...
ggml-ci
2025-06-09 12:57:58 +03:00
8780b33c64
consolidate imports
2025-06-09 17:48:54 +08:00
b460d16ae8
sycl: Add reorder to Q6_K mmvq implementation ( #13885 )
...
* Add Reorder to Q6_K mmvq implementation
* Address PR comments: clean up comments
* Remove unused parameter after refactoring q4_k
* Adding inline to function and removing unnecessary reference to int
---------
Signed-off-by: nscipione <nicolo.scipione@codeplay.com >
2025-06-09 11:47:07 +02:00
deece4be69
missed a build target
2025-06-09 17:05:56 +08:00
68ec00909b
updated lite (+1 squashed commits)
...
Squashed commits:
[375c5768b] updated lite
2025-06-09 16:33:42 +08:00
91a8ee6a6f
add geglu activation function ( #14074 )
...
Co-authored-by: dinhhuy <huy.dinh@brains-tech.co.jp >
2025-06-09 05:15:31 +01:00
056eb74534
CANN: Enable labeler for Ascend NPU ( #13914 )
2025-06-09 11:20:06 +08:00
247e5c6e44
cuda : fix buffer type check with integrated GPUs ( #14069 )
2025-06-08 11:39:56 -07:00
82d7c53b85
embeddings handle base64
2025-06-09 00:26:40 +08:00
7de88802f9
revert padding change for sd chroma
2025-06-08 23:48:46 +08:00
1cf7648305
fixed adapter
2025-06-08 23:24:11 +08:00
771bd7197b
updated lite (+1 squashed commits)
...
Squashed commits:
[907f10f2f] updated lite
2025-06-08 23:22:26 +08:00
6c5c8be48d
try to make rocm work for the github ci, requires disabling rocwmma
2025-06-08 21:52:29 +08:00
7f57846c2f
update bundled vcrts
2025-06-08 19:39:42 +08:00
2d4c1aa5a0
chroma support is now usable
2025-06-08 18:53:59 +08:00
30cf433ab4
merge base support for chroma, however its not working correctly
2025-06-08 18:06:23 +08:00
dcf88d6e78
Revert "make tts use gpu by default. use --ttscpu to disable"
...
This reverts commit 669f80265b
2025-06-08 17:08:04 +08:00
669f80265b
make tts use gpu by default. use --ttscpu to disable
2025-06-08 17:06:19 +08:00
7132d6b15c
test rocm rolling (+1 squashed commits)
...
Squashed commits:
[43c8f7fc6] test rocm rolling (+4 squashed commit)
Squashed commit:
[16a60aa77] test clobber 4
[a6c866450] test clobber 3
[9322f17f6] test clobber 2
[b7a420cbe] testing clobber
2025-06-08 15:33:05 +08:00
5d8f499f03
Remove 32GB of rocm dependencies with this one special trick ( #1585 )
...
* One file to remove them all
* That one lib wasn't versioned
2025-06-08 11:16:15 +08:00
a80dfa5c10
various minor fixes
2025-06-08 01:11:42 +08:00
5787b5da57
ci: add LoongArch cross-compile build ( #13944 )
2025-06-07 10:39:11 -03:00
228f34c9ce
SYCL: Implement few same quantized type copy kernels ( #13739 )
...
* SYCL: Implement few same quantized type copy kernels
* Use memcpy for copying contiguous tensors
ggml-ci
* feat(sycl): add contiguous tensor copy support and device checks
Adds a memcpy path for contiguous tensors of the same type to optimize data transfer. Updates device support checks to recognize contiguous tensor operations, improving compatibility and performance.
* refactor: replace specific block copy functions with template
The changes replace multiple redundant block copy functions (e.g., cpy_block_q8_0_q8_0, cpy_block_q5_0_q5_0) with a single templated function cpy_blck_q_q. This reduces code duplication by using a generic template that works for any block type, improving maintainability while preserving the same functionality. The template is instantiated with specific block types (e.g., block_q8_0) where needed.
* Exclude BF16 support for COPY tensors for now
ggml-ci
* perf: adjust SYCL copy kernel block sizes for efficiency
Use ceil_div to ensure full element coverage and update nd_range parameters to better align with SYCL block sizes, improving parallelism and device utilization in copy operations.
2025-06-07 18:58:20 +05:30
0974ad7a7c
llama : fix llama_model_chat_template with template name (LLM_KV with suffix) ( #14050 )
2025-06-07 14:13:12 +02:00
301450b1eb
attempt to use system glslc first before using bundled glslc
2025-06-07 16:54:25 +08:00
38ce7e06cc
updated readme
2025-06-07 10:23:41 +08:00
cfcdfd69bd
allow embeddings models to use mmap
2025-06-07 10:14:00 +08:00
abc272d89f
breaking change: standardize ci binary names
2025-06-07 00:40:46 +08:00
6effb65cfe
change singleinstance order
2025-06-06 21:20:30 +08:00
745aa5319b
llama : deprecate llama_kv_self_ API ( #14030 )
...
* llama : deprecate llama_kv_self_ API
ggml-ci
* llama : allow llama_memory_(nullptr)
ggml-ci
* memory : add flag for optional data clear in llama_memory_clear
ggml-ci
2025-06-06 14:11:15 +03:00
487a5e0401
context : fix SWA-related warning for multiple sequences ( #14045 )
2025-06-06 13:29:18 +03:00
d18938fc70
fixed build
2025-06-06 18:05:44 +08:00
d33c88b1f4
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# README.md
# ci/run.sh
# examples/embedding/embedding.cpp
# ggml/CMakeLists.txt
# ggml/src/CMakeLists.txt
# src/CMakeLists.txt
2025-06-06 17:56:51 +08:00
2b5d8e467b
updated lite
2025-06-06 17:49:56 +08:00
740f91e3fd
lower aria interval
2025-06-06 17:43:38 +08:00
8b141d8647
stick to cu12.1 for linux for now
2025-06-06 17:38:28 +08:00
0cc4m
Isaac McFadyen
Georgi Gerganov
Kai Pastor
Diego Devesa
Concedo
xctan
Xinpeng Dou
R0CKSTAR
Nicolò Scipione
Đinh Trọng Huy
Yuanhao Ji
henk717
吴小白
Akarshan Biswas
Sigbjørn Skjæret