300e20be6c
allow termux to launch existing downloaded models
2025-08-10 21:29:51 +08:00
8e6d27f629
handle if assistant_message_gen and assistant_message_gen!=assistant_message_start, replace final output tag with unspaced (gen) version if exists
2025-08-10 16:51:34 +08:00
204739e7f1
Adapter fixes ( #1659 )
...
* test adapters
* add assistant_gen adapter key
* add support for chat templates stored as .jinja files
* removed mistakenly commited gated-tokenizers link
* autoguess: Harmony: add missing newline prefixes to system_end
2025-08-10 16:19:50 +08:00
57db0ce9cd
allow uploading tagged pinned versions for rocm
2025-08-10 11:04:49 +08:00
1515d67c2c
oldpc build is now fixed (+2 squashed commit)
...
Squashed commit:
[d11ac6cef] temp test
[cfbc008b1] test no f16 as well
2025-08-10 10:52:45 +08:00
89266ac6b8
autoguess adapter make case insensitive
2025-08-10 00:58:47 +08:00
487d509b44
try fix oldpc cuda broken without flash attn since upstream pr14361 between 1.94 and 1.95 (+1 squashed commits)
...
Squashed commits:
[940f0c639] try fix oldpc cuda broken without flash attn since upstream pr14361 between 1.94 and 1.95
2025-08-10 00:10:37 +08:00
4c1faf61b2
increment version (+1 squashed commits)
...
Squashed commits:
[6e5080ad2] increment version
2025-08-09 20:53:26 +08:00
0fb25bb165
Merge branch 'upstream' into concedo_experimental
2025-08-09 20:31:36 +08:00
5f95fc1122
update lite
2025-08-09 20:31:15 +08:00
34c9d765bf
CUDA: add attention sinks for tile and wmma ( #15178 )
...
* CUDA: add attention sinks for tile and wmma
* Review: formatting changes + remove syncthreads from tile + remove warp_reduce_max from wmma
2025-08-09 20:00:24 +08:00
ced98823a1
kai api tool calling
2025-08-09 10:51:10 +08:00
4c7b82e982
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# scripts/server-bench.py
2025-08-09 10:34:24 +08:00
fc551470d4
updated lite
2025-08-09 10:33:59 +08:00
e54d41befc
gguf-py : add Numpy MXFP4 de/quantization support ( #15111 )
...
Python Type-Check / pyright type-check (push) Successful in 58s
* gguf-py : add MXFP4 de/quantization support
* ggml-quants : handle zero amax for MXFP4
2025-08-08 17:48:26 -04:00
4850b52aed
server-bench: external OAI servers, sqlite ( #15179 )
...
* server-bench: external OAI servers, sqlite
* Update scripts/server-bench.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update scripts/server-bench.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update scripts/server-bench.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* raise_for_status
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2025-08-08 23:04:36 +02:00
9e7a940ce4
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/softmax_4_f16.cl
# ggml/src/ggml-opencl/kernels/softmax_4_f32.cl
# ggml/src/ggml-opencl/kernels/softmax_f16.cl
# ggml/src/ggml-opencl/kernels/softmax_f32.cl
# ggml/src/ggml-rpc/ggml-rpc.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
2025-08-09 01:24:52 +08:00
7087aeb4bc
anti bsod only for nvidia
2025-08-09 01:23:38 +08:00
67e0072245
fixed clblast repacking
2025-08-09 01:08:02 +08:00
3468c2834d
fixed adv mode
2025-08-08 22:26:36 +08:00
866cc346ab
tweak OpenAI Harmony autoguess developer prefix and assistant end token ( #1673 )
...
* tweak OpenAI Harmony autoguess developer prefix
* use <|end|> for adapter end
2025-08-08 21:15:11 +08:00
cd6983d56d
ggml : fix field name when new ggml_backend ( #14944 )
2025-08-08 14:37:22 +02:00
6c7e9a5440
vendor: sync minja ( #15161 )
...
* vendor: sync minja
* Update minja.hpp
* Apply suggestions from code review
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2025-08-08 10:45:18 +01:00
1425f587a8
CUDA: attention sinks for mma FlashAttention ( #15157 )
2025-08-08 08:19:58 +02:00
aaa3d07ae7
opencl: support sink in soft_max (attn sinks) ( #15152 )
2025-08-07 21:47:03 -07:00
d5b5e79035
should fix vulkan bsod
2025-08-08 10:57:50 +08:00
eed5577aaa
fix unintended sd model quantization ( #1672 )
...
The recent ggml update added another quant type, GGML_TYPE_MXFP4,
which got the same value as SD_TYPE_COUNT. That made the embedded
sd.cpp quantize to GGML_TYPE_MXFP4 by default.
Photomaker in particular ends up crashing due to
"Missing CPY op for types: f32 mxfp4".
2025-08-08 10:19:58 +08:00
50aa938901
convert : support non-mxfp4 HF model ( #15153 )
...
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Successful in 55s
Python check requirements.txt / check-requirements (push) Failing after 13s
Python Type-Check / pyright type-check (push) Successful in 54s
* convert : support non-mxfp4 HF model
* rm redundant check
* disable debug check
2025-08-07 23:26:03 +02:00
c4f53563df
vulkan: support fattn sinks ( #15126 )
2025-08-07 22:44:20 +02:00
a0552c8bee
vulkan: Add env var to disable host visible vidmem ( #15109 )
2025-08-07 22:07:11 +02:00
99acbc9921
llama : Support intern-s1 ( #14875 )
...
* support internvl
* support interns1
* resolve comments
* put interns1 in tensor mapping
* resolve comment
* move tokenizer changes to sub class
2025-08-07 18:20:40 +02:00
8f15461bea
updated lite
2025-08-08 00:04:47 +08:00
7ad67ba9fe
HIP: add cmake option to enable compiler output of kernel resource usage metrics ( #15103 )
2025-08-07 16:44:14 +02:00
8a71eb03c0
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# ggml/cmake/ggml-config.cmake.in
# ggml/src/ggml-cann/CMakeLists.txt
# ggml/src/ggml-cann/common.h
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-cuda/fattn.cu
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# requirements/requirements-convert_hf_to_gguf.txt
# scripts/compare-llama-bench.py
# tests/test-chat-template.cpp
# tests/test-chat.cpp
# tools/llama-bench/llama-bench.cpp
2025-08-07 21:23:09 +08:00
338b1fe97e
readjusted mistral and oai template, fixed compile issue on termux, updated lite, show generated token ids in debug mode
2025-08-07 21:14:48 +08:00
9a96389544
ggml: Skip backend library linking code when GGML_BACKEND_DL=ON ( #15094 )
...
Python check requirements.txt / check-requirements (push) Failing after 15s
Python Type-Check / pyright type-check (push) Successful in 1m30s
Any available libraries are found and loaded dynamically at runtime.
2025-08-07 13:45:41 +02:00
1d72c84188
CUDA: GEMM for FP32/FP16/BF16 and ne11 <= 16 ( #15131 )
...
* CUDA: GEMM for FP32/FP16/BF16 and ne11 <= 16
2025-08-07 10:53:21 +02:00
20638e4f16
scripts: fix crash when --tool is not set ( #15133 )
2025-08-07 08:50:30 +02:00
36d3f00e14
requirements : fix PyTorch uint64 compatibility ( #15134 )
...
This commit addresses an issue with the convert_hf_to_gguf script
which is currently failing with:
```console
AttributeError: module 'torch' has no attribute 'uint64'
```
This occurred because safetensors expects torch.uint64 to be available
in the public API, but PyTorch 2.2.x only provides limited support for
unsigned types beyond uint8 it seems. The torch.uint64 dtype exists but
is not exposed in the standard torch namespace
(see pytorch/pytorch#58734 ).
PyTorch 2.4.0 properly exposes torch.uint64 in the public API, resolving
the compatibility issue with safetensors. This also required torchvision
to updated to =0.19.0 for compatibility.
Refs: https://huggingface.co/spaces/ggml-org/gguf-my-repo/discussions/186#68938de803e47d990aa087fb
Refs: https://github.com/pytorch/pytorch/issues/58734
2025-08-07 05:31:48 +02:00
5fd160bbd9
ggml: Add basic SET_ROWS support in WebGPU ( #15137 )
...
* Begin work on set_rows
* Work on set rows
* Add error buffers for reporting unsupported SET_ROWS indices
* Remove extra comments
2025-08-06 15:14:40 -07:00
756cfea826
fix profiling crash ( #15072 )
2025-08-06 14:17:51 -07:00
e725a1a982
opencl: add swiglu_oai and add_id ( #15121 )
...
* opencl: add `swiglu-oai`
* opencl: add `add_id`
* opencl: add missing `add_id.cl`
2025-08-06 12:12:17 -07:00
3db4da56a5
chat : support Granite model reasoning and tool call ( #14864 )
2025-08-06 20:27:30 +02:00
476aa3fd57
Fixed name -override-tensors to -override-tensor ( #15129 )
2025-08-06 17:28:48 +01:00
0d8831543c
ggml : fix fallback to CPU for ununsupported ops ( #15118 )
2025-08-06 14:37:35 +02:00
65c797c4fa
chat : fix yandex chat template ( #15116 )
2025-08-06 13:26:49 +02:00
25726898e8
chat : fix hunyuan auto-detection ( #15114 )
...
Signed-off-by: stevenkuang <stevenkuang@tencent.com >
2025-08-06 11:48:30 +02:00
61c19fea56
fixed glm4 sop, lower regex max stacks (+2 squashed commit)
...
Squashed commit:
[47e39ae5d] lower regex max stack again
[0a32ca232] lower regex max stack again
2025-08-06 17:10:57 +08:00
2241453252
CANN: add support for ACL Graph ( #15065 )
...
* feat(cann): add optional support for ACL Graph execution
This commit adds support for executing ggml computational graphs using
Huawei's ACL graph mode via the USE_CANN_GRAPH flag. The support can be
enabled at compile time using the CMake option:
-DUSE_CANN_GRAPH=ON
By default, ACL graph execution is **disabled**, and the fallback path
uses node-by-node execution.
Key additions:
- CMake option to toggle graph mode
- Graph capture and execution logic using
- Tensor property matching to determine whether graph update is required
- Safe fallback and logging if the environment variable LLAMA_SET_ROWS
is unset or invalid
This prepares the backend for performance improvements in repetitive graph
execution scenarios on Ascend devices.
Signed-off-by: noemotiovon <757486878@qq.com >
* Fix review comments
Signed-off-by: noemotiovon <757486878@qq.com >
* remane USE_CANN_GRAPH to USE_ACL_GRAPH
Signed-off-by: noemotiovon <757486878@qq.com >
* fix typo
Signed-off-by: noemotiovon <757486878@qq.com >
---------
Signed-off-by: noemotiovon <757486878@qq.com >
2025-08-06 14:12:42 +08:00
34487d3c02
gpt oss harmony template
2025-08-06 11:39:40 +08:00
Concedo
kallewoof
Aman Gupta
compilade
Johannes Gäßler
AN Long
Olivier Chafik
lhez
Wagner Bruna
Xuan-Son Nguyen
Jeff Bolz
RunningLeon
uvos
Christian Kastner
Daniel Bevenius
Reese Levine
rmatif
Sachin Desai
Juk Armstrong
Diego Devesa
Sigbjørn Skjæret
stevenkuang
Chenguang Li