* Linux CU13 CI
* Bump max CUDA arch
* CUDA13 Linux
* Upload the correct build to rolling (CUDA13)
* Downgrade cuda to get better compatibility
Runpod can't handle 13.1, and if they can't handle it neither can the people with a secondary GPU of an older generation.
* Add support for compute capability 89 in NVCCFLAGS
* hex-mm: process m-tail rows on HMX instead of HVX
* hmx-mm: unroll and optimize padded activation loop
---------
Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
common/arg.cpp:3719:9: error: function 'operator()' could be declared with attribute 'noreturn' [-Werror,-Wmissing-noreturn]
3719 | [](common_params & /*params*/, int /*value*/) {
| ^
common/arg.cpp:3726:9: error: function 'operator()' could be declared with attribute 'noreturn' [-Werror,-Wmissing-noreturn]
3726 | [](common_params & /*params*/, int /*value*/) {
| ^
common/arg.cpp:3733:9: error: function 'operator()' could be declared with attribute 'noreturn' [-Werror,-Wmissing-noreturn]
3733 | [](common_params & /*params*/, int /*value*/) {
| ^
common/arg.cpp:3740:9: error: function 'operator()' could be declared with attribute 'noreturn' [-Werror,-Wmissing-noreturn]
3740 | [](common_params & /*params*/, int /*value*/) {
| ^
common/arg.cpp:3747:9: error: function 'operator()' could be declared with attribute 'noreturn' [-Werror,-Wmissing-noreturn]
3747 | [](common_params & /*params*/, int /*value*/) {
| ^
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Previously, unknown tool names passed via --tools were silently ignored.
Now the server validates each tool name at startup and exits with an
error if an unrecognized tool is specified, listing the available tools.
Assisted-by: llama.cpp:local pi
* chat/autoparser: the fixes
* Move optspace() to chat-peg-parser, comment out server tests invalidated due to content now allowed with forced tool calls.
* Trim whitespace on apply instead
* sd: sync to master-593-3d6064b
* sd: use the same sdtype_adapter object for all builds
Since master-592-b8079e2, no sd.cpp source depends on the ggml
backend build anymore.
* sd: fix main_gpu selection
* sd: report backend devices to the Python layer
* docs : update speculative decoding parameters after refactor (#22397)
Update docs/speculative.md to reflect the new parameter naming scheme
introduced in PR #22397:
- Replace --draft-max/--draft-min with --spec-draft-n-max/--spec-draft-n-min
- Replace --spec-ngram-size-n/m with per-implementation variants
- Add documentation for all new --spec-ngram-*- parameters
- Update all example commands
Assisted-by: llama.cpp:local pi
* pi : add rule to use gh CLI for GitHub resources
Assisted-by: llama.cpp:local pi
* docs : run llama-gen-docs
* arg : fix typo
* shader(norm): add layer norm ops
* shader(norm): stablize floating point computation with Kahan summation and handle mixed types
* shader(norm): remove the non-contiguous strides
* shader(norm): use the original implementation rather than the kahan summation
Llama-architecture q_proj/k_proj weights need an axis-0 row permutation
to match GGML's RoPE convention. The BF16 path applies this in
LlamaModel.modify_tensors via LlamaModel.permute, but the NVFP4 path
bypasses modify_tensors and writes weights directly through
ModelBase._repack_nvfp4. Without the permutation, attention heads end
up scrambled at inference and the model produces gibberish.
This change overrides _repack_nvfp4 on LlamaModel and applies the same
permutation to both the nibble-packed weight and the per-block scale
before delegating to ModelBase._repack_nvfp4 via super(). Reuses the
existing LlamaModel.permute static helper and respects the existing
undo_permute flag, so subclasses (Mistral, Granite, Llama4, etc.)
inherit the fix automatically.
Verified on TinyLlama-1.1B reproducer: perplexity drops from 4419
(gibberish) to 43.9, matching the BF16-dequantized baseline (44.0).
Also verified end-to-end on ALIA-40b-instruct-2601 (BSC, Llama
architecture) with multilingual generation in Spanish/Catalan/Basque/
Galician all coherent with the fix applied.
Co-authored-by: Chema <chema@montevive.ai>
Closes#2178
The --debugmode help string previously read "Shows additional debug
info in the terminal" with no indication of what numeric values it
accepts or what each does — making the recommended troubleshooting
flag opaque (per #2178).
Document the three values actually checked in the source:
-1: Horde-quiet (suppresses non-essential prints; auto-applied
when --horde* args are set, see configure_horde_settings)
0: default
1: verbose (extra slot/cache info; larger utfprint buffer;
retains 'debug-' horde model prefix; etc.)
Also note that bare --debugmode (no value) implies 1, which is the
existing argparse behavior (nargs='?', const=1) but easy to miss.
Concedo
Wagner Bruna
henk717
Aleksander Grygier
zzzzwc
viggy
Sigbjørn Skjæret
fl0rianr
Trivikram Reddy
lhez
Radoslav Gerganov
Adrien Gallouët
Georgi Gerganov
Alessandro de Oliveira Faria (A.K.A.CABELO)
Ismail
Charles Xu
leonardHONG
Xuan-Son Nguyen
Shakhnazar Sailaukan
JusteLeo
Piotr Wilkin (ilintar)
Evan Huus
Nick Towle
Atomic-Germ
Chen Yuan
Aldehir Rojas
Julien Denize
JM Robles
Tai An