From 7bc11ff47026ba33d8d6d125a3f9bf85fb267192 Mon Sep 17 00:00:00 2001 From: CalamitousFelicitousness Date: Mon, 24 Aug 2026 02:01:19 +0100 Subject: [PATCH] docs(settings): name the sparse attention sequence floor The hint deferred to an unnamed built in threshold where every comparable hint in the file gives the value. It is a single static constant. --- ui/locale/locale_en.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ui/locale/locale_en.json b/ui/locale/locale_en.json index eed3b9616..0d83abfc0 100644 --- a/ui/locale/locale_en.json +++ b/ui/locale/locale_en.json @@ -1473,7 +1473,7 @@ {"id":"","label":"SDNQ Attention Quantize FP32","localized":"","hint":"Upcasts queries, keys and values to fp32 for the quantization step, meaning the mean subtraction, scale and rounding that produce the low precision operands. The matmuls themselves are unaffected, and the kernel applies the scales in fp32 either way.
Turned off, that arithmetic runs in the model's own precision. bf16 carries eight mantissa bits, so a scale derived in it is coarser than one derived in fp32, and SDNQ Attention use Smooth K loses the most from it, since a mean across the whole sequence is exactly the kind of sum that wants the extra bits.

Whether the upcast costs anything depends on how the GPU runs fp32 vector work against fp16 and bf16. NVIDIA and AMD run them at the same rate here, so there is nothing to save; Intel runs fp32 slower and takes a noticeable hit.

Enabled by default.","ui":"settings_cuda"}, {"id":"","label":"Sparse Attention","localized":"","hint":"Attention is computed on a subset of key tiles instead of all of them, which lowers cost on long sequences. Requires an attention backend that accepts a block mask, SDNQ attention or Flex attention in SDP overrides. Below the minimum sequence the setting stays inactive and attention is dense.","ui":"settings_cuda"}, {"id":"","label":"Sparse Attention KV budget","localized":"","hint":"Percentage of the eligible key tiles each query tile keeps. Lower budgets are faster and coarser. Text, conditioning and audio tokens are always kept, as are the tiles on the diagonal, so the budget applies only to the bulk image or video tokens.","ui":"settings_cuda"}, - {"id":"","label":"Sparse Attention minimum sequence","localized":"","hint":"Shortest sequence that is sparsified. Below it attention stays dense, because the selection costs more than it saves. 0 uses the built in threshold.","ui":"settings_cuda"}, + {"id":"","label":"Sparse Attention minimum sequence","localized":"","hint":"Shortest sequence that is sparsified. Below it attention stays dense, because the selection costs more than it saves. 0 uses 8192.","ui":"settings_cuda"}, {"id":"","label":"Sparse Attention dense steps","localized":"","hint":"Number of steps at the start and end of sampling that receive a larger budget, where composition and detail are set. 0 applies one budget to every step.","ui":"settings_cuda"}, {"id":"","label":"Sparse Attention dense step bonus","localized":"","hint":"Percentage points added to the budget on the dense steps.","ui":"settings_cuda"}, {"id":"","label":"Sparse Attention share selection across heads","localized":"","hint":"One selection is computed for all attention heads rather than one per head. Cheaper to select and coarser in what it keeps.","ui":"settings_cuda"}