The classic tick test used a bare object as the pipeline's transformer,
which context.begin hands to register_forward_pre_hook whenever sparse
attention is enabled in the config.
The sdnq Triton kernel gains a block mask input (on the submodule's feat/block-mask branch; the pointer here is unchanged), so the backend advertises the block_mask cap and passes the router's selection as block_mask plus its block sizes. Prepare probes the installed kernel's signature and narrows the cap when the input is missing, warning when sparse attention is on, so an older submodule stays dense instead of failing. PlanEntry carries the narrowed caps, and the router, the sparse stage and the report read those rather than the declaration.
- benchmark: int8-sparse100/50/30/15 and int8-radial30 rows feed the same producer as the flex rows into the quantized kernel, skipped on builds without block_mask; a --configs filter selects attention rows the way --block-configs selects block rows; sparse rows of either backend are held out of the settings advice
- test/test-attention-sdnq-sparse.py: the kernel contract against the token-expanded mask, the nesting filter, the public entry on the quantized paths, the backward against the same selection and against fp32 autograd, and the flex consumer on one selection, pinned to one tile per run with a --tiles sweep
- locale: the sparse attention hint names both capable backends
Batch matrix-matrix and Dynamic Attention BMM applied a legacy Attention
processor to pipe.unet, which a diffusion transformer does not have, so
they served unet models alone and said nothing elsewhere. The choices, the
processor and its slice helper are removed, an unrecognized method now
warns rather than selecting nothing, and a stored value is rewritten to
Scaled-Dot-Product on load.
attention_slicing holds one of Default, Enabled or Disabled, so testing the
string for truth sent Disabled down the enable branch and left the disable
call unreachable, while the log line below it reported the choice rather
than the action taken.
bypass_sdpa_hijacks and llm_context restore the pinned original over the
router and put the router back, including when the body raises. Captioners
and detailers run inside them, so the router must be removable.
The sdnq backend read six settings on every call; it now captures them
when the chain is built. Each backend declares the settings its call
captures, and webui registers one onchange over those names plus the
override set and the torch kernel flags, so a change rebuilds the chain
between jobs. When a compiled model is resident the rebuild also resets
dynamo, since its graphs hold the previous router.
SD_ATTN_DEBUG logs each distinct route once: backend, component role,
step, shapes, dtype and mask presence. The router takes an optional
observer for it, so the clean path carries one pointer check. report()
returns the active chain and generation context, and torch_info records
the whole chain as one string instead of the last prepared backend.
A module-level context tells attention consumers what is running: the
component role (transformer, text encoder, vae), the index of the
denoiser forward about to run, the pass length, and the model. It is
opened and closed around process_images, reset per denoising pass beside
the callback setup, and advanced by both step sources: the classic
callback passes the completed step plus one, the modular pre-forward
hook counts forwards. Roles come from the existing text encoder and vae
hijacks and the modular phase hooks. The step also lives in a device
scalar updated in place, so a compiled reader keeps its graph across
steps.
The flex backend never called the sdpa it replaced, so any backend
stacked before it was unreachable and every call it could not serve,
cpu or 3d inputs included, failed inside flex_attention. It is now an
ordinary entry gated on what flex_attention accepts: 4d tensors on one
non-cpu device. The mask path drops the 2d special case, which indexed
attn_mask.size and reshaped the mask onto the wrong axis; expanding to
(batch, heads, q, kv) already follows sdpa broadcast semantics.
Replace the six closure hijacks stacked in devices.set_sdpa_params with
a registry of declarative backends and one router installed in their
place. Each backend declares the constraints its closure carried as a
predicate, a priority matching its old stacking position, and a prepare
step that imports and configures the implementation; the router walks
the prepared entries by priority and hands declined calls to the
terminal backend (dynamic, flex) or the original sdpa, so fallback is
the router's job rather than each closure's.
- parity held: gates transcribed literally, the same kernel kwargs,
enable_gqa passed to the original only when set, torch_info keeps the
last prepared backend, the dynamic pin still set
- a backend enabled on a platform without it warns instead of silently
doing nothing
- the legacy set_* entry points are gone; devices.py installs the router
- test/test-attention-router.py checks every override subset against the
old stacking order, gate parity over 16,000 shape cases, dispatch,
terminal handoff and prepare isolation, offline
Vladimir Mandic
CalamitousFelicitousness