mirror of
https://github.com/vladmandic/automatic
synced 2026-09-18 00:34:33 +02:00
CalamitousFelicitousness
d75abce642
Sparse attention is a stage over the chain rather than a member of it: one switch, and the router hands the selection to whichever active backend advertises that it consumes a block mask, currently flex. A backend declares that through a capability set, so the quantized kernel joins later without touching the router. The stage gates on the component role, self attention, a minimum sequence length defaulting to the measured 8192 token crossover, and the absence of a token mask or causal flag, which flex cannot combine with a block only mask. Budgets follow a precomputed per step schedule with at most two distinct values. Enabling the feature with no capable backend in the chain warns and leaves attention dense rather than doing nothing quietly. The modular pre-forward hook now receives kwargs and publishes whatever token layout the pipeline passes by name, so a packed sequence gets its conditioning pinned without any model specific code. Without a layout the whole sequence is sparsified and that is logged once per length.