process_images swallows the interrupt assertion, so a cancel reaches the shared
video core as an empty result and was raised as 'processing failed' with 500,
leaving clients unable to tell a cancel from a crash. Mirrors the LTX path,
which already returns 499.
Queue.__exit__ returned _queue_lock, and a truthy __exit__ return suppresses the
exception in flight, so every `with queue_lock:` block discarded exceptions and
resumed with locals from the aborted block unassigned. The bare threading.Lock it
replaced returned None.
The refine-default comment claimed Condition variants are excluded from two-stage
refine, but supports_two_stage_refine has no such carve-out and ltx_process builds
a second condition set so conditioning survives the upsample. Two file references
had also drifted: ltx_process.py:179 moved, and the offload hook keying now lives
in sd_offload_balanced. Name the symbols instead of the line numbers.
New lora_sdnq_apply radio (exact, requantize) in the lora settings.
requantize keeps the previous behavior: every quantized layer takes the
dequantize-add-requantize path, with factor attach and svd hosting gated
off. A settings-only flip re-applies loaded networks: the mechanism
rides a per-module apply stamp and the network-changed signature, and
the activate fallthrough strips factors a closed gate leaves attached.
Requantize chosen by the setting logs as info instead of the
reduced-fidelity warning.
- locale hint covers fidelity and memory tradeoffs of both methods
- suite: gate, legacy routing and flip-transition tests
The lora block moves out of Networks into its own settings tab, with
header groups by what each option acts on: loading, prompt, application,
quantized models and metadata. Locale hints follow to the new section.
Plain svd truncation of hosted deltas is optimal in weight space but not
in output space: activations concentrate energy in a few input channels,
so scaling the delta by per-channel input RMS before the svd spends the
rank budget on output error instead. Statistics stream from the model's
own forwards on sub-8-bit SDNQ checkpoints and cache per checkpoint;
measured on real LoKR files this raises output-delta retention by ~0.05
at rank 256 and ~0.09 at rank 64, most on MLP down projections.
- modules/lora/lora_calib.py: capture hooks, per-checkpoint cache under
data/sdnq-calib, statistics land on layers as sdnq_calib_rms; gated by
lora_sdnq_host_calib, skipped when the model is compiled
- lora_sdnq.apply_hosted: weighted truncation when statistics exist,
calib count in the load summary
- cli/sdnq-calibrate.py: complete calibration now against a live server
- cli/lora-quant-fidelity.py --calib: hosted rho scored in the
activation-weighted norm
- test/test-sdnq-lora-factors.py: calibration category, 5 tests
Non-additive families (lokr, loha, oft, dora, full) merged into the
quantized weight and lost most of their delta on low-bit formats. On
sub-8-bit layers the set's calc_updown delta now rides the svd
side-channel as its top singular directions instead: factorable members
are subtracted out and appended exactly, so only the non-factorable
remainder is truncated. Truncation keeps the dominant part of the
effect and drops an orthogonal residual, where requantize keeps the
grid extrema and adds grid-shift noise of the delta's own magnitude;
on real lokr files retention rises from 0.04 to about 0.5 at the
default rank.
Hosted layers take no weight backup and unload bit-exactly. The svd
runs under a forked rng so generation seeds are unaffected. At 8 bits
and above requantize retains most of the delta and remains the path.
lora_sdnq_host_rank caps the hosted rank; 0 disables hosting.
- restore stashed svd factors onto the layer's current device; the
stash tuple does not follow module device moves, so an offload
between apply and remove left restored factors on a stale device
- recheck factor shapes for layers already in factor mode, so a
malformed stacked network downgrades to the legacy path instead of
raising in the concat
- clear the fallback log at activate entry so a raise mid-pass cannot
leak stale entries into the next report
- pin both behaviors in the suite and state the compute-dtype fidelity
floor in the module docstring
Baking a lora into a quantized weight requantizes it, and on low-bit
formats round-to-nearest erases sub-step deltas (uint4 retains roughly
2/group_size of the signal). Plain lora deltas now ride the sdnq svd
side-channel: factors append to svd_up/svd_down with the down factor
hadamard-rotated, applied by the dequantizer at full precision in every
forward mode. Apply and remove are exact and take no weight backup.
- non-factorable families (dora, lokr, loha, oft, cp mid, dense bias)
fall back to requantize with a per-pass summary warning
- native fuse now honors the quantized-model guard; fuse requantized in
place on every network swap and accumulated drift
- layers that fell back on a mixed set restore from backup before
re-entering the factor path; untargeted quantized layers are no
longer flagged
- test/test-sdnq-lora-factors.py pins the erasure law, factor-path
exactness, memory accounting and set transitions
dev added sdnq_attention_quantize_fp32 to modules/attention.py, which this
branch replaced with the modules/attention package. The option moves to the
sdnq backend in two places: the options dict the prepared call captures, and
the options tuple that rebuilds the chain when a captured option changes.
Reading it in only the first place would leave a setting that takes effect on
the next model load and not before.
The new option gets a hint alongside the rest of the section.
Batch matrix-matrix and Dynamic Attention BMM applied a legacy Attention
processor to pipe.unet, which a diffusion transformer does not have, so
they served unet models alone and said nothing elsewhere. The choices, the
processor and its slice helper are removed, an unrecognized method now
warns rather than selecting nothing, and a stored value is rewritten to
Scaled-Dot-Product on load.
attention_slicing holds one of Default, Enabled or Disabled, so testing the
string for truth sent Disabled down the enable branch and left the disable
call unreachable, while the log line below it reported the choice rather
than the action taken.
xformers_options had no reader, and Sub-quadratic has not been an
attention choice for a long time, so the hypertile branches keyed on it
never ran. Configs that still store xformers_options load without the
unknown-setting warning.
The sdnq backend read six settings on every call; it now captures them
when the chain is built. Each backend declares the settings its call
captures, and webui registers one onchange over those names plus the
override set and the torch kernel flags, so a change rebuilds the chain
between jobs. When a compiled model is resident the rebuild also resets
dynamo, since its graphs hold the previous router.
SD_ATTN_DEBUG logs each distinct route once: backend, component role,
step, shapes, dtype and mask presence. The router takes an optional
observer for it, so the clean path carries one pointer check. report()
returns the active chain and generation context, and torch_info records
the whole chain as one string instead of the last prepared backend.
A module-level context tells attention consumers what is running: the
component role (transformer, text encoder, vae), the index of the
denoiser forward about to run, the pass length, and the model. It is
opened and closed around process_images, reset per denoising pass beside
the callback setup, and advanced by both step sources: the classic
callback passes the completed step plus one, the modular pre-forward
hook counts forwards. Roles come from the existing text encoder and vae
hijacks and the modular phase hooks. The step also lives in a device
scalar updated in place, so a compiled reader keeps its graph across
steps.
The generate-time gate compared the stored processor name against the
sdp_overrides list, which can never be equal, so the check reduced to
the processor name alone and a changed override set was never applied
until the next model load. set_diffusers_attention now stamps the
override set it applied beside the processor name, the gate compares
both, and pipe switches carry the new attribute with the old one.
The flex backend never called the sdpa it replaced, so any backend
stacked before it was unreachable and every call it could not serve,
cpu or 3d inputs included, failed inside flex_attention. It is now an
ordinary entry gated on what flex_attention accepts: 4d tensors on one
non-cpu device. The mask path drops the 2d special case, which indexed
attn_mask.size and reshaped the mask onto the wrong axis; expanding to
(batch, heads, q, kv) already follows sdpa broadcast semantics.
Replace the six closure hijacks stacked in devices.set_sdpa_params with
a registry of declarative backends and one router installed in their
place. Each backend declares the constraints its closure carried as a
predicate, a priority matching its old stacking position, and a prepare
step that imports and configures the implementation; the router walks
the prepared entries by priority and hands declined calls to the
terminal backend (dynamic, flex) or the original sdpa, so fallback is
the router's job rather than each closure's.
- parity held: gates transcribed literally, the same kernel kwargs,
enable_gqa passed to the original only when set, torch_info keeps the
last prepared backend, the dynamic pin still set
- a backend enabled on a platform without it warns instead of silently
doing nothing
- the legacy set_* entry points are gone; devices.py installs the router
- test/test-attention-router.py checks every override subset against the
old stacking order, gate parity over 16,000 shape cases, dispatch,
terminal handoff and prepare isolation, offline
modules/attention.py becomes modules/attention/: hijacks.py keeps the six
sdpa monkeypatch setters, dispatcher.py the diffusers-side processor and
dispatcher setup with the kernels hub hijack, and the package facade
re-exports every public name so call sites are unchanged. The devices
import moves inside set_diffusers_attention, which removes the
devices <-> attention import cycle.
A delta that does not fit its target module cannot apply, and applying only
the layers that do fit leaves the model in a state nothing was trained for,
so try_load_chain drops the whole file when any family reports a mismatch.
Bias deltas were never checked against the target bias and could only surface
at apply time; a module with no bias stays a non-mismatch, since whole
architectures are built bias=False.
- check bias deltas against the module bias in the lora, norm and full loaders
- carry the mismatch count on the network so the chain can refuse the file
- record refused writes in the infotext so a partial apply is not read as clean
- point the krea2 full-diff test at a module that has a bias
Vladimir Mandic
Alejandro Alberto Aguilar Morales
CalamitousFelicitousness
Dity0