New lora_sdnq_apply radio (exact, requantize) in the lora settings.
requantize keeps the previous behavior: every quantized layer takes the
dequantize-add-requantize path, with factor attach and svd hosting gated
off. A settings-only flip re-applies loaded networks: the mechanism
rides a per-module apply stamp and the network-changed signature, and
the activate fallthrough strips factors a closed gate leaves attached.
Requantize chosen by the setting logs as info instead of the
reduced-fidelity warning.
- locale hint covers fidelity and memory tradeoffs of both methods
- suite: gate, legacy routing and flip-transition tests
The lora block moves out of Networks into its own settings tab, with
header groups by what each option acts on: loading, prompt, application,
quantized models and metadata. Locale hints follow to the new section.
Plain svd truncation of hosted deltas is optimal in weight space but not
in output space: activations concentrate energy in a few input channels,
so scaling the delta by per-channel input RMS before the svd spends the
rank budget on output error instead. Statistics stream from the model's
own forwards on sub-8-bit SDNQ checkpoints and cache per checkpoint;
measured on real LoKR files this raises output-delta retention by ~0.05
at rank 256 and ~0.09 at rank 64, most on MLP down projections.
- modules/lora/lora_calib.py: capture hooks, per-checkpoint cache under
data/sdnq-calib, statistics land on layers as sdnq_calib_rms; gated by
lora_sdnq_host_calib, skipped when the model is compiled
- lora_sdnq.apply_hosted: weighted truncation when statistics exist,
calib count in the load summary
- cli/sdnq-calibrate.py: complete calibration now against a live server
- cli/lora-quant-fidelity.py --calib: hosted rho scored in the
activation-weighted norm
- test/test-sdnq-lora-factors.py: calibration category, 5 tests
Non-additive families (lokr, loha, oft, dora, full) merged into the
quantized weight and lost most of their delta on low-bit formats. On
sub-8-bit layers the set's calc_updown delta now rides the svd
side-channel as its top singular directions instead: factorable members
are subtracted out and appended exactly, so only the non-factorable
remainder is truncated. Truncation keeps the dominant part of the
effect and drops an orthogonal residual, where requantize keeps the
grid extrema and adds grid-shift noise of the delta's own magnitude;
on real lokr files retention rises from 0.04 to about 0.5 at the
default rank.
Hosted layers take no weight backup and unload bit-exactly. The svd
runs under a forked rng so generation seeds are unaffected. At 8 bits
and above requantize retains most of the delta and remains the path.
lora_sdnq_host_rank caps the hosted rank; 0 disables hosting.
- restore stashed svd factors onto the layer's current device; the
stash tuple does not follow module device moves, so an offload
between apply and remove left restored factors on a stale device
- recheck factor shapes for layers already in factor mode, so a
malformed stacked network downgrades to the legacy path instead of
raising in the concat
- clear the fallback log at activate entry so a raise mid-pass cannot
leak stale entries into the next report
- pin both behaviors in the suite and state the compute-dtype fidelity
floor in the module docstring
Baking a lora into a quantized weight requantizes it, and on low-bit
formats round-to-nearest erases sub-step deltas (uint4 retains roughly
2/group_size of the signal). Plain lora deltas now ride the sdnq svd
side-channel: factors append to svd_up/svd_down with the down factor
hadamard-rotated, applied by the dequantizer at full precision in every
forward mode. Apply and remove are exact and take no weight backup.
- non-factorable families (dora, lokr, loha, oft, cp mid, dense bias)
fall back to requantize with a per-pass summary warning
- native fuse now honors the quantized-model guard; fuse requantized in
place on every network swap and accumulated drift
- layers that fell back on a mixed set restore from backup before
re-entering the factor path; untargeted quantized layers are no
longer flagged
- test/test-sdnq-lora-factors.py pins the erasure law, factor-path
exactness, memory accounting and set transitions
dev added sdnq_attention_quantize_fp32 to modules/attention.py, which this
branch replaced with the modules/attention package. The option moves to the
sdnq backend in two places: the options dict the prepared call captures, and
the options tuple that rebuilds the chain when a captured option changes.
Reading it in only the first place would leave a setting that takes effect on
the next model load and not before.
The new option gets a hint alongside the rest of the section.
Batch matrix-matrix and Dynamic Attention BMM applied a legacy Attention
processor to pipe.unet, which a diffusion transformer does not have, so
they served unet models alone and said nothing elsewhere. The choices, the
processor and its slice helper are removed, an unrecognized method now
warns rather than selecting nothing, and a stored value is rewritten to
Scaled-Dot-Product on load.
attention_slicing holds one of Default, Enabled or Disabled, so testing the
string for truth sent Disabled down the enable branch and left the disable
call unreachable, while the log line below it reported the choice rather
than the action taken.
xformers_options had no reader, and Sub-quadratic has not been an
attention choice for a long time, so the hypertile branches keyed on it
never ran. Configs that still store xformers_options load without the
unknown-setting warning.
The sdnq backend read six settings on every call; it now captures them
when the chain is built. Each backend declares the settings its call
captures, and webui registers one onchange over those names plus the
override set and the torch kernel flags, so a change rebuilds the chain
between jobs. When a compiled model is resident the rebuild also resets
dynamo, since its graphs hold the previous router.
SD_ATTN_DEBUG logs each distinct route once: backend, component role,
step, shapes, dtype and mask presence. The router takes an optional
observer for it, so the clean path carries one pointer check. report()
returns the active chain and generation context, and torch_info records
the whole chain as one string instead of the last prepared backend.
A module-level context tells attention consumers what is running: the
component role (transformer, text encoder, vae), the index of the
denoiser forward about to run, the pass length, and the model. It is
opened and closed around process_images, reset per denoising pass beside
the callback setup, and advanced by both step sources: the classic
callback passes the completed step plus one, the modular pre-forward
hook counts forwards. Roles come from the existing text encoder and vae
hijacks and the modular phase hooks. The step also lives in a device
scalar updated in place, so a compiled reader keeps its graph across
steps.
The generate-time gate compared the stored processor name against the
sdp_overrides list, which can never be equal, so the check reduced to
the processor name alone and a changed override set was never applied
until the next model load. set_diffusers_attention now stamps the
override set it applied beside the processor name, the gate compares
both, and pipe switches carry the new attribute with the old one.
The flex backend never called the sdpa it replaced, so any backend
stacked before it was unreachable and every call it could not serve,
cpu or 3d inputs included, failed inside flex_attention. It is now an
ordinary entry gated on what flex_attention accepts: 4d tensors on one
non-cpu device. The mask path drops the 2d special case, which indexed
attn_mask.size and reshaped the mask onto the wrong axis; expanding to
(batch, heads, q, kv) already follows sdpa broadcast semantics.
Replace the six closure hijacks stacked in devices.set_sdpa_params with
a registry of declarative backends and one router installed in their
place. Each backend declares the constraints its closure carried as a
predicate, a priority matching its old stacking position, and a prepare
step that imports and configures the implementation; the router walks
the prepared entries by priority and hands declined calls to the
terminal backend (dynamic, flex) or the original sdpa, so fallback is
the router's job rather than each closure's.
- parity held: gates transcribed literally, the same kernel kwargs,
enable_gqa passed to the original only when set, torch_info keeps the
last prepared backend, the dynamic pin still set
- a backend enabled on a platform without it warns instead of silently
doing nothing
- the legacy set_* entry points are gone; devices.py installs the router
- test/test-attention-router.py checks every override subset against the
old stacking order, gate parity over 16,000 shape cases, dispatch,
terminal handoff and prepare isolation, offline
modules/attention.py becomes modules/attention/: hijacks.py keeps the six
sdpa monkeypatch setters, dispatcher.py the diffusers-side processor and
dispatcher setup with the kernels hub hijack, and the package facade
re-exports every public name so call sites are unchanged. The devices
import moves inside set_diffusers_attention, which removes the
devices <-> attention import cycle.
A delta that does not fit its target module cannot apply, and applying only
the layers that do fit leaves the model in a state nothing was trained for,
so try_load_chain drops the whole file when any family reports a mismatch.
Bias deltas were never checked against the target bias and could only surface
at apply time; a module with no bias stays a non-mismatch, since whole
architectures are built bias=False.
- check bias deltas against the module bias in the lora, norm and full loaders
- carry the mismatch count on the network so the chain can refuse the file
- record refused writes in the infotext so a partial apply is not read as clean
- point the krea2 full-diff test at a module that has a bias
network_add_weights defaulted its base tensor to self.weight for the bias
delta as well, so in fuse mode a diff_b was added to the weight matrix and
the result written into the bias. Layers where in and out differ threw a
shape error and had the weight matrix installed as their bias, square layers
broadcast silently, and either way the summary still counted the delta as
applied.
- pick the base tensor from the bias flag
- name the layer, target and both shapes in the mismatch error
- return which of (weight, bias) took a write, count the rest as refused
- report refused= on partially applied and partially removed networks
- cover both apply paths in test/test-lora-apply.py
Its 2.x sibling always did, so only the 0.9 path hit a cpu weight against cuda
latents. Also logs why a run is refused, which until now travelled only in the
raised error.
TAEHV.decode_video already returns [-1,1] and framepack applied its own x2-1 on
top, landing near [-3,1] where the worker documents [-1,1]. The path was
unreachable until the taesd loader began honoring an explicit variant.
run_ltx reported failure by yielding a string, which is why LTX had no API.
run() is the core underneath: keyword arguments named as video_run.run names
them, a VideoResult back, VideoError out with 499 for an interrupt. The lock,
progress and summary stay in the adapter, whose signature is unchanged since
callers bind to it by keyword. Failure now closes the processing object and
deactivates networks, which abort never did.
The gallery branch took an upload handle or a PIL image, so an encoded string
fell through to .convert and was logged as a failed condition. Strings now go
to the api decoder, which reads base64 and upload refs rather than paths.
CalamitousFelicitousness
Vladimir Mandic
Dity0