New lora_sdnq_apply radio (exact, requantize) in the lora settings.
requantize keeps the previous behavior: every quantized layer takes the
dequantize-add-requantize path, with factor attach and svd hosting gated
off. A settings-only flip re-applies loaded networks: the mechanism
rides a per-module apply stamp and the network-changed signature, and
the activate fallthrough strips factors a closed gate leaves attached.
Requantize chosen by the setting logs as info instead of the
reduced-fidelity warning.
- locale hint covers fidelity and memory tradeoffs of both methods
- suite: gate, legacy routing and flip-transition tests
The lora block moves out of Networks into its own settings tab, with
header groups by what each option acts on: loading, prompt, application,
quantized models and metadata. Locale hints follow to the new section.
Plain svd truncation of hosted deltas is optimal in weight space but not
in output space: activations concentrate energy in a few input channels,
so scaling the delta by per-channel input RMS before the svd spends the
rank budget on output error instead. Statistics stream from the model's
own forwards on sub-8-bit SDNQ checkpoints and cache per checkpoint;
measured on real LoKR files this raises output-delta retention by ~0.05
at rank 256 and ~0.09 at rank 64, most on MLP down projections.
- modules/lora/lora_calib.py: capture hooks, per-checkpoint cache under
data/sdnq-calib, statistics land on layers as sdnq_calib_rms; gated by
lora_sdnq_host_calib, skipped when the model is compiled
- lora_sdnq.apply_hosted: weighted truncation when statistics exist,
calib count in the load summary
- cli/sdnq-calibrate.py: complete calibration now against a live server
- cli/lora-quant-fidelity.py --calib: hosted rho scored in the
activation-weighted norm
- test/test-sdnq-lora-factors.py: calibration category, 5 tests
Non-additive families (lokr, loha, oft, dora, full) merged into the
quantized weight and lost most of their delta on low-bit formats. On
sub-8-bit layers the set's calc_updown delta now rides the svd
side-channel as its top singular directions instead: factorable members
are subtracted out and appended exactly, so only the non-factorable
remainder is truncated. Truncation keeps the dominant part of the
effect and drops an orthogonal residual, where requantize keeps the
grid extrema and adds grid-shift noise of the delta's own magnitude;
on real lokr files retention rises from 0.04 to about 0.5 at the
default rank.
Hosted layers take no weight backup and unload bit-exactly. The svd
runs under a forked rng so generation seeds are unaffected. At 8 bits
and above requantize retains most of the delta and remains the path.
lora_sdnq_host_rank caps the hosted rank; 0 disables hosting.
dev added sdnq_attention_quantize_fp32 to modules/attention.py, which this
branch replaced with the modules/attention package. The option moves to the
sdnq backend in two places: the options dict the prepared call captures, and
the options tuple that rebuilds the chain when a captured option changes.
Reading it in only the first place would leave a setting that takes effect on
the next model load and not before.
The new option gets a hint alongside the rest of the section.
Everything under Cross Attention, SDNQ Attention and Attention Dispatcher
shipped with no hint, which left the layering invisible: the SDP kernel boxes
are candidates torch chooses from per call rather than a selection, and the
Flash box is torch's own build of the kernel rather than the flash-attn package
that the Flash attention override installs.
- cover attention method, sdp kernels, sdp overrides and attention slicing
- give the dynamic attention rates their unit and the estimate they compare against
- cover the sdnq attention kernel settings, including the head dimension clamp
that the hadamard group size resolves through
- cover the diffusers attention dispatcher and the backend names it takes
- record the constraints each override serves, leaving the throughput comparison
to the workload rather than naming a winner
Batch matrix-matrix and Dynamic Attention BMM applied a legacy Attention
processor to pipe.unet, which a diffusion transformer does not have, so
they served unet models alone and said nothing elsewhere. The choices, the
processor and its slice helper are removed, an unrecognized method now
warns rather than selecting nothing, and a stored value is rewritten to
Scaled-Dot-Product on load.
The offload section interleaved shared and mode-specific options. The
page now reads top down: general options, an Offload Overrides section
holding the exclusions shared by balanced and group offload, then one
tuning section per mode. Use torch streams becomes Prefetch with streams
and Record torch streams becomes Overlap stream transfers, the section
headers carry hints, and the hints reflect live behavior, including
dropping the model-reload badge from options that apply on change.
2.5 reuses the LTX-2 pipeline classes, so it is described through the
capability table: Gemma 4 text encoder, cross-timestep conditioning, and the
upsampler and stage 2 LoRA that now ship inside the model repo. The repo
carries the distilled scheduler config, so Dev rows restore the terminal
shift, and the Dev transformer sits in transformer_full.
Distilled rows stop forcing dynamic shifting on, which remapped their sigma
schedule. Auto duration hands the clip length to the duration head.
The progress response carries a stage label in textinfo, but the
button and hint composed their text from the job name alone, so
staged pipelines sat on a bare inference label. The stage now takes
precedence in the button label and appears next to the job in the
performance hint.
First natively modular model: the pipeline is driven directly through
ModularPipeline, with components fetched per workflow (fl2va covers
text and first/last-frame conditioning).
- per-generation overrides snap the canvas to /32, align frames to
the 17n+5 grid and duration window, and keep the bespoke scheduler
pair
- group offload for modular pipelines applied per component in
sd_offload; re-application is a guarded no-op
- audio checkbox pops the audio decode block so decode and muxing are
skipped
- frames=1 renders a single still image: the duration floor lifts per
instance and sub-floor latents pad at the vae decoder
- progress and interrupt handling via a transformer forward pre-hook
- vae scale factor override, tuple-safe patch size
Vae-class components never take group hooks, so group mode kept them
resident on the gpu; a MiniMax-class video vae holds about 10GB that
way while running only seconds per generation. Components above 1GB
now rest in system memory: the apply_forward_hook bridge on encode and
decode fires an on-demand hook that moves the whole module to the
device, so tiled calls find every weight already loaded, and the
processing seams return it to cpu once outputs are materialized. Small
vaes stay resident since the transfer would cost more than it frees.
- placement is decided per component by measured size and requires the
entry bridge; components without it stay resident
- move_model no longer forces on-demand vaes to the gpu for
non-txt2img tasks, and full_vae_encode onloads before binding the
input, which otherwise lands on the resting device
- mode switches clear the stamp and hook in both directions
Group offload is applied per component through one engine shared by
regular and modular pipelines. Each component carries a config
signature: re-application with unchanged settings is a no-op instead
of raising before the first forward or silently keeping a stale
config, and changed settings remove the hooks and reapply. Switching
offload modes cleans up the previous mode's hooks in both directions.
- text encoders always offload at leaf level without streams, so
their weights are never held in pinned host memory
- the vae never takes group hooks and stays resident: the hooks are
forward-scoped, while pipelines enter through encode/decode and
tiled calls re-enter per tile
- new pin offload memory option: disabled pins one group at a time
instead of holding the whole module in non-pageable memory, and
modules larger than half of system memory degrade automatically
- record stream is clamped to stream mode; the standalone
combination is rejected upstream
Entries in the always and never lists are matched against the pipeline
component name (text_encoder, vae) as well as the model class name, so
one entry covers every architecture instead of needing a new class name
per model. Class entries keep working unchanged.
Never is still tested first, so a class entry there exempts a single
model from a component entry in the always list.
This line resulted in the cleanup check never running if the current gallery has fewer than 1000 files, no matter how many excess entries there are in the cache.
Vladimir Mandic
CalamitousFelicitousness
awsr