At 1.5 the ramp hands nearly every layer to the style network mid-generation,
which on few-step models overwrites the forming subject before identity sets.
Alpha 0 freezes the schedule into a static per-layer split: the subject keeps
its layers for the whole generation and style keeps the layers where it is
more salient. Nonzero values remain the scheduled handover toward the second
network.
- locale hints describe both regimes; the mode hint no longer implies the
shift is always on
Text encoder networks now apply unconditionally in the native path.
- remove the option, the per-request parameter and the control threading
- collapse activate_filtered into plain activate at all call sites
- drop the toggle from the prompt embed cache key
- register the retired key so existing configs load without warnings
Add lora_stack_mode with ties, dare_ties, dare_linear and
magnitude_prune combination of per-network deltas when several loaded
networks target one layer; sum stays the default and the exact factor
path. Combined deltas ride the existing tail: hosted svd on sub-8-bit
SDNQ, requantize at int8 and above, direct add elsewhere. Text-encoder
layers and single-network sets keep plain summation. DARE masks draw
from per-layer sha256 seeds so re-applies and cache entries stay
deterministic; the stack settings join the activation and factor-cache
signatures so settings changes re-apply without a reload.
sparse_attention_exclude is a comma separated denylist matched case insensitively against the architecture, the pipeline class and the denoiser class, so one entry works whichever name is to hand. It resolves once per model rather than per call and declines with a log line. Seeded with CosmosTransformer3DModel, the transformer Anima runs, which returns banded noise at every budget tested against a sound dense baseline; listing the class rather than the architecture covers the other models built on it, none of which have been checked.
The option defaulted to 0 and a property turned that into 8192, which hid
the real default behind a sentinel and left no way to sparsify every
sequence. The default is now 8192 in the settings registry, 0 means no
floor, and the slider tops out at 32768 rather than a length nothing
reaches. A negative value is left unclaimed for a derived floor.
Sparse attention is a stage over the chain rather than a member of it:
one switch, and the router hands the selection to whichever active
backend advertises that it consumes a block mask, currently flex. A
backend declares that through a capability set, so the quantized kernel
joins later without touching the router.
The stage gates on the component role, self attention, a minimum
sequence length defaulting to the measured 8192 token crossover, and the
absence of a token mask or causal flag, which flex cannot combine with a
block only mask. Budgets follow a precomputed per step schedule with at
most two distinct values. Enabling the feature with no capable backend
in the chain warns and leaves attention dense rather than doing nothing
quietly.
The modular pre-forward hook now receives kwargs and publishes whatever
token layout the pipeline passes by name, so a packed sequence gets its
conditioning pinned without any model specific code. Without a layout
the whole sequence is sparsified and that is logged once per length.
Hosting a non-factorable set costs one truncated svd per layer on every
fresh apply. The factors are deterministic in the checkpoint, the loaded
set, the host rank and the calibration statistics, so they persist under
data/lora-factor-cache keyed by that identity and replay bit-identically
on later applies. lora_sdnq_host_cache sets the disk budget in GB
(0 disables), least-recently-used entries are evicted past it. With the
cost paid once per configuration, svd subspace iterations rise from 2 to
4: about two thirds of the captured-energy gap to an exact decomposition
for +45% one-time compute; an exact svd measures 200-400x slower at
these shapes and is not viable.
- modules/lora/lora_factor_cache.py: signature, store, flush, eviction
- lora_sdnq.apply_hosted: fetch before computing, store after, hits in
the load summary
- cli/lora-quant-fidelity.py: matching niter for the hosted mirror
- test/test-sdnq-lora-factors.py: factor-cache category, 3 tests
New lora_sdnq_apply radio (exact, requantize) in the lora settings.
requantize keeps the previous behavior: every quantized layer takes the
dequantize-add-requantize path, with factor attach and svd hosting gated
off. A settings-only flip re-applies loaded networks: the mechanism
rides a per-module apply stamp and the network-changed signature, and
the activate fallthrough strips factors a closed gate leaves attached.
Requantize chosen by the setting logs as info instead of the
reduced-fidelity warning.
- locale hint covers fidelity and memory tradeoffs of both methods
- suite: gate, legacy routing and flip-transition tests
The lora block moves out of Networks into its own settings tab, with
header groups by what each option acts on: loading, prompt, application,
quantized models and metadata. Locale hints follow to the new section.
Plain svd truncation of hosted deltas is optimal in weight space but not
in output space: activations concentrate energy in a few input channels,
so scaling the delta by per-channel input RMS before the svd spends the
rank budget on output error instead. Statistics stream from the model's
own forwards on sub-8-bit SDNQ checkpoints and cache per checkpoint;
measured on real LoKR files this raises output-delta retention by ~0.05
at rank 256 and ~0.09 at rank 64, most on MLP down projections.
- modules/lora/lora_calib.py: capture hooks, per-checkpoint cache under
data/sdnq-calib, statistics land on layers as sdnq_calib_rms; gated by
lora_sdnq_host_calib, skipped when the model is compiled
- lora_sdnq.apply_hosted: weighted truncation when statistics exist,
calib count in the load summary
- cli/sdnq-calibrate.py: complete calibration now against a live server
- cli/lora-quant-fidelity.py --calib: hosted rho scored in the
activation-weighted norm
- test/test-sdnq-lora-factors.py: calibration category, 5 tests
Non-additive families (lokr, loha, oft, dora, full) merged into the
quantized weight and lost most of their delta on low-bit formats. On
sub-8-bit layers the set's calc_updown delta now rides the svd
side-channel as its top singular directions instead: factorable members
are subtracted out and appended exactly, so only the non-factorable
remainder is truncated. Truncation keeps the dominant part of the
effect and drops an orthogonal residual, where requantize keeps the
grid extrema and adds grid-shift noise of the delta's own magnitude;
on real lokr files retention rises from 0.04 to about 0.5 at the
default rank.
Hosted layers take no weight backup and unload bit-exactly. The svd
runs under a forked rng so generation seeds are unaffected. At 8 bits
and above requantize retains most of the delta and remains the path.
lora_sdnq_host_rank caps the hosted rank; 0 disables hosting.
dev added sdnq_attention_quantize_fp32 to modules/attention.py, which this
branch replaced with the modules/attention package. The option moves to the
sdnq backend in two places: the options dict the prepared call captures, and
the options tuple that rebuilds the chain when a captured option changes.
Reading it in only the first place would leave a setting that takes effect on
the next model load and not before.
The new option gets a hint alongside the rest of the section.
xformers_options had no reader, and Sub-quadratic has not been an
attention choice for a long time, so the hypertile branches keyed on it
never ran. Configs that still store xformers_options load without the
unknown-setting warning.
On ROCm, host-to-device DMA from mmap'd safetensors pages stalls ~1s per
copy, so weights move to the GPU at ~27 MB/s instead of ~28 GB/s. With
offload enabled this re-copies weights every forward, so generation
appears to hang.
Adds `diffusers_disable_mmap` (Settings > Model Loading), off by default,
which makes diffusers read shards into anonymous memory instead. Costs
peak RAM equal to the model size, so it is opt-in.
SD3.5-large on RX 9070 (gfx1201), same prompt and steps:
off: no image after 120s
on: image in 20s
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The offload section interleaved shared and mode-specific options. The
page now reads top down: general options, an Offload Overrides section
holding the exclusions shared by balanced and group offload, then one
tuning section per mode. Use torch streams becomes Prefetch with streams
and Record torch streams becomes Overlap stream transfers, the section
headers carry hints, and the hints reflect live behavior, including
dropping the model-reload badge from options that apply on change.
Group offload is applied per component through one engine shared by
regular and modular pipelines. Each component carries a config
signature: re-application with unchanged settings is a no-op instead
of raising before the first forward or silently keeping a stale
config, and changed settings remove the hooks and reapply. Switching
offload modes cleans up the previous mode's hooks in both directions.
- text encoders always offload at leaf level without streams, so
their weights are never held in pinned host memory
- the vae never takes group hooks and stays resident: the hooks are
forward-scoped, while pipelines enter through encode/decode and
tiled calls re-enter per tile
- new pin offload memory option: disabled pins one group at a time
instead of holding the whole module in non-pageable memory, and
modules larger than half of system memory degrade automatically
- record stream is clamped to stream mode; the standalone
combination is rejected upstream
The secondary unet loads from the networks panel toggle or the settings
page; companion ordering and dynamic visibility pushes in ui_settings.py
are not worth their footprint for one setting.
One UNET override cannot serve dual-transformer arches: ideogram4
conditional/unconditional and wan combined-stage experts need separate
files, and previously a single override landed on both experts.
- sd_unet_secondary option with per-slot tracking, consumed-state sync,
arch-change reset, and incompatible-override fallback
- dropdown renders beside the primary, follows it into quicksettings,
and is visible only for dual-transformer model types
- ideogram4 native single-file spec with a quant-aware fused-qkv
converter; such converters run before comfy_quant detection via
TransformerSpec.converter_handles_quant
- quicksettings render in configured order (sort keyed on the option
object and always fell back to alphabetical)
- post-load dtype warning skips quantized transformers
Vladimir Mandic
CalamitousFelicitousness
Dity0
Christopher McGee