On ROCm, host-to-device DMA from mmap'd safetensors pages stalls ~1s per
copy, so weights move to the GPU at ~27 MB/s instead of ~28 GB/s. With
offload enabled this re-copies weights every forward, so generation
appears to hang.
Adds `diffusers_disable_mmap` (Settings > Model Loading), off by default,
which makes diffusers read shards into anonymous memory instead. Costs
peak RAM equal to the model size, so it is opt-in.
SD3.5-large on RX 9070 (gfx1201), same prompt and steps:
off: no image after 120s
on: image in 20s
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The loader applied balanced offload directly, which places nothing when the
offload mode is not balanced, leaving every component on cpu where the load
parked it. The dispatcher places them under whichever mode is configured, and
the vae seams pair the balanced call with an on-demand sweep so the vae returns
to cpu once its outputs are materialized.
Static inventory of every registered pipeline: component classes come from
the init type annotations, the component specs on modular pipelines, and
the video model definitions, so the real role function runs against real
classes with no weights loaded. The audit prints the role of every
component, lists which components turn resident under the shipped 22 GB
never-offload default, names the custom pipelines it cannot see, and
asserts that no component carries an undecorated entry point, that
opted-out classes route on-demand, and that every pipeline places a
per-step component.
The offload section interleaved shared and mode-specific options. The
page now reads top down: general options, an Offload Overrides section
holding the exclusions shared by balanced and group offload, then one
tuning section per mode. Use torch streams becomes Prefetch with streams
and Record torch streams becomes Overlap stream transfers, the section
headers carry hints, and the hints reflect live behavior, including
dropping the model-reload badge from options that apply on change.
Offline suite for the placement roles: the role table over the component
names sdnext loads, one dispatch arm per component with hooks landing on
text encoder wrappers, force sweeps scoped to stamped modules, a settings
change re-placing a resident component, on-demand contracts, enumeration
on both pipeline kinds, and the upstream markers the roles read.
The mage vae defined plain encode and decode methods, leaving offload
hooks nothing to fire on when the pipeline enters through them. The
diffusers entry decorator routes both through the hook path, the same as
every stock autoencoder.
Group offload ignored the offload exclusion settings and its options took
effect only at the next model load. Components named by the never-offload
list or the model-type exclusion now stay resident on the accelerator,
with matching shared between the balanced hook and the group roles. The
offload options re-place loaded components when changed: group modes
re-run the placement pass, and balanced mode rebuilds its hook, which
snapshots the exclusion lists at construction.
The force sweep enumerated pipeline components and moved every module to
cpu, including components with no hook to bring them back. It now moves
only modules stamped for on-demand placement, walks the pipe variants
through the shared enumeration instead of the components property that can
raise, keeps going past a component that fails to move, and collects
garbage once per sweep rather than per module.
Group offload placed components by name matching, which sent every modular
text encoder through a second apply with the denoiser profile and missed
vae-class components not named vae. Placement now derives from the component:
an encode or decode entry bridge and an upstream group-offload opt-out both
take whole-module on-demand placement, an undecorated encode or decode entry
stays resident with a warning, denoiser slot names take the per-step profile,
and everything else takes the leaf profile. One pass serves classic and
modular pipelines, enumerating modular components from their specs, with
hooks landing on text encoder wrappers rather than their inner models. A
signature sentinel marks a component mid-apply; a failed apply is torn down
on the next pass. The pass logs the placement of every component.
Implicit tokens are disabled process-wide, so the hijack that re-attaches the
token explicitly is what keeps gated repos reachable. It was installed by the
checkpoint loader and the caption paths, so loading a video model into a fresh
session sent unauthenticated requests and gated repos answered 401.
The stage 2 LoRA stores its connector deltas as
diffusion_model.{video,audio}_embeddings_connector, but
LTX2LoraLoaderMixin.lora_state_dict recognizes connectors only under the 2.3
text_embedding_projection prefix. All 3544 keys are routed into the transformer
namespace and peft drops the 224 that land nowhere, leaving refine with a
transformer-only adapter.
Wrapping lora_state_dict moves those keys onto the connectors component using
the rename table from the convert_ltx2_to_diffusers script. The wrapper is inert
once no misrouted keys appear, so it needs no version check.
2.5 reuses the LTX-2 pipeline classes, so it is described through the
capability table: Gemma 4 text encoder, cross-timestep conditioning, and the
upsampler and stage 2 LoRA that now ship inside the model repo. The repo
carries the distilled scheduler config, so Dev rows restore the terminal
shift, and the Dev transformer sits in transformer_full.
Distilled rows stop forcing dynamic shifting on, which remapped their sigma
schedule. Auto duration hands the clip length to the duration head.
The shared repo was written back onto the registry row, a module-level
singleton, so turning the setting off left the row pointing at the shared copy
for the rest of the session. It is chosen into locals instead.
Pipelines rarely report a sample rate, so the save path fell back to 24000.
LTX-2.3 and 2.5 run at 48k, and muxing at half the rate drops the track an
octave. The rate now comes from the vocoder, as the LTX tab already did.
The subfolder name doubles as the pipeline argument, so a repo shipping a
second transformer beside the default one had no way to load it: the argument
was dropped and the default weights stayed. dit_kwarg names the argument when
it differs from the folder.
The sampler shift override looked for flow_shift, which only UniPC-style
schedulers declare, so it never reached the flow-match schedulers 2.x uses.
Config keys are now restored only when the run wrote them, and a cached
upsampler is rebuilt once its repo or the model's VAE changes.
Adds the Gemma 4 text encoder, the duration head and the diffusion decoder.
Pipeline classes are unchanged, so 2.0 and 2.3 load as before.
Image conditioning is now re-compressed to match training, at CRF 18 for 2.5
and 33 for earlier versions.
Some paths left both to pipeline defaults, which track the current upstream
model: distilled picked up guidance it already bakes in, and 2.0 could land on
2.3's joint-sigma path.
The modular branch returned the pipe instead of assigning it, so the video
tab and the API failed with "model not loaded" and the whole post-load tail
was skipped along with it.
Vladimir Mandic
Christopher McGee
Dity0
CalamitousFelicitousness
Oleksandr Liutyi