The generate-time gate compared the stored processor name against the
sdp_overrides list, which can never be equal, so the check reduced to
the processor name alone and a changed override set was never applied
until the next model load. set_diffusers_attention now stamps the
override set it applied beside the processor name, the gate compares
both, and pipe switches carry the new attribute with the old one.
On ROCm, host-to-device DMA from mmap'd safetensors pages stalls ~1s per
copy, so weights move to the GPU at ~27 MB/s instead of ~28 GB/s. With
offload enabled this re-copies weights every forward, so generation
appears to hang.
Adds `diffusers_disable_mmap` (Settings > Model Loading), off by default,
which makes diffusers read shards into anonymous memory instead. Costs
peak RAM equal to the model size, so it is opt-in.
SD3.5-large on RX 9070 (gfx1201), same prompt and steps:
off: no image after 120s
on: image in 20s
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Group offload ignored the offload exclusion settings and its options took
effect only at the next model load. Components named by the never-offload
list or the model-type exclusion now stay resident on the accelerator,
with matching shared between the balanced hook and the group roles. The
offload options re-place loaded components when changed: group modes
re-run the placement pass, and balanced mode rebuilds its hook, which
snapshots the exclusion lists at construction.
None from a family loader means unhandled, so the chain falls through to the folder
loader, which for a modular pipe builds an object holding only its from_config helpers
and installs it. Refuse it when none of the from_pretrained specs materialized.
The local-folder skip in hf_auth_check keyed on model_index.json only,
so modular pipeline folders fell through to a hub auth check with a
filesystem path as the repo id.
Saving a modular pipeline now rewrites the component references in its
index to the destination folder, so a reload uses the saved quantized
weights instead of following the specs back to the source repositories.
The folder scan accepts modular_model_index.json for direct folders,
matching the snapshot branch, so saved pipelines list in the model
dropdown.
Modular pipelines run every stage inside one pipeline call, leaving
the ui on a single inference label. Forward hooks on the text encoder,
transformer and vae decoders now surface the current stage through
shared state, and the interrupt check runs in every stage so encodes
and tiled decodes abort promptly.
- saving a model registers a job instead of appearing idle
- group offload logs each component before the pin step instead of
only after completion
Reference entries for the bf16 repo and the sdnq uint4 quant load the
modular pipeline through the standard dispatch. Image tabs run the
model in still mode with audio off; the video tab keeps its own
overrides through the shared per-generation hook. Detailer is not
supported and is disabled with a warning.
First natively modular model: the pipeline is driven directly through
ModularPipeline, with components fetched per workflow (fl2va covers
text and first/last-frame conditioning).
- per-generation overrides snap the canvas to /32, align frames to
the 17n+5 grid and duration window, and keep the bespoke scheduler
pair
- group offload for modular pipelines applied per component in
sd_offload; re-application is a guarded no-op
- audio checkbox pops the audio decode block so decode and muxing are
skipped
- frames=1 renders a single still image: the duration floor lifts per
instance and sub-floor latents pad at the vae decoder
- progress and interrupt handling via a transformer forward pre-hook
- vae scale factor override, tuple-safe patch size
Model switches kept most of the previous model resident, and the next
load could stall in kernel reclaim while the freed memory was still held.
- strip group offload hooks in disable_offload so the meta move at unload
actually frees component weights; hook removal resolves wrapper
components that carry hooks on the inner model
- flush the torch pinned host cache in torch_gc so freed streaming
buffers return to the OS instead of staying cached in-process
- skip the pipe-level accelerator move for group-managed pipes: the
offload engine already placed every component, and the move only
dragged on-demand components to the accelerator for the trailing
eviction to undo
Tab runners pre-move the vae module to the gpu before generation,
which parks an on-demand component on the accelerator for the whole
denoise. Moves of a stamped module toward the accelerator now return
early; the entry bridge onloads it when its encode or decode runs.
A pipeline-level move to the accelerator carries on-demand components
along with it, since modular pipelines skip only group-hooked modules;
the load path then left a resting vae on the gpu until its first use.
Bulk moves now re-evict stamped components, whose entry points onload
them when needed.
Vae-class components never take group hooks, so group mode kept them
resident on the gpu; a MiniMax-class video vae holds about 10GB that
way while running only seconds per generation. Components above 1GB
now rest in system memory: the apply_forward_hook bridge on encode and
decode fires an on-demand hook that moves the whole module to the
device, so tiled calls find every weight already loaded, and the
processing seams return it to cpu once outputs are materialized. Small
vaes stay resident since the transfer would cost more than it frees.
- placement is decided per component by measured size and requires the
entry bridge; components without it stay resident
- move_model no longer forces on-demand vaes to the gpu for
non-txt2img tasks, and full_vae_encode onloads before binding the
input, which otherwise lands on the resting device
- mode switches clear the stamp and hook in both directions
Group offload hooks report the onload device at module level while the
weights rest on cpu, so every native apply took the parameter
replacement branch in assign_weight and detached the written layers
from the hook's group bookkeeping. The activation and deactivation
walks now remove a component's group hooks before its first weight
write and reapply offload at the end of the pass: writes land in place
on the resting tensors and fresh groups snapshot the result.
- hooks come off lazily, only for components with a covered layer or a
pending backup or factor-stash restore; repeat activations with an
unchanged set leave the hooks untouched
- remove_group_offload_component follows wrapper components to the
inner model that carries the hooks
Dynamo tracks a lifetime recompile counter per compiled function that
freed models leave climbing while their graphs and guards die, and the
compiled dequant runs fullgraph, so crossing the accumulated limit is a
hard FailOnRecompileLimitHit instead of an eager fallback; enough model
or quant switches in one process got there. unload_model_weights now
calls reset_compile_caches when the compiled dequant is active, dropping
the dead graphs and the counters in the same sweep as the unload gc.
Raised limits only move the wall; the reset removes it.
- scoped to the model unload branch: the reset is global and must only
run when the graphs' owner is being discarded
- regression test trips the wall under a lowered limit and recovers
through the same helper the unload path calls
One UNET override cannot serve dual-transformer arches: ideogram4
conditional/unconditional and wan combined-stage experts need separate
files, and previously a single override landed on both experts.
- sd_unet_secondary option with per-slot tracking, consumed-state sync,
arch-change reset, and incompatible-override fallback
- dropdown renders beside the primary, follows it into quicksettings,
and is visible only for dual-transformer model types
- ideogram4 native single-file spec with a quant-aware fused-qkv
converter; such converters run before comfy_quant detection via
TransformerSpec.converter_handles_quant
- quicksettings render in configured order (sort keyed on the option
object and always fell back to alphabetical)
- post-load dtype warning skips quantized transformers
diffusers pipeline downloads build subfolder config.json allow-patterns
with os.path.join, and huggingface_hub>=1.22 matches patterns with
fnmatchcase which does not normalize path separators
(huggingface/huggingface_hub#4435). On windows the resulting backslash
patterns match nothing, so per-component config.json files are never
downloaded and the incomplete snapshot still passes the diffusers
cache-completeness check, failing every subsequent load with
"no file named config.json".
Prefetch component configs with forward-slash patterns before pipeline
load. This covers all model families and also repairs snapshots already
broken by the bug on the next load attempt. No-op on linux, in offline
mode, and for local folder or single-file models.
When a checkpoint change switches the model type, a custom sd_text_encoder no
longer fits, so reset it to Default and clear loaded_te, mirroring the sd_unet
reset. The type is resolved with detect_pipeline on both the loaded and incoming
checkpoints, so same-arch switches (Krea2 Base and Turbo share one pipeline
class) do not reset. The checkpoint handler also returns sd_text_encoder
alongside sd_unet so the dropdown reflects it.
reload_text_encoder only hot-swapped T5-family encoders and ran only at initial
load, so changing sd_text_encoder for a model with a generic encoder (Krea2's
Qwen3-VL) never took effect until a full model reload. Track the loaded
selection and, for encoders with no in-place swap, reload the model on change,
triggered from the settings handler. The fresh model object also invalidates
the prompt cache.
Krea 2 is a 12.9B single-stream flow-matching DiT trained from scratch, using a Qwen3-VL-4B text encoder and the Qwen-Image VAE. The transformer is vendored as a diffusers ModelMixin whose module tree mirrors the checkpoint, so weights load with no key conversion; the pipeline ports the reference encode, flow-matching denoise, and VAE decode. The text encoder is shared at runtime via the existing dedup registry, so Base and Turbo reuse one Qwen3-VL-4B copy.
Covers text-to-image, image-to-image, native LoRA, and the single-file UNET override. Also completes SD.Next's partial Qwen-Image VAE support (5D decode input and TAESD preview mapping) that K2 shares.
Diffusers-native port of the 9.3B flow-matching DiT: dual-transformer asymmetric CFG, a 13-layer Qwen3-VL tap encoder deduped with VQA and prompt-enhance, the Flux.2 VAE, and a logit-normal schedule. Loads a published bf16 repo with SDNQ at load.
CalamitousFelicitousness
Vladimir Mandic
Christopher McGee
Dity0