Triton cannot compile e4m3 loads before sm_89, so fp8 weights fall back
to eager dequant there. The uint8-backed float8_e4m3fn_sdnq codec decodes
identically now that subnormals are handled (the two NaN codes become
+/-480), and its compiled dequant runs about 6x faster than eager native
fp8. Pre-quantized fp8 layers are viewed as uint8 at adoption when
compiled dequant is enabled on such hardware; the eager gate remains the
safety net for every other fp8 path.
detect_quant now reads _quantization_metadata as the authoritative quant
source when present, and resolves marker-file formats from the marked
layers' stored weight dtypes instead of file-wide dtype voting, which
mislabeled fp8 files carrying extra uint8 tensors and had no nvfp4
mapping at all. Schema bump so cached probe entries refresh.
nvfp4 layers keep their packed 4-bit codes and land on SDNQ grouped
quantization as float4_e2m1fn: the nibble order is swapped once at load,
the e4m3 block scales are unswizzled from the cuBLAS tile layout, and
the fp32 global scale folds into them as per-group scales. Marker
orig_shape acts as a cross-check and alignment padding is sliced
against the model dimensions.
- reject nvfp4 markers with unknown group sizes or convrot flags
- accept uint8 storage in the krea2 real-file check
Newer quantized checkpoints record per-layer formats in the safetensors
header _quantization_metadata instead of marker tensors. The native loader
now reads the header map, re-keys it through the same prefix strip as the
tensors, and transcodes it into marker tensors so both container forms
share one detection path; header entries win over markers.
- drop optional input_scale sidecars for marked layers
- map full_precision_matrix_mult onto the sdnq per-layer matmul exclusion list
- log the detection source (markers, header, both)
ConvRot is the regular Hadamard rotation SDNQ implements: identical
construction, normalization, axis, and dequant order. Per-layer markers
map onto the dequantizer's use_hadamard and hadamard_group_size; group
sizes must be powers of 4 and divide in_features, else base-repo
fallback. Detection carries per-layer metadata since files mix plain
and rotated layers.
One UNET override cannot serve dual-transformer arches: ideogram4
conditional/unconditional and wan combined-stage experts need separate
files, and previously a single override landed on both experts.
- sd_unet_secondary option with per-slot tracking, consumed-state sync,
arch-change reset, and incompatible-override fallback
- dropdown renders beside the primary, follows it into quicksettings,
and is visible only for dual-transformer model types
- ideogram4 native single-file spec with a quant-aware fused-qkv
converter; such converters run before comfy_quant detection via
TransformerSpec.converter_handles_quant
- quicksettings render in configured order (sort keyed on the option
object and always fell back to alphabetical)
- post-load dtype warning skips quantized transformers
load_transformer consumes the sd_unet dropdown selection during a full
model load but never marked it as loaded, so the queued sd_unet
onchange callback always forced a second full reload. Sync
sd_unet.loaded_unet once the override is successfully consumed; the
incompatible-override fallback keeps its reset to Default.
Parameterize the pre-quantized comfy_quant path by format so fp8
tensorwise checkpoints load alongside int8; both map onto SDNQ's
symmetric dequant for the matching weights dtype. Containers are
mislabeled in the wild, so the stored weight dtype is validated
against the marker, mixed-format files are rejected, and markers
carrying a convrot rotation flag fall back to the base repo since
they may require runtime inverse rotation.
Detect comfy_quant markers in native single-file transformer loads and
adopt the pre-quantized tensors as SDNQ int8 layers instead of rejecting
them. ComfyUI int8_tensorwise is a strict subset of SDNQ symmetric int8,
so weights and scales are taken bit-exact with no dequantize-requantize
round trip; quantized matmul and scale-dtype settings apply as usual.
- detect_comfy_quant/remap_comfy_quant helpers plus a prequantized
builder in native_transformer; file markers dictate the layer set
- unsupported formats and wrong-arch markers fall back to the base repo
- offline unit tests and an opt-in real-file krea2 test
All-in-one exports bundle the text encoder and VAE alongside the
transformer under LDM-style family prefixes (cond_stage_model.,
first_stage_model., text_encoders., vae.). The native loader treated
those keys as a mixed-prefix error and rejected the file. Drop known
companion families before prefix detection and log what was skipped;
keys matching neither a transformer prefix nor a known family still
raise. TE and VAE keep coming from the base repo or their own overrides.
strip_prefix returns the detected prefix instead of logging it, so the load
summary reports it alongside the key count and the redundant reading-state
marker is gone. Four near-identical cls/file lines become one.
Base ships last.up all-zeros so the branch is a no-op; pre-branch
finetunes omit both keys. A zero_init_missing spec field zero-fills
them on load instead of falling back to the base transformer.
Route nn.Embedding targets (and the SDNQEmbedding / ScaledWordEmbedding subclasses) through the linear LoRA path: the weight delta is up@down over the [vocab, dim] table, same shape and merge as a Linear.
Apply a companion bias delta (diff_b) as ex_bias on the same module rather than dropping it; collect diff_b into the LoRA group so it rides the existing module instead of a separate Full module that would collide on the network key.
Krea 2 is a 12.9B single-stream flow-matching DiT trained from scratch, using a Qwen3-VL-4B text encoder and the Qwen-Image VAE. The transformer is vendored as a diffusers ModelMixin whose module tree mirrors the checkpoint, so weights load with no key conversion; the pipeline ports the reference encode, flow-matching denoise, and VAE decode. The text encoder is shared at runtime via the existing dedup registry, so Base and Turbo reuse one Qwen3-VL-4B copy.
Covers text-to-image, image-to-image, native LoRA, and the single-file UNET override. Also completes SD.Next's partial Qwen-Image VAE support (5D decode input and TAESD preview mapping) that K2 shares.
transformer., bare-diffusers, and lora_transformer_ bases are already in
network-key form for every arch, yet each per-arch resolve_targets repeated the
same passthrough branch for them. Move that into a shared
PASSTHROUGH_PREFIXES_DEFAULT set consulted by resolve_group_targets, leaving each
arch's resolve_targets to only the prefixes it actually rewrites (kohya / BFL).
lycoris_ stays in flux2, the one arch that recognizes it.
Pure refactor: the same keys resolve to the same modules.
Add an end-to-end regression per diffusers arch (chroma, flux2, zimage, ernie)
that loads a lora_transformer_ diffusers-flat state dict and checks the
expected modules bind. The chroma suite also asserts the shared
resolve_group_targets passthrough runs above each arch's own resolve_targets,
which returns nothing for this prefix.
Drives txt2img/img2img with the full per-sampler scheduler-option matrix
(sigma method, prediction type, timestep spacing, beta schedule, solver
order, shift, low order, thresholding, dynamic shift, rescale betas), one
option at a time over a pinned baseline, and measures every case against
the default: applied, inert, marginal, rejected, or fallback. Detects
silent fallback via the infotext Scheduler class and annotates every saved
image with its full requested configuration. --sweep runs a reduced matrix
per sampler and emits an empirical capability report.
ERSDEScheduler now accepts use_karras_sigmas, use_exponential_sigmas,
use_beta_sigmas, and use_flow_sigmas, matching the other flow schedulers.
The VP path derives alpha/sigma/lambda from the k-diffusion sigma so the
karras/beta/exponential transforms can use fractional timesteps; the
default schedule is numerically unchanged. use_flow_sigmas triggers flow
mode and add_noise tolerates fractional timesteps.
Wire the new keys into the ER-SDE presets so the sigma method selector
drives them, and cover ER-SDE in the scheduler stability test.
validate_sampler_name only matched the exact, case-sensitive name, so near-miss client names such as lowercase variants were rejected while an omitted name silently used the model scheduler via the Default sentinel. Fall back to find_sampler and return the canonical name so create_sampler applies the intended sampler; unknown names still return 404. Add an API test covering case-insensitive resolution and rejection of unknown names.
diffusers shipped first-party Ideogram 4 (transformer + pipeline) in
9b0818cf, so drop the in-tree port and keep only SD.Next integration glue.
Bump the diffusers pin to 9b0818cf and build diffusers' Ideogram4Pipeline
from a thin loader with per-transformer SDNQ. A small subclass keeps the
text encoder resident for the Qwen3-VL tap under balanced offload, and the
step callback denormalizes the preview latent from vae.bn before unpatchify.
Deletes the ported transformer, pipeline, scheduler, text encoder, and
latent-norm constants.
A converter error or load_state_dict mismatch now raises OverrideArchMismatch, which load_transformer catches to drop the override and load the base transformer. No per-arch markers to maintain.
Surface YoloRestorer.restore() as a standalone operation: a Detailer
postprocessing script in the Process tab and a thin /sdapi/v1/detail
endpoint, neither requiring a base generation pass.
- modules/postprocess/yolo.py: YoloRestorer.make_processing() builds the
synthetic Img2Img processing object both entry points feed to restore(),
resolving the seed so the inpaint passes are reproducible
- modules/api/process.py: post_detail handler exposes the full detailer
parameter set and returns the detailed image plus optional annotations
as base64
- scripts/postprocessing_detailer.py: reuses shared.yolo.ui('extras') and
runs through make_processing()
- modules/postprocessing.py: run_extras takes a per-script script_args
dict, also letting the extras API drive other scripts such as Remove
background; omitting it leaves existing callers unchanged
- modules/api/models.py: ReqDetail / ResDetail
- modules/processing_info.py: guard create_infotext's Image/Hires CFG
reporting against an unset (None) cfg_image, matching the is-not-None
checks the other cfg_image readers use; the detailer inpaint pass runs
with it unset
- test/test-detailer-api.py: covers both paths; effect tests measure the
diff inside the detected region with extreme isolated parameter values,
and the suite disables model quantization for the run and restores the
original settings afterward
load_transformer threads **kwargs into its from_pretrained and from_single_file branches. The native dispatch branch bypasses both loaders and builds via cls.from_config, so those kwargs were dropped there. Thread them through load, build_component, and build_component_quantized into from_config so callers passing extra args alongside native_spec are honored instead of silently dropped. Siblings do not receive them.
217-line bespoke loader collapses to a 40-line ANIMA_SPEC in
pipelines/anima/__init__.py (Cosmos converter + llm_adapter sibling +
Cosmos 1.0 forbidden marker).
Drop the class-keyed REGISTRY: Anima and raw Cosmos share
CosmosTransformer3DModel but need different specs. Specs pass via
explicit native_spec= kwarg; make_default_spec(cls) covers the
auto-converter case.
Replaces anima_lora.py's bespoke try_load_lora / group_keys /
resolve_network_key with thin wrappers binding native_loader's generics
to anima's prefix tuples and resolve_targets, mirroring flux2 / zimage /
chroma / ernie. The hand-rolled apply_lora_alphas bake-with-balance
pass goes away; alpha / scale / dora_scale flow through NetworkWeights.w
to NetworkModule.calc_scale at apply time.
native_loader gains an optional network_prefix kwarg (str or
Callable[[prefix_used], str], default "lora_transformer_") used when
constructing network_key. Anima passes a callable picking
lora_transformer_ / lora_llm_adapter_ / lora_te_ per matched prefix.
Single-component siblings keep the default and are unchanged.
network.NetworkModule.apply_weight_decompose grows a dual-path DoRA
convention detector. The pre-fix implementation only handled per-input
dora_scale (DoRA paper / kohya, shape (1, in)), silently broadcasting
per-output LyCORIS / PEFT dora_scale (shape (out, 1)) into an incoherent
element-wise rescaling. Detection is structural: (out, 1, ...) routes
to per-output; everything else (including the square-weight 1D ambiguity)
defaults to per-input for legacy compat. Pre-existing bug surfaced by
the LoKR+DoRA LyCORIS files Anima now loads.
Behavior changes:
- LoHA via the generic try_load_loha (NetworkModuleHada); covers
scenery-anima-base and any other LyCORIS .hada_w* export.
- Kohya lora_te_ prefix recognized. The legacy resolver only matched
BFL text_encoders.qwen3_06b.transformer.model. and silently dropped
lora_te_layers_N_* keys (41% of BlueArcStyle's bases were unloaded).
- LoKR+DoRA LyCORIS files now apply correctly; the per-output dora_scale
is honored instead of silently scrambled.
Adds test/test-anima-native-adapters.py: 37 offline tests across all
five prefixes (LoRA + LoHA), every COSMOS_2_FLAT_RENAME entry, DoRA
threading, marker disambiguation, try_load_chain dispatch, calc_updown
sanity, and both DoRA conventions (per-input / per-output / 1D ambiguous).
Adapter mock mirrors AnimaLLMAdapter's real module tree.
Replaces ernie's four family loaders with thin wrappers binding
native_loader's generics to ernie's prefix tuples and resolve_targets.
ErnieImageAttention has fully split to_q / to_k / to_v with no fused QKV
and ErnieImageFeedForward has three separate Linear modules, so
resolve_targets is a straight passthrough across every recognized prefix.
BARE_DIFFUSERS_PREFIXES covers layers., adaLN_modulation., final_norm.,
final_linear. for bare-diffusers exports (e.g. via save_lora_adapter).
parse_key returns (prefix_used, base, suffix) instead of the old
(network_key, suffix); parse test updated.
Replaces chroma's four family loaders with thin wrappers binding
native_loader's generics to chroma's prefix tuples and resolve_targets.
resolve_targets folds the Flux-to-diffusers rename table and the two
fused-weight splits into one path-resolution step:
- img_attn.qkv / txt_attn.qkv: ChunkSpec(idx, total=3) for equal Q/K/V
- single_blocks.linear1: ChunkSpec(start, end) for the unequal
Q/K/V/proj_mlp partition
QKV_DIMS and LINEAR1_DIMS stay as module-level constants (tests patch
them for the scaled-down mock).
Behavior changes:
- LoKR on equal-chunk QKV now dispatches to NetworkModuleLokrChunk
instead of NetworkModuleLokrSliceChunk (the slice variant was used
pre-migration for both forms since chroma had no equal-chunk path).
- LoHA on fused img_attn.qkv now binds via NetworkModuleHadaChunk
instead of being skipped; the shared HadaChunk added in the flux2 PR
is general for equal-chunks dispatch.
parse_key returns (prefix_used, base, suffix) instead of the old
(flat_key, suffix); rename happens in resolve_targets. test_static_rename_table
replaced with test_resolve_targets_static_renames driving the same
remappings through the new interface.
Replaces zimage's four family loaders with thin wrappers binding
native_loader's generics to z-image's prefix tuples and resolve_targets.
resolve_targets folds the legacy attention.qkv split and attention.out
alias rename into the path-resolution step. Fused qkv now emits three
ChunkSpec(idx, total=3) entries; attention.out / attention.out.0 /
attention.wo aliases collapse to attention.to_out.0.
BARE_DIFFUSERS_PREFIXES allows bare paths starting with layers. /
noise_refiner. / context_refiner. to pass through to the loader. This
matches real Z-Image LoRAs exported via
ZImageTransformer2DModel.save_lora_adapter().
LoHA on fused qkv now binds via NetworkModuleHadaChunk (added to shared
infra by the flux2 PR) instead of being skipped. Test renamed to
test_loha_legacy_fused_qkv_chunked.
parse_key returns (prefix_used, base, suffix) instead of the old
(network_key, suffix); parse test updated.
Covers ernie's four-family surface (LoRA, LoKR, LoHA, OFT).
ErnieImageAttention has fully split to_q / to_k / to_v with no fused QKV,
and ErnieImageFeedForward has three separate Linear modules. The loader
has no chunking, renames, or fused-target dispatch.
Mock matches diffusers.ErnieImageTransformer2DModel
(ErnieImageSharedAdaLNBlock with self_attention + mlp + RMSNorms, plus
module-level adaLN_modulation Sequential and final_linear).
Formats exercised:
- BFL / AI-toolkit
- kohya
- BFL LoKR
Covers chroma's four-family surface plus the Flux-to-diffusers rename
and the unique single-block linear1 unequal-chunk slicing.
Mock transformer matches diffusers.ChromaTransformer2DModel
(transformer_blocks with FluxAttention(added_kv_proj_dim), single
transformer_blocks with pre-only attn + proj_mlp + proj_out, plus
distilled_guidance_layer).
Formats exercised:
- BFL / AI-toolkit
- kohya
- PEFT
- LyCORIS oft_diag
install_mock_pipe patches chroma_lora.QKV_DIMS and LINEAR1_DIMS to the
test scale (HIDDEN=96, MLP_HIDDEN=384); the module otherwise hardcodes
Chroma1-HD's 3072 / 12288.
Replaces flux2's eight family loaders with thin wrappers binding
native_loader's generics to flux2's prefix tuples and resolve_targets.
Constants and helpers (has_marker, parse_key, group_by_suffixes) are
re-exported from native_loader so the offline tests address them through
flux2_lora's surface unchanged.
resolve_targets now returns (diffusers_path, ChunkSpec | None) tuples
instead of (path, idx, num_chunks). Three parse-level tests updated.
PEFT-fallback path (apply_lora_alphas, preprocess_f2_keys, apply_patch)
stays in flux2_lora.
CalamitousFelicitousness
Vladimir Mandic