A full-weight extraction on Z-Image bound 308 modules and applied 172 of
them, silently dropping the rest, and left 71 more unmapped.
assign_network_names_to_compvis_modules puts every transformer module in
network_layer_mapping but skips stamping network_layer_name on norms,
which is the attribute the apply pass keys off. try_load_full bound those
modules through the mapping and they then never applied; stamp them
loader-locally, as try_load_norm already does.
Z-Image also names three module groups differently from the diffusers
tree: the qk-norms (q_norm/k_norm vs norm_q/norm_k), and the patch
embedder and final layer, which live in ModuleDicts keyed by
"{patch_size}-{f_patch_size}" and so carry a key the checkpoint has no
notion of. Read that key from the live model rather than hardcoding it.
The counts close exactly: 68 qk-norms plus 3 non-block targets are the 71
that went unmapped.
The linear layers bind their scaled-mm function at import, so
SDNQ_USE_TRITON_MM freezes the backend per process and comparing triton
against the torch fallback meant two runs, carrying clock drift into the
delta. --mm-backends rebinds the function on the consuming modules
between benches, so both rows measure the same quantized layer under the
same clock state.
- swap targets are the four linear modules that from-import a scaled-mm
function; every swap resets dynamo, since the layer forwards are
compiled and the traced graph pins the previous function
- the torch row is captured from kernel_wrappers rather than
reimplemented, so it is unavailable where triton is the device default
(rerun with SDNQ_USE_TRITON_MM=0) and says so
- rounds alternate order and keep the fastest per row, so drift cancels
instead of favouring whichever backend runs second
- flag output error when backends disagree past 1e-4: they are meant to
be numerically equivalent
probe_fp8 and the broken-compile bench path forced eager input prep by
toggling torch._dynamo.config.disable. torch 2.13+ raises 'found no
compiled frames' when a fullgraph-compiled function is called inside a
disable window, so the probe failed before reaching the attention kernel
and reported an environment error instead of the kernel's real verdict,
on fp8-capable gpus too. Swap the module-global prep function for its
eager inner instead, matching the correctness section.
The environment panel and json now carry python, platform, backend and
the backend runtime versions (cuda/cudnn/driver, hip, ipex, openvino,
directml via devices.get_gpu_info), so shared reports identify the stack
without inferring it from the torch version string.
The webui bootstrap log is gated behind an fd-level suppression for the
import window: the bootstrap reconfigures its loggers during import and
onnxruntime warns from c++, so logger levels alone cannot silence it.
Import failures still print after the gate lifts.
zimage, chroma, ernie and krea2 chained only lora/lokr/loha/oft while
flux2 and anima ran all eight families, so ia3/glora/norm/full files
(e.g. full-diff extractions with diff/diff_b keys) reported not loaded
on the short-chain arches. The generic family loaders are arch-agnostic;
wire the missing four into each chain.
- add ia3/glora/norm/full wrappers and chain entries to the four arch
modules, with matching suffix/marker re-exports
- add a chain-level full-diff test per suite: zimage covers the legacy
attention.out alias and the fused-qkv skip, chroma the proj rename,
ernie the passthrough
LyCORIS extraction with use_sparse_bias saves bias_indices/bias_values/
bias_size per module: the sparse weight-shaped remainder of the SVD
extraction, named bias for historical reasons. The keys were dropped by
both loader paths, so extracted adapters applied without the residual
correction; the dense-bias branch in finalize_updown that consumes it
was unreachable.
- rebuild the COO tensor in NetworkModule.__init__ (int16 indices cast
to long), shared by the native and generic loaders; kept sparse so
the dense += sparse in finalize_updown materializes per module at
apply instead of near-model-size densification at load
- accept the triplet suffixes in LORA_SUFFIXES; fused targets skip
with the weight-shaped-bias warning
- cover an extraction-faithful numeric round-trip and the fused skip
in the offline suite
ai-toolkit DoRA saves lora_A/B plus a 1-D per-output magnitude key in
place of alpha; PEFT and diffusers name the same quantity
lora_magnitude_vector. Neither key was in the suffix table, so such
adapters loaded as plain LoRA with the magnitude renormalization
silently missing. The semantics match LyCORIS wd_on_out=True row norms,
so both keys convert onto the existing dora_scale path.
- accept .magnitude and .lora_magnitude_vector in LORA_SUFFIXES and
convert at group level in try_load_lora
- reshape 1-D vectors to (out, 1): on square layers the apply-time
orientation detection would otherwise renormalize the wrong axis
- cover square-layer numeric equality, fused-qkv slicing and the PEFT
key form in the offline suite
finalize_updown ran apply_weight_decompose on the unscaled delta and
multiplied the result by alpha/rank afterward. LyCORIS and ComfyUI both
bake alpha/rank into the diff before computing the row norms, so any
DoRA with alpha != rank renormalized against the wrong merged weight
(64% relative delta error for kohya-style alpha=1 rank=8; exact only
when alpha == rank, which full-matrix LoKR forces).
- scale updown by calc_scale() before apply_weight_decompose; apply
only the multiplier afterward
- multiplier lerps the full merged delta (0 disables, 1 equals the
trainer output); LyCORIS weight-mode ratio interpolation leaves the
diff applied at multiplier 0 and is not used
- add a numeric regression test mirroring the LyCORIS forward reference
A diff_b bias delta on a fused BFL target passed through whole and
failed at apply with a shape mismatch. diff_b stores one value per
output feature, so it partitions with the fused rows exactly like the
up-weight; slice it with the chunk in the LoRA loader. The legacy
weight-shaped bias key (LyCORIS sparse-residual heritage) has no
defined partition on a fused target and no known emitter pairs it with
chunk-capable families, so the group is skipped with a warning in the
LoRA, LoKR and LoHA loaders.
- add slice_bias_delta beside slice_dora_scale; warn and skip
non-per-output diff_b shapes
- cover sliced diff_b flowing out as ex_bias and the legacy-bias skip
in the offline suite
BFL-format chroma adapters targeting the embedders, final projection or
the distilled guidance layer MLPs resolved verbatim and unmapped: the
embedder/final-layer names differ from diffusers outright, and the
approximator MLP leaves are in_layer/out_layer in BFL but linear_1/
linear_2 in the diffusers PixArt projection. Only in_proj, out_proj and
norms.N shared names and bound.
- add CHROMA_EXTRA_MAP (kohya form derived) and GUIDANCE_LEAF_MAP to
both target resolvers
- route bare distilled_guidance_layer keys through the resolver instead
of the diffusers passthrough so both leaf namings resolve; add
img_in/txt_in/final_layer bare prefixes
- cover all key forms and end-to-end binding in the offline suite
A LoKR group whose Kronecker product does not fit the resolved module
previously bound anyway and failed at apply time as a caught per-module
error, leaving the adapter partially applied with only an error log.
Reject the group at load with a warning instead, matching the LoRA
path's shapes_match gate.
- lokr_kron_shape derives (out, in_flat) from full, rank-decomposed or
Tucker-rebuilt factors, folding conv kernel dims into in_flat
- lokr_shapes_match honors SDNQ original shapes and chunk partitions:
equal chunks need total * out rows, row-range slices an exact range;
the input dim is never chunked
- cover non-fused and fused rejection in the offline suite
LyCORIS wd=True saves a dora_scale companion for LoRA/LoHA/LoKR; on
fused BFL targets the chunk paths passed it through whole, so apply
failed with a shape mismatch and the module was dropped. Per-output
magnitudes (wd_on_out=True, the default) partition exactly with the
fused rows; per-input magnitudes couple the chunks through shared
column norms and have no exact split.
- slice per-output dora_scale rows with the chunk in the LoRA, LoKR
and LoHA loaders
- skip per-input DoRA on fused targets with a specific warning
- cover sliced and skipped orientations in the offline suite
The lycoris_ save format is arch-independent: LyCORIS standalone wraps
the loaded diffusers model and emits the wrapped module path with dots
as underscores, so verbatim passthrough is correct for any arch. Only
flux2 handled it; zimage, chroma, ernie and krea2 reported such files
as not loaded.
- add lycoris_ to KNOWN_PREFIXES_DEFAULT and PASSTHROUGH_PREFIXES_DEFAULT
- drop flux2's per-arch prefix append and resolve_targets branch
- add lycoris_ to ANIMA_PREFIXES (anima replaces the default tuple);
network_prefix_for already routes it to the transformer namespace
- cover the passthrough with a zimage loader test
BFL-format adapters targeting the embedders, timestep/guidance MLPs,
modulation layers and the final layer resolved to nothing and were
dropped as unmapped, for every adapter family on the f2 native path.
- add F2_EXTRA_MAP exact-match lookups in both target resolvers, with
the kohya underscore form derived from the BFL path
- add guidance_in. to BARE_FLUX_PREFIXES; groups targeting the guidance
embedder stay unmapped on models built without guidance_embeds
- extend the offline test mock with the non-block targets and cover all
three key forms plus full-matrix LoKR with placeholder alpha
After the correctness section aborts on a sticky kernel fault, the
remaining sections cannot run; flush outputs and exit instead of
cascading into the next cuda call's traceback.
A kernel fault (misaligned address, illegal memory access) is sticky:
every later cuda call in the process fails, so one faulting check killed
the whole run through the next reference computation and lost the --save
and --json outputs. Compute the correctness reference inside the
per-check handler, detect a dead context and abort the section with a
clear message, and flush partial outputs from a top-level handler on any
crash or interrupt.
The fp8 attention probe can fail for reasons other than missing hardware
support (torch or triton compile issues); reporting those as 'not supported
on this gpu' is wrong on gpus where fp8 is native. Match the triton
pre-sm_89 signature and label anything else as an environment failure.
Toggling torch._dynamo.config.disable to force eager input prep fails on
newer torch: a fullgraph-compiled function called inside a disable window
raises 'found no compiled frames', failing every check and poisoning the
first compiled call afterwards. Swap the module-global prep function for
its eager inner instead, matching the dequant section's mechanism.
- store per-check errors and exceptions in the json report, so remote
reports carry the failure mode
The refine toggle doubles as the slot selector on the UNet/DiT page:
with it active a card click sets sd_unet_secondary instead of sd_unet,
mirroring base/refiner selection on the model page.
Benchmark weight dequantization alongside attention: eager vs compiled
dequantization, measured standalone and through the full linear forward,
plus the quantized matmul forward, across int/uint 8-6-4-2 and the
pre-quantized float formats (float8_e4m3fn, float8_e4m3fn_sdnq,
float4_e2m1fn group 16) at krea 2 layer geometry, against a bf16 nn.Linear
baseline and a true-fp32 output reference. Each row carries measured
storage size and one-shot quantize time; an svd/hadamard variants table
measures rotation and low-rank costs on top of the base dtypes.
A combined block section measures complete configurations (weights dtype x
matmul path x attention) end to end through a dit-style transformer block,
with output error at depth one and four against an fp32 reference block,
because component speedups and errors do not compose multiplicatively.
- recommendations weigh error against speed: a faster option is rejected
when it multiplies measured output error beyond 2x, and reason strings
cite both numbers; notes include measured size/error and speed/error
frontiers
- robustness: extreme-activation stress rows in the correctness matrix
(pass on finite output), max-token-error columns beside norm error, and
non-finite outputs labeled as verdicts; norm metrics alone hide
token-level corruption on outlier-heavy inputs
- probe compiled weight dequant for e4m3 and e5m2 storage and report the
fp8 compile gate status; on ampere e5m2 compiles while e4m3 does not
- new attention presets: sdxl-cross (cross-attention), qwen3-te (causal
gqa text encoder), wan22-cfg (batched cfg video); cross-attention
correctness check
- --sections, --dequant-dtypes, --dequant-variants and --block-configs
selectors, --json structured results output
Triton cannot compile e4m3 loads before sm_89, so fp8 weights fall back
to eager dequant there. The uint8-backed float8_e4m3fn_sdnq codec decodes
identically now that subnormals are handled (the two NaN codes become
+/-480), and its compiled dequant runs about 6x faster than eager native
fp8. Pre-quantized fp8 layers are viewed as uint8 at adoption when
compiled dequant is enabled on such hardware; the eager gate remains the
safety net for every other fp8 path.
detect_quant now reads _quantization_metadata as the authoritative quant
source when present, and resolves marker-file formats from the marked
layers' stored weight dtypes instead of file-wide dtype voting, which
mislabeled fp8 files carrying extra uint8 tensors and had no nvfp4
mapping at all. Schema bump so cached probe entries refresh.
nvfp4 layers keep their packed 4-bit codes and land on SDNQ grouped
quantization as float4_e2m1fn: the nibble order is swapped once at load,
the e4m3 block scales are unswizzled from the cuBLAS tile layout, and
the fp32 global scale folds into them as per-group scales. Marker
orig_shape acts as a cross-check and alignment padding is sliced
against the model dimensions.
- reject nvfp4 markers with unknown group sizes or convrot flags
- accept uint8 storage in the krea2 real-file check
Triton has no e4m3 conversions before sm_89, so any compiled graph
touching fp8 storage weights fails with an InductorError on Ampere.
Select the eager dequant and re-quantize paths for e4m3 weights when
the hardware cannot compile them; other dtypes keep compiled dequant.
SDNQ_ALLOW_FP8_COMPILE overrides the detection.
The secondary unet loads from the networks panel toggle or the settings
page; companion ordering and dynamic visibility pushes in ui_settings.py
are not worth their footprint for one setting.
Vladimir Mandic
CalamitousFelicitousness
Disty0