A delta that does not fit its target module cannot apply, and applying only
the layers that do fit leaves the model in a state nothing was trained for,
so try_load_chain drops the whole file when any family reports a mismatch.
Bias deltas were never checked against the target bias and could only surface
at apply time; a module with no bias stays a non-mismatch, since whole
architectures are built bias=False.
- check bias deltas against the module bias in the lora, norm and full loaders
- carry the mismatch count on the network so the chain can refuse the file
- record refused writes in the infotext so a partial apply is not read as clean
- point the krea2 full-diff test at a module that has a bias
The Krea 2 transformer keeps checkpoint-style module names while the
official krea/Krea-2-LoRA releases are saved with upstream-diffusers
names, so all 264 modules failed to bind and the LoRAs silently did
nothing. Krea 2 is the only native-LoRA arch with an sdnext-owned
transformer, so its module names diverge from the diffusers ecosystem.
- native_adapter.resolve_group_targets consults the arch resolver first
for passthrough prefixes, falling back to verbatim binding; a no-op
for arches that load the diffusers class
- krea2_lora maps diffusers attn/ff/text_fusion/embedder names onto the
checkpoint module tree; checkpoint-named LoRAs still bind verbatim
- add test/test-krea2-native-adapters.py
A full-weight extraction on Z-Image bound 308 modules and applied 172 of
them, silently dropping the rest, and left 71 more unmapped.
assign_network_names_to_compvis_modules puts every transformer module in
network_layer_mapping but skips stamping network_layer_name on norms,
which is the attribute the apply pass keys off. try_load_full bound those
modules through the mapping and they then never applied; stamp them
loader-locally, as try_load_norm already does.
Z-Image also names three module groups differently from the diffusers
tree: the qk-norms (q_norm/k_norm vs norm_q/norm_k), and the patch
embedder and final layer, which live in ModuleDicts keyed by
"{patch_size}-{f_patch_size}" and so carry a key the checkpoint has no
notion of. Read that key from the live model rather than hardcoding it.
The counts close exactly: 68 qk-norms plus 3 non-block targets are the 71
that went unmapped.
LyCORIS extraction with use_sparse_bias saves bias_indices/bias_values/
bias_size per module: the sparse weight-shaped remainder of the SVD
extraction, named bias for historical reasons. The keys were dropped by
both loader paths, so extracted adapters applied without the residual
correction; the dense-bias branch in finalize_updown that consumes it
was unreachable.
- rebuild the COO tensor in NetworkModule.__init__ (int16 indices cast
to long), shared by the native and generic loaders; kept sparse so
the dense += sparse in finalize_updown materializes per module at
apply instead of near-model-size densification at load
- accept the triplet suffixes in LORA_SUFFIXES; fused targets skip
with the weight-shaped-bias warning
- cover an extraction-faithful numeric round-trip and the fused skip
in the offline suite
ai-toolkit DoRA saves lora_A/B plus a 1-D per-output magnitude key in
place of alpha; PEFT and diffusers name the same quantity
lora_magnitude_vector. Neither key was in the suffix table, so such
adapters loaded as plain LoRA with the magnitude renormalization
silently missing. The semantics match LyCORIS wd_on_out=True row norms,
so both keys convert onto the existing dora_scale path.
- accept .magnitude and .lora_magnitude_vector in LORA_SUFFIXES and
convert at group level in try_load_lora
- reshape 1-D vectors to (out, 1): on square layers the apply-time
orientation detection would otherwise renormalize the wrong axis
- cover square-layer numeric equality, fused-qkv slicing and the PEFT
key form in the offline suite
A diff_b bias delta on a fused BFL target passed through whole and
failed at apply with a shape mismatch. diff_b stores one value per
output feature, so it partitions with the fused rows exactly like the
up-weight; slice it with the chunk in the LoRA loader. The legacy
weight-shaped bias key (LyCORIS sparse-residual heritage) has no
defined partition on a fused target and no known emitter pairs it with
chunk-capable families, so the group is skipped with a warning in the
LoRA, LoKR and LoHA loaders.
- add slice_bias_delta beside slice_dora_scale; warn and skip
non-per-output diff_b shapes
- cover sliced diff_b flowing out as ex_bias and the legacy-bias skip
in the offline suite
A LoKR group whose Kronecker product does not fit the resolved module
previously bound anyway and failed at apply time as a caught per-module
error, leaving the adapter partially applied with only an error log.
Reject the group at load with a warning instead, matching the LoRA
path's shapes_match gate.
- lokr_kron_shape derives (out, in_flat) from full, rank-decomposed or
Tucker-rebuilt factors, folding conv kernel dims into in_flat
- lokr_shapes_match honors SDNQ original shapes and chunk partitions:
equal chunks need total * out rows, row-range slices an exact range;
the input dim is never chunked
- cover non-fused and fused rejection in the offline suite
LyCORIS wd=True saves a dora_scale companion for LoRA/LoHA/LoKR; on
fused BFL targets the chunk paths passed it through whole, so apply
failed with a shape mismatch and the module was dropped. Per-output
magnitudes (wd_on_out=True, the default) partition exactly with the
fused rows; per-input magnitudes couple the chunks through shared
column norms and have no exact split.
- slice per-output dora_scale rows with the chunk in the LoRA, LoKR
and LoHA loaders
- skip per-input DoRA on fused targets with a specific warning
- cover sliced and skipped orientations in the offline suite
The lycoris_ save format is arch-independent: LyCORIS standalone wraps
the loaded diffusers model and emits the wrapped module path with dots
as underscores, so verbatim passthrough is correct for any arch. Only
flux2 handled it; zimage, chroma, ernie and krea2 reported such files
as not loaded.
- add lycoris_ to KNOWN_PREFIXES_DEFAULT and PASSTHROUGH_PREFIXES_DEFAULT
- drop flux2's per-arch prefix append and resolve_targets branch
- add lycoris_ to ANIMA_PREFIXES (anima replaces the default tuple);
network_prefix_for already routes it to the transformer namespace
- cover the passthrough with a zimage loader test
Route nn.Embedding targets (and the SDNQEmbedding / ScaledWordEmbedding subclasses) through the linear LoRA path: the weight delta is up@down over the [vocab, dim] table, same shape and merge as a Linear.
Apply a companion bias delta (diff_b) as ex_bias on the same module rather than dropping it; collect diff_b into the LoRA group so it rides the existing module instead of a separate Full module that would collide on the network key.
transformer., bare-diffusers, and lora_transformer_ bases are already in
network-key form for every arch, yet each per-arch resolve_targets repeated the
same passthrough branch for them. Move that into a shared
PASSTHROUGH_PREFIXES_DEFAULT set consulted by resolve_group_targets, leaving each
arch's resolve_targets to only the prefixes it actually rewrites (kohya / BFL).
lycoris_ stays in flux2, the one arch that recognizes it.
Pure refactor: the same keys resolve to the same modules.
OneTrainer saves LoRAs against the diffusers layout, keying each module as
'lora_transformer_' + the underscore-flattened module path with QKV pre-split.
That is sdnext's own network_layer_mapping namespace, but the native loader did
not list it as a known prefix, so parse_key dropped every key and the network
loaded zero modules ("not loaded").
Add lora_transformer_ to KNOWN_PREFIXES_DEFAULT and resolve it in a shared
resolve_group_targets helper that passes the base through unchanged, with no
rename or chunking. Routing every family loader through the helper gives all
diffusers arches (chroma, flux2, zimage, ernie) OneTrainer support without
per-arch wiring.
Fixes#4877
Frees the name for pipelines/native_transformer. Module covers the full
LyCORIS adapter family (LoRA/LoKR/LoHA/OFT/IA3/GLoRA/Norm/Full), not
just LoRA.
CalamitousFelicitousness
Vladimir Mandic