Discriminated from OFT by 4-D oft_blocks shape (boft_m, block_num,
block_size, block_size), mirroring LyCORIS algo_check. The cascade
of log_2(block_num)+1 Cayley rotations is applied via the
unflatten/transpose/flatten reshape sequence in NetworkModuleBOFT,
porting LyCORIS make_weight verbatim.
base_model.model. is the peft.save_pretrained wrapper; stripped
via _unwrap_peft_wrapper and re-parsed against the standard prefix
list (so the same code path handles whatever format peft wrapped).
lycoris_ is LyCORIS-standalone (e.g. SimpleTuner LoKR); base path
is an underscore-rendered diffusers path returned verbatim.
Adds 'f2' to allow_native and dispatches all eight native loaders
through load_safetensors. lora_force_diffusers continues to route
through lora_diffusers.load_diffusers as the explicit PEFT escape
hatch.
Targets the elementwise-affine RMSNorms inside attention (norm_q,
norm_k, norm_added_q, norm_added_k). lora_convert.py:502 skips
network_layer_name on transformer norm modules outside SD3, so this
loader stamps it directly on the modules it binds — surgical, no
shared-code edit, no impact on modules a Norm adapter doesn't target.
Loads the four ab.weight tensors plus alpha. Fused QKV is skipped:
GLoRA's target-dependent term doesn't row-slice cleanly without
recomputing against a fused proxy weight.
The .on_input marker disambiguates IA3 from other families that also
have .weight keys; the per-group gate insists on both. Fused QKV is
skipped with a warning.
Both kohya (oft_blocks + alpha-as-constraint) and LyCORIS (oft_diag)
layouts route through NetworkModuleOFT. Fused QKV in double_blocks is
skipped: the block-diagonal rotation's block_size = out_features /
num_blocks would need re-derivation per chunk to fit a Q/K/V split.
Standard non-Tucker LoHAs route through NetworkModuleHadaChunk for
fused QKV in double_blocks. Tucker (CP-decomposed) variants on fused
targets are skipped with a warning. Single-block linear1 and non-QKV
double-block targets work for any LoHA layout.
Adds per-family SUFFIX/MARKER tables and shared parse_key /
group_by_suffixes / resolve_targets helpers, mirroring the pattern
in pipelines/z_image/zimage_lora.py. try_load_lora and try_load_lokr
now share this scaffolding alongside the family loaders that follow.
Slices w1a/w2a at the assigned chunk's row range and computes the
partial Hadamard product, mirroring NetworkModuleLokrChunk. Used
when LoHA targets a fused weight (e.g. img_attn.qkv) on models
that expose split Q/K/V modules.
Without self.dim, NetworkModule.calc_scale returns 1.0 even when
alpha is present, silently dropping the trained scaling. Use
w1b.shape[0] as the rank, matching the network_lora and
network_hada conventions.
CalamitousFelicitousness
Vladimir Mandic
Oleksandr Liutyi
Oleksandr
awsr
Disty0