Add a per-model native LoRA path for Chroma that handles both PEFT-style
(diffusion_model.<dotted>.lora_A.weight, no alpha) and kohya-flat
(lora_unet_<flat> with .alpha) layouts, plus LoKR/LoHA/OFT/DoRA.
Resolves issue #4169 (ostris ai-toolkit Chroma LoRAs failed to load).
- new pipelines/chroma/chroma_lora.py with try_load_lora/lokr/loha/oft
- fused-QKV expansion: 3-way equal for img/txt qkv, 4-way unequal
[3072, 3072, 3072, 12288] for single linear1 (Q+K+V+proj_mlp)
- NetworkModuleLokrSliceChunk for unequal-row LoKR slicing
- path rewrite from Flux-source to ChromaTransformer2DModel naming
- lora_load.py: dispatch chroma to the native loader and narrow the
kohya-Flux convert gate to f1 only
Diffusers ships only ErnieImagePipeline (txt2img). Adds two
subclasses wrapping it: ErnieImageImg2ImgPipeline and
ErnieImageInpaintPipeline, registered into the auto-pipeline
mappings so set_diffuser_pipe swap finds them.
Encode mirrors the upstream decode: vae.encode -> _patchify_latents
-> BN-stats normalize using vae.bn.running_mean/running_var.
Inpaint blends the denoised latent with a noise-level-matched
init latent via callback_on_step_end. Verified end-to-end through
hires upscale and yolo detailer with face and eye masks.
Adds pipelines/ernie/ernie_lora.py with try_load_lora/lokr/loha/oft
entry points modeled on the z-image native loader, wires it into
lora_load.load_safetensors, and adds 'ernieimage' to allow_native.
ERNIE attention is not fused (separate to_q/to_k/to_v/to_out.0), so
the loader skips the qkv-split machinery the z-image loader needs and
supports all four families uniformly. Recognized prefixes are
diffusion_model., transformer., and lora_unet_; PEFT lora_A/lora_B
keys are normalized to lora_down/lora_up. Verified against PEFT, LoKR,
and Kohya/AIT files in the wild with zero unmapped or shape-mismatched
modules.
NVIDIA/Cosmos native exports prefix every key with net., including the
bundled llm_adapter half. Add it to KNOWN_PREFIXES so partition_adapter
runs on stripped keys.
Dispatch anima loras through a dedicated native loader covering kohya,
bfl/ai-toolkit, and hybrid (bfl with alpha plus qwen3 text encoder)
formats. Cosmos 2.0 path rename is mirrored from diffusers in flat
(underscore) form so rewritten paths match network_layer_mapping keys
without further conversion.
Split model_type from cosmos to anima so a future base-cosmos2 lora
path stays separable. Update flow_models, taesd supported list, and the
taesd wanvideo bucket so samplers and preview decoding keep working
after the split.
Extend assign_network_names_to_compvis_modules to walk pipe.llm_adapter
under the lora_llm_adapter_ prefix, and add llm_adapter to
default_components so activate and deactivate include it for anima
models while staying inert elsewhere via the existing getattr guards.
Anima community files bundle the llm_adapter inline alongside the
transformer, so CosmosTransformer3DModel.from_single_file mangles the
118 adapter keys via Cosmos 2.0 substring renames. New helper reads the
safetensors, strips the BFL-style prefix variants, partitions adapter
keys out, runs the rest through diffusers' converter, and loads each
half into its target. SDNQ and layerwise quantization honored; TRT and
GGUF skipped.
Add three adapter families to the z-image native loader and chain them
with the existing lora path through load_safetensors. Mixed-family
files (for example gta6_amateur_photography_zimagebase_v2.safetensors,
which carries lora and lokr groups in the same file) now load
completely instead of having one family silently dropped.
Shared helpers in pipelines/z_image/zimage_lora.py parse keys by
suffix list, rename legacy attention.out and attention.wo to
attention.to_out.0, and split fused attention.qkv into to_q/k/v. For
lora the split chunks the up weight along dim 0. For lokr the split
emits three NetworkModuleLokrChunk entries that share the tensors and
slice the kronecker product at apply time.
Fused attention.qkv for loha and oft is skipped with a warning. No
NetworkModuleHadaChunk exists, oft rotations are tied to out_features
and cannot be cleanly split across q/k/v, and no real z-image adapter
in that layout exists today.
load_safetensors for zimage chains try_load_lora, try_load_lokr,
try_load_loha and try_load_oft and merges their module dicts into a
single Network so mixed files load every module.
Add zimage to allow_native so lora_force_diffusers picks between
native and diffusers. Before this, zimage always took the diffusers
path regardless of the setting.
pipelines/z_image/zimage_lora.py reads the safetensors and writes
directly into network_layer_mapping, so Z-Image LoRAs no longer go
through the diffusers PEFT converter that raised KeyError on
state dicts with partial alpha keys.
Key formats handled: ai-toolkit, kohya lora_unet_, bare transformer.
and no-prefix. Pre-refactor fused attention.qkv is split into
to_q/k/v; attention.out and attention.wo are renamed to
attention.to_out.0. Alpha and dora_scale are preserved.
Vladimir Mandic
CalamitousFelicitousness
resonantsky
awsr