Anima 2.9B interleaves twelve new blocks among the 28 of Anima 1.0, so a block
index trained against 1.0 names a different block on the expanded model. Every
such key still resolves, since blocks 0 to 27 exist either way, so the
mismatch was silent. The Anima loader now shifts base-depth indices onto the
blocks that carry those weights, keyed by (base depth, expanded depth) and
applied only when the transformer is expanded and the LoRA stays inside the
base depth. Transformer keys move; llm_adapter and text encoder keys keep
their own numbering.
Anima-2.9B is a depth-expanded finetune of Anima 1.0 Base carrying 40
transformer blocks against the base repo's 28. The reference entry points at
the Diffusers conversion. Single-file releases load through the native loader:
TransformerSpec gains an infer_config hook, the Anima spec uses it to size
num_layers to the block indices in the file, and model_anima routes a
checkpoint-selected safetensors through the loader with the remaining
components from the base repo.
Route nn.Embedding targets (and the SDNQEmbedding / ScaledWordEmbedding subclasses) through the linear LoRA path: the weight delta is up@down over the [vocab, dim] table, same shape and merge as a Linear.
Apply a companion bias delta (diff_b) as ex_bias on the same module rather than dropping it; collect diff_b into the LoRA group so it rides the existing module instead of a separate Full module that would collide on the network key.
Replaces anima_lora.py's bespoke try_load_lora / group_keys /
resolve_network_key with thin wrappers binding native_loader's generics
to anima's prefix tuples and resolve_targets, mirroring flux2 / zimage /
chroma / ernie. The hand-rolled apply_lora_alphas bake-with-balance
pass goes away; alpha / scale / dora_scale flow through NetworkWeights.w
to NetworkModule.calc_scale at apply time.
native_loader gains an optional network_prefix kwarg (str or
Callable[[prefix_used], str], default "lora_transformer_") used when
constructing network_key. Anima passes a callable picking
lora_transformer_ / lora_llm_adapter_ / lora_te_ per matched prefix.
Single-component siblings keep the default and are unchanged.
network.NetworkModule.apply_weight_decompose grows a dual-path DoRA
convention detector. The pre-fix implementation only handled per-input
dora_scale (DoRA paper / kohya, shape (1, in)), silently broadcasting
per-output LyCORIS / PEFT dora_scale (shape (out, 1)) into an incoherent
element-wise rescaling. Detection is structural: (out, 1, ...) routes
to per-output; everything else (including the square-weight 1D ambiguity)
defaults to per-input for legacy compat. Pre-existing bug surfaced by
the LoKR+DoRA LyCORIS files Anima now loads.
Behavior changes:
- LoHA via the generic try_load_loha (NetworkModuleHada); covers
scenery-anima-base and any other LyCORIS .hada_w* export.
- Kohya lora_te_ prefix recognized. The legacy resolver only matched
BFL text_encoders.qwen3_06b.transformer.model. and silently dropped
lora_te_layers_N_* keys (41% of BlueArcStyle's bases were unloaded).
- LoKR+DoRA LyCORIS files now apply correctly; the per-output dora_scale
is honored instead of silently scrambled.
Adds test/test-anima-native-adapters.py: 37 offline tests across all
five prefixes (LoRA + LoHA), every COSMOS_2_FLAT_RENAME entry, DoRA
threading, marker disambiguation, try_load_chain dispatch, calc_updown
sanity, and both DoRA conventions (per-input / per-output / 1D ambiguous).
Adapter mock mirrors AnimaLLMAdapter's real module tree.
CalamitousFelicitousness