The metadata cache keeps a failed read forever and --no-metadata returns
nothing, and in both cases the head loader returned None silently while
the backbone still applied. The grid is now read from the file's own
header when the cached metadata lacks it, and head-shaped tensors
without a grid log a warning.
The projection a parallel head replaces waits out of the module tree,
so nothing moves it, and a weight still viewing its checkpoint shard
keeps the whole shard mapped. The stash now clones the projection's
tensors.
Group hooks on the no-stream path and the on-demand hook return a
component to cpu through a device copy, so a memory-mapped text encoder
sits in memory twice, as the mapped file its never-run vision tower
keeps alive and as the copies, and every encode pays a device-to-host
transfer of unchanged weights. The engine now records the cpu tensors
at onload and hands them back at offload; a component moved by any
other path still takes the copy.
MiniMaxH3Scheduler counts the terminal sigma in num_inference_steps, so
Steps N ran N-1 evaluations while every other model runs N. The shim
hands the scheduler p.steps + 1, the slider starts at 1, and the PDD
pin records the evaluation count while passing the scheduler its grid
argument. Metadata written before this change counted grid points.
Shift is a property of the trained schedule, not of the step count, so
the tab sliders take absolute values, defaulting to the shipped 12 and
3. video_minimax resolves each request from the request value or the
scheduler config inside apply_overrides, which the tab, the api and
the still path all call, so a request without values lands on the
shipped schedule. The api maps sampler_shift onto the video schedule
and gains audio_shift. Applied values are recorded as Video shift and
Audio shift; the PDD pin records what it enforces.
Layer-wise extractions (FastH3) ship diff_b on every projection and
diff on the norms; final_layer.norm had no rename onto norm_out.norm,
so its residual went unmapped.
PDD files pair a backbone LoRA with the output projections repeated per
interval of a training grid; each step fuses the heads of its block
into one projection. network_pdd reads the grid from the metadata,
swaps a ParallelHead in for each projection, fuses from the scheduler's
step_index and pins the step count and shift while heads are
installed. The MiniMax loader declares which scheduler each head
follows.
The pruned transformer stores W @ P for each AdaLN projection against
the rank-8 basis in time_embedder.basis, so deltas trained at the
released width failed the shape check. try_load_lora gains an
adapt_weights hook and the MiniMax loader refits lora_down as down @ P,
which is exact.
save_cache handed writefile the live HashStore objects, so an add_hash
from another thread during the deep copy raised "dictionary changed
size during iteration" and dropped the save. The stores are now copied
under a lock that also covers the write, so a later save never lands
under an earlier snapshot.
Every caller writes a whole JSON document, and a plain open/write can
leave a torn or empty file when the process dies or another writer
overlaps, which readfile then returns as {}. Writes now go through the
temp file and replace unless atomic=False or the mode is append. A read
waits out an open that Windows refuses while a replace of the same name
is in flight, and an empty file is reported when the read is not
silent, since a zero-byte config.json otherwise resets settings without
a trace.
The fasteners lock is inter-process only. On Linux fcntl record locks
belong to the process, so two threads writing the same file were never
serialized and could tear it; on Windows they were, until the lock
switched itself off. Every writer of these files runs inside one
multi-threaded process, so that is the case that matters.
readfile(lock=True) and writefile now take a re-entrant lock per
normalized path, held from the snapshot through the replace so writes
to one path land in call order. Other processes are covered by the
atomic replace. A .lock file left next to a JSON file by the old lock
is removed the first time that path is used.
writefile(atomic=True) renamed the temp file while it was still open,
which Windows refuses, so every atomic save failed there and left a temp
file behind. The temp file is now created with mkstemp, closed before
os.replace and removed when the save fails; the replace retries briefly
when another handle holds the target, which Windows reports as a
permission error.
Both helpers removed the .lock file after releasing it. That removal
raced other holders on both platforms, and any failure switched locking
off for the whole process without a log line. The lock file now stays in
place; a failed release or a lock timeout is logged instead.
writefile deep-copied and validated the live object in Python, so a
concurrent insert into a shared dict raised "dictionary changed size
during iteration" and dropped the save. dict.copy and list.copy run
under the GIL, so the deep copy and the per-key validation now walk that
snapshot; nested containers remain the caller's responsibility.
test/test-json-helpers.py covers all three on Linux and Windows.
A checkpoint-name key survives unload, so a reload of the same checkpoint
never printed the per-module stats block again, even with a different
quantization. Each component now carries its own reported stamp: a task
switch rebuilds the pipe around the same modules and stays quiet, while a
reload or a component swap brings new modules and reports them.
The group offload functions live in sd_offload_group, sd_offload_utils and
sd_offload_state rather than modules.sd_offload, so the tests and their
monkeypatches now import and patch those modules directly.
The parser no longer takes reference-name prefixes to tell bare
reference keys from bare diffusers keys. Any bare key carries the
sentinel and the arch resolver renames what it knows and passes the rest
through. Flux2 keeps its list for file-format detection only.
A bare key that matches no known prefix is parsed with the
bare-diffusers sentinel and handed to the resolver instead of being
dropped at parse time. The per-arch lists of bare diffusers prefixes are
gone, and a path that names no live module counts as unmapped.
A ChunkSpec can reorder equal row blocks of the rows it selects, so an
arch declares a swapped SwiGLU projection on the target instead of
permuting the state dict first. Only the LoRA family applies it; the
others skip a reordered target with a warning.
The reference fc1 is a fused [gate; value] SwiGLU projection and the
diffusers port stores [value; gate]. The native mapping did not swap the
halves, so gate and value deltas landed on each other's rows. The
mapping now also renames the standalone projections, reads a
metadata-only alpha, and accepts the musubi, peft dit and diffusers-named
layouts.
SDNQ codebook layers keep their Lloyd levels in the scale slot, so reading
scale.mean() as the grid step returned the levels' near-zero mean and sent
sub-step deltas to requantize, where the grid erases them. grid_step returns
the mean adjacent-level gap for those layers and the plain scale mean otherwise.
Anima 2.9B interleaves twelve new blocks among the 28 of Anima 1.0, so a block
index trained against 1.0 names a different block on the expanded model. Every
such key still resolves, since blocks 0 to 27 exist either way, so the
mismatch was silent. The Anima loader now shifts base-depth indices onto the
blocks that carry those weights, keyed by (base depth, expanded depth) and
applied only when the transformer is expanded and the LoRA stays inside the
base depth. Transformer keys move; llm_adapter and text encoder keys keep
their own numbering.
Anima-2.9B is a depth-expanded finetune of Anima 1.0 Base carrying 40
transformer blocks against the base repo's 28. The reference entry points at
the Diffusers conversion. Single-file releases load through the native loader:
TransformerSpec gains an infer_config hook, the Anima spec uses it to size
num_layers to the block indices in the file, and model_anima routes a
checkpoint-selected safetensors through the loader with the remaining
components from the base repo.
Two tables decide the native path: one says which architectures may take
it, the other says which loader they get. An entry in the second without
one in the first is a loader nothing can reach, and nothing checked that.
The error limiter halts a pass by raising, and nothing between the raise
and the caller put the model back. A halted pass left group offload hooks
stripped from every component the walk had reached, left a sequential
model on the cpu with offload disabled, and left the counters other
modules read describing the pass before it.
The epilogue moves into finish_pass under a finally, so the model returns
to its offload mode and the counters describe the pass that just ran. The
abort still reaches the caller.
Pass state is reset in one place in lora_sdnq now. Two of the six
accumulators were not being cleared at the start of a pass, and a stale
routed layer suppresses the fallback count for that layer next time.
The per-module walk carried four mechanisms inline, each repeating the
same tail: count the layer, stamp the pair that marks it current, advance
the bar, continue. Five copies of that tail and three of the backup probe
put the deepest arm nine levels in.
Each mechanism is now a function that either takes the layer or declines
to the next, and the walk reads as the four of them in order. The pass
state they share moves onto one object built before the walk starts, with
the accept tail, the stamp and the bar tick as its methods. That takes
network_activate from 218 lines to 55, none of it deeper than the module
loop.
Two shapes are deliberately not folded into that tail: the weight path
counts weights and bias separately and tracks what the module refused,
and the factor-strip restore stamps without counting. Hosting hands a
declined delta back rather than leaving it in a flag, so a pair of Nones
still reads as assembled and no layer is calculated twice.
The keys that mark a degradation as reported lived for the life of the
process, so a user who saw "flip=skipped weight=offloaded", changed the
offload mode and hit the same wall again was told nothing the second
time. Tie the set to the settings the warnings speak about: the stack
signature, the offload mode, the host rank and the checkpoint. Repeating
under one context still says it once.
The generic loader never offered a file to the boft type, so butterfly
OFT adapters reached the oft type instead, which claims any oft_blocks
key without checking its rank and then reads the block count as the lora
dim. Register boft ahead of oft; files with 3-d blocks still land on oft.
The nunchaku path replaced the loaded network list with the on-disk
entries it composed from, so reading a loaded network back hit an object
without the fields it expects: choosing the reported method reads
len(net.modules) and raised on every set change, costing that generation
its infotext and trigger tags. The adapter was already composed by then,
so the image was unaffected. Wrap the composed set in Network objects
and mutate the list in place.
The mechanism gate tests assert select_candidate declines under
requantize, and the apply-method hint names the cache option among
those the requantize choice disables.
The in-place weight installs, the promote-after-deactivate fuse
ordering, and the dynamo reset at model unload live in the shared
loader code; their regression pins belong in the campaign suite beside
the paths they protect.
<lora:name:1.0:lbw=VALUE> scales each targeted layer's delta by a slot of a
per-architecture block vector. VALUE is a preset name, a scalar, or a comma
vector; presets stretch onto the block count of the current model and the
a1111 17-slot and 12-slot layouts are accepted on sd and sdxl. The factor
enters through the module multiplier, so every apply path carries it: the
exact factor channel, hosting, requantize routing, dense stack combines and
select scoring.
- modules/lora/lora_blocks.py: slot classification from network_layer_mapping
(namespace-first, anchored chain prefixes), preset resolution reusing the
merge block-weight tables with BASE forced neutral, generated classic
segment names plus DOUBLE/SINGLE chain names, per-model memoization
- the raw spec stages through pending_config and promotes with the other
multipliers, keeping fuse removal consistent
- block weights join the activation signature, the per-module apply stamp
and the factor cache identity; entries without block weights keep their
existing signature bytes
- non-native load methods warn once and ignore the argument
The weight-kind schedule reset ran each winner's calc_updown on the target
weight's device, which on a block-swapped denoiser is the cpu; at hundreds
of layers per pass the cpu matmuls dominated every select generation. The
delta now computes on the accelerator and moves back, matching the activate
walk's convention.
- reset and flip execution log a debug timing line (materialize, select
loop, move/calc/apply split); the reset runs outside the activate walk,
so its cost was invisible to the load timers
Select scoring staged full fp32 copies, an abs copy and top-k workspace per
layer (hundreds of MB of transients that collide with block swapping on
offloaded denoisers) and recomputed scores from freshly assembled deltas on
every apply, which kept select modes out of the factor-cache fast path.
- score_pair: row-chunked fp32 interiors, fp64 accumulators, one device sync
- select scores persist in the factor cache as additive per-layer records
under the existing configuration signature
- apply_select_cached and register_weight_pair_cached replay a pair without
assembling deltas; the weight-kind winner is still computed at schedule
time
The klora and estlora balance factors accumulated across registrations
without ever resetting, so a multiplier change or pair swap blended the
previous registration into every later schedule. Balances now sum over
the live entries at finalize time, which keeps drop and re-register
consistent by construction.
- key flips one step early: step callbacks fire after the denoise, so
the winner is now live during the crossover step forward and a
final-step crossover engages instead of expiring
- include the calibration toggle in the factor cache signature so a hit
never replays factors computed under the other setting
- fall back to summation with a warning when hosting is disabled on a
quantized model instead of registering schedules that cannot flip
- drop the unused score_topk helper
EST-LoRA scores each layer by squared Frobenius energy, so a magnitude gap
between the two networks enters squared and the louder network wins nearly
every layer, starving the quieter one. The style side is now scaled by the
total-energy ratio (mirroring klora's gamma), making selection scale-invariant
so a network cannot take layers on magnitude alone. On the krea2 subject+style
pair this lifts the style network from 18% to 65% of the layer-step budget.
- lora_stack: accumulate per-mode energy totals, apply the balance in the est ramp
- test: content-louder est pair now hands over mid-schedule where raw scoring never would
- locale: note the est magnitude balance, and that a select mode gives each layer
to one network so both can be under-applied, while dense modes blend more fully
Requantizing a dense-combined delta into 8-bit weights is checkpoint-fragile:
on some checkpoints the round trip visibly damages the render while the same
combination hosted on the svd channel is clean. Dense-mode sets with two or
more contributing networks on a layer now ride the hosted path regardless of
bit width; single-set behavior at 8 bits and above is unchanged.
- host_candidate: dense multi-net layers qualify at any width
- suite: dense pair at int8 hosts; single non-factorable set at int8 keeps
the requantize fallback
Select pairs now ride the svd side channel on any SDNQ linear, not
only sub-8-bit ones: quantized backups are packed tensors, so the
weight rewrite path cannot recompute a winner from them and left
layers stripped mid-requantize. The sub-8-bit gate stays for dense
hosting, where requantize retains the delta at 8 bits and above.
Weight selection now only serves unquantized modules: finalize
iterates a snapshot so dead entries drop cleanly, materializes
balanced-offload modules before rewriting weights and skips modules
with stripped or quantized weights instead of corrupting the layer.
A select mode forced backup mode whenever it was merely set, so a
leftover setting changed behavior for ordinary single-network loads.
The fuse gate now engages only when the loaded set could actually
select (exactly two networks, compile permitting) or while selection
segments are still live on model layers. Re-application drops any
stale per-layer schedule so a later pass reset can never replay an
old winner over freshly applied weights. Fallback notices log per
activation instead of once per session; only the in-loop flip notice
stays latched.
The exact factor path was the only apply route with no log line; its
success read as silence. Track layers taking it beside the hosted and
fallback lists and report all three as key=value apply lines
(apply=exact/hosted/requantize); the stack fallback notices use the
same form. The suite pins its stack-mode baseline to sum so a mode
left set in user config cannot reroute tests that assume plain
summation.
Two-network subject+style sets select a winner per layer instead of
summing: scores are top-K magnitude sums (klora) or Frobenius energies
(estlora), and a timestep ramp shifts layers from the subject network
toward the style network across sampling, reduced to at most one
precomputed flip per layer per pass. On sub-8-bit SDNQ the pair rides
the side-channel as separate segments flipped in place; other layers
recompute the winner from the pristine backup, so select modes force
backup mode. Selection resets per pass from the callback setup and is
gated off under model compile. estlora's measured style-discrepancy
term is exposed as an option. Adds XYZ axes for the stack settings.
Add lora_stack_mode with ties, dare_ties, dare_linear and
magnitude_prune combination of per-network deltas when several loaded
networks target one layer; sum stays the default and the exact factor
path. Combined deltas ride the existing tail: hosted svd on sub-8-bit
SDNQ, requantize at int8 and above, direct add elsewhere. Text-encoder
layers and single-network sets keep plain summation. DARE masks draw
from per-layer sha256 seeds so re-applies and cache entries stay
deterministic; the stack settings join the activation and factor-cache
signatures so settings changes re-apply without a reload.
sparse_attention_exclude is a comma separated denylist matched case insensitively against the architecture, the pipeline class and the denoiser class, so one entry works whichever name is to hand. It resolves once per model rather than per call and declines with a log line. Seeded with CosmosTransformer3DModel, the transformer Anima runs, which returns banded noise at every budget tested against a sound dense baseline; listing the class rather than the architecture covers the other models built on it, none of which have been checked.
CalamitousFelicitousness
Vladimir Mandic
Yifan Chen
nan