- restore stashed svd factors onto the layer's current device; the
stash tuple does not follow module device moves, so an offload
between apply and remove left restored factors on a stale device
- recheck factor shapes for layers already in factor mode, so a
malformed stacked network downgrades to the legacy path instead of
raising in the concat
- clear the fallback log at activate entry so a raise mid-pass cannot
leak stale entries into the next report
- pin both behaviors in the suite and state the compute-dtype fidelity
floor in the module docstring
Three defects surfaced by running the analyzer over a full local
collection against shipped checkpoints.
The requantize path hardcoded use_svd=False, but network_add_weights
requantizes with the layer's own svd setting and rank. On an svd
checkpoint the dequantized weight is not on the plain integer grid, so
requantizing without svd produced an error dominated by the discarded
correction and roughly orthogonal to the delta, which read back as a
retention near 1.0 on a grid where the delta is 0.005 of a step. Thread
use_svd, svd_rank and svd_steps through and reuse the returned factors.
Targets whose weight is not a matrix (norm and scale parameters, 205 of
them in one z-image extraction) reached the quantizer and the module stub,
both of which unpack two dimensions and raised. Skip and count them before
either. An all-zero delta (some full-rank extractions ship empty .diff)
divided by its own norm and reported nan; its retention is undefined
rather than erased, so it is excluded from the medians and counted.
Kohya-style files carry an already-underscored base (lora_unet_layers_0_
mlp_gate_proj), which the loader resolves by comparing network_prefix +
path.replace('.', '_') against each module's stamped name, so both sides
are underscored and the file loads. The analyzer instead looked the base
up as a literal dotted module path, so every module of such a file was
reported unmatched: 76 files in a local collection, including 36 of 57
anima and 5 of 10 chroma.
Fall back to a stamped-name index when the direct lookup misses. Dotted
bases are unaffected.
The analyzer only mapped plain-lora groups, so a file carrying no plain
lora (a pure lokr, for example) analyzed zero modules and fell through to
a 1.0 default: it reported perfect fidelity for exactly the files that
degrade most. Measured on the shipped krea 2 uint4 checkpoint, those
files land between 0.04 and 0.34.
Every targeted module is now rebuilt with the loader's own module class
and its delta read from the production calc_updown, so lokr, loha, oft,
full, ia3, glora, norm and the dora / dense-bias / diff_b variants are
measured as they apply; factor-path eligibility is decided by calling the
loader's own predicate. Modules carrying several families sum their
deltas the way the loader stacks them, and a family the tool cannot
rebuild is reported instead of counting as clean.
- report per-module applied fidelity (1.0 on the factor path, measured
rho on the requantize path) as a median and an energy-weighted mean
- add --dtype bf16 to measure the unquantized reference rather than
assert it
- drop the per-module empty_cache: it cost 16ms per module against 1ms
of reuse, and the caching allocator already reuses the buffers
- keep shard handles open across modules
Checkpoints quantized without hadamard must attach factors unrotated;
checkpoints carrying their own svd correction must keep it under apply
and get the original factors back on remove. Both pinned in both svd
layouts.
Offline analyzer for a (model, lora) pair: maps lora modules onto the
transformer, measures per-module delta-to-step ratio and requantize
retention, and reports factor-path eligibility. Loads pre-quantized
sdnq repos or simulates quantization on bf16 repos; supports --json
and --fail-under for scripted checks.
Baking a lora into a quantized weight requantizes it, and on low-bit
formats round-to-nearest erases sub-step deltas (uint4 retains roughly
2/group_size of the signal). Plain lora deltas now ride the sdnq svd
side-channel: factors append to svd_up/svd_down with the down factor
hadamard-rotated, applied by the dequantizer at full precision in every
forward mode. Apply and remove are exact and take no weight backup.
- non-factorable families (dora, lokr, loha, oft, cp mid, dense bias)
fall back to requantize with a per-pass summary warning
- native fuse now honors the quantized-model guard; fuse requantized in
place on every network swap and accumulated drift
- layers that fell back on a mixed set restore from backup before
re-entering the factor path; untargeted quantized layers are no
longer flagged
- test/test-sdnq-lora-factors.py pins the erasure law, factor-path
exactness, memory accounting and set transitions
A delta that does not fit its target module cannot apply, and applying only
the layers that do fit leaves the model in a state nothing was trained for,
so try_load_chain drops the whole file when any family reports a mismatch.
Bias deltas were never checked against the target bias and could only surface
at apply time; a module with no bias stays a non-mismatch, since whole
architectures are built bias=False.
- check bias deltas against the module bias in the lora, norm and full loaders
- carry the mismatch count on the network so the chain can refuse the file
- record refused writes in the infotext so a partial apply is not read as clean
- point the krea2 full-diff test at a module that has a bias
CalamitousFelicitousness
Vladimir Mandic
Dity0