process_images swallows the interrupt assertion, so a cancel reaches the shared
video core as an empty result and was raised as 'processing failed' with 500,
leaving clients unable to tell a cancel from a crash. Mirrors the LTX path,
which already returns 499.
Queue.__exit__ returned _queue_lock, and a truthy __exit__ return suppresses the
exception in flight, so every `with queue_lock:` block discarded exceptions and
resumed with locals from the aborted block unassigned. The bare threading.Lock it
replaced returned None.
The refine-default comment claimed Condition variants are excluded from two-stage
refine, but supports_two_stage_refine has no such carve-out and ltx_process builds
a second condition set so conditioning survives the upsample. Two file references
had also drifted: ltx_process.py:179 moved, and the offload hook keying now lives
in sd_offload_balanced. Name the symbols instead of the line numbers.
New lora_sdnq_apply radio (exact, requantize) in the lora settings.
requantize keeps the previous behavior: every quantized layer takes the
dequantize-add-requantize path, with factor attach and svd hosting gated
off. A settings-only flip re-applies loaded networks: the mechanism
rides a per-module apply stamp and the network-changed signature, and
the activate fallthrough strips factors a closed gate leaves attached.
Requantize chosen by the setting logs as info instead of the
reduced-fidelity warning.
- locale hint covers fidelity and memory tradeoffs of both methods
- suite: gate, legacy routing and flip-transition tests
The lora block moves out of Networks into its own settings tab, with
header groups by what each option acts on: loading, prompt, application,
quantized models and metadata. Locale hints follow to the new section.
Plain svd truncation of hosted deltas is optimal in weight space but not
in output space: activations concentrate energy in a few input channels,
so scaling the delta by per-channel input RMS before the svd spends the
rank budget on output error instead. Statistics stream from the model's
own forwards on sub-8-bit SDNQ checkpoints and cache per checkpoint;
measured on real LoKR files this raises output-delta retention by ~0.05
at rank 256 and ~0.09 at rank 64, most on MLP down projections.
- modules/lora/lora_calib.py: capture hooks, per-checkpoint cache under
data/sdnq-calib, statistics land on layers as sdnq_calib_rms; gated by
lora_sdnq_host_calib, skipped when the model is compiled
- lora_sdnq.apply_hosted: weighted truncation when statistics exist,
calib count in the load summary
- cli/sdnq-calibrate.py: complete calibration now against a live server
- cli/lora-quant-fidelity.py --calib: hosted rho scored in the
activation-weighted norm
- test/test-sdnq-lora-factors.py: calibration category, 5 tests
Non-factorable modules on sub-8-bit formats report the hosted figure:
the seeded svd truncation at --host-rank realized through the bf16
materialize, mirroring the loader. The requantize figure stays in
requant_rho; --host-rank 0 restores the old scoring.
Non-additive families (lokr, loha, oft, dora, full) merged into the
quantized weight and lost most of their delta on low-bit formats. On
sub-8-bit layers the set's calc_updown delta now rides the svd
side-channel as its top singular directions instead: factorable members
are subtracted out and appended exactly, so only the non-factorable
remainder is truncated. Truncation keeps the dominant part of the
effect and drops an orthogonal residual, where requantize keeps the
grid extrema and adds grid-shift noise of the delta's own magnitude;
on real lokr files retention rises from 0.04 to about 0.5 at the
default rank.
Hosted layers take no weight backup and unload bit-exactly. The svd
runs under a forked rng so generation seeds are unaffected. At 8 bits
and above requantize retains most of the delta and remains the path.
lora_sdnq_host_rank caps the hosted rank; 0 disables hosting.
The analyzer scored factor-eligible modules applied_rho=1.0 by
construction. The side-channel stores the delta losslessly, but the
dequantizer materializes base + factors in the result dtype, so small
deltas round at the bf16 ulp of the base weight. Score the realized
delta through that rounding; sub-ulp loras now report the same floor an
unquantized bf16 model gives them instead of a false 1.0.
Also survive a broken file and keep completed work: per-lora failures
are recorded and skipped, the report json rewrites after every file,
and a complete flag marks a finished run.
- restore stashed svd factors onto the layer's current device; the
stash tuple does not follow module device moves, so an offload
between apply and remove left restored factors on a stale device
- recheck factor shapes for layers already in factor mode, so a
malformed stacked network downgrades to the legacy path instead of
raising in the concat
- clear the fallback log at activate entry so a raise mid-pass cannot
leak stale entries into the next report
- pin both behaviors in the suite and state the compute-dtype fidelity
floor in the module docstring
Three defects surfaced by running the analyzer over a full local
collection against shipped checkpoints.
The requantize path hardcoded use_svd=False, but network_add_weights
requantizes with the layer's own svd setting and rank. On an svd
checkpoint the dequantized weight is not on the plain integer grid, so
requantizing without svd produced an error dominated by the discarded
correction and roughly orthogonal to the delta, which read back as a
retention near 1.0 on a grid where the delta is 0.005 of a step. Thread
use_svd, svd_rank and svd_steps through and reuse the returned factors.
Targets whose weight is not a matrix (norm and scale parameters, 205 of
them in one z-image extraction) reached the quantizer and the module stub,
both of which unpack two dimensions and raised. Skip and count them before
either. An all-zero delta (some full-rank extractions ship empty .diff)
divided by its own norm and reported nan; its retention is undefined
rather than erased, so it is excluded from the medians and counted.
Kohya-style files carry an already-underscored base (lora_unet_layers_0_
mlp_gate_proj), which the loader resolves by comparing network_prefix +
path.replace('.', '_') against each module's stamped name, so both sides
are underscored and the file loads. The analyzer instead looked the base
up as a literal dotted module path, so every module of such a file was
reported unmatched: 76 files in a local collection, including 36 of 57
anima and 5 of 10 chroma.
Fall back to a stamped-name index when the direct lookup misses. Dotted
bases are unaffected.
The analyzer only mapped plain-lora groups, so a file carrying no plain
lora (a pure lokr, for example) analyzed zero modules and fell through to
a 1.0 default: it reported perfect fidelity for exactly the files that
degrade most. Measured on the shipped krea 2 uint4 checkpoint, those
files land between 0.04 and 0.34.
Every targeted module is now rebuilt with the loader's own module class
and its delta read from the production calc_updown, so lokr, loha, oft,
full, ia3, glora, norm and the dora / dense-bias / diff_b variants are
measured as they apply; factor-path eligibility is decided by calling the
loader's own predicate. Modules carrying several families sum their
deltas the way the loader stacks them, and a family the tool cannot
rebuild is reported instead of counting as clean.
- report per-module applied fidelity (1.0 on the factor path, measured
rho on the requantize path) as a median and an energy-weighted mean
- add --dtype bf16 to measure the unquantized reference rather than
assert it
- drop the per-module empty_cache: it cost 16ms per module against 1ms
of reuse, and the caching allocator already reuses the buffers
- keep shard handles open across modules
Checkpoints quantized without hadamard must attach factors unrotated;
checkpoints carrying their own svd correction must keep it under apply
and get the original factors back on remove. Both pinned in both svd
layouts.
Offline analyzer for a (model, lora) pair: maps lora modules onto the
transformer, measures per-module delta-to-step ratio and requantize
retention, and reports factor-path eligibility. Loads pre-quantized
sdnq repos or simulates quantization on bf16 repos; supports --json
and --fail-under for scripted checks.
Baking a lora into a quantized weight requantizes it, and on low-bit
formats round-to-nearest erases sub-step deltas (uint4 retains roughly
2/group_size of the signal). Plain lora deltas now ride the sdnq svd
side-channel: factors append to svd_up/svd_down with the down factor
hadamard-rotated, applied by the dequantizer at full precision in every
forward mode. Apply and remove are exact and take no weight backup.
- non-factorable families (dora, lokr, loha, oft, cp mid, dense bias)
fall back to requantize with a per-pass summary warning
- native fuse now honors the quantized-model guard; fuse requantized in
place on every network swap and accumulated drift
- layers that fell back on a mixed set restore from backup before
re-entering the factor path; untargeted quantized layers are no
longer flagged
- test/test-sdnq-lora-factors.py pins the erasure law, factor-path
exactness, memory accounting and set transitions
dev added sdnq_attention_quantize_fp32 to modules/attention.py, which this
branch replaced with the modules/attention package. The option moves to the
sdnq backend in two places: the options dict the prepared call captures, and
the options tuple that rebuilds the chain when a captured option changes.
Reading it in only the first place would leave a setting that takes effect on
the next model load and not before.
The new option gets a hint alongside the rest of the section.
Everything under Cross Attention, SDNQ Attention and Attention Dispatcher
shipped with no hint, which left the layering invisible: the SDP kernel boxes
are candidates torch chooses from per call rather than a selection, and the
Flash box is torch's own build of the kernel rather than the flash-attn package
that the Flash attention override installs.
- cover attention method, sdp kernels, sdp overrides and attention slicing
- give the dynamic attention rates their unit and the estimate they compare against
- cover the sdnq attention kernel settings, including the head dimension clamp
that the hadamard group size resolves through
- cover the diffusers attention dispatcher and the backend names it takes
- record the constraints each override serves, leaving the throughput comparison
to the workload rather than naming a winner
Batch matrix-matrix and Dynamic Attention BMM applied a legacy Attention
processor to pipe.unet, which a diffusion transformer does not have, so
they served unet models alone and said nothing elsewhere. The choices, the
processor and its slice helper are removed, an unrecognized method now
warns rather than selecting nothing, and a stored value is rewritten to
Scaled-Dot-Product on load.
attention_slicing holds one of Default, Enabled or Disabled, so testing the
string for truth sent Disabled down the enable branch and left the disable
call unreachable, while the log line below it reported the choice rather
than the action taken.
Dity0
Vladimir Mandic
Alejandro Alberto Aguilar Morales
CalamitousFelicitousness