Qwen3.5 runs most of its layers as gated delta linear attention. Without
flash-linear-attention, transformers falls back to a per-token torch loop that
runs sequentially over prefill and decode, so a caption takes minutes with no
indication of why.
ToriiGate 0.5 is a Qwen3.5 vision fine-tune trained on a single system prompt
and a single query structure, and it degrades on anything else. The shared qwen
handler strips angle brackets and underscores from the question, which mangles
the model's format templates, and the generic caption instructions are not what
it was trained on.
- hold the model's caption formats, system prompt and query builder in modules/caption/toriigate.py
- build the system prompt and user query from those templates in the qwen handler
- offer the native formats in the task dropdown and in the caption api prompt groups
- drop Normal Caption for this model, which has no format between short and long
"sage" is a different kernel per gpu: sageattention dispatches the pv dtype and
accumulator by arch and cuda version, and modules/attention.py forces the fp16-pv
cuda kernel on sm86, so one label across shared reports compares unlike kernels.
- resolve the row label from the dispatch: sm89 with cuda 12.8+ reads
"sage int8 qk + fp8 pv, fp32+fp16 accum", sm86 reads "fp16 pv, fp32 accum"
- name the sm86-only fp16-accum baseline for what it is
Notes ran to paragraph length and wrapped in the terminal, burying the numbers.
- lead with the topic, keep the numbers and the fix, drop the restatement
- one line per note at 140 columns, except the sd15 compile warning
sdnq no longer skips compile for fp8 storage, it upcasts e4m3 to the scale dtype
before the compiled dequant, so the tool's mirror of the deleted skip_fp8_compile
gate went dead and fed the raw e4m3 weight to its compiled dequant: on pre-ada gpus
that reports a compile failure for rows the webui runs fine.
- mirror the upcast in dequant_args, keep the hardware probe on the raw weight
- drop dequant_gated_to_eager and the gated-to-eager row, fp8 rows are measured now
- say upcast, not eager fallback, in the environment panel and dequant notes
Hiding the sdnext startup log by pointing fds 1 and 2 at a file kills the process
when stdout is a real console: sys.stdout writes through the win32 console api on
the handle behind fd 1, so the first bootstrap log line raises OSError 'the handle
is invalid', and with stderr broken the same way the interpreter aborts with no
message, no traceback and no results.
- swap sys.stdout/sys.stderr to a buffer along with the fd redirect
- capture the startup log instead of discarding it to devnull, replay it on failure
- catch SystemExit: loader.py and installer.py exit on fatal startup errors
A full-weight extraction on Z-Image bound 308 modules and applied 172 of
them, silently dropping the rest, and left 71 more unmapped.
assign_network_names_to_compvis_modules puts every transformer module in
network_layer_mapping but skips stamping network_layer_name on norms,
which is the attribute the apply pass keys off. try_load_full bound those
modules through the mapping and they then never applied; stamp them
loader-locally, as try_load_norm already does.
Z-Image also names three module groups differently from the diffusers
tree: the qk-norms (q_norm/k_norm vs norm_q/norm_k), and the patch
embedder and final layer, which live in ModuleDicts keyed by
"{patch_size}-{f_patch_size}" and so carry a key the checkpoint has no
notion of. Read that key from the live model rather than hardcoding it.
The counts close exactly: 68 qk-norms plus 3 non-block targets are the 71
that went unmapped.
The linear layers bind their scaled-mm function at import, so
SDNQ_USE_TRITON_MM freezes the backend per process and comparing triton
against the torch fallback meant two runs, carrying clock drift into the
delta. --mm-backends rebinds the function on the consuming modules
between benches, so both rows measure the same quantized layer under the
same clock state.
- swap targets are the four linear modules that from-import a scaled-mm
function; every swap resets dynamo, since the layer forwards are
compiled and the traced graph pins the previous function
- the torch row is captured from kernel_wrappers rather than
reimplemented, so it is unavailable where triton is the device default
(rerun with SDNQ_USE_TRITON_MM=0) and says so
- rounds alternate order and keep the fastest per row, so drift cancels
instead of favouring whichever backend runs second
- flag output error when backends disagree past 1e-4: they are meant to
be numerically equivalent
probe_fp8 and the broken-compile bench path forced eager input prep by
toggling torch._dynamo.config.disable. torch 2.13+ raises 'found no
compiled frames' when a fullgraph-compiled function is called inside a
disable window, so the probe failed before reaching the attention kernel
and reported an environment error instead of the kernel's real verdict,
on fp8-capable gpus too. Swap the module-global prep function for its
eager inner instead, matching the correctness section.
The environment panel and json now carry python, platform, backend and
the backend runtime versions (cuda/cudnn/driver, hip, ipex, openvino,
directml via devices.get_gpu_info), so shared reports identify the stack
without inferring it from the torch version string.
The webui bootstrap log is gated behind an fd-level suppression for the
import window: the bootstrap reconfigures its loggers during import and
onnxruntime warns from c++, so logger levels alone cannot silence it.
Import failures still print after the gate lifts.
zimage, chroma, ernie and krea2 chained only lora/lokr/loha/oft while
flux2 and anima ran all eight families, so ia3/glora/norm/full files
(e.g. full-diff extractions with diff/diff_b keys) reported not loaded
on the short-chain arches. The generic family loaders are arch-agnostic;
wire the missing four into each chain.
- add ia3/glora/norm/full wrappers and chain entries to the four arch
modules, with matching suffix/marker re-exports
- add a chain-level full-diff test per suite: zimage covers the legacy
attention.out alias and the fused-qkv skip, chroma the proj rename,
ernie the passthrough
LyCORIS extraction with use_sparse_bias saves bias_indices/bias_values/
bias_size per module: the sparse weight-shaped remainder of the SVD
extraction, named bias for historical reasons. The keys were dropped by
both loader paths, so extracted adapters applied without the residual
correction; the dense-bias branch in finalize_updown that consumes it
was unreachable.
- rebuild the COO tensor in NetworkModule.__init__ (int16 indices cast
to long), shared by the native and generic loaders; kept sparse so
the dense += sparse in finalize_updown materializes per module at
apply instead of near-model-size densification at load
- accept the triplet suffixes in LORA_SUFFIXES; fused targets skip
with the weight-shaped-bias warning
- cover an extraction-faithful numeric round-trip and the fused skip
in the offline suite
ai-toolkit DoRA saves lora_A/B plus a 1-D per-output magnitude key in
place of alpha; PEFT and diffusers name the same quantity
lora_magnitude_vector. Neither key was in the suffix table, so such
adapters loaded as plain LoRA with the magnitude renormalization
silently missing. The semantics match LyCORIS wd_on_out=True row norms,
so both keys convert onto the existing dora_scale path.
- accept .magnitude and .lora_magnitude_vector in LORA_SUFFIXES and
convert at group level in try_load_lora
- reshape 1-D vectors to (out, 1): on square layers the apply-time
orientation detection would otherwise renormalize the wrong axis
- cover square-layer numeric equality, fused-qkv slicing and the PEFT
key form in the offline suite
finalize_updown ran apply_weight_decompose on the unscaled delta and
multiplied the result by alpha/rank afterward. LyCORIS and ComfyUI both
bake alpha/rank into the diff before computing the row norms, so any
DoRA with alpha != rank renormalized against the wrong merged weight
(64% relative delta error for kohya-style alpha=1 rank=8; exact only
when alpha == rank, which full-matrix LoKR forces).
- scale updown by calc_scale() before apply_weight_decompose; apply
only the multiplier afterward
- multiplier lerps the full merged delta (0 disables, 1 equals the
trainer output); LyCORIS weight-mode ratio interpolation leaves the
diff applied at multiplier 0 and is not used
- add a numeric regression test mirroring the LyCORIS forward reference
A diff_b bias delta on a fused BFL target passed through whole and
failed at apply with a shape mismatch. diff_b stores one value per
output feature, so it partitions with the fused rows exactly like the
up-weight; slice it with the chunk in the LoRA loader. The legacy
weight-shaped bias key (LyCORIS sparse-residual heritage) has no
defined partition on a fused target and no known emitter pairs it with
chunk-capable families, so the group is skipped with a warning in the
LoRA, LoKR and LoHA loaders.
- add slice_bias_delta beside slice_dora_scale; warn and skip
non-per-output diff_b shapes
- cover sliced diff_b flowing out as ex_bias and the legacy-bias skip
in the offline suite
BFL-format chroma adapters targeting the embedders, final projection or
the distilled guidance layer MLPs resolved verbatim and unmapped: the
embedder/final-layer names differ from diffusers outright, and the
approximator MLP leaves are in_layer/out_layer in BFL but linear_1/
linear_2 in the diffusers PixArt projection. Only in_proj, out_proj and
norms.N shared names and bound.
- add CHROMA_EXTRA_MAP (kohya form derived) and GUIDANCE_LEAF_MAP to
both target resolvers
- route bare distilled_guidance_layer keys through the resolver instead
of the diffusers passthrough so both leaf namings resolve; add
img_in/txt_in/final_layer bare prefixes
- cover all key forms and end-to-end binding in the offline suite
A LoKR group whose Kronecker product does not fit the resolved module
previously bound anyway and failed at apply time as a caught per-module
error, leaving the adapter partially applied with only an error log.
Reject the group at load with a warning instead, matching the LoRA
path's shapes_match gate.
- lokr_kron_shape derives (out, in_flat) from full, rank-decomposed or
Tucker-rebuilt factors, folding conv kernel dims into in_flat
- lokr_shapes_match honors SDNQ original shapes and chunk partitions:
equal chunks need total * out rows, row-range slices an exact range;
the input dim is never chunked
- cover non-fused and fused rejection in the offline suite
LyCORIS wd=True saves a dora_scale companion for LoRA/LoHA/LoKR; on
fused BFL targets the chunk paths passed it through whole, so apply
failed with a shape mismatch and the module was dropped. Per-output
magnitudes (wd_on_out=True, the default) partition exactly with the
fused rows; per-input magnitudes couple the chunks through shared
column norms and have no exact split.
- slice per-output dora_scale rows with the chunk in the LoRA, LoKR
and LoHA loaders
- skip per-input DoRA on fused targets with a specific warning
- cover sliced and skipped orientations in the offline suite
The lycoris_ save format is arch-independent: LyCORIS standalone wraps
the loaded diffusers model and emits the wrapped module path with dots
as underscores, so verbatim passthrough is correct for any arch. Only
flux2 handled it; zimage, chroma, ernie and krea2 reported such files
as not loaded.
- add lycoris_ to KNOWN_PREFIXES_DEFAULT and PASSTHROUGH_PREFIXES_DEFAULT
- drop flux2's per-arch prefix append and resolve_targets branch
- add lycoris_ to ANIMA_PREFIXES (anima replaces the default tuple);
network_prefix_for already routes it to the transformer namespace
- cover the passthrough with a zimage loader test
BFL-format adapters targeting the embedders, timestep/guidance MLPs,
modulation layers and the final layer resolved to nothing and were
dropped as unmapped, for every adapter family on the f2 native path.
- add F2_EXTRA_MAP exact-match lookups in both target resolvers, with
the kohya underscore form derived from the BFL path
- add guidance_in. to BARE_FLUX_PREFIXES; groups targeting the guidance
embedder stay unmapped on models built without guidance_embeds
- extend the offline test mock with the non-block targets and cover all
three key forms plus full-matrix LoKR with placeholder alpha
After the correctness section aborts on a sticky kernel fault, the
remaining sections cannot run; flush outputs and exit instead of
cascading into the next cuda call's traceback.
Vladimir Mandic
Disty0
CalamitousFelicitousness
Oleksandr Liutyi