After the correctness section aborts on a sticky kernel fault, the
remaining sections cannot run; flush outputs and exit instead of
cascading into the next cuda call's traceback.
A kernel fault (misaligned address, illegal memory access) is sticky:
every later cuda call in the process fails, so one faulting check killed
the whole run through the next reference computation and lost the --save
and --json outputs. Compute the correctness reference inside the
per-check handler, detect a dead context and abort the section with a
clear message, and flush partial outputs from a top-level handler on any
crash or interrupt.
The fp8 attention probe can fail for reasons other than missing hardware
support (torch or triton compile issues); reporting those as 'not supported
on this gpu' is wrong on gpus where fp8 is native. Match the triton
pre-sm_89 signature and label anything else as an environment failure.
Toggling torch._dynamo.config.disable to force eager input prep fails on
newer torch: a fullgraph-compiled function called inside a disable window
raises 'found no compiled frames', failing every check and poisoning the
first compiled call afterwards. Swap the module-global prep function for
its eager inner instead, matching the dequant section's mechanism.
- store per-check errors and exceptions in the json report, so remote
reports carry the failure mode
The refine toggle doubles as the slot selector on the UNet/DiT page:
with it active a card click sets sd_unet_secondary instead of sd_unet,
mirroring base/refiner selection on the model page.
Benchmark weight dequantization alongside attention: eager vs compiled
dequantization, measured standalone and through the full linear forward,
plus the quantized matmul forward, across int/uint 8-6-4-2 and the
pre-quantized float formats (float8_e4m3fn, float8_e4m3fn_sdnq,
float4_e2m1fn group 16) at krea 2 layer geometry, against a bf16 nn.Linear
baseline and a true-fp32 output reference. Each row carries measured
storage size and one-shot quantize time; an svd/hadamard variants table
measures rotation and low-rank costs on top of the base dtypes.
A combined block section measures complete configurations (weights dtype x
matmul path x attention) end to end through a dit-style transformer block,
with output error at depth one and four against an fp32 reference block,
because component speedups and errors do not compose multiplicatively.
- recommendations weigh error against speed: a faster option is rejected
when it multiplies measured output error beyond 2x, and reason strings
cite both numbers; notes include measured size/error and speed/error
frontiers
- robustness: extreme-activation stress rows in the correctness matrix
(pass on finite output), max-token-error columns beside norm error, and
non-finite outputs labeled as verdicts; norm metrics alone hide
token-level corruption on outlier-heavy inputs
- probe compiled weight dequant for e4m3 and e5m2 storage and report the
fp8 compile gate status; on ampere e5m2 compiles while e4m3 does not
- new attention presets: sdxl-cross (cross-attention), qwen3-te (causal
gqa text encoder), wan22-cfg (batched cfg video); cross-attention
correctness check
- --sections, --dequant-dtypes, --dequant-variants and --block-configs
selectors, --json structured results output
Triton cannot compile e4m3 loads before sm_89, so fp8 weights fall back
to eager dequant there. The uint8-backed float8_e4m3fn_sdnq codec decodes
identically now that subnormals are handled (the two NaN codes become
+/-480), and its compiled dequant runs about 6x faster than eager native
fp8. Pre-quantized fp8 layers are viewed as uint8 at adoption when
compiled dequant is enabled on such hardware; the eager gate remains the
safety net for every other fp8 path.
detect_quant now reads _quantization_metadata as the authoritative quant
source when present, and resolves marker-file formats from the marked
layers' stored weight dtypes instead of file-wide dtype voting, which
mislabeled fp8 files carrying extra uint8 tensors and had no nvfp4
mapping at all. Schema bump so cached probe entries refresh.
nvfp4 layers keep their packed 4-bit codes and land on SDNQ grouped
quantization as float4_e2m1fn: the nibble order is swapped once at load,
the e4m3 block scales are unswizzled from the cuBLAS tile layout, and
the fp32 global scale folds into them as per-group scales. Marker
orig_shape acts as a cross-check and alignment padding is sliced
against the model dimensions.
- reject nvfp4 markers with unknown group sizes or convrot flags
- accept uint8 storage in the krea2 real-file check
Triton has no e4m3 conversions before sm_89, so any compiled graph
touching fp8 storage weights fails with an InductorError on Ampere.
Select the eager dequant and re-quantize paths for e4m3 weights when
the hardware cannot compile them; other dtypes keep compiled dequant.
SDNQ_ALLOW_FP8_COMPILE overrides the detection.
The secondary unet loads from the networks panel toggle or the settings
page; companion ordering and dynamic visibility pushes in ui_settings.py
are not worth their footprint for one setting.
The hidden change_refiner button passed key='sd_model_checkpoint', so
loading a card with the refine toggle active reloaded the base model
and never set sd_model_refiner.
json_helpers does its own error handling and file locking, so the
try/except wrappers, threading locks and isfile checks were dead
weight; probe cache file paths are now defined in paths.py.
Newer quantized checkpoints record per-layer formats in the safetensors
header _quantization_metadata instead of marker tensors. The native loader
now reads the header map, re-keys it through the same prefix strip as the
tensors, and transcodes it into marker tensors so both container forms
share one detection path; header entries win over markers.
- drop optional input_scale sidecars for marked layers
- map full_precision_matrix_mult onto the sdnq per-layer matmul exclusion list
- log the detection source (markers, header, both)
ConvRot is the regular Hadamard rotation SDNQ implements: identical
construction, normalization, axis, and dequant order. Per-layer markers
map onto the dequantizer's use_hadamard and hadamard_group_size; group
sizes must be powers of 4 and divide in_features, else base-repo
fallback. Detection carries per-layer metadata since files mix plain
and rotated layers.
transformer.dtype reports the storage dtype for fp8-quantized models
(fp8 params are floating, int8 params are not), so activations were
cast to fp8 and the forward crashed. Read the compute dtype from the
SDNQ dequantizer instead.
One UNET override cannot serve dual-transformer arches: ideogram4
conditional/unconditional and wan combined-stage experts need separate
files, and previously a single override landed on both experts.
- sd_unet_secondary option with per-slot tracking, consumed-state sync,
arch-change reset, and incompatible-override fallback
- dropdown renders beside the primary, follows it into quicksettings,
and is visible only for dual-transformer model types
- ideogram4 native single-file spec with a quant-aware fused-qkv
converter; such converters run before comfy_quant detection via
TransformerSpec.converter_handles_quant
- quicksettings render in configured order (sort keyed on the option
object and always fell back to alphabetical)
- post-load dtype warning skips quantized transformers
fp8_e4m3fn and fp8_e5m2 differ in kernel support, so a bare fp8 token
is not enough to know whether a file runs on a given architecture;
scaled_fp8 derives its token from the detected format.
Local probes read the marker tensor bytes for the true format string
(nvfp4, mxfp8, int8_tensorwise) instead of inferring from weight
dtypes; remote ranged peeks stay dtype-inferred. precision_token maps
quant format or dominant dtype to the short filename token.
peek-header now returns the full probe (architecture fingerprint,
dtypes, quant scheme) alongside the metadata block, and caches the
response persistently by civitai file id; file content per id is
immutable, so cached probes skip the ranged fetch entirely.
Analyzes safetensors/gguf headers without reading weights: curated
tensor-name fingerprints for common families, container kind detection
(model/lora/vae/text-encoder), element-weighted precision over core
tensors, comfy_quant/scaled-fp8/gguf quant detection, lora base
resolution via trainer metadata with shape arbitration, and full
__metadata__ passthrough. Results cached by mtime in
data/model_probe.json.
- analyze_header is pure and shared by remote ranged peeks and local
file reads
- fingerprints verified against the local library and live civitai
headers; unmatched families report unknown rather than guessing
The API carries no field distinguishing dual-transformer expert files;
conversion tools embed it in the header __metadata__. peek-header
range-fetches the JSON header and returns the metadata block without
downloading the file. Restricted to civitai.com urls.
CalamitousFelicitousness
Disty0
Vladimir Mandic