The refine toggle doubles as the slot selector on the UNet/DiT page:
with it active a card click sets sd_unet_secondary instead of sd_unet,
mirroring base/refiner selection on the model page.
Benchmark weight dequantization alongside attention: eager vs compiled
dequantization, measured standalone and through the full linear forward,
plus the quantized matmul forward, across int/uint 8-6-4-2 and the
pre-quantized float formats (float8_e4m3fn, float8_e4m3fn_sdnq,
float4_e2m1fn group 16) at krea 2 layer geometry, against a bf16 nn.Linear
baseline and a true-fp32 output reference. Each row carries measured
storage size and one-shot quantize time; an svd/hadamard variants table
measures rotation and low-rank costs on top of the base dtypes.
A combined block section measures complete configurations (weights dtype x
matmul path x attention) end to end through a dit-style transformer block,
with output error at depth one and four against an fp32 reference block,
because component speedups and errors do not compose multiplicatively.
- recommendations weigh error against speed: a faster option is rejected
when it multiplies measured output error beyond 2x, and reason strings
cite both numbers; notes include measured size/error and speed/error
frontiers
- robustness: extreme-activation stress rows in the correctness matrix
(pass on finite output), max-token-error columns beside norm error, and
non-finite outputs labeled as verdicts; norm metrics alone hide
token-level corruption on outlier-heavy inputs
- probe compiled weight dequant for e4m3 and e5m2 storage and report the
fp8 compile gate status; on ampere e5m2 compiles while e4m3 does not
- new attention presets: sdxl-cross (cross-attention), qwen3-te (causal
gqa text encoder), wan22-cfg (batched cfg video); cross-attention
correctness check
- --sections, --dequant-dtypes, --dequant-variants and --block-configs
selectors, --json structured results output
Triton cannot compile e4m3 loads before sm_89, so fp8 weights fall back
to eager dequant there. The uint8-backed float8_e4m3fn_sdnq codec decodes
identically now that subnormals are handled (the two NaN codes become
+/-480), and its compiled dequant runs about 6x faster than eager native
fp8. Pre-quantized fp8 layers are viewed as uint8 at adoption when
compiled dequant is enabled on such hardware; the eager gate remains the
safety net for every other fp8 path.
detect_quant now reads _quantization_metadata as the authoritative quant
source when present, and resolves marker-file formats from the marked
layers' stored weight dtypes instead of file-wide dtype voting, which
mislabeled fp8 files carrying extra uint8 tensors and had no nvfp4
mapping at all. Schema bump so cached probe entries refresh.
nvfp4 layers keep their packed 4-bit codes and land on SDNQ grouped
quantization as float4_e2m1fn: the nibble order is swapped once at load,
the e4m3 block scales are unswizzled from the cuBLAS tile layout, and
the fp32 global scale folds into them as per-group scales. Marker
orig_shape acts as a cross-check and alignment padding is sliced
against the model dimensions.
- reject nvfp4 markers with unknown group sizes or convrot flags
- accept uint8 storage in the krea2 real-file check
Triton has no e4m3 conversions before sm_89, so any compiled graph
touching fp8 storage weights fails with an InductorError on Ampere.
Select the eager dequant and re-quantize paths for e4m3 weights when
the hardware cannot compile them; other dtypes keep compiled dequant.
SDNQ_ALLOW_FP8_COMPILE overrides the detection.
The secondary unet loads from the networks panel toggle or the settings
page; companion ordering and dynamic visibility pushes in ui_settings.py
are not worth their footprint for one setting.
The hidden change_refiner button passed key='sd_model_checkpoint', so
loading a card with the refine toggle active reloaded the base model
and never set sd_model_refiner.
json_helpers does its own error handling and file locking, so the
try/except wrappers, threading locks and isfile checks were dead
weight; probe cache file paths are now defined in paths.py.
Newer quantized checkpoints record per-layer formats in the safetensors
header _quantization_metadata instead of marker tensors. The native loader
now reads the header map, re-keys it through the same prefix strip as the
tensors, and transcodes it into marker tensors so both container forms
share one detection path; header entries win over markers.
- drop optional input_scale sidecars for marked layers
- map full_precision_matrix_mult onto the sdnq per-layer matmul exclusion list
- log the detection source (markers, header, both)
ConvRot is the regular Hadamard rotation SDNQ implements: identical
construction, normalization, axis, and dequant order. Per-layer markers
map onto the dequantizer's use_hadamard and hadamard_group_size; group
sizes must be powers of 4 and divide in_features, else base-repo
fallback. Detection carries per-layer metadata since files mix plain
and rotated layers.
transformer.dtype reports the storage dtype for fp8-quantized models
(fp8 params are floating, int8 params are not), so activations were
cast to fp8 and the forward crashed. Read the compute dtype from the
SDNQ dequantizer instead.
One UNET override cannot serve dual-transformer arches: ideogram4
conditional/unconditional and wan combined-stage experts need separate
files, and previously a single override landed on both experts.
- sd_unet_secondary option with per-slot tracking, consumed-state sync,
arch-change reset, and incompatible-override fallback
- dropdown renders beside the primary, follows it into quicksettings,
and is visible only for dual-transformer model types
- ideogram4 native single-file spec with a quant-aware fused-qkv
converter; such converters run before comfy_quant detection via
TransformerSpec.converter_handles_quant
- quicksettings render in configured order (sort keyed on the option
object and always fell back to alphabetical)
- post-load dtype warning skips quantized transformers
fp8_e4m3fn and fp8_e5m2 differ in kernel support, so a bare fp8 token
is not enough to know whether a file runs on a given architecture;
scaled_fp8 derives its token from the detected format.
Local probes read the marker tensor bytes for the true format string
(nvfp4, mxfp8, int8_tensorwise) instead of inferring from weight
dtypes; remote ranged peeks stay dtype-inferred. precision_token maps
quant format or dominant dtype to the short filename token.
peek-header now returns the full probe (architecture fingerprint,
dtypes, quant scheme) alongside the metadata block, and caches the
response persistently by civitai file id; file content per id is
immutable, so cached probes skip the ranged fetch entirely.
Analyzes safetensors/gguf headers without reading weights: curated
tensor-name fingerprints for common families, container kind detection
(model/lora/vae/text-encoder), element-weighted precision over core
tensors, comfy_quant/scaled-fp8/gguf quant detection, lora base
resolution via trainer metadata with shape arbitration, and full
__metadata__ passthrough. Results cached by mtime in
data/model_probe.json.
- analyze_header is pure and shared by remote ranged peeks and local
file reads
- fingerprints verified against the local library and live civitai
headers; unmatched families report unknown rather than guessing
The API carries no field distinguishing dual-transformer expert files;
conversion tools embed it in the header __metadata__. peek-header
range-fetches the JSON header and returns the metadata block without
downloading the file. Restricted to civitai.com urls.
load_transformer consumes the sd_unet dropdown selection during a full
model load but never marked it as loaded, so the queued sd_unet
onchange callback always forced a second full reload. Sync
sd_unet.loaded_unet once the override is successfully consumed; the
incompatible-override fallback keeps its reset to Default.
Parameterize the pre-quantized comfy_quant path by format so fp8
tensorwise checkpoints load alongside int8; both map onto SDNQ's
symmetric dequant for the matching weights dtype. Containers are
mislabeled in the wild, so the stored weight dtype is validated
against the marker, mixed-format files are rejected, and markers
carrying a convrot rotation flag fall back to the base repo since
they may require runtime inverse rotation.
Versions bundle companion files typed by the file, not the model;
'Text Encoder' had no TYPE_MAP entry so companions landed in the
checkpoint folder where the loader never scans.
Detect comfy_quant markers in native single-file transformer loads and
adopt the pre-quantized tensors as SDNQ int8 layers instead of rejecting
them. ComfyUI int8_tensorwise is a strict subset of SDNQ symmetric int8,
so weights and scales are taken bit-exact with no dequantize-requantize
round trip; quantized matmul and scale-dtype settings apply as usual.
- detect_comfy_quant/remap_comfy_quant helpers plus a prequantized
builder in native_transformer; file markers dictate the layer set
- unsupported formats and wrong-arch markers fall back to the base repo
- offline unit tests and an opt-in real-file krea2 test
Interrupted downloads leave url-hash .tmp resume files that nothing
cleans up; the queue that knows about them is in-memory only.
- iter_type_roots lists every folder downloads resolve into
- partials older than 7 days by mtime are deleted when the download
manager starts; active partials always have a fresh mtime
Multi-variant versions serve one canonical filename per file; the
precision, size class, format, and scan status that distinguish them
were dropped at the model layer.
- CivitFile gains metadata (format, size, fp), pickle and virus scan
results, and scannedAt
History entries stored only the query or tag term, so restoring a
search dropped every filter, and filter-only browsing was never
recorded at all.
- SearchHistory.add takes an optional params dict stored on the entry
- get_search records non-default filters: type, sort, period, base
models, nsfw, username, favorites
- filter-only searches are recorded as 'filter' entries labeled with
the filter values
A 200 reply to a resumed download carries the full file, not the
remainder; appending it to the partial produced a corrupt file that
passed the size check and only failed at hash verification. Truncate
the temp file and restart from byte 0 when a Range request comes back
as 200 instead of 206.
The /model-versions endpoint now serializes availability as null, which
failed str validation and turned every version lookup into a 404. Coerce
null to the default like the creator username validator already does.
Pass through earlyAccessEndsAt and earlyAccessConfig so clients can see
that a download requires a Buzz purchase before requesting the file.
sdnext enables tf32 globally; a math-backend dispatch fallback (gqa,
some mask shapes) degrades the fp32 reference to tf32 precision,
measured 4.2e-4 vs 5.0e-7 relative to fp64. the default mem-efficient
backend is unaffected.
Recommendations must hold on any gpu, so every verdict now comes from
measuring the configuration the advice points to.
- run benchmarks in the webui's configured dtype, --dtype overrides
- bench the prep mode matching the environment: compiled, verified
dynamic=false workaround, or eager; label each with the webui change
the numbers correspond to
- add a prep column for the q/k/v quantization cost outside the
kernel; eager prep measured 1.9 ms vs 0.2 ms fused at sdxl,
inverting the quantized matmul verdict
- compare Use Quantized MatMul against the unquantized sdnq row so
quantization pays for its own prep
- star the lowest-error config within 5% of the fastest sdnq time;
the same rule drives the worth-enabling verdict vs torch sdpa
- drop the (auto, default) suffix; document the settings mapping and
star rule in the notes
- wrap correctness and shape tables in panels to separate sections
The pv matmul recommendation trusted the float8 hardware probe alone,
so gpus where fp8 pv compiles but runs slower than unquantized pv got
a recommendation that degrades generation speed and error. Measured on
an rtx 2000 ada, fp8 pv lands at x0.93 of the sdpa baseline while int8
qk alone reaches x1.26.
- add fp8 pv and fp8 qk bench configs, gated on the eager float8
probe so unsupported gpus keep clean tables
- include fp8 pv in the video presets since the pv stage matters most
at long sequence lengths
- recommend a quantized pv type only when the fastest measured pv
variant beats plain int8 qk by at least 5%, otherwise recommend auto
and name the variants that measured no gain
- cite the measured fp8 qk ratio in the matmul type reason instead of
the static compile claim
The environment panel reported triton as missing and float8 as
unsupported on ada gpus. The triton lookup used the distribution name,
which varies by platform (triton-windows on windows), and the fp8 probe
ran through the torch.compile input prep, so a broken compile toolchain
read as a hardware limitation with the actual error discarded.
- read the module version for the triton line instead of the
distribution name
- probe float8 with eager input prep so the result reflects gpu
capability; show the failure reason when genuinely unsupported
- probe the compiled input prep separately and cold, before any eager
kernel run: inductor lowers part of the dynamic-shape prep to a cpu
helper kernel, so a missing host c++ compiler fails every sdnq
attention call at generation, and a prior eager run in the same
process masks the cold-start failure the webui hits
- on failure, verify the SDNQ_COMPILE_KWARGS dynamic:false workaround
live and report it in the environment panel and recommendation notes
- benchmark with eager input prep when the compiled path is broken so
kernel numbers still fill in
- replace the unicode star marker with ascii: rich's legacy windows
renderer crashes with a cp1252 UnicodeEncodeError on piped output
Vladimir Mandic
CalamitousFelicitousness
Disty0