detect_quant now reads _quantization_metadata as the authoritative quant
source when present, and resolves marker-file formats from the marked
layers' stored weight dtypes instead of file-wide dtype voting, which
mislabeled fp8 files carrying extra uint8 tensors and had no nvfp4
mapping at all. Schema bump so cached probe entries refresh.
nvfp4 layers keep their packed 4-bit codes and land on SDNQ grouped
quantization as float4_e2m1fn: the nibble order is swapped once at load,
the e4m3 block scales are unswizzled from the cuBLAS tile layout, and
the fp32 global scale folds into them as per-group scales. Marker
orig_shape acts as a cross-check and alignment padding is sliced
against the model dimensions.
- reject nvfp4 markers with unknown group sizes or convrot flags
- accept uint8 storage in the krea2 real-file check
Triton has no e4m3 conversions before sm_89, so any compiled graph
touching fp8 storage weights fails with an InductorError on Ampere.
Select the eager dequant and re-quantize paths for e4m3 weights when
the hardware cannot compile them; other dtypes keep compiled dequant.
SDNQ_ALLOW_FP8_COMPILE overrides the detection.
The secondary unet loads from the networks panel toggle or the settings
page; companion ordering and dynamic visibility pushes in ui_settings.py
are not worth their footprint for one setting.
The hidden change_refiner button passed key='sd_model_checkpoint', so
loading a card with the refine toggle active reloaded the base model
and never set sd_model_refiner.
json_helpers does its own error handling and file locking, so the
try/except wrappers, threading locks and isfile checks were dead
weight; probe cache file paths are now defined in paths.py.
Newer quantized checkpoints record per-layer formats in the safetensors
header _quantization_metadata instead of marker tensors. The native loader
now reads the header map, re-keys it through the same prefix strip as the
tensors, and transcodes it into marker tensors so both container forms
share one detection path; header entries win over markers.
- drop optional input_scale sidecars for marked layers
- map full_precision_matrix_mult onto the sdnq per-layer matmul exclusion list
- log the detection source (markers, header, both)
ConvRot is the regular Hadamard rotation SDNQ implements: identical
construction, normalization, axis, and dequant order. Per-layer markers
map onto the dequantizer's use_hadamard and hadamard_group_size; group
sizes must be powers of 4 and divide in_features, else base-repo
fallback. Detection carries per-layer metadata since files mix plain
and rotated layers.
transformer.dtype reports the storage dtype for fp8-quantized models
(fp8 params are floating, int8 params are not), so activations were
cast to fp8 and the forward crashed. Read the compute dtype from the
SDNQ dequantizer instead.
One UNET override cannot serve dual-transformer arches: ideogram4
conditional/unconditional and wan combined-stage experts need separate
files, and previously a single override landed on both experts.
- sd_unet_secondary option with per-slot tracking, consumed-state sync,
arch-change reset, and incompatible-override fallback
- dropdown renders beside the primary, follows it into quicksettings,
and is visible only for dual-transformer model types
- ideogram4 native single-file spec with a quant-aware fused-qkv
converter; such converters run before comfy_quant detection via
TransformerSpec.converter_handles_quant
- quicksettings render in configured order (sort keyed on the option
object and always fell back to alphabetical)
- post-load dtype warning skips quantized transformers
fp8_e4m3fn and fp8_e5m2 differ in kernel support, so a bare fp8 token
is not enough to know whether a file runs on a given architecture;
scaled_fp8 derives its token from the detected format.
Local probes read the marker tensor bytes for the true format string
(nvfp4, mxfp8, int8_tensorwise) instead of inferring from weight
dtypes; remote ranged peeks stay dtype-inferred. precision_token maps
quant format or dominant dtype to the short filename token.
peek-header now returns the full probe (architecture fingerprint,
dtypes, quant scheme) alongside the metadata block, and caches the
response persistently by civitai file id; file content per id is
immutable, so cached probes skip the ranged fetch entirely.
Analyzes safetensors/gguf headers without reading weights: curated
tensor-name fingerprints for common families, container kind detection
(model/lora/vae/text-encoder), element-weighted precision over core
tensors, comfy_quant/scaled-fp8/gguf quant detection, lora base
resolution via trainer metadata with shape arbitration, and full
__metadata__ passthrough. Results cached by mtime in
data/model_probe.json.
- analyze_header is pure and shared by remote ranged peeks and local
file reads
- fingerprints verified against the local library and live civitai
headers; unmatched families report unknown rather than guessing
The API carries no field distinguishing dual-transformer expert files;
conversion tools embed it in the header __metadata__. peek-header
range-fetches the JSON header and returns the metadata block without
downloading the file. Restricted to civitai.com urls.
load_transformer consumes the sd_unet dropdown selection during a full
model load but never marked it as loaded, so the queued sd_unet
onchange callback always forced a second full reload. Sync
sd_unet.loaded_unet once the override is successfully consumed; the
incompatible-override fallback keeps its reset to Default.
Parameterize the pre-quantized comfy_quant path by format so fp8
tensorwise checkpoints load alongside int8; both map onto SDNQ's
symmetric dequant for the matching weights dtype. Containers are
mislabeled in the wild, so the stored weight dtype is validated
against the marker, mixed-format files are rejected, and markers
carrying a convrot rotation flag fall back to the base repo since
they may require runtime inverse rotation.
Versions bundle companion files typed by the file, not the model;
'Text Encoder' had no TYPE_MAP entry so companions landed in the
checkpoint folder where the loader never scans.
Detect comfy_quant markers in native single-file transformer loads and
adopt the pre-quantized tensors as SDNQ int8 layers instead of rejecting
them. ComfyUI int8_tensorwise is a strict subset of SDNQ symmetric int8,
so weights and scales are taken bit-exact with no dequantize-requantize
round trip; quantized matmul and scale-dtype settings apply as usual.
- detect_comfy_quant/remap_comfy_quant helpers plus a prequantized
builder in native_transformer; file markers dictate the layer set
- unsupported formats and wrong-arch markers fall back to the base repo
- offline unit tests and an opt-in real-file krea2 test
Interrupted downloads leave url-hash .tmp resume files that nothing
cleans up; the queue that knows about them is in-memory only.
- iter_type_roots lists every folder downloads resolve into
- partials older than 7 days by mtime are deleted when the download
manager starts; active partials always have a fresh mtime
Multi-variant versions serve one canonical filename per file; the
precision, size class, format, and scan status that distinguish them
were dropped at the model layer.
- CivitFile gains metadata (format, size, fp), pickle and virus scan
results, and scannedAt
History entries stored only the query or tag term, so restoring a
search dropped every filter, and filter-only browsing was never
recorded at all.
- SearchHistory.add takes an optional params dict stored on the entry
- get_search records non-default filters: type, sort, period, base
models, nsfw, username, favorites
- filter-only searches are recorded as 'filter' entries labeled with
the filter values
A 200 reply to a resumed download carries the full file, not the
remainder; appending it to the partial produced a corrupt file that
passed the size check and only failed at hash verification. Truncate
the temp file and restart from byte 0 when a Range request comes back
as 200 instead of 206.
The /model-versions endpoint now serializes availability as null, which
failed str validation and turned every version lookup into a 404. Coerce
null to the default like the creator username validator already does.
Pass through earlyAccessEndsAt and earlyAccessConfig so clients can see
that a download requires a Buzz purchase before requesting the file.
sdnext enables tf32 globally; a math-backend dispatch fallback (gqa,
some mask shapes) degrades the fp32 reference to tf32 precision,
measured 4.2e-4 vs 5.0e-7 relative to fp64. the default mem-efficient
backend is unaffected.
Recommendations must hold on any gpu, so every verdict now comes from
measuring the configuration the advice points to.
- run benchmarks in the webui's configured dtype, --dtype overrides
- bench the prep mode matching the environment: compiled, verified
dynamic=false workaround, or eager; label each with the webui change
the numbers correspond to
- add a prep column for the q/k/v quantization cost outside the
kernel; eager prep measured 1.9 ms vs 0.2 ms fused at sdxl,
inverting the quantized matmul verdict
- compare Use Quantized MatMul against the unquantized sdnq row so
quantization pays for its own prep
- star the lowest-error config within 5% of the fastest sdnq time;
the same rule drives the worth-enabling verdict vs torch sdpa
- drop the (auto, default) suffix; document the settings mapping and
star rule in the notes
- wrap correctness and shape tables in panels to separate sections
The pv matmul recommendation trusted the float8 hardware probe alone,
so gpus where fp8 pv compiles but runs slower than unquantized pv got
a recommendation that degrades generation speed and error. Measured on
an rtx 2000 ada, fp8 pv lands at x0.93 of the sdpa baseline while int8
qk alone reaches x1.26.
- add fp8 pv and fp8 qk bench configs, gated on the eager float8
probe so unsupported gpus keep clean tables
- include fp8 pv in the video presets since the pv stage matters most
at long sequence lengths
- recommend a quantized pv type only when the fastest measured pv
variant beats plain int8 qk by at least 5%, otherwise recommend auto
and name the variants that measured no gain
- cite the measured fp8 qk ratio in the matmul type reason instead of
the static compile claim
The environment panel reported triton as missing and float8 as
unsupported on ada gpus. The triton lookup used the distribution name,
which varies by platform (triton-windows on windows), and the fp8 probe
ran through the torch.compile input prep, so a broken compile toolchain
read as a hardware limitation with the actual error discarded.
- read the module version for the triton line instead of the
distribution name
- probe float8 with eager input prep so the result reflects gpu
capability; show the failure reason when genuinely unsupported
- probe the compiled input prep separately and cold, before any eager
kernel run: inductor lowers part of the dynamic-shape prep to a cpu
helper kernel, so a missing host c++ compiler fails every sdnq
attention call at generation, and a prior eager run in the same
process masks the cold-start failure the webui hits
- on failure, verify the SDNQ_COMPILE_KWARGS dynamic:false workaround
live and report it in the environment panel and recommendation notes
- benchmark with eager input prep when the compiled path is broken so
kernel numbers still fill in
- replace the unicode star marker with ascii: rich's legacy windows
renderer crashes with a cp1252 UnicodeEncodeError on piped output
Add the official Anima 1.0 Aesthetic and Turbo releases in diffusers
format. The previous turbo entry pointed at an unofficial build (base
with a fused community turbo lora); the official distilled checkpoint
replaces it.
- reference-base.json: add anima 1.0 aesthetic
- reference-distilled.json: point anima 1.0 turbo at the official v1.0
- add reference previews, remove the replaced turbo preview
allow_shared=False predates the Qwen-3 0.5B entries in shared_te_map, so
every Anima repo downloaded its own copy of the text encoder. Remove it
so all variants share the canonical Anima-1.0-Base text encoder.
Benchmark and validate sdnq attention on the local gpu: correctness
checks for mask, causal, gqa and padding code paths, float8 support
probe, per-shape speed and error comparison against torch sdpa and
sageattention, and measured recommendations for the settings section.
Tables fill in live with per-config compile/warmup/timing progress,
current settings are compared against recommended values, and --save
writes a plain-text transcript of the results. --config-timeout turns
stuck compiles into failed rows; sd15 skips its hadamard configs since
compiling hadamard with a non pow2 head dim currently hangs inductor.
- shape presets follow real model geometries: sd15, sdxl, anima,
flux2 klein, wan22 a14b, ltx 2.3, plus a masked joint-attention case
- runs the shipped kernel from modules/sdnq/kernels/triton_atten.py
directly, with stock sdpa restored for baselines and references
All-in-one exports bundle the text encoder and VAE alongside the
transformer under LDM-style family prefixes (cond_stage_model.,
first_stage_model., text_encoders., vae.). The native loader treated
those keys as a mixed-prefix error and rejected the file. Drop known
companion families before prefix detection and log what was skipped;
keys matching neither a transformer prefix nor a known family still
raise. TE and VAE keep coming from the base repo or their own overrides.
CalamitousFelicitousness
Vladimir Mandic
Disty0