Offline suite for the placement roles: the role table over the component
names sdnext loads, one dispatch arm per component with hooks landing on
text encoder wrappers, force sweeps scoped to stamped modules, a settings
change re-placing a resident component, on-demand contracts, enumeration
on both pipeline kinds, and the upstream markers the roles read.
The ref2va checkpoint partition conditions on reference images instead
of keyframes, so it gets its own registry row and reference card, and
the video core marshals PIL images into task_args as
MiniMaxH3ImageReference. Images are converted to RGB first, since the
reference encoder reads the array raw. The keyframe path is unchanged.
Validation runs before the model load in one funnel shared by the tab
and the API, so a rejected request costs nothing: references on a
non-reference model, a reference model with nothing to condition on,
more than nine images, non-images, and aspect outside 1:4 to 4:1 all
return 400. The image path rejects a reference pipe without references
instead of reaching a transformer that was never loaded.
Add POST /sdapi/v1/video plus GET /sdapi/v1/video/models and
GET /sdapi/v1/video/file. The generation body is extracted from the
gradio handler into a keyword-only core, video_run.run, which returns a
structured result and raises typed errors; the positional generate
signature is unchanged and now adapts to the core. Omitting engine and
model drives the currently loaded checkpoint when it is video-capable,
which covers models loaded from local folders without a registry entry.
- registry helpers in models_def (find, engines, pipeline_classes,
workflow_for_class); validate_pipeline reuses the shared class set
- modular pipes stamp their workflow so out-of-registry loads dispatch
onto the modular branch
- disk switches (mp4_*) and wire switches (send_*) are independent;
artifacts above the base64 cap fall back to path plus the file route,
which is jailed to the video output directory and serves video/mp4
with range support
- always-on video scripts get bootstrapped default args, matching the
txt2img handler; missing bootstrap raised a TypeError per frame
- checkpoint overrides are rejected with a pointer to the checkpoint
endpoint; unknown engine, model and sampler names return 404 with the
valid choices
- cli/api-video.py client, test/test-video-api.py suite and a
full-test.sh entry; video mimetypes registered; rate-limit cost set
- remove the unreferenced video_ui.run_video dispatcher
The Krea 2 transformer keeps checkpoint-style module names while the
official krea/Krea-2-LoRA releases are saved with upstream-diffusers
names, so all 264 modules failed to bind and the LoRAs silently did
nothing. Krea 2 is the only native-LoRA arch with an sdnext-owned
transformer, so its module names diverge from the diffusers ecosystem.
- native_adapter.resolve_group_targets consults the arch resolver first
for passthrough prefixes, falling back to verbatim binding; a no-op
for arches that load the diffusers class
- krea2_lora maps diffusers attn/ff/text_fusion/embedder names onto the
checkpoint module tree; checkpoint-named LoRAs still bind verbatim
- add test/test-krea2-native-adapters.py
A full-weight extraction on Z-Image bound 308 modules and applied 172 of
them, silently dropping the rest, and left 71 more unmapped.
assign_network_names_to_compvis_modules puts every transformer module in
network_layer_mapping but skips stamping network_layer_name on norms,
which is the attribute the apply pass keys off. try_load_full bound those
modules through the mapping and they then never applied; stamp them
loader-locally, as try_load_norm already does.
Z-Image also names three module groups differently from the diffusers
tree: the qk-norms (q_norm/k_norm vs norm_q/norm_k), and the patch
embedder and final layer, which live in ModuleDicts keyed by
"{patch_size}-{f_patch_size}" and so carry a key the checkpoint has no
notion of. Read that key from the live model rather than hardcoding it.
The counts close exactly: 68 qk-norms plus 3 non-block targets are the 71
that went unmapped.
zimage, chroma, ernie and krea2 chained only lora/lokr/loha/oft while
flux2 and anima ran all eight families, so ia3/glora/norm/full files
(e.g. full-diff extractions with diff/diff_b keys) reported not loaded
on the short-chain arches. The generic family loaders are arch-agnostic;
wire the missing four into each chain.
- add ia3/glora/norm/full wrappers and chain entries to the four arch
modules, with matching suffix/marker re-exports
- add a chain-level full-diff test per suite: zimage covers the legacy
attention.out alias and the fused-qkv skip, chroma the proj rename,
ernie the passthrough
LyCORIS extraction with use_sparse_bias saves bias_indices/bias_values/
bias_size per module: the sparse weight-shaped remainder of the SVD
extraction, named bias for historical reasons. The keys were dropped by
both loader paths, so extracted adapters applied without the residual
correction; the dense-bias branch in finalize_updown that consumes it
was unreachable.
- rebuild the COO tensor in NetworkModule.__init__ (int16 indices cast
to long), shared by the native and generic loaders; kept sparse so
the dense += sparse in finalize_updown materializes per module at
apply instead of near-model-size densification at load
- accept the triplet suffixes in LORA_SUFFIXES; fused targets skip
with the weight-shaped-bias warning
- cover an extraction-faithful numeric round-trip and the fused skip
in the offline suite
ai-toolkit DoRA saves lora_A/B plus a 1-D per-output magnitude key in
place of alpha; PEFT and diffusers name the same quantity
lora_magnitude_vector. Neither key was in the suffix table, so such
adapters loaded as plain LoRA with the magnitude renormalization
silently missing. The semantics match LyCORIS wd_on_out=True row norms,
so both keys convert onto the existing dora_scale path.
- accept .magnitude and .lora_magnitude_vector in LORA_SUFFIXES and
convert at group level in try_load_lora
- reshape 1-D vectors to (out, 1): on square layers the apply-time
orientation detection would otherwise renormalize the wrong axis
- cover square-layer numeric equality, fused-qkv slicing and the PEFT
key form in the offline suite
finalize_updown ran apply_weight_decompose on the unscaled delta and
multiplied the result by alpha/rank afterward. LyCORIS and ComfyUI both
bake alpha/rank into the diff before computing the row norms, so any
DoRA with alpha != rank renormalized against the wrong merged weight
(64% relative delta error for kohya-style alpha=1 rank=8; exact only
when alpha == rank, which full-matrix LoKR forces).
- scale updown by calc_scale() before apply_weight_decompose; apply
only the multiplier afterward
- multiplier lerps the full merged delta (0 disables, 1 equals the
trainer output); LyCORIS weight-mode ratio interpolation leaves the
diff applied at multiplier 0 and is not used
- add a numeric regression test mirroring the LyCORIS forward reference
A diff_b bias delta on a fused BFL target passed through whole and
failed at apply with a shape mismatch. diff_b stores one value per
output feature, so it partitions with the fused rows exactly like the
up-weight; slice it with the chunk in the LoRA loader. The legacy
weight-shaped bias key (LyCORIS sparse-residual heritage) has no
defined partition on a fused target and no known emitter pairs it with
chunk-capable families, so the group is skipped with a warning in the
LoRA, LoKR and LoHA loaders.
- add slice_bias_delta beside slice_dora_scale; warn and skip
non-per-output diff_b shapes
- cover sliced diff_b flowing out as ex_bias and the legacy-bias skip
in the offline suite
BFL-format chroma adapters targeting the embedders, final projection or
the distilled guidance layer MLPs resolved verbatim and unmapped: the
embedder/final-layer names differ from diffusers outright, and the
approximator MLP leaves are in_layer/out_layer in BFL but linear_1/
linear_2 in the diffusers PixArt projection. Only in_proj, out_proj and
norms.N shared names and bound.
- add CHROMA_EXTRA_MAP (kohya form derived) and GUIDANCE_LEAF_MAP to
both target resolvers
- route bare distilled_guidance_layer keys through the resolver instead
of the diffusers passthrough so both leaf namings resolve; add
img_in/txt_in/final_layer bare prefixes
- cover all key forms and end-to-end binding in the offline suite
A LoKR group whose Kronecker product does not fit the resolved module
previously bound anyway and failed at apply time as a caught per-module
error, leaving the adapter partially applied with only an error log.
Reject the group at load with a warning instead, matching the LoRA
path's shapes_match gate.
- lokr_kron_shape derives (out, in_flat) from full, rank-decomposed or
Tucker-rebuilt factors, folding conv kernel dims into in_flat
- lokr_shapes_match honors SDNQ original shapes and chunk partitions:
equal chunks need total * out rows, row-range slices an exact range;
the input dim is never chunked
- cover non-fused and fused rejection in the offline suite
LyCORIS wd=True saves a dora_scale companion for LoRA/LoHA/LoKR; on
fused BFL targets the chunk paths passed it through whole, so apply
failed with a shape mismatch and the module was dropped. Per-output
magnitudes (wd_on_out=True, the default) partition exactly with the
fused rows; per-input magnitudes couple the chunks through shared
column norms and have no exact split.
- slice per-output dora_scale rows with the chunk in the LoRA, LoKR
and LoHA loaders
- skip per-input DoRA on fused targets with a specific warning
- cover sliced and skipped orientations in the offline suite
The lycoris_ save format is arch-independent: LyCORIS standalone wraps
the loaded diffusers model and emits the wrapped module path with dots
as underscores, so verbatim passthrough is correct for any arch. Only
flux2 handled it; zimage, chroma, ernie and krea2 reported such files
as not loaded.
- add lycoris_ to KNOWN_PREFIXES_DEFAULT and PASSTHROUGH_PREFIXES_DEFAULT
- drop flux2's per-arch prefix append and resolve_targets branch
- add lycoris_ to ANIMA_PREFIXES (anima replaces the default tuple);
network_prefix_for already routes it to the transformer namespace
- cover the passthrough with a zimage loader test
BFL-format adapters targeting the embedders, timestep/guidance MLPs,
modulation layers and the final layer resolved to nothing and were
dropped as unmapped, for every adapter family on the f2 native path.
- add F2_EXTRA_MAP exact-match lookups in both target resolvers, with
the kohya underscore form derived from the BFL path
- add guidance_in. to BARE_FLUX_PREFIXES; groups targeting the guidance
embedder stay unmapped on models built without guidance_embeds
- extend the offline test mock with the non-block targets and cover all
three key forms plus full-matrix LoKR with placeholder alpha
Triton cannot compile e4m3 loads before sm_89, so fp8 weights fall back
to eager dequant there. The uint8-backed float8_e4m3fn_sdnq codec decodes
identically now that subnormals are handled (the two NaN codes become
+/-480), and its compiled dequant runs about 6x faster than eager native
fp8. Pre-quantized fp8 layers are viewed as uint8 at adoption when
compiled dequant is enabled on such hardware; the eager gate remains the
safety net for every other fp8 path.
detect_quant now reads _quantization_metadata as the authoritative quant
source when present, and resolves marker-file formats from the marked
layers' stored weight dtypes instead of file-wide dtype voting, which
mislabeled fp8 files carrying extra uint8 tensors and had no nvfp4
mapping at all. Schema bump so cached probe entries refresh.
nvfp4 layers keep their packed 4-bit codes and land on SDNQ grouped
quantization as float4_e2m1fn: the nibble order is swapped once at load,
the e4m3 block scales are unswizzled from the cuBLAS tile layout, and
the fp32 global scale folds into them as per-group scales. Marker
orig_shape acts as a cross-check and alignment padding is sliced
against the model dimensions.
- reject nvfp4 markers with unknown group sizes or convrot flags
- accept uint8 storage in the krea2 real-file check
Newer quantized checkpoints record per-layer formats in the safetensors
header _quantization_metadata instead of marker tensors. The native loader
now reads the header map, re-keys it through the same prefix strip as the
tensors, and transcodes it into marker tensors so both container forms
share one detection path; header entries win over markers.
- drop optional input_scale sidecars for marked layers
- map full_precision_matrix_mult onto the sdnq per-layer matmul exclusion list
- log the detection source (markers, header, both)
ConvRot is the regular Hadamard rotation SDNQ implements: identical
construction, normalization, axis, and dequant order. Per-layer markers
map onto the dequantizer's use_hadamard and hadamard_group_size; group
sizes must be powers of 4 and divide in_features, else base-repo
fallback. Detection carries per-layer metadata since files mix plain
and rotated layers.
One UNET override cannot serve dual-transformer arches: ideogram4
conditional/unconditional and wan combined-stage experts need separate
files, and previously a single override landed on both experts.
- sd_unet_secondary option with per-slot tracking, consumed-state sync,
arch-change reset, and incompatible-override fallback
- dropdown renders beside the primary, follows it into quicksettings,
and is visible only for dual-transformer model types
- ideogram4 native single-file spec with a quant-aware fused-qkv
converter; such converters run before comfy_quant detection via
TransformerSpec.converter_handles_quant
- quicksettings render in configured order (sort keyed on the option
object and always fell back to alphabetical)
- post-load dtype warning skips quantized transformers
load_transformer consumes the sd_unet dropdown selection during a full
model load but never marked it as loaded, so the queued sd_unet
onchange callback always forced a second full reload. Sync
sd_unet.loaded_unet once the override is successfully consumed; the
incompatible-override fallback keeps its reset to Default.
Parameterize the pre-quantized comfy_quant path by format so fp8
tensorwise checkpoints load alongside int8; both map onto SDNQ's
symmetric dequant for the matching weights dtype. Containers are
mislabeled in the wild, so the stored weight dtype is validated
against the marker, mixed-format files are rejected, and markers
carrying a convrot rotation flag fall back to the base repo since
they may require runtime inverse rotation.
Detect comfy_quant markers in native single-file transformer loads and
adopt the pre-quantized tensors as SDNQ int8 layers instead of rejecting
them. ComfyUI int8_tensorwise is a strict subset of SDNQ symmetric int8,
so weights and scales are taken bit-exact with no dequantize-requantize
round trip; quantized matmul and scale-dtype settings apply as usual.
- detect_comfy_quant/remap_comfy_quant helpers plus a prequantized
builder in native_transformer; file markers dictate the layer set
- unsupported formats and wrong-arch markers fall back to the base repo
- offline unit tests and an opt-in real-file krea2 test
All-in-one exports bundle the text encoder and VAE alongside the
transformer under LDM-style family prefixes (cond_stage_model.,
first_stage_model., text_encoders., vae.). The native loader treated
those keys as a mixed-prefix error and rejected the file. Drop known
companion families before prefix detection and log what was skipped;
keys matching neither a transformer prefix nor a known family still
raise. TE and VAE keep coming from the base repo or their own overrides.
strip_prefix returns the detected prefix instead of logging it, so the load
summary reports it alongside the key count and the redundant reading-state
marker is gone. Four near-identical cls/file lines become one.
Base ships last.up all-zeros so the branch is a no-op; pre-branch
finetunes omit both keys. A zero_init_missing spec field zero-fills
them on load instead of falling back to the base transformer.
Route nn.Embedding targets (and the SDNQEmbedding / ScaledWordEmbedding subclasses) through the linear LoRA path: the weight delta is up@down over the [vocab, dim] table, same shape and merge as a Linear.
Apply a companion bias delta (diff_b) as ex_bias on the same module rather than dropping it; collect diff_b into the LoRA group so it rides the existing module instead of a separate Full module that would collide on the network key.
Krea 2 is a 12.9B single-stream flow-matching DiT trained from scratch, using a Qwen3-VL-4B text encoder and the Qwen-Image VAE. The transformer is vendored as a diffusers ModelMixin whose module tree mirrors the checkpoint, so weights load with no key conversion; the pipeline ports the reference encode, flow-matching denoise, and VAE decode. The text encoder is shared at runtime via the existing dedup registry, so Base and Turbo reuse one Qwen3-VL-4B copy.
Covers text-to-image, image-to-image, native LoRA, and the single-file UNET override. Also completes SD.Next's partial Qwen-Image VAE support (5D decode input and TAESD preview mapping) that K2 shares.
transformer., bare-diffusers, and lora_transformer_ bases are already in
network-key form for every arch, yet each per-arch resolve_targets repeated the
same passthrough branch for them. Move that into a shared
PASSTHROUGH_PREFIXES_DEFAULT set consulted by resolve_group_targets, leaving each
arch's resolve_targets to only the prefixes it actually rewrites (kohya / BFL).
lycoris_ stays in flux2, the one arch that recognizes it.
Pure refactor: the same keys resolve to the same modules.
Add an end-to-end regression per diffusers arch (chroma, flux2, zimage, ernie)
that loads a lora_transformer_ diffusers-flat state dict and checks the
expected modules bind. The chroma suite also asserts the shared
resolve_group_targets passthrough runs above each arch's own resolve_targets,
which returns nothing for this prefix.
Drives txt2img/img2img with the full per-sampler scheduler-option matrix
(sigma method, prediction type, timestep spacing, beta schedule, solver
order, shift, low order, thresholding, dynamic shift, rescale betas), one
option at a time over a pinned baseline, and measures every case against
the default: applied, inert, marginal, rejected, or fallback. Detects
silent fallback via the infotext Scheduler class and annotates every saved
image with its full requested configuration. --sweep runs a reduced matrix
per sampler and emits an empirical capability report.
ERSDEScheduler now accepts use_karras_sigmas, use_exponential_sigmas,
use_beta_sigmas, and use_flow_sigmas, matching the other flow schedulers.
The VP path derives alpha/sigma/lambda from the k-diffusion sigma so the
karras/beta/exponential transforms can use fractional timesteps; the
default schedule is numerically unchanged. use_flow_sigmas triggers flow
mode and add_noise tolerates fractional timesteps.
Wire the new keys into the ER-SDE presets so the sigma method selector
drives them, and cover ER-SDE in the scheduler stability test.
validate_sampler_name only matched the exact, case-sensitive name, so near-miss client names such as lowercase variants were rejected while an omitted name silently used the model scheduler via the Default sentinel. Fall back to find_sampler and return the canonical name so create_sampler applies the intended sampler; unknown names still return 404. Add an API test covering case-insensitive resolution and rejection of unknown names.
diffusers shipped first-party Ideogram 4 (transformer + pipeline) in
9b0818cf, so drop the in-tree port and keep only SD.Next integration glue.
Bump the diffusers pin to 9b0818cf and build diffusers' Ideogram4Pipeline
from a thin loader with per-transformer SDNQ. A small subclass keeps the
text encoder resident for the Qwen3-VL tap under balanced offload, and the
step callback denormalizes the preview latent from vae.bn before unpatchify.
Deletes the ported transformer, pipeline, scheduler, text encoder, and
latent-norm constants.
A converter error or load_state_dict mismatch now raises OverrideArchMismatch, which load_transformer catches to drop the override and load the base transformer. No per-arch markers to maintain.
Surface YoloRestorer.restore() as a standalone operation: a Detailer
postprocessing script in the Process tab and a thin /sdapi/v1/detail
endpoint, neither requiring a base generation pass.
- modules/postprocess/yolo.py: YoloRestorer.make_processing() builds the
synthetic Img2Img processing object both entry points feed to restore(),
resolving the seed so the inpaint passes are reproducible
- modules/api/process.py: post_detail handler exposes the full detailer
parameter set and returns the detailed image plus optional annotations
as base64
- scripts/postprocessing_detailer.py: reuses shared.yolo.ui('extras') and
runs through make_processing()
- modules/postprocessing.py: run_extras takes a per-script script_args
dict, also letting the extras API drive other scripts such as Remove
background; omitting it leaves existing callers unchanged
- modules/api/models.py: ReqDetail / ResDetail
- modules/processing_info.py: guard create_infotext's Image/Hires CFG
reporting against an unset (None) cfg_image, matching the is-not-None
checks the other cfg_image readers use; the detailer inpaint pass runs
with it unset
- test/test-detailer-api.py: covers both paths; effect tests measure the
diff inside the detected region with extreme isolated parameter values,
and the suite disables model quantization for the run and restores the
original settings afterward
CalamitousFelicitousness
Dity0
Vladimir Mandic