Surface YoloRestorer.restore() as a standalone operation: a Detailer
postprocessing script in the Process tab and a thin /sdapi/v1/detail
endpoint, neither requiring a base generation pass.
- modules/postprocess/yolo.py: YoloRestorer.make_processing() builds the
synthetic Img2Img processing object both entry points feed to restore(),
resolving the seed so the inpaint passes are reproducible
- modules/api/process.py: post_detail handler exposes the full detailer
parameter set and returns the detailed image plus optional annotations
as base64
- scripts/postprocessing_detailer.py: reuses shared.yolo.ui('extras') and
runs through make_processing()
- modules/postprocessing.py: run_extras takes a per-script script_args
dict, also letting the extras API drive other scripts such as Remove
background; omitting it leaves existing callers unchanged
- modules/api/models.py: ReqDetail / ResDetail
- modules/processing_info.py: guard create_infotext's Image/Hires CFG
reporting against an unset (None) cfg_image, matching the is-not-None
checks the other cfg_image readers use; the detailer inpaint pass runs
with it unset
- test/test-detailer-api.py: covers both paths; effect tests measure the
diff inside the detected region with extreme isolated parameter values,
and the suite disables model quantization for the run and restores the
original settings afterward
create_paths() pre-created every per-type output folder via fix_path(), which resolves relative values against data_path (the repo root). With a base images/grids folder configured, those bare dirs are never written to (generation uses the base-joined path), so they accumulate as empty stray folders at the repo root on every launch. Guard the bare creations behind an unset base; the resolved base+specific block already creates the real targets.
taesd_layers < 3 drops spatial upsample blocks in the TAESD/TAEHV
decoders, shrinking preview output 2x/4x. Both UIs size the live preview
from the image's intrinsic pixel dimensions (modern via object-fit:
scale-down, standard via max(naturalWidth, 512px)), so lower layer counts
rendered the preview physically small.
Rescale the decoded preview spatially by 2^(3-layers) in
sd_vae_taesd.decode. Gated to TAESD and TAEHV, the only decoders that
honor taesd_layers; TAEM1 and Hybrid VAEs decode at full size and are left
untouched. Rank-agnostic so it covers both image (CHW) and video (TCHW)
previews, including single-frame video models used for txt2img.
The custom-transformer apply (DiT branch) and the revert-to-default path now call reload_model_weights(force=True) instead of load_diffuser, so both go through the same managed reload: unload-before-rebuild for a lower VRAM peak and consistent job/checkpoint bookkeeping.
Gate the cross-arch unet reset on an actual checkpoint change so a same-checkpoint reload skips the pipeline-class comparison, which could false-positive and clear the override when the model was left in an img2img/inpaint variant after an interrupted generation.
load_transformer threads **kwargs into its from_pretrained and from_single_file branches. The native dispatch branch bypasses both loaders and builds via cls.from_config, so those kwargs were dropped there. Thread them through load, build_component, and build_component_quantized into from_config so callers passing extra args alongside native_spec are honored instead of silently dropped. Siblings do not receive them.
The transformer load path now mirrors diffusers' load_model_dict_into_meta: build the component inside init_empty_weights(include_buffers=False) so parameter slots stay meta while computed buffers keep their __init__ values, then iterate the converted state_dict tensor by tensor. Each tensor is dispatched through SDNQQuantizer.check_if_quantized_param and create_quantized_param so Linear/Conv/Embed weights are packed to uint4 in flight, while non-quantizable parameters go through accelerate.set_module_tensor_to_device.
Peak memory during transformer load drops from the full bf16 model size to one layer's worth of bf16 plus the running uint4 result. The prior load_state_dict + .to(dtype) + post-walk sdnq_quantize_model sequence remains as the fallback path for siblings and non-SDNQ quant types.
generic.load_transformer already computes quant_args and quant_type at the top of the function, but native_transformer.load was redoing the same work and emitting a duplicate 'Quantization: module=Model ...' log line milliseconds later. Pass them through instead; fall back to computing them when called directly (model_anima still does so).
safetensors.torch.load_file opens the file by path internally, so the progress.open wrapper around it never advanced from 0/N - it just left a stuck '0.0/18.2 GB' bar on screen for the duration of the load. Skip the wrapper on that path; .ckpt and stream_load paths still get the working bar.
When reload_model_weights resets shared.opts.sd_unet to Default on a cross-arch swap, the backend value changed but the dropdown component kept showing the previous selection because sd_unet was not in the change_checkpoint button's outputs.
Add the sd_unet component to the outputs and return its post-reload value via gr.update.
The native loader was emitting one line at dispatch, a single line after prefix detection, and the total time at the end. For an 18 GB transformer that meant ~60s of silence with no indication anything was happening.
Add DEBUG lines before each long phase: read_state_dict, optional converter, load_state_dict into the model, and dtype cast. Total-time log at the end is unchanged.
A custom UNET selected via the UNET dropdown carried over silently when the user swapped to a base model of a different arch, then crashed inside the per-arch loader because shared.opts.sd_unet was still pointing at the previous arch's file.
reload_model_weights now runs sd_detect.detect_pipeline on the new checkpoint before unloading the old model, compares the detected pipeline class against the loaded model's class, and resets shared.opts.sd_unet to Default when they differ.
Loading a Chroma transformer override via UNET dropdown crashed at
inference with a shape mismatch in unpack_uint4: from_single_file does
not integrate quantization_config the way from_pretrained does, so the
dequantizer state was set up but the .weight tensor was never packed.
CHROMA_SPEC routes through native_transformer with the diffusers
Chroma converter, which loads bare and quantizes explicitly via
sdnq_quantize_model.
apply_quant called sdnq_quantize_model(transformer, op="transformer")
without the per-call skip lists, dropping modules_to_not_convert and
modules_dtype_dict that the caller passed into generic.load_transformer.
The from_pretrained path receives them via quantization_config; the
native path now threads them directly into the explicit quantizer call.
Plumbed load -> build_component -> apply_quant as keyword args. No
behavior change for callers that did not set the skip lists.
K3 and K5 share only the marketing name: different diffusers pipeline
classes (Kandinsky3Pipeline vs Kandinsky5T2I/I2IPipeline), different
denoiser classes (Kandinsky3UNet vs Kandinsky5Transformer3DModel), no
shared scaffolding. Separate packages so each spec lives next to the
arch it serves.
K2.1 and K2.2 stay outside both packages since they go through
KandinskyCombinedPipeline.from_pretrained directly with no spec.
Specs for chrono, cogview, flux2_klein, glm, hunyuandit, hyimage, joy,
kandinsky, longcat, nucleus, ovis, pixart and prx lived module-level in
model_<arch>.py. Move each into pipelines/<arch>/__init__.py to match
the layout used by anima, bria, ernie, f_lite, lens, qwen, step1x and
vibe.
Each model_<arch>.py now imports its spec lazily inside the load
function, so the package only gets pulled in when that arch is actually
loaded (kandinsky 2.x never goes through native dispatch and stays
untouched).
Also drop the "without this spec..." crash paragraph from the new
package docstrings plus ernie/__init__.py and bria/__init__.py.
load_unet returned early when sd_unet was set to 'Default'/'None',
leaving any previously-loaded custom UNET in memory. The model stayed
on the override and the dropdown looked stuck.
Now detects the transition from a custom selection to Default and
triggers sd_models.load_diffuser() to reload via the canonical
from_pretrained path.
Klein shares Flux2Transformer2DModel with full Flux 2 but uses a
smaller config. from_single_file instantiates with the class default
(= Flux 2 full size); loading a Klein-shaped file crashed at
load_model_dict_into_meta with:
Cannot load because double_stream_modulation_img.linear.weight
expected shape (36864, 6144), but got (24576, 4096)
FLUX2_KLEIN_SPEC fetches Klein's transformer/config.json from the base
repo and uses convert_flux2_transformer_checkpoint_to_diffusers
explicitly. Probed against snofsSexNudesAndOtherFunStuff_v14Base:
233/233 keys overlap.
Qwen-Image: diffusers registers a no-op identity lambda for
QwenImageTransformer2DModel; from_single_file silently accepted any
key naming and loaded with mismatches. QWEN_SPEC sets converter=None
to short-circuit the no-op.
Custom transformer classes (Lens, Step1X-Edit, VIBE Sana, f-lite):
runtime-registered onto diffusers, never in SINGLE_FILE_LOADABLE_CLASSES.
Specs land in each arch's existing pipelines/<arch>/__init__.py.
NextStep skipped: loader returns None up front.
CogView3+, CogView4, GLM-Image, HunyuanDiT, Joy, Kandinsky3 UNet,
PixArt, PRX. These classes lack from_single_file entirely; the UNET
dropdown fell through to cls.from_pretrained(local_file), which
decodes the .safetensors header as UTF-8 and crashes "is not a valid
JSON file".
TransformerSpec(cls=...) module-level in each model_<arch>.py.
Kandinsky3 also needs subfolder='unet' since it loads a UNet not a
transformer.
ChronoEdit, HunyuanImage, Kandinsky5, LongCat, Ovis, NucleusMoE, Bria,
BriaFibo. Each inherits from_single_file via FromOriginalModelMixin
but has no SINGLE_FILE_LOADABLE_CLASSES entry; calling it crashed with
"FromOriginalModelMixin is currently only compatible with [...]".
TransformerSpec(cls=...) for each. Bria specs in
pipelines/bria/__init__.py; the rest are module-level in
model_<arch>.py since they have no package directory.
The 40-line hand-rolled loader was generic.load_transformer plus a
VACE/standard class switch and a hardcoded GGUF rejection. Switch moves
to load_wan; GGUF rejection removed (generic handles it via
GGUFQuantizationConfig).
No native_spec passed: WanTransformer3DModel has a working diffusers
converter, so from_single_file via generic stays correct.
ErnieImageTransformer2DModel has no from_single_file; selecting an
Ernie finetune in the UNET dropdown previously crashed with
"is not a valid JSON file" from from_pretrained.
Probe of jibMixErnie_v20.safetensors: 409/409 keys overlap with the
model state dict after stripping model.diffusion_model., zero missing
or unexpected. Spec is the minimum TransformerSpec(cls=...).
New native_spec=None kwarg. When set and the UNET dropdown points at a
.safetensors, dispatches to native_transformer.load (threading
allow_quant/dtype/modules_to_not_convert/modules_dtype_dict). Pipelines
without a spec stay on cls.from_single_file unchanged.
217-line bespoke loader collapses to a 40-line ANIMA_SPEC in
pipelines/anima/__init__.py (Cosmos converter + llm_adapter sibling +
Cosmos 1.0 forbidden marker).
Drop the class-keyed REGISTRY: Anima and raw Cosmos share
CosmosTransformer3DModel but need different specs. Specs pass via
explicit native_spec= kwarg; make_default_spec(cls) covers the
auto-converter case.
Frees the name for pipelines/native_transformer. Module covers the full
LyCORIS adapter family (LoRA/LoKR/LoHA/OFT/IA3/GLoRA/Norm/Full), not
just LoRA.
Ryan Meador
Vladimir Mandic
Oleksandr Liutyi
CalamitousFelicitousness