Register the subclass with generic.set_pipeline and build it with
from_pretrained, passing the SDNQ transformers and shared text encoder
while the vae, scheduler, and tokenizer load from the repo.
The dynamic-prompt brace processor in apply_styles_to_prompts strips the
{} and [] out of a JSON caption, leaving non-JSON that trips the model's
weight-baked safety placeholder. Let a model opt out of style and wildcard
processing via keep_prompts and set it for Ideogram4, then normalize the
prompt in encode_prompt: valid JSON to the compact training form, plain
text wrapped into a minimal caption so basic prompts still generate.
Each denoise step runs both transformers, so balanced offload ping-pongs
them across PCIe every step. When both fit gpu_memory * max watermark, set
offload_never so the per-step pre-sweep skips them and they stay resident;
otherwise the normal offload path is kept, so smaller GPUs and bf16 fall
back to offloading unchanged.
diffusers shipped first-party Ideogram 4 (transformer + pipeline) in
9b0818cf, so drop the in-tree port and keep only SD.Next integration glue.
Bump the diffusers pin to 9b0818cf and build diffusers' Ideogram4Pipeline
from a thin loader with per-transformer SDNQ. A small subclass keeps the
text encoder resident for the Qwen3-VL tap under balanced offload, and the
step callback denormalizes the preview latent from vae.bn before unpatchify.
Deletes the ported transformer, pipeline, scheduler, text encoder, and
latent-norm constants.
- move the text encoder on-device for the tapped forward (bypasses the offload hook)
- wrap the denoise loop in the diffusers progress bar
- live preview through the shared TAE FLUX.2 decoder (Flux.2 VAE)
Diffusers-native port of the 9.3B flow-matching DiT: dual-transformer asymmetric CFG, a 13-layer Qwen3-VL tap encoder deduped with VQA and prompt-enhance, the Flux.2 VAE, and a logit-normal schedule. Loads a published bf16 repo with SDNQ at load.
A converter error or load_state_dict mismatch now raises OverrideArchMismatch, which load_transformer catches to drop the override and load the base transformer. No per-arch markers to maintain.
Surface YoloRestorer.restore() as a standalone operation: a Detailer
postprocessing script in the Process tab and a thin /sdapi/v1/detail
endpoint, neither requiring a base generation pass.
- modules/postprocess/yolo.py: YoloRestorer.make_processing() builds the
synthetic Img2Img processing object both entry points feed to restore(),
resolving the seed so the inpaint passes are reproducible
- modules/api/process.py: post_detail handler exposes the full detailer
parameter set and returns the detailed image plus optional annotations
as base64
- scripts/postprocessing_detailer.py: reuses shared.yolo.ui('extras') and
runs through make_processing()
- modules/postprocessing.py: run_extras takes a per-script script_args
dict, also letting the extras API drive other scripts such as Remove
background; omitting it leaves existing callers unchanged
- modules/api/models.py: ReqDetail / ResDetail
- modules/processing_info.py: guard create_infotext's Image/Hires CFG
reporting against an unset (None) cfg_image, matching the is-not-None
checks the other cfg_image readers use; the detailer inpaint pass runs
with it unset
- test/test-detailer-api.py: covers both paths; effect tests measure the
diff inside the detected region with extreme isolated parameter values,
and the suite disables model quantization for the run and restores the
original settings afterward
create_paths() pre-created every per-type output folder via fix_path(), which resolves relative values against data_path (the repo root). With a base images/grids folder configured, those bare dirs are never written to (generation uses the base-joined path), so they accumulate as empty stray folders at the repo root on every launch. Guard the bare creations behind an unset base; the resolved base+specific block already creates the real targets.
taesd_layers < 3 drops spatial upsample blocks in the TAESD/TAEHV
decoders, shrinking preview output 2x/4x. Both UIs size the live preview
from the image's intrinsic pixel dimensions (modern via object-fit:
scale-down, standard via max(naturalWidth, 512px)), so lower layer counts
rendered the preview physically small.
Rescale the decoded preview spatially by 2^(3-layers) in
sd_vae_taesd.decode. Gated to TAESD and TAEHV, the only decoders that
honor taesd_layers; TAEM1 and Hybrid VAEs decode at full size and are left
untouched. Rank-agnostic so it covers both image (CHW) and video (TCHW)
previews, including single-frame video models used for txt2img.
The custom-transformer apply (DiT branch) and the revert-to-default path now call reload_model_weights(force=True) instead of load_diffuser, so both go through the same managed reload: unload-before-rebuild for a lower VRAM peak and consistent job/checkpoint bookkeeping.
Gate the cross-arch unet reset on an actual checkpoint change so a same-checkpoint reload skips the pipeline-class comparison, which could false-positive and clear the override when the model was left in an img2img/inpaint variant after an interrupted generation.
load_transformer threads **kwargs into its from_pretrained and from_single_file branches. The native dispatch branch bypasses both loaders and builds via cls.from_config, so those kwargs were dropped there. Thread them through load, build_component, and build_component_quantized into from_config so callers passing extra args alongside native_spec are honored instead of silently dropped. Siblings do not receive them.
CalamitousFelicitousness
Vladimir Mandic
Ryan Meador
Oleksandr Liutyi