Public/automatic
1
0
Fork 0
mirror of https://github.com/vladmandic/automatic synced 2026-09-18 16:54:33 +02:00
Code Issues Packages Projects Releases Wiki Activity

refactor(ideogram4): adopt diffusers-native pipeline

Browse Source 
diffusers shipped first-party Ideogram 4 (transformer + pipeline) in
9b0818cf, so drop the in-tree port and keep only SD.Next integration glue.

Bump the diffusers pin to 9b0818cf and build diffusers' Ideogram4Pipeline
from a thin loader with per-transformer SDNQ. A small subclass keeps the
text encoder resident for the Qwen3-VL tap under balanced offload, and the
step callback denormalizes the preview latent from vae.bn before unpatchify.
Deletes the ported transformer, pipeline, scheduler, text encoder, and
latent-norm constants.
This commit is contained in:
CalamitousFelicitousness
2026-06-05 00:13:16 +01:00
parent 4866d2c8f3
commit bcebb6b81b
12 changed files with 69 additions and 1250 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/processing_callbacks.py
+20
View File
@@ -173,6 +173,26 @@ def diffusers_callback(pipe, step: int = 0, timestep: int = 0, kwargs: dict | No
current_noise_pred = current_noise_pred.view(b, h_patches, w_patches, channels, 2, 2)
current_noise_pred = current_noise_pred.permute(0, 3, 1, 4, 2, 5).reshape(b, channels, h_patches * 2, w_patches * 2)
shared.state.current_noise_pred = current_noise_pred
elif 'Ideogram4' in pipe.__class__.__name__: # packed normalized [B, seq, 128] -> Flux.2 latent space for TAE FLUX.2
latents = kwargs['latents']
if latents.ndim == 3:
b, seq_len, packed_ch = latents.shape
vae_scale = getattr(pipe, 'vae_scale_factor', 8)
patch = getattr(pipe, 'patch_size', 2)
grid_h = getattr(p, 'height', 1024) // (vae_scale * patch)
grid_w = getattr(p, 'width', 1024) // (vae_scale * patch)
if grid_h * grid_w != seq_len: # fallback to square assumption
grid_h = grid_w = int(seq_len ** 0.5)
bn = pipe.vae.bn
mean = bn.running_mean.view(1, 1, -1).to(device=latents.device, dtype=torch.float32)
std = torch.sqrt(bn.running_var + pipe.vae.config.batch_norm_eps).view(1, 1, -1).to(device=latents.device, dtype=torch.float32)
z = latents.float() * std + mean
ae_ch = packed_ch // (patch * patch)
z = z.view(b, grid_h, grid_w, patch, patch, ae_ch).permute(0, 5, 1, 3, 2, 4).reshape(b, ae_ch, grid_h * patch, grid_w * patch)
shared.state.current_latent = z
else:
shared.state.current_latent = latents
shared.state.current_noise_pred = current_noise_pred
else:
shared.state.current_latent = kwargs['latents']
shared.state.current_noise_pred = current_noise_pred