feat(ltx): canonical LTX-2.x Stage 2 recipe (LoRA + guidance + connectors)

Implement the Lightricks two-stage recipe (diffusers PR #13217) for the
LTX-2.x Dev family: Stage 1 at half-res with full four-way guidance,
2x latent upsample, Stage 2 with distilled LoRA + scheduler swap + identity
guidance on STAGE_2_DISTILLED_SIGMA_VALUES.

Extends to both LTX-2.0 and LTX-2.3 Dev via per-family distilled-LoRA
repos carried on the caps; Distilled variants take the same flow minus
the LoRA swap. Auto-couples Refine with a fixed 2x upsample on any Dev
variant with a known LoRA when the user enables Refine without Upsample.

- caps: is_ltx_2_3, use_cross_timestep, default_dynamic_shift,
  stage2_dev_lora_repo, supports_canonical_stage2, modality_default_scale,
  guidance_rescale_default; LTX-2.x defaults realigned to canonical
  cfg=3.0 / steps=30; per-variant STG block and four-way guidance wired
  for non-distilled 2.x
- process: canonical Stage 1/Stage 2 helpers, scheduler + opts snapshot
  under try/finally, per-family upsampler repo, audio latents threaded
  from Stage 1 into Stage 2, use_cross_timestep gated per caps
- overrides: skip the redundant unsharded LTX-2.3 connectors blob and
  share LTX2TextConnectors weights across 2.3 variants when te_shared_t5
- load: Gemma3 shared-TE path for LTX-2.3; gate use_dynamic_shifting=False
  override to 0.9.x only so LTX-2.x stays on its canonical token-count
  dynamic shift
This commit is contained in:
CalamitousFelicitousness
2026-04-19 03:37:25 +01:00
parent 05abd99285
commit 5cf46d2f81
4 changed files with 489 additions and 267 deletions
+12 -1
View File
@@ -88,6 +88,13 @@ def load_model(selected: models_def.Model):
selected.te = 'ai-forever/Kandinsky-5.0-T2V-Lite-sft-5s-Diffusers'
selected.te_folder = 'text_encoder'
selected.te_revision = None
if selected.te_cls.__name__ == 'Gemma3ForConditionalGeneration' and shared.opts.te_shared_t5:
if 'SDNQ' in selected.name:
selected.te = 'OzzyGT/LTX-2.3-sdnq-dynamic-int4'
else:
selected.te = 'OzzyGT/LTX-2.3'
selected.te_folder = 'text_encoder'
selected.te_revision = None
log.debug(f'Video load: module=te repo="{selected.te or selected.repo}" folder="{selected.te_folder}" cls={selected.te_cls.__name__} quant={model_quant.get_quant_type(quant_args)} loader={_loader("transformers")}')
kwargs["text_encoder"] = selected.te_cls.from_pretrained(
@@ -158,7 +165,11 @@ def load_model(selected: models_def.Model):
return msg
t1 = time.time()
if shared.sd_model.__class__.__name__.startswith("LTX"):
cls_name = shared.sd_model.__class__.__name__
# LTX 0.9.x is plain linear; pin use_dynamic_shifting=False against upstream config drift.
# LTX-2.x canonical is token-count-based dynamic shift (base_shift=0.95, max_shift=2.05);
# disabling it there would take the model off-distribution.
if cls_name.startswith("LTX") and not cls_name.startswith("LTX2"):
shared.sd_model.scheduler.config.use_dynamic_shifting = False
shared.sd_model.default_scheduler = copy.deepcopy(shared.sd_model.scheduler) if hasattr(shared.sd_model, "scheduler") else None
shared.sd_model.sd_checkpoint_info = sd_checkpoint.CheckpointInfo(selected.repo)