The I2V path forwards a last-frame image to the pipeline when one is supplied and the loaded pipeline accepts it, turning the run into first-last-frame interpolation. supports_last_frame() gates on the pipeline taking a last_image argument and not running expand_timesteps, which conditions on the first frame only, so a model that cannot use a last frame logs a warning instead of silently ignoring it.
Wan 2.2 A14B ships a per-model boundary_ratio (0.9 I2V, 0.875 T2V) that selects the high- or low-noise expert per step. The video and base-model image loaders both load the shipped value; the slider override is applied at generation time in set_pipeline_args, the one point both paths pass through before invoking the pipeline.
The denoising loop reads config.boundary_ratio each call, so tuning takes effect with no reload for video and base-model images alike. The slider defaults to -1, meaning use the model's value; 0 to 1 set the boundary explicitly. Single-expert stages stay load-time because they drop a transformer to free VRAM.
- monkey-patch LTX2ConnectorTransformer1d.forward to restore pre-#13564
padding logic when the upstream torch.flip pattern is detected; fixes
word-order scrambling in audio dialogue tracks
- reorganize LTX model entries into version-group separators (2.3 v1.1,
2.3 v1.0, 2.0, 0.9.x) with base/distilled subgroups; separators are
selectable no-ops handled in run_ltx
relabel toggle to 'LTX save audio' (default true) since audio always
generates on 2.x audio-capable models; the toggle gates mux only. hint
added to locale_en.json.
split add_audio_stream from write_audio. avformat_write_header runs on
first container.mux() and freezes the stream set, so audio added after
video packets has time_base=0/0 and raises 'Cannot rebase to zero time.'
atomic_save_video registers the audio stream before the encode loop.
LTX, video_run, and framepack_worker bypass process_images_inner, so
they call apply_video_interpolation explicitly before save_video. Save
receives already-inflated frames; the sentinel guard skips its own pass.
- LTX and video_run scale mp4_fps by interpolation_factor(p) so duration
is preserved instead of stretched (LTX is conditioned on source fps)
- FramePack pre-divides at gen time per get_latent_paddings, so save fps
stays at mp4_fps; worker passes p=None so save call uses
mp4_interpolate=0 to skip directly
- replaces the inline (mp4_interpolate+1) fps math at LTX with the
helper-driven equivalent
Promote RIFE interpolation from a save-time kwarg to a real stage of the
processing pipeline so per-frame work (detailer, color correction,
postprocess scripts) operates on source-rate frames and the inflated
stream becomes the saved output.
- new modules/processing_video.py with apply_video_interpolation,
interpolation_factor, expand_infotexts; PIL/tensor/numpy dispatch
- video_interpolate, video_interpolate_scale, video_interpolated fields
on StableDiffusionProcessingVideo
- process_images_inner runs the helper after the batch loop and inflates
infotexts in lockstep
- save_video in modules/video.py and modules/video_models/video_save.py
short-circuit re-interpolation when p.video_interpolated is set; the
user-facing kwarg still flows into metadata
video_save passes [-1,1]-range pixels to rife.interpolate_nchw, but
RIFE v4.25's IFNet explicitly clamps inputs to [0,1] (Head and IFNet
forward pass), turning every negative pixel value into zero. v3.9
silently extrapolated and produced soft artifacts; v4.25 produces
washout. Convert to [0,1] before the RIFE call and back to [-1,1] for
downstream save.
Both 2.3-1.1 I2V Distilled entries (full and SDNQ-4Bit) advertised
LTX2Pipeline. ltx_capabilities derives is_i2v partly from the cls_name
check, so I2V code paths (input media UI, supports_input_media, latent
prep) were not engaged for these models.
LTXCaps gains a `variant` field ('0.9', '2.0', '2.3') replacing
`is_ltx_2_3`; variant-specific branches check `caps.variant == '2.3'`
instead of grepping the model name.
ltx_util.load_upsample_2x mirrors load_upsample's contract so the 2.x
path owns a module-level cache and stops reloading ~2.3 GB every run.
The cached pipe is stamped with a synthetic
`CheckpointInfo('ltx-upsampler-2.x')` so it gets its own OffloadHook
slot and can go through apply_balanced_offload without invalidating
the main pipe's module map. The hardcoded `.to('cpu')` and post-pass
torch_gc are gone; the second apply_balanced_offload handles spill.
video_overrides comment on OzzyGT LTX-2.3 connectors states plainly
that mirrors pack weights twice by design; ignore_patterns is the
surgical workaround, not an hf_hub bug.
Drive-by: load_upsample log line used `__class__.__name__` (always
'type'); switched to `__name__`.
Each engine tab (Generic, FramePack, LTX) now owns its own size, seed,
frame count, and init imagery. The previously shared Size and Inputs
subtabs are removed; persistent encoding settings move to a single
Output subtab constructed first so engine tabs can consume its widgets.
FramePack drops its local fps/interpolate sliders and consumes the
shared Output widgets; LTX drops the redundant shared init_image
parameter and keeps only its own ltx_init_image, with run_ltx's
condition_image fallback removed in turn.
Implement the Lightricks two-stage recipe (diffusers PR #13217) for the
LTX-2.x Dev family: Stage 1 at half-res with full four-way guidance,
2x latent upsample, Stage 2 with distilled LoRA + scheduler swap + identity
guidance on STAGE_2_DISTILLED_SIGMA_VALUES.
Extends to both LTX-2.0 and LTX-2.3 Dev via per-family distilled-LoRA
repos carried on the caps; Distilled variants take the same flow minus
the LoRA swap. Auto-couples Refine with a fixed 2x upsample on any Dev
variant with a known LoRA when the user enables Refine without Upsample.
- caps: is_ltx_2_3, use_cross_timestep, default_dynamic_shift,
stage2_dev_lora_repo, supports_canonical_stage2, modality_default_scale,
guidance_rescale_default; LTX-2.x defaults realigned to canonical
cfg=3.0 / steps=30; per-variant STG block and four-way guidance wired
for non-distilled 2.x
- process: canonical Stage 1/Stage 2 helpers, scheduler + opts snapshot
under try/finally, per-family upsampler repo, audio latents threaded
from Stage 1 into Stage 2, use_cross_timestep gated per caps
- overrides: skip the redundant unsharded LTX-2.3 connectors blob and
share LTX2TextConnectors weights across 2.3 variants when te_shared_t5
- load: Gemma3 shared-TE path for LTX-2.3; gate use_dynamic_shifting=False
override to 0.9.x only so LTX-2.x stays on its canonical token-count
dynamic shift
Move cache tracking from ltx_util into video_load where shared.sd_model
lives, and invalidate the name-based hit when the cached class no longer
matches the current pipeline (e.g. after Unload Models triggers an
auto-reload of the default checkpoint).
- Drop the duplicate module-level loaded_model cache in ltx_util
- Add a pipe-class isinstance check around the cache hit in video_load
Rework the LTX Video tab so one UI handles every registered variant
(0.9.0 through 2.3, Dev/Distilled/SDNQ-4Bit, T2V/I2V/Condition). Per-
variant behavior is driven from a single capability lookup rather than
substring matching on model names scattered across the backend.
- modules/ltx/ltx_capabilities.py: new module computing family, is_i2v,
distilled, supports_input_media, supports_multi_condition,
supports_image_cond_noise_scale, supports_decode_timestep,
supports_stg, supports_audio, supports_frame_rate_kwarg, and the
default CFG / steps / sampler_shift for a given model name by reading
its registered repo_cls in models_def.
- modules/ltx/ltx_ui.py: capability-gated UI. Selecting a model rewires
accordion visibility, slider interactivity, and defaults via a single
model.change handler. New controls: dedicated image input slot inside
the LTX tab (replaces the disconnected shared init_image for I2V),
condition strength slider, CFG / sampler shift / dynamic shift
sliders that were previously unreachable. Input media accordion
restructured so the image slot is always-visible while the video /
gallery prefix tabs only appear on Condition pipelines.
- modules/ltx/ltx_process.py: route the base pass through
processing.process_images(p) so LTX inherits standard scheduler
wiring, extra_networks activation, VAE handling, and error plumbing
from StableDiffusionProcessingVideo. The multi-pass latent path
(upsample / refine) stays on direct pipeline calls for latent
re-entry. Refine noise control gets family-specific kwargs:
denoise_strength for 0.9.x LTXConditionPipeline, noise_scale for all
2.x pipelines; the prior strength= injection crashed on 2.x and only
affected conditioning intensity on 0.9.x. Add torch_gc between every
stage boundary (base to upsample to refine to vae decode) so the
CUDA allocator cache does not retain the prior pass's allocations
across stages. Remove the TypeError fallback that silently passed
raw latents to save_video when VAE decode returned None on OOM;
those errors now surface cleanly.
- modules/ltx/ltx_util.py: get_conditions grows a family parameter and
builds LTX2VideoCondition (frames, index, strength) for 2.x or
LTXVideoCondition (image, video, frame_index, strength) for 0.9.x.
get_bucket floors to max(32, vae_spatial_compression_ratio) since LTX
pipelines validate divisibility by 32 regardless of family.
- modules/video_models/video_overrides.py: extend the I2V generator
reset to cover LTX2ImageToVideoPipeline and both Condition classes.
Keep the strength= kwarg injection gated to 0.9.x
LTXConditionPipeline only; LTX2ConditionPipeline.__call__ does not
accept it (per-condition strength lives on the LTX2VideoCondition
dataclass instead).
The 2.3 I2V variants (Dev, Distilled, SDNQ-4Bit, Distilled SDNQ-4Bit)
were registered against LTX2Pipeline, whose __call__ does not accept an
image kwarg; init images were silently dropped. Route them through
LTX2ImageToVideoPipeline instead.
Also register the four 2.3 Condition variants under LTX2ConditionPipeline
so the multi-condition (image/video/gallery prefix) generation path is
reachable on LTX 2.3.
Replace "init image not set" / "last image not set" with actionable
messages that tell the user what to do. Matches the phrasing used by
the Caption tab for the same failure class.
Vladimir Mandic
CalamitousFelicitousness
QualiaRain
Ryan Meador
awsr