LTX refine pipe uses output_type='pil' so result.frames[0] returns a list, not a 5-D tensor. Convert via images_to_tensor before the helper sees it, mirroring what save_video already does for the same input.
Set p.video_interpolate at run() entry so process_images_inner picks up
the helper. Save calls keep their existing kwargs; the sentinel guard
skips re-interpolation when the helper already ran.
- animatediff, text2video, image2video, stablevideodiffusion route
mp4_interpolate into p.video_interpolate
- modules/control/run.py routes the request video_interpolate arg
- xyz_grid intentionally untouched: its end-of-axis save_video stitches
the cell slideshow, not per-cell videos; a future video_interpolate
axis would set p.video_interpolate per cell via axis_options
LTX, video_run, and framepack_worker bypass process_images_inner, so
they call apply_video_interpolation explicitly before save_video. Save
receives already-inflated frames; the sentinel guard skips its own pass.
- LTX and video_run scale mp4_fps by interpolation_factor(p) so duration
is preserved instead of stretched (LTX is conditioned on source fps)
- FramePack pre-divides at gen time per get_latent_paddings, so save fps
stays at mp4_fps; worker passes p=None so save call uses
mp4_interpolate=0 to skip directly
- replaces the inline (mp4_interpolate+1) fps math at LTX with the
helper-driven equivalent
Promote RIFE interpolation from a save-time kwarg to a real stage of the
processing pipeline so per-frame work (detailer, color correction,
postprocess scripts) operates on source-rate frames and the inflated
stream becomes the saved output.
- new modules/processing_video.py with apply_video_interpolation,
interpolation_factor, expand_infotexts; PIL/tensor/numpy dispatch
- video_interpolate, video_interpolate_scale, video_interpolated fields
on StableDiffusionProcessingVideo
- process_images_inner runs the helper after the batch loop and inflates
infotexts in lockstep
- save_video in modules/video.py and modules/video_models/video_save.py
short-circuit re-interpolation when p.video_interpolated is set; the
user-facing kwarg still flows into metadata
Color grading mutated pp.image but never wrote to pp.info, so the
chosen brightness/contrast/gamma/LUT/etc. were invisible in the
saved file's postprocessing/extras chunks and in the live Output
panel — unlike every other postprocessor (Upscale, Rembg, NudeNet,
PixelArt) which all stamp their settings.
Stamp every non-default GradingParams field as 'Grading <name>' so
the trail is consistent with the rest of the Process tab. LUT file
is recorded by basename rather than full path to keep the metadata
string compact.
read_info_from_image had three issues that together corrupted
metadata in the Process tab and img2img batch:
- items aliased image.info, so items.pop('parameters', ...) mutated
the source PIL image's info dict in place
- after popping, parameters was never restored to items, so callers
that re-stamp items onto a downstream image (postprocessing.py,
img2img.py) silently dropped the original generation params for
any input without EXIF UserComment
- synthetic width/height/mode keys were stuffed into items and ended
up persisted as PNG tEXt chunks even though no caller reads them
Result for the Process tab: upscaled outputs lacked the original
parameters chunk and instead had useless width/height/mode chunks
holding the source's pre-upscale dimensions. With this fix items is
a copy, parameters is restored after parsing, and width/height/mode
no longer leak into saved files.
Image.fromarray returns a fresh PIL image with empty .info, so the
source's parameters/UserComment metadata was silently dropped on the
grading roundtrip. In the Process tab this caused upscale + color
grading to save PNGs without the original generation parameters,
while upscale alone (via PIL resize/filter ops that preserve .info)
still worked.
torchvision 0.26 removed the torchvision.io.video submodule entirely
(only image I/O remains). The monkey-patch in install_requirements
(`torchvision.io.video.av = av`) was attempting to swap PyAV into a
torchvision-internal that no longer exists, raising AttributeError on
every FramePack run. The patch is fully vestigial against modern
torchvision; PyAV is still installed by `install('av')`, and the
sdnext save path uses PyAV directly via av.open().
LTX conditions the model on mp4_fps as the source frame rate, then
RIFE inflates frame count at save. Without compensation, save fps
stays equal to source fps and the video becomes (mp4_interpolate+1)x
slow-motion (e.g. 121 frames at 24 fps with mp4_interpolate=1 saved
as 242 frames at 24 fps = 10s instead of the intended 5s). Scale the
saved fps by the interpolation factor at the LTX call site so output
duration matches user intent.
FramePack already pre-divides mp4_fps at generation time (compute-
saving semantic baked into UI math), so it does not need this fix.
The pipelines now produce duration-correct output via two different
mechanisms; standardization is tracked for the RIFE-in-processing
follow-up.
video_save passes [-1,1]-range pixels to rife.interpolate_nchw, but
RIFE v4.25's IFNet explicitly clamps inputs to [0,1] (Head and IFNet
forward pass), turning every negative pixel value into zero. v3.9
silently extrapolated and produced soft artifacts; v4.25 produces
washout. Convert to [0,1] before the RIFE call and back to [-1,1] for
downstream save.
- Vendor IFNet_HDv3 v4.25 (5 IFBlocks, Head encoder, feat channel) and
v4 warplayer with explicit (tenFlow_div, backwarp_tenGrid) signature
- Rewrite RifeModel.inference for the new forward signature with
per-(H,W,device,dtype) caching of tenFlow_div and backwarp_tenGrid
- Force fp32 inference: bf16 produced visible checkerboard at the new
IFNet's depth (was hidden by v3.9's shallower architecture)
- Crop padded frames in interpolate_nchw before output (was missing,
produced gray bar on non-128-aligned inputs)
- Drop training scaffolding (AdamW, EPE/SOBEL, update method)
- Log obsolete legacy v3.9 weights file on first v4.25 load instead of
silently deleting user data
- Default download URL is HolyWu vs-rife mirror (MIT, byte-identical
upstream weights); swap to project-hosted URL before merge
NVIDIA/Cosmos native exports prefix every key with net., including the
bundled llm_adapter half. Add it to KNOWN_PREFIXES so partition_adapter
runs on stripped keys.
Distilled refine ran without the scheduler swap and identity guidance
kwargs prescribed by huggingface/diffusers#13217; only Dev got that
setup via the supports_canonical_stage2 branch. Distilled is already
trained at identity but still needs the recipe applied to avoid the
four-way composition double-dipping on top of the distilled sigma
schedule (oversaturation/striping).
Unify the branches under family == '2.x'; gate only the LoRA load and
unload on supports_canonical_stage2.
Both 2.3-1.1 I2V Distilled entries (full and SDNQ-4Bit) advertised
LTX2Pipeline. ltx_capabilities derives is_i2v partly from the cls_name
check, so I2V code paths (input media UI, supports_input_media, latent
prep) were not engaged for these models.
Conditions were built once at the user's chosen resolution, so condition
variants either ran same-res for both stages (blocked by the previous
gate) or fed base-res conditions to a final-res refine on the manual
upsample+refine path. Build conditions twice when staging is active:
once at base dims for Stage 1, once at final dims for Stage 2. Drop the
condition-class exclusion from supports_two_stage_refine.
auto_refine_upsample gated on supports_canonical_stage2 (Dev-only LoRA
path), so Distilled refine ran same-res for both stages and oversaturated.
Split the gate: supports_two_stage_refine covers any 2.x non-condition
variant and drives the staging plan plus UI Refine default;
supports_canonical_stage2 stays Dev-only for the LoRA branch.
Dispatch anima loras through a dedicated native loader covering kohya,
bfl/ai-toolkit, and hybrid (bfl with alpha plus qwen3 text encoder)
formats. Cosmos 2.0 path rename is mirrored from diffusers in flat
(underscore) form so rewritten paths match network_layer_mapping keys
without further conversion.
Split model_type from cosmos to anima so a future base-cosmos2 lora
path stays separable. Update flow_models, taesd supported list, and the
taesd wanvideo bucket so samplers and preview decoding keep working
after the split.
Extend assign_network_names_to_compvis_modules to walk pipe.llm_adapter
under the lora_llm_adapter_ prefix, and add llm_adapter to
default_components so activate and deactivate include it for anima
models while staying inert elsewhere via the existing getattr guards.
Anima community files bundle the llm_adapter inline alongside the
transformer, so CosmosTransformer3DModel.from_single_file mangles the
118 adapter keys via Cosmos 2.0 substring renames. New helper reads the
safetensors, strips the BFL-style prefix variants, partitions adapter
keys out, runs the rest through diffusers' converter, and loads each
half into its target. SDNQ and layerwise quantization honored; TRT and
GGUF skipped.
CalamitousFelicitousness
Vladimir Mandic
awsr