When a video script (animatediff, text2video, image2video, stablevideodiffusion)
runs via Control tab, both the script's save and control_run's end-of-run save
fired. The latter crashed silently because the local `video` cv2 capture name at
control_run:584 shadowed the modules.video import, so the duplicate was hidden
and the gallery video link never propagated.
- alias import as video_module to bypass the shadow
- p.video_saved marker set by each script
- control_run skips its end-of-run save when the marker is set
LTX refine pipe uses output_type='pil' so result.frames[0] returns a list, not a 5-D tensor. Convert via images_to_tensor before the helper sees it, mirroring what save_video already does for the same input.
Set p.video_interpolate at run() entry so process_images_inner picks up
the helper. Save calls keep their existing kwargs; the sentinel guard
skips re-interpolation when the helper already ran.
- animatediff, text2video, image2video, stablevideodiffusion route
mp4_interpolate into p.video_interpolate
- modules/control/run.py routes the request video_interpolate arg
- xyz_grid intentionally untouched: its end-of-axis save_video stitches
the cell slideshow, not per-cell videos; a future video_interpolate
axis would set p.video_interpolate per cell via axis_options
LTX, video_run, and framepack_worker bypass process_images_inner, so
they call apply_video_interpolation explicitly before save_video. Save
receives already-inflated frames; the sentinel guard skips its own pass.
- LTX and video_run scale mp4_fps by interpolation_factor(p) so duration
is preserved instead of stretched (LTX is conditioned on source fps)
- FramePack pre-divides at gen time per get_latent_paddings, so save fps
stays at mp4_fps; worker passes p=None so save call uses
mp4_interpolate=0 to skip directly
- replaces the inline (mp4_interpolate+1) fps math at LTX with the
helper-driven equivalent
Promote RIFE interpolation from a save-time kwarg to a real stage of the
processing pipeline so per-frame work (detailer, color correction,
postprocess scripts) operates on source-rate frames and the inflated
stream becomes the saved output.
- new modules/processing_video.py with apply_video_interpolation,
interpolation_factor, expand_infotexts; PIL/tensor/numpy dispatch
- video_interpolate, video_interpolate_scale, video_interpolated fields
on StableDiffusionProcessingVideo
- process_images_inner runs the helper after the batch loop and inflates
infotexts in lockstep
- save_video in modules/video.py and modules/video_models/video_save.py
short-circuit re-interpolation when p.video_interpolated is set; the
user-facing kwarg still flows into metadata
read_info_from_image had three issues that together corrupted
metadata in the Process tab and img2img batch:
- items aliased image.info, so items.pop('parameters', ...) mutated
the source PIL image's info dict in place
- after popping, parameters was never restored to items, so callers
that re-stamp items onto a downstream image (postprocessing.py,
img2img.py) silently dropped the original generation params for
any input without EXIF UserComment
- synthetic width/height/mode keys were stuffed into items and ended
up persisted as PNG tEXt chunks even though no caller reads them
Result for the Process tab: upscaled outputs lacked the original
parameters chunk and instead had useless width/height/mode chunks
holding the source's pre-upscale dimensions. With this fix items is
a copy, parameters is restored after parsing, and width/height/mode
no longer leak into saved files.
Image.fromarray returns a fresh PIL image with empty .info, so the
source's parameters/UserComment metadata was silently dropped on the
grading roundtrip. In the Process tab this caused upscale + color
grading to save PNGs without the original generation parameters,
while upscale alone (via PIL resize/filter ops that preserve .info)
still worked.
torchvision 0.26 removed the torchvision.io.video submodule entirely
(only image I/O remains). The monkey-patch in install_requirements
(`torchvision.io.video.av = av`) was attempting to swap PyAV into a
torchvision-internal that no longer exists, raising AttributeError on
every FramePack run. The patch is fully vestigial against modern
torchvision; PyAV is still installed by `install('av')`, and the
sdnext save path uses PyAV directly via av.open().
LTX conditions the model on mp4_fps as the source frame rate, then
RIFE inflates frame count at save. Without compensation, save fps
stays equal to source fps and the video becomes (mp4_interpolate+1)x
slow-motion (e.g. 121 frames at 24 fps with mp4_interpolate=1 saved
as 242 frames at 24 fps = 10s instead of the intended 5s). Scale the
saved fps by the interpolation factor at the LTX call site so output
duration matches user intent.
FramePack already pre-divides mp4_fps at generation time (compute-
saving semantic baked into UI math), so it does not need this fix.
The pipelines now produce duration-correct output via two different
mechanisms; standardization is tracked for the RIFE-in-processing
follow-up.
video_save passes [-1,1]-range pixels to rife.interpolate_nchw, but
RIFE v4.25's IFNet explicitly clamps inputs to [0,1] (Head and IFNet
forward pass), turning every negative pixel value into zero. v3.9
silently extrapolated and produced soft artifacts; v4.25 produces
washout. Convert to [0,1] before the RIFE call and back to [-1,1] for
downstream save.
- Vendor IFNet_HDv3 v4.25 (5 IFBlocks, Head encoder, feat channel) and
v4 warplayer with explicit (tenFlow_div, backwarp_tenGrid) signature
- Rewrite RifeModel.inference for the new forward signature with
per-(H,W,device,dtype) caching of tenFlow_div and backwarp_tenGrid
- Force fp32 inference: bf16 produced visible checkerboard at the new
IFNet's depth (was hidden by v3.9's shallower architecture)
- Crop padded frames in interpolate_nchw before output (was missing,
produced gray bar on non-128-aligned inputs)
- Drop training scaffolding (AdamW, EPE/SOBEL, update method)
- Log obsolete legacy v3.9 weights file on first v4.25 load instead of
silently deleting user data
- Default download URL is HolyWu vs-rife mirror (MIT, byte-identical
upstream weights); swap to project-hosted URL before merge
- new autocomplete_translations setting (default off); checkbox in script accordion + locale hint
- get_cached gates companion load on the opt; toggle-flip auto-syncs missing files in a background thread
- list_all_sync, local_names and config_json filter out *.translations.json so companion files never surface as selectable dicts
- one-time per-name warning when opt is on but companion is missing locally
Distilled refine ran without the scheduler swap and identity guidance
kwargs prescribed by huggingface/diffusers#13217; only Dev got that
setup via the supports_canonical_stage2 branch. Distilled is already
trained at identity but still needs the recipe applied to avoid the
four-way composition double-dipping on top of the distilled sigma
schedule (oversaturation/striping).
Unify the branches under family == '2.x'; gate only the LoRA load and
unload on supports_canonical_stage2.
Both 2.3-1.1 I2V Distilled entries (full and SDNQ-4Bit) advertised
LTX2Pipeline. ltx_capabilities derives is_i2v partly from the cls_name
check, so I2V code paths (input media UI, supports_input_media, latent
prep) were not engaged for these models.
Conditions were built once at the user's chosen resolution, so condition
variants either ran same-res for both stages (blocked by the previous
gate) or fed base-res conditions to a final-res refine on the manual
upsample+refine path. Build conditions twice when staging is active:
once at base dims for Stage 1, once at final dims for Stage 2. Drop the
condition-class exclusion from supports_two_stage_refine.
auto_refine_upsample gated on supports_canonical_stage2 (Dev-only LoRA
path), so Distilled refine ran same-res for both stages and oversaturated.
Split the gate: supports_two_stage_refine covers any 2.x non-condition
variant and drives the staging plan plus UI Refine default;
supports_canonical_stage2 stays Dev-only for the LoRA branch.
Dispatch anima loras through a dedicated native loader covering kohya,
bfl/ai-toolkit, and hybrid (bfl with alpha plus qwen3 text encoder)
formats. Cosmos 2.0 path rename is mirrored from diffusers in flat
(underscore) form so rewritten paths match network_layer_mapping keys
without further conversion.
Split model_type from cosmos to anima so a future base-cosmos2 lora
path stays separable. Update flow_models, taesd supported list, and the
taesd wanvideo bucket so samplers and preview decoding keep working
after the split.
Extend assign_network_names_to_compvis_modules to walk pipe.llm_adapter
under the lora_llm_adapter_ prefix, and add llm_adapter to
default_components so activate and deactivate include it for anima
models while staying inert elsewhere via the existing getattr guards.
Anima community files bundle the llm_adapter inline alongside the
transformer, so CosmosTransformer3DModel.from_single_file mangles the
118 adapter keys via Cosmos 2.0 substring renames. New helper reads the
safetensors, strips the BFL-style prefix variants, partitions adapter
keys out, runs the rest through diffusers' converter, and loads each
half into its target. SDNQ and layerwise quantization honored; TRT and
GGUF skipped.
Lightricks' production recipe for Dev is Stage 1 + 2x upsample + Stage 2
refine. Until now the UI reset Refine to False on every model change, so
users picking Dev got a single-pass generation that does not match the
recommended flow. Default Refine to on for variants that support the
canonical Stage 2 recipe; multi-condition variants stay off.
Modules already inventoried at load time; repeating the six-line dump at
each upsample or refine boundary is redundant. silent=True suppresses
the per-module DEBUG lines; op=init and Model class= INFO stay intact.
The upsample pipes receive shared.sd_model.vae as a constructor formality;
the forward pass is pure latent to latent. Main pipe already owns that
VAE's accelerate hook lifecycle, so walking it again from the upsample
pipe raises "Cannot copy out of meta tensor" when the main pipe has
offloaded params to meta. Skip it in the walk.
Vladimir Mandic
awsr
CalamitousFelicitousness