The shape branch read axis 1, which held the frame count back when animatediff
returned (batch, channels, frames, height, width). Diffusers now returns
(batch, frames, channels, height, width) and the modular video path puts
channels last, so axis 1 reads channels or height and the line printed a
plausible wrong number: a 124 frame 1024x576 generation logged frames=576.
len() is the frame count under both layouts and is what the sibling call site
in process_decode already uses.
The ref2va checkpoint partition conditions on reference images instead
of keyframes, so it gets its own registry row and reference card, and
the video core marshals PIL images into task_args as
MiniMaxH3ImageReference. Images are converted to RGB first, since the
reference encoder reads the array raw. The keyframe path is unchanged.
Validation runs before the model load in one funnel shared by the tab
and the API, so a rejected request costs nothing: references on a
non-reference model, a reference model with nothing to condition on,
more than nine images, non-images, and aspect outside 1:4 to 4:1 all
return 400. The image path rejects a reference pipe without references
instead of reaching a transformer that was never loaded.
Add POST /sdapi/v1/video plus GET /sdapi/v1/video/models and
GET /sdapi/v1/video/file. The generation body is extracted from the
gradio handler into a keyword-only core, video_run.run, which returns a
structured result and raises typed errors; the positional generate
signature is unchanged and now adapts to the core. Omitting engine and
model drives the currently loaded checkpoint when it is video-capable,
which covers models loaded from local folders without a registry entry.
- registry helpers in models_def (find, engines, pipeline_classes,
workflow_for_class); validate_pipeline reuses the shared class set
- modular pipes stamp their workflow so out-of-registry loads dispatch
onto the modular branch
- disk switches (mp4_*) and wire switches (send_*) are independent;
artifacts above the base64 cap fall back to path plus the file route,
which is jailed to the video output directory and serves video/mp4
with range support
- always-on video scripts get bootstrapped default args, matching the
txt2img handler; missing bootstrap raised a TypeError per frame
- checkpoint overrides are rejected with a pointer to the checkpoint
endpoint; unknown engine, model and sampler names return 404 with the
valid choices
- cli/api-video.py client, test/test-video-api.py suite and a
full-test.sh entry; video mimetypes registered; rate-limit cost set
- remove the unreferenced video_ui.run_video dispatcher
Reference entries for the bf16 repo and the sdnq uint4 quant load the
modular pipeline through the standard dispatch. Image tabs run the
model in still mode with audio off; the video tab keeps its own
overrides through the shared per-generation hook. Detailer is not
supported and is disabled with a warning.
First natively modular model: the pipeline is driven directly through
ModularPipeline, with components fetched per workflow (fl2va covers
text and first/last-frame conditioning).
- per-generation overrides snap the canvas to /32, align frames to
the 17n+5 grid and duration window, and keep the bespoke scheduler
pair
- group offload for modular pipelines applied per component in
sd_offload; re-application is a guarded no-op
- audio checkbox pops the audio decode block so decode and muxing are
skipped
- frames=1 renders a single still image: the duration floor lifts per
instance and sub-floor latents pad at the vae decoder
- progress and interrupt handling via a transformer forward pre-hook
- vae scale factor override, tuple-safe patch size
Vae-class components never take group hooks, so group mode kept them
resident on the gpu; a MiniMax-class video vae holds about 10GB that
way while running only seconds per generation. Components above 1GB
now rest in system memory: the apply_forward_hook bridge on encode and
decode fires an on-demand hook that moves the whole module to the
device, so tiled calls find every weight already loaded, and the
processing seams return it to cpu once outputs are materialized. Small
vaes stay resident since the transfer would cost more than it frees.
- placement is decided per component by measured size and requires the
entry bridge; components without it stay resident
- move_model no longer forces on-demand vaes to the gpu for
non-txt2img tasks, and full_vae_encode onloads before binding the
input, which otherwise lands on the resting device
- mode switches clear the stamp and hook in both directions
Network activation ran after prompt encoding, so text encoder lora
weights never affected embeds on the first generation and the stale
result was then served from the embed cache. The trailing unfiltered
activate in network_load also overrode the te exclude filter, so the
lora_apply_te setting was never honored.
- parse and activate networks in process_base before pipeline args are built
- activate_filtered gates text encoder components on per-request or global
lora_apply_te; used by base, hires, detailer and faceid call sites
- network_load accepts activate=False for callers that run their own
deactivate/activate sequence with include/exclude
- network_activate walks excluded components in restore-only mode so a
filtered text encoder reverts to backup instead of keeping stale deltas
- loaded_loras cache is single-entry since per-filter entries go stale when
the setting toggles
- prompt embed cache key includes the effective lora_apply_te value
review feedback from vladmandic on commit 80fde086f.
- run_ltx: drop the inline random.seed() / randrange resolve and use
processing.fix_seed(p) right after the StableDiffusionProcessingVideo
construction. p.seed carries the resolved int across every stage
(latent_pass, both upsample paths, refine, post-refine vae_decode).
matches the existing fix_seed(p) call sites in img2img.py:30,
video_run.py:101, xyz_grid.py:266 and 8 others.
- process_decode: add AudioFrameList(list) subclass and attach_audio
helper to carry output.audio onto the returned frame list. mirrors
the existing output.bytes early-return contract: samples.audio
survives downstream so process_images_inner can collect it via the
Processed.audio kwarg.
- drop the p.audio_capture transit in process_diffusers and the
fallback read in process_images_inner. p is input params, not
output state.
process_decode strips video pipeline output to a flat list of frames at the
PIL early-return (processing_diffusers.py:461-465), so any output.audio is
lost before processing.process_images returns. video pipelines that produce
synchronized audio (LTX-2 audio-capable models) were getting silent mp4s on
the non-latent path.
stash output.audio on p.audio_capture before process_decode runs and let
processing read it back as a fallback when samples is a flat list.
ltx_process non-latent branch strips the (B, 2, N) batch dim with [0] so
write_audio's .T+contiguous() path produces interleaved bytes for AAC s16.
The else branch in process_decode was returning a numpy array directly
from vae_decode, while the if branch properly converted results to a
list. This caused process_samples to return early with an empty infotext
list, and zip(numpy_array, []) produced zero iterations, resulting in
images=0 for detailer, inpainting, and img2img operations.
Bug introduced in 3e8dec929 (Dec 2024), exposed by 63a180be1 (Nov 2025).
CalamitousFelicitousness
Vladimir Mandic
awsr
Disty0