The stage 2 LoRA stores its connector deltas as
diffusion_model.{video,audio}_embeddings_connector, but
LTX2LoraLoaderMixin.lora_state_dict recognizes connectors only under the 2.3
text_embedding_projection prefix. All 3544 keys are routed into the transformer
namespace and peft drops the 224 that land nowhere, leaving refine with a
transformer-only adapter.
Wrapping lora_state_dict moves those keys onto the connectors component using
the rename table from the convert_ltx2_to_diffusers script. The wrapper is inert
once no misrouted keys appear, so it needs no version check.
2.5 reuses the LTX-2 pipeline classes, so it is described through the
capability table: Gemma 4 text encoder, cross-timestep conditioning, and the
upsampler and stage 2 LoRA that now ship inside the model repo. The repo
carries the distilled scheduler config, so Dev rows restore the terminal
shift, and the Dev transformer sits in transformer_full.
Distilled rows stop forcing dynamic shifting on, which remapped their sigma
schedule. Auto duration hands the clip length to the duration head.
The shared repo was written back onto the registry row, a module-level
singleton, so turning the setting off left the row pointing at the shared copy
for the rest of the session. It is chosen into locals instead.
Pipelines rarely report a sample rate, so the save path fell back to 24000.
LTX-2.3 and 2.5 run at 48k, and muxing at half the rate drops the track an
octave. The rate now comes from the vocoder, as the LTX tab already did.
The subfolder name doubles as the pipeline argument, so a repo shipping a
second transformer beside the default one had no way to load it: the argument
was dropped and the default weights stayed. dit_kwarg names the argument when
it differs from the folder.
The sampler shift override looked for flow_shift, which only UniPC-style
schedulers declare, so it never reached the flow-match schedulers 2.x uses.
Config keys are now restored only when the run wrote them, and a cached
upsampler is rebuilt once its repo or the model's VAE changes.
Adds the Gemma 4 text encoder, the duration head and the diffusion decoder.
Pipeline classes are unchanged, so 2.0 and 2.3 load as before.
Image conditioning is now re-compressed to match training, at CRF 18 for 2.5
and 33 for earlier versions.
Some paths left both to pipeline defaults, which track the current upstream
model: distilled picked up guidance it already bakes in, and 2.0 could land on
2.3's joint-sigma path.
The modular branch returned the pipe instead of assigning it, so the video
tab and the API failed with "model not loaded" and the whole post-load tail
was skipped along with it.
The registry helpers filtered only the 'None' placeholder, so the eleven
LTX separator rows resolved as models and a POST naming one reached
load_model with a null repo. is_model() now covers both sentinel kinds.
DiffusionPipeline.download derives ignore_patterns from the passed components and
never reads the caller's, so the kwarg set in load_override was inert and diffusers
logged it as an unexpected keyword. The unsharded connectors duplicate it was meant
to skip was fetched on every LTX-2.3 mirror load.
Resolve the repo to a snapshot built from those patterns and hand from_pretrained
the folder instead of the id. That skips its download path, so the passed component
folders it would have pruned are pruned here instead. The connectors override drops
its own copy of the kwarg, which ModelMixin never read either.
Selected from the specs the pipe already declares, update_components first and
load_components for the rest, so only what the workflow declares gets fetched. Weights
land in hfcache. Covers the TODO on component names.
Encoder sharing left off: shared_te_map matches a substring of the repo name and 4b hits
_dynamic_4bit, redirecting Qwen3-VL to another repo.
None from a family loader means unhandled, so the chain falls through to the folder
loader, which for a modular pipe builds an object holding only its from_config helpers
and installs it. Refuse it when none of the from_pretrained specs materialized.
load_components leaves a failed component as None and carries on, so the pipe reaches
generation short of one and fails somewhere unrelated. Compare against what the workflow
declares and refuse a pipe missing any of it. The conditioner fallback goes too, since
loading it another way just defers the failure into generation.
The ref2va checkpoint partition conditions on reference images instead
of keyframes, so it gets its own registry row and reference card, and
the video core marshals PIL images into task_args as
MiniMaxH3ImageReference. Images are converted to RGB first, since the
reference encoder reads the array raw. The keyframe path is unchanged.
Validation runs before the model load in one funnel shared by the tab
and the API, so a rejected request costs nothing: references on a
non-reference model, a reference model with nothing to condition on,
more than nine images, non-images, and aspect outside 1:4 to 4:1 all
return 400. The image path rejects a reference pipe without references
instead of reaching a transformer that was never loaded.
Add POST /sdapi/v1/video plus GET /sdapi/v1/video/models and
GET /sdapi/v1/video/file. The generation body is extracted from the
gradio handler into a keyword-only core, video_run.run, which returns a
structured result and raises typed errors; the positional generate
signature is unchanged and now adapts to the core. Omitting engine and
model drives the currently loaded checkpoint when it is video-capable,
which covers models loaded from local folders without a registry entry.
- registry helpers in models_def (find, engines, pipeline_classes,
workflow_for_class); validate_pipeline reuses the shared class set
- modular pipes stamp their workflow so out-of-registry loads dispatch
onto the modular branch
- disk switches (mp4_*) and wire switches (send_*) are independent;
artifacts above the base64 cap fall back to path plus the file route,
which is jailed to the video output directory and serves video/mp4
with range support
- always-on video scripts get bootstrapped default args, matching the
txt2img handler; missing bootstrap raised a TypeError per frame
- checkpoint overrides are rejected with a pointer to the checkpoint
endpoint; unknown engine, model and sampler names return 404 with the
valid choices
- cli/api-video.py client, test/test-video-api.py suite and a
full-test.sh entry; video mimetypes registered; rate-limit cost set
- remove the unreferenced video_ui.run_video dispatcher
The modular override forces Default instead of the None sentinel:
Default restores the model's own scheduler, which is the bespoke pair.
The reference entries request sampler: Default to match.
The local-folder skip in hf_auth_check keyed on model_index.json only,
so modular pipeline folders fell through to a hub auth check with a
filesystem path as the repo id.
Saving a modular pipeline now rewrites the component references in its
index to the destination folder, so a reload uses the saved quantized
weights instead of following the specs back to the source repositories.
The folder scan accepts modular_model_index.json for direct folders,
matching the snapshot branch, so saved pipelines list in the model
dropdown.
The progress response carries a stage label in textinfo, but the
button and hint composed their text from the job name alone, so
staged pipelines sat on a bare inference label. The stage now takes
precedence in the button label and appears next to the job in the
performance hint.
CalamitousFelicitousness
Vladimir Mandic
Oleksandr Liutyi
Dity0