The ref2va checkpoint partition conditions on reference images instead
of keyframes, so it gets its own registry row and reference card, and
the video core marshals PIL images into task_args as
MiniMaxH3ImageReference. Images are converted to RGB first, since the
reference encoder reads the array raw. The keyframe path is unchanged.
Validation runs before the model load in one funnel shared by the tab
and the API, so a rejected request costs nothing: references on a
non-reference model, a reference model with nothing to condition on,
more than nine images, non-images, and aspect outside 1:4 to 4:1 all
return 400. The image path rejects a reference pipe without references
instead of reaching a transformer that was never loaded.
Add POST /sdapi/v1/video plus GET /sdapi/v1/video/models and
GET /sdapi/v1/video/file. The generation body is extracted from the
gradio handler into a keyword-only core, video_run.run, which returns a
structured result and raises typed errors; the positional generate
signature is unchanged and now adapts to the core. Omitting engine and
model drives the currently loaded checkpoint when it is video-capable,
which covers models loaded from local folders without a registry entry.
- registry helpers in models_def (find, engines, pipeline_classes,
workflow_for_class); validate_pipeline reuses the shared class set
- modular pipes stamp their workflow so out-of-registry loads dispatch
onto the modular branch
- disk switches (mp4_*) and wire switches (send_*) are independent;
artifacts above the base64 cap fall back to path plus the file route,
which is jailed to the video output directory and serves video/mp4
with range support
- always-on video scripts get bootstrapped default args, matching the
txt2img handler; missing bootstrap raised a TypeError per frame
- checkpoint overrides are rejected with a pointer to the checkpoint
endpoint; unknown engine, model and sampler names return 404 with the
valid choices
- cli/api-video.py client, test/test-video-api.py suite and a
full-test.sh entry; video mimetypes registered; rate-limit cost set
- remove the unreferenced video_ui.run_video dispatcher
The modular override forces Default instead of the None sentinel:
Default restores the model's own scheduler, which is the bespoke pair.
The reference entries request sampler: Default to match.
The local-folder skip in hf_auth_check keyed on model_index.json only,
so modular pipeline folders fell through to a hub auth check with a
filesystem path as the repo id.
Saving a modular pipeline now rewrites the component references in its
index to the destination folder, so a reload uses the saved quantized
weights instead of following the specs back to the source repositories.
The folder scan accepts modular_model_index.json for direct folders,
matching the snapshot branch, so saved pipelines list in the model
dropdown.
An empty prompt tokenizes to zero tokens and the conditioner fails
reshaping an empty sequence; a single space keeps the generate-on-empty
behavior other models have.
Modular pipelines run every stage inside one pipeline call, leaving
the ui on a single inference label. Forward hooks on the text encoder,
transformer and vae decoders now surface the current stage through
shared state, and the interrupt check runs in every stage so encodes
and tiled decodes abort promptly.
- saving a model registers a job instead of appearing idle
- group offload logs each component before the pin step instead of
only after completion
The modular text encoder quant config now excludes the vision tower.
The default skip pool covers diffusion module names, so qwen-vl vision
blocks quantized alongside the language layers; vision blocks have no
validated quantization precedent and run only for keyframe workflows.
Modular pipelines flagged as base models in the video registry cache
into the diffusers folder, so the folder scan and the model dropdown
pick them up once downloaded and the reference card reports them as
ready; video-only models keep caching into the shared cache and stay
out of the dropdown. The snapshot scan accepts modular_model_index.json
when the plain index is absent.
The modular loader passes sdnq quantization configs to load_components
as a per-component dict: transformers take the model config, the text
encoder takes the te config, and components without an entry load
unquantized. Pre-quantized repositories keep their own config, which
diffusers detects before a passed config applies.
Reference entries for the bf16 repo and the sdnq uint4 quant load the
modular pipeline through the standard dispatch. Image tabs run the
model in still mode with audio off; the video tab keeps its own
overrides through the shared per-generation hook. Detailer is not
supported and is disabled with a warning.
First natively modular model: the pipeline is driven directly through
ModularPipeline, with components fetched per workflow (fl2va covers
text and first/last-frame conditioning).
- per-generation overrides snap the canvas to /32, align frames to
the 17n+5 grid and duration window, and keep the bespoke scheduler
pair
- group offload for modular pipelines applied per component in
sd_offload; re-application is a guarded no-op
- audio checkbox pops the audio decode block so decode and muxing are
skipped
- frames=1 renders a single still image: the duration floor lifts per
instance and sub-floor latents pad at the vae decoder
- progress and interrupt handling via a transformer forward pre-hook
- vae scale factor override, tuple-safe patch size
Move the per-component placement walk from the api server into memstats
next to ram and gpu stats, and add it to the --monitor tick so offload
placement is visible in a log rather than only over the api.
- guard the walk internally: the supervisor loop logs its monitor line
unguarded, and the walk can race a reload or an offload rewrap
- monitor reports gb and merges into a fresh dict, since memory_stats
returns a module global that the per-generation log also prints
- endpoint keeps reporting raw bytes
Add a model section to /sdapi/v1/memory with loaded-model bytes summed
per pipeline component and device, so clients can tell resident weights
from offloaded ones and loop-critical components from edge ones.
- walk components over parameters and buffers, dedupe shared storages,
key by component name then device type
- read the raw model slot so a memory poll never triggers a model load
- section is exception-isolated like ram and cuda; reports an error
string if the walk races a reload
The guard compared component size against half of total system memory and
silently dropped to unpinned streaming above that. Total is the wrong
quantity: a component that fits comfortably in free memory gets denied, and
unpinned leaf streaming is slow out of proportion to its size because every
per-leaf transfer pays staging cost.
Budget against memory free at apply time less a reserve, and cache the
verdict on the module since a granted pin lowers the same reading it derives
from. A denied pin now also drops streams and falls back to block_level,
since leaf plus stream is only the right shape when weights are pinned, and
logs the per-step transfer volume so the cost of the slow path is visible up
front.
Model switches kept most of the previous model resident, and the next
load could stall in kernel reclaim while the freed memory was still held.
- strip group offload hooks in disable_offload so the meta move at unload
actually frees component weights; hook removal resolves wrapper
components that carry hooks on the inner model
- flush the torch pinned host cache in torch_gc so freed streaming
buffers return to the OS instead of staying cached in-process
- skip the pipe-level accelerator move for group-managed pipes: the
offload engine already placed every component, and the move only
dragged on-demand components to the accelerator for the trailing
eviction to undo
Tab runners pre-move the vae module to the gpu before generation,
which parks an on-demand component on the accelerator for the whole
denoise. Moves of a stamped module toward the accelerator now return
early; the entry bridge onloads it when its encode or decode runs.
The per-component stats block printed only from the balanced offload
path, so group mode loads showed no component classes, sizes or
quantization. Group and modular applies now print the same block once
per loaded model, with sizes measured directly when no balanced hook
map exists.
A pipeline-level move to the accelerator carries on-demand components
along with it, since modular pipelines skip only group-hooked modules;
the load path then left a resting vae on the gpu until its first use.
Bulk moves now re-evict stamped components, whose entry points onload
them when needed.
Vae-class components never take group hooks, so group mode kept them
resident on the gpu; a MiniMax-class video vae holds about 10GB that
way while running only seconds per generation. Components above 1GB
now rest in system memory: the apply_forward_hook bridge on encode and
decode fires an on-demand hook that moves the whole module to the
device, so tiled calls find every weight already loaded, and the
processing seams return it to cpu once outputs are materialized. Small
vaes stay resident since the transfer would cost more than it frees.
- placement is decided per component by measured size and requires the
entry bridge; components without it stay resident
- move_model no longer forces on-demand vaes to the gpu for
non-txt2img tasks, and full_vae_encode onloads before binding the
input, which otherwise lands on the resting device
- mode switches clear the stamp and hook in both directions
Group offload hooks report the onload device at module level while the
weights rest on cpu, so every native apply took the parameter
replacement branch in assign_weight and detached the written layers
from the hook's group bookkeeping. The activation and deactivation
walks now remove a component's group hooks before its first weight
write and reapply offload at the end of the pass: writes land in place
on the resting tensors and fresh groups snapshot the result.
- hooks come off lazily, only for components with a covered layer or a
pending backup or factor-stash restore; repeat activations with an
unchanged set leave the hooks untouched
- remove_group_offload_component follows wrapper components to the
inner model that carries the hooks
Group offload is applied per component through one engine shared by
regular and modular pipelines. Each component carries a config
signature: re-application with unchanged settings is a no-op instead
of raising before the first forward or silently keeping a stale
config, and changed settings remove the hooks and reapply. Switching
offload modes cleans up the previous mode's hooks in both directions.
- text encoders always offload at leaf level without streams, so
their weights are never held in pinned host memory
- the vae never takes group hooks and stays resident: the hooks are
forward-scoped, while pipelines enter through encode/decode and
tiled calls re-enter per tile
- new pin offload memory option: disabled pins one group at a time
instead of holding the whole module in non-pageable memory, and
modules larger than half of system memory degrade automatically
- record stream is clamped to stream mode; the standalone
combination is rejected upstream
Entries in the always and never lists are matched against the pipeline
component name (text_encoder, vae) as well as the model class name, so
one entry covers every architecture instead of needing a new class name
per model. Class entries keep working unchanged.
Never is still tested first, so a class entry there exempts a single
model from a component entry in the always list.
Autotune sweeps and kernel compiles run inside the first forward pass at a
new shape and can take minutes with no indication in the log or UI. Add
listeners on the triton autotuning and compilation knobs plus a wrap of the
per-candidate benchmark: a sweep draws a console progress bar over its
candidates and mirrors the count in the live progress text, the completion
line records the kernel, shape, duration and compile share, and standalone
compiles over 1s are logged.
- fix timer_sdnq reading bench_time from the python wrapper functions
instead of the autotuner kernel objects, which left the two matmul
autotune timers permanently empty
- restore the pre-tuning progress text across chained sweeps, so an
abandoned sweep cannot strand its own tuning label in the UI
The audio waveform rides as an attribute on the decoded sample list, but
the batch script hooks rewrap samples into a plain list before the
capture ran, so joint audio-video models muxed silent files whenever a
script runner was attached. Capture the attribute before the hooks run.
The keep-current-scheduler sentinel leaked into user-facing surfaces:
infotexts and filenames recorded Sampler: None, restoring such an
infotext pushed an invalid value into the dropdown, and the API
samplers list never offered Default at all.
- infotext always records the sampler, mapping a stray None to Default
- parse maps legacy Sampler: None infotexts to Default
- api samplers list leads with Default; the remaining excluded config
keys are shared templates, not samplers
Dynamo tracks a lifetime recompile counter per compiled function that
freed models leave climbing while their graphs and guards die, and the
compiled dequant runs fullgraph, so crossing the accumulated limit is a
hard FailOnRecompileLimitHit instead of an eager fallback; enough model
or quant switches in one process got there. unload_model_weights now
calls reset_compile_caches when the compiled dequant is active, dropping
the dead graphs and the counters in the same sweep as the unload gc.
Raised limits only move the wall; the reset removes it.
- scoped to the model unload branch: the reset is global and must only
run when the graphs' owner is being discarded
- regression test trips the wall under a lowered limit and recovers
through the same helper the unload path calls
NATIVE_DISPATCH is the documented registration surface for per-arch
native loaders and is read cross-module by the fidelity analyzer, so
the private marker signaled the opposite of its role and enforced
nothing.
safe_open exposes keys() but implements no __iter__, so the dict idiom applied
in d11a619a6 raises TypeError on every non-streamer load of a pre-quantized
checkpoint. The threaded method reaches the same loop, since load_threaded
delegates one file at a time.
Cached networks are shared objects, and network_load overwrote their
multipliers before network_deactivate ran, so fuse-mode removal recomputed
the subtraction delta with the new values: a strength edit froze at its
first applied value and a later removal left residue in the model weights.
network_load now stages the values on the net and network_activate promotes
them, so the removal pass always subtracts the delta that was applied.
Backup mode restores from stored tensors and was unaffected.
Backup-mode apply and restore installed fresh Parameters. Matmul kernel
selection is sensitive to operand placement, so the first load/remove cycle
shifted otherwise deterministic renders once per process even though every
weight restored byte-exact: bit-identical inputs entered the first post-cycle
unet forward and a different output left it. Copying into the existing
parameter keeps each touched module on its load-time allocation and drops the
per-layer transient of holding old and new weights side by side.
- assign_weight writes weight and bias installs in place when shape, dtype
and device match; quantized fallback layers keep their rebuild path
- regression test pins storage stability across the activate walk
Under a pressed balanced offload, dispatched modules hold meta tensors
whose data lives in the accelerate offload map. The factor path raised
trying to move a meta svd tensor and aborted activation mid-pass; the
legacy requantize path silently skipped those layers. Both left the
model with a partially applied network.
Rebuild the offload state with apply_balanced_offload(force) at
activate and deactivate entry: modules come back real on cpu with
hooks intact and the execution device unchanged, so both paths see
usable tensors and the next forward re-onloads under the watermark.
network_load seeded net.dyn_dim with extra_networks_default_multiplier
when no dyn_dims list was passed, so a float multiplier landed where
consumers expect a rank and slice with it. The prompt path always builds
a per-network list of ints or None, which is why the crash never fired
from the UI; any direct network_load caller hits it in both
rebuild_conventional and the sdnq factor path.
Vladimir Mandic
CalamitousFelicitousness
Dity0