When download_civit_preview short-circuits at 304 (file exists), the
rescan path now falls back to backfill_preview_parameters: checks
whether the file already carries an embedded parameters chunk and, if
not, embeds img.meta in place. Existing libraries get retroactive
metadata via the existing "Search metadata" rescan, no model
re-download required.
Civitai's CDN strips PNG tEXt/EXIF chunks during delivery, leaving the
modernUI Preview metadata tab empty for downloaded networks. Format the
version API's images[].meta into sdnext's standard parameters channel
(PNG tEXt / JPEG-WEBP EXIF UserComment) so read_info_from_image picks
it up through the existing path. Embedding is wrapped in try/except
and never blocks the download.
Upstream LTX2 encode_prompt is not @torch.no_grad-decorated; only __call__
is. Calling encode_prompt directly from ltx_process recorded the full
Gemma3 forward graph.
Add a per-model native LoRA path for Chroma that handles both PEFT-style
(diffusion_model.<dotted>.lora_A.weight, no alpha) and kohya-flat
(lora_unet_<flat> with .alpha) layouts, plus LoKR/LoHA/OFT/DoRA.
Resolves issue #4169 (ostris ai-toolkit Chroma LoRAs failed to load).
- new pipelines/chroma/chroma_lora.py with try_load_lora/lokr/loha/oft
- fused-QKV expansion: 3-way equal for img/txt qkv, 4-way unequal
[3072, 3072, 3072, 12288] for single linear1 (Q+K+V+proj_mlp)
- NetworkModuleLokrSliceChunk for unequal-row LoKR slicing
- path rewrite from Flux-source to ChromaTransformer2DModel naming
- lora_load.py: dispatch chroma to the native loader and narrow the
kohya-Flux convert gate to f1 only
Adds pipelines/ernie/ernie_lora.py with try_load_lora/lokr/loha/oft
entry points modeled on the z-image native loader, wires it into
lora_load.load_safetensors, and adds 'ernieimage' to allow_native.
ERNIE attention is not fused (separate to_q/to_k/to_v/to_out.0), so
the loader skips the qkv-split machinery the z-image loader needs and
supports all four families uniformly. Recognized prefixes are
diffusion_model., transformer., and lora_unet_; PEFT lora_A/lora_B
keys are normalized to lora_down/lora_up. Verified against PEFT, LoKR,
and Kohya/AIT files in the wild with zero unmapped or shape-mismatched
modules.
review feedback from vladmandic on commit 80fde086f.
- run_ltx: drop the inline random.seed() / randrange resolve and use
processing.fix_seed(p) right after the StableDiffusionProcessingVideo
construction. p.seed carries the resolved int across every stage
(latent_pass, both upsample paths, refine, post-refine vae_decode).
matches the existing fix_seed(p) call sites in img2img.py:30,
video_run.py:101, xyz_grid.py:266 and 8 others.
- process_decode: add AudioFrameList(list) subclass and attach_audio
helper to carry output.audio onto the returned frame list. mirrors
the existing output.bytes early-return contract: samples.audio
survives downstream so process_images_inner can collect it via the
Processed.audio kwarg.
- drop the p.audio_capture transit in process_diffusers and the
fallback read in process_images_inner. p is input params, not
output state.
- resolve seed=-1 once at top of run_ltx and thread the int through every
stage (StableDiffusionProcessingVideo, _latent_pass, upsample 0.9/2.x,
refine, post-refine vae_decode). get_generator(-1) reseeds globally per
call so each stage was rolling an uncorrelated value; p.seed now carries
the resolved seed so reruns reproduce.
- pre-encode prompts via shared.sd_model.encode_prompt before the latent
path; park the four tensors on CPU and pass them as prompt_embeds /
*_attention_mask kwargs through _latent_pass and refine_args. Stage 2
reuses Stage 1's embeds instead of re-running the text encoder. The
manual encode is outside pipe.__call__ so the post-forward offload hook
never fires; apply_balanced_offload(force=True) re-anchors the device
map so the text encoder doesn't stay pinned through Stage 1.
- skip audio_vae.decode + vocoder on refine when audio_enable=False:
output_type='latent' bypasses the internal audio + video decode and
hands off to the post-refine vae_decode block. Per-step audio
cross-attention still runs for video conditioning.
drop the project-specific stage 1 direct audio decode (a6870f7d2). smoke
testing showed broadband tinniness on distilled is BWE-bound, not stage 2
corruption, so the deviation didn't fix the underlying issue.
revert to canonical:
- _latent_pass returns video latents only; result.audio is unused.
- stage 2 receives audio_latents=None (default), prepare_audio_latents
generates fresh gaussian noise, audio scheduler runs the 3 stage-2
sigmas under identity guidance, video<->audio cross-attention
conditions the audio branch.
- capture stage 2 result.audio[0].float().cpu() for save.
upstream evidence: pipeline_ltx2.py:937-940 documents audio_latents as
pre-generated noisy latents (initial gaussian, not stage 1 output); no
caller in diffusers threads stage outputs into the kwarg.
non-latent path is unchanged and continues to work via p.audio_capture.
process_decode strips video pipeline output to a flat list of frames at the
PIL early-return (processing_diffusers.py:461-465), so any output.audio is
lost before processing.process_images returns. video pipelines that produce
synchronized audio (LTX-2 audio-capable models) were getting silent mp4s on
the non-latent path.
stash output.audio on p.audio_capture before process_decode runs and let
processing read it back as a fallback when samples is a flat list.
ltx_process non-latent branch strips the (B, 2, N) batch dim with [0] so
write_audio's .T+contiguous() path produces interleaved bytes for AAC s16.
stop threading stage 1 audio_latents into stage 2 refine. Lightricks/LTX-2#126
reports the two-stage pipeline degrades audio quality, confirmed locally as
clean speech with tinny ambient/foley/music on the threaded path.
root cause: stage 2 prepare_audio_latents calls _create_noised_state at
noise_scale=0.909 (pipeline_ltx2.py:704-714, 598-603), keeping ~9% of stage 1
signal. 3 refine steps recover speech via video<->audio cross-attention but
not broadband content.
new path: _latent_pass decodes audio_latents to waveform via audio_vae +
vocoder mirroring pipeline_ltx2.py:1471-1473 exactly (input cast to
audio_vae.dtype, no module dtype mutation). stage 2 result.audio is
discarded; cross-attention still runs each block for video conditioning.
relabel toggle to 'LTX save audio' (default true) since audio always
generates on 2.x audio-capable models; the toggle gates mux only. hint
added to locale_en.json.
split add_audio_stream from write_audio. avformat_write_header runs on
first container.mux() and freezes the stream set, so audio added after
video packets has time_base=0/0 and raises 'Cannot rebase to zero time.'
atomic_save_video registers the audio stream before the encode loop.
CalamitousFelicitousness
Vladimir Mandic
awsr
Disty0