feat(video): condition minimax h3 ref2va on reference images

The ref2va checkpoint partition conditions on reference images instead
of keyframes, so it gets its own registry row and reference card, and
the video core marshals PIL images into task_args as
MiniMaxH3ImageReference. Images are converted to RGB first, since the
reference encoder reads the array raw. The keyframe path is unchanged.

Validation runs before the model load in one funnel shared by the tab
and the API, so a rejected request costs nothing: references on a
non-reference model, a reference model with nothing to condition on,
more than nine images, non-images, and aspect outside 1:4 to 4:1 all
return 400. The image path rejects a reference pipe without references
instead of reaching a transformer that was never loaded.
This commit is contained in:
CalamitousFelicitousness
2026-08-09 23:15:15 +01:00
parent ff42f1631c
commit 3e8f0372ad
8 changed files with 130 additions and 9 deletions
+5
View File
@@ -574,6 +574,11 @@ def validate_pipeline(p: processing.StableDiffusionProcessing):
elif not is_video_model and is_video_pipeline:
log.error(f'Mismatch: type={shared.sd_model_type} cls={shared.sd_model.__class__.__name__} request={p.__class__.__name__} non-video model with video pipeline')
return False
if getattr(shared.sd_model, 'sdnext_video_workflow', None) == 'ref2va' and p.task_args.get('references', None) is None:
# the reference workflow loads its own transformer partition alone: without references the pipeline
# dispatches to the keyframe path and reaches a transformer that was never loaded
log.error(f'Mismatch: type={shared.sd_model_type} cls={shared.sd_model.__class__.__name__} request={p.__class__.__name__} reference workflow requires reference images: use the video tab or the video api')
return False
return True