Public/automatic
1
0
Fork 0
mirror of https://github.com/vladmandic/automatic synced 2026-09-19 17:24:32 +02:00
Code Issues Packages Projects Releases Wiki Activity

update api and parameters for video endpoint

CalamitousFelicitousness
2026-08-08 22:00:13 +01:00
parent 4114a2e8c2
commit 5bca12bb6c
2 changed files with 94 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
API.md
+50 -1
@@ -27,7 +27,7 @@ SD.Next includes API examples in both Python and JavaScript.
All examples are in the `/cli/` folder:
- Generate endpoints
`api-txt2img.js api-txt2img.py api-img2img.py api-control.py api-faceid.py api-pulid.js`
`api-txt2img.js api-txt2img.py api-img2img.py api-control.py api-video.py api-faceid.py api-pulid.js`
- Status endpoints
`api-progress.py api-history.py api-json.py`
- Utility endpoints
@@ -122,6 +122,55 @@ The loaded model can be set per-request through `override_settings`, for example
By default `override_settings_restore_afterwards` is `true`, so an overridden option is restored once the request completes. For a checkpoint override this means the model is loaded before the request and the previous model is reloaded afterwards, which adds two model loads per request whenever the requested model differs from the one already loaded. An override that matches the current value is dropped and triggers no reload. For a workflow that issues many requests against the same model, setting the model once through `/sdapi/v1/options` or sending `override_settings_restore_afterwards: false` avoids the repeated reloads.
## Video Generation
Video models are served by a dedicated endpoint group:
- `POST /sdapi/v1/video`: generate a video, or a single still frame, using a video model
- `GET /sdapi/v1/video/models`: enumerate available video engines and models
- `GET /sdapi/v1/video/file`: download a saved video artifact
Like `txt2img`, the generate endpoint is synchronous: the request blocks until generation finishes and the result is returned in the response. Progress is reported on `/sdapi/v1/progress`, and a running generation is cancelled with `/sdapi/v1/interrupt`.
### Model selection
The model is selected in one of two ways:
- **Registry model**: pass `engine` and `model` together, using a pair returned by `GET /sdapi/v1/video/models`. The model is loaded on demand before generation.
- **Loaded checkpoint**: omit both fields to drive the currently loaded checkpoint, provided it is a video-capable pipeline. This covers video models loaded from local folders that have no registry entry.
Passing only one of the two fields is rejected with `400`. An unknown engine, model, or sampler name is rejected with `404`, and the valid names are listed in the error detail.
The model list reports each model's input mode: `t2v` (text only), `i2v` (requires `init_image`), `flf2v` (requires `init_image` and `last_image`), `vace`, `animate`, or `workflow` for modular models that select the workflow from the inputs provided. On workflow models, `frames: 1` produces a single still image, returned base64-encoded in `frames`.
### Video outputs
Disk and response outputs are controlled independently:
- `mp4_video`, `mp4_frames`, `mp4_sf`, and `mp4_thumb` select which artifacts are written to disk: the video container, individual frame images, raw frames as safetensors, and a thumbnail.
- `send_video`, `send_frames`, and `send_thumbnail` select which artifacts are returned base64-encoded in the response.
A video larger than the base64 size cap is returned with `video` empty and `video_path` set; the file is then downloaded with `GET /sdapi/v1/video/file?file={video_path}`. The file endpoint only serves paths inside the video output directory; anything else is rejected with `403 Forbidden`.
Switching checkpoints through `override_settings` is not supported on this endpoint and is rejected with `400`; the model is selected per-request through `engine` and `model`, or switched beforehand with `POST /sdapi/v1/checkpoint`.
A minimal request:
```json
{
"engine": "WAN Video",
"model": "WAN 2.2 5B T2V",
"prompt": "a cat walking through tall grass",
"width": 832,
"height": 480,
"frames": 33,
"steps": 20,
"mp4_fps": 16
}
```
The full field list is documented under Video Generation Parameters in [Parameters](Parameters) and served as a live schema at `/docs`. A complete client example is available at `cli/api-video.py`.
## Examples
### T2I with ControlNet
Parameters.md
+44
@@ -31,6 +31,7 @@ The [API](API) guide is the place to look for how to build a request: payload sh
- [Script Parameters](#script-parameters)
- [Override Parameters](#override-parameters)
- [Video Processing Parameters](#video-processing-parameters)
- [Video Generation Parameters](#video-generation-parameters)
- [Resize modes](#resize-modes)
- [Context-aware scaling](#context-aware-scaling)
- [Notes](#notes)
@@ -280,6 +281,49 @@ These settings control postprocessing operations that run after the main image g
| `scheduler_shift` | `float` | `0.0` | `Syntax: scheduler_shift: 0.5` | Shift scheduler timing for animation effects. Creates motion and transformation effects between frames. | sampler shift |
| `vae_tile_frames` | `int` | `0` | `Syntax: vae_tile_frames: 8` | Number of frames to tile in VAE for memory efficiency. Helps process longer videos on limited GPU memory. | tile frames |
## Video Generation Parameters
These parameters apply to the dedicated video endpoint `POST /sdapi/v1/video` and mirror the Video tab controls. Model selection, input modes, output handling, and error behavior are covered in the Video Generation section of the [API](API) guide; the live request schema is served at `/docs` and `/redocs`.
| Parameter | Type | Default | Details / Syntax | Description | UI Label |
| :------------------ | :---------- | :----------- | :------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------- | :--------------------- |
| `engine` | `str` | `None` | `Values: engine names from /sdapi/v1/video/models` | Video engine family. Omitted together with `model`, the currently loaded checkpoint is used. | Video engine |
| `model` | `str` | `None` | `Values: model names from /sdapi/v1/video/models` | Video model name within the engine; loaded on demand before generation. | Video model |
| `prompt` | `str` | `""` | `Syntax: prompt: your text here` | The positive text prompt. | Prompt |
| `negative_prompt` | `str` | `""` | `Syntax: negative_prompt: blurry, low quality` | The negative text prompt. | Negative prompt |
| `styles` | `List[str]` | `[]` | `Syntax: styles: [style1, style2]` | List of prompt style names to apply. | Styles |
| `width` | `int` | `832` | `Range: 64 to 4096` | Output width in pixels; snapped to the model canvas multiple. | Width |
| `height` | `int` | `480` | `Range: 64 to 4096` | Output height in pixels; snapped to the model canvas multiple. | Height |
| `frames` | `int` | `17` | `Range: 1 to 1024` | Number of frames to generate. A value of 1 produces a single still image on workflow models. | Frames |
| `steps` | `int` | `50` | `Range: 1 to 200` | Number of inference steps. | Steps |
| `sampler_name` | `str` | `"Default"` | `Values: see /sdapi/v1/samplers` | Sampler name. `Default` keeps the scheduler the model ships with. | Sampling method |
| `sampler_shift` | `float` | `-1.0` | `Range: -1.0 to 20.0` | Scheduler flow shift. `-1` keeps the model default. | Sampler shift |
| `dynamic_shift` | `bool` | `False` | `Values: True, False` | Enables resolution-dependent scheduler shifting. | Dynamic shift |
| `seed` | `int` | `-1` | `Values: -1 (random), or any positive integer` | Generation seed. | Initial seed |
| `guidance_scale` | `float` | `-1.0` | `Range: -1.0 to 14.0` | Classifier-free guidance scale. `-1` keeps the model default. | Guidance scale |
| `guidance_true` | `float` | `-1.0` | `Range: -1.0 to 14.0` | True CFG scale. `-1` keeps the model default. | True guidance |
| `init_image` | `str` | `None` | `Values: base64 data, data URI, or upload reference` | First-frame input image; required by i2v and flf2v models. | |
| `init_strength` | `float` | `0.8` | `Range: 0.0 to 1.0` | Denoising strength for the init image. | Init strength |
| `last_image` | `str` | `None` | `Values: base64 data, data URI, or upload reference` | Last-frame input image; required by flf2v models. | |
| `vae_type` | `str` | `"Default"` | `Values: "Default", "Tiny", "Remote", "Upscale"` | VAE decode variant. | VAE decode |
| `vae_tile_frames` | `int` | `16` | `Range: 1 to 64` | Number of frames per VAE decode tile; lower values reduce memory use. | Tile frames |
| `audio` | `bool` | `True` | `Values: True, False` | Generates audio on models that support it. | Audio |
| `mp4_fps` | `int` | `24` | `Range: 1 to 60` | Frames per second of the saved video. | FPS |
| `mp4_interpolate` | `int` | `0` | `Range: 0 to 10` | RIFE interpolation passes between frames. | Video interpolation |
| `mp4_codec` | `str` | `"libx264"` | `Values: "none", "libx264"` | Video codec. `none` skips video encoding. | Video codec |
| `mp4_ext` | `str` | `"mp4"` | `Syntax: mp4_ext: webm` | Container extension; the muxer is inferred from it. | Video format |
| `mp4_opt` | `str` | `"crf:16"` | `Syntax: "key:value, key:value"` | Encoder options as comma-separated key:value pairs. | Video options |
| `mp4_video` | `bool` | `True` | `Values: True, False` | Writes the video container to disk. | Video save video |
| `mp4_frames` | `bool` | `False` | `Values: True, False` | Writes individual frame images to disk. | Video save frames |
| `mp4_sf` | `bool` | `False` | `Values: True, False` | Writes raw frames as a safetensors file. | Video save safetensors |
| `mp4_thumb` | `bool` | `True` | `Values: True, False` | Writes a thumbnail image next to the video. | Video save thumbnail |
| `override_settings` | `Dict` | `{}` | `Syntax: override_settings: {key: value}` | Setting overrides applied for this generation only. Checkpoint switching is rejected on this endpoint. | Override settings |
| `script_args` | `list` | `[]` | `Syntax: script_args: [arg1, arg2]` | Positional arguments for a selectable script. | |
| `alwayson_scripts` | `Dict` | `{}` | `Syntax: alwayson_scripts: {script name: {args: [...]}}` | Per-script argument overrides, keyed by script name. | |
| `send_video` | `bool` | `True` | `Values: True, False` | Returns the video base64-encoded in the response. | |
| `send_frames` | `bool` | `False` | `Values: True, False` | Returns every frame base64-encoded in the response. Forced on for still results. | |
| `send_thumbnail` | `bool` | `True` | `Values: True, False` | Returns the thumbnail base64-encoded in the response. | |
## Resize modes
TODO