mirror of
https://github.com/vladmandic/automatic
synced 2026-09-19 09:14:35 +02:00
update wiki
Signed-off-by: Vladimir Mandic <mandic00@live.com>
+92
@@ -0,0 +1,92 @@
|
||||
# Loading Models
|
||||
|
||||
> [!NOTE]
|
||||
> This guide is intended for ALL model types other than legacy **SD1.5/SD2.1/SD-XL** models that are loaded directly from a single file
|
||||
|
||||
## Base Models
|
||||
|
||||
> [!IMPORTANT]
|
||||
> For all base models, start with the **Reference Models** listed in the **Networks** panel on the right sidebar
|
||||
> SD.Next will automatically download and cache the required model files for you
|
||||
|
||||
### Variants
|
||||
|
||||
A single base model may have multiple available variants. In the Networks panel, variants are grouped by category
|
||||
|
||||
- **Base** - the original model
|
||||
This is the reference implementation for the family and should be used first
|
||||
- **Distilled** - a smaller or faster version of the base model
|
||||
Distilled variants often trade some quality for faster performance
|
||||
- **Quantized** - a model version pre-quantized using [SDNQ-Quantization](SDNQ-Quantization)
|
||||
These versions are much smaller (often ~4× smaller) and may run faster on limited memory
|
||||
- **Community** - community-created variants
|
||||
These can include fine-tuning, special-purpose training, or other custom modifications
|
||||
- **Nunchaku** - optimized variants for NVIDIA GPUs
|
||||
These may include specialized kernels or execution optimizations
|
||||
|
||||
> [!TIP]
|
||||
> When choosing a large model, prefer a *Quantized* variant first, unless you specifically need a *Base* model
|
||||
|
||||
### Downloaded Cache
|
||||
|
||||
SD.Next separates large shared model components from smaller model definition files:
|
||||
|
||||
- `models/huggingface/<model_name>/`: Large model components
|
||||
These are typically the heavy weights such as the *transformer*, and *text encoder*
|
||||
- `models/Diffusers/<model_name>/`: Model definitions, configs
|
||||
Also includes smaller supporting modules such as *VAE*
|
||||
|
||||
This separation allows multiple models to share the same large components when possible
|
||||
For example, different models may reuse the same text encoder, so the large encoder weights are downloaded once and reused
|
||||
|
||||
## Finetunes & Custom Models
|
||||
|
||||
### Downloading Safetensors
|
||||
|
||||
When downloading safetensors from external sources such as [CivitAI](https://civitai.com/), put the files in the correct model folder so SD.Next can detect them properly
|
||||
|
||||
- **SD15 / SD21 / SD-XL** → `models/Stable-diffusion/`
|
||||
- **All other diffusion models** → `models/UNET/`
|
||||
|
||||
This includes models such as Flux.1, Flux.2, Chroma, Z-Image, Anima, Krea.2, Qwen, and similar families
|
||||
|
||||
Other model component types belong in these folders:
|
||||
|
||||
- **VAE** → `models/VAE/`
|
||||
- **LoRA** → `models/Lora/`
|
||||
- **Text Encoder** → `models/Text-encoder/`
|
||||
- **Embedding** → `models/Embeddings/`
|
||||
- **ControlNet** → `models/control/`
|
||||
- **Detailer** → `models/yolo/`
|
||||
- **Upscaler** → `models/chainner/`
|
||||
|
||||
> [!TIP]
|
||||
> The root `models` folder is the base location for all model files
|
||||
> You can change this path in **Settings → Paths**
|
||||
|
||||
### Loading finetunes
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Always load the reference base model first, then load the finetune on top of it
|
||||
|
||||
A finetune is usually NOT a complete standalone model
|
||||
It modifies a single component of the base model (typically, the *transformer* component), so the full base model and its standard components (for example, the *VAE* and *text encoder*) must be loaded first
|
||||
|
||||
To load a finetune:
|
||||
|
||||
1. Load the reference base model from the **Networks → Model** panel
|
||||
2. Select the finetune model in **Networks → UNET** panel
|
||||
|
||||
### Loading via Settings
|
||||
|
||||
You can also configure base and finetune models in **Settings → Model Loading**.
|
||||
|
||||
- **Base model**: usually found in the *Quicksettings* section
|
||||
- **UNET model**: can also be added to *Quicksettings* for easier access
|
||||
|
||||
### Common pitfalls
|
||||
|
||||
- Custom models downloaded from external sources should be placed in the correct folder and may require restarting SD.Next or refreshing the model list
|
||||
- If a finetune is loaded before its reference model, the model may fail to initialize
|
||||
- If the file is in the wrong `models/` subfolder, SD.Next may not detect it automatically
|
||||
- When a model uses separate encoder or VAE files, those files must be placed in the correct folder and loaded by the matching **Settings** option
|
||||
@@ -0,0 +1,61 @@
|
||||
# Performance Timers
|
||||
|
||||
After every generation, SD.Next prints a performance line below the image and in the server log that breaks down where the time went:
|
||||
|
||||
> Time: 5.52s | total 8.57 pipeline 3.60 decode 1.80 onload 1.02 prompt 0.77 preview 0.76 move 0.46 | GPU 11238 MB 46% | RAM 15.73 GB 24%
|
||||
|
||||
- First number is the wall-clock duration of the inference, including UI overhead of all stages
|
||||
- The middle section lists per-stage timers, sorted by duration
|
||||
- Stages taking less than 0.25 sec are hidden
|
||||
- **GPU** and **RAM** show peak memory usage for the run
|
||||
|
||||
Reading this line is the fastest way to find out *why* a generation is slow and which setting to tune
|
||||
|
||||
## Stage reference
|
||||
|
||||
### Model execution
|
||||
|
||||
| Timer | Meaning | Note |
|
||||
| ----- | ------- | ---- |
|
||||
| `pipeline` | The main denoising loop: UNet or DiT/transformer forward passes for each step | This is the core compute of the generation |
|
||||
| `hires` | Second-pass processing when HiRes fix is enabled | |
|
||||
| `refine` | Refiner pass when a refiner model is enabled | |
|
||||
| `decode` | The final decode stage: turning latents into the output image, including image post-processing |
|
||||
| `vae` | Time spent inside actual VAE encode/decode calls, e.g. encoding the input image for img2img/inpaint | Overlaps with `decode` on the output side |
|
||||
| `te` | Text encoder forward pass: turning the parsed prompt into conditioning tensors |
|
||||
| `prompt` | Prompt parse and text-encode: attention syntax, prompt scheduling and building the embeddings | Overlaps with `te` |
|
||||
| `lora` | Loading, applying and removing LoRA weights | See [LoRA](LoRA) for the performance impact of different LoRA modes |
|
||||
|
||||
### Memory management
|
||||
|
||||
| Timer | Meaning | Note |
|
||||
| ----- | ------- | ---- |
|
||||
| `sync` | GPU synchronization time: waiting for GPU to finish one task before moving to the next one | can be (unsafe) disabled in *Settings -> Backend Settings* |
|
||||
| `onload` | Moving model components from RAM to VRAM when offloading is active | See [Offload](Offload) |
|
||||
| `offload` | Moving model components from VRAM back to RAM | See [Offload](Offload) |
|
||||
| `move` | Module component forced device moves | Used by legacy pipelines only, modern models report `onload`/`offload` instead |
|
||||
| `gc` | Garbage collection: clearing unused RAM and VRAM between stages |
|
||||
|
||||
### Everything else
|
||||
|
||||
| Timer | Meaning | Note |
|
||||
| ----- | ------- | ---- |
|
||||
| `preview` | Generating live previews during sampling | Runs asynchronously on a separate thread, so it only partially adds to wall-clock time |
|
||||
| `callback` | Per-step callbacks executed during sampling |
|
||||
| `init` | Pipeline and sampler initialization |
|
||||
| `prepare`, `pre` | Applying processing modifiers (IP-Adapter, HiDiffusion, PAG, etc.) before the run |
|
||||
| `post` | Unapplying processing modifiers and post-processing after the run |
|
||||
| `process` | Overall per-batch processing wrapper | overlaps with the stages above |
|
||||
| `validate` | Validating decoded samples and converting them to image format |
|
||||
| `proc` | Running Control input processors (canny, depth, pose, etc.) on the input image |
|
||||
|
||||
> [!NOTE]
|
||||
> Video models (LTX, FramePack) report their own stage timers such as `base`, `upsample`, `sample`, `encode` and `vision` which follow the same principle: each named stage is the time spent in that part of the video pipeline.
|
||||
|
||||
## What to tune
|
||||
|
||||
- **High `pipeline`**: this is the actual model compute. Fewer steps, a faster sampler, quantization, or model compile. See [Performance Tuning](Performance-Tuning)
|
||||
- **High `onload`/`offload`/`move`**: model parts are being shuffled between RAM and VRAM every generation. If you have VRAM headroom, raise the offload low watermark or exempt specific model/module types from offloading in *Settings -> Models & Loading*. Note that offloading exists for a reason: workflows that spike VRAM usage, such as HiRes to a much higher resolution, need that headroom for decode. See [Offload](Offload)
|
||||
- **High `decode`/`vae`**: consider a faster VAE, or check whether VAE tiling/slicing is enabled unnecessarily on a high-VRAM system. See [VAE](VAE)
|
||||
- **High `lora`**: LoRA apply cost depends heavily on LoRA type and mode; see the measurements in [Offload](Offload#performance-notes)
|
||||
- **High `te`/`prompt`**: large text encoders (T5, LLMs) are expensive; offloading them adds onload cost on every prompt change. Reusing the same prompt avoids re-encoding
|
||||
Reference in New Issue
Block a user