update wiki

Signed-off-by: Vladimir Mandic <mandic00@live.com>
Vladimir Mandic
2026-07-05 20:04:48 +02:00
parent 9512e471af
commit e2a4ceb0a9
2 changed files with 153 additions and 0 deletions
+92
@@ -0,0 +1,92 @@
# Loading Models
> [!NOTE]
> This guide is intended for ALL model types other than legacy **SD1.5/SD2.1/SD-XL** models that are loaded directly from a single file
## Base Models
> [!IMPORTANT]
> For all base models, start with the **Reference Models** listed in the **Networks** panel on the right sidebar
> SD.Next will automatically download and cache the required model files for you
### Variants
A single base model may have multiple available variants. In the Networks panel, variants are grouped by category
- **Base** - the original model
This is the reference implementation for the family and should be used first
- **Distilled** - a smaller or faster version of the base model
Distilled variants often trade some quality for faster performance
- **Quantized** - a model version pre-quantized using [SDNQ-Quantization](SDNQ-Quantization)
These versions are much smaller (often ~4× smaller) and may run faster on limited memory
- **Community** - community-created variants
These can include fine-tuning, special-purpose training, or other custom modifications
- **Nunchaku** - optimized variants for NVIDIA GPUs
These may include specialized kernels or execution optimizations
> [!TIP]
> When choosing a large model, prefer a *Quantized* variant first, unless you specifically need a *Base* model
### Downloaded Cache
SD.Next separates large shared model components from smaller model definition files:
- `models/huggingface/<model_name>/`: Large model components
These are typically the heavy weights such as the *transformer*, and *text encoder*
- `models/Diffusers/<model_name>/`: Model definitions, configs
Also includes smaller supporting modules such as *VAE*
This separation allows multiple models to share the same large components when possible
For example, different models may reuse the same text encoder, so the large encoder weights are downloaded once and reused
## Finetunes & Custom Models
### Downloading Safetensors
When downloading safetensors from external sources such as [CivitAI](https://civitai.com/), put the files in the correct model folder so SD.Next can detect them properly
- **SD15 / SD21 / SD-XL** → `models/Stable-diffusion/`
- **All other diffusion models** → `models/UNET/`
This includes models such as Flux.1, Flux.2, Chroma, Z-Image, Anima, Krea.2, Qwen, and similar families
Other model component types belong in these folders:
- **VAE** → `models/VAE/`
- **LoRA** → `models/Lora/`
- **Text Encoder** → `models/Text-encoder/`
- **Embedding** → `models/Embeddings/`
- **ControlNet** → `models/control/`
- **Detailer** → `models/yolo/`
- **Upscaler** → `models/chainner/`
> [!TIP]
> The root `models` folder is the base location for all model files
> You can change this path in **Settings → Paths**
### Loading finetunes
> [!IMPORTANT]
> Always load the reference base model first, then load the finetune on top of it
A finetune is usually NOT a complete standalone model
It modifies a single component of the base model (typically, the *transformer* component), so the full base model and its standard components (for example, the *VAE* and *text encoder*) must be loaded first
To load a finetune:
1. Load the reference base model from the **Networks → Model** panel
2. Select the finetune model in **Networks → UNET** panel
### Loading via Settings
You can also configure base and finetune models in **Settings → Model Loading**.
- **Base model**: usually found in the *Quicksettings* section
- **UNET model**: can also be added to *Quicksettings* for easier access
### Common pitfalls
- Custom models downloaded from external sources should be placed in the correct folder and may require restarting SD.Next or refreshing the model list
- If a finetune is loaded before its reference model, the model may fail to initialize
- If the file is in the wrong `models/` subfolder, SD.Next may not detect it automatically
- When a model uses separate encoder or VAE files, those files must be placed in the correct folder and loaded by the matching **Settings** option
+61
@@ -0,0 +1,61 @@
# Performance Timers
After every generation, SD.Next prints a performance line below the image and in the server log that breaks down where the time went:
> Time: 5.52s | total 8.57 pipeline 3.60 decode 1.80 onload 1.02 prompt 0.77 preview 0.76 move 0.46 | GPU 11238 MB 46% | RAM 15.73 GB 24%
- First number is the wall-clock duration of the inference, including UI overhead of all stages
- The middle section lists per-stage timers, sorted by duration
- Stages taking less than 0.25 sec are hidden
- **GPU** and **RAM** show peak memory usage for the run
Reading this line is the fastest way to find out *why* a generation is slow and which setting to tune
## Stage reference
### Model execution
| Timer | Meaning | Note |
| ----- | ------- | ---- |
| `pipeline` | The main denoising loop: UNet or DiT/transformer forward passes for each step | This is the core compute of the generation |
| `hires` | Second-pass processing when HiRes fix is enabled | |
| `refine` | Refiner pass when a refiner model is enabled | |
| `decode` | The final decode stage: turning latents into the output image, including image post-processing |
| `vae` | Time spent inside actual VAE encode/decode calls, e.g. encoding the input image for img2img/inpaint | Overlaps with `decode` on the output side |
| `te` | Text encoder forward pass: turning the parsed prompt into conditioning tensors |
| `prompt` | Prompt parse and text-encode: attention syntax, prompt scheduling and building the embeddings | Overlaps with `te` |
| `lora` | Loading, applying and removing LoRA weights | See [LoRA](LoRA) for the performance impact of different LoRA modes |
### Memory management
| Timer | Meaning | Note |
| ----- | ------- | ---- |
| `sync` | GPU synchronization time: waiting for GPU to finish one task before moving to the next one | can be (unsafe) disabled in *Settings -> Backend Settings* |
| `onload` | Moving model components from RAM to VRAM when offloading is active | See [Offload](Offload) |
| `offload` | Moving model components from VRAM back to RAM | See [Offload](Offload) |
| `move` | Module component forced device moves | Used by legacy pipelines only, modern models report `onload`/`offload` instead |
| `gc` | Garbage collection: clearing unused RAM and VRAM between stages |
### Everything else
| Timer | Meaning | Note |
| ----- | ------- | ---- |
| `preview` | Generating live previews during sampling | Runs asynchronously on a separate thread, so it only partially adds to wall-clock time |
| `callback` | Per-step callbacks executed during sampling |
| `init` | Pipeline and sampler initialization |
| `prepare`, `pre` | Applying processing modifiers (IP-Adapter, HiDiffusion, PAG, etc.) before the run |
| `post` | Unapplying processing modifiers and post-processing after the run |
| `process` | Overall per-batch processing wrapper | overlaps with the stages above |
| `validate` | Validating decoded samples and converting them to image format |
| `proc` | Running Control input processors (canny, depth, pose, etc.) on the input image |
> [!NOTE]
> Video models (LTX, FramePack) report their own stage timers such as `base`, `upsample`, `sample`, `encode` and `vision` which follow the same principle: each named stage is the time spent in that part of the video pipeline.
## What to tune
- **High `pipeline`**: this is the actual model compute. Fewer steps, a faster sampler, quantization, or model compile. See [Performance Tuning](Performance-Tuning)
- **High `onload`/`offload`/`move`**: model parts are being shuffled between RAM and VRAM every generation. If you have VRAM headroom, raise the offload low watermark or exempt specific model/module types from offloading in *Settings -> Models & Loading*. Note that offloading exists for a reason: workflows that spike VRAM usage, such as HiRes to a much higher resolution, need that headroom for decode. See [Offload](Offload)
- **High `decode`/`vae`**: consider a faster VAE, or check whether VAE tiling/slicing is enabled unnecessarily on a high-VRAM system. See [VAE](VAE)
- **High `lora`**: LoRA apply cost depends heavily on LoRA type and mode; see the measurements in [Offload](Offload#performance-notes)
- **High `te`/`prompt`**: large text encoders (T5, LLMs) are expensive; offloading them adds onload cost on every prompt change. Reusing the same prompt avoids re-encoding