Offload
Offload moves a model, or model parts, between GPU memory (VRAM) and system memory (RAM). This reduces VRAM usage and helps run larger models on lower-VRAM GPUs.
Offload mode is set in Settings -> Models Offloading
Offload mode can be:
- Balanced: offload only when VRAM usage exceeds the user-specified threshold (default)
- Group: offload model parts in groups, based on the model component type
- Sequential: offload model parts one by one, layer by layer
- Model: offload model parts marked as offload-compatible (deprecated)
- None: no offloading, all model parts remain in VRAM (not recommended)
Balanced
Balanced offload is the default mode for most environments. Balanced offload works differently than all other offloading methods as it performs offloading only when the VRAM usage exceeds the user-specified threshold.
- The default mode for any detected GPU, suitable across the memory range.
- Faster than other offload modes, but requires platform compatibility and enough VRAM.
- Model parts move based on configured thresholds, keeping VRAM usage within the limit.
- The high threshold sets the maximum memory usage allowed for weights of a single model component.
- The low threshold controls when unused model parts are offloaded back to RAM. If VRAM usage is above the low threshold, offloading runs. Otherwise, it does nothing.
- Thresholds are configured under Settings -> Models & Loading -> Offload low watermark / Offload GPU high watermark.
Defaults are selected from detected GPU memory. Every detected GPU uses balanced offload; only the explicit --lowvram flag switches to sequential, and --medvram forces balanced with a low watermark of 0.
- 12-24 GB (default): balanced, low/high watermark 0.2 / 0.6
- 4-12 GB: balanced, low watermark 0.0, large text encoders always offloaded
- 4 GB or less: balanced, low watermark 0.0, plus the
--lowvramlow-memory optimizations (VAE tiling, aggressive garbage collection) - 24 GB or more: balanced, high watermark 0.8, large text encoders always offloaded, CLIP and VAE never offloaded
Group
Group offloading partitions a sub-model’s layers into smaller clusters, dynamically loading one group into GPU memory for execution while offloading the inactive groups back to CPU RAM.
Unlike standard balanced offloading—which swaps entire sub-networks like the text encoder, UNet, or VAE sequentially—group offloading operates at a finer granularity within the transformer or attention blocks themselves.
This provides a middle ground between full model offloading and ultra-granular sequential layer offloading.
Group offloading comes with two modes: leaf_level and block_level:
- block-level offloading swaps entire transformer layers or module groups in larger bulk transfers to balance low VRAM usage with fast asynchronous streaming
- leaf-level offloading swaps parameters down to individual primitive layers (like single linear or convolution weights) for absolute minimum VRAM footprint at the cost of significantly higher overhead
Group offloading performance can be improved by enabling streams usage, both prefetch and overlap, but at the cost of increased VRAM usage
Sequential
Works layer by layer for each model component marked as offload-compatible.
- Recommended for very low VRAM, around 4 GB or less
- Much slower, but can run large models such as FLUX on GPUs with 2-4GB VRAM
- Not supported by all models
Note
Using
--lowvramautomatically enables sequential offload
Model
Works at the model-component level by offloading components marked as offload-compatible. Obsolete as balanced offload can cover all use cases historically covered by model offload.
None
No offloading, all model parts remain in VRAM.
Tip
If you have sufficient VRAM and dont need offloading at all, it is still recommended to use balanced offload
And set desired model type in Model types not to offload (e.g.sdorsdxl)
That way model will still be managed, but not unnecessarily offloaded