Public/automatic
1
0
Fork 0
mirror of https://github.com/vladmandic/automatic synced 2026-09-20 01:31:13 +02:00
Code Issues Packages Projects Releases Wiki Activity

update 

Signed-off-by: Vladimir Mandic <mandic00@live.com>
Vladimir Mandic
2025-01-07 12:55:07 -05:00
parent 06094cad46
commit ba50d70e47
1 changed files with 8 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Quantization.md
+8 -2
@@ -40,7 +40,7 @@ Quantization can be done in multiple ways:
## Quantization Engines
> [!TIP]
> If you're on Windows with a compatible GPU, you may try WSL2 for broader feature compatibiliy
> If you're on Windows with a compatible GPU, you may try **WSL2** for broader feature compatibiliy
> See [WSL Wiki](https://github.com/vladmandic/automatic/wiki/WSL) for more details
SD.Next supports multiple quantization engines, each with multiple quantization schemes:
@@ -49,18 +49,24 @@ SD.Next supports multiple quantization engines, each with multiple quantization
- `Optimium.Quanto` 3 int-based and 2 float-based quantizations schemes
- `GGUF` with pre-quantized weights
> [!IMPORTANT]
> Not all quantization engines are available on all platforms, see notes below for details!
> Using any quantization engine for the first time may result in failure as required libraries are downloaded and installed
> Restart SD.Next and try again if you encounter any issues
### BitsAndBytes
Typical models pre-quantized with `bitsandbytes` would have look like `*nf4.safetensors` or `*fp8.safetensors`
> [!NOTE]
> BnB is the only quantization method that allows for usage of balanced offload as well as quantization on-the-fly during load, thus it is considered most versatile choice, but it is not available on all platforms.
> BnB is allows for usage of balanced offload as well as fast quantization on-the-fly during load, thus it is considered most versatile choice, but it is not available on all platforms.
Limitations:
- default `bitsandbytes` package only supports nVidia GPUs
some quantization types require newer GPU with supported CUDA ops: e.g. *nVidia Turing* GPUs or newer
- `bitsandbytes` relies on `triton` packages which are not available on windows unless manually compiled/installed
without them, performance is significantly reduced
- for nVidia: automatically installed as needed
- for AMD/ROCm: [link](https://huggingface.co/docs/bitsandbytes/main/en/installation?backend=AMD+ROCm#amd-gpu)
- for Intel/IPEX: [link](https://huggingface.co/docs/bitsandbytes/main/en/installation?backend=Intel+CPU+%2B+GPU#multi-backend)