Public/automatic
1
0
Fork 0
mirror of https://github.com/vladmandic/automatic synced 2026-09-20 01:31:13 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'dev' into patch-2

Browse Source
This commit is contained in:
CalamitousFelicitousness
2025-10-01 20:58:58 +01:00
committed by GitHub
parent 78820a14dc 450464aa92
commit 78711fb1d4
156 changed files with 4077 additions and 1307 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.eslintrc.json
+2 -1
View File
@@ -116,7 +116,8 @@
"idbDel": "readonly",
"idbAdd": "readonly",
"initChangelog": "readonly",
"sendNotification": "readonly"
"sendNotification": "readonly",
"monitorConnection": "readonly"
},
"ignorePatterns": [
"node_modules",
.gitignore
-8
View File
@@ -46,14 +46,6 @@ tunableop_results*.csv
!package.json
!requirements.txt
# pyinstaller
*.spec
build/
dist/
# dynamically generated
/repositories/deepseek-vl2/
# all dynamic stuff
/extensions/**/*
/outputs/**/*
.gitmodules
-4
View File
@@ -14,10 +14,6 @@
path = extensions-builtin/stable-diffusion-webui-rembg
url = https://github.com/vladmandic/sd-extension-rembg
ignore = dirty
[submodule "extensions-builtin/sd-webui-agent-scheduler"]
path = extensions-builtin/sd-webui-agent-scheduler
url = https://github.com/ArtVentureX/sd-webui-agent-scheduler
ignore = dirty
[submodule "extensions-builtin/sdnext-modernui"]
path = extensions-builtin/sdnext-modernui
url = https://github.com/BinaryQuantumSoul/sdnext-modernui
.pylintrc
+1
View File
@@ -44,6 +44,7 @@ ignore-paths=/usr/lib/.*$,
pipelines/segmoe,
scripts/consistory,
scripts/ctrlx,
scripts/daam,
scripts/demofusion,
scripts/freescale,
scripts/infiniteyou,
.ruff.toml
+1
View File
@@ -24,6 +24,7 @@ exclude = [
"pipelines/segmoe",
"scripts/lbm",
"scripts/daam",
"scripts/xadapter",
"scripts/pulid",
"scripts/instantir",
CHANGELOG.md
+100 -2
View File
@@ -1,5 +1,98 @@
# Change Log for SD.Next
## Update for 2025-10-01
- **Models**
- [WAN 2.2 14B VACE](https://huggingface.co/alibaba-pai/Wan2.2-VACE-Fun-A14B)
available for *text-to-image* and *text-to-video* and *image-to-video* workflows
- [Qwen Image Edit 2509](https://huggingface.co/Qwen/Qwen-Image-Edit-2509) and [Nunchaku Qwen Image Edit 2509](https://huggingface.co/nunchaku-tech/nunchaku-qwen-image-edit-2509)
updated version of Qwen Image Edit with improved image consistency
- [HiDream E1.1](https://huggingface.co/HiDream-ai/HiDream-E1-1)
updated version of E1 image editing model
- [Tencent FLUX.1 Dev SRPO](https://huggingface.co/tencent/SRPO)
SRPO is trained by Tencent with specific technique: directly aligning the full diffusion trajectory with fine-grained human preference
- [Nunchaku SDXL](https://huggingface.co/nunchaku-tech/nunchaku-sdxl) and [Nunchaku SDXL Turbo](https://huggingface.co/nunchaku-tech/nunchaku-sdxl-turbo)
impact of nunchaku engine on unet-based model such as sdxl is much less than on a dit-based models, but its still significantly faster than baseline
note that nunchaku optimized and prequantized unet is replacement for base unet, so its only applicable to base models, not any of finetunes
*how to use*: enable nunchaku in settings -> quantization and then load either sdxl-base or sdxl-base-turbo reference models
*note*: sdxl support for nunchaku is not in released version of `nunchaku==1.0.0`, so you need to build [nunchaku](https://nunchaku.tech/docs/nunchaku/installation/installation.html) from source
- **Features**
- [Cache-DiT](https://github.com/vipshop/cache-dit)
cache-dit is a unified, flexible and training-free cache acceleration framework
compatible with many dit-based models such as FLUX.1, Qwen, HunyuanImage, Wan2.2, Chroma, etc.
enable in *settings -> pipeline modifers -> cache-dit*
- [Nunchaku Flux.1 PulID](https://nunchaku.tech/docs/nunchaku/python_api/nunchaku.pipeline.pipeline_flux_pulid.html)
automatically enabled if loaded model is FLUX.1 with Nunchaku engine enabled and when PulID script is enabled
- **Compute**
- **ROCm** for Windows
support for both official torch preview release of `torch-rocm` for windows and **TheRock** unoffical `torch-rocm` builds for windows
note that rocm for windows is still in preview and has limited gpu support, please check rocm docs for details
- **DirectML** warn as end-of-life
`torch-directml` received no updates in over 1 year and its currently superceded by `rocm` or `zluda`
- command line params `--use-zluda` and `--use-rocm` will attempt desired operation or fail if not possible
previously sdnext was performing a fallback to `torch-cpu` which is not desired
- **installer**: warn if cuda or rocm are available and `torch-cpu` is installed
- **Extensions**
- [Agent-Scheduler](https://github.com/SipherAGI/sd-webui-agent-scheduler)
was a high-value built-in extension, but it has not been maintained for 1.5 years
it also does not work with control and video tabs which are the core of sdnext nowadays
so it has been removed from built-in extensions: manual installation is still possible
- [DAAM: Diffusion Attentive Attribution Maps](https://github.com/castorini/daam)
create heatmap visualizations of which parts of the prompt influenced which parts of the image
available in scripts for sdxl text-to-image workflows
- **Offloading**
- improve offloading for pipelines with multiple stages such as *wan-2.2-14b*
- add timers to measure onload/offload times during generate
- experimental offloading using `torch.streams`
enable in settings -> model offloading
- new feature to specify which models types not to offload
in *settings -> model offloading -> model types not to offload*
- **UI**
- **connection monitor**
main logo in top-left corner now indicates server connection status and hovering over it shows connection details
- separate guidance and detail sections
- networks ability to filter lora by base model version
- **Other**
- server will note when restart is recommended due to package updates
- **interrrupt** will now show last known preview image
*keep incomplete* setting is now *save interrupted*
- **logging** enable `debug`, `docs` and `api-docs` by default
- **logging** add detailed ram/vram utilization info to log
logging frequency can be specified using `--monitor x` command line param, where x is number of seconds
- **ipex** simplify internal implementation
- refactor to use new libraries
- styles and wildcards now use same seed as main generate for reproducible results
- **api** new endpoint POST `/sdapi/v1/civitai` to trigger civitai models metadata update
accepts optional `page` parameter to search specific networks page
- **reference models** additional example images, thanks @liutyi
- **reference models** add model size and release date, thanks @alerikaisattera
- **video** support for configurable multi-stage models such as WAN-2.2-14B
- **video** new LTX model selection
- replace `pynvml` with `nvidia-ml-py` for gpu monitoring
- update **loopback** script with radon seed option, thanks @rabanti
- **vae** slicing enable for lowvram/medvram, tiling for lowvram, both disabled otherwise
- **attention** remove split-attention and add explicitly attention slicing enable/disable option
enable in *settings -> compute settings*
can be combined with sdp, enabling may improve stability when used on iGPU or shared memory systems
- **Experimental**
- `new` command line flag enables new `pydantic` and `albumentations` packages
- **modular pipelines**: enable in *settings -> model options*
only compatible with some pipelines, invalidates preview generation
- **modular guiders**: automatically used for compatible pipelines when *modular pipelines* is enabled
allows for using many different guidance methods:
*CFG, CFGZero, PAG, APG, SLG, SEG, TCFG, FDG*
- **Fixes**
- **Microsoft Florence 2** both base and large variants
*note* this will trigger download of the new variant of the model, feel free to delete older variant in `huggingface` folder
- **MiaoshouAI PromptGen** 1.5/2.0 in both base and large variants
- ui: fix image metadata display when switching selected image in control tab
- framepack: add explicit hf-login before framepack load
- framepack: patch solver for unsupported gpus
- benchmark: remove forced sampler from system info benchmark
- xyz-grid: fix xyz grid with random seeds
- reference: fix download for sd15/sdxl reference models
- fix checks in init/mask image decode
## Update for 2025-09-15
### Highlights for 2025-09-15
@@ -82,6 +175,13 @@ And check out new **history** tab in the right panel, it now shows visualization
*experimental*: requires new pydantic package which *may* break other things, to enable start sdnext with `--new` flag
*note*: this is model quantization only, no support for tensorRT inference yet
- **Other**
- **LoRA** allow specifying module to apply lora on
*example*: `<lora:mylora:1.0:module=unet>` would apply lora *only* on unet regardless of lora content
this is particularly useful when you have multiple loras and you want to apply them on different parts of the model
*example*: `<lora:firstlora:1.0:high>` and `<lora:secondlora:1.0:low>`
*note*: `low` is shorthand for `module=transformer_2` and `high` is shortcut for `module=transformer`
- **Detailer** allow manually setting processing resolution
*note*: this does not impact the actual image resolution, only the resolution at which detailer internally operates
- refactor reuse-seed and add functionality to all tabs
- refactor modernui js codebase
- move zluda flash attenion to *Triton Flash attention* option
@@ -96,8 +196,6 @@ And check out new **history** tab in the right panel, it now shows visualization
- add deprecation warning for `python==3.9`
- allow setting denoise strength to 0 in control/img2img
this allows to run workflows which only refine or detail existing image without changing it
- **Detailer** allow manually setting processing resolution
*note*: this does not impact the actual image resolution, only the resolution at which detailer internally operates
- **Fixes**
- normalize path hanlding when deleting images
- unified compile upscalers
README.md
+1
View File
@@ -6,6 +6,7 @@
![Last update](https://img.shields.io/github/last-commit/vladmandic/sdnext?svg=true)
![License](https://img.shields.io/github/license/vladmandic/sdnext?svg=true)
[![Discord](https://img.shields.io/discord/1101998836328697867?logo=Discord&svg=true)](https://discord.gg/VjvR2tabEX)
[![DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/vladmandic/sdnext)
[![Sponsors](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/vladmandic)
[Docs](https://vladmandic.github.io/sdnext-docs/) | [Wiki](https://github.com/vladmandic/sdnext/wiki) | [Discord](https://discord.gg/VjvR2tabEX) | [Changelog](CHANGELOG.md)
TODO.md
+6 -6
View File
@@ -13,13 +13,9 @@ Main ToDo list can be found at [GitHub projects](https://github.com/users/vladma
- Video Core: API
- Video LTX: TeaCache and others, API, Conditioning preprocess Video: LTX API
### Blocked items
- Upgrade: `albumentations`
- blocked by `insightface`
### Under Consideration
- [Inf-DiT](https://github.com/zai-org/Inf-DiT)
- [X-Omni](https://github.com/X-Omni-Team/X-Omni/blob/main/README.md)
- [DiffSynth Studio](https://github.com/modelscope/DiffSynth-Studio)
- [IPAdapter negative guidance](https://github.com/huggingface/diffusers/discussions/7167)
@@ -40,14 +36,18 @@ Main ToDo list can be found at [GitHub projects](https://github.com/users/vladma
### New models
- [Kandinsky 5](https://github.com/ai-forever/Kandinsky-5)
- [Bytedance Lynx](https://github.com/bytedance/lynx)
- [HunyuanImage](https://huggingface.co/tencent/HunyuanImage-2.1)
- [Phantom HuMo](https://github.com/Phantom-video/Phantom)
- [Lumina-DiMOO](https://huggingface.co/Alpha-VLLM/Lumina-DiMOO)
- [Wan2.2 Fun](https://huggingface.co/collections/alibaba-pai/wan22-fun-68958eabec343b948f1225c5) (includes VACE, Control, etc)
- [Wan2.2-Animate-14B](https://huggingface.co/Wan-AI/Wan2.2-Animate-14B)
- [Magi](https://github.com/SandAI-org/MAGI-1)(https://github.com/huggingface/diffusers/pull/11713)
- [SEVA](https://github.com/huggingface/diffusers/pull/11440)
- [Ming](https://github.com/inclusionAI/Ming)
- [Liquid](https://github.com/FoundationVision/Liquid)
- [Step1X](https://github.com/stepfun-ai/Step1X-Edit)
- [LucyEdit](https://github.com/huggingface/diffusers/pull/12340)
- [SD3 UltraEdit](https://github.com/HaozheZhao/UltraEdit)
- [WAN2GP](https://github.com/deepbeepmeep/Wan2GP)
- [SelfForcing](https://github.com/guandeh17/Self-Forcing)
cli/localize.js
+4 -3
View File
@@ -7,9 +7,10 @@ const { GoogleGenerativeAI } = require('@google/generative-ai');
const api_key = process.env.GOOGLE_AI_API_KEY;
const model = 'gemini-2.5-flash';
const prompt = `
// eslint-disable-next-line max-len
Translate attached JSON from English to {language} using following rules: fields id, label and reload should be preserved from original, field localized should be a translated version of field label and field hint should be translated in-place. if field is less than 3 characters, do not translate it and keep it as is. Every JSON entry should have id, label, localized, reload and hint fields. Output should be pure JSON without any additional text. To better match translation, context of the text is related to Stable Diffusion and topic of Generative AI.`;
const prompt = `Translate attached JSON from English to {language} using following rules: fields id, label and reload should be preserved from original, field localized should be a translated version of field label and field hint should be translated in-place.
if field is less than 3 characters, do not translate it and keep it as is.
Every JSON entry should have id, label, localized, reload and hint fields.
Output should be pure JSON without any additional text. To better match translation, context of the text is related to Stable Diffusion and topic of Generative AI.`;
const languages = {
hr: 'Croatian',
de: 'German',
configs/Dockerfile.cuda
+2 -2
View File
@@ -41,11 +41,11 @@ ENV SD_DOCKER=true
# tcmalloc is not required but it is highly recommended
ENV LD_PRELOAD=libtcmalloc.so.4
# sdnext will run all necessary pip install ops and then exit
RUN ["python", "/app/launch.py", "--debug", "--uv", "--use-cuda", "--log", "sdnext.log", "--test", "--optional"]
RUN ["python", "/app/launch.py", "--uv", "--use-cuda", "--log", "sdnext.log", "--test", "--optional"]
# preinstall additional packages to avoid installation during runtime
# actually run sdnext
CMD ["python", "launch.py", "--debug", "--skip-all", "--listen", "--quick", "--api-log", "--log", "sdnext.log"]
CMD ["python", "launch.py", "--listen", "--quick", "--log", "sdnext.log"]
# expose port
EXPOSE 7860
extensions-builtin/sd-extension-system-info
+1 -1
Submodule extensions-builtin/sd-extension-system-info updated: 615d2f8103...19a1b1b722
extensions-builtin/sd-webui-agent-scheduler
-1
Submodule extensions-builtin/sd-webui-agent-scheduler deleted from a33753321b
extensions-builtin/sdnext-modernui
+1 -1
Submodule extensions-builtin/sdnext-modernui updated: 430c8140c0...e56b4f6954
html/locale_en.json
-1
View File
@@ -32,7 +32,6 @@
{"id":"","label":"","localized":"","reload":"","hint":"Sort by time, descending"}
],
"main": [
{"id":"","label":"SD.Next","localized":"","reload":"","hint":"SD.Next<br>All-in-one WebUI for AI generative image and video creation"},
{"id":"","label":"Prompt","localized":"","reload":"","hint":"Describe image you want to generate"},
{"id":"","label":"Start","localized":"","reload":"","hint":"Start"},
{"id":"","label":"End","localized":"","reload":"","hint":"End"},
html/previews.json
+6 -5
View File
@@ -1,6 +1,6 @@
{
"sd-v21-512-ema": "models/Reference/stabilityai--stable-diffusion-2-1-base.jpg",
"stabilityai--stable-diffusion-xl-base-1.0": "models/Reference/sd_xl_base_1.0.jpg",
"stabilityai--stable-diffusion-xl-base-1.0": "models/Reference/stabilityai--stable-diffusion-xl-base-1.0.jpg",
"stabilityai--stable-diffusion-3-medium-diffusers": "models/Reference/stabilityai--stable-diffusion-3.jpg",
"stabilityai--stable-diffusion-3.5-medium": "models/Reference/stabilityai--stable-diffusion-3_5-medium.jpg",
"stabilityai--stable-diffusion-3.5-large": "models/Reference/stabilityai--stable-diffusion-3_5-large.jpg",
@@ -20,10 +20,11 @@
"HiDream-I1-Full": "models/Reference/HiDream-I1 Full",
"lodestones--Chroma1-Base": "models/Reference/lodestones--Chroma-Base.jpg",
"lodestones--Chroma1-HD": "models/Reference/lodestones--Chroma-HD.jpg",
"chroma-unlocked-v50": "models/Reference/lodestones Chroma Unlocked HD",
"chroma-unlocked-v50-annealed": "models/Reference/lodestones Chroma Unlocked HD",
"chroma-unlocked-v50": "models/Reference/lodestones--Chroma-detail.jpg",
"chroma-unlocked-v50-annealed": "models/Reference/lodestones--Chroma-annealed.jpg",
"vladmandic--Qwen-Lightning": "models/Reference/Qwen-Lightning.jpg",
"vladmandic--Qwen-Lightning-Edit": "models/Reference/Qwen-Lightning.jpg",
"Wan-AI--Wan2.2-T2V-A14B-Diffusers": "models/Reference/Wan2.2-T2V-A14B.jpg",
"Wan-AI--Wan2.1-T2V-14B-Diffusers": "models/Reference/Wan-AI--Wan2.1.jpg"
"Wan-AI--Wan2.2-T2V-A14B-Diffusers": "models/Reference/Wan-AI--Wan2.2-T2V-A14B-Diffusers.jpg",
"Wan-AI--Wan2.1-T2V-14B-Diffusers": "models/Reference/Wan-AI--Wan2.1-T2V-14B-Diffusers.jpg",
"linoyts--Wan2.2-VACE-Fun-14B-diffusers": "models/Reference/linoyts--Wan2.2-VACE-Fun-14B-diffusers.jpg"
}
html/reference.json
+293 -114
View File
@@ -1,26 +1,26 @@
{
"Tempest-by-Vlad XL": {
"path": "tempestByVlad_baseV01.safetensors@https://civitai.com/api/download/models/1301775",
"preview": "tempest-by-vlad-base.jpg",
"preview": "tempestByVlad_baseV01.jpg",
"desc": "Flexible SDXL model with custom encoder and finetuned for larger landscape resolutions with high details and high contrast.",
"extras": ""
},
"Tempest-by-Vlad XL Hyper": {
"path": "tempestByVlad_hyperV01.safetensors@https://civitai.com/api/download/models/1343512",
"preview": "tempest-by-vlad-hyper.jpg",
"preview": "tempestByVlad_hyperV01.jpg",
"desc": "Custom distilled variant with goal to get as-normal-as-possible model that works with low steps and guidance-free",
"extras": ""
},
"Juggernaut XL XI": {
"path": "juggernautXL_juggXIByRundiffusion.safetensors@https://civitai.com/api/download/models/782002",
"preview": "juggernautXL_v9Rundiffusionphoto2.jpg",
"preview": "juggernautXL_juggXIByRundiffusion.jpg",
"desc": "Showcase finetuned model based on Stable diffusion XL",
"extras": "sampler: DEIS, steps: 20, cfg_scale: 6.0"
},
"Juggernaut XL XI Lightning": {
"path": "juggernautXL_juggXILightningByRD.safetensors@https://civitai.com/api/download/models/920957",
"preview": "juggernautXL_v9Rdphoto2Lightning.jpg",
"preview": "juggernautXL_juggXILightningByRD.jpg",
"desc": "Showcase finetuned model based on Stable diffusion XL",
"extras": "sampler: DPM SDE, steps: 6, cfg_scale: 2.0"
},
@@ -37,7 +37,9 @@
"path": "v1-5-pruned-fp16-emaonly.safetensors@https://huggingface.co/Aptronym/SDNext/resolve/main/Reference/v1-5-pruned-fp16-emaonly.safetensors?download=true",
"preview": "v1-5-pruned-fp16-emaonly.jpg",
"desc": "Stable Diffusion 1.5 is the base model all other 1.5 checkpoint were trained from. It's a latent text-to-image diffusion model capable of generating photo-realistic images given any text input. The Stable-Diffusion-v1-5 checkpoint was initialized with the weights of the Stable-Diffusion-v1-2 checkpoint and subsequently fine-tuned on 595k steps at resolution 512x512.",
"extras": "width: 512, height: 512, sampler: DEIS, steps: 20, cfg_scale: 6.0"
"extras": "width: 512, height: 512, sampler: DEIS, steps: 20, cfg_scale: 6.0",
"size": 2.28,
"date": "2022 October"
},
"StabilityAI StableDiffusion 2.1": {
"path": "huggingface/stabilityai/stable-diffusion-2-1-base",
@@ -45,7 +47,9 @@
"skip": true,
"variant": "fp16",
"desc": "This stable-diffusion-2-1-base model fine-tunes stable-diffusion-2-base (512-base-ema.ckpt) with 220k extra steps taken",
"extras": "width: 512, height: 512, sampler: DEIS, steps: 20, cfg_scale: 6.0"
"extras": "width: 512, height: 512, sampler: DEIS, steps: 20, cfg_scale: 6.0",
"size": 2.58,
"date": "2022 December"
},
"StabilityAI StableDiffusion 2.1 V": {
"path": "huggingface/stabilityai/stable-diffusion-2-1",
@@ -55,11 +59,23 @@
"desc": "This stable-diffusion-2 model is resumed from stable-diffusion-2-base (512-base-ema.ckpt) and trained for 150k steps using a v-objective on the same dataset. Resumed for another 140k steps on 768x768 images",
"extras": "width: 768, height: 768, sampler: DEIS, steps: 20, cfg_scale: 6.0"
},
"StabilityAI StableDiffusion XL 1.0 Base": {
"path": "sd_xl_base_1.0.safetensors@https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors?download=true",
"preview": "sd_xl_base_1.0.jpg",
"desc": "Stable Diffusion XL (SDXL) is the latest AI image generation model that is tailored towards more photorealistic outputs with more detailed imagery and composition compared to previous SD models, including SD 2.1. It can make realistic faces, legible text within the images, and better image composition, all while using shorter and simpler prompts at a greatly increased base resolution of 1024x1024. Just like its predecessors, SDXL has the ability to generate image variations using image-to-image prompting, inpainting (reimagining of the selected parts of an image), and outpainting (creating new parts that lie outside the image borders).",
"extras": "sampler: DEIS, steps: 20, cfg_scale: 6.0"
"StabilityAI StableDiffusion XL": {
"path": "stabilityai/stable-diffusion-xl-base-1.0",
"preview": "stabilityai--stable-diffusion-xl-base-1.0.jpg",
"desc": "Stable Diffusion XL (SDXL) is AI image generation model that is tailored towards more photorealistic outputs with more detailed imagery and composition compared to previous SD models, including SD 2.1. It can make realistic faces and better image composition, all while using shorter and simpler prompts at a greatly increased base resolution of 1024x1024. Just like its predecessors, SDXL has the ability to generate image variations using image-to-image prompting, inpainting (reimagining of the selected parts of an image), and outpainting (creating new parts that lie outside the image borders).",
"skip": true,
"variant": "fp16",
"extras": "",
"size": 6.94,
"date": "2023 July"
},
"StabilityAI StableDiffusion XL Turbo": {
"path": "stabilityai/sdxl-turbo",
"preview": "stabilityai--sdxl-turbo.jpg",
"desc": "SDXL-Turbo is a fast generative text-to-image model that can synthesize photorealistic images from a text prompt in a 1-4 steps.",
"skip": true,
"variant": "fp16",
"extras": "steps: 4, cfg_scale: 0.0"
},
"StabilityAI Stable Cascade": {
"path": "huggingface/stabilityai/stable-cascade",
@@ -67,7 +83,9 @@
"variant": "bf16",
"desc": "Stable Cascade is a diffusion model built upon the Würstchen architecture and its main difference to other models like Stable Diffusion is that it is working at a much smaller latent space. Why is this important? The smaller the latent space, the faster you can run inference and the cheaper the training becomes. How small is the latent space? Stable Diffusion uses a compression factor of 8, resulting in a 1024x1024 image being encoded to 128x128. Stable Cascade achieves a compression factor of 42, meaning that it is possible to encode a 1024x1024 image to 24x24, while maintaining crisp reconstructions. The text-conditional model is then trained in the highly compressed latent space. Previous versions of this architecture, achieved a 16x cost reduction over Stable Diffusion 1.5",
"preview": "stabilityai--stable-cascade.jpg",
"extras": "sampler: Default, cfg_scale: 4.0, image_cfg_scale: 1.0"
"extras": "sampler: Default, cfg_scale: 4.0, image_cfg_scale: 1.0",
"size": 11.82,
"date": "2024 February"
},
"StabilityAI Stable Cascade Lite": {
"path": "huggingface/stabilityai/stable-cascade-lite",
@@ -75,7 +93,9 @@
"variant": "bf16",
"desc": "Stable Cascade is a diffusion model built upon the Würstchen architecture and its main difference to other models like Stable Diffusion is that it is working at a much smaller latent space. Why is this important? The smaller the latent space, the faster you can run inference and the cheaper the training becomes. How small is the latent space? Stable Diffusion uses a compression factor of 8, resulting in a 1024x1024 image being encoded to 128x128. Stable Cascade achieves a compression factor of 42, meaning that it is possible to encode a 1024x1024 image to 24x24, while maintaining crisp reconstructions. The text-conditional model is then trained in the highly compressed latent space. Previous versions of this architecture, achieved a 16x cost reduction over Stable Diffusion 1.5",
"preview": "stabilityai--stable-cascade-lite.jpg",
"extras": "sampler: Default, cfg_scale: 4.0, image_cfg_scale: 1.0"
"extras": "sampler: Default, cfg_scale: 4.0, image_cfg_scale: 1.0",
"size": 4.97,
"date": "2024 February"
},
"StabilityAI Stable Diffusion 3 Medium": {
"path": "stabilityai/stable-diffusion-3-medium-diffusers",
@@ -83,7 +103,9 @@
"variant": "fp16",
"desc": "Stable Diffusion 3 Medium is a Multimodal Diffusion Transformer (MMDiT) text-to-image model that features greatly improved performance in image quality, typography, complex prompt understanding, and resource-efficiency",
"preview": "stabilityai--stable-diffusion-3.jpg",
"extras": "sampler: Default, cfg_scale: 7.0"
"extras": "sampler: Default, cfg_scale: 7.0",
"size": 15.14,
"date": "2024 June"
},
"StabilityAI Stable Diffusion 3.5 Medium": {
"path": "stabilityai/stable-diffusion-3.5-medium",
@@ -91,7 +113,9 @@
"variant": "fp16",
"desc": "Stable Diffusion 3.5 Medium is a Multimodal Diffusion Transformer with improvements (MMDiT-X) text-to-image model that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency.",
"preview": "stabilityai--stable-diffusion-3_5-medium.jpg",
"extras": "sampler: Default, cfg_scale: 7.0"
"extras": "sampler: Default, cfg_scale: 7.0",
"size": 15.89,
"date": "2024 October"
},
"StabilityAI Stable Diffusion 3.5 Large": {
"path": "stabilityai/stable-diffusion-3.5-large",
@@ -99,7 +123,9 @@
"variant": "fp16",
"desc": "Stable Diffusion 3.5 Large is a Multimodal Diffusion Transformer (MMDiT) text-to-image model that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency.",
"preview": "stabilityai--stable-diffusion-3_5-large.jpg",
"extras": "sampler: Default, cfg_scale: 7.0"
"extras": "sampler: Default, cfg_scale: 7.0",
"size": 26.98,
"date": "2024 October"
},
"StabilityAI Stable Diffusion 3.5 Turbo": {
"path": "stabilityai/stable-diffusion-3.5-large-turbo",
@@ -115,107 +141,153 @@
"preview": "black-forest-labs--FLUX.1-dev.jpg",
"desc": "FLUX.1 models are based on a hybrid architecture of multimodal and parallel diffusion transformer blocks, scaled to 12B parameters and builing on flow matching",
"skip": true,
"extras": "sampler: Default, cfg_scale: 3.5"
"extras": "sampler: Default, cfg_scale: 3.5",
"size": 32.93,
"date": "2024 August"
},
"Black Forest Labs FLUX.1 Schnell": {
"path": "black-forest-labs/FLUX.1-schnell",
"preview": "black-forest-labs--FLUX.1-schnell.jpg",
"desc": "FLUX.1 models are based on a hybrid architecture of multimodal and parallel diffusion transformer blocks, scaled to 12B parameters and builing on flow matching. Trained using latent adversarial diffusion distillation, FLUX.1 [schnell] can generate high-quality images in only 1 to 4 steps",
"skip": true,
"extras": "sampler: Default, cfg_scale: 3.5"
"extras": "sampler: Default, cfg_scale: 3.5",
"size": 32.93,
"date": "2024 August"
},
"Black Forest Labs FLUX.1 Kontext Dev": {
"path": "black-forest-labs/FLUX.1-Kontext-dev",
"preview": "black-forest-labs--FLUX.1-Kontext-dev.jpg",
"desc": "FLUX.1 Kontext [dev] is a 12 billion parameter rectified flow transformer capable of editing images based on text instructions.",
"skip": true,
"extras": "sampler: Default, cfg_scale: 3.5"
"extras": "sampler: Default, cfg_scale: 3.5",
"size": 32.93,
"date": "2025 June"
},
"Black Forest Labs FLUX.1 Krea Dev": {
"path": "black-forest-labs/FLUX.1-Krea-dev",
"preview": "black-forest-labs--FLUX.1-Krea-dev.jpg",
"desc": "FLUX.1 Krea [dev] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions.",
"skip": true,
"extras": "sampler: Default, cfg_scale: 4.5",
"size": 32.93,
"date": "2025 July"
},
"Tencent FLUX.1 Dev SRPO": {
"path": "vladmandic/flux.1-dev-SRPO",
"preview": "vladmandic--flux.1-dev-SRPO.jpg",
"desc": "FLUX.1 Dev SRPO is Tencent trained with specific technique: Directly Aligning the Full Diffusion Trajectory with Fine-Grained Human Preference",
"skip": true,
"extras": "sampler: Default, cfg_scale: 4.5"
},
"lodestones Chroma1 HD": {
"path": "lodestones/Chroma1-HD",
"preview": "lodestones--Chroma-HD.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. This is the high-res fine-tune of the Chroma1-Base at a 1024x1024 resolution.",
"skip": true,
"extras": ""
},
"lodestones Chroma1 Base": {
"path": "lodestones/Chroma1-Base",
"preview": "lodestones--Chroma-Base.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. This is the core 512x512 model. It's a solid, all-around foundation for pretty much any creative project.",
"skip": true,
"extras": ""
},
"lodestones Chroma1 Flash": {
"path": "lodestones/Chroma1-Flash",
"preview": "lodestones--Chroma-flash.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. A fine-tuned version of the Chroma1-Base made to find the best way to make these flow matching models faster.",
"skip": true,
"extras": ""
},
"lodestones Chroma1 v50 Preview Annealed": {
"path": "vladmandic/chroma-unlocked-v50-annealed",
"preview": "lodestones--Chroma-annealed.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. Re-tweaked variant with extra noise added.",
"skip": true,
"extras": ""
},
"lodestones Chroma1 v48 Preview": {
"path": "vladmandic/chroma-unlocked-v48",
"preview": "lodestones--Chroma.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. Last raw version of Chroma before final finetuning.",
"skip": true,
"extras": ""
},
"lodestones Chroma1 v48 Preview Calibrated": {
"path": "vladmandic/chroma-unlocked-v48-detail-calibrated",
"preview": "lodestones--Chroma-detail.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. Last raw version of Chroma before final finetuning but with some detail calibration.",
"skip": true,
"extras": ""
},
"Qwen-Image": {
"path": "Qwen/Qwen-Image",
"preview": "Qwen--Qwen-Image.jpg",
"desc": " Qwen-Image, an image generation foundation model in the Qwen series that achieves significant advances in complex text rendering and precise image editing.",
"skip": true,
"extras": ""
"extras": "",
"size": 56.1,
"date": "2025 August"
},
"Qwen-Image-Edit": {
"path": "Qwen/Qwen-Image-Edit",
"preview": "Qwen--Qwen-Image-Edit.jpg",
"desc": "Qwen-Image-Edit, the image editing version of Qwen-Image. Built upon our 20B Qwen-Image model, Qwen-Image-Edit successfully extends Qwen-Image’s unique text rendering capabilities to image editing tasks, enabling precise text editing.",
"skip": true,
"extras": ""
"extras": "",
"size": 56.1,
"date": "2025 August"
},
"Qwen-Image-Edit-2509": {
"path": "Qwen/Qwen-Image-Edit-2509",
"preview": "Qwen--Qwen-Image-Edit-2509.jpg",
"desc": "Qwen-Image-Edit, the image editing version of Qwen-Image. Built upon our 20B Qwen-Image model, Qwen-Image-Edit successfully extends Qwen-Image’s unique text rendering capabilities to image editing tasks, enabling precise text editing.",
"skip": true,
"extras": "",
"size": 56.1,
"date": "2025 September"
},
"Qwen-Image-Lightning": {
"path": "vladmandic/Qwen-Lightning",
"preview": "vladmandic--Qwen-Lightning.jpg",
"desc": "Qwen-Lightning is step-distilled from Qwen-Image to allow for generation in 8 steps.",
"skip": true,
"extras": "steps: 8"
"extras": "steps: 8",
"size": 56.1,
"date": "2025 August"
},
"Qwen-Image-Distill": {
"path": "SahilCarterr/Qwen-Image-Distill-Full",
"preview": "SahilCarterr--Qwen-Image-Distill-Full.jpg",
"desc": "Qwen-Image-Distill is a distilled and accelerated version of Qwen-Image by DiffSynth-Studio.",
"skip": true,
"extras": "steps: 15"
"extras": "steps: 15",
"size": 56.1,
"date": "2025 August"
},
"Qwen-Image-Lightning-Edit": {
"path": "vladmandic/Qwen-Lightning-Edit",
"preview": "vladmandic--Qwen-Lightning-Edit.jpg",
"desc": " Qwen-Lightning-Edit is step-distilled from Qwen-Image-Edit to allow for generation in 8 steps.",
"skip": true,
"extras": "steps: 8"
"extras": "steps: 8",
"size": 56.1,
"date": "2025 August"
},
"lodestones Chroma1 HD": {
"path": "lodestones/Chroma1-HD",
"preview": "lodestones--Chroma-HD.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. This is the high-res fine-tune of the Chroma1-Base at a 1024x1024 resolution.",
"skip": true,
"extras": "",
"size": 26.84,
"date": "2025 July"
},
"lodestones Chroma1 Base": {
"path": "lodestones/Chroma1-Base",
"preview": "lodestones--Chroma-Base.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. This is the core 512x512 model. It's a solid, all-around foundation for pretty much any creative project.",
"skip": true,
"extras": "",
"size": 26.84,
"date": "2025 July"
},
"lodestones Chroma1 Flash": {
"path": "lodestones/Chroma1-Flash",
"preview": "lodestones--Chroma-flash.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. A fine-tuned version of the Chroma1-Base made to find the best way to make these flow matching models faster.",
"skip": true,
"extras": "",
"size": 26.84,
"date": "2025 July"
},
"lodestones Chroma1 v50 Preview Annealed": {
"path": "vladmandic/chroma-unlocked-v50-annealed",
"preview": "lodestones--Chroma-annealed.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. Re-tweaked variant with extra noise added.",
"skip": true,
"extras": "",
"size": 26.84,
"date": "2025 July"
},
"lodestones Chroma1 v48 Preview": {
"path": "vladmandic/chroma-unlocked-v48",
"preview": "lodestones--Chroma.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. Last raw version of Chroma before final finetuning.",
"skip": true,
"extras": "",
"size": 26.84,
"date": "2025 July"
},
"lodestones Chroma1 v48 Preview Calibrated": {
"path": "vladmandic/chroma-unlocked-v48-detail-calibrated",
"preview": "lodestones--Chroma-detail.jpg",
"desc": "Chroma is a 8.9B parameter model based on FLUX.1-schnell. It’s fully Apache 2.0 licensed, ensuring that anyone can use, modify, and build on top of it—no corporate gatekeeping. Last raw version of Chroma before final finetuning but with some detail calibration.",
"skip": true,
"extras": "",
"size": 26.84,
"date": "2025 July"
},
"Ostris Flex.2 Preview": {
@@ -223,14 +295,18 @@
"preview": "ostris--Flex.2-preview.jpg",
"desc": "Open Source 8B parameter Text to Image Diffusion Model with universal control and inpainting support built in. Early access preview release. The next version of Flex.1-alpha",
"skip": true,
"extras": "sampler: Default, cfg_scale: 3.5"
"extras": "sampler: Default, cfg_scale: 3.5",
"size": 25.65,
"date": "2025 April"
},
"Ostris Flex.1 Alpha": {
"path": "ostris/Flex.1-alpha",
"preview": "ostris--Flex.1-alpha.jpg",
"desc": "Flex.1 alpha is a pre-trained base 8 billion parameter rectified flow transformer capable of generating images from text descriptions. It has a similar architecture to FLUX.1-dev, but with fewer double transformer blocks (8 vs 19)",
"skip": true,
"extras": "sampler: Default, cfg_scale: 3.5"
"extras": "sampler: Default, cfg_scale: 3.5",
"size": 25.65,
"date": "2025 January"
},
"Wan-AI Wan2.1 1.3B": {
@@ -238,14 +314,18 @@
"preview": "Wan-AI--Wan2.1-T2V-1.3B-Diffusers.jpg",
"desc": "Wan is an advanced and powerful visual generation model developed by Tongyi Lab of Alibaba Group. It can generate videos based on text, images, and other control signals. The Wan2.1 series models are now fully open-source.",
"skip": true,
"extras": "sampler: Default"
"extras": "sampler: Default",
"size": 27.72,
"date": "2025 February"
},
"Wan-AI Wan2.1 14B": {
"path": "Wan-AI/Wan2.1-T2V-14B-Diffusers",
"preview": "Wan-AI--Wan2.1-T2V-14B-Diffusers.jpg",
"desc": "Wan is an advanced and powerful visual generation model developed by Tongyi Lab of Alibaba Group. It can generate videos based on text, images, and other control signals. The Wan2.1 series models are now fully open-source.",
"skip": true,
"extras": "sampler: Default"
"extras": "sampler: Default",
"size": 78.52,
"date": "2025 February"
},
"Wan-AI Wan2.2 5B": {
"path": "Wan-AI/Wan2.2-TI2V-5B-Diffusers",
@@ -268,27 +348,40 @@
"skip": true,
"extras": "sampler: Default"
},
"Wan-AI Wan2.2 14B VACE": {
"path": "linoyts/Wan2.2-VACE-Fun-14B-diffusers",
"preview": "linoyts--Wan2.2-VACE-Fun-14B-diffusers.jpg",
"desc": "Wan2.2, offering more powerful capabilities, better performance, and superior visual quality. With Wan2.2, we have focused on incorporating the following technical innovations: MoE Architecture, Data Scalling, Cinematic Aesthetics, Efficient High-Definition Hybrid",
"skip": true,
"extras": "sampler: Default"
},
"Freepik F-Lite": {
"path": "Freepik/F-Lite",
"preview": "Freepik--F-Lite.jpg",
"desc": "F Lite is a 10B parameter diffusion model created by Freepik and Fal, trained exclusively on copyright-safe and SFW content. The model was trained on Freepik's internal dataset comprising approximately 80 million copyright-safe images, making it the first publicly available model of this scale trained exclusively on legally compliant and SFW content.",
"skip": true,
"extras": "sampler: Default, cfg_scale: 3.5"
"extras": "sampler: Default, cfg_scale: 3.5",
"size": 19.81,
"date": "2025 May"
},
"Freepik F-Lite Texture": {
"path": "Freepik/F-Lite-Texture",
"preview": "Freepik--F-Lite-Texture.jpg",
"desc": "F Lite is a 10B parameter diffusion model created by Freepik and Fal, trained exclusively on copyright-safe and SFW content. The model was trained on Freepik's internal dataset comprising approximately 80 million copyright-safe images, making it the first publicly available model of this scale trained exclusively on legally compliant and SFW content.",
"skip": true,
"extras": "sampler: Default, cfg_scale: 3.5"
"extras": "sampler: Default, cfg_scale: 3.5",
"size": 19.81,
"date": "2025 May"
},
"Freepik F-Lite 7B": {
"path": "Freepik/F-Lite-7B",
"preview": "Freepik--F-Lite-7B.jpg",
"desc": "F Lite is a 10B parameter diffusion model created by Freepik and Fal, trained exclusively on copyright-safe and SFW content. The model was trained on Freepik's internal dataset comprising approximately 80 million copyright-safe images, making it the first publicly available model of this scale trained exclusively on legally compliant and SFW content.",
"skip": true,
"extras": "sampler: Default, cfg_scale: 3.5"
"extras": "sampler: Default, cfg_scale: 3.5",
"size": 13.89,
"date": "2025 May"
},
"SDXS DreamShaper 512": {
@@ -309,13 +402,17 @@
"path": "Efficient-Large-Model/SANA1.5_1.6B_1024px_diffusers",
"desc": "Sana is an efficient model with scaling of training-time and inference time techniques. SANA-1.5 delivers: efficient model growth from 1.6B Sana-1.0 model to 4.8B, achieving similar or better performance than training from scratch and saving 60% training cost; efficient model depth pruning, slimming any model size as you want; powerful VLM selection based inference scaling, smaller model+inference scaling > larger model.",
"preview": "Efficient-Large-Model--SANA1.5_1.6B_1024px_diffusers.jpg",
"skip": true
"skip": true,
"size": 9.49,
"date": "2025 March"
},
"NVLabs Sana 1.5 4.8B 1k": {
"path": "Efficient-Large-Model/SANA1.5_4.8B_1024px_diffusers",
"desc": "Sana is an efficient model with scaling of training-time and inference time techniques. SANA-1.5 delivers: efficient model growth from 1.6B Sana-1.0 model to 4.8B, achieving similar or better performance than training from scratch and saving 60% training cost; efficient model depth pruning, slimming any model size as you want; powerful VLM selection based inference scaling, smaller model+inference scaling > larger model.",
"preview": "Efficient-Large-Model--SANA1.5_4.8B_1024px_diffusers.jpg",
"skip": true
"skip": true,
"size": 15.58,
"date": "2025 March"
},
"NVLabs Sana 1.5 1.6B 1k Sprint": {
"path": "Efficient-Large-Model/Sana_Sprint_1.6B_1024px_diffusers",
@@ -327,63 +424,83 @@
"path": "Efficient-Large-Model/Sana_1600M_4Kpx_BF16_diffusers",
"desc": "Sana is a text-to-image framework that can efficiently generate images up to 4096 × 4096 resolution. Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU.",
"preview": "Efficient-Large-Model--Sana_1600M_4Kpx_BF16_diffusers.jpg",
"skip": true
"skip": true,
"size": 12.63,
"date": "2024 November"
},
"NVLabs Sana 1.0 1.6B 2k": {
"path": "Efficient-Large-Model/Sana_1600M_2Kpx_BF16_diffusers",
"desc": "Sana is a text-to-image framework that can efficiently generate images up to 4096 × 4096 resolution. Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU.",
"preview": "Efficient-Large-Model--Sana_1600M_2Kpx_BF16_diffusers.jpg",
"skip": true
"skip": true,
"size": 12.63,
"date": "2024 November"
},
"NVLabs Sana 1.0 1.6B 1k": {
"path": "Efficient-Large-Model/Sana_1600M_1024px_diffusers",
"desc": "Sana is a text-to-image framework that can efficiently generate images up to 4096 × 4096 resolution. Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU.",
"preview": "Efficient-Large-Model--Sana_1600M_1024px_diffusers.jpg",
"skip": true
"skip": true,
"size": 12.63,
"date": "2024 November"
},
"NVLabs Sana 1.0 0.6B 0.5k": {
"path": "Efficient-Large-Model/Sana_600M_512px_diffusers",
"desc": "Sana is a text-to-image framework that can efficiently generate images up to 4096 × 4096 resolution. Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU.",
"preview": "Efficient-Large-Model--Sana_600M_512px_diffusers.jpg",
"skip": true
"skip": true,
"size": 7.51,
"date": "2024 November"
},
"nVidia Cosmos-Predict2 T2I 2B": {
"path": "nvidia/Cosmos-Predict2-2B-Text2Image",
"desc": "Cosmos-Predict2: A family of highly performant pre-trained world foundation models purpose-built for generating physics-aware images, videos and world states for physical AI development.",
"preview": "nvidia--Cosmos-Predict2-2B-Text2Image.jpg",
"skip": true
"skip": true,
"size": 13.32,
"date": "2025 June"
},
"nVidia Cosmos-Predict2 T2I 14B": {
"path": "nvidia/Cosmos-Predict2-14B-Text2Image",
"desc": "Cosmos-Predict2: A family of highly performant pre-trained world foundation models purpose-built for generating physics-aware images, videos and world states for physical AI development.",
"preview": "nvidia--Cosmos-Predict2-14B-Text2Image.jpg",
"skip": true
"skip": true,
"size": 37.36,
"date": "2025 June"
},
"VectorSpaceLab OmniGen v1": {
"path": "Shitao/OmniGen-v1-diffusers",
"desc": "OmniGen is a unified image generation model that can generate a wide range of images from multi-modal prompts. It is designed to be simple, flexible and easy to use.",
"preview": "Shitao--OmniGen-v1.jpg",
"skip": true
"skip": true,
"size": 15.47,
"date": "2024 October"
},
"VectorSpaceLab OmniGen v2": {
"path": "OmniGen2/OmniGen2",
"desc": "OmniGen2 is a powerful and efficient unified multimodal model. Unlike OmniGen v1, OmniGen2 features two distinct decoding pathways for text and image modalities, utilizing unshared parameters and a decoupled image tokenizer.",
"preview": "OmniGen2--OmniGen2.jpg",
"skip": true
"skip": true,
"size": 30.5,
"date": "2025 June"
},
"AuraFlow 0.3": {
"path": "fal/AuraFlow-v0.3",
"desc": "AuraFlow v0.3 is the fully open-sourced flow-based text-to-image generation model. The model was trained with more compute compared to the previous version, AuraFlow-v0.2. Compared to AuraFlow-v0.2, the model is fine-tuned on more aesthetic datasets and now supports various aspect ratio, (now width and height up to 1536 pixels).",
"preview": "fal--AuraFlow-v0.3.jpg",
"skip": true
"skip": true,
"size": 31.9,
"date": "2024 August"
},
"AuraFlow 0.2": {
"path": "fal/AuraFlow-v0.2",
"desc": "AuraFlow v0.2 is the fully open-sourced largest flow-based text-to-image generation model. The model was trained with more compute compared to the previous version, AuraFlow-v0.1",
"preview": "fal--AuraFlow-v0.2.jpg",
"skip": true
"skip": true,
"size": 31.9,
"date": "2024 July"
},
"Segmind Vega": {
@@ -392,7 +509,9 @@
"desc": "The Segmind-Vega Model is a distilled version of the Stable Diffusion XL (SDXL), offering a remarkable 70% reduction in size and an impressive 100% speedup while retaining high-quality text-to-image generation capabilities. Trained on diverse datasets, including Grit and Midjourney scrape data, it excels at creating a wide range of visual content based on textual prompts. Employing a knowledge distillation strategy, Segmind-Vega leverages the teachings of several expert models, including SDXL, ZavyChromaXL, and JuggernautXL, to combine their strengths and produce compelling visual outputs.",
"variant": "fp16",
"skip": true,
"extras": "sampler: Default, cfg_scale: 9.0"
"extras": "sampler: Default, cfg_scale: 9.0",
"size": 6.43,
"date": "2023 November"
},
"Segmind SSD-1B": {
"path": "huggingface/segmind/SSD-1B",
@@ -400,13 +519,17 @@
"desc": "The Segmind Stable Diffusion Model (SSD-1B) offers a compact, efficient, and distilled version of the SDXL model. At 50% smaller and 60% faster than Stable Diffusion XL (SDXL), it provides quick and seamless performance without sacrificing image quality.",
"variant": "fp16",
"skip": true,
"extras": "sampler: Default, cfg_scale: 9.0"
"extras": "sampler: Default, cfg_scale: 9.0",
"size": 8.72,
"date": "2023 October"
},
"Segmind Tiny": {
"path": "segmind/tiny-sd",
"preview": "segmind--tiny-sd.jpg",
"desc": "Segmind's Tiny-SD offers a compact, efficient, and distilled version of Realistic Vision 4.0 and is up to 80% faster than SD1.5",
"extras": "width: 512, height: 512, sampler: Default, cfg_scale: 9.0"
"extras": "width: 512, height: 512, sampler: Default, cfg_scale: 9.0",
"size": 1.03,
"date": "2023 July"
},
"Segmind SegMoE SD 4x2": {
"path": "segmind/SegMoE-SD-4x2-v0",
@@ -430,7 +553,9 @@
"path": "PixArt-alpha/PixArt-XL-2-1024-MS",
"desc": "PixArt-α is a Transformer-based T2I diffusion model whose image generation quality is competitive with state-of-the-art image generators (e.g., Imagen, SDXL, and even Midjourney), and the training speed markedly surpasses existing large-scale T2I models. Extensive experiments demonstrate that PIXART-α excels in image quality, artistry, and semantic control. It can directly generate 1024px images from text prompts within a single sampling process.",
"preview": "PixArt-alpha--PixArt-XL-2-1024-MS.jpg",
"extras": "sampler: Default, cfg_scale: 2.0"
"extras": "sampler: Default, cfg_scale: 2.0",
"size": 21.3,
"date": "2023 November"
},
"Pixart-Σ Small": {
"path": "huggingface/PixArt-alpha/PixArt-Sigma-XL-2-512-MS",
@@ -451,14 +576,18 @@
"desc": "PixArt-Σ, a Diffusion Transformer model (DiT) capable of directly generating images at 4K resolution. PixArt-Σ represents a significant advancement over its predecessor, PixArt-α, offering images of markedly higher fidelity and improved alignment with text prompts.",
"preview": "PixArt-alpha--PixArt-Sigma-XL-2-2K-MS.jpg",
"skip": true,
"extras": "sampler: Default, cfg_scale: 2.0"
"extras": "sampler: Default, cfg_scale: 2.0",
"size": 21.3,
"date": "2024 April"
},
"Tencent HunyuanDiT 1.2": {
"path": "Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers",
"desc": "Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding.",
"preview": "Tencent-Hunyuan--HunyuanDiT-v1.2-Diffusers.jpg",
"extras": "sampler: Default, cfg_scale: 2.0"
"extras": "sampler: Default, cfg_scale: 2.0",
"size": 14.09,
"date": "2024 May"
},
"Tencent HunyuanDiT 1.2 Distilled": {
"path": "Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers-Distilled",
@@ -484,14 +613,18 @@
"desc": "The Lumina-Next-SFT is a Next-DiT model containing 2B parameters and utilizes Gemma-2B as the text encoder, enhanced through high-quality supervised fine-tuning (SFT).",
"preview": "Alpha-VLLM--Lumina-Next-SFT-diffusers.jpg",
"skip": true,
"extras": "sampler: Default"
"extras": "sampler: Default",
"size": 8.67,
"date": "2024 June"
},
"AlphaVLLM Lumina 2": {
"path": "Alpha-VLLM/Lumina-Image-2.0",
"desc": "A Unified and Efficient Image Generative Model. Lumina-Image-2.0 is a 2 billion parameter flow-based diffusion transformer capable of generating images from text descriptions.",
"preview": "Alpha-VLLM--Lumina-Image-2.0.jpg",
"skip": true,
"extras": "sampler: Default"
"extras": "sampler: Default",
"size": 20.75,
"date": "2025 January"
},
"HiDream-I1 Fast": {
@@ -499,21 +632,27 @@
"desc": "HiDream-I1 is a new open-source image generative foundation model with 17B parameters that achieves state-of-the-art image generation quality within seconds.",
"preview": "HiDream-ai--HiDream-I1-Fast.jpg",
"skip": true,
"extras": "sampler: Default"
"extras": "sampler: Default",
"size": 58.4,
"date": "2025 April"
},
"HiDream-I1 Dev": {
"path": "HiDream-ai/HiDream-I1-Dev",
"desc": "HiDream-I1 is a new open-source image generative foundation model with 17B parameters that achieves state-of-the-art image generation quality within seconds.",
"preview": "HiDream-ai--HiDream-I1-Dev.jpg",
"skip": true,
"extras": "sampler: Default"
"extras": "sampler: Default",
"size": 58.4,
"date": "2025 April"
},
"HiDream-I1 Full": {
"path": "HiDream-ai/HiDream-I1-Full",
"desc": "HiDream-I1 is a new open-source image generative foundation model with 17B parameters that achieves state-of-the-art image generation quality within seconds.",
"preview": "HiDream-ai--HiDream-I1-Full.jpg",
"skip": true,
"extras": "sampler: Default"
"extras": "sampler: Default",
"size": 58.4,
"date": "2025 April"
},
"HiDream-E1 Full": {
"path": "HiDream-ai/HiDream-E1-Full",
@@ -522,40 +661,58 @@
"skip": true,
"extras": "sampler: Default"
},
"HiDream-E1.1": {
"path": "HiDream-ai/HiDream-E1-1",
"desc": "HiDream-E1 is an image editing model built on HiDream-I1.",
"preview": "HiDream-ai--HiDream-E1-Full.jpg",
"skip": true,
"extras": "sampler: Default",
"experimental": true
},
"Kwai Kolors": {
"path": "Kwai-Kolors/Kolors-diffusers",
"desc": "Kolors is a large-scale text-to-image generation model based on latent diffusion, developed by the Kuaishou Kolors team. Trained on billions of text-image pairs, Kolors exhibits significant advantages over both open-source and proprietary models in visual quality, complex semantic accuracy, and text rendering for both Chinese and English characters. Furthermore, Kolors supports both Chinese and English inputs",
"preview": "Kwai-Kolors--Kolors-diffusers.jpg",
"skip": true,
"extras": "width: 1024, height: 1024"
"extras": "width: 1024, height: 1024",
"size": 17.40,
"date": "2024 July"
},
"Kandinsky 2.1": {
"path": "kandinsky-community/kandinsky-2-1",
"desc": "Kandinsky 2.1 is a text-conditional diffusion model based on unCLIP and latent diffusion, composed of a transformer-based image prior model, a unet diffusion model, and a decoder. Kandinsky 2.1 inherits best practices from Dall-E 2 and Latent diffusion while introducing some new ideas. It uses the CLIP model as a text and image encoder, and diffusion image prior (mapping) between latent spaces of CLIP modalities. This approach increases the visual performance of the model and unveils new horizons in blending images and text-guided image manipulation.",
"preview": "kandinsky-community--kandinsky-2-1.jpg",
"extras": "width: 768, height: 768, sampler: Default"
"extras": "width: 768, height: 768, sampler: Default",
"size": 5.15,
"date": "2023 April"
},
"Kandinsky 2.2": {
"path": "kandinsky-community/kandinsky-2-2-decoder",
"desc": "Kandinsky 2.2 is a text-conditional diffusion model (+0.1!) based on unCLIP and latent diffusion, composed of a transformer-based image prior model, a unet diffusion model, and a decoder. Kandinsky 2.2 inherits best practices from Dall-E 2 and Latent diffusion while introducing some new ideas. It uses the CLIP model as a text and image encoder, and diffusion image prior (mapping) between latent spaces of CLIP modalities. This approach increases the visual performance of the model and unveils new horizons in blending images and text-guided image manipulation.",
"preview": "kandinsky-community--kandinsky-2-2-decoder.jpg",
"extras": "width: 768, height: 768, sampler: Default"
"extras": "width: 768, height: 768, sampler: Default",
"size": 5.15,
"date": "2023 July"
},
"Kandinsky 3": {
"path": "kandinsky-community/kandinsky-3",
"desc": "Kandinsky 3.0 is an open-source text-to-image diffusion model built upon the Kandinsky2-x model family. In comparison to its predecessors, Kandinsky 3.0 incorporates more data and specifically related to Russian culture, which allows to generate pictures related to Russin culture. Furthermore, enhancements have been made to the text understanding and visual quality of the model, achieved by increasing the size of the text encoder and Diffusion U-Net models, respectively.",
"preview": "kandinsky-community--kandinsky-3.jpg",
"variant": "fp16",
"extras": "sampler: Default"
"extras": "sampler: Default",
"size": 27.72,
"date": "2023 November"
},
"Playground v1": {
"path": "playgroundai/playground-v1",
"desc": "Playground v1 is a latent diffusion model that improves the overall HDR quality to get more stunning images.",
"preview": "playgroundai--playground-v1.jpg",
"extras": "width: 512, height: 512, sampler: Default"
"extras": "width: 512, height: 512, sampler: Default",
"size": 4.95,
"date": "2023 December"
},
"Playground v2 Small": {
"path": "playgroundai/playground-v2-256px-base",
@@ -578,22 +735,28 @@
"Playground v2.5": {
"path": "playgroundai/playground-v2.5-1024px-aesthetic",
"desc": "Playground v2.5 is a diffusion-based text-to-image generative model, and a successor to Playground v2. Playground v2.5 is the state-of-the-art open-source model in aesthetic quality.",
"preview": "playgroundai--playground-v2_5-1024px-aesthetic.jpg",
"preview": "playgroundai--playground-v2.5-1024px-aesthetic.jpg",
"variant": "fp16",
"extras": "sampler: DPM++ 2M EDM"
"extras": "sampler: DPM++ 2M EDM",
"size": 13.35,
"date": "2023 December"
},
"CogView 4": {
"path": "zai-org/CogView4-6B",
"desc": "An innovative cascaded framework that enhances the performance of text-to-image diffusion. CogView is the first model implementing relay diffusion in the realm of text-to-image generation, executing the task by first creating low-resolution images and subsequently applying relay-based super-resolution.",
"preview": "THUDM--CogView4-6B.jpg",
"skip": true
"skip": true,
"size": 30.39,
"date": "2025 March"
},
"CogView 3 Plus": {
"path": "zai-org/CogView3-Plus-3B",
"desc": "An innovative cascaded framework that enhances the performance of text-to-image diffusion. CogView is the first model implementing relay diffusion in the realm of text-to-image generation, executing the task by first creating low-resolution images and subsequently applying relay-based super-resolution.",
"preview": "THUDM--CogView3-Plus-3B.jpg",
"skip": true
"skip": true,
"size": 24.96,
"date": "2024 October"
},
"ShuttleAI Shuttle 3.0 Diffusion": {
@@ -619,14 +782,18 @@
"path": "briaai/BRIA-3.2",
"desc": "Bria 3.2 is the next-generation commercial-ready text-to-image model. With just 4 billion parameters, it provides exceptional aesthetics and text rendering, evaluated to provide on par results to leading open-source models, and outperforming other licensed models.",
"preview": "briaai--BRIA-3.2.jpg",
"skip": true
"skip": true,
"size": 18.66,
"date": "2025 June"
},
"Meissonic": {
"path": "MeissonFlow/Meissonic",
"desc": "Meissonic is a non-autoregressive mask image modeling text-to-image synthesis model that can generate high-resolution images. It is designed to run on consumer graphics cards.",
"preview": "MeissonFlow--Meissonic.jpg",
"skip": true
"skip": true,
"size": 3.64,
"date": "2024 October"
},
"aMUSEd 256": {
@@ -647,7 +814,9 @@
"path": "warp-ai/wuerstchen",
"desc": "Würstchen is a diffusion model whose text-conditional model works in a highly compressed latent space of images. Why is this important? Compressing data can reduce computational costs for both training and inference by magnitudes. Training on 1024x1024 images, is way more expensive than training at 32x32. Usually, other works make use of a relatively small compression, in the range of 4x - 8x spatial compression. Würstchen takes this to an extreme. Through its novel design, we achieve a 42x spatial compression. Würstchen employs a two-stage compression, what we call Stage A and Stage B. Stage A is a VQGAN, and Stage B is a Diffusion Autoencoder (more details can be found in the paper). A third model, Stage C, is learned in that highly compressed latent space. This training requires fractions of the compute used for current top-performing models, allowing also cheaper and faster inference.",
"preview": "warp-ai--wuerstchen.jpg",
"extras": "sampler: Default, cfg_scale: 4.0, image_cfg_scale: 0.0"
"extras": "sampler: Default, cfg_scale: 4.0, image_cfg_scale: 0.0",
"size": 12.16,
"date": "2023 August"
},
"KOALA 700M": {
@@ -656,7 +825,9 @@
"skip": true,
"desc": "Fast text-to-image model, called KOALA, by compressing SDXL's U-Net and distilling knowledge from SDXL into our model. KOALA-700M can generate a 1024x1024 image in less than 1.5 seconds on an NVIDIA 4090 GPU, which is more than 2x faster than SDXL.",
"preview": "etri-vilab--koala-700m-llava-cap.jpg",
"extras": "sampler: Default"
"extras": "sampler: Default",
"size": 6.58,
"date": "2024 January"
},
"HDM-XUT 340M Anime": {
@@ -671,13 +842,17 @@
"path": "thu-ml/unidiffuser-v1",
"desc": "UniDiffuser is a unified diffusion framework to fit all distributions relevant to a set of multi-modal data in one transformer. UniDiffuser is able to perform image, text, text-to-image, image-to-text, and image-text pair generation by setting proper timesteps without additional overhead.\nSpecifically, UniDiffuser employs a variation of transformer, called U-ViT, which parameterizes the joint noise prediction network. Other components perform as encoders and decoders of different modalities, including a pretrained image autoencoder from Stable Diffusion, a pretrained image ViT-B/32 CLIP encoder, a pretrained text ViT-L CLIP encoder, and a GPT-2 text decoder finetuned by ourselves.",
"preview": "thu-ml--unidiffuser-v1.jpg",
"extras": "width: 512, height: 512, sampler: Default"
"extras": "width: 512, height: 512, sampler: Default",
"size": 5.37,
"date": "2023 May"
},
"SalesForce BLIP-Diffusion": {
"path": "salesforce/blipdiffusion",
"desc": "BLIP-Diffusion, a new subject-driven image generation model that supports multimodal control which consumes inputs of subject images and text prompts. Unlike other subject-driven generation models, BLIP-Diffusion introduces a new multimodal encoder which is pre-trained to provide subject representation.",
"preview": "salesforce--blipdiffusion.jpg"
"preview": "salesforce--blipdiffusion.jpg",
"size": 7.23,
"date": "2023 July"
},
"InstaFlow 0.9B": {
@@ -690,13 +865,17 @@
"path": "DeepFloyd/IF-I-M-v1.0",
"desc": "DeepFloyd-IF is a pixel-based text-to-image triple-cascaded diffusion model, that can generate pictures with new state-of-the-art for photorealism and language understanding. The result is a highly efficient model that outperforms current state-of-the-art models, achieving a zero-shot FID-30K score of 6.66 on the COCO dataset. It is modular and composed of frozen text mode and three pixel cascaded diffusion modules, each designed to generate images of increasing resolution: 64x64, 256x256, and 1024x1024.",
"preview": "DeepFloyd--IF-I-M-v1.0.jpg",
"extras": "sampler: Default"
"extras": "sampler: Default",
"size": 12.79,
"date": "2023 April"
},
"DeepFloyd IF Large": {
"path": "DeepFloyd/IF-I-L-v1.0",
"desc": "DeepFloyd-IF is a pixel-based text-to-image triple-cascaded diffusion model, that can generate pictures with new state-of-the-art for photorealism and language understanding. The result is a highly efficient model that outperforms current state-of-the-art models, achieving a zero-shot FID-30K score of 6.66 on the COCO dataset. It is modular and composed of frozen text mode and three pixel cascaded diffusion modules, each designed to generate images of increasing resolution: 64x64, 256x256, and 1024x1024.",
"preview": "DeepFloyd--IF-I-L-v1.0.jpg",
"extras": "sampler: Default"
"extras": "sampler: Default",
"size": 15.48,
"date": "2023 April"
}
}
installer.py
+146 -151
View File
@@ -56,6 +56,7 @@ args = Dot({
})
git_commit = "unknown"
diffusers_commit = "unknown"
restart_required = False
extensions_commit = { # force specific commit for extensions
'sd-webui-controlnet': 'ecd33eb',
'adetailer': 'a89c01d'
@@ -345,6 +346,8 @@ def installed(package, friendly: str = None, reload = False, quiet = False): # p
log.warning(f'Install: package="{p[0]}" installed={pkg_version} required={p[1]} allowing experimental')
else:
log.warning(f'Install: package="{p[0]}" installed={pkg_version} required={p[1]} version mismatch')
global restart_required # pylint: disable=global-statement
restart_required = True
ok = ok and (exact or args.experimental)
else:
if not quiet:
@@ -605,7 +608,9 @@ def check_diffusers():
if args.skip_git:
install('diffusers')
return
sha = '5e181eddfe7e44c1444a2511b0d8e21d177850a0' # diffusers commit hash
sha = '64a5187d96f9376c7cf5123db810f2d2da79d7d0' # diffusers commit hash
if args.use_rocm or args.use_zluda or args.use_directml:
sha = '043ab2520f6a19fce78e6e060a68dbc947edb9f9' # lock diffusers versions for now
pkg = pkg_resources.working_set.by_key.get('diffusers', None)
minor = int(pkg.version.split('.')[1] if pkg is not None else -1)
cur = opts.get('diffusers_version', '') if minor > -1 else ''
@@ -626,19 +631,19 @@ def check_transformers():
t_start = time.time()
if args.skip_all or args.skip_git or args.experimental:
return
pkg_transofmers = pkg_resources.working_set.by_key.get('transformers', None)
pkg_transformers = pkg_resources.working_set.by_key.get('transformers', None)
pkg_tokenizers = pkg_resources.working_set.by_key.get('tokenizers', None)
if args.use_directml:
target_transformers = '4.52.4'
target_tokenizers = '0.21.4'
else:
target_transformers = '4.56.1'
target_tokenizers = '0.22.0'
if (pkg_transofmers is None) or ((pkg_transofmers.version != target_transformers) or (pkg_tokenizers is None) or ((pkg_tokenizers.version != target_tokenizers) and (not args.experimental))):
if pkg_transofmers is None:
target_transformers = '4.56.2'
target_tokenizers = '0.22.1'
if (pkg_transformers is None) or ((pkg_transformers.version != target_transformers) or (pkg_tokenizers is None) or ((pkg_tokenizers.version != target_tokenizers) and (not args.experimental))):
if pkg_transformers is None:
log.info(f'Transformers install: version={target_transformers}')
else:
log.info(f'Transformers update: current={pkg_transofmers.version} target={target_transformers}')
log.info(f'Transformers update: current={pkg_transformers.version} target={target_transformers}')
pip('uninstall --yes transformers', ignore=True, quiet=True, uv=False)
pip(f'install --upgrade tokenizers=={target_tokenizers}', ignore=False, quiet=True, uv=False)
pip(f'install --upgrade transformers=={target_transformers}', ignore=False, quiet=True, uv=False)
@@ -674,82 +679,83 @@ def install_rocm_zluda():
if args.skip_all or args.skip_requirements:
return torch_command
from modules import rocm
if not rocm.is_installed:
log.warning('ROCm: could not find ROCm toolkit installed')
log.info('Using CPU-only torch')
return os.environ.get('TORCH_COMMAND', 'torch torchvision')
log.info('ROCm: AMD toolkit detected')
# if not is_windows:
# os.environ.setdefault('TENSORFLOW_PACKAGE', 'tensorflow-rocm')
device = None
amd_gpus = []
try:
amd_gpus = rocm.get_agents()
if len(amd_gpus) == 0:
log.warning('ROCm: no agent was found')
if sys.platform == "win32" and not rocm.is_installed:
amd_gpus = rocm.driver_get_agents()
else:
log.info(f'ROCm: agents={[gpu.name for gpu in amd_gpus]}')
if args.device_id is None:
index = 0
for idx, gpu in enumerate(amd_gpus):
index = idx
# if gpu.name.startswith('gfx11') and os.environ.get('TENSORFLOW_PACKAGE') == 'tensorflow-rocm': # do not use tensorflow-rocm for navi 3x
# os.environ['TENSORFLOW_PACKAGE'] = 'tensorflow==2.13.0'
if not gpu.is_apu:
# although apu was found, there can be a dedicated card. do not break loop.
# if no dedicated card was found, apu will be used.
break
os.environ.setdefault('HIP_VISIBLE_DEVICES', str(index))
device = amd_gpus[index]
else:
device_id = int(args.device_id)
if device_id < len(amd_gpus):
device = amd_gpus[device_id]
amd_gpus = rocm.get_agents()
log.info('ROCm: AMD toolkit detected')
except Exception as e:
log.warning(f'ROCm agent enumerator failed: {e}')
#os.environ.setdefault('TENSORFLOW_PACKAGE', 'tensorflow')
device = None
if len(amd_gpus) == 0:
log.warning('ROCm: no agent was found')
else:
log.info(f'ROCm: agents={[gpu.name for gpu in amd_gpus]}')
if args.device_id is None:
index = 0
for idx, gpu in enumerate(amd_gpus):
index = idx
if not gpu.is_apu:
# although apu was found, there can be a dedicated card. do not break loop.
# if no dedicated card was found, apu will be used.
break
os.environ.setdefault('HIP_VISIBLE_DEVICES', str(index))
device = amd_gpus[index]
else:
device_id = int(args.device_id)
if device_id < len(amd_gpus):
device = amd_gpus[device_id]
if sys.platform == "win32" and args.use_rocm and not rocm.is_installed:
check_python(supported_minors=[11, 12, 13], reason='ROCm backend requires a Python version between 3.11 and 3.13')
install(f"rocm rocm-sdk-core --index-url https://rocm.nightlies.amd.com/v2-staging/{device.therock}")
rocm.refresh()
msg = f'ROCm: version={rocm.version}'
if device is not None:
msg += f', using agent {device.name}'
log.info(msg)
if sys.platform == "win32": # TODO install: enable ROCm for windows when available
#check_python(supported_minors=[10, 11, 12, 13], reason='ZLUDA backend requires a Python version between 3.10 and 3.13')
if sys.platform == "win32":
if args.use_rocm: # TODO install: switch to pytorch source when it becomes available
if device is not None and isinstance(rocm.environment, rocm.PythonPackageEnvironment): # TheRock
check_python(supported_minors=[11, 12, 13], reason='ROCm backend requires a Python version between 3.11 and 3.13')
torch_command = os.environ.get('TORCH_COMMAND', f'torch torchvision --index-url https://rocm.nightlies.amd.com/v2-staging/{device.therock}')
else:
check_python(supported_minors=[12], reason='ROCm Windows preview requires Python version 3.12')
torch_command = os.environ.get('TORCH_COMMAND', '--no-cache-dir https://repo.radeon.com/rocm/windows/rocm-rel-6.4.4/torch-2.8.0a0%2Bgitfc14c65-cp312-cp312-win_amd64.whl https://repo.radeon.com/rocm/windows/rocm-rel-6.4.4/torchvision-0.24.0a0%2Bc85f008-cp312-cp312-win_amd64.whl')
else:
#check_python(supported_minors=[10, 11, 12, 13], reason='ZLUDA backend requires a Python version between 3.10 and 3.13')
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.7.1+cu118 torchvision==0.22.1+cu118 --index-url https://download.pytorch.org/whl/cu118')
if args.device_id is not None:
if os.environ.get('HIP_VISIBLE_DEVICES', None) is not None:
log.warning('Setting HIP_VISIBLE_DEVICES and --device-id at the same time may be mistake.')
os.environ['HIP_VISIBLE_DEVICES'] = args.device_id
del args.device_id
if args.device_id is not None:
if os.environ.get('HIP_VISIBLE_DEVICES', None) is not None:
log.warning('Setting HIP_VISIBLE_DEVICES and --device-id at the same time may be mistake.')
os.environ['HIP_VISIBLE_DEVICES'] = args.device_id
del args.device_id
error = None
from modules import zluda_installer
try:
if args.reinstall or zluda_installer.is_reinstall_needed():
zluda_installer.uninstall()
zluda_installer.install()
zluda_installer.set_default_agent(device)
except Exception as e:
error = e
log.warning(f'Failed to install ZLUDA: {e}')
from modules import zluda_installer
try:
if args.reinstall or zluda_installer.is_reinstall_needed():
zluda_installer.uninstall()
zluda_installer.install()
zluda_installer.set_default_agent(device)
except Exception as e:
log.warning(f'Failed to install ZLUDA: {e}')
if error is None:
try:
zluda_installer.load()
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.7.1+cu118 torchvision==0.22.1+cu118 --index-url https://download.pytorch.org/whl/cu118')
except Exception as e:
error = e
log.warning(f'Failed to load ZLUDA: {e}')
if error is not None:
log.info('Using CPU-only torch')
torch_command = os.environ.get('TORCH_COMMAND', 'torch torchvision')
else:
#check_python(supported_minors=[10, 11, 12, 13], reason='ROCm backend requires a Python version between 3.10 and 3.13')
if os.environ.get("TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL", None) is None:
os.environ.setdefault('TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL', '1')
if args.use_nightly:
if rocm.version is None or float(rocm.version) >= 6.4: # assume the latest if version check fails
torch_command = os.environ.get('TORCH_COMMAND', '--upgrade --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/rocm6.4')
@@ -779,7 +785,7 @@ def install_rocm_zluda():
log.info(f'ROCm: HSA_OVERRIDE_GFX_VERSION auto config skipped: device={device.name if device is not None else None} version={os.environ.get("HSA_OVERRIDE_GFX_VERSION", None)}')
else:
gfx_ver = device.get_gfx_version()
if gfx_ver is not None:
if gfx_ver is not None and device.name.removeprefix("gfx") != gfx_ver.replace(".", ""):
os.environ.setdefault('HSA_OVERRIDE_GFX_VERSION', gfx_ver)
log.info(f'ROCm: HSA_OVERRIDE_GFX_VERSION config overridden: device={device.name} version={os.environ.get("HSA_OVERRIDE_GFX_VERSION", None)}')
@@ -793,27 +799,6 @@ def install_ipex():
args.use_ipex = True # pylint: disable=attribute-defined-outside-init
log.info('IPEX: Intel OneAPI toolkit detected')
if os.environ.get("NEOReadDebugKeys", None) is None:
os.environ.setdefault('NEOReadDebugKeys', '1')
if os.environ.get("ClDeviceGlobalMemSizeAvailablePercent", None) is None:
os.environ.setdefault('ClDeviceGlobalMemSizeAvailablePercent', '100')
if os.environ.get("SYCL_CACHE_PERSISTENT", None) is None:
os.environ.setdefault('SYCL_CACHE_PERSISTENT', '1') # Jit cache
if os.environ.get("PYTORCH_ENABLE_XPU_FALLBACK", None) is None:
os.environ.setdefault('PYTORCH_ENABLE_XPU_FALLBACK', '1') # CPU fallback for unsupported ops
if os.environ.get("UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS", None) is None:
os.environ.setdefault('UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS', '1') # Work around the 4G alloc limit on Alchemist
# FP64 emulation causes random UR Errors
#if os.environ.get("OverrideDefaultFP64Settings", None) is None:
# os.environ.setdefault('OverrideDefaultFP64Settings', '1')
#if os.environ.get("IGC_EnableDPEmulation", None) is None:
# os.environ.setdefault('IGC_EnableDPEmulation', '1') # FP64 Emulation
if args.use_nightly:
torch_command = os.environ.get('TORCH_COMMAND', '--upgrade --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/xpu')
else:
@@ -827,20 +812,17 @@ def install_ipex():
def install_openvino():
t_start = time.time()
log.info('OpenVINO: selected')
#check_python(supported_minors=[10, 11, 12, 13], reason='OpenVINO backend requires a Python version between 3.10 and 3.13')
os.environ.setdefault('PYTORCH_TRACING_MODE', 'TORCHFX')
#check_python(supported_minors=[10, 11, 12, 13], reason='OpenVINO backend requires a Python version between 3.10 and 3.13')
if sys.platform == 'darwin':
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.8.0 torchvision==0.23.0')
else:
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.8.0+cpu torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cpu')
install(os.environ.get('OPENVINO_COMMAND', 'openvino==2025.3.0'), 'openvino')
install(os.environ.get('NNCF_COMMAND', 'nncf==2.18.0'), 'nncf')
os.environ.setdefault('PYTORCH_TRACING_MODE', 'TORCHFX')
if os.environ.get("NEOReadDebugKeys", None) is None:
os.environ.setdefault('NEOReadDebugKeys', '1')
if os.environ.get("ClDeviceGlobalMemSizeAvailablePercent", None) is None:
os.environ.setdefault('ClDeviceGlobalMemSizeAvailablePercent', '100')
if not (args.skip_all or args.skip_requirements):
install(os.environ.get('OPENVINO_COMMAND', 'openvino==2025.3.0'), 'openvino')
install(os.environ.get('NNCF_COMMAND', 'nncf==2.18.0'), 'nncf')
ts('openvino', t_start)
return torch_command
@@ -874,6 +856,7 @@ def install_torch_addons():
install('pillow-jxl-plugin==1.3.4', 'pillow-jxl-plugin')
if not args.experimental:
uninstall('wandb', quiet=True)
uninstall('pynvml', quiet=True)
ts('addons', t_start)
@@ -921,9 +904,9 @@ def check_torch():
if torch_command != '':
pass
else:
is_cuda_available = allow_cuda and (shutil.which('nvidia-smi') is not None or args.use_xformers or os.path.exists(os.path.join(os.environ.get('SystemRoot') or r'C:\Windows', 'System32', 'nvidia-smi.exe')))
is_rocm_available = allow_rocm and rocm.is_installed
is_ipex_available = allow_ipex and (args.use_ipex or shutil.which('sycl-ls') is not None or shutil.which('sycl-ls.exe') is not None or os.environ.get('ONEAPI_ROOT') is not None or os.path.exists('/opt/intel/oneapi') or os.path.exists("C:/Program Files (x86)/Intel/oneAPI") or os.path.exists("C:/oneAPI"))
is_cuda_available = allow_cuda and (args.use_cuda or shutil.which('nvidia-smi') is not None or args.use_xformers or os.path.exists(os.path.join(os.environ.get('SystemRoot') or r'C:\Windows', 'System32', 'nvidia-smi.exe')))
is_rocm_available = allow_rocm and (args.use_rocm or args.use_zluda or rocm.is_installed)
is_ipex_available = allow_ipex and (args.use_ipex or shutil.which('sycl-ls') is not None or shutil.which('sycl-ls.exe') is not None or os.environ.get('ONEAPI_ROOT') is not None or os.path.exists('/opt/intel/oneapi') or os.path.exists("C:/Program Files (x86)/Intel/oneAPI") or os.path.exists("C:/oneAPI") or os.path.exists("C:/Program Files/Intel/Intel Graphics Software"))
if is_cuda_available and args.use_cuda: # prioritize cuda
torch_command = install_cuda()
@@ -950,63 +933,61 @@ def check_torch():
install(torch_command, 'torch torchvision')
install('onnxruntime-directml', 'onnxruntime-directml', ignore=True)
else:
if args.use_zluda:
log.warning("ZLUDA failed to initialize: no HIP SDK found")
log.warning('Torch: CPU-only version installed')
torch_command = os.environ.get('TORCH_COMMAND', 'torch torchvision')
if 'torch' in torch_command and not args.version:
if args.version:
return
if 'torch' in torch_command:
if not installed('torch'):
log.info(f'Torch: download and install in progress... cmd="{torch_command}"')
install('--upgrade pip', 'pip', reinstall=True) # pytorch rocm is too large for older pip
install(torch_command, 'torch torchvision', quiet=True)
else:
try:
import torch
try:
import torch
log.info(f'Torch {torch.__version__}')
if args.use_ipex and allow_ipex:
try:
import intel_extension_for_pytorch as ipex # pylint: disable=import-error, unused-import
log.info(f'Torch backend: Intel IPEX {ipex.__version__}')
except Exception:
log.warning('IPEX: not found')
if shutil.which('icpx') is not None:
log.info(f'{os.popen("icpx --version").read().rstrip()}')
for device in range(torch.xpu.device_count()):
log.info(f'Torch detected GPU: {torch.xpu.get_device_name(device)} VRAM {round(torch.xpu.get_device_properties(device).total_memory / 1024 / 1024)} Compute Units {torch.xpu.get_device_properties(device).max_compute_units}')
elif torch.cuda.is_available() and (allow_cuda or allow_rocm):
# log.debug(f'Torch allocator: {torch.cuda.get_allocator_backend()}')
if torch.version.cuda and allow_cuda:
log.info(f'Torch backend: nVidia CUDA {torch.version.cuda} cuDNN {torch.backends.cudnn.version() if torch.backends.cudnn.is_available() else "N/A"}')
elif torch.version.hip and allow_rocm:
log.info(f'Torch backend: AMD ROCm HIP {torch.version.hip}')
else:
log.warning('Unknown Torch backend')
for device in [torch.cuda.device(i) for i in range(torch.cuda.device_count())]:
log.info(f'Torch detected GPU: {torch.cuda.get_device_name(device)} VRAM {round(torch.cuda.get_device_properties(device).total_memory / 1024 / 1024)} Arch {torch.cuda.get_device_capability(device)} Cores {torch.cuda.get_device_properties(device).multi_processor_count}')
import intel_extension_for_pytorch as ipex # pylint: disable=import-error, unused-import
log.info(f'Torch backend: type=IPEX version={ipex.__version__}')
except Exception:
pass
if 'cpu' in torch.__version__:
if is_cuda_available:
log.warning(f'Torch: version="{torch.__version__}" CPU version installed and CUDA is available - consider reinstalling')
elif is_rocm_available:
log.warning(f'Torch: version="{torch.__version__}" CPU version installed and ROCm is available - consider reinstalling')
if hasattr(torch, "xpu") and torch.xpu.is_available() and allow_ipex:
if shutil.which('icpx') is not None:
log.info(f'{os.popen("icpx --version").read().rstrip()}')
for device in range(torch.xpu.device_count()):
log.info(f'Torch detected: gpu="{torch.xpu.get_device_name(device)}" vram={round(torch.xpu.get_device_properties(device).total_memory / 1024 / 1024)} units={torch.xpu.get_device_properties(device).max_compute_units}')
elif torch.cuda.is_available() and (allow_cuda or allow_rocm):
if torch.version.cuda and allow_cuda:
log.info(f'Torch backend: version="{torch.__version__}" type=CUDA CUDA={torch.version.cuda} cuDNN={torch.backends.cudnn.version() if torch.backends.cudnn.is_available() else "N/A"}')
elif torch.version.hip and allow_rocm:
log.info(f'Torch backend: version="{torch.__version__}" type=ROCm HIP={torch.version.hip}')
else:
try:
if args.use_directml and allow_directml:
import torch_directml # pylint: disable=import-error
dml_ver = pkg_resources.get_distribution("torch-directml")
log.info(f'Torch backend: DirectML ({dml_ver})')
for i in range(0, torch_directml.device_count()):
log.info(f'Torch detected GPU: {torch_directml.device_name(i)}')
except Exception:
log.warning("Torch reports CUDA not available")
except Exception as e:
log.error(f'Torch cannot load: {e}')
if not args.ignore:
sys.exit(1)
if rocm.is_installed:
if sys.platform == "win32": # CPU, DirectML, ZLUDA
rocm.conceal()
elif rocm.is_wsl: # WSL ROCm
log.warning('Unknown Torch backend')
for device in [torch.cuda.device(i) for i in range(torch.cuda.device_count())]:
log.info(f'Torch detected: gpu="{torch.cuda.get_device_name(device)}" vram={round(torch.cuda.get_device_properties(device).total_memory / 1024 / 1024)} arch={torch.cuda.get_device_capability(device)} cores={torch.cuda.get_device_properties(device).multi_processor_count}')
else:
try:
rocm.load_hsa_runtime()
except OSError:
log.error("ROCm: failed to preload HSA runtime")
if args.version:
return
if args.use_directml and allow_directml:
import torch_directml # pylint: disable=import-error
dml_ver = pkg_resources.get_distribution("torch-directml")
log.warning(f'Torch backend: DirectML ({dml_ver})')
log.warning('DirectML: end-of-life')
for i in range(0, torch_directml.device_count()):
log.info(f'Torch detected GPU: {torch_directml.device_name(i)}')
except Exception:
log.warning("Torch reports CUDA not available")
except Exception as e:
log.error(f'Torch cannot load: {e}')
if not args.ignore:
sys.exit(1)
if rocm.is_installed:
rocm.postinstall()
if not args.skip_all:
install_torch_addons()
check_cudnn()
@@ -1264,6 +1245,17 @@ def install_pydantic():
reload('pydantic', '1.10.21')
def install_insightface():
install('git+https://github.com/deepinsight/insightface@29b6cd65aa0e9ae3b6602de3c52e9d8949c8ee86#subdirectory=python-package', 'insightface') # insightface==0.7.3 with patches
if args.new:
uninstall('albumentations')
install('albumentationsx')
else:
uninstall('albumentationsx')
install('albumentations==1.4.3', ignore=True, quiet=True)
install_pydantic()
def install_optional():
t_start = time.time()
log.info('Installing optional requirements...')
@@ -1277,8 +1269,6 @@ def install_optional():
install('nvidia-ml-py', ignore=True, quiet=True)
install('ultralytics==8.3.40', ignore=True, quiet=True)
install('Cython', ignore=True, quiet=True)
install('git+https://github.com/deepinsight/insightface@554a05561cb71cfebb4e012dfea48807f845a0c2#subdirectory=python-package', 'insightface') # insightface==0.7.3 with patches
install('albumentations==1.4.3', ignore=True, quiet=True)
install('av', ignore=True, quiet=True)
install('gguf', ignore=True)
try:
@@ -1358,8 +1348,14 @@ def set_environment():
os.environ.setdefault('PYTORCH_CUDA_ALLOC_CONF', allocator)
os.environ.setdefault('PYTORCH_HIP_ALLOC_CONF', allocator)
log.debug(f'Torch allocator: "{allocator}"')
if sys.platform == 'darwin':
os.environ.setdefault('PYTORCH_ENABLE_MPS_FALLBACK', '1')
os.environ.setdefault('TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL', '1')
os.environ.setdefault('NEOReadDebugKeys', '1')
os.environ.setdefault('ClDeviceGlobalMemSizeAvailablePercent', '100')
os.environ.setdefault('SYCL_CACHE_PERSISTENT', '1')
os.environ.setdefault('UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS', '1')
os.environ.setdefault('PYTORCH_ENABLE_XPU_FALLBACK', '1')
os.environ.setdefault('PYTORCH_ENABLE_MPS_FALLBACK', '1')
os.environ.setdefault('TOKENIZERS_PARALLELISM', '0')
def check_extensions():
@@ -1634,7 +1630,6 @@ def add_args(parser):
group_http.add_argument("--cors-regex", type=str, default=os.environ.get("SD_CORSREGEX", None), help="Allowed CORS origins as regular expression, default: %(default)s")
group_http.add_argument('--subpath', type=str, default=os.environ.get("SD_SUBPATH", None), help='Customize the URL subpath for usage with reverse proxy')
group_http.add_argument("--autolaunch", default=os.environ.get("SD_AUTOLAUNCH", False), action='store_true', help="Open the UI URL in the system's default browser upon launch")
group_http.add_argument('--docs', default=os.environ.get("SD_DOCS", False), action='store_true', help = "Mount API docs, default: %(default)s")
group_http.add_argument("--auth", type=str, default=os.environ.get("SD_AUTH", None), help='Set access authentication like "user:pwd,user:pwd""')
group_http.add_argument("--auth-file", type=str, default=os.environ.get("SD_AUTHFILE", None), help='Set access authentication using file, default: %(default)s')
group_http.add_argument("--allowed-paths", nargs='+', default=[], type=str, required=False, help="add additional paths to paths allowed for web access")
@@ -1655,11 +1650,11 @@ def add_args(parser):
group_log = parser.add_argument_group('Logging')
group_log.add_argument("--log", type=str, default=os.environ.get("SD_LOG", None), help="Set log file, default: %(default)s")
group_log.add_argument('--debug', default=os.environ.get("SD_DEBUG",False), action='store_true', help="Run with debug logging, default: %(default)s")
group_log.add_argument('--debug', default=not os.environ.get("SD_NODEBUG",False), action='store_true', help="Run with debug logging, default: %(default)s")
group_log.add_argument("--trace", default=os.environ.get("SD_TRACE", False), action='store_true', help="Run with trace logging, default: %(default)s")
group_log.add_argument("--profile", default=os.environ.get("SD_PROFILE", False), action='store_true', help="Run profiler, default: %(default)s")
group_log.add_argument('--docs', default=os.environ.get("SD_DOCS", False), action='store_true', help="Mount API docs, default: %(default)s")
group_log.add_argument("--api-log", default=os.environ.get("SD_APILOG", False), action='store_true', help="Log all API requests")
group_log.add_argument('--docs', default=not os.environ.get("SD_NODOCS", False), action='store_true', help = "Mount API docs, default: %(default)s")
group_log.add_argument("--api-log", default=not os.environ.get("SD_NOAPILOG", False), action='store_true', help="Log all API requests")
group_nargs = parser.add_argument_group('Other')
group_nargs.add_argument('args', type=str, nargs='*', help=argparse.SUPPRESS)
javascript/extraNetworks.js
+3 -2
View File
@@ -293,10 +293,11 @@ function extraNetworksSearchButton(event) {
}
function extraNetworksFilterVersion(event) {
// log('extraNetworksFilterVersion', event);
log('extraNetworksFilterVersion', event);
const version = event.target.textContent.trim();
const activeTab = getENActiveTab();
const cardContainer = gradioApp().querySelector(`#${activeTab}_model_cards`);
const activePage = getENActivePage().toLowerCase();
const cardContainer = gradioApp().querySelector(`#${activeTab}_${activePage}_cards`);
if (!cardContainer) return;
if (cardContainer.dataset.activeVersion === version) {
cardContainer.dataset.activeVersion = '';
javascript/guidance.js
+20
View File
@@ -0,0 +1,20 @@
const guiders = {
None: '',
'LSC: LayerSkipConfig': 'https://github.com/huggingface/diffusers/blob/041501aea92919c9c7f36e189fc9cf7d865ebb96/src/diffusers/hooks/layer_skip.py#L41',
'CFG: ClassifierFreeGuidance': 'https://huggingface.co/docs/diffusers/v0.35.1/en/api/modular_diffusers/guiders#diffusers.ClassifierFreeGuidance',
'Auto: AutoGuidance': 'https://huggingface.co/docs/diffusers/v0.35.1/en/api/modular_diffusers/guiders#diffusers.AutoGuidance',
'Zero: ClassifierFreeZeroStar': 'https://huggingface.co/docs/diffusers/v0.35.1/en/api/modular_diffusers/guiders#diffusers.ClassifierFreeZeroStarGuidance',
'PAG: PerturbedAttentionGuidance': 'https://huggingface.co/docs/diffusers/v0.35.1/en/api/modular_diffusers/guiders#diffusers.PerturbedAttentionGuidance',
'APG: AdaptiveProjectedGuidance': 'https://huggingface.co/docs/diffusers/v0.35.1/en/api/modular_diffusers/guiders#diffusers.AdaptiveProjectedGuidance',
'SLG: SkipLayerGuidance': 'https://huggingface.co/docs/diffusers/v0.35.1/en/api/modular_diffusers/guiders#diffusers.SkipLayerGuidance',
'SEG: SmoothedEnergyGuidance': 'https://huggingface.co/docs/diffusers/v0.35.1/en/api/modular_diffusers/guiders#diffusers.SmoothedEnergyGuidance',
'TCFG: TangentialClassifierFreeGuidance': 'https://huggingface.co/docs/diffusers/v0.35.1/en/api/modular_diffusers/guiders#diffusers.TangentialClassifierFreeGuidance',
'FDG: FrequencyDecoupledGuidance': 'https://huggingface.co/docs/diffusers/v0.35.1/en/api/modular_diffusers/guiders#diffusers.FrequencyDecoupledGuidance',
};
function getGuidanceDocs(guider) {
if (guider.label) guider = guider.label;
const url = guiders[guider];
log('getGuidanceDocs', guider, url);
if (url) window.open(url, '_blank');
}
javascript/logMonitor.js
+1
View File
@@ -78,6 +78,7 @@ async function logMonitor() {
for (const line of lines) addLogLine(line);
if (!logConnected) {
logConnected = true;
monitorConnection();
xhrPost(`${window.api}/log`, { debug: 'connected' });
}
} else {
javascript/monitor.js
+35
View File
@@ -0,0 +1,35 @@
async function updateIndicator(online, data, msg) {
const el = document.getElementById('logo_nav');
if (!el || !data) return;
const status = online ? '<b style="color:lime">online</b>' : '<b style="color:darkred">offline</b>';
const date = new Date();
const template = `
Version: <b>${data.updated}</b><br>
Commit: <b>${data.hash}</b><br>
Branch: <b>${data.branch}</b><br>
Status: ${status}<br>
Since: ${date.toLocaleString()}<br>
`;
if (online) {
el.dataset.hint = template;
el.style.backgroundColor = 'var(--sd-main-accent-color)';
log('monitorConnection: online', data);
} else {
el.dataset.hint = template;
el.style.backgroundColor = 'var(--color-error)';
log('monitorConnection: offline', msg);
}
}
async function monitorConnection() {
try {
const res = await fetch(`${window.api}/version`);
const data = await res.json();
const url = res.url.split('/sdapi')[0].replace('http', 'ws'); // update global url as ws need fqdn
const ws = new WebSocket(`${url}/queue/join`);
ws.onopen = () => updateIndicator(true, data, '');
ws.onclose = () => updateIndicator(false, data, '');
ws.onerror = (e) => updateIndicator(false, data, e.message);
ws.onmessage = (evt) => log('monitorConnection: message', evt.data);
} catch { /**/ }
}
javascript/ui.js
+1
View File
@@ -601,4 +601,5 @@ async function reconnectUI() {
const sd_model_observer = new MutationObserver(sd_model_callback);
sd_model_observer.observe(sd_model, { attributes: true, childList: true, subtree: true });
log('reconnectUI');
monitorConnection();
}
launch.py
+3
View File
@@ -297,6 +297,9 @@ def main():
installer.log.info(f'Installer time: {init_summary()}')
get_custom_args()
if installer.restart_required:
installer.log.warning('Restart is recommended due to packages updates...')
uv, instance = start_server(immediate=True, server=None)
t_server = time.time()
t_monitor = time.time()
models/Reference/Qwen--Qwen-Image-Edit-2509.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 85 KiB

models/Reference/Qwen-Image-Distill.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 75 KiB

models/Reference/briaai--BRIA-3.2.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 31 KiB

After

Width:  |  Height:  |  Size: 48 KiB

models/Reference/fal--AuraFlow-v0.3.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 87 KiB

After

Width:  |  Height:  |  Size: 102 KiB

models/Reference/juggernautXL_v9Rundiffusionphoto2.jpg → models/Reference/juggernautXL_juggXIByRundiffusion.jpg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 68 KiB

After

Width:  |  Height:  |  Size: 68 KiB

models/Reference/juggernautXL_v9Rdphoto2Lightning.jpg → models/Reference/juggernautXL_juggXILightningByRD.jpg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 69 KiB

models/Reference/kandinsky-community--kandinsky-2-1.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 82 KiB

After

Width:  |  Height:  |  Size: 131 KiB

models/Reference/kandinsky-community--kandinsky-2-2-decoder.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 63 KiB

After

Width:  |  Height:  |  Size: 39 KiB

models/Reference/kandinsky-community--kandinsky-3.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 53 KiB

After

Width:  |  Height:  |  Size: 67 KiB

models/Reference/linoyts--Wan2.2-VACE-Fun-14B-diffusers.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 80 KiB

models/Reference/playgroundai--playground-v2_5-1024px-aesthetic.jpg → models/Reference/playgroundai--playground-v2.5-1024px-aesthetic.jpg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 73 KiB

models/Reference/segmind--SSD-1B.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 49 KiB

After

Width:  |  Height:  |  Size: 84 KiB

models/Reference/segmind--SegMoE-4x2-v0.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 34 KiB

After

Width:  |  Height:  |  Size: 59 KiB

models/Reference/segmind--SegMoE-SD-4x2-v0.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 51 KiB

models/Reference/segmind--Segmind-Vega.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 76 KiB

models/Reference/segmind--tiny-sd.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 88 KiB

After

Width:  |  Height:  |  Size: 80 KiB

models/Reference/stabilityai--sdxl-turbo.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

models/Reference/sd_xl_base_1.0.jpg → models/Reference/stabilityai--stable-diffusion-xl-base-1.0.jpg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 36 KiB

After

Width:  |  Height:  |  Size: 36 KiB

models/Reference/tempest-by-vlad-base.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 34 KiB

models/Reference/tempest-by-vlad-hyper.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 26 KiB

models/Reference/tempestByVlad_baseV01.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 74 KiB

models/Reference/tempestByVlad_hyperV01.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 83 KiB

models/Reference/vladmandic--flux.1-dev-SRPO.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 53 KiB

modules/cachedit.py
+64
View File
@@ -0,0 +1,64 @@
import os
from installer import install
from modules import shared
def apply_cache_dit(pipe):
if not shared.opts.cache_dit_enabled:
return
install('git+https://github.com/vipshop/cache-dit', 'cache_dit')
os.environ.setdefault("CACHE_DIT_LOG_LEVEL", "error")
try:
import cache_dit
except Exception as e:
shared.log.error(f'Cache-DIT: {e}')
return
_, supported = cache_dit.supported_pipelines()
supported = [s.replace('*', '') for s in supported]
if not any(pipe.__class__.__name__.startswith(s) for s in supported):
shared.log.error(f'Cache-DiT: pipeline={pipe.__class__.__name__} unsupported')
return
if getattr(pipe, 'has_cache_dit', False):
unapply_cache_dir(pipe)
config_args = {}
if shared.opts.cache_dit_fcompute >= 0:
config_args['Fn_compute_blocks'] = int(shared.opts.cache_dit_fcompute)
if shared.opts.cache_dit_bcompute >= 0:
config_args['Bn_compute_blocks'] = int(shared.opts.cache_dit_bcompute)
if shared.opts.cache_dit_threshold >= 0:
config_args['residual_diff_threshold'] = float(shared.opts.cache_dit_threshold)
if shared.opts.cache_dit_warmup >= 0:
config_args['max_warmup_steps'] = int(shared.opts.cache_dit_warmup)
cache_config = cache_dit.BasicCacheConfig(**config_args)
if shared.opts.cache_dit_calibrator == "TaylorSeer":
calibrator_config = cache_dit.TaylorSeerCalibratorConfig(taylorseer_order=1)
elif shared.opts.cache_dit_calibrator == "FoCa":
calibrator_config = cache_dit.FoCaCalibratorConfig()
else:
calibrator_config = None
shared.log.info(f'Apply Cache-DiT: config="{cache_config.strify()}" calibrator="{calibrator_config.strify() if calibrator_config else "None"}"')
try:
cache_dit.enable_cache(
pipe,
cache_config=cache_config,
calibrator_config=calibrator_config,
)
shared.sd_model.has_cache_dit = True
except Exception as e:
shared.log.error(f'Cache-DiT: {e}')
return
def unapply_cache_dir(pipe):
if not shared.opts.cache_dit_enabled or not getattr(pipe, 'has_cache_dit', False):
return
try:
import cache_dit
# stats = cache_dit.summary(pipe)
# shared.log.critical(f'Unapply Cache-DiT: {stats}')
cache_dit.disable_cache(pipe)
pipe.has_cache_dit = False
except Exception:
return
modules/civitai/api_civitai.py
+9
View File
@@ -54,6 +54,15 @@ def get_civitai(
return JSONResponse(content=[], status_code=200)
def post_civitai(page:str=None):
from modules.civitai import metadata_civitai
result = []
for r in metadata_civitai.civit_search_metadata(title=page, raw=True):
result = r # get the last yielded result
return result
def register_api():
from modules.shared import api
api.add_api_route("/sdapi/v1/civitai", get_civitai, methods=["GET"], response_model=list)
api.add_api_route("/sdapi/v1/civitai", post_civitai, methods=["POST"], response_model=list)
modules/civitai/download_civitai.py
+5 -4
View File
@@ -103,10 +103,11 @@ def download_civit_model_thread(model_name: str, model_url: str, model_path: str
if os.path.isfile(temp_file):
starting_pos = os.path.getsize(temp_file)
headers['Range'] = f'bytes={starting_pos}-'
if token is None or len(token) == 0:
token = shared.opts.civitai_token
if token is not None and len(token) > 0:
headers['Authorization'] = f'Bearer {token}'
if 'civit' in model_url.lower(): # downloader can be used for other urls too
if token is None or len(token) == 0:
token = shared.opts.civitai_token
if (token is not None) and (len(token) > 0):
headers['Authorization'] = f'Bearer {token}'
r = shared.req(model_url, headers=headers, stream=True)
total_size = int(r.headers.get('content-length', 0))
modules/civitai/metadata_civitai.py
+7 -7
View File
@@ -26,7 +26,7 @@ class CivitModel:
self.status = 'Not found'
def civit_update_metadata():
def civit_update_metadata(raw:bool=False):
def create_update_metadata_table(rows: list[CivitModel]):
html = """
<table class="simple-table">
@@ -102,8 +102,8 @@ def civit_update_metadata():
model.status = 'Update available'
break
results.append(model)
yield create_update_metadata_table(results)
return create_update_metadata_table(results)
yield results if raw else create_update_metadata_table(results)
yield results if raw else create_update_metadata_table(results)
def civit_search_model(name, tag, model_type):
@@ -224,7 +224,7 @@ def atomic_civit_search_metadata(item, results):
results.append(result)
def civit_search_metadata(title: str = None):
def civit_search_metadata(title: str = None, raw: bool = False):
def create_search_metadata_table(rows):
html = """
<table class="simple-table">
@@ -258,7 +258,7 @@ def civit_search_metadata(title: str = None):
re_skip = [r.strip() for r in opts.extra_networks_scan_skip.split(',') if len(r.strip()) > 0]
for page in get_pages():
if type(title) == str:
if page.title != title:
if page.title.lower() != title.lower():
continue
if page.name == 'style' or page.name == 'wildcards':
continue
@@ -278,8 +278,8 @@ def civit_search_metadata(title: str = None):
future_items[executor.submit(atomic_civit_search_metadata, fn, results)] = fn
for future in concurrent.futures.as_completed(future_items):
future.result()
yield create_search_metadata_table(results)
yield results if raw else create_search_metadata_table(results)
t1 = time.time()
log.debug(f'CivitAI search metadata: scanned={scanned} skipped={skipped} time={t1-t0:.2f}')
yield create_search_metadata_table(results)
yield results if raw else create_search_metadata_table(results)
modules/control/run.py
+11 -3
View File
@@ -255,6 +255,7 @@ def control_run(state: str = '', # pylint: disable=keyword-arg-before-vararg
prompt: str = '', negative_prompt: str = '', styles: List[str] = [],
steps: int = 20, sampler_index: int = None,
seed: int = -1, subseed: int = -1, subseed_strength: float = 0, seed_resize_from_h: int = -1, seed_resize_from_w: int = -1,
guidance_name: str = 'Default', guidance_scale: float = 6.0, guidance_rescale: float = 0.0, guidance_start: float = 0.0, guidance_stop: float = 1.0,
cfg_scale: float = 6.0, clip_skip: float = 1.0, image_cfg_scale: float = 6.0, diffusers_guidance_rescale: float = 0.7, pag_scale: float = 0.0, pag_adaptive: float = 0.5, cfg_end: float = 1.0,
vae_type: str = 'Full', tiling: bool = False, hidiffusion: bool = False,
detailer_enabled: bool = True, detailer_prompt: str = '', detailer_negative: str = '', detailer_steps: int = 10, detailer_strength: float = 0.3, detailer_resolution: int = 1024,
@@ -306,7 +307,13 @@ def control_run(state: str = '', # pylint: disable=keyword-arg-before-vararg
seed_resize_from_h = seed_resize_from_h,
seed_resize_from_w = seed_resize_from_w,
denoising_strength = denoising_strength,
# advanced
# modular guidance
guidance_name = guidance_name,
guidance_scale = guidance_scale,
guidance_rescale = guidance_rescale,
guidance_start = guidance_start,
guidance_stop = guidance_stop,
# legacy guidance
cfg_scale = cfg_scale,
cfg_end = cfg_end,
clip_skip = clip_skip,
@@ -314,6 +321,7 @@ def control_run(state: str = '', # pylint: disable=keyword-arg-before-vararg
diffusers_guidance_rescale = diffusers_guidance_rescale,
pag_scale = pag_scale,
pag_adaptive = pag_adaptive,
# advanced
vae_type = vae_type,
tiling = tiling,
hidiffusion = hidiffusion,
@@ -396,9 +404,9 @@ def control_run(state: str = '', # pylint: disable=keyword-arg-before-vararg
# hires/refine defined outside of main init
vae_scale_factor = sd_vae.get_vae_scale_factor()
if p.enable_hr and (p.hr_resize_x == 0 or p.hr_resize_y == 0):
p.hr_upscale_to_x, p.hr_upscale_to_y = vae_scale_factor * int(p.width_before * p.hr_scale / vae_scale_factor), vae_scale_factor * int(p.height_before * p.hr_scale / vae_scale_factor)
p.hr_upscale_to_x, p.hr_upscale_to_y = int(vae_scale_factor * int(p.width_before * p.hr_scale / vae_scale_factor)), int(vae_scale_factor * int(p.height_before * p.hr_scale / vae_scale_factor))
elif p.enable_hr and (p.hr_upscale_to_x == 0 or p.hr_upscale_to_y == 0):
p.hr_upscale_to_x, p.hr_upscale_to_y = 8 * int(p.hr_resize_x / vae_scale_factor), vae_scale_factor * int(p.hr_resize_y / vae_scale_factor)
p.hr_upscale_to_x, p.hr_upscale_to_y = 8 * int(p.hr_resize_x / vae_scale_factor), int(vae_scale_factor * int(p.hr_resize_y / vae_scale_factor))
global p_extra_args # pylint: disable=global-statement
for k, v in p_extra_args.items():
modules/devices.py
+4
View File
@@ -49,6 +49,10 @@ def has_xpu() -> bool:
return bool(hasattr(torch, 'xpu') and torch.xpu.is_available())
def has_rocm() -> bool:
return bool(torch.version.hip is not None and torch.cuda.is_available())
def has_zluda() -> bool:
if not cuda_ok:
return False
modules/face/insightface.py
+2 -4
View File
@@ -10,11 +10,9 @@ instightface_mp = None
def get_app(mp_name, threshold=0.5, resolution=640):
global insightface_app, instightface_mp # pylint: disable=global-statement
from installer import install, installed, install_pydantic
from installer import install, installed, install_insightface
if not installed('insightface', reload=False, quiet=True):
install('git+https://github.com/deepinsight/insightface@554a05561cb71cfebb4e012dfea48807f845a0c2#subdirectory=python-package', 'insightface') # insightface==0.7.3 with patches
install('albumentations==1.4.3', ignore=False, reinstall=True)
install_pydantic()
install_insightface()
if not installed('ip_adapter', reload=False, quiet=True):
install('git+https://github.com/tencent-ailab/IP-Adapter.git', 'ip_adapter', ignore=False)
modules/framepack/framepack_load.py
+4
View File
@@ -124,6 +124,10 @@ def load_model(variant:str=None, pipeline:str=None, text_encoder:str=None, text_
sd_models.unload_model_weights()
t0 = time.time()
sd_models.hf_auth_check(model["transformer"]["repo"])
sd_models.hf_auth_check(model["text_encoder"]["repo"])
sd_models.hf_auth_check(model["text_encoder_2"]["repo"])
shared.log.debug(f'FramePack load: module=llm {model["text_encoder"]}')
load_args, quant_args = model_quant.get_dit_args({}, module='TE', device_map=True)
text_encoder = LlamaModel.from_pretrained(model["text_encoder"]["repo"], subfolder=model["text_encoder"]["subfolder"], cache_dir=shared.opts.hfcache_dir, **load_args, **quant_args)
modules/framepack/framepack_worker.py
+2 -2
View File
@@ -54,7 +54,7 @@ def worker(
from modules.framepack.pipeline import hunyuan
from modules.framepack.pipeline import utils
from modules.framepack.pipeline.k_diffusion_hunyuan import sample_hunyuan
from modules.framepack.pipeline import k_diffusion_hunyuan
is_f1 = variant == 'forward-only'
total_generated_frames = 0
@@ -244,7 +244,7 @@ def worker(
transformer.initialize_teacache(enable_teacache=use_teacache, num_steps=steps, rel_l1_thresh=shared.opts.teacache_thresh)
t_sample = time.time()
generated_latents = sample_hunyuan(
generated_latents = k_diffusion_hunyuan.sample_hunyuan(
transformer=transformer,
sampler='unipc',
width=width,
modules/framepack/pipeline/uni_pc_fm.py
+33 -3
View File
@@ -5,7 +5,7 @@
import torch
import numpy as np
from tqdm.auto import trange
@@ -13,6 +13,36 @@ def expand_dims(v, dims):
return v[(...,) + (None,) * (dims - 1)]
torch_linalg_solve = None
def test_solver():
from modules import devices, shared
try:
a = torch.randn(50, 50).to(device=devices.device, dtype=torch.float32)
b = torch.randn(50, 2).to(device=devices.device, dtype=torch.float32)
_x = torch.linalg.solve(a, b)
return True
except Exception as e:
shared.log.debug(f'FramePack: solver=cpu {e}')
return False
def linalg_solve(A, B, device):
global torch_linalg_solve # pylint: disable=global-statement
if torch_linalg_solve is None:
torch_linalg_solve = test_solver()
if torch_linalg_solve:
X = torch.linalg.solve(A, B)
return X
else:
A_np = A.float().cpu().numpy()
B_np = B.float().cpu().numpy()
X_np = np.linalg.solve(A_np, B_np)
X = torch.from_numpy(X_np).to(device=device, dtype=A.dtype)
return X
class FlowMatchUniPC:
def __init__(self, model, extra_args, variant='bh1'):
self.model = model
@@ -78,7 +108,7 @@ class FlowMatchUniPC:
if order == 2:
rhos_p = torch.tensor([0.5], device=b.device)
else:
rhos_p = torch.linalg.solve(R[:-1, :-1], b[:-1])
rhos_p = linalg_solve(R[:-1, :-1], b[:-1], x.device)
else:
D1s = None
rhos_p = None
@@ -86,7 +116,7 @@ class FlowMatchUniPC:
if order == 1:
rhos_c = torch.tensor([0.5], device=b.device)
else:
rhos_c = torch.linalg.solve(R, b)
rhos_c = linalg_solve(R, b, x.device)
x_t_ = expand_dims(t / t_prev_0, dims) * x - expand_dims(h_phi_1, dims) * model_prev_0
modules/img2img.py
+7 -4
View File
@@ -161,10 +161,8 @@ def img2img(id_task: str, state: str, mode: int,
vae_type, tiling, hidiffusion,
detailer_enabled, detailer_prompt, detailer_negative, detailer_steps, detailer_strength, detailer_resolution,
n_iter, batch_size,
cfg_scale, image_cfg_scale,
diffusers_guidance_rescale,
pag_scale, pag_adaptive,
cfg_end,
guidance_name, guidance_scale, guidance_rescale, guidance_start, guidance_stop,
cfg_scale, image_cfg_scale, diffusers_guidance_rescale, pag_scale, pag_adaptive, cfg_end,
refiner_start,
clip_skip,
denoising_strength,
@@ -255,6 +253,11 @@ def img2img(id_task: str, state: str, mode: int,
batch_size=batch_size,
n_iter=n_iter,
steps=steps,
guidance_name=guidance_name,
guidance_scale=guidance_scale,
guidance_rescale=guidance_rescale,
guidance_start=guidance_start,
guidance_stop=guidance_stop,
cfg_scale=cfg_scale,
cfg_end=cfg_end,
clip_skip=clip_skip,
modules/intel/ipex/__init__.py
+2 -2
View File
@@ -152,7 +152,7 @@ def ipex_init(): # pylint: disable=too-many-statements
# torch._int_mm directly uses onednn quantized matmul
# onednn qlinear is a wrapper around onednn quantized matmul
if hasattr(torch.ops, "onednn") and hasattr(torch.ops.onednn, "qlinear_pointwise"):
def onednn_mm(x: torch.Tensor, y: torch.Tensor):
def onednn_mm(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor:
# supports int8, fp32, fp16, and bf16 matmul with accumulation using a different dtype
# int8 matmul with onednn is slower than 16 bit with dim_size < 4096
return torch.ops.onednn.qlinear_pointwise.default(x, 1.0, 0, y, torch.ones(1, device=y.device), torch.zeros(1, device=y.device), None, 1.0, 0, torch.float32, "none", [], "none")
@@ -165,7 +165,7 @@ def ipex_init(): # pylint: disable=too-many-statements
pass
# Memory:
if 'linux' in sys.platform and "WSL2" in os.popen("uname -a").read():
if "linux" in sys.platform and "WSL2" in os.popen("uname -a").read():
torch.xpu.empty_cache = lambda: None
torch.cuda.empty_cache = torch.xpu.empty_cache
modules/intel/ipex/attention.py
+2 -2
View File
@@ -8,8 +8,8 @@ from functools import cache, wraps
# ARC GPUs can't allocate more than 4GB to a single block so we slice the attention layers
dynamic_attention_slice_rate = float(os.environ.get('IPEX_SDPA_SLICE_TRIGGER_RATE', 1))
dynamic_attention_trigger_rate = float(os.environ.get('IPEX_ATTENTION_SLICE_RATE', 0.5))
dynamic_attention_slice_rate = float(os.environ.get("IPEX_SDPA_SLICE_TRIGGER_RATE", "1"))
dynamic_attention_trigger_rate = float(os.environ.get("IPEX_ATTENTION_SLICE_RATE", "0.5"))
# Find something divisible with the input_tokens
@cache
modules/intel/ipex/hijacks.py
+72 -157
View File
@@ -1,11 +1,9 @@
from typing import Optional
import os
from functools import wraps
from contextlib import nullcontext
import torch
import numpy as np
from modules import devices, errors
from modules import devices
torch_version = torch.__version__[:4]
@@ -16,38 +14,25 @@ torch_version[0], torch_version[1] = int(torch_version[0]), int(torch_version[1]
device_supports_fp64 = torch.xpu.has_fp64_dtype() if hasattr(torch.xpu, "has_fp64_dtype") else torch.xpu.get_device_properties(devices.device).has_fp64
if os.environ.get('IPEX_FORCE_ATTENTION_SLICE', '0') == '0':
if torch_version[0] > 2 or (torch_version[0] == 2 and torch_version[1] >= 7):
use_dynamic_attention = False # torch 2.7 has flash atten support
else:
use_dynamic_attention = True
else:
use_dynamic_attention = bool(os.environ.get('IPEX_FORCE_ATTENTION_SLICE', '0') == '1')
# pylint: disable=protected-access, missing-function-docstring, line-too-long, unnecessary-lambda, no-else-return
class DummyDataParallel(torch.nn.Module): # pylint: disable=missing-class-docstring, unused-argument, too-few-public-methods
def __new__(cls, module, device_ids=None, output_device=None, dim=0): # pylint: disable=unused-argument
if isinstance(device_ids, list) and len(device_ids) > 1:
errors.log.error("IPEX backend doesn't support DataParallel on multiple XPU devices")
return module.to(devices.device)
def return_null_context(*args, **kwargs): # pylint: disable=unused-argument
return nullcontext()
@property
def is_cuda(self):
return self.device.type == "xpu" or self.device.type == "cuda"
def check_device_type(device, device_type: str) -> bool:
if device is None or type(device) not in {str, int, torch.device}:
return False
else:
return bool(torch.device(device).type == device_type)
def check_cuda(device) -> bool:
return bool(isinstance(device, int) or check_device_type(device, "cuda"))
def return_xpu(device): # keep the device instance type, aka return string if the input is string
return devices.device if device is None else f"xpu:{device.split(':')[-1]}" if isinstance(device, str) and ":" in device else f"xpu:{device}" if isinstance(device, int) else torch.device(f"xpu:{device.index}" if device.index is not None else "xpu") if isinstance(device, torch.device) else "xpu"
@@ -95,8 +80,8 @@ def torch_get_autocast_dtype(device_type=None):
# IPEX 2.5 and above has partial support but doesn't really work most of the time.
original_interpolate = torch.nn.functional.interpolate
@wraps(torch.nn.functional.interpolate)
def interpolate(tensor, size=None, scale_factor=None, mode='nearest', align_corners=None, recompute_scale_factor=None, antialias=False): # pylint: disable=too-many-arguments
if mode in {'bicubic', 'bilinear'}:
def interpolate(tensor, size=None, scale_factor=None, mode="nearest", align_corners=None, recompute_scale_factor=None, antialias=False): # pylint: disable=too-many-arguments
if mode in {"bicubic", "bilinear"}:
return_device = tensor.device
return_dtype = tensor.dtype
return original_interpolate(tensor.to("cpu", dtype=torch.float32), size=size, scale_factor=scale_factor, mode=mode,
@@ -106,6 +91,32 @@ def interpolate(tensor, size=None, scale_factor=None, mode='nearest', align_corn
align_corners=align_corners, recompute_scale_factor=recompute_scale_factor, antialias=antialias)
# SwinIR BF16:
original_functional_pad = torch.nn.functional.pad
@wraps(torch.nn.functional.pad)
def functional_pad(input, pad, mode="constant", value=None):
if mode == "reflect" and input.dtype == torch.bfloat16:
return original_functional_pad(input.to(torch.float32), pad, mode=mode, value=value).to(dtype=torch.bfloat16)
else:
return original_functional_pad(input, pad, mode=mode, value=value)
# Diffusers FreeU
original_fft_fftn = torch.fft.fftn
@wraps(torch.fft.fftn)
def fft_fftn(input, s=None, dim=None, norm=None, *, out=None):
return_dtype = input.dtype
return original_fft_fftn(input.to(dtype=torch.float32), s=s, dim=dim, norm=norm, out=out).to(dtype=return_dtype)
# Diffusers FreeU
original_fft_ifftn = torch.fft.ifftn
@wraps(torch.fft.ifftn)
def fft_ifftn(input, s=None, dim=None, norm=None, *, out=None):
return_dtype = input.dtype
return original_fft_ifftn(input.to(dtype=torch.float32), s=s, dim=dim, norm=norm, out=out).to(dtype=return_dtype)
# Diffusers Float64 (Alchemist GPUs doesn't support 64 bit):
original_from_numpy = torch.from_numpy
@wraps(torch.from_numpy)
@@ -115,6 +126,7 @@ def from_numpy(ndarray):
else:
return original_from_numpy(ndarray)
original_as_tensor = torch.as_tensor
@wraps(torch.as_tensor)
def as_tensor(data, dtype=None, device=None):
@@ -126,120 +138,6 @@ def as_tensor(data, dtype=None, device=None):
return original_as_tensor(data, dtype=dtype, device=device)
if not use_dynamic_attention:
original_scaled_dot_product_attention = torch.nn.functional.scaled_dot_product_attention
else:
# 32 bit attention workarounds for Alchemist:
try:
from .attention import dynamic_scaled_dot_product_attention as original_scaled_dot_product_attention
except ImportError:
original_scaled_dot_product_attention = torch.nn.functional.scaled_dot_product_attention
@wraps(torch.nn.functional.scaled_dot_product_attention)
def scaled_dot_product_attention(query: torch.FloatTensor, key: torch.FloatTensor, value: torch.FloatTensor, attn_mask: Optional[torch.FloatTensor] = None, dropout_p: float = 0.0, is_causal: bool = False, scale: Optional[float] = None, enable_gqa: bool = False, **kwargs) -> torch.FloatTensor:
if query.dtype != key.dtype:
key = key.to(dtype=query.dtype)
if query.dtype != value.dtype:
value = value.to(dtype=query.dtype)
if attn_mask is not None and query.dtype != attn_mask.dtype:
attn_mask = attn_mask.to(dtype=query.dtype)
if enable_gqa:
kwargs["enable_gqa"] = enable_gqa
result = original_scaled_dot_product_attention(query, key, value, attn_mask=attn_mask, dropout_p=dropout_p, is_causal=is_causal, scale=scale, **kwargs)
if result.dtype != query.dtype:
result = result.to(dtype=query.dtype)
return result
# Data Type Errors:
original_torch_bmm = torch.bmm
@wraps(torch.bmm)
def torch_bmm(input, mat2, *, out=None):
if input.dtype != mat2.dtype:
mat2 = mat2.to(dtype=input.dtype)
return original_torch_bmm(input, mat2, out=out)
# Diffusers FreeU
original_fft_fftn = torch.fft.fftn
@wraps(torch.fft.fftn)
def fft_fftn(input, s=None, dim=None, norm=None, *, out=None):
return_dtype = input.dtype
return original_fft_fftn(input.to(dtype=torch.float32), s=s, dim=dim, norm=norm, out=out).to(dtype=return_dtype)
# Diffusers FreeU
original_fft_ifftn = torch.fft.ifftn
@wraps(torch.fft.ifftn)
def fft_ifftn(input, s=None, dim=None, norm=None, *, out=None):
return_dtype = input.dtype
return original_fft_ifftn(input.to(dtype=torch.float32), s=s, dim=dim, norm=norm, out=out).to(dtype=return_dtype)
# A1111 FP16
original_functional_group_norm = torch.nn.functional.group_norm
@wraps(torch.nn.functional.group_norm)
def functional_group_norm(input, num_groups, weight=None, bias=None, eps=1e-05):
if weight is not None and input.dtype != weight.data.dtype:
input = input.to(dtype=weight.data.dtype)
if bias is not None and weight is not None and bias.data.dtype != weight.data.dtype:
bias.data = bias.data.to(dtype=weight.data.dtype)
return original_functional_group_norm(input, num_groups, weight=weight, bias=bias, eps=eps)
# A1111 BF16
original_functional_layer_norm = torch.nn.functional.layer_norm
@wraps(torch.nn.functional.layer_norm)
def functional_layer_norm(input, normalized_shape, weight=None, bias=None, eps=1e-05):
if weight is not None and input.dtype != weight.data.dtype:
input = input.to(dtype=weight.data.dtype)
if bias is not None and weight is not None and bias.data.dtype != weight.data.dtype:
bias.data = bias.data.to(dtype=weight.data.dtype)
return original_functional_layer_norm(input, normalized_shape, weight=weight, bias=bias, eps=eps)
# Training
original_functional_linear = torch.nn.functional.linear
@wraps(torch.nn.functional.linear)
def functional_linear(input, weight, bias=None):
if input.dtype != weight.data.dtype:
input = input.to(dtype=weight.data.dtype)
if bias is not None and bias.data.dtype != weight.data.dtype:
bias.data = bias.data.to(dtype=weight.data.dtype)
return original_functional_linear(input, weight, bias=bias)
original_functional_conv1d = torch.nn.functional.conv1d
@wraps(torch.nn.functional.conv1d)
def functional_conv1d(input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1):
if input.dtype != weight.data.dtype:
input = input.to(dtype=weight.data.dtype)
if bias is not None and bias.data.dtype != weight.data.dtype:
bias.data = bias.data.to(dtype=weight.data.dtype)
return original_functional_conv1d(input, weight, bias=bias, stride=stride, padding=padding, dilation=dilation, groups=groups)
original_functional_conv2d = torch.nn.functional.conv2d
@wraps(torch.nn.functional.conv2d)
def functional_conv2d(input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1):
if input.dtype != weight.data.dtype:
input = input.to(dtype=weight.data.dtype)
if bias is not None and bias.data.dtype != weight.data.dtype:
bias.data = bias.data.to(dtype=weight.data.dtype)
return original_functional_conv2d(input, weight, bias=bias, stride=stride, padding=padding, dilation=dilation, groups=groups)
# LTX Video
original_functional_conv3d = torch.nn.functional.conv3d
@wraps(torch.nn.functional.conv3d)
def functional_conv3d(input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1):
if input.dtype != weight.data.dtype:
input = input.to(dtype=weight.data.dtype)
if bias is not None and bias.data.dtype != weight.data.dtype:
bias.data = bias.data.to(dtype=weight.data.dtype)
return original_functional_conv3d(input, weight, bias=bias, stride=stride, padding=padding, dilation=dilation, groups=groups)
# SwinIR BF16:
original_functional_pad = torch.nn.functional.pad
@wraps(torch.nn.functional.pad)
def functional_pad(input, pad, mode='constant', value=None):
if mode == 'reflect' and input.dtype == torch.bfloat16:
return original_functional_pad(input.to(torch.float32), pad, mode=mode, value=value).to(dtype=torch.bfloat16)
else:
return original_functional_pad(input, pad, mode=mode, value=value)
original_torch_tensor = torch.tensor
@wraps(torch.tensor)
def torch_tensor(data, *args, dtype=None, device=None, **kwargs):
@@ -254,20 +152,20 @@ def torch_tensor(data, *args, dtype=None, device=None, **kwargs):
dtype = torch.float32
return original_torch_tensor(data, *args, dtype=dtype, device=device, **kwargs)
torch.Tensor.original_Tensor_to = torch.Tensor.to
@wraps(torch.Tensor.to)
def Tensor_to(self, device=None, *args, **kwargs):
global device_supports_fp64
if check_cuda(device):
if not device_supports_fp64 and kwargs.get("dtype", None) == torch.float64:
device = return_xpu(device)
if not device_supports_fp64:
if kwargs.get("dtype", None) == torch.float64 and ((device is None and self.device.type == "xpu") or (device is not None and torch.device(device).type == "xpu")):
kwargs["dtype"] = torch.float32
return self.original_Tensor_to(return_xpu(device), *args, **kwargs)
else:
if not device_supports_fp64:
if kwargs.get("dtype", None) == torch.float64 and ((device is None and self.device.type == "xpu") or (device is not None and torch.device(device).type == "xpu")):
kwargs["dtype"] = torch.float32
elif device == torch.float64 and self.device.type == "xpu":
device = torch.float32
return self.original_Tensor_to(device, *args, **kwargs)
elif device == torch.float64 and self.device.type == "xpu":
device = torch.float32
return self.original_Tensor_to(device, *args, **kwargs)
original_Tensor_cuda = torch.Tensor.cuda
@wraps(torch.Tensor.cuda)
@@ -277,6 +175,7 @@ def Tensor_cuda(self, device=None, *args, **kwargs):
else:
return original_Tensor_cuda(self, device, *args, **kwargs)
original_Tensor_pin_memory = torch.Tensor.pin_memory
@wraps(torch.Tensor.pin_memory)
def Tensor_pin_memory(self, device=None, *args, **kwargs):
@@ -285,6 +184,7 @@ def Tensor_pin_memory(self, device=None, *args, **kwargs):
else:
return original_Tensor_pin_memory(self, device, *args, **kwargs)
original_UntypedStorage_init = torch.UntypedStorage.__init__
@wraps(torch.UntypedStorage.__init__)
def UntypedStorage_init(*args, device=None, **kwargs):
@@ -293,6 +193,7 @@ def UntypedStorage_init(*args, device=None, **kwargs):
else:
return original_UntypedStorage_init(*args, device=device, **kwargs)
if torch_version[0] > 2 or (torch_version[0] == 2 and torch_version[1] >= 4):
original_UntypedStorage_to = torch.UntypedStorage.to
@wraps(torch.UntypedStorage.to)
@@ -310,6 +211,7 @@ if torch_version[0] > 2 or (torch_version[0] == 2 and torch_version[1] >= 4):
else:
return original_UntypedStorage_cuda(self, device=device, non_blocking=non_blocking, **kwargs)
original_torch_empty = torch.empty
@wraps(torch.empty)
def torch_empty(*args, device=None, **kwargs):
@@ -318,15 +220,17 @@ def torch_empty(*args, device=None, **kwargs):
else:
return original_torch_empty(*args, device=device, **kwargs)
original_torch_randn = torch.randn
@wraps(torch.randn)
def torch_randn(*args, device=None, dtype=None, **kwargs):
if dtype is bytes:
dtype = None
if check_cuda(device):
return original_torch_randn(*args, device=return_xpu(device), **kwargs)
return original_torch_randn(*args, device=return_xpu(device), dtype=dtype, **kwargs)
else:
return original_torch_randn(*args, device=device, **kwargs)
return original_torch_randn(*args, device=device, dtype=dtype, **kwargs)
original_torch_ones = torch.ones
@wraps(torch.ones)
@@ -336,6 +240,7 @@ def torch_ones(*args, device=None, **kwargs):
else:
return original_torch_ones(*args, device=device, **kwargs)
original_torch_zeros = torch.zeros
@wraps(torch.zeros)
def torch_zeros(*args, device=None, **kwargs):
@@ -344,6 +249,7 @@ def torch_zeros(*args, device=None, **kwargs):
else:
return original_torch_zeros(*args, device=device, **kwargs)
original_torch_full = torch.full
@wraps(torch.full)
def torch_full(*args, device=None, **kwargs):
@@ -352,6 +258,7 @@ def torch_full(*args, device=None, **kwargs):
else:
return original_torch_full(*args, device=device, **kwargs)
original_torch_linspace = torch.linspace
@wraps(torch.linspace)
def torch_linspace(*args, device=None, **kwargs):
@@ -360,6 +267,7 @@ def torch_linspace(*args, device=None, **kwargs):
else:
return original_torch_linspace(*args, device=device, **kwargs)
original_torch_eye = torch.eye
@wraps(torch.eye)
def torch_eye(*args, device=None, **kwargs):
@@ -368,6 +276,7 @@ def torch_eye(*args, device=None, **kwargs):
else:
return original_torch_eye(*args, device=device, **kwargs)
original_torch_load = torch.load
@wraps(torch.load)
def torch_load(f, map_location=None, *args, **kwargs):
@@ -376,6 +285,7 @@ def torch_load(f, map_location=None, *args, **kwargs):
else:
return original_torch_load(f, *args, map_location=map_location, **kwargs)
@wraps(torch.cuda.synchronize)
def torch_cuda_synchronize(device=None):
if check_cuda(device):
@@ -383,6 +293,7 @@ def torch_cuda_synchronize(device=None):
else:
return torch.xpu.synchronize(device)
@wraps(torch.cuda.device)
def torch_cuda_device(device):
if check_cuda(device):
@@ -390,6 +301,7 @@ def torch_cuda_device(device):
else:
return torch.xpu.device(device)
@wraps(torch.cuda.set_device)
def torch_cuda_set_device(device):
if check_cuda(device):
@@ -397,6 +309,7 @@ def torch_cuda_set_device(device):
else:
torch.xpu.set_device(device)
# torch.Generator has to be a class for isinstance checks
original_torch_Generator = torch.Generator
class torch_Generator(original_torch_Generator):
@@ -434,24 +347,14 @@ def ipex_hijacks():
torch.Generator = torch_Generator
torch._C.Generator = torch_Generator
torch.backends.cuda.sdp_kernel = return_null_context
torch.nn.DataParallel = DummyDataParallel
torch.UntypedStorage.is_cuda = is_cuda
torch.amp.autocast_mode.autocast.__init__ = autocast_init
torch.nn.functional.interpolate = interpolate
torch.nn.functional.scaled_dot_product_attention = scaled_dot_product_attention
torch.nn.functional.group_norm = functional_group_norm
torch.nn.functional.layer_norm = functional_layer_norm
torch.nn.functional.linear = functional_linear
torch.nn.functional.conv1d = functional_conv1d
torch.nn.functional.conv2d = functional_conv2d
torch.nn.functional.conv3d = functional_conv3d
torch.nn.functional.pad = functional_pad
torch.bmm = torch_bmm
torch.fft.fftn = fft_fftn
torch.fft.ifftn = fft_ifftn
if not device_supports_fp64:
torch.from_numpy = from_numpy
torch.as_tensor = as_tensor
@@ -462,6 +365,18 @@ def ipex_hijacks():
except Exception:
pass
if os.environ.get("IPEX_FORCE_ATTENTION_SLICE", "0") == "0":
if torch_version[0] > 2 or (torch_version[0] == 2 and torch_version[1] >= 7):
use_dynamic_attention = False # torch 2.7 has flash atten support
else:
use_dynamic_attention = True
else:
use_dynamic_attention = bool(os.environ.get("IPEX_FORCE_ATTENTION_SLICE", "0") == "1")
if use_dynamic_attention:
from .attention import dynamic_scaled_dot_product_attention
torch.nn.functional.scaled_dot_product_attention = dynamic_scaled_dot_product_attention
# AMP:
torch.amp.grad_scaler.GradScaler.__init__ = GradScaler_init
torch.is_autocast_enabled = torch_is_autocast_enabled
modules/interrogate/vqa.py
+40 -14
View File
@@ -13,7 +13,7 @@ from modules import shared, devices, errors, model_quant, sd_models, sd_models_c
processor = None
model = None
loaded: str = None
quant_args = {}
quant_args = None
vlm_default = "Alibaba Qwen 2.5 VL 4B"
vlm_models = {
"Google Gemma 3 4B": "google/gemma-3-4b-it",
@@ -27,12 +27,18 @@ vlm_models = {
"Apple FastVLM 0.5B": "apple/FastVLM-0.5B",
"Apple FastVLM 1.5B": "apple/FastVLM-1.5B",
"Apple FastVLM 7B": "apple/FastVLM-7B",
"Microsoft Florence 2 Base": "microsoft/Florence-2-base-ft", # 0.5GB
"Microsoft Florence 2 Large": "microsoft/Florence-2-large-ft", # 1.5GB
"MiaoshouAI PromptGen 1.5 Base": "MiaoshouAI/Florence-2-base-PromptGen-v1.5@c06a5f02cc6071a5d65ee5d294cf3732d3097540", # 1.1GB
"MiaoshouAI PromptGen 1.5 Large": "MiaoshouAI/Florence-2-large-PromptGen-v1.5@28a42440e39c9c32b83f7ae74ec2b3d1540404f0", # 3.3GB
"MiaoshouAI PromptGen 2.0 Base": "MiaoshouAI/Florence-2-base-PromptGen-v2.0", # 1.1GB
"MiaoshouAI PromptGen 2.0 Large": "MiaoshouAI/Florence-2-large-PromptGen-v2.0", # 3.3GB
# "Microsoft Florence 2 Base": "microsoft/Florence-2-base-ft", # 0.5GB
# "Microsoft Florence 2 Large": "microsoft/Florence-2-large-ft", # 1.5GB
"Microsoft Florence 2 Base": "florence-community/Florence-2-base-ft", # 0.5GB
"Microsoft Florence 2 Large": "florence-community/Florence-2-large-ft", # 1.5GB
#"MiaoshouAI PromptGen 1.5 Base": "MiaoshouAI/Florence-2-base-PromptGen-v1.5@c06a5f02cc6071a5d65ee5d294cf3732d3097540", # 1.1GB
#"MiaoshouAI PromptGen 1.5 Large": "MiaoshouAI/Florence-2-large-PromptGen-v1.5@28a42440e39c9c32b83f7ae74ec2b3d1540404f0", # 3.3GB
#"MiaoshouAI PromptGen 2.0 Base": "MiaoshouAI/Florence-2-base-PromptGen-v2.0", # 1.1GB
#"MiaoshouAI PromptGen 2.0 Large": "MiaoshouAI/Florence-2-large-PromptGen-v2.0", # 3.3GB
"MiaoshouAI PromptGen 1.5 Base": "Disty0/Florence-2-base-PromptGen-v1.5", # 0.5GB
"MiaoshouAI PromptGen 1.5 Large": "Disty0/Florence-2-large-PromptGen-v1.5", # 1.5GB
"MiaoshouAI PromptGen 2.0 Base": "Disty0/Florence-2-base-PromptGen-v2.0", # 0.5GB
"MiaoshouAI PromptGen 2.0 Large": "Disty0/Florence-2-large-PromptGen-v2.0", # 1.5GB
"CogFlorence 2.0 Large": "thwri/CogFlorence-2-Large-Freeze", # 1.6GB
"CogFlorence 2.2 Large": "thwri/CogFlorence-2.2-Large", # 1.6GB
"Moondream 2": "vikhyatk/moondream2", # 3.7GB
@@ -88,14 +94,19 @@ def b64(image):
def clean(response, question):
strip = ['---', '\r', '\t', '**', '"', '“', '”', 'Assistant:', 'Caption:', '<|im_end|>', '<pad>']
if isinstance(response, dict):
if 'task' in response:
response = response['task']
if isinstance(response, str):
response = response.strip()
elif isinstance(response, dict):
if 'answer' in response:
response = response['answer']
response = json.dumps(response)
if isinstance(response, list):
elif 'task' in response:
response = response['task']
else:
response = json.dumps(response)
elif isinstance(response, list):
response = response[0]
else:
response = str(response)
question = question.replace('<', '').replace('>', '').replace('_', ' ')
if question in response:
response = response.split(question, 1)[1]
@@ -354,7 +365,6 @@ def smol(question: str, image: Image.Image, repo: str = None, system_prompt: str
repo,
cache_dir=shared.opts.hfcache_dir,
torch_dtype=devices.dtype,
_attn_implementation="eager",
**quant_args,
)
processor = transformers.AutoProcessor.from_pretrained(repo, cache_dir=shared.opts.hfcache_dir)
@@ -524,11 +534,13 @@ def moondream(question: str, image: Image.Image, repo: str = None):
def florence(question: str, image: Image.Image, repo: str = None, revision: str = None):
global processor, model, loaded # pylint: disable=global-statement
_get_imports = transformers.dynamic_module_utils.get_imports
def get_imports(f):
R = _get_imports(f)
if "flash_attn" in R:
R.remove("flash_attn") # flash_attn is optional
return R
revision = None
if '@' in repo:
repo, revision = repo.split('@')
@@ -536,6 +548,7 @@ def florence(question: str, image: Image.Image, repo: str = None, revision: str
shared.log.debug(f'Interrogate load: vlm="{repo}" path="{shared.opts.hfcache_dir}"')
transformers.dynamic_module_utils.get_imports = get_imports
model = None
"""
model = transformers.AutoModelForCausalLM.from_pretrained(
repo,
trust_remote_code=True,
@@ -544,6 +557,15 @@ def florence(question: str, image: Image.Image, repo: str = None, revision: str
cache_dir=shared.opts.hfcache_dir,
**quant_args,
)
"""
model = transformers.Florence2ForConditionalGeneration.from_pretrained(
repo,
dtype=torch.bfloat16,
revision=revision,
torch_dtype=devices.dtype,
cache_dir=shared.opts.hfcache_dir,
**quant_args,
)
processor = transformers.AutoProcessor.from_pretrained(repo, trust_remote_code=True, revision=revision, cache_dir=shared.opts.hfcache_dir)
transformers.dynamic_module_utils.get_imports = _get_imports
loaded = repo
@@ -607,7 +629,8 @@ def interrogate(question:str='', system_prompt:str=None, prompt:str=None, image:
global quant_args # pylint: disable=global-statement
jobid = shared.state.begin('Interrogate LLM')
t0 = time.time()
quant_args = model_quant.create_config(module='LLM')
if quant_args is None:
quant_args = model_quant.create_config(module='LLM')
model_name = model_name or shared.opts.interrogate_vlm_model
if isinstance(image, list):
image = image[0] if len(image) > 0 else None
@@ -623,9 +646,12 @@ def interrogate(question:str='', system_prompt:str=None, prompt:str=None, image:
question = prompt
if len(question) < 2:
question = "Describe the image."
"""
if shared.sd_loaded:
from modules.sd_models import apply_balanced_offload # prevent circular import
apply_balanced_offload(shared.sd_model)
"""
from modules import modelloader
modelloader.hf_login()
modules/ipadapter.py
+12
View File
@@ -354,6 +354,18 @@ def apply(pipe, p: processing.StableDiffusionProcessing, adapter_names=[], adapt
adapter_scales[i] = 0.00
pipe.set_ip_adapter_scale(adapter_scales if len(adapter_scales) > 1 else adapter_scales[0])
ip_str = [f'{os.path.splitext(adapter)[0]}:{scale}:{start}:{end}:{crop}' for adapter, scale, start, end, crop in zip(adapter_names, adapter_scales, adapter_starts, adapter_ends, adapter_crops)]
if hasattr(pipe, 'transformer') and 'Nunchaku' in pipe.transformer.__class__.__name__:
if isinstance(repos, str):
sd_models.clear_caches(full=True)
import accelerate
accelerate.hooks.remove_hook_from_module(pipe.transformer, recurse=True)
pipe.transformer = pipe.transformer.to(devices.device)
from nunchaku.models.ip_adapter.diffusers_adapters import apply_IPA_on_pipe
apply_IPA_on_pipe(pipe, ip_adapter_scale=adapter_scales[0], repo_id=repos)
pipe = sd_models.apply_balanced_offload(pipe)
shared.log.debug(f'IP adapter load: engine=nunchaku scale={adapter_scales[0]} repo="{repos}"')
else:
shared.log.error('IP adapter: Nunchaku only supports single adapter')
p.task_args['ip_adapter_image'] = crop_images(adapter_images, adapter_crops)
if len(adapter_masks) > 0:
p.cross_attention_kwargs = { 'ip_adapter_masks': adapter_masks }
modules/loader.py
+13 -8
View File
@@ -123,15 +123,20 @@ timer.startup.record("pydantic")
import tqdm as tqdm_lib # pylint: disable=C0411
from tqdm.rich import tqdm # pylint: disable=W0611,C0411
import diffusers.utils.import_utils # pylint: disable=W0611,C0411
diffusers.utils.import_utils._k_diffusion_available = True # pylint: disable=protected-access # monkey-patch since we use k-diffusion from git
diffusers.utils.import_utils._k_diffusion_version = '0.0.12' # pylint: disable=protected-access
try:
import diffusers.utils.import_utils # pylint: disable=W0611,C0411
diffusers.utils.import_utils._k_diffusion_available = True # pylint: disable=protected-access # monkey-patch since we use k-diffusion from git
diffusers.utils.import_utils._k_diffusion_version = '0.0.12' # pylint: disable=protected-access
import diffusers # pylint: disable=W0611,C0411
import diffusers.loaders.single_file # pylint: disable=W0611,C0411
diffusers.loaders.single_file.logging.tqdm = partial(tqdm, unit='C')
logging.getLogger("diffusers.loaders.single_file").setLevel(logging.ERROR)
timer.startup.record("diffusers")
import diffusers # pylint: disable=W0611,C0411
import diffusers.loaders.single_file # pylint: disable=W0611,C0411
diffusers.loaders.single_file.logging.tqdm = partial(tqdm, unit='C')
logging.getLogger("diffusers.loaders.single_file").setLevel(logging.ERROR)
timer.startup.record("diffusers")
except Exception as e:
errors.log.error(f'Loader: diffusers=={diffusers.__version__ if "diffusers" in sys.modules else None} {e}')
errors.log.error('Please restart re-run the installer')
sys.exit(1)
import huggingface_hub # pylint: disable=W0611,C0411
timer.startup.record("hfhub")
modules/lora/extra_networks_lora.py
+36 -13
View File
@@ -2,7 +2,7 @@ from typing import List
import os
import re
import numpy as np
from modules.lora import networks, lora_overrides, lora_load
from modules.lora import networks, lora_overrides, lora_load, lora_diffusers
from modules.lora import lora_common as l
from modules import extra_networks, shared, sd_models
@@ -83,21 +83,34 @@ def infotext(p):
p.extra_generation_params["LoRA hashes"] = ", ".join(network_hashes)
def to_float(value):
try:
return float(value)
except (ValueError, TypeError):
return value
def parse(p, params_list, step=0):
names = []
te_multipliers = []
unet_multipliers = []
dyn_dims = []
lora_modules = []
for params in params_list:
assert params.items
names.append(params.positional[0])
te_multiplier = params.named.get("te", params.positional[1] if len(params.positional) > 1 else shared.opts.extra_networks_default_multiplier)
default_multiplier = params.positional[1] if len(params.positional) > 1 else shared.opts.extra_networks_default_multiplier
default_multiplier = to_float(default_multiplier)
if isinstance(default_multiplier, str) and "@" not in default_multiplier:
default_multiplier = shared.opts.extra_networks_default_multiplier
te_multiplier = params.named.get("te", default_multiplier)
if isinstance(te_multiplier, str) and "@" in te_multiplier:
te_multiplier = get_stepwise(te_multiplier, step, p.steps)
else:
te_multiplier = float(te_multiplier)
unet_multiplier = [params.positional[2] if len(params.positional) > 2 else te_multiplier] * 3
unet_multiplier = [params.named.get("unet", unet_multiplier[0])] * 3
te_multiplier = to_float(te_multiplier)
unet_multiplier = 3 * [params.named.get("unet", te_multiplier)] # fill all 3 with same value
unet_multiplier[0] = params.named.get("in", unet_multiplier[0])
unet_multiplier[1] = params.named.get("mid", unet_multiplier[1])
unet_multiplier[2] = params.named.get("out", unet_multiplier[2])
@@ -105,13 +118,23 @@ def parse(p, params_list, step=0):
if isinstance(unet_multiplier[i], str) and "@" in unet_multiplier[i]:
unet_multiplier[i] = get_stepwise(unet_multiplier[i], step, p.steps)
else:
unet_multiplier[i] = float(unet_multiplier[i])
dyn_dim = int(params.positional[3]) if len(params.positional) > 3 else None
dyn_dim = int(params.named["dyn"]) if "dyn" in params.named else dyn_dim
unet_multiplier[i] = to_float(unet_multiplier[i])
dyn_dim = int(params.named["dyn"]) if "dyn" in params.named else None
te_multipliers.append(te_multiplier)
unet_multipliers.append(unet_multiplier)
dyn_dims.append(dyn_dim)
return names, te_multipliers, unet_multipliers, dyn_dims
lora_module = []
if 'high' in params.positional or 'HIGH 14B' in params.positional[0]:
lora_module.append('transformer')
if 'low' in params.positional or 'LOW 14B' in params.positional[0]:
lora_module.append('transformer_2')
if params.named.get('module', None) is not None:
lora_module.append(params.named['module'].lower())
lora_modules.append(lora_module)
return names, te_multipliers, unet_multipliers, dyn_dims, lora_modules
def unload_diffusers():
@@ -168,7 +191,7 @@ class ExtraNetworkLora(extra_networks.ExtraNetwork):
if len(params_list) > 0 and not self.active: # activate patches once
self.active = True
self.model = shared.opts.sd_model_checkpoint
names, te_multipliers, unet_multipliers, dyn_dims = parse(p, params_list, step)
names, te_multipliers, unet_multipliers, dyn_dims, lora_modules = parse(p, params_list, step)
requested = self.signature(names, te_multipliers, unet_multipliers)
load_method = lora_overrides.get_method()
@@ -181,7 +204,7 @@ class ExtraNetworkLora(extra_networks.ExtraNetwork):
has_changed = False # diffusers handles its own loading
if len(exclude) == 0:
jobid = shared.state.begin('LoRA')
lora_load.network_load(names, te_multipliers, unet_multipliers, dyn_dims) # load only on first call
lora_load.network_load(names, te_multipliers, unet_multipliers, dyn_dims, lora_modules) # load only on first call
sd_models.set_diffuser_offload(shared.sd_model, op="model")
shared.state.end(jobid)
elif load_method == 'nunchaku':
@@ -208,7 +231,7 @@ class ExtraNetworkLora(extra_networks.ExtraNetwork):
shared.log.info(f'Network load: type=LoRA apply={[n.name for n in l.loaded_networks]} method={load_method} mode={"fuse" if shared.opts.lora_fuse_diffusers else "backup"} te={te_multipliers} unet={unet_multipliers} time={l.timer.summary}')
def deactivate(self, p):
if len(lora_load.diffuser_loaded) > 0:
if len(lora_diffusers.diffuser_loaded) > 0:
if not (shared.compiled_model_state is not None and shared.compiled_model_state.is_compiled is True):
unload_diffusers()
if self.active and l.debug:
modules/lora/lora_diffusers.py
+92
View File
@@ -0,0 +1,92 @@
from typing import Union
import os
import time
import diffusers
from modules import shared, errors
from modules.lora import network
from modules.lora import lora_common as l
diffuser_loaded = []
diffuser_scales = []
def load_per_module(sd_model: diffusers.DiffusionPipeline, filename: str, adapter_name: str, lora_modules: list[str]):
shared.log.debug(f'LoRA load: modules={lora_modules}')
try:
state_dict = sd_model.lora_state_dict(filename)
if isinstance(state_dict, tuple) and len(state_dict) == 2:
state_dict, network_alphas = state_dict
else:
network_alphas = {}
except Exception as e:
shared.log.error(f'LoRA load: {e}')
if l.debug:
errors.display(e, "LoRA")
return None
for lora_module in lora_modules:
if lora_module == 'transformer':
if hasattr(sd_model, 'transformer') and sd_model.transformer is not None:
sd_model.load_lora_into_transformer(state_dict, transformer=sd_model.transformer, adapter_name=adapter_name)
else:
shared.log.warning(f'LoRA load: requested={lora_module} missing')
elif lora_module == 'transformer_2':
if hasattr(sd_model, 'transformer_2') and sd_model.transformer_2 is not None:
sd_model.load_lora_into_transformer(state_dict, transformer=sd_model.transformer_2, adapter_name=adapter_name)
else:
shared.log.warning(f'LoRA load: requested={lora_module} missing')
elif lora_module == 'unet':
if hasattr(sd_model, 'unet') and sd_model.unet is not None:
sd_model.load_lora_into_unet(state_dict, network_alphas, unet=sd_model.unet, adapter_name=adapter_name)
else:
shared.log.warning(f'LoRA load: requested={lora_module} missing')
elif lora_module == 'text_encoder' or lora_module == 'te':
if hasattr(sd_model, 'text_encoder') and sd_model.text_encoder is not None:
sd_model.load_lora_into_text_encoder(state_dict, network_alphas, text_encoder=sd_model.text_encoder, adapter_name=adapter_name)
else:
shared.log.warning(f'LoRA load: requested={lora_module} missing')
else:
shared.log.warning(f'LoRA load: requested={lora_module} unknown')
return adapter_name
def load_diffusers(name: str, network_on_disk: network.NetworkOnDisk, lora_scale:float=shared.opts.extra_networks_default_multiplier, lora_module=None) -> Union[network.Network, None]:
t0 = time.time()
name = name.replace(".", "_")
sd_model: diffusers.DiffusionPipeline = getattr(shared.sd_model, "pipe", shared.sd_model)
shared.log.debug(f'Network load: type=LoRA name="{name}" file="{network_on_disk.filename}" detected={network_on_disk.sd_version} method=diffusers scale={lora_scale} fuse={shared.opts.lora_fuse_diffusers}')
if not hasattr(sd_model, 'load_lora_weights'):
shared.log.error(f'Network load: type=LoRA class={sd_model.__class__} does not implement load lora')
return None
try:
if lora_module is not None and isinstance(lora_module, list) and len(lora_module) > 0:
name = load_per_module(sd_model, network_on_disk.filename, adapter_name=name, lora_modules=lora_module)
else:
sd_model.load_lora_weights(network_on_disk.filename, adapter_name=name)
except Exception as e:
if 'already in use' in str(e):
pass
else:
if 'following keys have not been correctly renamed' in str(e):
shared.log.error(f'Network load: type=LoRA name="{name}" diffusers unsupported format')
elif 'object has no attribute' in str(e):
shared.log.error(f'Network load: type=LoRA name="{name}" diffusers empty module')
else:
shared.log.error(f'Network load: type=LoRA name="{name}" {e}')
if l.debug:
errors.display(e, "LoRA")
return None
if name is None:
return None
if name not in diffuser_loaded:
list_adapters = sd_model.get_list_adapters()
list_adapters = [adapter for adapters in list_adapters.values() for adapter in adapters]
if name not in list_adapters:
shared.log.error(f'Network load: type=LoRA name="{name}" adapters={list_adapters} not loaded')
else:
diffuser_loaded.append(name)
diffuser_scales.append(lora_scale)
net = network.Network(name, network_on_disk)
net.mtime = os.path.getmtime(network_on_disk.filename)
l.timer.activate += time.time() - t0
return net
modules/lora/lora_load.py
+13 -48
View File
@@ -3,12 +3,10 @@ import os
import time
import concurrent
from modules import shared, errors, sd_models, sd_models_compile, files_cache
from modules.lora import network, lora_overrides, lora_convert
from modules.lora import network, lora_overrides, lora_convert, lora_diffusers
from modules.lora import lora_common as l
diffuser_loaded = []
diffuser_scales = []
lora_cache = {}
available_networks = {}
available_network_aliases = {}
@@ -17,41 +15,6 @@ available_network_hash_lookup = {}
dump_lora_keys = os.environ.get('SD_LORA_DUMP', None) is not None
def load_diffusers(name, network_on_disk, lora_scale=shared.opts.extra_networks_default_multiplier) -> Union[network.Network, None]:
t0 = time.time()
name = name.replace(".", "_")
sd_model = getattr(shared.sd_model, "pipe", shared.sd_model)
shared.log.debug(f'Network load: type=LoRA name="{name}" file="{network_on_disk.filename}" detected={network_on_disk.sd_version} method=diffusers scale={lora_scale} fuse={shared.opts.lora_fuse_diffusers}')
if not hasattr(sd_model, 'load_lora_weights'):
shared.log.error(f'Network load: type=LoRA class={sd_model.__class__} does not implement load lora')
return None
try:
sd_model.load_lora_weights(network_on_disk.filename, adapter_name=name)
except Exception as e:
if 'already in use' in str(e):
pass
else:
if 'The following keys have not been correctly renamed' in str(e):
shared.log.error(f'Network load: type=LoRA name="{name}" diffusers unsupported format')
else:
shared.log.error(f'Network load: type=LoRA name="{name}" {e}')
if l.debug:
errors.display(e, "LoRA")
return None
if name not in diffuser_loaded:
list_adapters = sd_model.get_list_adapters()
list_adapters = [adapter for adapters in list_adapters.values() for adapter in adapters]
if name not in list_adapters:
shared.log.error(f'Network load: type=LoRA name="{name}" adapters={list_adapters} not loaded')
else:
diffuser_loaded.append(name)
diffuser_scales.append(lora_scale)
net = network.Network(name, network_on_disk)
net.mtime = os.path.getmtime(network_on_disk.filename)
l.timer.activate += time.time() - t0
return net
def lora_dump(lora, dct):
import tempfile
sd_model = getattr(shared.sd_model, "pipe", shared.sd_model)
@@ -73,7 +36,7 @@ def lora_dump(lora, dct):
f.write(line + "\n")
def load_safetensors(name, network_on_disk) -> Union[network.Network, None]:
def load_safetensors(name, network_on_disk: network.NetworkOnDisk) -> Union[network.Network, None]:
if not shared.sd_loaded:
return None
@@ -261,15 +224,15 @@ def gather_networks(names):
return networks_on_disk
def network_load(names, te_multipliers=None, unet_multipliers=None, dyn_dims=None):
def network_load(names, te_multipliers=None, unet_multipliers=None, dyn_dims=None, lora_modules=None):
networks_on_disk = gather_networks(names)
failed_to_load_networks = []
recompile_model, skip_lora_load = maybe_recompile_model(names, te_multipliers)
sd_model = getattr(shared.sd_model, "pipe", shared.sd_model)
l.loaded_networks.clear()
diffuser_loaded.clear()
diffuser_scales.clear()
lora_diffusers.diffuser_loaded.clear()
lora_diffusers.diffuser_scales.clear()
t0 = time.time()
for i, (network_on_disk, name) in enumerate(zip(networks_on_disk, names)):
@@ -279,11 +242,13 @@ def network_load(names, te_multipliers=None, unet_multipliers=None, dyn_dims=Non
if l.debug:
shared.log.debug(f'Network load: type=LoRA name="{name}" file="{network_on_disk.filename}" hash="{shorthash}"')
try:
lora_scale = te_multipliers[i] if te_multipliers else shared.opts.extra_networks_default_multiplier
lora_module = lora_modules[i] if lora_modules and len(lora_modules) > i else None
if recompile_model:
shared.compiled_model_state.lora_model.append(f"{name}:{te_multipliers[i] if te_multipliers else shared.opts.extra_networks_default_multiplier}")
shared.compiled_model_state.lora_model.append(f"{name}:{lora_scale}")
lora_method = lora_overrides.get_method(shorthash)
if lora_method == 'diffusers':
net = load_diffusers(name, network_on_disk, lora_scale=te_multipliers[i] if te_multipliers else shared.opts.extra_networks_default_multiplier)
net = lora_diffusers.load_diffusers(name, network_on_disk, lora_scale, lora_module)
elif lora_method == 'nunchaku':
pass # handled directly from extra_networks_lora.load_nunchaku
else:
@@ -311,21 +276,21 @@ def network_load(names, te_multipliers=None, unet_multipliers=None, dyn_dims=Non
name = next(iter(lora_cache))
lora_cache.pop(name, None)
if not skip_lora_load and len(diffuser_loaded) > 0:
shared.log.debug(f'Network load: type=LoRA loaded={diffuser_loaded} available={sd_model.get_list_adapters()} active={sd_model.get_active_adapters()} scales={diffuser_scales}')
if not skip_lora_load and len(lora_diffusers.diffuser_loaded) > 0:
shared.log.debug(f'Network load: type=LoRA loaded={lora_diffusers.diffuser_loaded} available={sd_model.get_list_adapters()} active={sd_model.get_active_adapters()} scales={lora_diffusers.diffuser_scales}')
try:
t1 = time.time()
if l.debug:
shared.log.trace(f'Network load: type=LoRA list={sd_model.get_list_adapters()}')
shared.log.trace(f'Network load: type=LoRA active={sd_model.get_active_adapters()}')
sd_model.set_adapters(adapter_names=diffuser_loaded, adapter_weights=diffuser_scales)
sd_model.set_adapters(adapter_names=lora_diffusers.diffuser_loaded, adapter_weights=lora_diffusers.diffuser_scales)
except Exception as e:
shared.log.error(f'Network load: type=LoRA action=set {e}')
if l.debug:
errors.display(e, 'LoRA')
try:
if shared.opts.lora_fuse_diffusers and not lora_overrides.disable_fuse():
sd_model.fuse_lora(adapter_names=diffuser_loaded, lora_scale=1.0, fuse_unet=True, fuse_text_encoder=True) # diffusers with fuse uses fixed scale since later apply does the scaling
sd_model.fuse_lora(adapter_names=lora_diffusers.diffuser_loaded, lora_scale=1.0, fuse_unet=True, fuse_text_encoder=True) # diffusers with fuse uses fixed scale since later apply does the scaling
sd_model.unload_lora_weights()
l.timer.activate += time.time() - t1
except Exception as e:
modules/lora/lora_nunchaku.py
+4 -1
View File
@@ -15,6 +15,9 @@ def load_nunchaku(names, strengths):
is_changed = loras != previously_loaded
if not is_changed:
return False
if not hasattr(shared.sd_model, 'transformer') or not hasattr(shared.sd_model.transformer, 'update_lora_params'):
shared.log.error(f'Network load: type=LoRA method=nunchaku model={shared.sd_model.__class__.__name__} unsupported')
return False
previously_loaded = loras
try:
@@ -27,7 +30,7 @@ def load_nunchaku(names, strengths):
lora_common.timer.load = t1 - t0
shared.log.debug(f"Network load: type=LoRA method=nunchaku loras={names} strength={strengths} time={t1-t0:.3f}")
except Exception as e:
shared.log.errors(f'Network load: type=LoRA method=nunchaku {e}')
shared.log.error(f'Network load: type=LoRA method=nunchaku {e}')
if lora_common.debug:
errors.display(e, 'LoRA')
return is_changed
modules/lora/lora_overrides.py
+3 -1
View File
@@ -61,7 +61,9 @@ def get_method(shorthash=''):
use_diffusers = use_diffusers or any(x.startswith(shorthash) for x in maybe_diffusers)
if shared.opts.lora_force_diffusers and len(shorthash) > 4:
use_diffusers = use_diffusers or any(x.startswith(shorthash) for x in force_diffusers)
use_nunchaku = hasattr(shared.sd_model, 'transformer') and 'Nunchaku' in shared.sd_model.transformer.__class__.__name__
nunchaku_dit = hasattr(shared.sd_model, 'transformer') and 'Nunchaku' in shared.sd_model.transformer.__class__.__name__
nunchaku_unet = hasattr(shared.sd_model, 'unet') and 'Nunchaku' in shared.sd_model.unet.__class__.__name__
use_nunchaku = nunchaku_dit or nunchaku_unet
if use_nunchaku:
return 'nunchaku'
elif use_diffusers:
modules/ltx/ltx_ui.py
+17
View File
@@ -3,17 +3,34 @@ import gradio as gr
from modules import shared, ui_sections, ui_symbols, ui_common
from modules.ui_components import ToolButton
from modules.video_models.video_utils import get_codecs
from modules.video_models.models_def import models
from modules.ltx import ltx_process
debug = shared.log.trace if os.environ.get('SD_VIDEO_DEBUG', None) is not None else lambda *args, **kwargs: None
def load_model(model_name):
if model_name is None or model_name == 'None':
shared.log.info('LTX model unload')
from modules import sd_models
sd_models.unload_model_weights()
return
else:
model_instance = [m for m in models['LTX Video'] if m.name == model_name][0]
from modules.video_models import video_load
video_load.load_model(model_instance)
def create_ui(prompt, negative, styles, overrides):
with gr.Row():
with gr.Column(variant='compact', elem_id="ltx_settings", elem_classes=['settings-column'], scale=1):
with gr.Row():
generate = gr.Button('Generate', elem_id="ltx_generate_btn", variant='primary', visible=False)
with gr.Row():
ltx_models = [m.name for m in models['LTX Video']]
model = gr.Dropdown(label='LTX model', choices=ltx_models, value=ltx_models[0])
model.change(fn=load_model, inputs=[model], outputs=[], show_progress=True)
with gr.Accordion(open=True, label="LTX size", elem_id='ltx_generate_accordion'):
with gr.Row():
width, height = ui_sections.create_resolution_inputs('ltx', default_width=832, default_height=480)
modules/memstats.py
+19 -2
View File
@@ -54,11 +54,28 @@ def ram_stats():
ram_total = 100 * res.rss / process.memory_percent()
ram_total = min(ram_total, get_docker_limit(), get_runpod_limit())
ram['total'] = gb(ram_total)
ram['used'] = gb(res.rss)
ram['free'] = round(ram['total'] - ram['used'])
ram['rss'] = gb(res.rss)
except Exception as e:
ram['total'] = 0
ram['rss'] = 0
ram['error'] = str(e)
if not fail_once:
shared.log.error(f'RAM stats: {e}')
errors.display(e, 'RAM stats')
fail_once = True
try:
vmem = psutil.virtual_memory()
ram['used'] = gb(vmem.used) if hasattr(vmem, 'used') else 0
ram['free'] = gb(vmem.free) if hasattr(vmem, 'free') else 0
ram['avail'] = gb(vmem.available) if hasattr(vmem, 'available') else 0
ram['buffers'] = gb(vmem.buffers) if hasattr(vmem, 'buffers') else 0
ram['cached'] = gb(vmem.cached) if hasattr(vmem, 'cached') else 0
except Exception as e:
ram['used'] = 0
ram['free'] = 0
ram['avail'] = 0
ram['buffers'] = 0
ram['cached'] = 0
ram['error'] = str(e)
if not fail_once:
shared.log.error(f'RAM stats: {e}')
modules/model_quant.py
-5
View File
@@ -501,11 +501,6 @@ def sdnq_quantize_model(model, op=None, sd_model=None, do_gc: bool = True, weigh
modules_to_not_convert.extend(model._skip_layerwise_casting_patterns) # pylint: disable=protected-access
if model.__class__.__name__ == "ChromaTransformer2DModel":
modules_to_not_convert.append("distilled_guidance_layer")
if model.__class__.__name__ == "QwenImageTransformer2DModel":
if "minimum_6bit" not in modules_dtype_dict.keys():
modules_dtype_dict["minimum_6bit"] = ["img_mod", "pos_embed", "time_text_embed", "img_in", "txt_in", "norm_out"]
else:
modules_dtype_dict["minimum_6bit"].extend(["img_mod", "pos_embed", "time_text_embed", "img_in", "txt_in", "norm_out"])
sdnq_modules_to_not_convert = [m.strip() for m in re.split(';|,| ', shared.opts.sdnq_modules_to_not_convert) if len(m.strip()) > 1]
if len(sdnq_modules_to_not_convert) > 0:
modules/modular.py
+64
View File
@@ -0,0 +1,64 @@
import time
import diffusers
from modules import shared
modular_map= {
'StableDiffusionXLPipeline': 'StableDiffusionXLAutoBlocks',
'StableDiffusionXLImg2ImgPipeline': 'StableDiffusionXLAutoBlocks',
'StableDiffusionXLInpaintPipeline': 'StableDiffusionXLAutoBlocks',
'FluxPipeline': 'FluxAutoBlocks',
'FluxImg2ImgPipeline': 'FluxAutoBlocks',
'FluxInpaintPipeline': 'FluxAutoBlocks',
'WanPipeline': 'WanAutoBlocks',
'WanImageToVideoPipeline': 'WanAutoBlocks',
'QwenImagePipeline': 'QwenImageAutoBlocks',
'QwenImageEditPipeline': 'QwenImageEditAutoBlocks',
}
def is_compatible(diffusion_pipeline: diffusers.DiffusionPipeline) -> bool:
if not shared.opts.model_modular_enable:
return False
compatible = diffusion_pipeline.__class__.__name__ in modular_map
if not compatible:
shared.log.debug(f'Modular: source={diffusion_pipeline.__class__.__name__} incompatible pipeline')
return compatible
def convert_to_modular(diffusion_pipeline: diffusers.DiffusionPipeline) -> diffusers.ModularPipeline:
modular_pipe = None
try:
t0 = time.time()
modular_cls = modular_map.get(diffusion_pipeline.__class__.__name__, None)
if modular_cls is None:
raise ValueError(f'unknown: cls={diffusion_pipeline.__class__.__name__}')
modular_cls = getattr(diffusers, modular_cls, None)
if modular_cls is None:
raise ValueError(f'invalid: cls={diffusion_pipeline.__class__.__name__}')
modular_blocks = modular_cls()
modular_pipe = modular_blocks.init_pipeline()
components_dct = {k: v for k, v in diffusion_pipeline.components.items() if v is not None}
modular_pipe.update_components(**components_dct, **diffusion_pipeline.parameters)
modular_pipe.original_pipe = diffusion_pipeline
t1 = time.time()
shared.log.debug(f'Modular: source={diffusion_pipeline.__class__.__name__} target={modular_pipe.__class__.__name__} time={t1 - t0:.2f}')
"""
for expected_input_param in modular_pipe.blocks.inputs:
name = expected_input_param.name
default = expected_input_param.default
kwargs_type = expected_input_param.kwargs_type
shared.log.trace(f'Modular input: name={name} type={kwargs_type} default={default}')
"""
except Exception as e:
shared.log.error(f'Modular: {e}')
raise e
return modular_pipe
def restore_standard(modular_pipe):
if hasattr(modular_pipe, 'original_pipe'):
shared.log.debug(f'Modular: source={modular_pipe.__class__.__name__} target={modular_pipe.original_pipe.__class__.__name__}')
return modular_pipe.original_pipe
return modular_pipe
modules/modular_guiders.py
+87
View File
@@ -0,0 +1,87 @@
import diffusers
from modules import shared, errors, processing
# ['Default', 'CFG', 'Zero', 'PAG', 'APG', 'SLG', 'SEG', 'TCFG', 'FDG']
guiders = {
# 'None': { 'cls': None, 'args': {}, },
'Default': { 'cls': None, 'args': {}, },
'CFG: ClassifierFreeGuidance': { 'cls': diffusers.ClassifierFreeGuidance, 'args': {} },
'Auto: AutoGuidance': { 'cls': diffusers.AutoGuidance, 'args': { 'dropout': 1.0, 'auto_guidance_layers': [7, 8, 9], 'auto_guidance_config': None } },
'Zero: ClassifierFreeZeroStar': { 'cls': diffusers.ClassifierFreeZeroStarGuidance, 'args': { 'zero_init_steps': 1 } },
'PAG: PerturbedAttentionGuidance': { 'cls': diffusers.PerturbedAttentionGuidance, 'args': { 'perturbed_guidance_scale': 2.8, 'perturbed_guidance_start': 0.01, 'perturbed_guidance_stop': 0.2, 'perturbed_guidance_layers': [7, 8, 9], 'perturbed_guidance_config': None } },
'APG: AdaptiveProjectedGuidance': { 'cls': diffusers.AdaptiveProjectedGuidance, 'args': { 'adaptive_projected_guidance_momentum': -1, 'adaptive_projected_guidance_rescale': 15.0 } },
'SLG: SkipLayerGuidance': { 'cls': diffusers.SkipLayerGuidance, 'args': { 'skip_layer_guidance_scale': 2.8, 'skip_layer_guidance_start': 0.01, 'skip_layer_guidance_stop': 0.2, 'skip_layer_guidance_layers': [7, 8, 9], 'skip_layer_config': None } },
'SEG: SmoothedEnergyGuidance': { 'cls': diffusers.SmoothedEnergyGuidance, 'args': { 'seg_guidance_scale': 3.0, 'seg_blur_sigma': 9999999.0, 'seg_blur_threshold_inf': 9999.0, 'seg_guidance_start': 0.0, 'seg_guidance_stop': 1.0, 'seg_guidance_layers': [7, 8, 9], 'seg_guidance_config': None } },
'TCFG: TangentialClassifierFreeGuidance': { 'cls': diffusers.TangentialClassifierFreeGuidance, 'args': {} },
'FDG: FrequencyDecoupledGuidance': { 'cls': diffusers.FrequencyDecoupledGuidance, 'args': { 'guidance_scales': [10.0, 5.0], 'parallel_weights': 1.0, 'guidance_rescale_space': "data" } },
}
base_args = {
'guidance_scale': 6.0,
'guidance_rescale': 0.0,
'start': 0.0,
'stop': 1.0,
}
def set_guider(p: processing.StableDiffusionProcessing):
guidance_name = p.guidance_name or 'Default'
if guidance_name not in guiders:
return
if guidance_name == 'Default':
if hasattr(shared.sd_model, 'default_guider'):
guider_info = shared.sd_model.default_guider
shared.sd_model.update_components(guider=guider_info)
else:
guider_info = shared.sd_model.get_component_spec("guider")
shared.sd_model.default_guider = guider_info
guider_cls = guider_info.type_hint
if guider_info is not None and guider_cls is not None and guider_info.config is not None:
guider_args = {k: v for k, v in guider_info.config.items() if not k.startswith('_') and v is not None}
else:
guider_args = {}
shared.log.info(f'Guider: name={guidance_name} cls={guider_cls.__name__} args={guider_args}')
return
if guidance_name == 'None':
shared.sd_model.update_components(guider=None) # breaks the pipeline
shared.log.info(f'Guider: name={guidance_name}')
return
guider_info = guiders[guidance_name]
guider_cls = guider_info['cls']
guider_args = {}
for k, v in base_args.items():
if v is not None and v >= 0.0:
guider_args[k] = v
shared.log.warning('Guiders: partially implemented') # TODO: guiders
for k, v in guider_info['args'].items():
try:
if k is None:
pass
elif k.endswith('_layers') and isinstance(v, str):
guider_args[k] = [int(x.strip()) for x in v.split(',') if x.strip().isdigit()]
elif k.endswith('_config'):
# if lsc_enabled
# guider_args[k] = diffusers.LayerSkipConfig(...)
pass
elif isinstance(v, list) and len(v) > 0:
guider_args[k] = v
elif isinstance(v, int) and (v >= 0):
guider_args[k] = int(v)
elif isinstance(v, float) and (v >= 0.0):
guider_args[k] = float(v)
elif isinstance(v, str) and (len(v) > 0):
guider_args[k] = v
except Exception as e:
shared.log.error(f'Guiders: arg={k} value={v} error={e}')
errors.display(e, 'Guiders')
# guider_args.update(guider_info['args'])
if guider_cls is not None:
try:
guider_instance = guider_cls(**guider_args)
shared.log.info(f'Guider: name={guidance_name} cls={guider_cls.__name__} args={guider_args}')
shared.sd_model.update_components(guider=guider_instance)
except Exception as e:
shared.log.error(f'Guider: name={guidance_name} cls={guider_cls.__name__} args={guider_args} {e}')
return
modules/processing.py
+10 -3
View File
@@ -130,7 +130,7 @@ def get_processed(*args, **kwargs):
def process_images(p: StableDiffusionProcessing) -> Processed:
timer.process.reset()
debug(f'Process images: {vars(p)}')
debug(f'Process images: class={p.__class__.__name__} {vars(p)}')
if not hasattr(p.sd_model, 'sd_checkpoint_info'):
shared.log.error('Processing: incomplete model')
return None
@@ -427,13 +427,20 @@ def process_images_inner(p: StableDiffusionProcessing) -> Processed:
for script_image, script_infotext in zip(results.images, results.infotexts):
output_images.append(script_image)
infotexts.append(script_infotext)
if samples is None:
from modules.processing_diffusers import process_diffusers
samples = process_diffusers(p)
timer.process.record('process')
if not shared.opts.keep_incomplete and shared.state.interrupted:
samples = []
if shared.state.interrupted:
shared.log.debug(f'Process: batch={n+1}/{p.n_iter} interrupted')
p.do_not_save_samples = not shared.opts.keep_incomplete
if shared.state.current_image is not None and isinstance(shared.state.current_image, Image.Image):
samples = [shared.state.current_image]
infotexts = [create_infotext(p, p.all_prompts, p.all_seeds, p.all_subseeds, index=0)]
else:
samples = []
if p.scripts is not None and isinstance(p.scripts, scripts_manager.ScriptRunner):
p.scripts.postprocess_batch(p, samples, batch_number=n)
modules/processing_args.py
+54 -9
View File
@@ -18,6 +18,26 @@ debug_log = shared.log.trace if debug_enabled else lambda *args, **kwargs: None
disable_pbar = os.environ.get('SD_DISABLE_PBAR', None) is not None
def task_modular_kwargs(p, model): # pylint: disable=unused-argument
# model_cls = model.__class__.__name__
task_args = {}
p.ops.append('modular')
processing_helpers.resize_init_images(p)
task_args['width'] = p.width
task_args['height'] = p.height
if len(getattr(p, 'init_images', [])) > 0:
task_args['image'] = p.init_images
task_args['strength'] = p.denoising_strength
mask_image = p.task_args.get('image_mask', None) or getattr(p, 'image_mask', None) or getattr(p, 'mask', None)
if mask_image is not None:
task_args['mask_image'] = mask_image
if debug_enabled:
debug_log(f'Process task specific args: {task_args}')
return task_args
def task_specific_kwargs(p, model):
model_cls = model.__class__.__name__
vae_scale_factor = sd_vae.get_vae_scale_factor(model)
@@ -103,9 +123,9 @@ def task_specific_kwargs(p, model):
}
# model specific args
if model_cls == 'QwenImageEditPipeline' and len(getattr(p, 'init_images', [])) == 0:
if 'QwenImageEdit' in model_cls and len(getattr(p, 'init_images', [])) == 0:
task_args['image'] = [Image.new('RGB', (p.width, p.height), (0, 0, 0))] # monkey-patch so qwen-image-edit pipeline does not error-out on t2i
if model_cls == 'LatentConsistencyModelPipeline' and hasattr(p, 'init_images') and len(p.init_images) > 0:
if 'LatentConsistencyModelPipeline' in model_cls and hasattr(p, 'init_images') and len(p.init_images) > 0:
p.ops.append('lcm')
init_latents = [processing_vae.vae_encode(image, model=shared.sd_model, vae_type=p.vae_type).squeeze(dim=0) for image in p.init_images]
init_latent = torch.stack(init_latents, dim=0).to(shared.device)
@@ -116,7 +136,7 @@ def task_specific_kwargs(p, model):
'width': p.width if hasattr(p, 'width') else None,
'height': p.height if hasattr(p, 'height') else None,
}
if model_cls == 'BlipDiffusionPipeline':
if 'BlipDiffusionPipeline' in model_cls:
if len(getattr(p, 'init_images', [])) == 0:
shared.log.error('BLiP diffusion requires init image')
return task_args
@@ -126,14 +146,26 @@ def task_specific_kwargs(p, model):
'target_subject_category': getattr(p, 'prompt', '').split()[-1],
'output_type': 'pil',
}
if model.__class__.__name__ == 'WanImageToVideoPipeline' and hasattr(p, 'init_images') and len(p.init_images) > 0:
if ('WanImageToVideoPipeline' in model_cls) and (getattr(p, 'init_images', None) is not None) and (len(p.init_images) > 0):
task_args['image'] = p.init_images[0]
if ('WanVACEPipeline' in model_cls) and (getattr(p, 'init_images', None) is not None) and (len(p.init_images) > 0):
task_args['reference_images'] = p.init_images
if debug_enabled:
debug_log(f'Process task specific args: {task_args}')
return task_args
def get_params(model):
if hasattr(model, 'blocks') and hasattr(model.blocks, 'inputs'): # modular pipeline
possible = [input_param.name for input_param in model.blocks.inputs]
return possible
else:
signature = inspect.signature(type(model).__call__, follow_wrapped=True)
possible = list(signature.parameters)
return possible
def set_pipeline_args(p, model, prompts:list, negative_prompts:list, prompts_2:typing.Optional[list]=None, negative_prompts_2:typing.Optional[list]=None, prompt_attention:typing.Optional[str]=None, desc:typing.Optional[str]='', **kwargs):
t0 = time.time()
shared.sd_model = sd_models.apply_balanced_offload(shared.sd_model)
@@ -149,8 +181,8 @@ def set_pipeline_args(p, model, prompts:list, negative_prompts:list, prompts_2:t
model.set_progress_bar_config(bar_format='Progress {rate_fmt}{postfix} {bar} {percentage:3.0f}% {n_fmt}/{total_fmt} {elapsed} {remaining} ' + '\x1b[38;5;71m' + desc, ncols=80, colour='#327fba', disable=disable_pbar)
else:
model.set_progress_bar_config(bar_format='Progress {rate_fmt}{postfix} {bar} {percentage:3.0f}% {n_fmt}/{total_fmt} {elapsed} {remaining} ' + '\x1b[38;5;71m' + desc, ncols=80, colour='#327fba')
signature = inspect.signature(type(model).__call__, follow_wrapped=True)
possible = list(signature.parameters)
possible = get_params(model)
if debug_enabled:
debug_log(f'Process pipeline possible: {possible}')
@@ -307,6 +339,14 @@ def set_pipeline_args(p, model, prompts:list, negative_prompts:list, prompts_2:t
args['control_strength'] = p.denoising_strength
args['width'] = p.width
args['height'] = p.height
if 'WanVACEPipeline' in model.__class__.__name__:
if isinstance(args['prompt'], list):
args['prompt'] = args['prompt'][0] if len(args['prompt']) > 0 else ''
if isinstance(args.get('negative_prompt', None), list):
args['negative_prompt'] = args['negative_prompt'][0] if len(args['negative_prompt']) > 0 else ''
if isinstance(args['generator'], list) and len(args['generator']) > 0:
args['generator'] = args['generator'][0]
# set callbacks
if 'prior_callback_steps' in possible: # Wuerstchen / Cascade
args['prior_callback_steps'] = 1
@@ -347,7 +387,11 @@ def set_pipeline_args(p, model, prompts:list, negative_prompts:list, prompts_2:t
args[arg] = kwargs[arg]
# handle task specific args
task_kwargs = task_specific_kwargs(p, model)
if sd_models.get_diffusers_task(model) == sd_models.DiffusersTaskType.MODULAR:
task_kwargs = task_modular_kwargs(p, model)
else:
task_kwargs = task_specific_kwargs(p, model)
pipe_args = getattr(p, 'task_args', {})
model_args = getattr(model, 'task_args', {})
task_kwargs.update(pipe_args or {})
@@ -397,8 +441,9 @@ def set_pipeline_args(p, model, prompts:list, negative_prompts:list, prompts_2:t
# handle implicit controlnet
if 'control_image' in possible and 'control_image' not in args and 'image' in args:
debug_log('Process: set control image')
args['control_image'] = args['image']
if sd_models.get_diffusers_task(model) != sd_models.DiffusersTaskType.MODULAR:
debug_log('Process: set control image')
args['control_image'] = args['image']
sd_hijack_hypertile.hypertile_set(p, hr=len(getattr(p, 'init_images', [])) > 0)
modules/processing_class.py
+20 -9
View File
@@ -36,7 +36,13 @@ class StableDiffusionProcessing:
sampler_name: str = None,
hr_sampler_name: str = None,
eta: float = None,
# guidance
# modular guidance
guidance_name: str = 'Default',
guidance_scale: float = 6.0,
guidance_rescale: float = 0.0,
guidance_start: float = 0.0,
guidance_stop: float = 1.0,
# legacy guidance
cfg_scale: float = 6.0,
cfg_end: float = 1,
diffusers_guidance_rescale: float = 0.0,
@@ -247,6 +253,11 @@ class StableDiffusionProcessing:
self.do_not_save_grid = do_not_save_grid
self.override_settings_restore_afterwards = override_settings_restore_afterwards
self.eta = eta
self.guidance_name = guidance_name
self.guidance_scale = guidance_scale
self.guidance_rescale = guidance_rescale
self.guidance_start = guidance_start
self.guidance_stop = guidance_stop
self.cfg_scale = cfg_scale
self.cfg_end = cfg_end
self.diffusers_guidance_rescale = diffusers_guidance_rescale
@@ -445,14 +456,14 @@ class StableDiffusionProcessingTxt2Img(StableDiffusionProcessing):
self.hr_upscale_to_y = int(self.height * scale)
else:
if self.hr_resize_y == 0:
self.hr_upscale_to_x = self.hr_resize_x
self.hr_upscale_to_y = self.hr_resize_x * self.height // self.width
self.hr_upscale_to_x = int(self.hr_resize_x)
self.hr_upscale_to_y = int(self.hr_resize_x * self.height // self.width)
elif self.hr_resize_x == 0:
self.hr_upscale_to_x = self.hr_resize_y * self.width // self.height
self.hr_upscale_to_y = self.hr_resize_y
self.hr_upscale_to_x = int(self.hr_resize_y * self.width // self.height)
self.hr_upscale_to_y = int(self.hr_resize_y)
elif self.hr_resize_x > 0 and self.hr_resize_y > 0:
self.hr_upscale_to_x = self.hr_resize_x
self.hr_upscale_to_y = self.hr_resize_y
self.hr_upscale_to_x = int(self.hr_resize_x)
self.hr_upscale_to_y = int(self.hr_resize_y)
shared.log.debug(f'Init hires: upscaler="{self.hr_upscaler}" sampler="{self.hr_sampler_name}" resize={self.hr_resize_x}x{self.hr_resize_y} upscale={self.hr_upscale_to_x}x{self.hr_upscale_to_y}')
@@ -572,9 +583,9 @@ class StableDiffusionProcessingControl(StableDiffusionProcessingImg2Img):
self.hr_force = force
if use_scale:
vae_scale_factor = sd_vae.get_vae_scale_factor()
self.hr_upscale_to_x, self.hr_upscale_to_y = vae_scale_factor * int(self.width * scale / vae_scale_factor), vae_scale_factor * int(self.height * scale / vae_scale_factor)
self.hr_upscale_to_x, self.hr_upscale_to_y = int(vae_scale_factor * int(self.width * scale / vae_scale_factor)), int(vae_scale_factor * int(self.height * scale / vae_scale_factor))
else:
self.hr_upscale_to_x, self.hr_upscale_to_y = self.hr_resize_x, self.hr_resize_y
self.hr_upscale_to_x, self.hr_upscale_to_y = int(self.hr_resize_x), int(self.hr_resize_y)
def switch_class(p: StableDiffusionProcessing, new_class: type, dct: dict = None):
modules/processing_diffusers.py
+16 -5
View File
@@ -67,7 +67,7 @@ def restore_state(p: processing.StableDiffusionProcessing):
def process_pre(p: processing.StableDiffusionProcessing):
from modules import ipadapter, sd_hijack_freeu, para_attention, teacache, hidiffusion, ras, pag, cfgzero, transformer_cache, token_merge, linfusion
from modules import ipadapter, sd_hijack_freeu, para_attention, teacache, hidiffusion, ras, pag, cfgzero, transformer_cache, token_merge, linfusion, cachedit
shared.log.info('Processing modifiers: apply')
try:
@@ -80,6 +80,7 @@ def process_pre(p: processing.StableDiffusionProcessing):
pag.apply(p)
cfgzero.apply(p)
linfusion.apply(shared.sd_model)
cachedit.apply_cache_dit(shared.sd_model)
# apply-only
sd_hijack_freeu.apply_freeu(p)
@@ -95,11 +96,20 @@ def process_pre(p: processing.StableDiffusionProcessing):
# sd_models.move_model(shared.sd_model.unet, devices.device)
# if hasattr(shared.sd_model, 'transformer'):
# sd_models.move_model(shared.sd_model.transformer, devices.device)
from modules import modular
if modular.is_compatible(shared.sd_model):
modular_pipe = modular.convert_to_modular(shared.sd_model)
if modular_pipe is not None:
shared.sd_model = modular_pipe
from modules import modular_guiders
modular_guiders.set_guider(p)
timer.process.record('pre')
def process_post(p: processing.StableDiffusionProcessing):
from modules import ipadapter, hidiffusion, ras, pag, cfgzero, token_merge, linfusion
from modules import ipadapter, hidiffusion, ras, pag, cfgzero, token_merge, linfusion, cachedit
shared.log.info('Processing modifiers: unapply')
try:
@@ -111,6 +121,7 @@ def process_post(p: processing.StableDiffusionProcessing):
pag.unapply()
cfgzero.unapply()
linfusion.unapply(shared.sd_model)
cachedit.unapply_cache_dir(shared.sd_model)
except Exception as e:
shared.log.error(f'Processing unapply: {e}')
errors.display(e, 'unapply')
@@ -229,8 +240,8 @@ def process_hires(p: processing.StableDiffusionProcessing, output):
p.hr_upscaler = p.resize_name
p.hr_resize_mode = p.resize_mode
p.hr_resize_context = p.resize_context
p.hr_upscale_to_x = p.width * p.hr_scale if p.hr_resize_x == 0 else p.hr_resize_x
p.hr_upscale_to_y = p.height * p.hr_scale if p.hr_resize_y == 0 else p.hr_resize_y
p.hr_upscale_to_x = int(p.width * p.hr_scale) if p.hr_resize_x == 0 else p.hr_resize_x
p.hr_upscale_to_y = int(p.height * p.hr_scale) if p.hr_resize_y == 0 else p.hr_resize_y
# hires runs on original pipeline
if hasattr(shared.sd_model, 'restore_pipeline') and (shared.sd_model.restore_pipeline is not None) and (not shared.opts.control_hires):
@@ -487,7 +498,7 @@ def validate_pipeline(p: processing.StableDiffusionProcessing):
if m.repo_cls is not None:
models_cls.append(m.repo_cls.__name__)
is_video_model = shared.sd_model.__class__.__name__ in models_cls
override_video_pipelines = ['WanPipeline', 'WanImageToVideoPipeline']
override_video_pipelines = ['WanPipeline', 'WanImageToVideoPipeline', 'WanVACEPipeline']
is_video_pipeline = ('video' in p.__class__.__name__.lower()) or (shared.sd_model.__class__.__name__ in override_video_pipelines)
if is_video_model and not is_video_pipeline:
shared.log.error(f'Mismatch: type={shared.sd_model_type} cls={shared.sd_model.__class__.__name__} request={p.__class__.__name__} video model with non-video pipeline')
modules/processing_helpers.py
+44 -6
View File
@@ -9,6 +9,7 @@ import cv2
from PIL import Image
from blendmodes.blend import blendLayers, BlendType
from modules import shared, devices, images, sd_models, sd_samplers, sd_vae, sd_hijack_hypertile, processing_vae, timer
from modules.api import helpers
debug = shared.log.trace if os.environ.get('SD_PROCESS_DEBUG', None) is not None else lambda *args, **kwargs: None
@@ -16,6 +17,10 @@ debug_steps = shared.log.trace if os.environ.get('SD_STEPS_DEBUG', None) is not
debug_steps('Trace: STEPS')
def is_modular():
return sd_models.get_diffusers_task(shared.sd_model) == sd_models.DiffusersTaskType.MODULAR
def is_txt2img():
return sd_models.get_diffusers_task(shared.sd_model) == sd_models.DiffusersTaskType.TEXT_2_IMAGE
@@ -275,10 +280,38 @@ def validate_sample(tensor):
return cast
def decode_images(image):
if isinstance(image, list):
decoded = []
for i, img in enumerate(image):
if isinstance(img, str):
try:
decoded.append(helpers.decode_base64_to_image(img, quiet=True))
except Exception as e:
shared.log.error(f'Decode image[{i}]: {e}')
elif isinstance(img, Image.Image):
decoded.append(img)
else:
shared.log.error(f'Decode image[{i}]: {type(img)} unknown type')
return decoded
elif isinstance(image, str):
try:
return helpers.decode_base64_to_image(image, quiet=True)
except Exception as e:
shared.log.error(f'Decode image: {e}')
elif isinstance(image, Image.Image):
return image
else:
shared.log.error(f'Decode image: {type(image)} unknown type')
return None
def resize_init_images(p):
if getattr(p, 'image', None) is not None and getattr(p, 'init_images', None) is None:
p.init_images = [p.image]
if getattr(p, 'init_images', None) is not None and len(p.init_images) > 0:
p.init_images = decode_images(p.init_images)
vae_scale_factor = sd_vae.get_vae_scale_factor()
tgt_width, tgt_height = vae_scale_factor * math.ceil(p.init_images[0].width / vae_scale_factor), vae_scale_factor * math.ceil(p.init_images[0].height / vae_scale_factor)
if p.init_images[0].size != (tgt_width, tgt_height):
@@ -287,11 +320,14 @@ def resize_init_images(p):
p.height = tgt_height
p.width = tgt_width
sd_hijack_hypertile.hypertile_set(p)
if getattr(p, 'mask', None) is not None and p.mask.size != (tgt_width, tgt_height):
if getattr(p, 'mask', None) is not None and p.mask is not None and p.mask.size != (tgt_width, tgt_height):
p.mask = decode_images(p.mask)
p.mask = images.resize_image(1, p.mask, tgt_width, tgt_height, upscaler_name=None)
if getattr(p, 'init_mask', None) is not None and p.init_mask.size != (tgt_width, tgt_height):
if getattr(p, 'init_mask', None) is not None and p.init_mask is not None and p.init_mask.size != (tgt_width, tgt_height):
p.init_mask = decode_images(p.init_mask)
p.init_mask = images.resize_image(1, p.init_mask, tgt_width, tgt_height, upscaler_name=None)
if getattr(p, 'mask_for_overlay', None) is not None and p.mask_for_overlay.size != (tgt_width, tgt_height):
if getattr(p, 'mask_for_overlay', None) is not None and p.mask_for_overlay is not None and p.mask_for_overlay.size != (tgt_width, tgt_height):
p.mask_for_overlay = decode_images(p.mask_for_overlay)
p.mask_for_overlay = images.resize_image(1, p.mask_for_overlay, tgt_width, tgt_height, upscaler_name=None)
return tgt_width, tgt_height
return p.width, p.height
@@ -372,7 +408,9 @@ def calculate_base_steps(p, use_denoise_start, use_refiner_start):
if len(getattr(p, 'timesteps', [])) > 0:
return None
cls = shared.sd_model.__class__.__name__
if 'Flex' in cls or 'Kontext' in cls or 'Edit' in cls:
if 'Flex' in cls or 'Kontext' in cls or 'Edit' in cls or 'Wan' in cls:
steps = p.steps
elif is_modular():
steps = p.steps
elif not is_txt2img():
if cls in sd_models.i2i_pipes:
@@ -393,7 +431,7 @@ def calculate_base_steps(p, use_denoise_start, use_refiner_start):
def calculate_hires_steps(p):
cls = shared.sd_model.__class__.__name__
if 'Flex' in cls or 'HiDreamImageEditingPipeline' in cls or 'Kontext' in cls:
if 'Flex' in cls or 'Kontext' in cls or 'Edit' in cls or 'Wan' in cls:
steps = p.steps
elif p.hr_second_pass_steps > 0:
steps = (p.hr_second_pass_steps // p.denoising_strength) + 1
@@ -407,7 +445,7 @@ def calculate_hires_steps(p):
def calculate_refiner_steps(p):
cls = shared.sd_model.__class__.__name__
if 'Flex' in cls or 'HiDreamImageEditingPipeline' in cls or 'Kontext' in cls:
if 'Flex' in cls or 'Kontext' in cls or 'Edit' in cls or 'Wan' in cls:
steps = p.steps
elif "StableDiffusionXL" in shared.sd_refiner.__class__.__name__:
if p.refiner_start > 0 and p.refiner_start < 1:
modules/processing_info.py
+4 -4
View File
@@ -31,13 +31,13 @@ def create_infotext(p: StableDiffusionProcessing, all_prompts=None, all_seeds=No
if all_subseeds is None:
all_subseeds = p.all_subseeds or [p.subseed]
while len(all_prompts) <= index:
all_prompts.append(all_prompts[-1])
all_prompts.insert(0, p.prompt)
while len(all_seeds) <= index:
all_seeds.append(all_seeds[-1])
all_seeds.insert(0, int(p.seed))
while len(all_subseeds) <= index:
all_subseeds.append(all_subseeds[-1])
all_subseeds.insert(0, int(p.subseed))
while len(all_negative_prompts) <= index:
all_negative_prompts.append(all_negative_prompts[-1])
all_negative_prompts.insert(0, p.negative_prompt)
comment = ', '.join(comments) if comments is not None and type(comments) is list else None
ops = list(set(p.ops))
args = {
modules/processing_vae.py
+2 -2
View File
@@ -173,7 +173,7 @@ def full_vae_decode(latents, model):
log_debug(f'VAE memory: {shared.mem_mon.read()}')
vae_name = os.path.splitext(os.path.basename(sd_vae.loaded_vae_file))[0] if sd_vae.loaded_vae_file is not None else "default"
vae_scale_factor = sd_vae.get_vae_scale_factor(model)
shared.log.debug(f'Decode: vae="{vae_name}" scale={vae_scale_factor} upcast={upcast} slicing={getattr(model.vae, "use_slicing", None)} tiling={getattr(model.vae, "use_tiling", None)} latents={list(latents.shape)}:{latents.device}:{latents.dtype} time={t1-t0:.3f}')
shared.log.debug(f'Decode: vae="{vae_name}" scale={vae_scale_factor} upcast={upcast} slicing={getattr(model.vae, "use_slicing", None)} tiling={getattr(model.vae, "use_tiling", None)} latents={list(latents.shape)}:{latents.device} dtype={latents.dtype} time={t1-t0:.3f}')
return decoded
@@ -221,7 +221,7 @@ def taesd_vae_decode(latents):
else:
decoded = sd_vae_taesd.decode(latents)
t1 = time.time()
shared.log.debug(f'Decode: vae="taesd" latents={latents.shape}:{latents.dtype}:{latents.device} time={t1-t0:.3f}')
shared.log.debug(f'Decode: vae="taesd" latents={latents.shape}:{latents.device} dtype={latents.dtype} time={t1-t0:.3f}')
return decoded
modules/rocm.py
+176 -78
View File
@@ -3,9 +3,9 @@ import sys
import ctypes
import shutil
import subprocess
import importlib.metadata
from typing import Union, List
from enum import Enum
from functools import wraps
def resolve_link(path_: str) -> str:
@@ -29,15 +29,30 @@ def load_library_global(path_: str):
ctypes.CDLL(path_, mode=ctypes.RTLD_GLOBAL)
def conceal():
os.environ.pop("ROCM_HOME", None)
os.environ.pop("ROCM_PATH", None)
paths = os.environ["PATH"].split(";")
paths_no_rocm = []
for path_ in paths:
if "rocm" not in path_.lower():
paths_no_rocm.append(path_)
os.environ["PATH"] = ";".join(paths_no_rocm)
class Environment:
pass
# rocm is installed system-wide
class ROCmEnvironment(Environment):
path: str
def __init__(self, path: str):
self.path = path
# rocm-sdk package is installed
class PythonPackageEnvironment(Environment):
hip: ctypes.CDLL
def __init__(self):
import _rocm_sdk_core
if sys.platform == "win32":
path = os.path.join(_rocm_sdk_core.__path__[0], "bin", "amdhip64_7.dll")
else:
raise NotImplementedError
# This library will be loaded/used by PyTorch. So it won't make conflicts.
self.hip = ctypes.CDLL(path)
class MicroArchitecture(Enum):
@@ -80,8 +95,22 @@ class Agent:
self.is_apu = (self.gfx_version & 0xFFF0 == 0x1150) or self.gfx_version in (0x801, 0x902, 0x90c, 0x1013, 0x1033, 0x1035, 0x1036, 0x1103,)
self.blaslt_supported = os.path.exists(os.path.join(blaslt_tensile_libpath, f"Kernels.so-000-{name}.hsaco" if sys.platform == "win32" else f"extop_{name}.co"))
@property
def therock(self) -> str:
if (self.gfx_version & 0xFFF0) == 0x1100:
return "gfx110X-dgpu"
if self.gfx_version == 0x1151:
return "gfx1151"
if (self.gfx_version & 0xFFF0) == 0x1200:
return "gfx120X-all"
if (self.gfx_version & 0xFFF0) == 0x940:
return "gfx94X-dcgpu"
if self.gfx_version == 0x950:
return "gfx950-dcgpu"
raise RuntimeError(f"Unsupported GPU architecture: {self.name}")
def get_gfx_version(self) -> Union[str, None]:
if self.gfx_version >= 0x1101 and self.gfx_version < 0x1200:
if self.gfx_version >= 0x1100 and self.gfx_version < 0x1200:
return "11.0.0"
elif self.gfx_version != 0x1030 and self.gfx_version >= 0x1000 and self.gfx_version < 0x1100:
# gfx1010 users had to override gfx version to 10.3.0 in Linux
@@ -90,26 +119,22 @@ class Agent:
return None
def get_version_torch() -> Union[str, None]:
version_ = None
try:
version_ = importlib.metadata.version("torch")
except importlib.metadata.PackageNotFoundError:
return None
if "+rocm" not in version_: # unofficial build, non-rocm torch.
return None
return version_.split("+rocm")[1]
def find() -> Union[Environment, None]:
try: # TheRock
import _rocm_sdk_core # pylint: disable=unused-import
return PythonPackageEnvironment()
except ImportError:
pass
# system-wide installation
hip_path = shutil.which("hipconfig")
if hip_path is not None:
return ROCmEnvironment(dirname(resolve_link(hip_path), 2))
if sys.platform == "win32":
def find() -> Union[str, None]:
hip_path = shutil.which("hipconfig")
if hip_path is not None:
return dirname(resolve_link(hip_path), 2)
if sys.platform == "win32":
hip_path = os.environ.get("HIP_PATH", None)
if hip_path is not None:
return hip_path
return ROCmEnvironment(hip_path)
program_files = os.environ.get('ProgramFiles', r'C:\Program Files')
hip_path = rf'{program_files}\AMD\ROCm'
@@ -146,29 +171,111 @@ if sys.platform == "win32":
if latest is None:
return None
return os.path.join(hip_path, str(latest))
def get_version() -> str: # cannot just run hipconfig as it requires Perl installed on Windows.
return os.path.basename(path) or os.path.basename(os.path.dirname(path))
def get_agents() -> List[Agent]:
return [Agent(x.split(' ')[-1].strip()) for x in spawn("hipinfo", cwd=os.path.join(path, 'bin')).split("\n") if x.startswith('gcnArchName:')]
is_wsl: bool = False
version_torch = None
else:
def find() -> Union[str, None]:
rocm_path = shutil.which("hipconfig")
if rocm_path is not None:
return dirname(resolve_link(rocm_path), 2)
return ROCmEnvironment(os.path.join(hip_path, str(latest)))
else:
if not os.path.exists("/opt/rocm"):
return None
return resolve_link("/opt/rocm")
return ROCmEnvironment(resolve_link("/opt/rocm"))
def get_version() -> str:
arr = spawn("hipconfig --version", cwd=os.path.join(path, 'bin')).split(".")
return f'{arr[0]}.{arr[1]}' if len(arr) >= 2 else None
def get_version() -> str:
if isinstance(environment, ROCmEnvironment):
# We don't load the hip library that will not be used by PyTorch.
if sys.platform == "win32":
# ROCm is system-wide installed. Assume the version is the folder name. (e.g. C:\Program Files\AMD\ROCm\6.4)
# hipconfig requires Perl
return os.path.basename(environment.path) or os.path.basename(os.path.dirname(environment.path))
else:
arr = spawn("hipconfig --version", cwd=os.path.join(environment.path, 'bin')).split(".")
return f'{arr[0]}.{arr[1]}' if len(arr) >= 2 else None
else:
# If rocm-sdk package is installed, the hip library may be used by PyTorch.
ver = ctypes.c_int()
environment.hip.hipRuntimeGetVersion(ctypes.byref(ver))
major = ver.value // 10000000
minor = (ver.value // 100000) % 100
#patch = version.value % 100000
return f"{major}.{minor}"
def get_flash_attention_command(agent: Agent) -> str:
default = "git+https://github.com/ROCm/flash-attention"
if agent.gfx_version >= 0x1100 and agent.gfx_version < 0x1200 and os.environ.get("FLASH_ATTENTION_USE_TRITON_ROCM", "false").lower() != "true":
# use the navi_rotary_fix fork because the original doesn't support rotary_emb for transformers
# original: "git+https://github.com/ROCm/flash-attention@howiejay/navi_support"
default = "git+https://github.com/Disty0/flash-attention@navi_rotary_fix"
return "--no-build-isolation " + os.environ.get("FLASH_ATTENTION_PACKAGE", default)
if sys.platform == "win32":
def get_agents() -> List[Agent]:
if isinstance(environment, ROCmEnvironment):
out = spawn("amdgpu-arch", cwd=os.path.join(environment.path, 'bin'))
else:
# Assume that amdgpu-arch is in PATH (venv/Scripts/amdgpu-arch.exe)
out = spawn("amdgpu-arch")
out = out.strip()
return [Agent(x.split(' ')[-1].strip()) for x in out.split("\n")]
def driver_get_agents() -> List[Agent]:
# unsafe and experimental feature
from modules import windows_hip_ffi
hip = windows_hip_ffi.HIP()
count = hip.get_device_count()
agents = [None] * count
for i in range(count):
prop = hip.get_device_properties(i)
name = prop.gcnArchName.decode('utf-8').strip('\x00')
agents[i] = Agent(name)
del hip
return agents
def postinstall():
import torch
if torch.version.hip is None:
os.environ.pop("ROCM_HOME", None)
os.environ.pop("ROCM_PATH", None)
paths = os.environ["PATH"].split(";")
paths_no_rocm = []
for path_ in paths:
if "rocm" not in path_.lower():
paths_no_rocm.append(path_)
os.environ["PATH"] = ";".join(paths_no_rocm)
return
def rocm_init():
try:
import torch
import numpy as np
original_cholesky_ex = torch.linalg.cholesky_ex
@wraps(original_cholesky_ex)
def cholesky_ex(A: torch.Tensor, upper=False, check_errors=False, out=None) -> torch.return_types.linalg_cholesky_ex:
assert not check_errors
return_device = A.device
L = torch.from_numpy(np.linalg.cholesky(A.to("cpu").numpy(), upper=upper)).to(return_device)
info = torch.tensor(0, dtype=torch.int32, device=return_device)
if out is not None:
out[0].copy_(L)
out[1].copy_(info)
return torch.return_types.linalg_cholesky_ex((L, info), {})
torch.linalg.cholesky_ex = cholesky_ex
original_cholesky = torch.linalg.cholesky
@wraps(original_cholesky)
def cholesky(A: torch.Tensor, upper=False, out=None) -> torch.Tensor:
return_device = A.device
L = torch.from_numpy(np.linalg.cholesky(A.to("cpu").numpy(), upper=upper)).to(return_device)
if out is not None:
out.copy_(L)
return L
torch.linalg.cholesky = cholesky
except Exception as e:
return False, e
return True, None
is_wsl: bool = False
else:
def get_agents() -> List[Agent]:
try:
agents = spawn("rocm_agent_enumerator").split("\n")
@@ -178,41 +285,32 @@ else:
agents = [x.strip().split(" ")[-1] for x in agents if x.startswith(' Name:') and "CPU" not in x]
return [Agent(x) for x in agents]
def load_hsa_runtime() -> None:
try:
# Preload stdc++ library. This will ignore Anaconda stdc++ library.
load_library_global("/lib/x86_64-linux-gnu/libstdc++.so.6")
# Use tcmalloc if possible.
load_library_global("/usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4")
except OSError:
pass
# Preload HSA Runtime library.
load_library_global("/opt/rocm/lib/libhsa-runtime64.so")
def postinstall():
if is_wsl:
try:
if shutil.which("conda") is not None:
# Preload stdc++ library. This will bypass Anaconda stdc++ library.
load_library_global("/lib/x86_64-linux-gnu/libstdc++.so.6")
# Preload rocr4wsl. The user don't have to replace the library file.
load_library_global("/opt/rocm/lib/libhsa-runtime64.so")
except OSError:
pass
def set_blaslt_enabled(enabled: bool) -> None:
if enabled:
os.environ["HIPBLASLT_TENSILE_LIBPATH"] = blaslt_tensile_libpath
else:
os.environ["TORCH_BLAS_PREFER_HIPBLASLT"] = "0"
def get_blaslt_enabled() -> bool:
return version == version_torch and bool(int(os.environ.get("TORCH_BLAS_PREFER_HIPBLASLT", "1")))
def get_flash_attention_command(agent: Agent):
default = "git+https://github.com/ROCm/flash-attention"
if agent.gfx_version >= 0x1100 and agent.gfx_version < 0x1200 and os.environ.get("FLASH_ATTENTION_USE_TRITON_ROCM", "false").lower() != "true":
# use the navi_rotary_fix fork because the original doesn't support rotary_emb for transformers
# original: "git+https://github.com/ROCm/flash-attention@howiejay/navi_support"
default = "git+https://github.com/Disty0/flash-attention@navi_rotary_fix"
return "--no-build-isolation " + os.environ.get("FLASH_ATTENTION_PACKAGE", default)
def rocm_init():
return True, None
is_wsl: bool = os.environ.get('WSL_DISTRO_NAME', 'unknown' if spawn('wslpath -w /') else None) is not None
version_torch = get_version_torch()
path = find()
environment = None
blaslt_tensile_libpath = ""
is_installed = False
version = None
if path is not None:
blaslt_tensile_libpath = os.environ.get("HIPBLASLT_TENSILE_LIBPATH", os.path.join(path, "bin" if sys.platform == "win32" else "lib", "hipblaslt", "library"))
is_installed = True
version = get_version()
def refresh():
global environment, blaslt_tensile_libpath, is_installed, version # pylint: disable=global-statement
environment = find()
if environment is not None:
if isinstance(environment, ROCmEnvironment):
blaslt_tensile_libpath = os.environ.get("HIPBLASLT_TENSILE_LIBPATH", os.path.join(environment.path, "bin" if sys.platform == "win32" else "lib", "hipblaslt", "library"))
is_installed = True
version = get_version()
refresh()
modules/sd_detect.py
+2
View File
@@ -115,6 +115,8 @@ def guess_by_name(fn, current_guess):
return 'Kandinsky 3.0'
elif 'hunyuanimage' in fn.lower():
return 'HunyuanImage'
elif 'sdxl-turbo' in fn.lower() or 'stable-diffusion-xl' in fn.lower():
return 'Stable Diffusion XL'
return current_guess
modules/sd_hijack_vae.py
+1 -1
View File
@@ -18,7 +18,7 @@ def hijack_vae_decode(*args, **kwargs):
latents = args[0].to(device=devices.device, dtype=shared.sd_model.vae.dtype) # upcast to vae dtype
res = shared.sd_model.vae.orig_decode(latents, *args[1:], **kwargs)
t1 = time.time()
shared.log.debug(f'Decode: vae={shared.sd_model.vae.__class__.__name__} slicing={getattr(shared.sd_model.vae, "use_slicing", None)} tiling={getattr(shared.sd_model.vae, "use_tiling", None)} latents={list(latents.shape)}:{latents.device}:{latents.dtype} time={t1-t0:.3f}')
shared.log.debug(f'Decode: vae={shared.sd_model.vae.__class__.__name__} slicing={getattr(shared.sd_model.vae, "use_slicing", None)} tiling={getattr(shared.sd_model.vae, "use_tiling", None)} latents={list(latents.shape)}:{latents.device} dtype={latents.dtype} time={t1-t0:.3f}')
else:
res = shared.sd_model.vae.orig_decode(*args, **kwargs)
except Exception as e:
modules/sd_models.py
+80 -61
View File
@@ -411,65 +411,70 @@ def load_diffuser_folder(model_type, pipeline, checkpoint_info, diffusers_load_c
files = shared.walk_files(checkpoint_info.path, ['.safetensors', '.bin', '.ckpt'])
if 'variant' not in diffusers_load_config and any('diffusion_pytorch_model.fp16' in f for f in files): # deal with diffusers lack of variant fallback when loading
diffusers_load_config['variant'] = 'fp16'
if (model_type is not None) and (pipeline is not None) and ('ONNX' in model_type): # forced pipeline
try:
sd_model = pipeline.from_pretrained(checkpoint_info.path)
except Exception as e:
shared.log.error(f'Load {op}: type=ONNX path="{checkpoint_info.path}" {e}')
if debug_load:
errors.display(e, 'Load')
return None
else:
err1, err2, err3 = None, None, None
if os.path.exists(checkpoint_info.path) and os.path.isdir(checkpoint_info.path):
if os.path.exists(os.path.join(checkpoint_info.path, 'unet', 'diffusion_pytorch_model.bin')):
shared.log.debug(f'Load {op}: type=pickle')
diffusers_load_config['use_safetensors'] = False
err0, err1, err2, err3 = None, None, None, None
if os.path.exists(checkpoint_info.path) and os.path.isdir(checkpoint_info.path):
if os.path.exists(os.path.join(checkpoint_info.path, 'unet', 'diffusion_pytorch_model.bin')):
shared.log.debug(f'Load {op}: type=pickle')
diffusers_load_config['use_safetensors'] = False
if debug_load:
shared.log.debug(f'Load {op}: args={diffusers_load_config}')
try: #0 - using detected model type and pipeline
if (model_type is not None) and (pipeline is not None):
sd_model = pipeline.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
sd_model.model_type = sd_model.__class__.__name__
except Exception as e:
err0 = e
if debug_load:
shared.log.debug(f'Load {op}: args={diffusers_load_config}')
try: # 1 - autopipeline, best choice but not all pipelines are available
try:
errors.display(e, 'Load Detected')
try: # 1 - autopipeline, best choice but not all pipelines are available
try:
if err0 is not None:
sd_model = diffusers.AutoPipelineForText2Image.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
sd_model.model_type = sd_model.__class__.__name__
except ValueError as e:
if 'no variant default' in str(e):
shared.log.warning(f'Load {op}: variant={diffusers_load_config["variant"]} model="{checkpoint_info.path}" using default variant')
diffusers_load_config.pop('variant', None)
sd_model = diffusers.AutoPipelineForText2Image.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
sd_model.model_type = sd_model.__class__.__name__
elif 'safetensors found in directory' in str(err1):
shared.log.warning(f'Load {op}: type=pickle')
diffusers_load_config['use_safetensors'] = False
sd_model = diffusers.AutoPipelineForText2Image.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
sd_model.model_type = sd_model.__class__.__name__
else:
raise ValueError from e # reraise
except Exception as e:
err1 = e
if debug_load:
errors.display(e, 'Load AutoPipeline')
# shared.log.error(f'AutoPipeline: {e}')
try: # 2 - diffusion pipeline, works for most non-linked pipelines
if err1 is not None:
sd_model = diffusers.DiffusionPipeline.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
except ValueError as e:
if 'no variant default' in str(e):
shared.log.warning(f'Load {op}: variant={diffusers_load_config["variant"]} model="{checkpoint_info.path}" using default variant')
diffusers_load_config.pop('variant', None)
sd_model = diffusers.AutoPipelineForText2Image.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
sd_model.model_type = sd_model.__class__.__name__
except Exception as e:
err2 = e
if debug_load:
errors.display(e, "Load DiffusionPipeline")
# shared.log.error(f'DiffusionPipeline: {e}')
try: # 3 - try basic pipeline just in case
if err2 is not None:
sd_model = diffusers.StableDiffusionPipeline.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
elif 'safetensors found in directory' in str(err1):
shared.log.warning(f'Load {op}: type=pickle')
diffusers_load_config['use_safetensors'] = False
sd_model = diffusers.AutoPipelineForText2Image.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
sd_model.model_type = sd_model.__class__.__name__
except Exception as e:
err3 = e # ignore last error
shared.log.error(f"StableDiffusionPipeline: {e}")
if debug_load:
errors.display(e, "Load StableDiffusionPipeline")
if err3 is not None:
shared.log.error(f'Load {op}: {checkpoint_info.path} auto={err1} diffusion={err2}')
return None
else:
raise ValueError from e # reraise
except Exception as e:
err1 = e
if debug_load:
errors.display(e, 'Load AutoPipeline')
try: # 2 - diffusion pipeline, works for most non-linked pipelines
if err1 is not None:
sd_model = diffusers.DiffusionPipeline.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
sd_model.model_type = sd_model.__class__.__name__
except Exception as e:
err2 = e
if debug_load:
errors.display(e, "Load DiffusionPipeline")
try: # 3 - try basic pipeline just in case
if err2 is not None:
sd_model = diffusers.StableDiffusionPipeline.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
sd_model.model_type = sd_model.__class__.__name__
except Exception as e:
err3 = e # ignore last error
shared.log.error(f"StableDiffusionPipeline: {e}")
if debug_load:
errors.display(e, "Load StableDiffusionPipeline")
if err3 is not None:
shared.log.error(f'Load {op}: {checkpoint_info.path} detected={err0} auto={err1} diffusion={err2} base={err3}')
return None
return sd_model
@@ -667,7 +672,7 @@ def load_diffuser(checkpoint_info=None, op='model', revision=None): # pylint: di
sd_model.scheduler.name = 'DDIM'
if hasattr(sd_model, "unet") and model_type not in ['Stable Cascade']: # others calls load_diffuser again
sd_unet.load_unet(sd_model)
sd_unet.load_unet(sd_model, checkpoint_info.path)
add_noise_pred_to_diffusers_callback(sd_model)
@@ -738,6 +743,7 @@ class DiffusersTaskType(Enum):
IMAGE_2_IMAGE = 2
INPAINTING = 3
INSTRUCT = 4
MODULAR = 5
def get_diffusers_task(pipe: diffusers.DiffusionPipeline) -> DiffusersTaskType:
@@ -748,6 +754,8 @@ def get_diffusers_task(pipe: diffusers.DiffusionPipeline) -> DiffusersTaskType:
return DiffusersTaskType.IMAGE_2_IMAGE
elif 'Instruct' in cls:
return DiffusersTaskType.INSTRUCT
elif 'Modular' in cls:
return DiffusersTaskType.MODULAR
elif pipe.__class__ in diffusers.pipelines.auto_pipeline.AUTO_IMAGE2IMAGE_PIPELINES_MAPPING.values():
return DiffusersTaskType.IMAGE_2_IMAGE
elif pipe.__class__ in diffusers.pipelines.auto_pipeline.AUTO_INPAINT_PIPELINES_MAPPING.values():
@@ -945,6 +953,9 @@ def set_diffuser_pipe(pipe, new_pipe_type):
if get_diffusers_task(pipe) == new_pipe_type:
return pipe
if get_diffusers_task(pipe) == DiffusersTaskType.MODULAR:
return pipe
# skip specific pipelines
cls = pipe.__class__.__name__
if cls in pipe_switch_task_exclude:
@@ -1029,7 +1040,13 @@ def set_diffusers_attention(pipe, quiet:bool=False):
return
# other models uses their own attention processor
if pipe.__class__.__name__.startswith("StableDiffusion") and hasattr(pipe, "unet"):
pipe.unet.set_attn_processor(attention)
try:
pipe.unet.set_attn_processor(attention)
except Exception as e:
if 'Nunchaku' in pipe.unet.__class__.__name__:
pass
else:
shared.log.error(f"Attention: {name if name is not None else attention.__class__.__name__} pipe={pipe.__class__.__name__} {e}")
elif not quiet:
shared.log.warning(f"Attention: {name if name is not None else attention.__class__.__name__} is not compatible with {pipe.__class__.__name__}")
@@ -1053,17 +1070,19 @@ def set_diffusers_attention(pipe, quiet:bool=False):
pipe.enable_xformers_memory_efficient_attention()
else:
shared.log.warning(f"Attention: xFormers is not compatible with {pipe.__class__.__name__}")
elif shared.opts.cross_attention_optimization == "Split attention":
if hasattr(pipe, "enable_attention_slicing"):
pipe.enable_attention_slicing()
else:
shared.log.warning(f"Attention: Split attention is not compatible with {pipe.__class__.__name__}")
elif shared.opts.cross_attention_optimization == "Batch matrix-matrix":
set_attn(pipe, p.AttnProcessor(), name="Batch matrix-matrix")
elif shared.opts.cross_attention_optimization == "Dynamic Attention BMM":
from modules.sd_hijack_dynamic_atten import DynamicAttnProcessorBMM
set_attn(pipe, DynamicAttnProcessorBMM(), name="Dynamic Attention BMM")
if shared.opts.attention_slicing != "Default" and hasattr(pipe, "enable_attention_slicing") and hasattr(pipe, "disable_attention_slicing"):
if shared.opts.attention_slicing:
pipe.enable_attention_slicing()
else:
pipe.disable_attention_slicing()
shared.log.debug(f"Attention: slicing={shared.opts.attention_slicing}")
pipe.current_attn_name = shared.opts.cross_attention_optimization
modules/sd_offload.py
+31 -9
View File
@@ -18,6 +18,7 @@ offload_post = ['h1']
offload_hook_instance = None
balanced_offload_exclude = ['CogView4Pipeline', 'MeissonicPipeline']
accelerate_dtype_byte_size = None
move_stream = None
def dtype_byte_size(dtype: torch.dtype):
@@ -176,7 +177,7 @@ class OffloadHook(accelerate.hooks.ModelHook):
self.last_post = None
self.last_cls = None
gpu = f'{(shared.gpu_memory * shared.opts.diffusers_offload_min_gpu_memory):.2f}-{(shared.gpu_memory * shared.opts.diffusers_offload_max_gpu_memory):.2f}:{shared.gpu_memory:.2f}'
shared.log.info(f'Offload: type=balanced op=init watermark={self.min_watermark}-{self.max_watermark} gpu={gpu} cpu={shared.cpu_memory:.3f} limit={shared.opts.cuda_mem_fraction:.2f} always={self.offload_always} never={self.offload_never} pre={shared.opts.diffusers_offload_pre}')
shared.log.info(f'Offload: type=balanced op=init watermark={self.min_watermark}-{self.max_watermark} gpu={gpu} cpu={shared.cpu_memory:.3f} limit={shared.opts.cuda_mem_fraction:.2f} always={self.offload_always} never={self.offload_never} pre={shared.opts.diffusers_offload_pre} streams={shared.opts.diffusers_offload_streams}')
self.validate()
super().__init__()
@@ -206,12 +207,18 @@ class OffloadHook(accelerate.hooks.ModelHook):
return False
if hasattr(module, 'nets') and any(hasattr(n, "offload_never") for n in module.nets):
return False
if shared.sd_model_type.lower() in [m.lower().strip() for m in re.split(r'[ ,]+', shared.opts.models_not_to_offload)]:
return False
return True
def pre_forward(self, module, *args, **kwargs):
_id = id(module)
if (self.last_pre != _id) and (module.__class__.__name__ != self.last_cls) and self.offload_allowed(module): # offload every other module first time when new module starts pre-forward
do_offload = (self.last_pre != _id) or (module.__class__.__name__ != self.last_cls)
if do_offload and self.offload_allowed(module): # offload every other module first time when new module starts pre-forward
if shared.opts.diffusers_offload_pre:
t0 = time.time()
debug_move(f'Offload: type=balanced op=pre module={module.__class__.__name__}')
for pipe in get_pipe_variants():
for module_name in get_module_names(pipe):
@@ -220,15 +227,16 @@ class OffloadHook(accelerate.hooks.ModelHook):
if (_id != id(module_instance)) and (module_cls not in self.offload_never) and (not devices.same_device(module_instance.device, devices.cpu)):
apply_balanced_offload_to_module(module_instance, op='pre')
self.last_cls = module.__class__.__name__
self.last_pre = _id
process_timer.add('offload', time.time() - t0)
if not devices.same_device(module.device, devices.device): # move-to-device
t0 = time.time()
device_index = torch.device(devices.device).index
if device_index is None:
device_index = 0
max_memory = { device_index: self.gpu, "cpu": self.cpu }
device_map = getattr(module, "balanced_offload_device_map", None)
if device_map is None or max_memory != getattr(module, "balanced_offload_max_memory", None):
if (device_map is None) or (max_memory != getattr(module, "balanced_offload_max_memory", None)):
device_map = accelerate.infer_auto_device_map(module, max_memory=max_memory)
offload_dir = getattr(module, "offload_dir", os.path.join(shared.opts.accelerate_offload_path, module.__class__.__name__))
if devices.backend == "directml":
@@ -241,13 +249,15 @@ class OffloadHook(accelerate.hooks.ModelHook):
module._hf_hook.execution_device = torch.device(devices.device) # pylint: disable=protected-access
module.balanced_offload_device_map = device_map
module.balanced_offload_max_memory = max_memory
process_timer.add('onload', time.time() - t0)
if debug:
for pipe in get_pipe_variants():
for _i, pipe in enumerate(get_pipe_variants()):
for module_name in get_module_names(pipe):
module_instance = getattr(pipe, module_name, None)
shared.log.trace(f'Offload: type=balanced op=pre:status module={module_instance.__class__.__name__} device={module_instance.device} dtype={module_instance.dtype}')
shared.log.trace(f'Offload: type=balanced op=pre:status forward={module.__class__.__name__} module={module_name} class={module_instance.__class__.__name__} pipe={_i} device={module_instance.device} dtype={module_instance.dtype}')
self.last_pre = _id
return args, kwargs
def post_forward(self, module, output):
@@ -283,6 +293,7 @@ def get_module_names(pipe=None, exclude=[]):
modules_names = get_signature(pipe).keys()
modules_names = [m for m in modules_names if m not in exclude and not m.startswith('_')]
modules_names = [m for m in modules_names if isinstance(getattr(pipe, m, None), torch.nn.Module)]
modules_names = sorted(set(modules_names))
return modules_names
@@ -308,6 +319,17 @@ def get_module_sizes(pipe=None, exclude=[]):
def move_module_to_cpu(module, op='unk', force:bool=False):
def do_move(module):
if shared.opts.diffusers_offload_streams:
global move_stream # pylint: disable=global-statement
if move_stream is None:
move_stream = torch.cuda.Stream(device=devices.device)
with torch.cuda.stream(move_stream):
module = module.to(devices.cpu)
else:
module = module.to(devices.cpu)
return module
try:
module_name = getattr(module, "module_name", module.__class__.__name__)
module_size = offload_hook_instance.offload_map.get(module_name, offload_hook_instance.model_size())
@@ -318,17 +340,17 @@ def move_module_to_cpu(module, op='unk', force:bool=False):
op = f'{op}:skip'
if force:
op = f'{op}:force'
module = module.to(devices.cpu)
module = do_move(module)
used_gpu -= module_size
elif module_cls in offload_hook_instance.offload_never:
op = f'{op}:never'
elif module_cls in offload_hook_instance.offload_always:
op = f'{op}:always'
module = module.to(devices.cpu)
module = do_move(module)
used_gpu -= module_size
elif perc_gpu > shared.opts.diffusers_offload_min_gpu_memory:
op = f'{op}:mem'
module = module.to(devices.cpu)
module = do_move(module)
used_gpu -= module_size
if debug:
quant = getattr(module, "quantization_method", None)
modules/sd_unet.py
+36 -3
View File
@@ -1,5 +1,5 @@
import os
from modules import shared, devices, files_cache, sd_models
from modules import shared, devices, files_cache, sd_models, model_quant
unet_dict = {}
@@ -8,22 +8,55 @@ failed_unet = []
debug = os.environ.get('SD_LOAD_DEBUG', None) is not None
dit_models = ['Flux', 'StableDiffusion3', 'HiDream', 'Lumina2', 'Chroma', 'Wan']
dit_models = ['Flux', 'StableDiffusion3', 'HiDream', 'Lumina2', 'Chroma', 'Wan', 'Qwen']
def load_unet(model):
def load_unet_sdxl_nunchaku(repo_id):
try:
from nunchaku.models.unets.unet_sdxl import NunchakuSDXLUNet2DConditionModel
except Exception:
shared.log.error(f'Load module: quant=Nunchaku module=unet repo="{repo_id}" low nunchaku version')
return None
if 'turbo' in repo_id.lower():
nunchaku_repo = 'nunchaku-tech/nunchaku-sdxl-turbo/svdq-int4_r32-sdxl-turbo.safetensors'
else:
nunchaku_repo = 'nunchaku-tech/nunchaku-sdxl/svdq-int4_r32-sdxl.safetensors'
shared.log.debug(f'Load module: quant=Nunchaku module=unet repo="{nunchaku_repo}" offload={shared.opts.nunchaku_offload}')
unet = NunchakuSDXLUNet2DConditionModel.from_pretrained(
nunchaku_repo,
offload=shared.opts.nunchaku_offload,
torch_dtype=devices.dtype,
cache_dir=shared.opts.hfcache_dir,
)
unet.quantization_method = 'SVDQuant'
return unet
def load_unet(model, repo_id:str=None):
global loaded_unet # pylint: disable=global-statement
if ("StableDiffusionXLPipeline" in model.__class__.__name__) and (('stable-diffusion-xl-base' in repo_id) or ('sdxl-turbo' in repo_id)):
if model_quant.check_nunchaku('Model'):
unet = load_unet_sdxl_nunchaku(repo_id)
if unet is not None:
model.unet = unet
return
if shared.opts.sd_unet == 'Default' or shared.opts.sd_unet == 'None':
return
if shared.opts.sd_unet not in list(unet_dict):
shared.log.error(f'Load module: type=UNet not found: {shared.opts.sd_unet}')
return
config_file = os.path.splitext(unet_dict[shared.opts.sd_unet])[0] + '.json'
if os.path.exists(config_file):
config = shared.readfile(config_file)
else:
config = None
config_file = 'default'
try:
if shared.opts.sd_unet == loaded_unet or shared.opts.sd_unet in failed_unet:
pass
modules/sdnq/__init__.py
+26 -11
View File
@@ -75,6 +75,10 @@ def sdnq_quantize_layer(layer, weights_dtype="int8", torch_dtype=None, group_siz
use_quantized_matmul = group_channel_size >= 32 and output_channel_size >= 32
if use_quantized_matmul and not dtype_dict[weights_dtype]["is_integer"]:
use_quantized_matmul = output_channel_size % 16 == 0 and group_channel_size % 16 == 0
if use_quantized_matmul and dtype_dict[weights_dtype]["num_bits"] == 8:
result_shape = layer.weight.shape
layer.weight.data = layer.weight.flatten(1,-1)
reduction_axes = -1
elif layer_class_name in conv_transpose_types:
if not quant_conv:
return layer
@@ -128,7 +132,8 @@ def sdnq_quantize_layer(layer, weights_dtype="int8", torch_dtype=None, group_siz
num_of_groups = int(num_of_groups)
if num_of_groups > 1:
result_shape = layer.weight.shape
if result_shape is None:
result_shape = layer.weight.shape
new_shape = list(result_shape)
if is_conv_type:
# output_channel_size, channel_size, X, X
@@ -165,17 +170,15 @@ def sdnq_quantize_layer(layer, weights_dtype="int8", torch_dtype=None, group_siz
re_quantize_for_matmul = (num_of_groups > 1 or zero_point is not None)
if use_quantized_matmul and not re_quantize_for_matmul:
if is_conv_type:
result_shape = layer.weight.shape
layer.weight.data = layer.weight.reshape(output_channel_size, -1)
scale.transpose_(0,1)
layer.weight.transpose_(0,1)
if not dtype_dict[weights_dtype]["is_integer"]:
weight_stride = layer.weight.stride()
if not (weight_stride[0] == 1 and weight_stride[1] > 1):
layer.weight.data = layer.weight.t().contiguous().t()
if not use_tensorwise_fp8_matmul:
scale = scale.to(torch.float32)
if layer.weight.is_contiguous():
if devices.backend != "ipex":
layer.weight.data = layer.weight.t_().contiguous().t_()
elif devices.backend == "ipex":
layer.weight.data = layer.weight.contiguous()
if not use_tensorwise_fp8_matmul and not dtype_dict[weights_dtype]["is_integer"]:
scale = scale.to(torch.float32)
layer.sdnq_dequantizer = dequantizer_dict[weights_dtype](
scale=scale,
@@ -402,7 +405,7 @@ class SDNQQuantizer(DiffusersQuantizer):
def _process_model_after_weight_loading(self, model, **kwargs): # pylint: disable=unused-argument
if shared.opts.diffusers_offload_mode != "none":
model = model.to(devices.cpu)
devices.torch_gc(force=True, reason='sdnq')
devices.torch_gc(force=True, reason="sdnq")
return model
def get_accelerator_warm_up_factor(self):
@@ -420,6 +423,12 @@ class SDNQQuantizer(DiffusersQuantizer):
"""
return config
def update_ep_plan(self, config):
"""
needed for transformers compatibilty, no-op function
"""
return config
def update_unexpected_keys(self, model, unexpected_keys: List[str], prefix: str) -> List[str]: # pylint: disable=unused-argument
"""
needed for transformers compatibilty, no-op function
@@ -432,6 +441,12 @@ class SDNQQuantizer(DiffusersQuantizer):
"""
return missing_keys
def update_state_dict_with_metadata(self, state_dict: dict, metadata: dict) -> dict: # pylint: disable=unused-argument
"""
needed for transformers compatibilty, no-op function
"""
return state_dict
def update_expected_keys(self, model, expected_keys: List[str], loaded_keys: List[str]) -> List[str]: # pylint: disable=unused-argument
"""
needed for transformers compatibilty, no-op function
modules/sdnq/common.py
+7 -1
View File
@@ -1,7 +1,9 @@
# pylint: disable=redefined-builtin,no-member,protected-access
import os
from functools import partial
import torch
from modules import shared
torch_version = float(torch.__version__[:3])
@@ -32,7 +34,7 @@ if hasattr(torch, "float8_e5m2fnuz"):
dtype_dict["float8_e5m2fnuz"] = {"min": -57344, "max": 57344, "num_bits": 8, "target_dtype": "fp8", "torch_dtype": torch.float8_e5m2fnuz, "storage_dtype": torch.float8_e5m2fnuz, "is_unsigned": False, "is_integer": False}
use_torch_compile = shared.opts.sdnq_dequantize_compile # this setting requires a full restart of the webui to apply
use_tensorwise_fp8_matmul = os.environ.get('SDNQ_USE_TENSORWISE_FP8_MATMUL', "1").lower() not in {"0", "false", "no"} # row-wise FP8 only exist on H100 hardware, sdnq will use software row-wise with tensorwise hardware with this setting
use_tensorwise_fp8_matmul = os.environ.get("SDNQ_USE_TENSORWISE_FP8_MATMUL", "1").lower() not in {"0", "false", "no"} # row-wise FP8 only exist on H100 hardware, sdnq will use software row-wise with tensorwise hardware with this setting
linear_types = ("Linear",)
conv_types = ("Conv1d", "Conv2d", "Conv3d")
@@ -42,3 +44,7 @@ allowed_types = linear_types + conv_types + conv_transpose_types
if use_torch_compile:
torch._dynamo.config.cache_size_limit = max(8192, torch._dynamo.config.cache_size_limit)
torch._dynamo.config.accumulated_recompile_limit = max(8192, torch._dynamo.config.accumulated_recompile_limit)
compile_func = partial(torch.compile, fullgraph=True)
else:
def compile_func(fn, **kwargs): # pylint: disable=unused-argument
return fn
modules/sdnq/dequantizer.py
+9 -19
View File
@@ -4,7 +4,7 @@ from typing import Tuple
import torch
from .common import dtype_dict, use_torch_compile
from .common import dtype_dict, compile_func
from .packed_int import pack_int_symetric, unpack_int_symetric, pack_int_asymetric, unpack_int_asymetric
@@ -226,21 +226,11 @@ dequantizer_dict = {
}
if use_torch_compile:
dequantize_asymmetric_compiled = torch.compile(dequantize_asymmetric, fullgraph=True, dynamic=False)
dequantize_symmetric_compiled = torch.compile(dequantize_symmetric, fullgraph=True, dynamic=False)
dequantize_packed_int_asymmetric_compiled = torch.compile(dequantize_packed_int_asymmetric, fullgraph=True, dynamic=False)
dequantize_packed_int_symmetric_compiled = torch.compile(dequantize_packed_int_symmetric, fullgraph=True, dynamic=False)
re_quantize_matmul_asymmetric_compiled = torch.compile(re_quantize_matmul_asymmetric, fullgraph=True, dynamic=False)
re_quantize_matmul_symmetric_compiled = torch.compile(re_quantize_matmul_symmetric, fullgraph=True, dynamic=False)
re_quantize_matmul_packed_int_asymmetric_compiled = torch.compile(re_quantize_matmul_packed_int_asymmetric, fullgraph=True, dynamic=False)
re_quantize_matmul_packed_int_symmetric_compiled = torch.compile(re_quantize_matmul_packed_int_symmetric, fullgraph=True, dynamic=False)
else:
dequantize_asymmetric_compiled = dequantize_asymmetric
dequantize_symmetric_compiled = dequantize_symmetric
dequantize_packed_int_asymmetric_compiled = dequantize_packed_int_asymmetric
dequantize_packed_int_symmetric_compiled = dequantize_packed_int_symmetric
re_quantize_matmul_asymmetric_compiled = re_quantize_matmul_asymmetric
re_quantize_matmul_symmetric_compiled = re_quantize_matmul_symmetric
re_quantize_matmul_packed_int_asymmetric_compiled = re_quantize_matmul_packed_int_asymmetric
re_quantize_matmul_packed_int_symmetric_compiled = re_quantize_matmul_packed_int_symmetric
dequantize_asymmetric_compiled = compile_func(dequantize_asymmetric)
dequantize_symmetric_compiled = compile_func(dequantize_symmetric)
dequantize_packed_int_asymmetric_compiled = compile_func(dequantize_packed_int_asymmetric)
dequantize_packed_int_symmetric_compiled = compile_func(dequantize_packed_int_symmetric)
re_quantize_matmul_asymmetric_compiled = compile_func(re_quantize_matmul_asymmetric)
re_quantize_matmul_symmetric_compiled = compile_func(re_quantize_matmul_symmetric)
re_quantize_matmul_packed_int_asymmetric_compiled = compile_func(re_quantize_matmul_packed_int_asymmetric)
re_quantize_matmul_packed_int_symmetric_compiled = compile_func(re_quantize_matmul_packed_int_symmetric)
modules/sdnq/layers/conv/conv_fp8.py
+4 -3
View File
@@ -4,8 +4,9 @@ from typing import List
import torch
from ...common import use_torch_compile # noqa: TID252
from ...common import compile_func # noqa: TID252
from ..linear.linear_fp8 import quantize_fp8_matmul_input # noqa: TID252
from ..linear.forward import check_mats # noqa: TID252
from .forward import get_conv_args, process_conv_input
@@ -23,6 +24,7 @@ def conv_fp8_matmul(
return_dtype = input.dtype
input, mm_output_shape = process_conv_input(conv_type, input, reversed_padding_repeated_twice, padding_mode, result_shape, stride, padding, dilation)
input, input_scale = quantize_fp8_matmul_input(input)
input, weight = check_mats(input, weight)
if groups == 1:
if bias is not None and bias.dtype != torch.bfloat16:
@@ -68,5 +70,4 @@ def quantized_conv_forward_fp8_matmul(self, input) -> torch.FloatTensor:
)
if use_torch_compile:
conv_fp8_matmul = torch.compile(conv_fp8_matmul, fullgraph=True, dynamic=False)
conv_fp8_matmul = compile_func(conv_fp8_matmul)
modules/sdnq/layers/conv/conv_fp8_tensorwise.py
+4 -3
View File
@@ -4,9 +4,10 @@ from typing import List
import torch
from ...common import use_torch_compile # noqa: TID252
from ...common import compile_func # noqa: TID252
from ...dequantizer import dequantize_symmetric, dequantize_symmetric_with_bias # noqa: TID252
from ..linear.linear_fp8_tensorwise import quantize_fp8_matmul_input_tensorwise # noqa: TID252
from ..linear.forward import check_mats # noqa: TID252
from .forward import get_conv_args, process_conv_input
@@ -24,6 +25,7 @@ def conv_fp8_matmul_tensorwise(
return_dtype = input.dtype
input, mm_output_shape = process_conv_input(conv_type, input, reversed_padding_repeated_twice, padding_mode, result_shape, stride, padding, dilation)
input, scale = quantize_fp8_matmul_input_tensorwise(input, scale)
input, weight = check_mats(input, weight)
dummy_input_scale = torch.ones(1, device=input.device, dtype=torch.float32)
if groups == 1:
@@ -63,5 +65,4 @@ def quantized_conv_forward_fp8_matmul_tensorwise(self, input) -> torch.FloatTens
)
if use_torch_compile:
conv_fp8_matmul_tensorwise = torch.compile(conv_fp8_matmul_tensorwise, fullgraph=True, dynamic=False)
conv_fp8_matmul_tensorwise = compile_func(conv_fp8_matmul_tensorwise)
modules/sdnq/layers/conv/conv_int8.py
+4 -3
View File
@@ -4,10 +4,11 @@ from typing import List
import torch
from ...common import use_torch_compile # noqa: TID252
from ...common import compile_func # noqa: TID252
from ...packed_int import unpack_int_symetric # noqa: TID252
from ...dequantizer import dequantize_symmetric, dequantize_symmetric_with_bias # noqa: TID252
from ..linear.linear_int8 import quantize_int8_matmul_input # noqa: TID252
from ..linear.forward import check_mats # noqa: TID252
from .forward import get_conv_args, process_conv_input
@@ -29,6 +30,7 @@ def conv_int8_matmul(
input, scale = quantize_int8_matmul_input(input, scale)
if quantized_weight_shape is not None:
weight = unpack_int_symetric(weight, quantized_weight_shape, weights_dtype, dtype=torch.int8)
input, weight = check_mats(input, weight)
if groups == 1:
result = torch._int_mm(input, weight)
@@ -75,5 +77,4 @@ def quantized_conv_forward_int8_matmul(self, input) -> torch.FloatTensor:
)
if use_torch_compile:
conv_int8_matmul = torch.compile(conv_int8_matmul, fullgraph=True, dynamic=False)
conv_int8_matmul = compile_func(conv_int8_matmul)
modules/sdnq/layers/linear/forward.py
+12
View File
@@ -1,7 +1,19 @@
# pylint: disable=relative-beyond-top-level,redefined-builtin,protected-access
from typing import Tuple
import torch
def check_mats(input: torch.Tensor, weight: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
input = input.contiguous()
if weight.is_contiguous():
if weight.device.type != "xpu":
weight = weight.t().contiguous().t()
elif weight.device.type == "xpu":
weight = weight.contiguous()
return input, weight
def quantized_linear_forward(self, input: torch.FloatTensor) -> torch.FloatTensor:
return torch.nn.functional.linear(input, self.sdnq_dequantizer(self.weight), self.bias)
modules/sdnq/layers/linear/linear_fp8.py
+4 -3
View File
@@ -4,8 +4,9 @@ from typing import Tuple
import torch
from ...common import use_torch_compile # noqa: TID252
from ...common import compile_func # noqa: TID252
from ...dequantizer import quantize_fp8 # noqa: TID252
from .forward import check_mats
def quantize_fp8_matmul_input(input: torch.FloatTensor) -> Tuple[torch.Tensor, torch.FloatTensor]:
@@ -23,6 +24,7 @@ def fp8_matmul(
return_dtype = input.dtype
output_shape = (*input.shape[:-1], weight.shape[-1])
input, input_scale = quantize_fp8_matmul_input(input)
input, weight = check_mats(input, weight)
if bias is not None and bias.dtype != torch.bfloat16:
bias = bias.to(dtype=torch.bfloat16)
return torch._scaled_mm(input, weight, scale_a=input_scale, scale_b=scale, bias=bias, out_dtype=torch.bfloat16).view(output_shape).to(return_dtype)
@@ -34,5 +36,4 @@ def quantized_linear_forward_fp8_matmul(self, input: torch.FloatTensor) -> torch
return fp8_matmul(input, self.weight, self.bias, self.sdnq_dequantizer.scale)
if use_torch_compile:
fp8_matmul = torch.compile(fp8_matmul, fullgraph=True, dynamic=False)
fp8_matmul = compile_func(fp8_matmul)
modules/sdnq/layers/linear/linear_fp8_tensorwise.py
+4 -3
View File
@@ -4,8 +4,9 @@ from typing import Tuple
import torch
from ...common import use_torch_compile # noqa: TID252
from ...common import compile_func # noqa: TID252
from ...dequantizer import quantize_fp8, dequantize_symmetric, dequantize_symmetric_with_bias # noqa: TID252
from .forward import check_mats
def quantize_fp8_matmul_input_tensorwise(input: torch.FloatTensor, scale: torch.FloatTensor) -> Tuple[torch.Tensor, torch.FloatTensor]:
@@ -27,6 +28,7 @@ def fp8_matmul_tensorwise(
output_shape = (*input.shape[:-1], weight.shape[-1])
dummy_input_scale = torch.ones(1, device=input.device, dtype=torch.float32)
input, scale = quantize_fp8_matmul_input_tensorwise(input, scale)
input, weight = check_mats(input, weight)
if bias is not None:
return dequantize_symmetric_with_bias(torch._scaled_mm(input, weight, scale_a=dummy_input_scale, scale_b=dummy_input_scale, bias=None, out_dtype=scale.dtype), scale, bias, return_dtype, output_shape)
else:
@@ -39,5 +41,4 @@ def quantized_linear_forward_fp8_matmul_tensorwise(self, input: torch.FloatTenso
return fp8_matmul_tensorwise(input, self.weight, self.bias, self.sdnq_dequantizer.scale)
if use_torch_compile:
fp8_matmul_tensorwise = torch.compile(fp8_matmul_tensorwise, fullgraph=True, dynamic=False)
fp8_matmul_tensorwise = compile_func(fp8_matmul_tensorwise)
modules/sdnq/layers/linear/linear_int8.py
+4 -3
View File
@@ -4,9 +4,10 @@ from typing import Tuple
import torch
from ...common import use_torch_compile # noqa: TID252
from ...common import compile_func # noqa: TID252
from ...packed_int import unpack_int_symetric # noqa: TID252
from ...dequantizer import quantize_int8, dequantize_symmetric, dequantize_symmetric_with_bias # noqa: TID252
from .forward import check_mats
def quantize_int8_matmul_input(input: torch.FloatTensor, scale: torch.FloatTensor) -> Tuple[torch.CharTensor, torch.FloatTensor]:
@@ -31,6 +32,7 @@ def int8_matmul(
return_dtype = input.dtype
output_shape = (*input.shape[:-1], weight.shape[-1])
input, scale = quantize_int8_matmul_input(input, scale)
input, weight = check_mats(input, weight)
if bias is not None:
return dequantize_symmetric_with_bias(torch._int_mm(input, weight), scale, bias, return_dtype, output_shape)
else:
@@ -50,5 +52,4 @@ def quantized_linear_forward_int8_matmul(self, input: torch.FloatTensor) -> torc
return int8_matmul(input, weight, self.bias, scale, quantized_weight_shape, self.sdnq_dequantizer.weights_dtype)
if use_torch_compile:
int8_matmul = torch.compile(int8_matmul, fullgraph=True, dynamic=False)
int8_matmul = compile_func(int8_matmul)
modules/shared.py
+24 -6
View File
@@ -70,9 +70,8 @@ state = shared_state.State()
# early select backend
backend = Backend.DIFFUSERS
if not hasattr(cmd_opts, "use_openvino"):
cmd_opts.use_openvino = False
if cmd_opts.use_openvino: # override for openvino
os.environ.setdefault('PYTORCH_TRACING_MODE', 'TORCHFX')
from modules.intel.openvino import get_device_list as get_openvino_device_list # pylint: disable=ungrouped-imports
elif cmd_opts.use_ipex or devices.has_xpu():
from modules.intel.ipex import ipex_init
@@ -84,6 +83,11 @@ elif cmd_opts.use_directml:
ok, e = directml_init()
if not ok:
log.error(f'DirectML initialization failed: {e}')
elif cmd_opts.use_rocm or devices.has_rocm():
from modules.rocm import rocm_init
ok, e = rocm_init()
if not ok:
log.error(f'ROCm initialization failed: {e}')
devices.backend = devices.get_backend(cmd_opts)
devices.device = devices.get_optimal_device()
mem_stat = memory_stats()
@@ -155,6 +159,8 @@ options_templates.update(options_section(('sd', "Model Loading"), {
}))
options_templates.update(options_section(('model_options', "Model Options"), {
"model_modular_sep": OptionInfo("<h2>Modular Pipelines</h2>", "", gr.HTML),
"model_modular_enable": OptionInfo(False, "Enable modular pipelines (experimental)"),
"model_sd3_sep": OptionInfo("<h2>Stable Diffusion 3.x</h2>", "", gr.HTML),
"model_sd3_disable_te5": OptionInfo(False, "Disable T5 text encoder"),
"model_h1_sep": OptionInfo("<h2>HiDream</h2>", "", gr.HTML),
@@ -170,9 +176,11 @@ options_templates.update(options_section(('offload', "Model Offloading"), {
"diffusers_offload_nonblocking": OptionInfo(False, "Non-blocking move operations"),
"offload_balanced_sep": OptionInfo("<h2>Balanced Offload</h2>", "", gr.HTML),
"diffusers_offload_pre": OptionInfo(True, "Offload during pre-forward"),
"diffusers_offload_streams": OptionInfo(False, "Offload using streams"),
"diffusers_offload_min_gpu_memory": OptionInfo(startup_offload_min_gpu, "Offload low watermark", gr.Slider, {"minimum": 0, "maximum": 1, "step": 0.01 }),
"diffusers_offload_max_gpu_memory": OptionInfo(startup_offload_max_gpu, "Offload GPU high watermark", gr.Slider, {"minimum": 0.1, "maximum": 1, "step": 0.01 }),
"diffusers_offload_max_cpu_memory": OptionInfo(0.90, "Offload CPU high watermark", gr.Slider, {"minimum": 0, "maximum": 1, "step": 0.01, "visible": False }),
"models_not_to_offload": OptionInfo("", "Model types not to offload"),
"diffusers_offload_always": OptionInfo(startup_offload_always, "Modules to always offload"),
"diffusers_offload_never": OptionInfo(startup_offload_never, "Modules to never offload"),
"offload_group_sep": OptionInfo("<h2>Group Offload</h2>", "", gr.HTML),
@@ -183,7 +191,7 @@ options_templates.update(options_section(('offload', "Model Offloading"), {
}))
options_templates.update(options_section(("quantization", "Model Quantization"), {
"models_not_to_quant": OptionInfo("", "List of model types not to quantize"),
"models_not_to_quant": OptionInfo("", "Model types not to quantize"),
"sdnq_quantize_sep": OptionInfo("<h2>SDNQ: SD.Next Quantization</h2>", "", gr.HTML),
"sdnq_quantize_weights": OptionInfo([], "Quantization enabled", gr.CheckboxGroup, {"choices": ["Model", "TE", "LLM", "Control", "VAE"]}),
@@ -248,8 +256,8 @@ options_templates.update(options_section(('vae_encoder', "Variational Auto Encod
"sd_vae": OptionInfo("Automatic", "VAE model", gr.Dropdown, lambda: {"choices": shared_items.sd_vae_items()}, refresh=shared_items.refresh_vae_list),
"diffusers_vae_upcast": OptionInfo("default", "VAE upcasting", gr.Radio, {"choices": ['default', 'true', 'false']}),
"no_half_vae": OptionInfo(False if not cmd_opts.use_openvino else True, "Full precision (--no-half-vae)"),
"diffusers_vae_slicing": OptionInfo(True, "VAE slicing", gr.Checkbox),
"diffusers_vae_tiling": OptionInfo(cmd_opts.lowvram or cmd_opts.medvram, "VAE tiling", gr.Checkbox),
"diffusers_vae_slicing": OptionInfo(cmd_opts.lowvram or cmd_opts.medvram, "VAE slicing", gr.Checkbox),
"diffusers_vae_tiling": OptionInfo(cmd_opts.lowvram, "VAE tiling", gr.Checkbox),
"diffusers_vae_tile_size": OptionInfo(0, "VAE tile size", gr.Slider, {"minimum": 0, "maximum": 4096, "step": 8 }),
"diffusers_vae_tile_overlap": OptionInfo(0.25, "VAE tile overlap", gr.Slider, {"minimum": 0, "maximum": 0.95, "step": 0.05 }),
"remote_vae_type": OptionInfo('raw', "Remote VAE image type", gr.Dropdown, {"choices": ['raw', 'jpg', 'png']}),
@@ -282,6 +290,8 @@ options_templates.update(options_section(('cuda', "Compute Settings"), {
"cross_attention_sep": OptionInfo("<h2>Cross Attention</h2>", "", gr.HTML),
"cross_attention_optimization": OptionInfo(startup_cross_attention, "Attention optimization method", gr.Radio, lambda: {"choices": shared_items.list_crossattention()}),
"attention_": OptionInfo("<h2>Cross Attention</h2>", "", gr.HTML),
"attention_slicing": OptionInfo('Default', "Attention slicing", gr.Radio, {"choices": ['Default', 'Enabled', 'Disabled']}),
"sdp_options": OptionInfo(startup_sdp_options, "SDP options", gr.CheckboxGroup, {"choices": startup_sdp_choices}),
"xformers_options": OptionInfo(['Flash attention'], "xFormers options", gr.CheckboxGroup, {"choices": ['Flash attention'] }),
"dynamic_attention_slice_rate": OptionInfo(0.5, "Dynamic Attention slicing rate in GB", gr.Slider, {"minimum": 0.01, "maximum": max(gpu_memory,4), "step": 0.01}),
@@ -354,6 +364,14 @@ options_templates.update(options_section(('advanced', "Pipeline Modifiers"), {
"pab_spacial_skip_start": OptionInfo(100, "PAB spacial skip start", gr.Slider, {"minimum": 0, "maximum": 1000, "step": 1}),
"pab_spacial_skip_end": OptionInfo(800, "PAB spacial skip end", gr.Slider, {"minimum": 0, "maximum": 1000, "step": 1}),
"cache_dit_sep": OptionInfo("<h2>Cache-DiT</h2>", "", gr.HTML),
"cache_dit_enabled": OptionInfo(False, "Cache-DiT enabled"),
"cache_dit_calibrator": OptionInfo("None", "Cache-DiT calibrator", gr.Radio, {"choices": ["None", "TaylorSeer", "FoCa"]}),
"cache_dit_fcompute": OptionInfo(-1, "Cache-DiT F-compute blocks", gr.Slider, {"minimum": -1, "maximum": 32, "step": 1}),
"cache_dit_bcompute": OptionInfo(-1, "Cache-DiT B-compute blocks", gr.Slider, {"minimum": -1, "maximum": 32, "step": 1}),
"cache_dit_threshold": OptionInfo(-1, "Cache-DiT residual diff threshold", gr.Slider, {"minimum": -1.0, "maximum": 1.0, "step": 0.01}),
"cache_dit_warmup": OptionInfo(-1, "Cache-DiT warmup steps", gr.Slider, {"minimum": -1, "maximum": 50, "step": 1}),
"faster_cache__sep": OptionInfo("<h2>Faster Cache</h2>", "", gr.HTML),
"faster_cache_enabled": OptionInfo(False, "FasterCache cache enabled"),
"fc_spacial_skip_range": OptionInfo(2, "FasterCache spacial skip range", gr.Slider, {"minimum": 1, "maximum": 4, "step": 1}),
@@ -453,7 +471,7 @@ options_templates.update(options_section(('system-paths', "System Paths"), {
options_templates.update(options_section(('saving-images', "Image Options"), {
"samples_save": OptionInfo(True, "Save all generated images"),
"keep_incomplete": OptionInfo(True, "Keep incomplete images"),
"keep_incomplete": OptionInfo(True, "Save interrupted images"),
"samples_format": OptionInfo('jpg', 'File format', gr.Dropdown, {"choices": ["jpg", "png", "webp", "tiff", "jp2", "jxl"]}),
"jpeg_quality": OptionInfo(90, "Image quality", gr.Slider, {"minimum": 1, "maximum": 100, "step": 1}),
"img_max_size_mp": OptionInfo(1000, "Maximum image size (MP)", gr.Slider, {"minimum": 100, "maximum": 2000, "step": 1}),
modules/shared_items.py
-1
View File
@@ -120,7 +120,6 @@ def list_crossattention():
"Scaled-Dot-Product",
"xFormers",
"Batch matrix-matrix",
"Split attention",
"Dynamic Attention BMM"
]
modules/styles.py
+10 -1
View File
@@ -123,7 +123,7 @@ def apply_wildcards_to_prompt(prompt, all_wildcards, seed=-1, silent=False):
if replaced and not silent:
shared.log.debug(f'Apply wildcards: {replaced} path="{shared.opts.wildcards_dir}" type=style time={t1-t0:.2f}')
if (len(replaced_file) > 0 or len(not_found) > 0) and not silent:
shared.log.debug(f'Apply wildcards: found={replaced_file} missing={not_found} path="{shared.opts.wildcards_dir}" type=file time={t2-t2:.2f} ')
shared.log.debug(f'Apply wildcards: found={replaced_file} missing={not_found} path="{shared.opts.wildcards_dir}" type=file seed={seed} time={t2-t2:.2f}')
if old_state is not None:
random.setstate(old_state)
return prompt
@@ -168,6 +168,7 @@ def apply_styles_to_extra(p, style: Style):
params = []
settings = []
skipped = []
for k, v in extra.items():
k = k.lower().replace(' ', '_')
if k in name_map: # rename some fields
@@ -322,16 +323,24 @@ class StyleDatabase:
jobid = shared.state.begin('Styles')
parsed_positive = []
parsed_negative = []
random_state = random.getstate()
for i in range(len(prompts)):
if seeds[i]> 0:
random.seed(seeds[i])
prompt = prompts[i]
prompt = apply_styles_to_prompt(prompt, [self.find_style(x).prompt for x in styles])
prompt = apply_wildcards_to_prompt(prompt, [self.find_style(x).wildcards for x in styles], seeds[i])
parsed_positive.append(prompt)
for i in range(len(negatives)):
if seeds[i]> 0:
random.seed(seeds[i])
prompt = negatives[i]
prompt = apply_styles_to_prompt(prompt, [self.find_style(x).negative_prompt for x in styles])
prompt = apply_wildcards_to_prompt(prompt, [self.find_style(x).wildcards for x in styles], seeds[i])
parsed_negative.append(prompt)
random.setstate(random_state)
shared.state.end(jobid)
return parsed_positive, parsed_negative
modules/txt2img.py
+6
View File
@@ -14,6 +14,7 @@ def txt2img(id_task, state,
vae_type, tiling, hidiffusion,
detailer_enabled, detailer_prompt, detailer_negative, detailer_steps, detailer_strength, detailer_resolution,
n_iter, batch_size,
guidance_name, guidance_scale, guidance_rescale, guidance_start, guidance_stop,
cfg_scale, image_cfg_scale, diffusers_guidance_rescale, pag_scale, pag_adaptive, cfg_end,
clip_skip,
seed, subseed, subseed_strength, seed_resize_from_h, seed_resize_from_w,
@@ -55,6 +56,11 @@ def txt2img(id_task, state,
batch_size=batch_size,
n_iter=n_iter,
steps=steps,
guidance_name=guidance_name,
guidance_scale=guidance_scale,
guidance_rescale=guidance_rescale,
guidance_start=guidance_start,
guidance_stop=guidance_stop,
cfg_scale=cfg_scale,
image_cfg_scale=image_cfg_scale,
diffusers_guidance_rescale=diffusers_guidance_rescale,

Some files were not shown because too many files have changed in this diff Show More