Public/automatic
1
0
Fork 0
mirror of https://github.com/vladmandic/automatic synced 2026-09-18 16:54:33 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #2540 from vladmandic/dev

Browse Source 
merge dev to master
This commit is contained in:
Vladimir Mandic
2023-11-23 10:40:09 -05:00
committed by GitHub
parent e70f3eecda 567b9dd435
commit 50102fdeca
68 changed files with 4381 additions and 726 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.git-blame-ignore-revs
-1
View File
@@ -1 +0,0 @@
4c7792ed011b233cdb6e9e42327085f4d66701f2
.github/ISSUE_TEMPLATE/bug_report.yml
+27 -12
View File
@@ -38,18 +38,18 @@ body:
- type: markdown
attributes:
value: |
If issue is setup, installation or startup related, please check `sdnext.log` before reporting
- type: markdown
attributes:
value: |
If you have additional extensions installed, try to reproduce the issue with user extensions disabled
And if the issue is with compatibility with specific extension, mark it as such when creating the issue
Try running with `--safe` command line flag with disables loading of user-installed extensions
- type: markdown
attributes:
value: |
If possible update to latest version before reporting the issue as older versions cannot be properly supported
And search existing **issues** and **discussions** before creating a new one
Please follow these guidelines when creating an issue:
- If possible update to latest version before reporting the issue as older versions cannot be properly supported
And search existing **issues** and **discussions** before creating a new one
- If issue cannot be clearly seen in the provided log, run SD.Next with debug logging enabled by adding `--debug` to your command line parameters
- If issue is setup, installation or startup related, please check `sdnext.log` before reporting
And if possible, perform reinstallation of required dependencies by running SD.Next with `--reinstall` command line flag
- If issue is after recent update of SD.Next, try to reproduce it using clean installation to avoid issues due to old configuration
- If you have additional extensions installed, try to reproduce the issue with user extensions disabled
And if the issue is with compatibility with specific extension, mark it as such when creating the issue
Try running with `--safe` command line flag with disables loading of user-installed extensions
- If issue is in functionality marked as *experimental* or relates to *dev* branch, reach out on Discord before creating an issue
To see which issues are new/updated/experimental, check [CHANGELOG](https://github.com/vladmandic/automatic/blob/master/CHANGELOG.md)
- type: textarea
id: logs
attributes:
@@ -72,6 +72,17 @@ body:
default: 0
validations:
required: true
- type: dropdown
id: branch
attributes:
label: Branch
description: What is the branch you're using?
options:
- Master
- Dev
default: 0
validations:
required: true
- type: dropdown
id: model
attributes:
@@ -79,7 +90,11 @@ body:
description: What is the model type you're using?
options:
- SD 1.5
- SD 2.1
- SD-XL
- LCM
- PixArt
- SSD
- Kandinsky
- Other
default: 0
CHANGELOG.md
+45 -10
View File
@@ -1,42 +1,77 @@
# Change Log for SD.Next
## Update for 2023-11-11
## Update for 2023-11-23
- **Diffusers**
- **LCM** support for any *SD 1.5* or *SD-XL* model!
- download [lcm-lora-sd15](https://huggingface.co/latent-consistency/lcm-lora-sdv1-5/tree/main) and/or [lcm-lora-sdxl](https://huggingface.co/latent-consistency/lcm-lora-sdxl/tree/main)
- load for favorite *SD 1.5* or *SD-XL* model *(original LCM was SD 1.5 only, this is both)*
- load **lcm lora**
- load **lcm lora** *(note: lcm lora is processed differently than any other lora)*
- set **sampler** to **LCM**
- set number of steps to some low number, for SD-XL 6-7 steps is normally sufficient
note: LCM scheduler does not support steps higher than 50
- set cfg to 1 or 2
note: LCM scheduler does not support steps higher than 50
- set CFG to between 1 and 2
- Add `cli/lcm-convert.py` script to convert any SD 1.5 or SD-XL model to LCM model
by baking in LORA and uploading to Huggingface, thanks @Disty0
- Support for [Stable Fast](https://github.com/chengzeyi/stable-fast) model compile on *Windows/Linux/WSL2* with *CUDA*
See [Wiki:Benchmark](https://github.com/vladmandic/automatic/wiki/Benchmark) for details and comparisment
of different backends, precision modes, advanced settings and compile modes
*Hint*: **70+ it/s** is possible on *RTX4090* with no special tweaks
- Add additional pipeline types for manual model loads when loading from `safetensors`
- Updated logic for calculating **steps** when using base/hires/refiner workflows
- Improve **model offloading** for both model and sequential cpu offload when dealing with meta tensors
- Safe model offloading for non-standard models
- Fix **DPM SDE** scheduler
- Better support for SD 1.5 **inpainting** models
- Update to `diffusers==0.23.0`
- Better support for SD 1.5 **inpainting** models
- Add support for **OpenAI Consistency decoder VAE**
- Enhance prompt parsing with long prompts and support for *BREAK* keyword
Change-in-behavior: new line in prompt now means *BREAK*
- Add alternative Lora loading algorithm, triggered if `SD_LORA_DIFFUSERS` is set
- **Models**
- **Model merge**
- completely redesigned, now based on best-of-class `meh` by @s1dlx
and heavily modified for additional functionality and fully integrated by @AI-Casanova (thanks!)
- merge SD or SD-XL models using *simple merge* (12 methods),
using one of *presets* (20 built-in presets) or custom block merge values
- merge with ReBasin permuatations and/or clipping protection
- fully multithreaded for fastest merge possible
- **Model update**
- under UI -> Models - Update
- scan existing models for updated metadata on CivitAI and
provide download functionality for models with available
- **Extra networks**
- Use multi-threading for 5x load speedup
- Better Lora trigger words support
- **General**:
- Auto refresh styles on change
- **General**
- Many **mobile UI** optimizations, thanks @iDeNoh
- Support for **Torch 2.1.1** with CUDA 12.1 or CUDA 11.8
- Configurable location for HF cache folder
Default is standard `~/.cache/huggingface/hub`
- Reworked parser when pasting previously generated images/prompts
includes all `txt2img`, `img2img` and `override` params
- Add refiner options to XYZ Grid
- Reworked **model compile**
- Support custom upscalers in subfolders
- Add additional image info when loading image in process tab
- Better file locking when sharing config and/or models between multiple instances
- Handle custom API endpoints when using auth
- Show logged in user in log when accessing via UI and/or API
- Support `--ckpt none` to skip loading a model
- **XYZ grid**
- Add refiner options to XYZ Grid
- Add option to create only subgrids in XYZ grid, thanks @midcoastal
- Allow custom font, background and text color in settings
- **Fixes**
- Fix `params.txt` saved before actual image
- Fix inpaint
- Fix manual grid image save
- Fix img2img init image save
- Fix upscale in txt2img for batch counts when no hires is used
- More uniform models paths
- Safe scripts callback execution
- Improve extension compatibility
- Improve BF16 support
- Improved extension compatibility
- Improved BF16 support
- Match previews for reference models with downloaded models
## Update for 2023-11-06
README.md
+42 -39
View File
@@ -1,54 +1,57 @@
<div align="center">
# SD.Next
**Stable Diffusion implementation with advanced features**
<br>
[![](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/vladmandic)
[![Sponsors](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/vladmandic)
![Last Commit](https://img.shields.io/github/last-commit/vladmandic/automatic?svg=true)
![License](https://img.shields.io/github/license/vladmandic/automatic?svg=true)
[![Discord](https://img.shields.io/discord/1101998836328697867?logo=Discord&svg=true)](https://discord.gg/VjvR2tabEX)
### [Wiki](https://github.com/vladmandic/automatic/wiki) | [Discord](https://discord.gg/VjvR2tabEX) | [Changelog](CHANGELOG.md)
[Wiki](https://github.com/vladmandic/automatic/wiki) | [Discord](https://discord.gg/VjvR2tabEX) | [Changelog](CHANGELOG.md)
</div>
</br>
## Top-13 Features (10 wasn't enough)
## Notable features
All Individual features are not listed here, instead check [ChangeLog](CHANGELOG.md) for full list of changes.
- Support for multiple backends!
**original** and **diffusers**
- Support for multiple diffusion models!
**Stable Diffusion, SD-XL, LCM, Segmind, Kandinsky, Pixart-α, Wuerstchen, DeepFloyd IF, UniDiffusion, SD-Distilled, etc.**
- Fully multiplatform with platform specific autodetection and tuning performed on install
**Windows / Linux / MacOS with CPU / nVidia / AMD / IntelArc / DirectML / OpenVINO / ONNX+Olive**
- Optimized processing with latest **torch** developments
Including built-in support for `torch.compile`
All individual features are not listed here, instead check [ChangeLog](CHANGELOG.md) for full list of changes
- Multiple backends!
▹ **Original | Diffusers**
- Multiple diffusion models!
▹ **Stable Diffusion | SD-XL | LCM | Segmind | Kandinsky | Pixart-α | Würstchen | DeepFloyd IF | UniDiffusion | SD-Distilled | etc.**
- Multiplatform!
▹ **Windows | Linux | MacOS with CPU | nVidia | AMD | IntelArc | DirectML | OpenVINO | ONNX+Olive**
- Platform specific autodetection and tuning performed on install
- Optimized processing with latest `torch` developments with built-in support for `torch.compile` and multiple compile backends
- Improved prompt parser
- Enhanced *Lora*/*Locon*/*Lyco* code supporting latest trends in training
- Enhanced *Lora*/*LoCon*/*Lyco* code supporting latest trends in training
- Built-in queue management
- Advanced metadata caching and handling to speed up operations
- Enterprise level logging and hardened API
- Modern localization and hints engine
- Broad compatibility with existing extensions ecosystem and new extensions manager
- Built in installer with automatic updates and dependency management
- Modernized UI with theme support and number of built-in themes
- Modernized UI with theme support and number of built-in themes *(dark and light)*
<br>![screenshot](html/black-teal.jpg)<br>
<br>
![Screenshot-Dark](html/black-teal.jpg)
![Screenshot-Light](html/light-teal.jpg)
<br>
## Backend support
**SD.Next** supports two main backends: *Original* and *Diffusers* which can be switched on-the-fly:
**SD.Next** supports two main backends: *Original* and *Diffusers*:
- **Original**: Based on [LDM](https://github.com/Stability-AI/stablediffusion) reference implementation and significantly expanded on by [A1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
This is the default backend and it is fully compatible with all existing functionality and extensions
It supports **SD 1.x** and **SD 2.x** models
All other model types such as SD-XL, LCM, PixArt, Segmind, Kandinsky, etc. require backend **Diffusers**
Supports **SD 1.x** and **SD 2.x** models
All other model types such as *SD-XL, LCM, PixArt, Segmind, Kandinsky, etc.* require backend **Diffusers**
- **Diffusers**: Based on new [Huggingface Diffusers](https://huggingface.co/docs/diffusers/index) implementation
It supports *original* SD models as well as *all* models listed below
Supports *original* SD models as well as *all* models listed below
See [wiki article](https://github.com/vladmandic/automatic/wiki/Diffusers) for more information
## Model support
@@ -66,11 +69,11 @@ Additional models will be added as they become available and there is public int
- [DeepFloyd IF](https://github.com/deep-floyd/IF) Medium and Large
- [Segmind SD Distilled](https://huggingface.co/blog/sd_distillation) *(all variants)*
*Notes*:
- Loading any model other than standard SD 1.x / SD 2.x requires use of backend **Diffusers**
Loading any other models using **Original** backend is not supproted
- Loading manually download model `.safetensors` files is supported for SD 1.x / SD 2.x / SD-XL models only
For all other model types, use backend **Diffusers** and use built in Model downloader or
> [!IMPORTANT]
> - Loading any model other than standard SD 1.x / SD 2.x requires use of backend **Diffusers**
> - Loading any other models using **Original** backend is not supproted
> - Loading manually download model `.safetensors` files is supported for SD 1.x / SD 2.x / SD-XL models only
> - For all other model types, use backend **Diffusers** and use built in Model downloader or
select model from Networks -> Models -> Reference list in which case it will be auto-downloaded and loaded
## Platform support
@@ -85,21 +88,21 @@ Additional models will be added as they become available and there is public int
- *Apple M1/M2* on *OSX* using built-in support in Torch with **MPS** optimizations
- *ONNX/Olive* (experimental)
## Install & Run
## Install
- [Step-by-step install guide](https://github.com/vladmandic/automatic/wiki/Installation)
- [Advanced install notes](https://github.com/vladmandic/automatic/wiki/Advanced-Install)
### Installation Notes
- [Common installation errors](https://github.com/vladmandic/automatic/discussions/1627)
- [FAQ](https://github.com/vladmandic/automatic/discussions/1011)
- Server can run without virtual environment,
but it is recommended to use it to avoid library version conflicts with other applications
- **nVidia/CUDA** / **AMD/ROCm** / **Intel/OneAPI** are auto-detected if present and available,
> [!TIP]
> - Server can run without virtual environment,
Recommended to use `VENV` to avoid library version conflicts with other applications
> - **nVidia/CUDA** / **AMD/ROCm** / **Intel/OneAPI** are auto-detected if present and available,
For any other use case such as **DirectML**, **ONNX/Olive**, **OpenVINO** specify required parameter explicitly
or wrong packages may be installed as installer will assume CPU-only environment
- Full startup sequence is logged in `sdnext.log`, so if you encounter any issues, please check it first
> - Full startup sequence is logged in `sdnext.log`,
so if you encounter any issues, please check it first
### Run
@@ -161,9 +164,9 @@ SD.Next comes with several extensions pre-installed:
### **Collab**
- To avoid having this repo rely just on me, I'd love to have additional maintainers with full admin rights. If you're interested, ping me!
- We'd love to have additional maintainers with full admin rights. If you're interested, ping us!
- In addition to general cross-platform code, desire is to have a lead for each of the main platforms.
This should be fully cross-platform, but I would really love to have additional contributors and/or maintainers to join and help lead the efforts on different platforms.
This should be fully cross-platform, but we'd really love to have additional contributors and/or maintainers to join and help lead the efforts on different platforms.
### **Goals**
@@ -172,7 +175,7 @@ but although it diverged considerably, any substantial features to original work
The idea behind the fork is to enable latest technologies and advances in text-to-image generation.
> *Sometimes this is not the same as "as simple as possible to use".*
*Sometimes this is not the same as "as simple as possible to use".*
General goals:
cli/process.py
+1 -1
View File
@@ -182,7 +182,7 @@ def interrogate_image(res: Result, tag: str = None):
def resize_image(res: Result):
resized = res.image
resized.thumbnail((options.process.target_size, options.process.target_size), Image.HAMMING)
resized.thumbnail((options.process.target_size, options.process.target_size), Image.Resampling.HAMMING)
res.image = resized
res.ops.append('resize')
return res
cli/requirements.txt
-2
View File
@@ -2,5 +2,3 @@ mediapipe
colormap
filetype
albumentations
accelerate==0.19.0
diffusers==0.10.2
cli/run-benchmark.py
+1 -2
View File
@@ -28,8 +28,7 @@ options = Map({
# batch = [1, 1, 2, 4, 8, 12, 16, 24, 32, 48, 64, 96, 128]
# batch = [1, 1, 2, 4, 8, 12, 16]
batch = [4, 4]
batch = [1, 1, 2, 4, 8, 12, 16]
oom = 0
cli/train.py
+7 -2
View File
@@ -109,7 +109,7 @@ def parse_args():
group_train.add_argument('--algo', type=str, default=None, choices=['locon', 'loha', 'lokr', 'ia3'], required=False, help='alternative lyco algoritm, default: %(default)s')
group_train.add_argument('--args', type=str, default=None, required=False, help='lora/lyco additional network arguments, default: %(default)s')
group_train.add_argument('--optimizer', type=str, default='AdamW', required=False, help='optimizer type, default: %(default)s')
group_train.add_argument('--precision', type=str, choices=['fp16', 'fp32'], default='fp16', required=True, help='training precision, default: %(default)s')
group_train.add_argument('--precision', type=str, choices=['fp16', 'fp32'], default='fp16', required=False, help='training precision, default: %(default)s')
group_train.add_argument('--sdxl', default = False, action='store_true', help = "run sdxl training, default: %(default)s")
# AdamW (default), AdamW8bit, PagedAdamW8bit, Lion8bit, PagedLion8bit, Lion, SGDNesterov, SGDNesterov8bit, DAdaptation(DAdaptAdamPreprint), DAdaptAdaGrad, DAdaptAdam, DAdaptAdan, DAdaptAdanIP, DAdaptLion, DAdaptSGD, AdaFactor
@@ -164,8 +164,13 @@ def verify_args():
exit(1)
base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if args.type == 'lora' and not server_ok and not args.dir:
log.error('offline lora training requires lora')
log.error('offline lora training requires --dir <lora folder>')
exit(1)
if args.type == 'lora':
import transformers
if transformers.__version__ != '4.30.2':
log.error(f'lora training requires specific transformers version: current {transformers.__version__} required transformers==4.30.2')
exit(1)
args.lora_dir = server_options.options.lora_dir or args.dir
if not os.path.isabs(args.lora_dir):
args.lora_dir = os.path.join(base_dir, args.lora_dir)
extensions-builtin/Lora/extra_networks_lora.py
+2
View File
@@ -64,6 +64,8 @@ class ExtraNetworkLora(extra_networks.ExtraNetwork):
self.active = False
def deactivate(self, p):
if shared.backend == shared.Backend.DIFFUSERS and hasattr(shared.sd_model, "unload_lora_weights"):
shared.sd_model.unload_lora_weights()
if not self.active and getattr(networks, "originals", None ) is not None:
networks.originals.undo() # remove patches
if networks.debug:
extensions-builtin/Lora/lora_patches.py
+3 -3
View File
@@ -1,6 +1,6 @@
import torch
import networks
from modules import patches
from modules import patches, shared
class LoraPatches:
@@ -18,7 +18,7 @@ class LoraPatches:
self.MultiheadAttention_load_state_dict = None
def apply(self):
if self.active:
if self.active or (shared.opts.cuda_compile and shared.opts.cuda_compile_backend == "openvino_fx"): # OpenVINO only works with Diffusers LoRa loading
return
self.Linear_forward = patches.patch(__name__, torch.nn.Linear, 'forward', networks.network_Linear_forward)
self.Linear_load_state_dict = patches.patch(__name__, torch.nn.Linear, '_load_from_state_dict', networks.network_Linear_load_state_dict)
@@ -36,7 +36,7 @@ class LoraPatches:
self.active = True
def undo(self):
if not self.active:
if not self.active or (shared.opts.cuda_compile and shared.opts.cuda_compile_backend == "openvino_fx"): # OpenVINO only works with Diffusers LoRa loading
return
self.Linear_forward = patches.undo(__name__, torch.nn.Linear, 'forward') # pylint: disable=E1128
self.Linear_load_state_dict = patches.undo(__name__, torch.nn.Linear, '_load_from_state_dict') # pylint: disable=E1128
extensions-builtin/Lora/networks.py
+36 -5
View File
@@ -14,7 +14,7 @@ import network_norm
import lora_convert
import torch
import diffusers.models.lora
from modules import shared, devices, sd_models, errors, scripts, sd_hijack
from modules import shared, devices, sd_models, sd_models_compile, errors, scripts, sd_hijack
debug = os.environ.get('SD_LORA_DEBUG', None)
@@ -74,6 +74,26 @@ def assign_network_names_to_compvis_modules(sd_model):
sd_model.network_layer_mapping = network_layer_mapping
def load_diffusers(name, network_on_disk, lora_scale=1.0):
t0 = time.time()
cached = lora_cache.get(name, None)
# if debug:
shared.log.debug(f'LoRA load: name={name} file={network_on_disk.filename} type=diffusers {"cached" if cached else ""}')
if cached is not None:
return cached
if shared.backend != shared.Backend.DIFFUSERS:
return None
shared.sd_model.load_lora_weights(network_on_disk.filename)
if shared.opts.cuda_compile and shared.opts.cuda_compile_backend == "openvino_fx":
shared.sd_model.fuse_lora(lora_scale=lora_scale)
net = network.Network(name, network_on_disk)
net.mtime = os.path.getmtime(network_on_disk.filename)
lora_cache[name] = net
t1 = time.time()
timer['load'] += t1 - t0
return net
def load_network(name, network_on_disk):
t0 = time.time()
cached = lora_cache.get(name, None)
@@ -128,11 +148,13 @@ def load_networks(names, te_multipliers=None, unet_multipliers=None, dyn_dims=No
for i, name in enumerate(names):
if shared.compiled_model_state.lora_model[i] != f"{name}:{te_multipliers[i] if te_multipliers else 1.0}":
recompile_model = True
shared.compiled_model_state.lora_model = []
break
else:
recompile_model = True
shared.compiled_model_state.lora_model = []
shared.compiled_model_state.lora_model = []
if recompile_model:
shared.compiled_model_state.lora_compile = True
sd_models.unload_model_weights(op='model')
shared.opts.cuda_compile = False
sd_models.reload_model_weights(op='model')
@@ -142,9 +164,18 @@ def load_networks(names, te_multipliers=None, unet_multipliers=None, dyn_dims=No
net = None
if network_on_disk is not None:
try:
net = load_network(name, network_on_disk)
if recompile_model:
shared.compiled_model_state.lora_model.append(f"{name}:{te_multipliers[i] if te_multipliers else 1.0}")
if shared.backend == shared.Backend.DIFFUSERS and (os.environ.get('SD_LORA_DIFFUSERS', None)
or getattr(network_on_disk, 'shorthash', None) == 'aaebf6360f7d' # lcm sd15
or getattr(network_on_disk, 'shorthash', None) == '3d18b05e4f56' # lcm sdxl
or (shared.opts.cuda_compile and shared.opts.cuda_compile_backend == "openvino_fx")):
# OpenVINO only works with Diffusers LoRa loading.
net = load_diffusers(name, network_on_disk, lora_scale=te_multipliers[i] if te_multipliers else 1.0)
else:
net = load_network(name, network_on_disk)
except Exception as e:
shared.log.error(f"LoRA load failed: file={network_on_disk.filename}")
shared.log.error(f"LoRA load failed: file={network_on_disk.filename} {e}")
if debug:
errors.display(e, f"LoRA load failed file={network_on_disk.filename}")
continue
@@ -170,7 +201,7 @@ def load_networks(names, te_multipliers=None, unet_multipliers=None, dyn_dims=No
if recompile_model:
shared.log.info("LoRA recompiling model")
sd_models.compile_diffusers(shared.sd_model)
sd_models_compile.compile_diffusers(shared.sd_model)
def network_restore_weights_from_backup(self: Union[torch.nn.Conv2d, torch.nn.Linear, torch.nn.GroupNorm, torch.nn.LayerNorm, torch.nn.MultiheadAttention, diffusers.models.lora.LoRACompatibleLinear, diffusers.models.lora.LoRACompatibleConv]):
extensions-builtin/Lora/ui_extra_networks_lora.py
+9 -2
View File
@@ -56,7 +56,8 @@ class ExtraNetworksPageLora(ui_extra_networks.ExtraNetworksPage):
words = [str(w).replace('.json', '') for w in words]
if words[0] == '{}':
words[0] = 0
tags[' '.join(words[1:])] = words[0]
tag = ' '.join(words[1:])
tags[tag] = words[0]
versions = info.get('modelVersions', []) # trigger words from info json
for v in versions:
possible_tags = v.get('trainedWords', [])
@@ -73,9 +74,15 @@ class ExtraNetworksPageLora(ui_extra_networks.ExtraNetworksPage):
if len(list(tags)) == 0:
tags = search
bad_chars = [';', ':', '<', ">", "*", '?', '\'', '\"']
clean_tags = {}
for k, v in tags.items():
tag = ''.join(i for i in k if not i in bad_chars)
clean_tags[tag] = v
item["info"] = info
item["description"] = self.find_description(l.filename, info) # use existing info instead of double-read
item["tags"] = tags
item["tags"] = clean_tags
item["search_term"] = f'{self.search_terms_from_path(l.filename)} {" ".join(tags.keys())} {" ".join(search.keys())}'
return item
html/light-teal.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 120 KiB

After

Width:  |  Height:  |  Size: 255 KiB

html/locale_en.json
+13 -1
View File
@@ -14,6 +14,7 @@
{"id":"","label":"⊗","localized":"","hint":"Clear prompt"},
{"id":"","label":"🗁","localized":"","hint":"Show/hide extra networks"},
{"id":"","label":"⇰","localized":"","hint":"Apply selected styles to current prompt"},
{"id":"","label":"⇨","localized":"","hint":"Apply preset to Manual Block Merge tab"},
{"id":"","label":"⇩","localized":"","hint":"Save parameters from last generated image as style template"},
{"id":"","label":"🕮","localized":"","hint":"Save parameters from last generated image as style template"},
{"id":"","label":"⇕","localized":"","hint":"Sort by: Name asc/desc, Size largest/smallest, Time newest/oldest"},
@@ -255,7 +256,18 @@
{"id":"","label":"specify model variant","localized":"","hint":""},
{"id":"","label":"specify model revision","localized":"","hint":""},
{"id":"","label":"huggingface token","localized":"","hint":""},
{"id":"","label":"huggingface mirror","localized":"","hint":""}
{"id":"","label":"huggingface mirror","localized":"","hint":""},
{"id":"","label":"Weights Clip","localized":"","hint":"Forced merged weights to be no heavier than the original model, preventing burn in and overly saturated models"},
{"id":"","label":"ReBasin","localized":"","hint":"Performs multiple merges with permutations in order to keep more features from both models"},
{"id":"","label":"Number of ReBasin Iterations","localized":"","hint":"Number of times to merge and permute the model before saving"},
{"id":"","label":"cpu","localized":"","hint":"Uses cpu and RAM only: slowest but least likely to OOM"},
{"id":"","label":"shuffle","localized":"","hint":"Loads full model in RAM and calculates on VRAM: Less speedup, suggested for SDXL merges"},
{"id":"","label":"cuda","localized":"","hint":"Loads models into VRAM automatically unloading current model: fastest option but unlikely to handle SDXL Models without OOM"},
{"id":"","label":"Base","localized":"","hint":"Text Encoder and a few unaligned keys (1 value)"},
{"id":"","label":"In Blocks","localized":"","hint":"Downsampling Blocks of the UNet (12 values for SD1.5, 9 values for SDXL)"},
{"id":"","label":"Mid Block","localized":"","hint":"Central Block of the UNet (1 value)"},
{"id":"","label":"Out Block","localized":"","hint":"Upsampling Blocks of the UNet (12 values for SD1.5, 9 values for SDXL)"},
{"id":"","label":"Preset Interpolation Ratio","localized":"","hint":"If two presets are selected, interpolate between them"}
],
"train tabs": [
{"id":"","label":"Preprocess","localized":"","hint":""},
html/reference.json
+5 -5
View File
@@ -17,7 +17,7 @@
"Segmind SSD-1B": {
"path": "segmind/SSD-1B",
"desc": "The Segmind Stable Diffusion Model (SSD-1B) offers a compact, efficient, and distilled version of the SDXL model. At 50% smaller and 60% faster than Stable Diffusion XL (SDXL), it provides quick and seamless performance without sacrificing image quality.",
"preview": "segmind--ssd-1b.jpg"
"preview": "segmind--SSD-1B.jpg"
},
"Segmind Tiny": {
"path": "segmind/tiny-sd",
@@ -27,17 +27,17 @@
"LCM SD-1.5 Dreamshaper 7": {
"path": "SimianLuo/LCM_Dreamshaper_v7",
"desc": "Latent Consistencey Models enable swift inference with minimal steps on any pre-trained LDMs, including Stable Diffusion. By distilling classifier-free guidance into the model's input, LCM can generate high-quality images in very short inference time. LCM can generate quality images in as few as 3-4 steps, making it blazingly fast.",
"preview": "simianluo--lcm_dreamshaper_v7.jpg"
"preview": "SimianLuo--LCM_Dreamshaper_v7.jpg"
},
"Pixart-α XL 2 Medium 512": {
"path": "PixArt-alpha/PixArt-XL-2-512x512",
"desc": "PixArt-α is a Transformer-based T2I diffusion model whose image generation quality is competitive with state-of-the-art image generators (e.g., Imagen, SDXL, and even Midjourney), and the training speed markedly surpasses existing large-scale T2I models. Extensive experiments demonstrate that PIXART-α excels in image quality, artistry, and semantic control. It can directly generate 512px images from text prompts within a single sampling process.",
"preview": "pixart-alpha--pixart-xl-2-512x512.jpg"
"preview": "PixArt-alpha--PixArt-XL-2-512x512.jpg"
},
"Pixart-α XL 2 Large 1024": {
"path": "PixArt-alpha/PixArt-XL-2-1024-MS",
"desc": "PixArt-α is a Transformer-based T2I diffusion model whose image generation quality is competitive with state-of-the-art image generators (e.g., Imagen, SDXL, and even Midjourney), and the training speed markedly surpasses existing large-scale T2I models. Extensive experiments demonstrate that PIXART-α excels in image quality, artistry, and semantic control. It can directly generate 1024px images from text prompts within a single sampling process.",
"preview": "pixart-alpha--pixart-xl-2-1024-ms.jpg"
"preview": "PixArt-alpha--PixArt-XL-2-1024-MS.jpg"
},
"Warp Wuerstchen": {
"path": "warp-ai/wuerstchen",
@@ -57,7 +57,7 @@
"DeepFloyd IF Medium": {
"path": "DeepFloyd/IF-I-M-v1.0",
"desc": "DeepFloyd-IF is a pixel-based text-to-image triple-cascaded diffusion model, that can generate pictures with new state-of-the-art for photorealism and language understanding. The result is a highly efficient model that outperforms current state-of-the-art models, achieving a zero-shot FID-30K score of 6.66 on the COCO dataset. It is modular and composed of frozen text mode and three pixel cascaded diffusion modules, each designed to generate images of increasing resolution: 64x64, 256x256, and 1024x1024.",
"preview": "deepfloyd--if-i-m-v1.0.jpg"
"preview": "DeepFloyd--IF-I-M-v1.0.jpg"
},
"Tsinghua UniDiffuser": {
"path": "thu-ml/unidiffuser-v1",
installer.py
+17 -15
View File
@@ -190,7 +190,7 @@ def installed(package, friendly: str = None, reload = False, quiet = False):
if args.experimental:
log.warning(f"Package allowing experimental: {p[0]} {package_version} required {p[1]}")
else:
log.warning(f"Package wrong version: {p[0]} {package_version} required {p[1]}")
log.warning(f"Package version mismatch: {p[0]} {package_version} required {p[1]}")
else:
if not quiet:
log.debug(f"Package not found: {p[0]}")
@@ -442,12 +442,11 @@ def check_torch():
torchvision_pip = 'https://github.com/Nuullll/intel-extension-for-pytorch/releases/download/v2.0.110%2Bxpu-master%2Bdll-bundle/torchvision-0.15.2a0+fa99a53-cp310-cp310-win_amd64.whl'
ipex_pip = 'https://github.com/Nuullll/intel-extension-for-pytorch/releases/download/v2.0.110%2Bxpu-master%2Bdll-bundle/intel_extension_for_pytorch-2.0.110+gitc6ea20b-cp310-cp310-win_amd64.whl'
torch_command = os.environ.get('TORCH_COMMAND', f'{pytorch_pip} {torchvision_pip} {ipex_pip}')
uninstall('openvino-nightly')
install('openvino', 'openvino', ignore=True)
install('onnxruntime-openvino', 'onnxruntime-openvino', ignore=True) # TODO numpy version conflicts with tensorflow and doesn't support Python 3.11
elif allow_openvino and args.use_openvino:
log.info('Using OpenVINO')
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.1.0 torchvision==0.16.0 --index-url https://download.pytorch.org/whl/cpu')
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.1.1 torchvision==0.16.1 --index-url https://download.pytorch.org/whl/cpu')
else:
machine = platform.machine()
if sys.platform == 'darwin':
@@ -747,26 +746,29 @@ def install_requirements():
# set environment variables controling the behavior of various libraries
def set_environment():
log.debug('Setting environment tuning')
os.environ.setdefault('USE_TORCH', '1')
os.environ.setdefault('TF_CPP_MIN_LOG_LEVEL', '2')
os.environ.setdefault('ACCELERATE', 'True')
os.environ.setdefault('FORCE_CUDA', '1')
os.environ.setdefault('ATTN_PRECISION', 'fp16')
os.environ.setdefault('PYTORCH_CUDA_ALLOC_CONF', 'garbage_collection_threshold:0.8,max_split_size_mb:512')
os.environ.setdefault('CUDA_LAUNCH_BLOCKING', '0')
os.environ.setdefault('CUDA_CACHE_DISABLE', '0')
os.environ.setdefault('CUDA_AUTO_BOOST', '1')
os.environ.setdefault('CUDA_MODULE_LOADING', 'LAZY')
os.environ.setdefault('CUDA_CACHE_DISABLE', '0')
os.environ.setdefault('CUDA_DEVICE_DEFAULT_PERSISTING_L2_CACHE_PERCENTAGE_LIMIT', '0')
os.environ.setdefault('CUDA_LAUNCH_BLOCKING', '0')
os.environ.setdefault('CUDA_MODULE_LOADING', 'LAZY')
os.environ.setdefault('TORCH_CUDNN_V8_API_ENABLED', '1')
os.environ.setdefault('FORCE_CUDA', '1')
os.environ.setdefault('GRADIO_ANALYTICS_ENABLED', 'False')
os.environ.setdefault('SAFETENSORS_FAST_GPU', '1')
os.environ.setdefault('HF_HUB_DISABLE_EXPERIMENTAL_WARNING', '1')
os.environ.setdefault('HF_HUB_DISABLE_TELEMETRY', '1')
os.environ.setdefault('K_DIFFUSION_USE_COMPILE', '0')
os.environ.setdefault('NUMEXPR_MAX_THREADS', '16')
os.environ.setdefault('PYTHONHTTPSVERIFY', '0')
os.environ.setdefault('HF_HUB_DISABLE_TELEMETRY', '1')
os.environ.setdefault('HF_HUB_DISABLE_EXPERIMENTAL_WARNING', '1')
os.environ.setdefault('UVICORN_TIMEOUT_KEEP_ALIVE', '60')
os.environ.setdefault('K_DIFFUSION_USE_COMPILE', '0')
os.environ.setdefault('PYTORCH_CUDA_ALLOC_CONF', 'garbage_collection_threshold:0.8,max_split_size_mb:512')
os.environ.setdefault('SAFETENSORS_FAST_GPU', '1')
os.environ.setdefault('TF_CPP_MIN_LOG_LEVEL', '2')
os.environ.setdefault('TF_ENABLE_ONEDNN_OPTS', '0')
os.environ.setdefault('USE_TORCH', '1')
os.environ.setdefault('UVICORN_TIMEOUT_KEEP_ALIVE', '60')
os.environ.setdefault('HF_HUB_CACHE', opts.get('hfcache_dir', os.path.join(os.path.expanduser('~'), '.cache', 'huggingface', 'hub')))
log.debug(f'Cache folder: {os.environ.get("HF_HUB_CACHE")}')
if sys.platform == 'darwin':
os.environ.setdefault('PYTORCH_ENABLE_MPS_FALLBACK', '1')
javascript/extraNetworks.js
+23 -2
View File
@@ -40,7 +40,7 @@ const setENState = (state) => {
// methods
function showCardDetails(event) {
console.log('showCardDetails', event)
console.log('showCardDetails', event);
const tabname = getENActiveTab();
const btn = gradioApp().getElementById(`${tabname}_extra_details_btn`);
btn.click();
@@ -183,7 +183,7 @@ function sortExtraNetworks() {
return `sort page ${pagename} cards ${num} by ${desc}`;
}
function refreshExtraNetworks(tabname) {
function refreshENInput(tabname) {
log('refreshExtraNetworks', tabname, gradioApp().querySelector(`#${tabname}_extra_networks textarea`)?.value);
gradioApp().querySelector(`#${tabname}_extra_networks textarea`)?.dispatchEvent(new Event('input'));
}
@@ -234,6 +234,27 @@ function quickSaveStyle() {
if (btnSave) btnSave.click();
}
let enDirty = false;
function closeDetailsEN(args) {
// log('closeDetailsEN');
enDirty = true;
const tabname = getENActiveTab();
const btnClose = gradioApp().getElementById(`${tabname}_extra_details_close`);
if (btnClose) setTimeout(() => btnClose.click(), 100);
const btnRefresh = gradioApp().getElementById(`${tabname}_extra_refresh`);
if (btnRefresh && enDirty) setTimeout(() => btnRefresh.click(), 100);
return args;
}
function refeshDetailsEN(args) {
log(`refeshDetailsEN: ${enDirty}`);
const tabname = getENActiveTab();
const btnRefresh = gradioApp().getElementById(`${tabname}_extra_refresh`);
if (btnRefresh && enDirty) setTimeout(() => btnRefresh.click(), 100);
enDirty = false;
return args;
}
// init
function setupExtraNetworksForTab(tabname) {
javascript/loader.js
+4 -1
View File
@@ -39,7 +39,10 @@ async function createSplash() {
document.getElementById('splash').insertAdjacentHTML('afterbegin', imgEl);
fetch('/sdapi/v1/motd')
.then((res) => res.text())
.then((text) => document.getElementById('motd').innerHTML = text.replace(/["]+/g, ''))
.then((text) => {
const motdEl = document.getElementById('motd');
if (motdEl) motdEl.innerHTML = text.replace(/["]+/g, '');
})
.catch((err) => console.error('getMOTD:', err));
}
javascript/logMonitor.js
+1 -1
View File
@@ -39,7 +39,7 @@ async function initLogMonitor() {
<table id="logMonitor" style="width: 100%;">
<thead style="display: block; text-align: left; border-bottom: solid 1px var(--button-primary-border-color)">
<tr>
<th style="width: 170px">Time</th>
<th style="width: 160px">Time</th>
<th>Level</th>
<th style="width: 72px">Facility</th>
<th style="width: 124px">Module</th>
javascript/sdnext.css
+71 -19
View File
@@ -94,7 +94,8 @@ div#extras_scale_to_tab div.form{ flex-direction: row; }
#mode_img2img .gradio-image > div.fixed-height, #mode_img2img .gradio-image > div.fixed-height img{ height: 480px !important; max-height: 480px !important; min-height: 480px !important; }
#img2img_sketch, #img2maskimg, #inpaint_sketch { overflow: overlay !important; resize: auto; background: var(--panel-background-fill); z-index: 5; }
.image-buttons button{ min-width: auto; }
.infotext { overflow-wrap: break-word; }
.infotext { overflow-wrap: break-word; line-height: 1.5em; }
.infotext > p { padding-left: 1em; text-indent: -1em; }
.tooltip { display: block; position: fixed; top: 1em; right: 1em; padding: 0.5em; background: var(--input-background-fill); color: var(--body-text-color); border: 1pt solid var(--button-primary-border-color);
width: 22em; min-height: 1.3em; font-size: 0.8em; transition: opacity 0.2s ease-in; pointer-events: none; opacity: 0; z-index: 999; }
.tooltip-show { opacity: 0.9; }
@@ -279,25 +280,76 @@ table.settings-value-table td { padding: 0.4em; border: 1px solid #ccc; max-widt
--spacing-xxl: 6px;
}
/* Apply different styles for devices with coarse pointers dependant on screen resolution */
@media (hover: none) and (pointer: coarse) {
/* Screens 400px and smaller */
@media (max-width: 425px) {
:root, .light, .dark { --left-column: 100%; }
#settings {display: flex;gap: var(--layout-gap);flex-wrap: wrap;flex-direction: row;}
#txt2img_results, #img2img_results, #extras_results { background-color: var(--background-color); padding: 0; min-width: calc(min(320px, 100%)) !important;}
.gradio-slider input[type="number"] { width: 4em; font-size: 0.8rem; height: 16px; text-align: center; }
}
/* Screens 400px and larger up to 1080px */
@media (max-width: 1080px) and (min-width:425px) {
:root, .light, .dark { --left-column: 39%; }
#scripts_alwayson_txt2img div { max-width: 99%; }
#settings {display: flex;gap: var(--layout-gap);flex-wrap: wrap;flex-direction: row;}
#txt2img_results, #img2img_results, #extras_results { background-color: var(--background-color); padding: 0;}
.gradio-slider input[type="number"] { width: 4em; font-size: 0.8rem; height: 16px; text-align: center; }
}
/* Do not affect displays larger than 1024px wide. */
@media (max-width: 1024px) {
/* Screens smaller than 424px wide */
@media (max-width: 399px) {
:root, .light, .dark { --left-column: 100%; }
/* maintain single column for from image operations on larger mobile devices */
#txt2img_results, #img2img_results, #extras_results { min-width: calc(min(320px, 100%)) !important;}
}
/* Screens larger than 425px wide */
@media (min-width: 425px) {
:root, .light, .dark {--left-column: 50% ;}
/* adjust extension panel to fit within resized sidebar */
#scripts_alwayson_txt2img div { max-width: 99%; }
/* maintain side by side split on larger mobile displays for from text */
#txt2img_results, #extras_results { min-width: 50% !important;}
}
#txt2img_prompt_container, #img2img_prompt_container { resize:vertical !important; }
/* make generate and enqueue buttons take up the entire width of their rows. */
#txt2img_generate_box, #txt2img_enqueue_wrapper { min-width: 100% !important;}
/*make interrogate buttons take up appropriate space. */
#img2img_toprow > div.gradio-column {flex-grow: 1 !important;}
#img2img_actions_column {display: flex; min-width: fit-content !important; flex-direction: row;justify-content: space-evenly; align-items: center;}
#txt2img_generate_box, #img2img_generate_box, #txt2img_enqueue_wrapper,#img2img_enqueue_wrapper {display: flex;flex-direction: column;height: 4em !important;align-items: stretch;justify-content: space-evenly;}
/* maintain single column for from image operations on larger mobile devices */
#img2img_settings, #img2img_results { min-width: 100% !important; max-width: 100% !important;}
/* fix inpaint image display being too large for mobile displays */
#img2img_sketch, #img2maskimg, #inpaint_sketch {display: flex;alignment-baseline:after-edge !important;overflow: auto !important;resize: none !important;}
#img2maskimg canvas { width: 100% !important; max-height: 100% !important; height: auto !important; }
/* fix from text/image UI elements to prevent them from moving around within the UI */
#txt2img_sampler, #txt2img_batch, #txt2img_seed_group, #txt2img_advanced, #txt2img_second_pass, #img2img_sampling_group, #img2img_resize_group, #img2img_batch_group, #img2img_seed_group, #img2img_denoise_group, #img2img_advanced_group { width: 100% !important; }
#img2img_resize_group .gradio-radio > div { display: flex; flex-direction: column; width: unset !important; }
#inpaint_controls div {display:flex;flex-direction: row;}
#inpaint_controls .gradio-radio > div { display: flex; flex-direction: column !important; }
/* move image preview/output on models page to bottom of page */
#models_tab { flex-direction: column-reverse !important; }
/* fix settings for agent scheduler */
#enqueue_keyboard_shortcut_modifiers, #enqueue_keyboard_shortcut_key div { max-width: 40% !important;}
/* adjust width of certain settings item to allow aligning as row, but not have it go off the screen */
#settings { display: flex; flex-direction: row; flex-wrap: wrap; max-width: 100% !important; }
#settings div.tab-content > div > div > div { max-width: 80% !important;}
#settings div .gradio-radio { width: unset !important; }
/* enable scrolling on extensions tab */
#tab_extensions table { border-collapse: collapse; display: block; overflow-x:auto !important;}
/* increase scrollbar size to make it finger friendly */
::-webkit-scrollbar { width: 25px !important; height:25px; }
/* adjust dropdown size to make them easier to select individual items on mobile. */
.gradio-dropdown ul.options {max-height: 41vh !important; }
.gradio-dropdown ul.options li.item {height: 40px !important; display: flex; align-items: center;}
/* adjust slider input fields as they were too large for mobile devices. */
.gradio-slider input[type="number"] { width: 4em; font-size: 0.8rem; height: 16px; text-align: center; }
#txt2img_settings .block .padded:not(.gradio-accordion) {padding: 0 !important;margin-right: 0; min-width: 100% !important; width:100% !important;}
}
}
launch.py
+1 -1
View File
@@ -233,7 +233,7 @@ if __name__ == "__main__":
if round(time.time()) % 120 == 0:
state = f'job="{instance.state.job}" {instance.state.job_no}/{instance.state.job_count}' if instance.state.job != '' or instance.state.job_no != 0 or instance.state.job_count != 0 else 'idle'
uptime = round(time.time() - instance.state.server_start)
installer.log.debug(f'Server: alive={alive} jobs={instance.state.total_jobs} requests={requests} uptime={uptime} memory={get_memory_stats()} backend={instance.backend} {state}')
installer.log.debug(f'Server: alive={alive} jobs={instance.state.total_jobs} requests={requests} uptime={uptime} memory={get_memory_stats()} backend={instance.backend} state={state}')
if not alive:
if uv is not None and uv.wants_restart:
installer.log.info('Server restarting...')
models/Reference/deepfloyd--if-i-m-v1.0.jpg → models/Reference/DeepFloyd--IF-I-M-v1.0.jpg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 93 KiB

After

Width:  |  Height:  |  Size: 93 KiB

models/Reference/pixart-alpha--pixart-xl-2-1024-ms.jpg → models/Reference/PixArt-alpha--PixArt-XL-2-1024-MS.jpg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 79 KiB

After

Width:  |  Height:  |  Size: 79 KiB

models/Reference/pixart-alpha--pixart-xl-2-512x512.jpg → models/Reference/PixArt-alpha--PixArt-XL-2-512x512.jpg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

models/Reference/simianluo--lcm_dreamshaper_v7.jpg → models/Reference/SimianLuo--LCM_Dreamshaper_v7.jpg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 76 KiB

models/Reference/deepfloyd--if-i-m-v1.0.thumb.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 36 KiB

models/Reference/kandinsky-community--kandinsky-2-1.thumb.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 43 KiB

models/Reference/kandinsky-community--kandinsky-3.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 118 KiB

models/Reference/pixart-alpha--pixart-xl-2-1024-ms.thumb.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 30 KiB

models/Reference/segmind--ssd-1b.jpg → models/Reference/segmind--SSD-1B.jpg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 114 KiB

After

Width:  |  Height:  |  Size: 114 KiB

models/Reference/segmind--ssd-1b.thumb.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 42 KiB

models/Reference/segmind--tiny-sd.thumb.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 29 KiB

models/Reference/simianluo--lcm_dreamshaper_v7.thumb.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 27 KiB

modules/api/api.py
+6 -3
View File
@@ -145,7 +145,7 @@ class Api:
self.add_api_route("/sdapi/v1/reload-checkpoint", self.reloadapi, methods=["POST"])
self.add_api_route("/sdapi/v1/scripts", self.get_scripts_list, methods=["GET"], response_model=models.ScriptsList)
self.add_api_route("/sdapi/v1/script-info", self.get_script_info, methods=["GET"], response_model=List[models.ScriptInfo])
self.add_api_route("/sdapi/v1/log", self.get_log_buffer, methods=["GET"], response_model=List) # bypass auth
self.add_api_route("/sdapi/v1/log", self.get_log_buffer, methods=["GET"], response_model=List)
self.add_api_route("/sdapi/v1/start", self.session_start, methods=["GET"])
self.add_api_route("/sdapi/v1/motd", self.get_motd, methods=["GET"], response_model=str)
self.add_api_route("/sdapi/v1/extra-networks", self.get_extra_networks, methods=["GET"], response_model=List[models.ExtraNetworkItem])
@@ -153,11 +153,12 @@ class Api:
self.default_script_arg_img2img = []
def add_api_route(self, path: str, endpoint, **kwargs):
if shared.cmd_opts.auth or shared.cmd_opts.auth_file:
if (shared.cmd_opts.auth or shared.cmd_opts.auth_file) and shared.cmd_opts.api_only:
return self.app.add_api_route(path, endpoint, dependencies=[Depends(self.auth)], **kwargs)
return self.app.add_api_route(path, endpoint, **kwargs)
def auth(self, credentials: HTTPBasicCredentials = Depends(HTTPBasic())):
# this is only needed for api-only since otherwise auth is handled in gradio/routes.py
if credentials.username in self.credentials:
if compare_digest(credentials.password, self.credentials[credentials.username]):
return True
@@ -170,7 +171,9 @@ class Api:
return lines
def session_start(self, req: Request, agent: Optional[str] = None):
shared.log.info(f'Browser session: client={req.client.host} agent={agent}')
token = req.cookies.get("access-token") or req.cookies.get("access-token-unsecure")
user = self.app.tokens.get(token)
shared.log.info(f'Browser session: user={user} client={req.client.host} agent={agent}')
return {}
def get_motd(self):
modules/devices.py
+1 -1
View File
@@ -134,7 +134,7 @@ def torch_gc(force=False):
if oom > previous_oom:
previous_oom = oom
log.warning(f'GPU out-of-memory error: {mem}')
if used > shared.opts.torch_gc_threshold:
if used >= shared.opts.torch_gc_threshold:
log.info(f'GPU high memory utilization: {used}% {mem}')
force = True
if not force:
modules/extras.py
+104 -138
View File
@@ -1,18 +1,17 @@
import os
import re
import html
import json
import time
import shutil
import torch
import tqdm
import gradio as gr
import safetensors.torch
from modules.merging.merge import merge_models
from modules.merging.merge_utils import TRIPLE_METHODS
from modules import shared, images, sd_models, sd_vae, sd_models_config
checkpoint_dict_skip_on_merge = ["cond_stage_model.transformer.text_model.embeddings.position_ids"]
from modules import shared, images, sd_models, sd_vae, sd_models_config, devices
def run_pnginfo(image):
@@ -31,6 +30,7 @@ def create_config(ckpt_result, config_source, a, b, c):
def config(x):
res = sd_models_config.find_checkpoint_config_near_filename(x) if x else None
return res if res != shared.sd_default_config else None
if config_source == 0:
cfg = config(a) or config(b) or config(c)
elif config_source == 1:
@@ -53,158 +53,119 @@ def to_half(tensor, enable):
return tensor
def run_modelmerger(id_task, primary_model_name, secondary_model_name, tertiary_model_name, interp_method, multiplier, save_as_half, custom_name, checkpoint_format, config_source, bake_in_vae, discard_weights, save_metadata): # pylint: disable=unused-argument
def run_modelmerger(id_task, **kwargs): # pylint: disable=unused-argument
shared.state.begin('merge')
save_as_half = save_as_half == 0
t0 = time.time()
def fail(message):
shared.state.textinfo = message
shared.state.end()
return [*[gr.update() for _ in range(4)], message]
def weighted_sum(theta0, theta1, alpha):
return ((1 - alpha) * theta0) + (alpha * theta1)
def get_difference(theta1, theta2):
return theta1 - theta2
def add_difference(theta0, theta1_2_diff, alpha):
return theta0 + (alpha * theta1_2_diff)
def filename_weighted_sum():
a = primary_model_info.model_name
b = secondary_model_info.model_name
Ma = round(1 - multiplier, 2)
Mb = round(multiplier, 2)
return f"{Ma}({a}) + {Mb}({b})"
def filename_add_difference():
a = primary_model_info.model_name
b = secondary_model_info.model_name
c = tertiary_model_info.model_name
M = round(multiplier, 2)
return f"{a} + {M}({b} - {c})"
def filename_nothing():
return primary_model_info.model_name
theta_funcs = {
"Weighted sum": (filename_weighted_sum, None, weighted_sum),
"Add difference": (filename_add_difference, get_difference, add_difference),
"No interpolation": (filename_nothing, None, None),
kwargs["models"] = {
"model_a": sd_models.get_closet_checkpoint_match(kwargs.get("primary_model_name", None)).filename,
"model_b": sd_models.get_closet_checkpoint_match(kwargs.get("secondary_model_name", None)).filename,
}
filename_generator, theta_func1, theta_func2 = theta_funcs[interp_method]
shared.state.job_count = (1 if theta_func1 else 0) + (1 if theta_func2 else 0)
if not primary_model_name or primary_model_name == 'None':
if kwargs.get("primary_model_name", None) in [None, 'None']:
return fail("Failed: Merging requires a primary model.")
primary_model_info = sd_models.checkpoints_list[primary_model_name]
if theta_func2 and (not secondary_model_name or secondary_model_name == 'None'):
primary_model_info = sd_models.get_closet_checkpoint_match(kwargs.get("primary_model_name", None))
if kwargs.get("secondary_model_name", None) in [None, 'None']:
return fail("Failed: Merging requires a secondary model.")
secondary_model_info = sd_models.checkpoints_list[secondary_model_name] if theta_func2 else None
if theta_func1 and (not tertiary_model_name or tertiary_model_name == 'None'):
return fail(f"Failed: Interpolation method ({interp_method}) requires a tertiary model.")
tertiary_model_info = sd_models.checkpoints_list[tertiary_model_name] if theta_func1 else None
result_is_inpainting_model = False
result_is_instruct_pix2pix_model = False
if theta_func2:
shared.state.textinfo = "Loading B"
shared.log.info(f"Model merge loading secondary model: {secondary_model_info.filename}")
theta_1 = sd_models.read_state_dict(secondary_model_info.filename)
secondary_model_info = sd_models.get_closet_checkpoint_match(kwargs.get("secondary_model_name", None))
if kwargs.get("tertiary_model_name", None) in [None, 'None'] and kwargs.get("merge_mode", None) in TRIPLE_METHODS:
return fail(f"Failed: Interpolation method ({kwargs.get('merge_mode', None)}) requires a tertiary model.")
tertiary_model_info = sd_models.get_closet_checkpoint_match(kwargs.get("tertiary_model_name", None)) if kwargs.get("merge_mode", None) in TRIPLE_METHODS else None
del kwargs["primary_model_name"]
del kwargs["secondary_model_name"]
if kwargs.get("tertiary_model_name", None) is not None:
kwargs["models"] |= {"model_c": sd_models.get_closet_checkpoint_match(kwargs.get("tertiary_model_name", None)).filename}
del kwargs["tertiary_model_name"]
if hasattr(kwargs, "alpha_base") and hasattr(kwargs, "alpha_in_blocks") and hasattr(kwargs, "alpha_mid_block") and hasattr(kwargs, "alpha_out_blocks"):
try:
alpha = [float(x) for x in
[kwargs["alpha_base"]] + kwargs["alpha_in_blocks"].split(",") + [kwargs["alpha_mid_block"]] + kwargs["alpha_out_blocks"].split(",")]
assert len(alpha) == 26 or len(alpha) == 20, "Alpha Block Weights are wrong length (26 or 20 for SDXL) falling back"
kwargs["alpha"] = alpha
except KeyError as ke:
shared.log.warning(f"Merge: Malformed manual block weight: {ke}")
elif hasattr(kwargs, "alpha_preset") or hasattr(kwargs, "alpha"):
kwargs["alpha"] = kwargs.get("alpha_preset", kwargs["alpha"])
kwargs.pop("alpha_base", None)
kwargs.pop("alpha_in_blocks", None)
kwargs.pop("alpha_mid_block", None)
kwargs.pop("alpha_out_blocks", None)
kwargs.pop("alpha_preset", None)
if hasattr(kwargs, "beta_base") and hasattr(kwargs, "beta_in_blocks") and hasattr(kwargs, "beta_mid_block") and hasattr(kwargs, "beta_out_blocks"):
try:
beta = [float(x) for x in
[kwargs["beta_base"]] + kwargs["beta_in_blocks"].split(",") + [kwargs["beta_mid_block"]] + kwargs["beta_out_blocks"].split(",")]
assert len(beta) == 26 or len(beta) == 20, "Beta Block Weights are wrong length (26 or 20 for SDXL) falling back"
kwargs["beta"] = beta
except KeyError as ke:
shared.log.warning(f"Merge: Malformed manual block weight: {ke}")
elif hasattr(kwargs, "beta_preset") or hasattr(kwargs, "beta"):
kwargs["beta"] = kwargs.get("beta_preset", kwargs["beta"])
kwargs.pop("beta_base", None)
kwargs.pop("beta_in_blocks", None)
kwargs.pop("beta_mid_block", None)
kwargs.pop("beta_out_blocks", None)
kwargs.pop("beta_preset", None)
if kwargs["device"] == "gpu":
kwargs["device"] = devices.device
elif kwargs["device"] == "shuffle":
kwargs["device"] = torch.device("cpu")
kwargs["work_device"] = devices.device
else:
theta_1 = None
if theta_func1:
shared.state.textinfo = "Loading C"
shared.log.info(f"Model merge loading tertiary model: {tertiary_model_info.filename}")
theta_2 = sd_models.read_state_dict(tertiary_model_info.filename)
shared.state.textinfo = 'Merging B and C'
shared.state.sampling_steps = len(theta_1.keys())
for key in tqdm.tqdm(theta_1.keys()):
if key in checkpoint_dict_skip_on_merge:
continue
if 'model' in key:
if key in theta_2:
t2 = theta_2.get(key, torch.zeros_like(theta_1[key]))
theta_1[key] = theta_func1(theta_1[key], t2)
else:
theta_1[key] = torch.zeros_like(theta_1[key])
shared.state.sampling_step += 1
del theta_2
shared.state.nextjob()
shared.state.textinfo = f"Loading {primary_model_info.filename}..."
shared.log.info(f"Model merge loading primary model: {primary_model_info.filename}")
theta_0 = sd_models.read_state_dict(primary_model_info.filename)
shared.log.info("Model merge: running")
shared.state.textinfo = 'Merging A and B'
shared.state.sampling_steps = len(theta_0.keys())
for key in tqdm.tqdm(theta_0.keys()):
if theta_1 and 'model' in key and key in theta_1:
if key in checkpoint_dict_skip_on_merge:
continue
a = theta_0[key]
b = theta_1[key]
# this enables merging an inpainting model (A) with another one (B);
# where normal model would have 4 channels, for latenst space, inpainting model would
# have another 4 channels for unmasked picture's latent space, plus one channel for mask, for a total of 9
if a.shape != b.shape and a.shape[0:1] + a.shape[2:] == b.shape[0:1] + b.shape[2:]:
if a.shape[1] == 4 and b.shape[1] == 9:
raise RuntimeError("When merging inpainting model with a normal one, A must be the inpainting model.")
if a.shape[1] == 4 and b.shape[1] == 8:
raise RuntimeError("When merging instruct-pix2pix model with a normal one, A must be the instruct-pix2pix model.")
if a.shape[1] == 8 and b.shape[1] == 4:#If we have an Instruct-Pix2Pix model...
theta_0[key][:, 0:4, :, :] = theta_func2(a[:, 0:4, :, :], b, multiplier)#Merge only the vectors the models have in common. Otherwise we get an error due to dimension mismatch.
result_is_instruct_pix2pix_model = True
else:
assert a.shape[1] == 9 and b.shape[1] == 4, f"Bad dimensions for merged layer {key}: A={a.shape}, B={b.shape}"
theta_0[key][:, 0:4, :, :] = theta_func2(a[:, 0:4, :, :], b, multiplier)
result_is_inpainting_model = True
else:
theta_0[key] = theta_func2(a, b, multiplier)
theta_0[key] = to_half(theta_0[key], save_as_half)
shared.state.sampling_step += 1
del theta_1
bake_in_vae_filename = sd_vae.vae_dict.get(bake_in_vae, None)
kwargs["device"] = torch.device("cpu")
if kwargs.pop("unload", False):
sd_models.unload_model_weights()
try:
theta_0 = merge_models(**kwargs)
except Exception as e:
return fail(f"{e}")
try:
theta_0 = theta_0.to_dict() #TensorDict -> Dict if necessary
except:
pass
bake_in_vae_filename = sd_vae.vae_dict.get(kwargs.get("bake_in_vae", None), None)
if bake_in_vae_filename is not None:
shared.log.info(f"Model merge: baking in VAE: {bake_in_vae_filename}")
shared.state.textinfo = 'Baking in VAE'
shared.log.info(f"Merge VAE='{bake_in_vae_filename}'")
shared.state.textinfo = 'Merge VAE'
vae_dict = sd_vae.load_vae_dict(bake_in_vae_filename)
for key in vae_dict.keys():
theta_0_key = 'first_stage_model.' + key
if theta_0_key in theta_0:
theta_0[theta_0_key] = to_half(vae_dict[key], save_as_half)
theta_0[theta_0_key] = to_half(vae_dict[key], kwargs.get("precision", "fp16") == "fp16")
del vae_dict
if save_as_half and not theta_func2:
for key in theta_0.keys():
theta_0[key] = to_half(theta_0[key], save_as_half)
if discard_weights:
regex = re.compile(discard_weights)
for key in list(theta_0):
if re.search(regex, key):
theta_0.pop(key, None)
ckpt_dir = shared.opts.ckpt_dir or sd_models.model_path
filename = filename_generator() if custom_name == '' else custom_name
filename += ".inpainting" if result_is_inpainting_model else ""
filename += ".instruct-pix2pix" if result_is_instruct_pix2pix_model else ""
filename += "." + checkpoint_format
filename = kwargs.get("custom_name", "Unnamed_Merge")
filename += "." + kwargs.get("checkpoint_format", None)
output_modelname = os.path.join(ckpt_dir, filename)
shared.state.nextjob()
shared.state.textinfo = "Saving"
shared.state.textinfo = "merge saving"
metadata = None
if save_metadata:
if kwargs.get("save_metadata", False):
metadata = {"format": "pt", "sd_merge_models": {}}
merge_recipe = {
"type": "webui", # indicate this model was merged with webui's built-in merger
"type": "SDNext", # indicate this model was merged with webui's built-in merger
"primary_model_hash": primary_model_info.sha256,
"secondary_model_hash": secondary_model_info.sha256 if secondary_model_info else None,
"tertiary_model_hash": tertiary_model_info.sha256 if tertiary_model_info else None,
"interp_method": interp_method,
"multiplier": multiplier,
"save_as_half": save_as_half,
"custom_name": custom_name,
"config_source": config_source,
"bake_in_vae": bake_in_vae,
"discard_weights": discard_weights,
"is_inpainting": result_is_inpainting_model,
"is_instruct_pix2pix": result_is_instruct_pix2pix_model
"merge_mode": kwargs.get('merge_mode', None),
"alpha": kwargs.get('alpha', None),
"beta": kwargs.get('beta', None),
"precision": kwargs.get('precision', None),
"custom_name": kwargs.get("custom_name", "Unamed_Merge"),
}
metadata["sd_merge_recipe"] = json.dumps(merge_recipe)
@@ -225,22 +186,27 @@ def run_modelmerger(id_task, primary_model_name, secondary_model_name, tertiary_
metadata["sd_merge_models"] = json.dumps(metadata["sd_merge_models"])
_, extension = os.path.splitext(output_modelname)
if os.path.exists(output_modelname) and not kwargs.get("overwrite", False):
return [*[gr.Dropdown.update(choices=sd_models.checkpoint_tiles()) for _ in range(4)], f"Model alredy exists: {output_modelname}"]
if extension.lower() == ".safetensors":
safetensors.torch.save_file(theta_0, output_modelname, metadata=metadata)
else:
torch.save(theta_0, output_modelname)
t1 = time.time()
shared.log.info(f"Merge complete: saved='{output_modelname}' time={t1-t0:.2f}")
sd_models.list_models()
created_model = next((ckpt for ckpt in sd_models.checkpoints_list.values() if ckpt.name == filename), None)
if created_model:
created_model.calculate_shorthash()
create_config(output_modelname, config_source, primary_model_info, secondary_model_info, tertiary_model_info)
shared.log.info(f"Model merge saved: {output_modelname}.")
shared.state.textinfo = "Checkpoint saved"
devices.torch_gc(force=True)
shared.state.end()
return [*[gr.Dropdown.update(choices=sd_models.checkpoint_tiles()) for _ in range(4)], "Checkpoint saved to " + output_modelname]
return [*[gr.Dropdown.update(choices=sd_models.checkpoint_tiles()) for _ in range(4)], f"Model saved to {output_modelname}"]
def run_modelconvert(model, checkpoint_formats, precision, conv_type, custom_name, unet_conv, text_encoder_conv, vae_conv, others_conv, fix_clip):
def run_modelconvert(model, checkpoint_formats, precision, conv_type, custom_name, unet_conv, text_encoder_conv,
vae_conv, others_conv, fix_clip):
# position_ids in clip is int64. model_ema.num_updates is int32
dtypes_to_fp16 = {torch.float32, torch.float64, torch.bfloat16}
dtypes_to_bf16 = {torch.float32, torch.float64, torch.float16}
@@ -278,7 +244,6 @@ def run_modelconvert(model, checkpoint_formats, precision, conv_type, custom_nam
state_dict = m["state_dict"] if "state_dict" in m else m
return state_dict
def fix_model(model, fix_clip=False):
# code from model-toolkit
nai_keys = {
@@ -340,6 +305,7 @@ def run_modelconvert(model, checkpoint_formats, precision, conv_type, custom_nam
ok[wk] = t
elif conv_t == "delete":
return
shared.log.info("Model convert: running")
if conv_type == "ema-only":
for k in tqdm.tqdm(state_dict):
modules/generation_parameters_copypaste.py
+1 -2
View File
@@ -212,7 +212,7 @@ def parse_generation_parameters(x: str):
remaining = x[7:] if x.startswith('Prompt: ') else x
remaining = x[11:] if x.startswith('parameters: ') else x
if 'Steps: ' in remaining and 'Negative prompt: ' not in remaining:
remaining = remaining.replace('Steps: ', 'Negative prompt: , Steps: ')
remaining = remaining.replace('Steps: ', 'Negative prompt: Steps: ')
prompt, remaining = remaining.strip().split('Negative prompt: ', maxsplit=1) if 'Negative prompt: ' in remaining else (remaining, '')
res["Prompt"] = prompt.strip()
negative, remaining = remaining.strip().split('Steps: ', maxsplit=1) if 'Steps: ' in remaining else (remaining, None)
@@ -247,7 +247,6 @@ infotext_to_setting_name_mapping = [
('VAE', 'sd_vae'),
('Parser', 'prompt_attention'),
('Color correction', 'img2img_color_correction'),
('LoRA method', 'diffusers_lora_loader'),
# Samplers
('Sampler Eta', 'scheduler_eta'),
('Sampler ENSD', 'eta_noise_seed_delta'),
modules/hashes.py
+1 -1
View File
@@ -16,7 +16,7 @@ def dump_cache():
def cache(subsection):
global cache_data # pylint: disable=global-statement
if cache_data is None:
cache_data = {} if not os.path.isfile(cache_filename) else shared.readfile(cache_filename)
cache_data = {} if not os.path.isfile(cache_filename) else shared.readfile(cache_filename, lock=True)
s = cache_data.get(subsection, {})
cache_data[subsection] = s
return s
modules/images.py
+42 -27
View File
@@ -18,10 +18,8 @@ import piexif.helper
from PIL import Image, ImageFont, ImageDraw, PngImagePlugin, ExifTags
from modules import sd_samplers, shared, script_callbacks, errors, paths
LANCZOS = (Image.Resampling.LANCZOS if hasattr(Image, 'Resampling') else Image.LANCZOS)
debug = errors.log.info if os.environ.get('SD_PATH_DEBUG', None) is not None else lambda *args, **kwargs: None
try:
from pi_heif import register_heif_opener
register_heif_opener()
@@ -56,7 +54,7 @@ def image_grid(imgs, batch_size=1, rows=None):
params = script_callbacks.ImageGridLoopParams(imgs, cols, rows)
script_callbacks.image_grid_callback(params)
w, h = imgs[0].size
grid = Image.new('RGB', size=(params.cols * w, params.rows * h), color='black')
grid = Image.new('RGB', size=(params.cols * w, params.rows * h), color=shared.opts.grid_background)
for i, img in enumerate(params.imgs):
grid.paste(img, box=(i % params.cols * w, i // params.cols * h))
return grid
@@ -122,7 +120,7 @@ class GridAnnotation:
self.size = None
def draw_grid_annotations(im, width, height, hor_texts, ver_texts, margin=0):
def draw_grid_annotations(im, width, height, hor_texts, ver_texts, margin=0, title=None):
def wrap(drawing, text, font, line_length):
lines = ['']
for word in text.split():
@@ -141,55 +139,63 @@ def draw_grid_annotations(im, width, height, hor_texts, ver_texts, margin=0):
def draw_texts(drawing: ImageDraw, draw_x, draw_y, lines, initial_fnt, initial_fontsize):
for line in lines:
fnt = initial_fnt
font = initial_fnt
fontsize = initial_fontsize
while drawing.multiline_textsize(line.text, font=fnt)[0] > line.allowed_width and fontsize > 0:
while drawing.multiline_textbbox((0,0), text=line.text, font=font)[0] > line.allowed_width and fontsize > 0:
fontsize -= 1
fnt = get_font(fontsize)
drawing.multiline_text((draw_x, draw_y + line.size[1] / 2), line.text, font=fnt, fill=color_active if line.is_active else color_inactive, anchor="mm", align="center")
font = get_font(fontsize)
drawing.multiline_text((draw_x, draw_y + line.size[1] / 2), line.text, font=font, fill=shared.opts.font_color if line.is_active else color_inactive, anchor="mm", align="center")
if not line.is_active:
drawing.line((draw_x - line.size[0] // 2, draw_y + line.size[1] // 2, draw_x + line.size[0] // 2, draw_y + line.size[1] // 2), fill=color_inactive, width=4)
draw_y += line.size[1] + line_spacing
fontsize = (width + height) // 25
line_spacing = fontsize // 2
fnt = get_font(fontsize)
color_active = (0, 0, 0)
color_inactive = (153, 153, 153)
font = get_font(fontsize)
color_inactive = (127, 127, 127)
pad_left = 0 if sum([sum([len(line.text) for line in lines]) for lines in ver_texts]) == 0 else width * 3 // 4
cols = im.width // width
rows = im.height // height
assert cols == len(hor_texts), f'bad number of horizontal texts: {len(hor_texts)}; must be {cols}'
assert rows == len(ver_texts), f'bad number of vertical texts: {len(ver_texts)}; must be {rows}'
calc_img = Image.new("RGB", (1, 1), "white")
calc_img = Image.new("RGB", (1, 1), shared.opts.grid_background)
calc_d = ImageDraw.Draw(calc_img)
for texts, allowed_width in zip(hor_texts + ver_texts, [width] * len(hor_texts) + [pad_left] * len(ver_texts)):
title_texts = [title] if title else [[GridAnnotation()]]
for texts, allowed_width in zip(hor_texts + ver_texts + title_texts, [width] * len(hor_texts) + [pad_left] * len(ver_texts) + [(width+margin)*cols]):
items = [] + texts
texts.clear()
for line in items:
wrapped = wrap(calc_d, line.text, fnt, allowed_width)
wrapped = wrap(calc_d, line.text, font, allowed_width)
texts += [GridAnnotation(x, line.is_active) for x in wrapped]
for line in texts:
bbox = calc_d.multiline_textbbox((0, 0), line.text, font=fnt)
bbox = calc_d.multiline_textbbox((0, 0), line.text, font=font)
line.size = (bbox[2] - bbox[0], bbox[3] - bbox[1])
line.allowed_width = allowed_width
hor_text_heights = [sum([line.size[1] + line_spacing for line in lines]) - line_spacing for lines in hor_texts]
ver_text_heights = [sum([line.size[1] + line_spacing for line in lines]) - line_spacing * len(lines) for lines in ver_texts]
pad_top = 0 if sum(hor_text_heights) == 0 else max(hor_text_heights) + line_spacing * 2
result = Image.new("RGB", (im.width + pad_left + margin * (cols-1), im.height + pad_top + margin * (rows-1)), "white")
title_pad = 0
if title:
title_text_heights = [sum([line.size[1] + line_spacing for line in lines]) - line_spacing for lines in title_texts] # pylint: disable=unsubscriptable-object
title_pad = 0 if sum(title_text_heights) == 0 else max(title_text_heights) + line_spacing * 2
result = Image.new("RGB", (im.width + pad_left + margin * (cols-1), im.height + pad_top + title_pad + margin * (rows-1)), shared.opts.grid_background)
for row in range(rows):
for col in range(cols):
cell = im.crop((width * col, height * row, width * (col+1), height * (row+1)))
result.paste(cell, (pad_left + (width + margin) * col, pad_top + (height + margin) * row))
result.paste(cell, (pad_left + (width + margin) * col, pad_top + title_pad + (height + margin) * row))
d = ImageDraw.Draw(result)
if title:
x = pad_left + ((width+margin)*cols) / 2
y = title_pad / 2 - title_text_heights[0] / 2
draw_texts(d, x, y, title_texts[0], font, fontsize)
for col in range(cols):
x = pad_left + (width + margin) * col + width / 2
y = pad_top / 2 - hor_text_heights[col] / 2
draw_texts(d, x, y, hor_texts[col], fnt, fontsize)
y = (pad_top / 2 - hor_text_heights[col] / 2) + title_pad
draw_texts(d, x, y, hor_texts[col], font, fontsize)
for row in range(rows):
x = pad_left / 2
y = pad_top + (height + margin) * row + height / 2 - ver_text_heights[row] / 2
draw_texts(d, x, y, ver_texts[row], fnt, fontsize)
y = (pad_top + (height + margin) * row + height / 2 - ver_text_heights[row] / 2) + title_pad
draw_texts(d, x, y, ver_texts[row], font, fontsize)
return result
@@ -204,6 +210,7 @@ def draw_prompt_matrix(im, width, height, all_prompts, margin=0):
def resize_image(resize_mode, im, width, height, upscaler_name=None, output_type='image'):
# shared.log.debug(f'Image resize: mode={resize_mode} resolution={width}x{height} upscaler={upscaler_name}')
"""
Resizes an image with the specified resize_mode, width, and height.
Args:
@@ -221,7 +228,7 @@ def resize_image(resize_mode, im, width, height, upscaler_name=None, output_type
def resize(im, w, h):
if upscaler_name is None or upscaler_name == "None" or im.mode == 'L':
return im.resize((w, h), resample=LANCZOS)
return im.resize((w, h), resample=Image.Resampling.LANCZOS)
scale = max(w / im.width, h / im.height)
if scale > 1.0:
upscalers = [x for x in shared.sd_upscalers if x.name == upscaler_name]
@@ -232,7 +239,7 @@ def resize_image(resize_mode, im, width, height, upscaler_name=None, output_type
upscaler = upscalers[0]
im = upscaler.scaler.upscale(im, scale, upscaler.data_path)
if im.width != w or im.height != h:
im = im.resize((w, h), resample=LANCZOS)
im = im.resize((w, h), resample=Image.Resampling.LANCZOS)
return im
if resize_mode == 0:
@@ -245,7 +252,7 @@ def resize_image(resize_mode, im, width, height, upscaler_name=None, output_type
src_w = width if ratio > src_ratio else im.width * height // im.height
src_h = height if ratio <= src_ratio else im.height * width // im.width
resized = resize(im, src_w, src_h)
res = Image.new("RGB", (width, height))
res = Image.new(im.mode, (width, height))
res.paste(resized, box=(width // 2 - src_w // 2, height // 2 - src_h // 2))
else:
ratio = width / height
@@ -253,7 +260,7 @@ def resize_image(resize_mode, im, width, height, upscaler_name=None, output_type
src_w = width if ratio < src_ratio else im.width * height // im.height
src_h = height if ratio >= src_ratio else im.height * width // im.width
resized = resize(im, src_w, src_h)
res = Image.new("RGB", (width, height))
res = Image.new(im.mode, (width, height))
res.paste(resized, box=(width // 2 - src_w // 2, height // 2 - src_h // 2))
if ratio < src_ratio:
fill_height = height // 2 - src_h // 2
@@ -628,7 +635,7 @@ def safe_decode_string(s: bytes):
return None
def read_info_from_image(image):
def read_info_from_image(image: Image):
items = image.info or {}
geninfo = items.pop('parameters', None)
if geninfo is None:
@@ -680,6 +687,14 @@ Negative prompt: {json_info["uc"]}
Steps: {json_info["steps"]}, Sampler: {sampler}, CFG scale: {json_info["scale"]}, Seed: {json_info["seed"]}, Size: {image.width}x{image.height}, Clip skip: 2, ENSD: 31337"""
except Exception as e:
errors.display(e, 'novelai image parser')
try:
items['width'] = image.width
items['height'] = image.height
items['mode'] = image.mode
except Exception:
pass
return geninfo, items
modules/intel/ipex/hijacks.py
+4 -3
View File
@@ -103,9 +103,6 @@ def ipex_hijacks():
CondFunc('torch.empty',
lambda orig_func, *args, device=None, **kwargs: orig_func(*args, device=return_xpu(device), **kwargs),
lambda orig_func, *args, device=None, **kwargs: check_device(device))
CondFunc('torch.load',
lambda orig_func, *args, map_location=None, **kwargs: orig_func(*args, return_xpu(map_location), **kwargs),
lambda orig_func, *args, map_location=None, **kwargs: map_location is None or check_device(map_location))
CondFunc('torch.randn',
lambda orig_func, *args, device=None, **kwargs: orig_func(*args, device=return_xpu(device), **kwargs),
lambda orig_func, *args, device=None, **kwargs: check_device(device))
@@ -121,6 +118,10 @@ def ipex_hijacks():
CondFunc('torch.linspace',
lambda orig_func, *args, device=None, **kwargs: orig_func(*args, device=return_xpu(device), **kwargs),
lambda orig_func, *args, device=None, **kwargs: check_device(device))
CondFunc('torch.load',
lambda orig_func, f, map_location=None, pickle_module=None, *, weights_only=False, mmap=None, **kwargs:
orig_func(orig_func, f, map_location=return_xpu(map_location), pickle_module=pickle_module, weights_only=weights_only, mmap=mmap, **kwargs),
lambda orig_func, f, map_location=None, pickle_module=None, *, weights_only=False, mmap=None, **kwargs: check_device(map_location))
CondFunc('torch.Generator',
lambda orig_func, device=None: torch.xpu.Generator(device),
modules/intel/openvino/__init__.py
+7 -20
View File
@@ -33,10 +33,7 @@ def BUILD_MAP_UNPACK(self, inst):
)
tmp_torch = sys.modules["torch"]
tmp_torch.BUILD_MAP_UNPACK_WITH_CALL = BUILD_MAP_UNPACK
compiled_cache = {}
max_openvino_partitions = 0
partitioned_modules = {}
DEFAULT_OPENVINO_PYTHON_CONFIG = MappingProxyType(
{
@@ -180,13 +177,6 @@ def execute_cached(compiled_model, *args):
result = [torch.from_numpy(res[out]) for out in compiled_model.outputs]
return result
def openvino_clear_caches():
global partitioned_modules
global compiled_cache
compiled_cache.clear()
partitioned_modules.clear()
def openvino_compile(gm: GraphModule, *args, model_hash_str: str = None, file_name=""):
core = Core()
@@ -204,7 +194,7 @@ def openvino_compile(gm: GraphModule, *args, model_hash_str: str = None, file_na
for input_data in args:
if isinstance(input_data, torch.SymInt):
input_types.append(torch.SymInt)
input_shapes.append([1])
input_shapes.append(1)
else:
input_types.append(input_data.type())
input_shapes.append(input_data.size())
@@ -279,21 +269,20 @@ def openvino_execute(gm: GraphModule, *args, executor_parameters=None, partition
"use_python_fusion_cache",
DEFAULT_OPENVINO_PYTHON_CONFIG["use_python_fusion_cache"],
)
global compiled_cache
model_hash_str = executor_parameters.get("model_hash_str", None)
if model_hash_str is not None:
model_hash_str = model_hash_str + str(partition_id)
if use_cache and (partition_id in compiled_cache):
compiled = compiled_cache[partition_id]
if use_cache and (partition_id in shared.compiled_model_state.compiled_cache):
compiled = shared.compiled_model_state.compiled_cache[partition_id]
else:
if (shared.compiled_model_state.cn_model != [] and file_name is not None
and os.path.isfile(file_name + ".xml") and os.path.isfile(file_name + ".bin")):
compiled = openvino_compile_cached_model(file_name, *args)
else:
compiled = openvino_compile(gm, *args, model_hash_str=model_hash_str, file_name=file_name)
compiled_cache[partition_id] = compiled
shared.compiled_model_state.compiled_cache[partition_id] = compiled
flat_args, _ = tree_flatten(args)
ov_inputs = [a.detach().cpu().numpy() for a in flat_args]
@@ -308,8 +297,6 @@ def openvino_execute(gm: GraphModule, *args, executor_parameters=None, partition
def openvino_execute_partitioned(gm: GraphModule, *args, executor_parameters=None, file_name=""):
executor_parameters = executor_parameters or DEFAULT_OPENVINO_PYTHON_CONFIG
global partitioned_modules
use_python_fusion_cache = executor_parameters.get(
"use_python_fusion_cache",
DEFAULT_OPENVINO_PYTHON_CONFIG["use_python_fusion_cache"],
@@ -323,11 +310,11 @@ def openvino_execute_partitioned(gm: GraphModule, *args, executor_parameters=Non
else:
signature = signature + "_" + str(idx) + ":" + type(input_data).__name__ + ":val(" + str(input_data) + ")"
if signature not in partitioned_modules:
partitioned_modules[signature] = partition_graph(gm, use_python_fusion_cache=use_python_fusion_cache,
if signature not in shared.compiled_model_state.partitioned_modules:
shared.compiled_model_state.partitioned_modules[signature] = partition_graph(gm, use_python_fusion_cache=use_python_fusion_cache,
model_hash_str=model_hash_str, file_name=file_name)
return partitioned_modules[signature](*args)
return shared.compiled_model_state.partitioned_modules[signature](*args)
def partition_graph(gm: GraphModule, use_python_fusion_cache: bool, model_hash_str: str = None, file_name=""):
global max_openvino_partitions
modules/interrogate.py
+1 -35
View File
@@ -12,27 +12,23 @@ from modules import devices, paths, shared, lowvram, modelloader, errors
blip_image_eval_size = 384
clip_model_name = 'ViT-L/14'
Category = namedtuple("Category", ["name", "topn", "items"])
re_topn = re.compile(r"\.top(\d+)\.")
def category_types():
return [f.stem for f in Path(shared.interrogator.content_dir).glob('*.txt')]
def download_default_clip_interrogate_categories(content_dir):
shared.log.info("Downloading CLIP categories...")
tmpdir = f"{content_dir}_tmp"
cat_types = ["artists", "flavors", "mediums", "movements"]
try:
os.makedirs(tmpdir, exist_ok=True)
for category_type in cat_types:
torch.hub.download_url_to_file(f"https://raw.githubusercontent.com/pharmapsychotic/clip-interrogator/main/clip_interrogator/data/{category_type}.txt", os.path.join(tmpdir, f"{category_type}.txt"))
os.rename(tmpdir, content_dir)
except Exception as e:
errors.display(e, "downloading default CLIP interrogate categories")
finally:
@@ -56,10 +52,8 @@ class InterrogateModels:
def categories(self):
if not os.path.exists(self.content_dir):
download_default_clip_interrogate_categories(self.content_dir)
if self.loaded_categories is not None and self.skip_categories == shared.opts.interrogate_clip_skip_categories:
return self.loaded_categories
self.loaded_categories = []
if os.path.exists(self.content_dir):
@@ -74,14 +68,12 @@ class InterrogateModels:
with open(filename, "r", encoding="utf8") as file:
lines = [x.strip() for x in file.readlines()]
self.loaded_categories.append(Category(name=filename.stem, topn=topn, items=lines))
return self.loaded_categories
def create_fake_fairscale(self):
class FakeFairscale:
def checkpoint_wrapper(self):
pass
sys.modules["fairscale.nn.checkpoint.checkpoint_activations"] = FakeFairscale
def load_blip_model(self):
@@ -95,7 +87,6 @@ class InterrogateModels:
ext_filter=[".pth"],
download_name='model_base_caption_capfilt_large.pth',
)
blip_model = models.blip.blip_decoder(pretrained=files[0], image_size=blip_image_eval_size, vit='base', med_config=os.path.join(paths.paths["BLIP"], "configs", "med_config.json")) # pylint: disable=c-extension-no-member
blip_model.eval()
@@ -103,15 +94,12 @@ class InterrogateModels:
def load_clip_model(self):
import clip
if self.running_on_cpu:
model, preprocess = clip.load(clip_model_name, device="cpu", download_root=shared.opts.clip_models_path)
else:
model, preprocess = clip.load(clip_model_name, download_root=shared.opts.clip_models_path)
model.eval()
model = model.to(devices.device_interrogate)
return model, preprocess
def load(self):
@@ -119,16 +107,12 @@ class InterrogateModels:
self.blip_model = self.load_blip_model()
if not shared.opts.no_half and not self.running_on_cpu:
self.blip_model = self.blip_model.half()
self.blip_model = self.blip_model.to(devices.device_interrogate)
if self.clip_model is None:
self.clip_model, self.clip_preprocess = self.load_clip_model()
if not shared.opts.no_half and not self.running_on_cpu:
self.clip_model = self.clip_model.half()
self.clip_model = self.clip_model.to(devices.device_interrogate)
self.dtype = next(self.clip_model.parameters()).dtype
def send_clip_to_ram(self):
@@ -144,27 +128,21 @@ class InterrogateModels:
def unload(self):
self.send_clip_to_ram()
self.send_blip_to_ram()
devices.torch_gc()
def rank(self, image_features, text_array, top_count=1):
import clip
devices.torch_gc()
if shared.opts.interrogate_clip_dict_limit != 0:
text_array = text_array[0:int(shared.opts.interrogate_clip_dict_limit)]
top_count = min(top_count, len(text_array))
text_tokens = clip.tokenize(list(text_array), truncate=True).to(devices.device_interrogate)
text_features = self.clip_model.encode_text(text_tokens).type(self.dtype)
text_features /= text_features.norm(dim=-1, keepdim=True)
similarity = torch.zeros((1, len(text_array))).to(devices.device_interrogate)
for i in range(image_features.shape[0]):
similarity += (100.0 * image_features[i].unsqueeze(0) @ text_features.T).softmax(dim=-1)
similarity /= image_features.shape[0]
top_probs, top_labels = similarity.cpu().topk(top_count, dim=-1)
return [(text_array[top_labels[0][i].numpy()], (top_probs[0][i].numpy()*100)) for i in range(top_count)]
@@ -174,10 +152,8 @@ class InterrogateModels:
transforms.ToTensor(),
transforms.Normalize((0.48145466, 0.4578275, 0.40821073), (0.26862954, 0.26130258, 0.27577711))
])(pil_image).unsqueeze(0).type(self.dtype).to(devices.device_interrogate)
with devices.inference_context():
caption = self.blip_model.generate(gpu_image, sample=False, num_beams=shared.opts.interrogate_clip_num_beams, min_length=shared.opts.interrogate_clip_min_length, max_length=shared.opts.interrogate_clip_max_length)
return caption[0]
def interrogate(self, pil_image):
@@ -187,22 +163,15 @@ class InterrogateModels:
if shared.cmd_opts.lowvram or shared.cmd_opts.medvram:
lowvram.send_everything_to_cpu()
devices.torch_gc()
self.load()
caption = self.generate_caption(pil_image)
self.send_blip_to_ram()
devices.torch_gc()
res = caption
clip_image = self.clip_preprocess(pil_image).unsqueeze(0).type(self.dtype).to(devices.device_interrogate)
with devices.inference_context(), devices.autocast():
image_features = self.clip_model.encode_image(clip_image).type(self.dtype)
image_features /= image_features.norm(dim=-1, keepdim=True)
for _name, topn, items in self.categories():
matches = self.rank(image_features, items, top_count=topn)
for match, score in matches:
@@ -210,12 +179,9 @@ class InterrogateModels:
res += f", ({match}:{score/100:.3f})"
else:
res += f", {match}"
except Exception as e:
errors.display(e, 'interrogate')
res += "<error>"
self.unload()
shared.state.end()
return res
modules/merging/merge.py
+402
View File
@@ -0,0 +1,402 @@
import os
from concurrent.futures import ThreadPoolExecutor
from contextlib import contextmanager
from typing import Dict, Optional, Tuple
import safetensors.torch
import torch
from tensordict import TensorDict
import modules.memstats
import modules.devices as devices
from modules.shared import log, console
from modules.sd_models import read_state_dict
from modules.merging import merge_methods
from modules.merging.merge_utils import WeightClass
from modules.merging.merge_rebasin import (
apply_permutation,
sdunet_permutation_spec,
update_model_a,
weight_matching,
)
##########################################################
# Files in modules.merging are heavily modified
# versions of sd-meh by @s1dxl used with his blessing
# orginal code can be found @ https://github.com/s1dlx/meh
##########################################################
MAX_TOKENS = 77
KEY_POSITION_IDS = ".".join(
[
"cond_stage_model",
"transformer",
"text_model",
"embeddings",
"position_ids",
]
)
def fix_clip(model: Dict) -> Dict:
if KEY_POSITION_IDS in model.keys():
model[KEY_POSITION_IDS] = torch.tensor(
[list(range(MAX_TOKENS))],
dtype=torch.int64,
device=model[KEY_POSITION_IDS].device,
)
return model
def prune_sd_model(model: Dict) -> Dict:
keys = list(model.keys())
for k in keys:
if (
not k.startswith("model.diffusion_model.")
# and not k.startswith("first_stage_model.")
and not k.startswith("cond_stage_model.")
):
del model[k]
return model
def restore_sd_model(original_model: Dict, merged_model: Dict) -> Dict:
for k in original_model:
if k not in merged_model:
merged_model[k] = original_model[k]
return merged_model
def log_vram(txt=""):
log.debug(f"Merge {txt}: {modules.memstats.memory_stats()}")
def load_thetas(
models: Dict[str, os.PathLike | str],
prune: bool,
device: torch.device,
precision: str,
) -> Dict:
if prune:
thetas = {k: prune_sd_model(TensorDict.from_dict(read_state_dict(m, "cpu"))) for k, m in models.items()}
else:
thetas = {k: TensorDict.from_dict(read_state_dict(m, device)) for k, m in models.items()}
for model_key, model in thetas.items():
for key, block in model.items():
if precision == "fp16":
thetas[model_key].update({key: block.to(device).half()})
else:
thetas[model_key].update({key: block.to(device)})
log_vram("models loaded")
return thetas
def merge_models(
models: Dict[str, os.PathLike | str],
merge_mode: str,
precision: str = "fp16",
weights_clip: bool = False,
re_basin: bool = False,
device: torch.device = None,
work_device: torch.device = None,
prune: bool = False,
threads: int = 4,
**kwargs,
) -> Dict:
thetas = load_thetas(models, prune, device, precision)
# log.info(f'Merge start: models={models.values()} precision={precision} clip={weights_clip} rebasin={re_basin} prune={prune} threads={threads}')
weight_matcher = WeightClass(thetas["model_a"], **kwargs)
if re_basin:
merged = rebasin_merge(
thetas,
weight_matcher,
merge_mode,
precision=precision,
weights_clip=weights_clip,
iterations=kwargs.get("re_basin_iterations", 1),
device=device,
work_device=work_device,
threads=threads,
)
else:
merged = simple_merge(
thetas,
weight_matcher,
merge_mode,
precision=precision,
weights_clip=weights_clip,
device=device,
work_device=work_device,
threads=threads,
)
return un_prune_model(merged, thetas, models, device, prune, precision)
def un_prune_model(
merged: Dict,
thetas: Dict,
models: Dict,
device: torch.device,
prune: bool,
precision: str,
) -> Dict:
if prune:
log.info("Merge restoring pruned keys")
del thetas
devices.torch_gc(force=False)
original_a = TensorDict.from_dict(read_state_dict(models["model_a"], device))
unpruned = 0
for key in original_a.keys():
if KEY_POSITION_IDS in key:
continue
if "model" in key and key not in merged.keys():
merged.update({key: original_a[key]})
unpruned += 1
if precision == "fp16":
merged.update({key: merged[key].half()})
if unpruned > 248: # VAE has 248 keys, and we are purposely restoring it here
log.debug(f"Merge restored from primary model: keys={unpruned - 248}")
unpruned = 0
del original_a
original_b = TensorDict.from_dict(read_state_dict(models["model_b"], device))
for key in original_b.keys():
if KEY_POSITION_IDS in key:
continue
if "model" in key and key not in merged.keys():
merged.update({key: original_b[key]})
unpruned += 1
if precision == "fp16":
merged.update({key: merged[key].half()})
if unpruned != 0:
log.debug(f"Merge restored from secondary model: keys={unpruned}")
del original_b
devices.torch_gc(force=False)
return fix_clip(merged)
def simple_merge(
thetas: Dict[str, Dict],
weight_matcher: WeightClass,
merge_mode: str,
precision: str = "fp16",
weights_clip: bool = False,
device: torch.device = None,
work_device: torch.device = None,
threads: int = 4,
) -> Dict:
futures = []
# with tqdm(thetas["model_a"].keys(), desc="Merge") as progress:
import rich.progress as p
with p.Progress(p.TextColumn('[cyan]{task.description}'), p.BarColumn(), p.TaskProgressColumn(), p.TimeRemainingColumn(), p.TimeElapsedColumn(), p.TextColumn('[cyan]keys={task.fields[keys]}'), console=console) as progress:
task = progress.add_task(description="Merging", total=len(thetas["model_a"].keys()), keys=len(thetas["model_a"].keys()))
with ThreadPoolExecutor(max_workers=threads) as executor:
for key in thetas["model_a"].keys():
future = executor.submit(
simple_merge_key,
progress,
task,
key,
thetas,
weight_matcher,
merge_mode,
precision,
weights_clip,
device,
work_device,
)
futures.append(future)
for res in futures:
res.result()
if len(thetas["model_b"]) > 0:
log.debug(f'Merge update thetas: keys={len(thetas["model_b"])}')
for key in thetas["model_b"].keys():
if KEY_POSITION_IDS in key:
continue
if "model" in key and key not in thetas["model_a"].keys():
thetas["model_a"].update({key: thetas["model_b"][key]})
if precision == "fp16":
thetas["model_a"].update({key: thetas["model_a"][key].half()})
return fix_clip(thetas["model_a"])
def rebasin_merge(
thetas: Dict[str, os.PathLike | str],
weight_matcher: WeightClass,
merge_mode: str,
precision: str = "fp16",
weights_clip: bool = False,
iterations: int = 1,
device: torch.device = None,
work_device: torch.device = None,
threads: int = 1,
):
# not sure how this does when 3 models are involved...
model_a = thetas["model_a"].clone()
perm_spec = sdunet_permutation_spec()
for it in range(iterations):
log_vram(f"rebasin: iteration={it}")
weight_matcher.set_it(it)
# normal block merge we already know and love
thetas["model_a"] = simple_merge(
thetas,
weight_matcher,
merge_mode,
precision,
False,
device,
work_device,
threads,
)
# find permutations
perm_1, y = weight_matching(
perm_spec,
model_a,
thetas["model_a"],
max_iter=it,
init_perm=None,
usefp16=precision == "fp16",
device=device,
)
thetas["model_a"] = apply_permutation(perm_spec, perm_1, thetas["model_a"])
perm_2, z = weight_matching(
perm_spec,
thetas["model_b"],
thetas["model_a"],
max_iter=it,
init_perm=None,
usefp16=precision == "fp16",
device=device,
)
new_alpha = torch.nn.functional.normalize(
torch.sigmoid(torch.Tensor([y, z])), p=1, dim=0
).tolist()[0]
thetas["model_a"] = update_model_a(
perm_spec, perm_2, thetas["model_a"], new_alpha
)
if weights_clip:
clip_thetas = thetas.copy()
clip_thetas["model_a"] = model_a
thetas["model_a"] = clip_weights(thetas, thetas["model_a"])
return thetas["model_a"]
def simple_merge_key(progress, task, key, thetas, *args, **kwargs):
with merge_key_context(key, thetas, *args, **kwargs) as result:
if result is not None:
thetas["model_a"].update({key: result.detach().clone()})
progress.update(task, advance=1)
def merge_key( # pylint: disable=inconsistent-return-statements
key: str,
thetas: Dict,
weight_matcher: WeightClass,
merge_mode: str,
precision: str = "fp16",
weights_clip: bool = False,
device: torch.device = None,
work_device: torch.device = None,
) -> Optional[Tuple[str, Dict]]:
if work_device is None:
work_device = device
if KEY_POSITION_IDS in key:
return
for theta in thetas.values():
if key not in theta.keys():
return
current_bases = weight_matcher(key)
try:
merge_method = getattr(merge_methods, merge_mode)
except AttributeError as e:
raise ValueError(f"{merge_mode} not implemented, aborting merge!") from e
merge_args = get_merge_method_args(current_bases, thetas, key, work_device)
# dealing with pix2pix and inpainting models
if (a_size := merge_args["a"].size()) != (b_size := merge_args["b"].size()):
if a_size[1] > b_size[1]:
merged_key = merge_args["a"]
else:
merged_key = merge_args["b"]
else:
merged_key = merge_method(**merge_args).to(device)
if weights_clip:
merged_key = clip_weights_key(thetas, merged_key, key)
if precision == "fp16":
merged_key = merged_key.half()
return merged_key
def clip_weights(thetas, merged):
for k in thetas["model_a"].keys():
if k in thetas["model_b"].keys():
merged.update({k: clip_weights_key(thetas, merged[k], k)})
return merged
def clip_weights_key(thetas, merged_weights, key):
t0 = thetas["model_a"][key]
t1 = thetas["model_b"][key]
maximums = torch.maximum(t0, t1)
minimums = torch.minimum(t0, t1)
return torch.minimum(torch.maximum(merged_weights, minimums), maximums)
@contextmanager
def merge_key_context(*args, **kwargs):
result = merge_key(*args, **kwargs)
try:
yield result
finally:
if result is not None:
del result
def get_merge_method_args(
current_bases: Dict,
thetas: Dict,
key: str,
work_device: torch.device,
) -> Dict:
merge_method_args = {
"a": thetas["model_a"][key].to(work_device),
"b": thetas["model_b"][key].to(work_device),
**current_bases,
}
if "model_c" in thetas:
merge_method_args["c"] = thetas["model_c"][key].to(work_device)
return merge_method_args
def save_model(model, output_file, file_format) -> None:
log.info(f"Merge saving: model='{output_file}'")
if file_format == "safetensors":
safetensors.torch.save_file(
model if type(model) == dict else model.to_dict(),
f"{output_file}.safetensors",
metadata={"format": "pt"},
)
else:
torch.save({"state_dict": model}, f"{output_file}.ckpt")
modules/merging/merge_methods.py
+247
View File
@@ -0,0 +1,247 @@
import math
from typing import Tuple
import torch
from torch import Tensor
__all__ = [
"weighted_sum",
"weighted_subtraction",
"tensor_sum",
"add_difference",
"sum_twice",
"triple_sum",
"euclidean_add_difference",
"multiply_difference",
"top_k_tensor_sum",
"similarity_add_difference",
"distribution_crossover",
"ties_add_difference",
]
EPSILON = 1e-10 # Define a small constant EPSILON to prevent division by zero
def weighted_sum(a: Tensor, b: Tensor, alpha: float, **kwargs) -> Tensor: # pylint: disable=unused-argument
"""
Basic Merge:
alpha 0 returns Primary Model
alpha 1 returns Secondary Model
"""
return (1 - alpha) * a + alpha * b
def weighted_subtraction(a: Tensor, b: Tensor, alpha: float, beta: float, **kwargs) -> Tensor: # pylint: disable=unused-argument
"""
The inverse of a Weighted Sum Merge
Returns Primary Model when alpha*beta = 0
High values of alpha*beta are likely to break the merged model
"""
# Adjust beta if both alpha and beta are 1.0 to avoid division by zero
if alpha == 1.0 and beta == 1.0:
beta -= EPSILON
return (a - alpha * beta * b) / (1 - alpha * beta)
def tensor_sum(a: Tensor, b: Tensor, alpha: float, beta: float, **kwargs) -> Tensor: # pylint: disable=unused-argument
"""
Takes a slice of Secondary Model and pastes it into Primary Model
Alpha sets the width of the slice
Beta sets the start point of the slice
ie Alpha = 0.5 Beta = 0.25 is (ABBA) Alpha = 0.25 Beta = 0 is (BAAA)
"""
if alpha + beta <= 1:
tt = a.clone()
talphas = int(a.shape[0] * beta)
talphae = int(a.shape[0] * (alpha + beta))
tt[talphas:talphae] = b[talphas:talphae].clone()
else:
talphas = int(a.shape[0] * (alpha + beta - 1))
talphae = int(a.shape[0] * beta)
tt = b.clone()
tt[talphas:talphae] = a[talphas:talphae].clone()
return tt
def add_difference(a: Tensor, b: Tensor, c: Tensor, alpha: float, **kwargs) -> Tensor: # pylint: disable=unused-argument
"""
Classic Add Difference Merge
"""
return a + alpha * (b - c)
def sum_twice(a: Tensor, b: Tensor, c: Tensor, alpha: float, beta: float, **kwargs) -> Tensor: # pylint: disable=unused-argument
"""
Stacked Basic Merge:
Equivalent to Merging Primary and Secondary @ alpha
Then merging the result with Tertiary @ beta
"""
return (1 - beta) * ((1 - alpha) * a + alpha * b) + beta * c
def triple_sum(a: Tensor, b: Tensor, c: Tensor, alpha: float, beta: float, **kwargs) -> Tensor: # pylint: disable=unused-argument
"""
Weights Secondary and Tertiary at alpha and beta respectively
Fills in the rest with Primary
Expect odd results if alpha + beta > 1 as Primary will be merged with a negative ratio
"""
return (1 - alpha - beta) * a + alpha * b + beta * c
def euclidean_add_difference(a: Tensor, b: Tensor, c: Tensor, alpha: float, **kwargs) -> Tensor: # pylint: disable=unused-argument
"""
Subtract Primary and Secondary from Tertiary
Compare the remainders via Euclidean distance
Add to Tertiary
Note: Slow
"""
a_diff = a.float() - c.float()
b_diff = b.float() - c.float()
a_diff = torch.nan_to_num(a_diff / torch.linalg.norm(a_diff))
b_diff = torch.nan_to_num(b_diff / torch.linalg.norm(b_diff))
distance = (1 - alpha) * a_diff**2 + alpha * b_diff**2
distance = torch.sqrt(distance)
sum_diff = weighted_sum(a.float(), b.float(), alpha) - c.float()
distance = torch.copysign(distance, sum_diff)
target_norm = torch.linalg.norm(sum_diff)
return c + distance / torch.linalg.norm(distance) * target_norm
def multiply_difference(a: Tensor, b: Tensor, c: Tensor, alpha: float, beta: float, **kwargs) -> Tensor: # pylint: disable=unused-argument
"""
Similar to Add Difference but with geometric mean instead of arithmatic mean
"""
diff_a = torch.pow(torch.abs(a.float() - c), (1 - alpha))
diff_b = torch.pow(torch.abs(b.float() - c), alpha)
difference = torch.copysign(diff_a * diff_b, weighted_sum(a, b, beta) - c)
return c + difference.to(c.dtype)
def top_k_tensor_sum(a: Tensor, b: Tensor, alpha: float, beta: float, **kwargs) -> Tensor: # pylint: disable=unused-argument
"""
Redistributes the largest weights of Secondary Model into Primary Model
"""
a_flat = torch.flatten(a)
a_dist = torch.msort(a_flat)
b_indices = torch.argsort(torch.flatten(b), stable=True)
redist_indices = torch.argsort(b_indices)
start_i, end_i, region_is_inverted = ratio_to_region(alpha, beta, torch.numel(a))
start_top_k = kth_abs_value(a_dist, start_i)
end_top_k = kth_abs_value(a_dist, end_i)
indices_mask = (start_top_k < torch.abs(a_dist)) & (torch.abs(a_dist) <= end_top_k)
if region_is_inverted:
indices_mask = ~indices_mask
indices_mask = torch.gather(indices_mask.float(), 0, redist_indices)
a_redist = torch.gather(a_dist, 0, redist_indices)
a_redist = (1 - indices_mask) * a_flat + indices_mask * a_redist
return a_redist.reshape_as(a)
def kth_abs_value(a: Tensor, k: int) -> Tensor:
if k <= 0:
return torch.tensor(-1, device=a.device)
else:
return torch.kthvalue(torch.abs(a.float()), k)[0]
def ratio_to_region(width: float, offset: float, n: int) -> Tuple[int, int, bool]:
if width < 0:
offset += width
width = -width
width = min(width, 1)
if offset < 0:
offset = 1 + offset - int(offset)
offset = math.fmod(offset, 1.0)
if width + offset <= 1:
inverted = False
start = offset * n
end = (width + offset) * n
else:
inverted = True
start = (width + offset - 1) * n
end = offset * n
return round(start), round(end), inverted
def similarity_add_difference(a: Tensor, b: Tensor, c: Tensor, alpha: float, beta: float, **kwargs) -> Tensor: # pylint: disable=unused-argument
"""
Weighted Sum where A and B are similar and Add Difference where A and B are dissimilar
"""
threshold = torch.maximum(torch.abs(a), torch.abs(b))
similarity = ((a * b / threshold**2) + 1) / 2
similarity = torch.nan_to_num(similarity * beta, nan=beta)
ab_diff = a + alpha * (b - c)
ab_sum = (1 - alpha / 2) * a + (alpha / 2) * b
return (1 - similarity) * ab_diff + similarity * ab_sum
def distribution_crossover(a: Tensor, b: Tensor, c: Tensor, alpha: float, beta: float, **kwargs): # pylint: disable=unused-argument
"""
From the creator:
It's Primary high-passed + Secondary low-passed. Takes the fourrier transform of the weights of
Primary and Secondary when ordered with respect to Tertiary. Split the frequency domain
using a linear function. Alpha is the split frequency and Beta is the inclination of the line.
add everything under the line as the contribution of Primary and everything over the line as the contribution of Secondary
"""
if a.shape == ():
return alpha * a + (1 - alpha) * b
c_indices = torch.argsort(torch.flatten(c))
a_dist = torch.gather(torch.flatten(a), 0, c_indices)
b_dist = torch.gather(torch.flatten(b), 0, c_indices)
a_dft = torch.fft.rfft(a_dist.float())
b_dft = torch.fft.rfft(b_dist.float())
dft_filter = torch.arange(0, torch.numel(a_dft), device=a_dft.device).float()
dft_filter /= torch.numel(a_dft)
if beta > EPSILON:
dft_filter = (dft_filter - alpha) / beta + 1 / 2
dft_filter = torch.clamp(dft_filter, 0.0, 1.0)
else:
dft_filter = (dft_filter >= alpha).float()
x_dft = (1 - dft_filter) * a_dft + dft_filter * b_dft
x_dist = torch.fft.irfft(x_dft, a_dist.shape[0])
x_values = torch.gather(x_dist, 0, torch.argsort(c_indices))
return x_values.reshape_as(a)
def ties_add_difference(a: Tensor, b: Tensor, c: Tensor, alpha: float, beta: float, **kwargs) -> Tensor: # pylint: disable=unused-argument
"""
An implementation of arXiv:2306.01708
"""
deltas = []
signs = []
for m in [a, b]:
deltas.append(filter_top_k(m - c, beta))
signs.append(torch.sign(deltas[-1]))
signs = torch.stack(signs, dim=0)
final_sign = torch.sign(torch.sum(signs, dim=0))
delta_filters = (signs == final_sign).float()
res = torch.zeros_like(c, device=c.device)
for delta_filter, delta in zip(delta_filters, deltas):
res += delta_filter * delta
param_count = torch.sum(delta_filters, dim=0)
return c + alpha * torch.nan_to_num(res / param_count)
def filter_top_k(a: Tensor, k: float):
k = max(int((1 - k) * torch.numel(a)), 1)
k_value, _ = torch.kthvalue(torch.abs(a.flatten()).float(), k)
top_k_filter = (torch.abs(a) >= k_value).float()
return a * top_k_filter
modules/merging/merge_presets.py
+62
View File
@@ -0,0 +1,62 @@
BLOCK_WEIGHTS_PRESETS = {
"GRAD_V": [0, 1, 0.9166666667, 0.8333333333, 0.75, 0.6666666667, 0.5833333333, 0.5, 0.4166666667, 0.3333333333, 0.25, 0.1666666667, 0.0833333333, 0, 0.0833333333, 0.1666666667, 0.25, 0.3333333333, 0.4166666667, 0.5, 0.5833333333, 0.6666666667, 0.75, 0.8333333333, 0.9166666667, 1.0],
"GRAD_A": [0, 0, 0.0833333333, 0.1666666667, 0.25, 0.3333333333, 0.4166666667, 0.5, 0.5833333333, 0.6666666667, 0.75, 0.8333333333, 0.9166666667, 1.0, 0.9166666667, 0.8333333333, 0.75, 0.6666666667, 0.5833333333, 0.5, 0.4166666667, 0.3333333333, 0.25, 0.1666666667, 0.0833333333, 0],
"FLAT_25": [0, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25],
"FLAT_75": [0, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75],
"WRAP08": [0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1],
"WRAP12": [0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1],
"WRAP14": [0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1],
"WRAP16": [0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1],
"MID12_50": [0, 0, 0, 0, 0, 0, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0, 0, 0, 0, 0, 0],
"OUT07": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1],
"OUT12": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
"OUT12_5": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
"RING08_SOFT": [0, 0, 0, 0, 0, 0, 0.5, 1, 1, 1, 0.5, 0, 0, 0, 0, 0, 0.5, 1, 1, 1, 0.5, 0, 0, 0, 0, 0],
"RING08_5": [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0],
"RING10_5": [0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0],
"RING10_3": [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0],
"SMOOTHSTEP": [0, 0, 0.00506365740740741, 0.0196759259259259, 0.04296875, 0.0740740740740741, 0.112123842592593, 0.15625, 0.205584490740741, 0.259259259259259, 0.31640625, 0.376157407407407, 0.437644675925926, 0.5, 0.562355324074074, 0.623842592592592, 0.68359375, 0.740740740740741, 0.794415509259259, 0.84375, 0.887876157407408, 0.925925925925926, 0.95703125, 0.980324074074074, 0.994936342592593, 1],
"REVERSE_SMOOTHSTEP": [0, 1, 0.994936342592593, 0.980324074074074, 0.95703125, 0.925925925925926, 0.887876157407407, 0.84375, 0.794415509259259, 0.740740740740741, 0.68359375, 0.623842592592593, 0.562355324074074, 0.5, 0.437644675925926, 0.376157407407408, 0.31640625, 0.259259259259259, 0.205584490740741, 0.15625, 0.112123842592592, 0.0740740740740742, 0.0429687499999996, 0.0196759259259258, 0.00506365740740744, 0],
"2SMOOTHSTEP": [0, 0, 0.0101273148148148, 0.0393518518518519, 0.0859375, 0.148148148148148, 0.224247685185185, 0.3125, 0.411168981481482, 0.518518518518519, 0.6328125, 0.752314814814815, 0.875289351851852, 1.0, 0.875289351851852, 0.752314814814815, 0.6328125, 0.518518518518519, 0.411168981481481, 0.3125, 0.224247685185184, 0.148148148148148, 0.0859375, 0.0393518518518512, 0.0101273148148153, 0],
"2R_SMOOTHSTEP": [0, 1, 0.989872685185185, 0.960648148148148, 0.9140625, 0.851851851851852, 0.775752314814815, 0.6875, 0.588831018518519, 0.481481481481481, 0.3671875, 0.247685185185185, 0.124710648148148, 0.0, 0.124710648148148, 0.247685185185185, 0.3671875, 0.481481481481481, 0.588831018518519, 0.6875, 0.775752314814816, 0.851851851851852, 0.9140625, 0.960648148148149, 0.989872685185185, 1],
"3SMOOTHSTEP": [0, 0, 0.0151909722222222, 0.0590277777777778, 0.12890625, 0.222222222222222, 0.336371527777778, 0.46875, 0.616753472222222, 0.777777777777778, 0.94921875, 0.871527777777778, 0.687065972222222, 0.5, 0.312934027777778, 0.128472222222222, 0.0507812500000004, 0.222222222222222, 0.383246527777778, 0.53125, 0.663628472222223, 0.777777777777778, 0.87109375, 0.940972222222222, 0.984809027777777, 1],
"3R_SMOOTHSTEP": [0, 1, 0.984809027777778, 0.940972222222222, 0.87109375, 0.777777777777778, 0.663628472222222, 0.53125, 0.383246527777778, 0.222222222222222, 0.05078125, 0.128472222222222, 0.312934027777778, 0.5, 0.687065972222222, 0.871527777777778, 0.94921875, 0.777777777777778, 0.616753472222222, 0.46875, 0.336371527777777, 0.222222222222222, 0.12890625, 0.0590277777777777, 0.0151909722222232, 0],
"4SMOOTHSTEP": [0, 0, 0.0202546296296296, 0.0787037037037037, 0.171875, 0.296296296296296, 0.44849537037037, 0.625, 0.822337962962963, 0.962962962962963, 0.734375, 0.49537037037037, 0.249421296296296, 0.0, 0.249421296296296, 0.495370370370371, 0.734375000000001, 0.962962962962963, 0.822337962962962, 0.625, 0.448495370370369, 0.296296296296297, 0.171875, 0.0787037037037024, 0.0202546296296307, 0],
"4R_SMOOTHSTEP": [0, 1, 0.97974537037037, 0.921296296296296, 0.828125, 0.703703703703704, 0.55150462962963, 0.375, 0.177662037037037, 0.0370370370370372, 0.265625, 0.50462962962963, 0.750578703703704, 1.0, 0.750578703703704, 0.504629629629629, 0.265624999999999, 0.0370370370370372, 0.177662037037038, 0.375, 0.551504629629631, 0.703703703703703, 0.828125, 0.921296296296298, 0.979745370370369, 1],
"HALF_SMOOTHSTEP": [0, 0, 0.0196759259259259, 0.0740740740740741, 0.15625, 0.259259259259259, 0.376157407407407, 0.5, 0.623842592592593, 0.740740740740741, 0.84375, 0.925925925925926, 0.980324074074074, 1.0, 0.980324074074074, 0.925925925925926, 0.84375, 0.740740740740741, 0.623842592592593, 0.5, 0.376157407407407, 0.259259259259259, 0.15625, 0.0740740740740741, 0.0196759259259259, 0],
"HALF_R_SMOOTHSTEP": [0, 1, 0.980324074074074, 0.925925925925926, 0.84375, 0.740740740740741, 0.623842592592593, 0.5, 0.376157407407407, 0.259259259259259, 0.15625, 0.0740740740740742, 0.0196759259259256, 0.0, 0.0196759259259256, 0.0740740740740742, 0.15625, 0.259259259259259, 0.376157407407407, 0.5, 0.623842592592593, 0.740740740740741, 0.84375, 0.925925925925926, 0.980324074074074, 1],
"ONE_THIRD_SMOOTHSTEP": [0, 0, 0.04296875, 0.15625, 0.31640625, 0.5, 0.68359375, 0.84375, 0.95703125, 1.0, 0.95703125, 0.84375, 0.68359375, 0.5, 0.31640625, 0.15625, 0.04296875, 0.0, 0.04296875, 0.15625, 0.31640625, 0.5, 0.68359375, 0.84375, 0.95703125, 1],
"ONE_THIRD_R_SMOOTHSTEP": [0, 1, 0.95703125, 0.84375, 0.68359375, 0.5, 0.31640625, 0.15625, 0.04296875, 0.0, 0.04296875, 0.15625, 0.31640625, 0.5, 0.68359375, 0.84375, 0.95703125, 1.0, 0.95703125, 0.84375, 0.68359375, 0.5, 0.31640625, 0.15625, 0.04296875, 0],
"ONE_FOURTH_SMOOTHSTEP": [0, 0, 0.0740740740740741, 0.259259259259259, 0.5, 0.740740740740741, 0.925925925925926, 1.0, 0.925925925925926, 0.740740740740741, 0.5, 0.259259259259259, 0.0740740740740741, 0.0, 0.0740740740740741, 0.259259259259259, 0.5, 0.740740740740741, 0.925925925925926, 1.0, 0.925925925925926, 0.740740740740741, 0.5, 0.259259259259259, 0.0740740740740741, 0],
"ONE_FOURTH_R_SMOOTHSTEP": [0, 1, 0.925925925925926, 0.740740740740741, 0.5, 0.259259259259259, 0.0740740740740742, 0.0, 0.0740740740740742, 0.259259259259259, 0.5, 0.740740740740741, 0.925925925925926, 1.0, 0.925925925925926, 0.740740740740741, 0.5, 0.259259259259259, 0.0740740740740742, 0.0, 0.0740740740740742, 0.259259259259259, 0.5, 0.740740740740741, 0.925925925925926, 1],
"COSINE": [0, 1, 0.995722430686905, 0.982962913144534, 0.961939766255643, 0.933012701892219, 0.896676670145617, 0.853553390593274, 0.80438071450436, 0.75, 0.691341716182545, 0.62940952255126, 0.565263096110026, 0.5, 0.434736903889974, 0.37059047744874, 0.308658283817455, 0.25, 0.195619285495639, 0.146446609406726, 0.103323329854382, 0.0669872981077805, 0.0380602337443566, 0.0170370868554658, 0.00427756931309475, 0],
"REVERSE_COSINE": [0, 0, 0.00427756931309475, 0.0170370868554659, 0.0380602337443566, 0.0669872981077808, 0.103323329854383, 0.146446609406726, 0.19561928549564, 0.25, 0.308658283817455, 0.37059047744874, 0.434736903889974, 0.5, 0.565263096110026, 0.62940952255126, 0.691341716182545, 0.75, 0.804380714504361, 0.853553390593274, 0.896676670145618, 0.933012701892219, 0.961939766255643, 0.982962913144534, 0.995722430686905, 1],
"CUBIC_HERMITE": [0, 0, 0.157576195987654, 0.28491512345679, 0.384765625, 0.459876543209877, 0.512996720679012, 0.546875, 0.564260223765432, 0.567901234567901, 0.560546875, 0.544945987654321, 0.523847415123457, 0.5, 0.476152584876543, 0.455054012345679, 0.439453125, 0.432098765432099, 0.435739776234568, 0.453125, 0.487003279320987, 0.540123456790124, 0.615234375, 0.71508487654321, 0.842423804012347, 1],
"REVERSE_CUBIC_HERMITE": [0, 1, 0.842423804012346, 0.71508487654321, 0.615234375, 0.540123456790123, 0.487003279320988, 0.453125, 0.435739776234568, 0.432098765432099, 0.439453125, 0.455054012345679, 0.476152584876543, 0.5, 0.523847415123457, 0.544945987654321, 0.560546875, 0.567901234567901, 0.564260223765432, 0.546875, 0.512996720679013, 0.459876543209876, 0.384765625, 0.28491512345679, 0.157576195987653, 0],
"ALL_A": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
"ALL_B": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
}
SDXL_BLOCK_WEIGHTS_PRESETS = {
"SDXL_GRAD_V": [0, 1.0, 0.888889, 0.777778, 0.666667, 0.555556, 0.444444, 0.333333, 0.222222, 0.111111, 0.0, 0.111111, 0.222222, 0.333333, 0.444444, 0.555556, 0.666667, 0.777778, 0.888889, 1.0],
"SDXL_GRAD_A": [0, 0.0, 0.111111, 0.222222, 0.333333, 0.444444, 0.555556, 0.666667, 0.777778, 0.888889, 1.0, 0.888889, 0.777778, 0.666667, 0.555556, 0.444444, 0.333333, 0.222222, 0.111111, 0.0],
"SDXL_FLAT_25": [0, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25],
"SDXL_FLAT_75": [0, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75],
"SDXL_WRAP08": [0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1],
"SDXL_WRAP12": [0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1],
"SDXL_WRAP14": [0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1],
"SDXL_OUT07": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1],
"SDXL_SMOOTHSTEP": [0, 0, 0.008916, 0.034294, 0.074074, 0.126200, 0.188615, 0.259259, 0.336077, 0.417010, 0.500000, 0.582990, 0.663923, 0.740741, 0.811385, 0.873800, 0.925926, 0.965706, 0.991084, 1],
"SDXL_REVERSE_SMOOTHSTEP": [0, 1, 0.991084, 0.965706, 0.925926, 0.873800, 0.811385, 0.740741, 0.663923, 0.582990, 0.500000, 0.417010, 0.336077, 0.259259, 0.188615, 0.126200, 0.074074, 0.034294, 0.008916, 0],
"SDXL_HALF_SMOOTHSTEP": [0, 0, 0.034294, 0.126200, 0.259259, 0.417010, 0.582990, 0.740741, 0.873800, 0.965706, 1, 0.965706, 0.873800, 0.740741, 0.582990, 0.417010, 0.259259, 0.126200, 0.034294, 0],
"SDXL_HALF_R_SMOOTHSTEP": [0, 1, 0.965706, 0.873800, 0.740741, 0.582990, 0.417010, 0.259259, 0.126200, 0.034294, 0, 0.034294, 0.126200, 0.259259, 0.417010, 0.582990, 0.740741, 0.873800, 0.965706, 1],
"SDXL_ONE_THIRD_SMOOTHSTEP": [0, 0, 0.074074, 0.259259, 0.500000, 0.740741, 0.925926, 1, 0.907407, 0.592593, 0, 0.592593, 0.907407, 1, 0.925926, 0.740741, 0.500000, 0.259259, 0.074074, 0],
"SDXL_ONE_THIRD_R_SMOOTHSTEP": [0, 1, 0.925926, 0.740741, 0.500000, 0.259259, 0.074074, 0, 0.092593, 0.407407, 1, 0.407407, 0.092593, 0, 0.074074, 0.259259, 0.500000, 0.740741, 0.925926, 1],
"SDXL_COSINE": [0, 1, 0.992404, 0.969846, 0.933013, 0.883022, 0.821394, 0.750000, 0.671010, 0.586824, 0.500000, 0.413176, 0.328990, 0.250000, 0.178606, 0.116978, 0.066987, 0.030154, 0.007596, 0],
"SDXL_REVERSE_COSINE": [0, 0, 0.007596, 0.030154, 0.066987, 0.116978, 0.178606, 0.250000, 0.328990, 0.413176, 0.500000, 0.586824, 0.671010, 0.750000, 0.821394, 0.883022, 0.933013, 0.969846, 0.992404, 1],
"SDXL_CUBIC_HERMITE": [0, 0, 0.268023, 0.461058, 0.588477, 0.659656, 0.683966, 0.670782, 0.629477, 0.569425, 0.500000, 0.430575, 0.370523, 0.329218, 0.316034, 0.340344, 0.411523, 0.538942, 0.731977, 1],
"SDXL_REVERSE_CUBIC_HERMITE": [0, 1, 0.731977, 0.538942, 0.411523, 0.340344, 0.316034, 0.329218, 0.370523, 0.430575, 0.500000, 0.569425, 0.629477, 0.670782, 0.683966, 0.659656, 0.588477, 0.461058, 0.268023, 0],
"SDXL_ALL_A": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
"SDXL_ALL_B": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
}
modules/merging/merge_rebasin.py
+2211
View File
File diff suppressed because it is too large Load Diff
modules/merging/merge_utils.py
+111
View File
@@ -0,0 +1,111 @@
import inspect
import re
from modules.merging import merge_methods
from modules.merging.merge_presets import BLOCK_WEIGHTS_PRESETS, SDXL_BLOCK_WEIGHTS_PRESETS
ALL_PRESETS = {}
ALL_PRESETS.update(BLOCK_WEIGHTS_PRESETS)
ALL_PRESETS.update(SDXL_BLOCK_WEIGHTS_PRESETS)
MERGE_METHODS = dict(inspect.getmembers(merge_methods, inspect.isfunction))
BETA_METHODS = [
name
for name, fn in MERGE_METHODS.items()
if "beta" in inspect.getfullargspec(fn)[0]
]
TRIPLE_METHODS = [
name
for name, fn in MERGE_METHODS.items()
if "c" in inspect.getfullargspec(fn)[0]
]
def interpolate(values, interp_lambda):
interpolated = []
for i in range(len(values[0])):
interpolated.append((1 - interp_lambda) * values[0][i] + interp_lambda * values[1][i])
return interpolated
class WeightClass:
def __init__(self,
model_a,
**kwargs,
):
self.SDXL = "model.diffusion_model.middle_block.1.transformer_blocks.9.norm3.weight" in model_a.keys()
self.NUM_INPUT_BLOCKS = 12 if not self.SDXL else 9
self.NUM_MID_BLOCK = 1
self.NUM_OUTPUT_BLOCKS = 12 if not self.SDXL else 9
self.NUM_TOTAL_BLOCKS = self.NUM_INPUT_BLOCKS + self.NUM_MID_BLOCK + self.NUM_OUTPUT_BLOCKS
self.iterations = kwargs.get("iterations", 1)
self.it = 0
self.re_basin = kwargs.get("re_basin", False)
self.ratioDict = {}
for key, value in kwargs.items():
if isinstance(value, list) or (key.lower() not in ["alpha", "beta"]):
self.ratioDict[key.lower()] = value
else:
self.ratioDict[key.lower()] = [value]
for key, value in self.ratioDict.items():
if key in ["alpha", "beta"]:
for i, v in enumerate(value):
if isinstance(v, str) and v.upper() in BLOCK_WEIGHTS_PRESETS.keys():
value[i] = BLOCK_WEIGHTS_PRESETS[v.upper()]
else:
value[i] = [float(x) for x in v.split(",")] if isinstance(v, str) else v
if not isinstance(value[i], list):
value[i] = [value[i]] * (self.NUM_TOTAL_BLOCKS + 1)
if len(value) > 1 and isinstance(value[0], list):
self.ratioDict[key] = interpolate(value, self.ratioDict.get(key + "_lambda", 0))
else:
self.ratioDict[key] = self.ratioDict[key][0]
def __call__(self, key, it=0):
current_bases = {}
if self.ratioDict.get("alpha", None):
current_bases["alpha"] = self.step_weights_and_bases(self.ratioDict["alpha"])
if self.ratioDict.get("beta", None):
current_bases["beta"] = self.step_weights_and_bases(self.ratioDict["beta"])
weight_index = 0
if "model" in key:
if "model.diffusion_model." in key:
weight_index = -1
re_inp = re.compile(r"\.input_blocks\.(\d+)\.") # 12
re_mid = re.compile(r"\.middle_block\.(\d+)\.") # 1
re_out = re.compile(r"\.output_blocks\.(\d+)\.") # 12
if "time_embed" in key:
weight_index = 0 # before input blocks
elif ".out." in key:
weight_index = self.NUM_TOTAL_BLOCKS - 1 # after output blocks
elif m := re_inp.search(key):
weight_index = int(m.groups()[0])
elif re_mid.search(key):
weight_index = self.NUM_INPUT_BLOCKS
elif m := re_out.search(key):
weight_index = self.NUM_INPUT_BLOCKS + self.NUM_MID_BLOCK + int(m.groups()[0])
if weight_index >= self.NUM_TOTAL_BLOCKS:
raise ValueError(f"illegal block index {key}")
current_bases = {k: w[weight_index] for k, w in current_bases.items()}
return current_bases
def step_weights_and_bases(self, ratio):
if not self.re_basin:
return ratio
new_ratio = [
1 - (1 - (1 + self.it) * v / self.iterations) / (1 - self.it * v / self.iterations)
if self.it > 0
else v / self.iterations
for v in ratio
]
return new_ratio
def set_it(self, it):
self.it = it
modules/middleware.py
+5 -4
View File
@@ -1,6 +1,5 @@
import ssl
import time
import datetime
import logging
from asyncio.exceptions import CancelledError
import anyio
@@ -14,6 +13,7 @@ from fastapi.encoders import jsonable_encoder
from installer import log
import modules.errors as errors
errors.install()
@@ -42,11 +42,12 @@ def setup_middleware(app: FastAPI, cmd_opts):
duration = str(round(time.time() - ts, 4))
res.headers["X-Process-Time"] = duration
endpoint = req.scope.get('path', 'err')
token = req.cookies.get("access-token") or req.cookies.get("access-token-unsecure")
if (cmd_opts.api_log or cmd_opts.api_only) and endpoint.startswith('/sdapi'):
if endpoint.endswith('/sdapi/v1/log'):
if '/sdapi/v1/log' in endpoint:
return res
log.info('API {t} {code} {prot}/{ver} {method} {endpoint} {cli} {duration}'.format( # pylint: disable=consider-using-f-string, logging-format-interpolation
t = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f"),
log.info('API {user} {code} {prot}/{ver} {method} {endpoint} {cli} {duration}'.format( # pylint: disable=consider-using-f-string, logging-format-interpolation
user = app.tokens.get(token),
code = res.status_code,
ver = req.scope.get('http_version', '0.0'),
cli = req.scope.get('client', ('0:0.0.0', 0))[0],
modules/modelloader.py
+5 -4
View File
@@ -58,13 +58,11 @@ def walk(top, onerror:callable=None):
def download_civit_meta(model_path: str, model_id):
fn = os.path.splitext(model_path)[0] + '.json'
if os.path.exists(fn):
return ''
url = f'https://civitai.com/api/v1/models/{model_id}'
r = shared.req(url)
if r.status_code == 200:
try:
shared.writefile(r.json(), fn, silent=True)
shared.writefile(r.json(), filename=fn, mode='w', silent=True)
msg = f'CivitAI download: id={model_id} url={url} file={fn}'
shared.log.info(msg)
return msg
@@ -72,7 +70,8 @@ def download_civit_meta(model_path: str, model_id):
msg = f'CivitAI download error: id={model_id} url={url} file={fn} {e}'
shared.log.error(msg)
return msg
return ''
return f'CivitAI download error: id={model_id} url={url} code={r.status_code}'
def download_civit_preview(model_path: str, preview_url: str):
ext = os.path.splitext(preview_url)[1]
@@ -274,6 +273,8 @@ def load_diffusers_models(model_path: str, command_path: str = None, clear=True)
try:
if "--" not in folder:
continue
if folder.endswith("-prior"):
continue
_, name = folder.split("--", maxsplit=1)
name = name.replace("--", "/")
folder = os.path.join(place, folder)
modules/processing.py
+9 -10
View File
@@ -58,7 +58,7 @@ def apply_color_correction(correction, original_image):
return image
def apply_overlay(image, paste_loc, index, overlays):
def apply_overlay(image: Image, paste_loc, index, overlays):
if overlays is None or index >= len(overlays):
return image
overlay = overlays[index]
@@ -686,6 +686,7 @@ def process_images(p: StableDiffusionProcessing) -> Processed:
p.override_settings.pop(k, None)
for k in p.override_settings.keys():
stored_opts[k] = shared.opts.data.get(k, None) or shared.opts.data_labels[k].default
res = None
try:
# if no checkpoint override or the override checkpoint can't be found, remove override entry and load opts checkpoint
if p.override_settings.get('sd_model_checkpoint', None) is not None and modules.sd_models.checkpoint_aliases.get(p.override_settings.get('sd_model_checkpoint')) is None:
@@ -1210,7 +1211,6 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
def init(self, all_prompts, all_seeds, all_subseeds):
if shared.backend == shared.Backend.DIFFUSERS and self.image_mask is not None:
shared.sd_model = modules.sd_models.set_diffuser_pipe(self.sd_model, modules.sd_models.DiffusersTaskType.INPAINTING)
# self.sd_model.dtype = self.sd_model.unet.dtype
elif shared.backend == shared.Backend.DIFFUSERS and self.image_mask is None:
shared.sd_model = modules.sd_models.set_diffuser_pipe(self.sd_model, modules.sd_models.DiffusersTaskType.IMAGE_2_IMAGE)
@@ -1225,6 +1225,7 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
else:
self.ops.append('img2img')
crop_region = None
image_mask = self.image_mask
if image_mask is not None:
if type(image_mask) == list:
@@ -1250,6 +1251,7 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
self.mask_for_overlay = Image.fromarray(np_mask)
self.overlay_images = []
latent_mask = self.latent_mask if self.latent_mask is not None else image_mask
add_color_corrections = shared.opts.img2img_color_correction and self.color_corrections is None
if add_color_corrections:
self.color_corrections = []
@@ -1280,14 +1282,12 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
if crop_region is not None:
image = image.crop(crop_region)
image = images.resize_image(3, image, self.width, self.height)
if shared.backend == shared.Backend.DIFFUSERS:
unprocessed.append(image)
self.init_images = [image] # assign early for diffusers
if image_mask is not None:
if self.inpainting_fill != 1:
image = modules.masking.fill(image, latent_mask)
if image_mask is not None and self.inpainting_fill != 1:
image = modules.masking.fill(image, latent_mask)
if add_color_corrections:
self.color_corrections.append(setup_color_correction(image))
if shared.backend == shared.Backend.DIFFUSERS:
unprocessed.append(image) # assign early for diffusers
image = np.array(image).astype(np.float32) / 255.0
image = np.moveaxis(image, 2, 0)
imgs.append(image)
@@ -1304,8 +1304,7 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
else:
raise RuntimeError(f"bad number of images passed: {len(imgs)}; expecting {self.batch_size} or less")
if shared.backend == shared.Backend.DIFFUSERS:
# we've already set self.init_images and self.mask and we dont need any more processing
return
return # we've already set self.init_images and self.mask and we dont need any more processing
image = torch.from_numpy(batch_images)
image = 2. * image - 1.
modules/processing_diffusers.py
+62 -27
View File
@@ -31,10 +31,10 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
p.height = tgt_height
p.width = tgt_width
hypertile_set(p)
if getattr(p, 'mask', None) is not None:
p.mask = images.resize_image(1, p.mask, tgt_width, tgt_height, upscaler_name=None)
if getattr(p, 'mask_for_overlay', None) is not None:
p.mask_for_overlay = images.resize_image(1, p.mask_for_overlay, tgt_width, tgt_height, upscaler_name=None)
if getattr(p, 'mask', None) is not None and p.mask.size != (tgt_width, tgt_height):
p.mask = images.resize_image(1, p.mask, tgt_width, tgt_height, upscaler_name=None)
if getattr(p, 'mask_for_overlay', None) is not None and p.mask_for_overlay.size != (tgt_width, tgt_height):
p.mask_for_overlay = images.resize_image(1, p.mask_for_overlay, tgt_width, tgt_height, upscaler_name=None)
def hires_resize(latents): # input=latents output=pil
latent_upscaler = shared.latent_upscale_modes.get(p.hr_upscaler, None)
@@ -42,16 +42,14 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
if latent_upscaler is not None:
latents = torch.nn.functional.interpolate(latents, size=(p.hr_upscale_to_y // 8, p.hr_upscale_to_x // 8), mode=latent_upscaler["mode"], antialias=latent_upscaler["antialias"])
first_pass_images = vae_decode(latents=latents, model=shared.sd_model, full_quality=p.full_quality, output_type='pil')
p.init_images = []
resized_images = []
for img in first_pass_images:
if latent_upscaler is None:
init_image = images.resize_image(1, img, p.hr_upscale_to_x, p.hr_upscale_to_y, upscaler_name=p.hr_upscaler)
resized_image = images.resize_image(1, img, p.hr_upscale_to_x, p.hr_upscale_to_y, upscaler_name=p.hr_upscaler)
else:
init_image = img
# if is_refiner_enabled:
# init_image = vae_encode(init_image, model=shared.sd_model, full_quality=p.full_quality)
p.init_images.append(init_image)
return p.init_images
resized_image = img
resized_images.append(resized_image)
return resized_images
def save_intermediate(latents, suffix):
for i in range(len(latents)):
@@ -98,7 +96,7 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
model.vae.to(devices.device)
latents.to(model.vae.device)
upcast = (model.vae.dtype == torch.float16) and model.vae.config.force_upcast and hasattr(model, 'upcast_vae')
upcast = (model.vae.dtype == torch.float16) and getattr(model.vae.config, 'force_upcast', False) and hasattr(model, 'upcast_vae')
if upcast: # this is done by diffusers automatically if output_type != 'latent'
model.upcast_vae()
latents = latents.to(next(iter(model.vae.post_quant_conv.parameters())).dtype)
@@ -157,7 +155,8 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
decoded = full_vae_decode(latents=latents, model=shared.sd_model)
else:
decoded = taesd_vae_decode(latents=latents)
# decoded = validate_sample(decoded) # TODO validate sample
# TODO validate decoded sample diffusers
# decoded = validate_sample(decoded)
imgs = model.image_processor.postprocess(decoded, output_type=output_type)
shared.state.job = prev_job
return imgs
@@ -183,6 +182,8 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
negative_prompts = [negative_prompts]
while len(negative_prompts) < len(prompts):
negative_prompts.append(negative_prompts[-1])
while len(prompts) < len(negative_prompts):
prompts.append(prompts[-1])
if type(prompts_2) is str:
prompts_2 = [prompts_2]
if type(prompts_2) is list:
@@ -213,7 +214,26 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
p.mask = TF.to_pil_image(torch.ones_like(TF.to_tensor(p.init_images[0]))).convert("L")
width = 8 * math.ceil(p.init_images[0].width / 8)
height = 8 * math.ceil(p.init_images[0].height / 8)
# option-1: use images as inputs
task_args = {"image": p.init_images, "mask_image": p.mask, "strength": p.denoising_strength, "height": height, "width": width}
""" # option-2: preprocess images into latents using diffusers
vae_scale_factor = 2 ** (len(model.vae.config.block_out_channels) - 1)
image_processor = diffusers.image_processor.VaeImageProcessor(vae_scale_factor=vae_scale_factor)
mask_processor = diffusers.image_processor.VaeImageProcessor(vae_scale_factor=vae_scale_factor, do_normalize=False, do_binarize=True, do_convert_grayscale=True)
init_image = image_processor.preprocess(p.init_images[0], width=width, height=height)
mask_image = mask_processor.preprocess(p.mask, width=width, height=height)
task_args = {"image": p.init_images, "mask_image": p.mask, "strength": p.denoising_strength, "height": height, "width": width}
"""
""" # option-2: manually assemble masked image latents
masked_image_latents = []
mask_image = TF.to_tensor(p.mask)
for init_image in p.init_images:
init_image = TF.to_tensor(p.init_images[0])
masked_image = init_image * (mask_image > 0.5)
masked_image_latents.append(torch.cat([masked_image, mask_image], dim=0))
masked_image_latents = torch.stack(masked_image_latents, dim=0).to(shared.device)
task_args = {"image": p.init_images, "mask_image": mask_image, "masked_image_latents": masked_image_latents, "strength": p.denoising_strength, "height": height, "width": width}
"""
if model.__class__.__name__ == 'LatentConsistencyModelPipeline' and hasattr(p, 'init_images') and len(p.init_images) > 0:
init_latents = [vae_encode(image, model=shared.sd_model, full_quality=p.full_quality).squeeze(dim=0) for image in p.init_images]
init_latent = torch.stack(init_latents, dim=0).to(shared.device)
@@ -305,6 +325,8 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
clean['image'] = type(clean['image'])
if 'mask_image' in clean:
clean['mask_image'] = type(clean['mask_image'])
if 'masked_image_latents' in clean:
clean['masked_image_latents'] = type(clean['masked_image_latents'])
if 'prompt' in clean:
clean['prompt'] = len(clean['prompt'])
if 'negative_prompt' in clean:
@@ -381,27 +403,41 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
use_denoise_start = bool(is_img2img and p.refiner_start > 0 and p.refiner_start < 1)
def calculate_base_steps():
steps = p.steps
if use_refiner_start:
steps = (p.steps // (1.0 - p.refiner_start)) if shared.sd_model_type == 'sdxl' else p.steps
if is_img2img:
if use_denoise_start and shared.sd_model_type == 'sdxl':
steps = p.steps // (1 - p.refiner_start)
else:
steps = (p.steps // p.denoising_strength) + 1
elif use_refiner_start and shared.sd_model_type == 'sdxl':
steps = (p.steps // p.refiner_start) + 1
else:
steps = p.steps
if os.environ.get('SD_STEPS_DEBUG', None) is not None:
shared.log.debug(f'Steps: type=base input={p.steps} output={steps} refiner={use_refiner_start}')
return max(2, int(steps))
def calculate_hires_steps():
# denoising strength is applied to steps by diffusers so this is no-op
# steps = (p.hr_second_pass_steps * p.denoising_strength) if p.hr_second_pass_steps > 0 else (p.steps * p.denoising_strength)
steps = p.hr_second_pass_steps if p.hr_second_pass_steps > 0 else p.steps
if p.hr_second_pass_steps > 0:
steps = (p.hr_second_pass_steps // p.denoising_strength) + 1
else:
steps = (p.steps // p.denoising_strength) + 1
if os.environ.get('SD_STEPS_DEBUG', None) is not None:
shared.log.debug(f'Steps: type=hires input={p.hr_second_pass_steps} output={steps} denoise={p.denoising_strength}')
return max(2, int(steps))
def calculate_refiner_steps():
# diffusers apply additional math to refiner steps, but we leave numbers as-is without correction
if p.refiner_start > 0 and p.refiner_start < 1:
steps = ((1 - p.refiner_start) * p.refiner_steps) if p.refiner_steps > 0 else ((1 - p.refiner_start) * p.steps)
if "StableDiffusionXL" in shared.sd_refiner.__class__.__name__:
if p.refiner_start > 0 and p.refiner_start < 1:
#steps = p.refiner_steps // (1 - p.refiner_start) # SDXL with denoise strenght
steps = (p.refiner_steps // (1 - p.refiner_start) // 2) + 1
else:
steps = (p.refiner_steps // p.denoising_strength) + 1
else:
steps = (p.denoising_strength * p.refiner_steps) if p.refiner_steps > 0 else (p.denoising_strength * p.steps)
#steps = p.refiner_steps # SD 1.5 with denoise strenght
steps = (p.refiner_steps * 1.25) + 1
if os.environ.get('SD_STEPS_DEBUG', None) is not None:
shared.log.debug(f'Steps: type=refiner input={p.refiner_steps} output={steps} start={p.refiner_start} denoise={p.denoising_strength}')
return max(2, int(steps))
@@ -429,7 +465,6 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
desc='Base',
)
shared.state.sampling_steps = base_args['num_inference_steps']
p.extra_generation_params['CFG rescale'] = p.diffusers_guidance_rescale
p.extra_generation_params["Sampler Eta"] = shared.opts.scheduler_eta if shared.opts.scheduler_eta is not None and shared.opts.scheduler_eta > 0 and shared.opts.scheduler_eta < 1 else None
try:
output = shared.sd_model(**base_args) # pylint: disable=not-callable
@@ -449,7 +484,7 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
return results
# optional hires pass
if p.enable_hr and p.hr_upscaler != 'None' and p.denoising_strength > 0 and len(getattr(p, 'init_images', [])) == 0:
if p.enable_hr and getattr(p, 'hr_upscaler', 'None') != 'None' and len(getattr(p, 'init_images', [])) == 0:
p.is_hr_pass = True
latent_scale_mode = shared.latent_upscale_modes.get(p.hr_upscaler, None) if (hasattr(p, "hr_upscaler") and p.hr_upscaler is not None) else shared.latent_upscale_modes.get(shared.latent_upscale_default_mode, "None")
if p.is_hr_pass:
@@ -461,7 +496,7 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
save_intermediate(latents=output.images, suffix="-before-hires")
shared.state.job = 'upscale'
output.images = hires_resize(latents=output.images)
if latent_scale_mode is not None or p.hr_force:
if (latent_scale_mode is not None or p.hr_force) and p.denoising_strength > 0:
p.ops.append('hires')
shared.sd_model = sd_models.set_diffuser_pipe(shared.sd_model, sd_models.DiffusersTaskType.IMAGE_2_IMAGE)
recompile_model(hires=True)
@@ -478,7 +513,7 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
guidance_rescale=p.diffusers_guidance_rescale,
output_type='latent' if hasattr(shared.sd_model, 'vae') else 'np',
clip_skip=p.clip_skip,
image=p.init_images,
image=output.images,
strength=p.denoising_strength,
desc='Hires',
)
modules/prompt_parser.py
+5 -2
View File
@@ -14,7 +14,7 @@ from typing import List
import lark
import torch
from compel import Compel
from modules.shared import opts, log
from modules.shared import opts, log, backend, Backend
# a prompt like this: "fantasy landscape with a [mountain:lake:0.25] and [an oak:a christmas tree:0.75][ in foreground::0.6][ in background:0.25] [shoddy:masterful:0.5]"
# will be represented with prompt_schedule like this (assuming steps=100):
@@ -320,7 +320,10 @@ def parse_prompt_attention(text):
whitespace = ''
else:
re_attention = re_attention_v1
text = text.replace('\\n', ' ')
if backend == Backend.DIFFUSERS:
text = text.replace('\n', ' BREAK ')
else:
text = text.replace('\n', ' ')
whitespace = ' '
def multiply_range(start_position, multiplier):
modules/prompt_parser_diffusers.py
+35 -20
View File
@@ -3,7 +3,7 @@ import typing
import torch
from compel import ReturnedEmbeddingsType
from compel.embeddings_provider import BaseTextualInversionManager, EmbeddingsProvider
from modules import shared, prompt_parser
from modules import shared, prompt_parser, devices
debug = shared.log.info if os.environ.get('SD_PROMPT_DEBUG', None) is not None else lambda *args, **kwargs: None
@@ -69,7 +69,7 @@ def encode_prompts(pipeline, prompts: list, negative_prompts: list, clip_skip: t
negative_embeds = []
negative_pooleds = []
for i in range(len(prompts)):
prompt_embed, positive_pooled, negative_embed, negative_pooled = get_weighted_text_embeddings_sdxl(pipeline,prompts[i], negative_prompts[i], clip_skip)
prompt_embed, positive_pooled, negative_embed, negative_pooled = get_weighted_text_embeddings(pipeline, prompts[i], negative_prompts[i], clip_skip)
prompt_embeds.append(prompt_embed)
positive_pooleds.append(positive_pooled)
negative_embeds.append(negative_embed)
@@ -97,6 +97,7 @@ def get_prompts_with_weights(prompt: str):
def prepare_embedding_providers(pipe, clip_skip):
device = pipe.device if str(pipe.device) != 'meta' else devices.device
embeddings_providers = []
if 'XL' in pipe.__class__.__name__:
embedding_type = ReturnedEmbeddingsType.PENULTIMATE_HIDDEN_STATES_NON_NORMALIZED
@@ -106,20 +107,21 @@ def prepare_embedding_providers(pipe, clip_skip):
clip_skip = 2
embedding_type = CLIP_SKIP_MAPPING[clip_skip]
if getattr(pipe, "tokenizer", None) is not None and getattr(pipe, "text_encoder", None) is not None:
embedding = EmbeddingsProvider(tokenizer=pipe.tokenizer, text_encoder=pipe.text_encoder, truncate=False, returned_embeddings_type=embedding_type, device=pipe.device)
embedding = EmbeddingsProvider(tokenizer=pipe.tokenizer, text_encoder=pipe.text_encoder, truncate=False, returned_embeddings_type=embedding_type, device=device)
embeddings_providers.append(embedding)
if getattr(pipe, "tokenizer_2", None) is not None and getattr(pipe, "text_encoder_2", None) is not None:
embedding = EmbeddingsProvider(tokenizer=pipe.tokenizer_2, text_encoder=pipe.text_encoder_2, truncate=False, returned_embeddings_type=embedding_type, device=pipe.device)
embedding = EmbeddingsProvider(tokenizer=pipe.tokenizer_2, text_encoder=pipe.text_encoder_2, truncate=False, returned_embeddings_type=embedding_type, device=device)
embeddings_providers.append(embedding)
return embeddings_providers
def pad_to_same_length(embeds):
def pad_to_same_length(pipe, embeds):
device = pipe.device if str(pipe.device) != 'meta' else devices.device
try: #SDXL
empty_embed = shared.sd_model.encode_prompt("")
except Exception: #SD1.5
empty_embed = shared.sd_model.encode_prompt("",shared.sd_model.device, 1, False)
empty_batched = torch.cat([empty_embed[0]] * embeds[0].shape[0])
empty_embed = shared.sd_model.encode_prompt("", device, 1, False)
empty_batched = torch.cat([empty_embed[0].to(embeds[0].device)] * embeds[0].shape[0])
max_token_count = max([embed.shape[1] for embed in embeds])
for i, embed in enumerate(embeds):
while embed.shape[1] < max_token_count:
@@ -127,7 +129,9 @@ def pad_to_same_length(embeds):
embeds[i] = embed
return embeds
def get_weighted_text_embeddings_sdxl(pipe, prompt: str = "", neg_prompt: str = "", clip_skip: int = None):
def get_weighted_text_embeddings(pipe, prompt: str = "", neg_prompt: str = "", clip_skip: int = None):
device = pipe.device if str(pipe.device) != 'meta' else devices.device
prompt_2 = prompt.split("TE2:")[-1]
neg_prompt_2 = neg_prompt.split("TE2:")[-1]
prompt = prompt.split("TE2:")[0]
@@ -150,33 +154,44 @@ def get_weighted_text_embeddings_sdxl(pipe, prompt: str = "", neg_prompt: str =
negative_prompt_embeds = []
pooled_prompt_embeds = None
negative_pooled_prompt_embeds = None
for i in range(len(embedding_providers)):
embed, ptokens = embedding_providers[i].get_embeddings_for_weighted_prompt_fragments(text_batch=[positives[i]], fragment_weights_batch=[positive_weights[i]], device=pipe.device, should_return_tokens=True)
prompt_embeds.append(embed)
embed, ntokens = embedding_providers[i].get_embeddings_for_weighted_prompt_fragments(text_batch=[negatives[i]], fragment_weights_batch=[negative_weights[i]],device=pipe.device, should_return_tokens=True)
# add BREAK keyword that splits the prompt into multiple fragments
text = positives[i]
weights = positive_weights[i]
text.append('BREAK')
weights.append(-1)
provider_embed = []
while 'BREAK' in text:
pos = text.index('BREAK')
embed, ptokens = embedding_providers[i].get_embeddings_for_weighted_prompt_fragments(text_batch=[text[:pos]], fragment_weights_batch=[weights[:pos]], device=device, should_return_tokens=True)
provider_embed.append(embed)
text = text[pos+1:]
weights = weights[pos+1:]
prompt_embeds.append(torch.cat(provider_embed, dim=1))
# negative prompt has no keywords
embed, ntokens = embedding_providers[i].get_embeddings_for_weighted_prompt_fragments(text_batch=[negatives[i]], fragment_weights_batch=[negative_weights[i]], device=device, should_return_tokens=True)
negative_prompt_embeds.append(embed)
if prompt_embeds[-1].shape[-1] > 768:
if shared.opts.diffusers_pooled == "weighted":
pooled_prompt_embeds = prompt_embeds[-1][
torch.arange(prompt_embeds[-1].shape[0], device=pipe.device),
(ptokens.to(dtype=torch.int, device=pipe.device) == 49407)
torch.arange(prompt_embeds[-1].shape[0], device=device),
(ptokens.to(dtype=torch.int, device=device) == 49407)
.int()
.argmax(dim=-1),
]
negative_pooled_prompt_embeds = negative_prompt_embeds[-1][
torch.arange(negative_prompt_embeds[-1].shape[0], device=pipe.device),
(ntokens.to(dtype=torch.int, device=pipe.device) == 49407)
torch.arange(negative_prompt_embeds[-1].shape[0], device=device),
(ntokens.to(dtype=torch.int, device=device) == 49407)
.int()
.argmax(dim=-1),
]
else:
pooled_prompt_embeds = embedding_providers[-1].get_pooled_embeddings(texts=[prompt_2], device=pipe.device) if prompt_embeds[-1].shape[-1] > 768 else None
negative_pooled_prompt_embeds = embedding_providers[-1].get_pooled_embeddings(texts=[neg_prompt_2], device=pipe.device) if negative_prompt_embeds[-1].shape[-1] > 768 else None
pooled_prompt_embeds = embedding_providers[-1].get_pooled_embeddings(texts=[prompt_2], device=device) if prompt_embeds[-1].shape[-1] > 768 else None
negative_pooled_prompt_embeds = embedding_providers[-1].get_pooled_embeddings(texts=[neg_prompt_2], device=device) if negative_prompt_embeds[-1].shape[-1] > 768 else None
prompt_embeds = torch.cat(prompt_embeds, dim=-1) if len(prompt_embeds) > 1 else prompt_embeds[0]
negative_prompt_embeds = torch.cat(negative_prompt_embeds, dim=-1) if len(negative_prompt_embeds) > 1 else negative_prompt_embeds[0]
if prompt_embeds.shape[1] != negative_prompt_embeds.shape[1]:
[prompt_embeds, negative_prompt_embeds] = pad_to_same_length([prompt_embeds, negative_prompt_embeds])
[prompt_embeds, negative_prompt_embeds] = pad_to_same_length(pipe, [prompt_embeds, negative_prompt_embeds])
return prompt_embeds, pooled_prompt_embeds, negative_prompt_embeds, negative_pooled_prompt_embeds
modules/sd_hijack.py
+5 -2
View File
@@ -178,14 +178,14 @@ class StableDiffusionModelHijack:
except Exception as err:
shared.log.warning(f"IPEX Optimize not supported: {err}")
if opts.cuda_compile and opts.cuda_compile_backend != 'none' and shared.backend == shared.Backend.ORIGINAL:
if (opts.cuda_compile or opts.cuda_compile_vae or opts.cuda_compile_upscaler) and shared.opts.cuda_compile_backend != 'none' and shared.backend == shared.Backend.ORIGINAL:
try:
import logging
shared.log.info(f"Compiling pipeline={m.model.__class__.__name__} mode={opts.cuda_compile_backend}")
import torch._dynamo # pylint: disable=unused-import,redefined-outer-name
if shared.opts.cuda_compile_backend == "openvino_fx":
torch._dynamo.reset() # pylint: disable=protected-access
from modules.intel.openvino import openvino_fx, openvino_clear_caches # pylint: disable=unused-import
from modules.intel.openvino import openvino_fx, openvino_clear_caches # pylint: disable=unused-import, no-name-in-module
openvino_clear_caches()
torch._dynamo.eval_frame.check_if_dynamo_supported = lambda: True # pylint: disable=protected-access
log_level = logging.WARNING if opts.cuda_compile_verbose else logging.CRITICAL # pylint: disable=protected-access
@@ -202,6 +202,9 @@ class StableDiffusionModelHijack:
shared.log.info("Model complilation done.")
except Exception as err:
shared.log.warning(f"Model compile not supported: {err}")
finally:
from installer import setup_logging
setup_logging()
self.optimization_method = apply_optimizations()
self.clip = m.cond_stage_model
modules/sd_models.py
+49 -84
View File
@@ -20,8 +20,7 @@ import tomesd
from transformers import logging as transformers_logging
import ldm.modules.midas as midas
from ldm.util import instantiate_from_config
from modules import paths, shared, shared_items, shared_state, modelloader, devices, script_callbacks, sd_vae, sd_disable_initialization, errors, hashes, sd_models_config
from modules.sd_hijack_inpainting import do_inpainting_hijack
from modules import paths, shared, shared_items, shared_state, modelloader, devices, script_callbacks, sd_vae, sd_disable_initialization, errors, hashes, sd_models_config, sd_models_compile, sd_hijack_inpainting
from modules.timer import Timer
from modules.memstats import memory_stats
from modules.paths import models_path, script_path
@@ -119,18 +118,6 @@ class CheckpointInfo:
return self.shorthash
#Used by OpenVINO, can be used with TensorRT or Olive
class CompiledModelState:
def __init__(self):
self.first_pass = True
self.height = 512
self.width = 512
self.batch_size = 1
self.partition_id = 0
self.cn_model = []
self.lora_model = []
class NoWatermark:
def apply_watermark(self, img):
return img
@@ -340,7 +327,7 @@ def read_metadata_from_safetensors(filename):
if not os.path.isfile(sd_metadata_file):
sd_metadata = {}
else:
sd_metadata = shared.readfile(sd_metadata_file)
sd_metadata = shared.readfile(sd_metadata_file, lock=True)
res = sd_metadata.get(filename, None)
if res is not None:
return res
@@ -400,7 +387,7 @@ def read_state_dict(checkpoint_file, map_location=None): # pylint: disable=unuse
return None
try:
pl_sd = None
with progress.open(checkpoint_file, 'rb', description=f'[cyan]Loading weights: [yellow]{checkpoint_file}', auto_refresh=True, console=shared.console) as f:
with progress.open(checkpoint_file, 'rb', description=f'[cyan]Loading model: [yellow]{checkpoint_file}', auto_refresh=True, console=shared.console) as f:
_, extension = os.path.splitext(checkpoint_file)
if extension.lower() == ".ckpt" and shared.opts.sd_disable_ckpt:
shared.log.warning(f"Checkpoint loading disabled: {checkpoint_file}")
@@ -596,7 +583,7 @@ model_data = ModelData()
def change_backend():
shared.log.info(f'Backend changed: {shared.backend}')
shared.log.warning('Server restart required to apply all changes')
shared.log.warning('Full server restart required to apply all changes')
if shared.backend == shared.Backend.ORIGINAL:
change_from = shared.Backend.DIFFUSERS
else:
@@ -610,51 +597,52 @@ def change_backend():
refresh_vae_list()
def detect_pipeline(f: str, op: str = 'model'):
def detect_pipeline(f: str, op: str = 'model', warning=True):
if not f.endswith('.safetensors'):
return None, None
guess = shared.opts.diffusers_pipeline
warn = shared.log.warning if warning else lambda *args, **kwargs: None
if guess == 'Autodetect':
try:
size = round(os.path.getsize(f) / 1024 / 1024)
if size < 128:
shared.log.warning(f'Model size smaller than expected: {f} size={size} MB')
warn(f'Model size smaller than expected: {f} size={size} MB')
elif (size >= 316 and size <= 324) or (size >= 156 and size <= 164): # 320 or 160
shared.log.warning(f'Model detected as VAE model, but attempting to load as model: {op}={f} size={size} MB')
warn(f'Model detected as VAE model, but attempting to load as model: {op}={f} size={size} MB')
guess = 'VAE'
elif size >= 5351 and size <= 5359: # 5353
guess = 'Stable Diffusion' # SD v2
elif size >= 5791 and size <= 5799: # 5795
if shared.backend == shared.Backend.ORIGINAL:
shared.log.warning(f'Model detected as SD-XL refiner model, but attempting to load using backend=original: {op}={f} size={size} MB')
warn(f'Model detected as SD-XL refiner model, but attempting to load using backend=original: {op}={f} size={size} MB')
if op == 'model':
shared.log.warning(f'Model detected as SD-XL refiner model, but attempting to load a base model: {op}={f} size={size} MB')
warn(f'Model detected as SD-XL refiner model, but attempting to load a base model: {op}={f} size={size} MB')
guess = 'Stable Diffusion XL'
elif (size >= 6611 and size <= 6619) or (size >= 6771 and size <= 6779): # 6617, HassakuXL is 6776
if shared.backend == shared.Backend.ORIGINAL:
shared.log.warning(f'Model detected as SD-XL base model, but attempting to load using backend=original: {op}={f} size={size} MB')
warn(f'Model detected as SD-XL base model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Stable Diffusion XL'
elif size >= 3361 and size <= 3369: # 3368
if shared.backend == shared.Backend.ORIGINAL:
shared.log.warning(f'Model detected as SD upscale model, but attempting to load using backend=original: {op}={f} size={size} MB')
warn(f'Model detected as SD upscale model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Stable Diffusion Upscale'
elif size >= 4891 and size <= 4899: # 4897
if shared.backend == shared.Backend.ORIGINAL:
shared.log.warning(f'Model detected as SD XL inpaint model, but attempting to load using backend=original: {op}={f} size={size} MB')
warn(f'Model detected as SD XL inpaint model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Stable Diffusion XL Inpaint'
elif size >= 9791 and size <= 9799: # 9794
if shared.backend == shared.Backend.ORIGINAL:
shared.log.warning(f'Model detected as SD XL instruct pix2pix model, but attempting to load using backend=original: {op}={f} size={size} MB')
warn(f'Model detected as SD XL instruct pix2pix model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Stable Diffusion XL Instruct'
else:
guess = 'Stable Diffusion'
if 'LCM_' in f or 'LCM-' in f:
if shared.backend == shared.Backend.ORIGINAL:
shared.log.warning(f'Model detected as LCM model, but attempting to load using backend=original: {op}={f} size={size} MB')
warn(f'Model detected as LCM model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Latent Consistency Model'
if 'PixArt' in f:
if shared.backend == shared.Backend.ORIGINAL:
shared.log.warning(f'Model detected as PixArt Alpha model, but attempting to load using backend=original: {op}={f} size={size} MB')
warn(f'Model detected as PixArt Alpha model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'PixArt Alpha'
pipeline = shared_items.get_pipelines().get(guess, None)
shared.log.info(f'Autodetect: {op}="{guess}" class={pipeline.__name__} file="{f}" size={size}MB')
@@ -675,55 +663,6 @@ def detect_pipeline(f: str, op: str = 'model'):
return pipeline, guess
def compile_diffusers(sd_model):
try:
if shared.opts.ipex_optimize:
import intel_extension_for_pytorch as ipex # pylint: disable=import-error, unused-import
sd_model.unet.training = False
sd_model.unet = ipex.optimize(sd_model.unet, dtype=devices.dtype_unet, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
if hasattr(sd_model, 'vae'):
sd_model.vae.training = False
sd_model.vae = ipex.optimize(sd_model.vae, dtype=devices.dtype_vae, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
if hasattr(sd_model, 'movq'):
sd_model.movq.training = False
sd_model.movq = ipex.optimize(sd_model.movq, dtype=devices.dtype_vae, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
shared.log.info("Applied IPEX Optimize.")
except Exception as err:
shared.log.warning(f"IPEX Optimize not supported: {err}")
try:
if (shared.opts.cuda_compile or shared.opts.cuda_compile_vae or shared.opts.cuda_compile_upscaler) and shared.opts.cuda_compile_backend != 'none':
shared.log.info(f"Compiling pipeline={sd_model.__class__.__name__} shape={8 * sd_model.unet.config.sample_size} mode={shared.opts.cuda_compile_backend}")
import torch._dynamo # pylint: disable=unused-import,redefined-outer-name
if shared.opts.cuda_compile_backend == "openvino_fx":
torch._dynamo.reset() # pylint: disable=protected-access
from modules.intel.openvino import openvino_fx, openvino_clear_caches # pylint: disable=unused-import
openvino_clear_caches()
torch._dynamo.eval_frame.check_if_dynamo_supported = lambda: True # pylint: disable=protected-access
if shared.compiled_model_state is None:
shared.compiled_model_state = CompiledModelState()
shared.compiled_model_state.first_pass = True if not shared.opts.cuda_compile_precompile else False
log_level = logging.WARNING if shared.opts.cuda_compile_verbose else logging.CRITICAL # pylint: disable=protected-access
if hasattr(torch, '_logging'):
torch._logging.set_logs(dynamo=log_level, aot=log_level, inductor=log_level) # pylint: disable=protected-access
torch._dynamo.config.verbose = shared.opts.cuda_compile_verbose # pylint: disable=protected-access
torch._dynamo.config.suppress_errors = shared.opts.cuda_compile_errors # pylint: disable=protected-access
if shared.opts.cuda_compile:
sd_model.unet = torch.compile(sd_model.unet, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph) # pylint: disable=attribute-defined-outside-init
if shared.opts.cuda_compile_vae:
if hasattr(sd_model, 'vae'):
sd_model.vae.decode = torch.compile(sd_model.vae.decode, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph) # pylint: disable=attribute-defined-outside-init
if hasattr(sd_model, 'movq'):
sd_model.movq.decode = torch.compile(sd_model.movq.decode, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph) # pylint: disable=attribute-defined-outside-init
from installer import setup_logging
setup_logging()
if shared.opts.cuda_compile_precompile:
sd_model("dummy prompt")
shared.log.info("Complilation done.")
except Exception as err:
shared.log.warning(f"Model compile not supported: {err}")
def set_diffuser_options(sd_model, vae, op: str):
if sd_model is None:
shared.log.warning(f'{op} is not loaded')
@@ -780,10 +719,8 @@ def set_diffuser_options(sd_model, vae, op: str):
sd_model.vae = vae
if shared.opts.diffusers_vae_upcast != 'default':
if shared.opts.diffusers_vae_upcast == 'true':
# sd_model.vae.config["force_upcast"] = True
sd_model.vae.config.force_upcast = True
else:
# sd_model.vae.config["force_upcast"] = False
sd_model.vae.config.force_upcast = False
if shared.opts.no_half_vae:
devices.dtype_vae = torch.float32
@@ -791,6 +728,18 @@ def set_diffuser_options(sd_model, vae, op: str):
shared.log.debug(f'Setting {op} VAE: name={sd_vae.loaded_vae_file} upcast={sd_model.vae.config.get("force_upcast", None)}')
if shared.opts.cross_attention_optimization == "xFormers" and hasattr(sd_model, 'enable_xformers_memory_efficient_attention'):
sd_model.enable_xformers_memory_efficient_attention()
if shared.opts.diffusers_eval:
if hasattr(sd_model, "unet"):
sd_model.unet.requires_grad_(False)
sd_model.unet.eval()
if hasattr(sd_model, "vae"):
sd_model.vae.requires_grad_(False)
sd_model.vae.eval()
if hasattr(sd_model, "text_encoder"):
sd_model.text_encoder.requires_grad_(False)
sd_model.text_encoder.eval()
if shared.opts.opt_channelslast and hasattr(sd_model, 'unet'):
shared.log.debug(f'Setting {op}: enable channels last')
sd_model.unet.to(memory_format=torch.channels_last)
@@ -978,7 +927,7 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No
elif not getattr(sd_model, 'has_accelerate', False):
sd_model.to(devices.device)
compile_diffusers(sd_model)
sd_models_compile.compile_diffusers(sd_model)
if sd_model is None:
shared.log.error('Diffuser model not loaded')
@@ -1042,9 +991,25 @@ def set_diffuser_pipe(pipe, new_pipe_type):
has_accelerate = getattr(pipe, "has_accelerate", None)
embedding_db = getattr(pipe, "embedding_db", None)
if shared.opts.diffusers_force_inpaint:
if new_pipe_type == DiffusersTaskType.IMAGE_2_IMAGE:
new_pipe_type = DiffusersTaskType.INPAINTING # sdxl may work better with init mask
# TODO implement alternative diffusion pipelines
"""
from collections import OrderedDict
AUTO_TEXT2IMAGE_PIPELINES_MAPPING = OrderedDict(
[
("stable-diffusion", diffusers.StableDiffusionPipeline),
("stable-diffusion-xl", diffusers.StableDiffusionXLPipeline),
("if", diffusers.IFPipeline),
("kandinsky", diffusers.KandinskyCombinedPipeline),
("kandinsky22", diffusers.KandinskyV22CombinedPipeline),
("stable-diffusion-controlnet", diffusers.StableDiffusionControlNetPipeline),
("stable-diffusion-xl-controlnet", diffusers.StableDiffusionXLControlNetPipeline),
("wuerstchen", diffusers.WuerstchenCombinedPipeline),
("lcm", diffusers.LatentConsistencyModelPipeline),
("pixart", diffusers.PixArtAlphaPipeline),
]
)
diffusers.pipelines.auto_pipeline.AUTO_TEXT2IMAGE_PIPELINES_MAPPING = AUTO_TEXT2IMAGE_PIPELINES_MAPPING
"""
try:
if new_pipe_type == DiffusersTaskType.TEXT_2_IMAGE:
new_pipe = diffusers.AutoPipelineForText2Image.from_pipe(pipe)
@@ -1108,7 +1073,7 @@ def load_model(checkpoint_info=None, already_loaded_state_dict=None, timer=None,
current_checkpoint_info = model_data.sd_refiner.sd_checkpoint_info
unload_model_weights(op=op)
do_inpainting_hijack()
sd_hijack_inpainting.do_inpainting_hijack()
devices.set_cuda_params()
if already_loaded_state_dict is not None:
state_dict = already_loaded_state_dict
modules/sd_models_compile.py
+142
View File
@@ -0,0 +1,142 @@
import time
import logging
import torch
from modules import shared, devices
from installer import setup_logging
#Used by OpenVINO, can be used with TensorRT or Olive
class CompiledModelState:
def __init__(self):
self.first_pass = True
self.height = 512
self.width = 512
self.batch_size = 1
self.partition_id = 0
self.cn_model = []
self.lora_model = []
self.lora_compile = False
self.compiled_cache = {}
self.partitioned_modules = {}
def optimize_ipex(sd_model):
try:
t0 = time.time()
import intel_extension_for_pytorch as ipex # pylint: disable=import-error, unused-import
sd_model.unet.training = False
sd_model.unet = ipex.optimize(sd_model.unet, dtype=devices.dtype_unet, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
if hasattr(sd_model, 'vae'):
sd_model.vae.training = False
sd_model.vae = ipex.optimize(sd_model.vae, dtype=devices.dtype_vae, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
if hasattr(sd_model, 'movq'):
sd_model.movq.training = False
sd_model.movq = ipex.optimize(sd_model.movq, dtype=devices.dtype_vae, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
t1 = time.time()
shared.log.info(f"Model compile: mode=IPEX-optimize time={t1-t0:.2f}")
except Exception as e:
shared.log.warning(f"Model compile: task=IPEX-optimize error: {e}")
def optimize_openvino():
try:
from modules.intel.openvino import openvino_fx # pylint: disable=unused-import
torch._dynamo.eval_frame.check_if_dynamo_supported = lambda: True # pylint: disable=protected-access
if shared.compiled_model_state is None:
shared.compiled_model_state = CompiledModelState()
else:
if not shared.compiled_model_state.lora_compile:
shared.compiled_model_state.lora_compile = False
shared.compiled_model_state.lora_model = []
shared.compiled_model_state.compiled_cache.clear()
shared.compiled_model_state.partitioned_modules.clear()
shared.compiled_model_state.first_pass = True if not shared.opts.cuda_compile_precompile else False
except Exception as e:
shared.log.warning(f"Model compile: task=OpenVINO: {e}")
def compile_stablefast(sd_model):
try:
import sfast.compilers.stable_diffusion_pipeline_compiler as sf
except Exception as e:
shared.log.warning(f'Model compile using stable-fast: {e}')
return sd_model
config = sf.CompilationConfig.Default()
try:
import xformers # pylint: disable=unused-import
config.enable_xformers = True
except Exception:
pass
try:
import triton # pylint: disable=unused-import
config.enable_triton = True
except Exception:
pass
import warnings
warnings.filterwarnings("ignore", category=torch.jit.TracerWarning)
config.enable_cuda_graph = shared.opts.cuda_compile_fullgraph
config.enable_jit_freeze = shared.opts.diffusers_eval
config.memory_format = torch.channels_last if shared.opts.opt_channelslast else torch.contiguous_format
# config.enable_cnn_optimization
# config.prefer_lowp_gemm
try:
t0 = time.time()
sd_model = sf.compile(sd_model, config)
setup_logging() # compile messes with logging so reset is needed
if shared.opts.cuda_compile_precompile:
sd_model("dummy prompt")
t1 = time.time()
shared.log.info(f"Model compile: task=Stable-fast config={config.__dict__} time={t1-t0:.2f}")
except Exception as e:
shared.log.info(f"Model compile: task=Stable-fast error: {e}")
return sd_model
def compile_torch(sd_model):
try:
import torch._dynamo # pylint: disable=unused-import,redefined-outer-name
torch._dynamo.reset() # pylint: disable=protected-access
shared.log.debug(f"Model compile available backends: {torch._dynamo.list_backends()}") # pylint: disable=protected-access
if shared.opts.ipex_optimize:
optimize_ipex(sd_model)
if shared.opts.cuda_compile_backend == "openvino_fx":
optimize_openvino()
log_level = logging.WARNING if shared.opts.cuda_compile_verbose else logging.CRITICAL # pylint: disable=protected-access
if hasattr(torch, '_logging'):
torch._logging.set_logs(dynamo=log_level, aot=log_level, inductor=log_level) # pylint: disable=protected-access
torch._dynamo.config.verbose = shared.opts.cuda_compile_verbose # pylint: disable=protected-access
torch._dynamo.config.suppress_errors = shared.opts.cuda_compile_errors # pylint: disable=protected-access
t0 = time.time()
if shared.opts.cuda_compile:
sd_model.unet = torch.compile(sd_model.unet, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph)
if shared.opts.cuda_compile_vae:
if hasattr(sd_model, 'vae'):
sd_model.vae.decode = torch.compile(sd_model.vae.decode, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph)
if hasattr(sd_model, 'movq'):
sd_model.movq.decode = torch.compile(sd_model.movq.decode, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph)
setup_logging() # compile messes with logging so reset is needed
if shared.opts.cuda_compile_precompile:
sd_model("dummy prompt")
t1 = time.time()
shared.log.info(f"Model compile: time={t1-t0:.2f}")
except Exception as e:
shared.log.warning(f"Model compile error: {e}")
return sd_model
def compile_diffusers(sd_model):
if not (shared.opts.cuda_compile or shared.opts.cuda_compile_vae or shared.opts.cuda_compile_upscaler):
return sd_model
if not hasattr(sd_model, 'unet') or not hasattr(sd_model.unet, 'config'):
shared.log.warning('Model compile enabled but model has no Unet')
return sd_model
if shared.opts.cuda_compile_backend == 'none':
shared.log.warning('Model compile enabled but no backend specified')
return sd_model
size = 8*getattr(sd_model.unet.config, 'sample_size', 0)
shared.log.info(f"Model compile: pipeline={sd_model.__class__.__name__} shape={size} mode={shared.opts.cuda_compile_mode} backend={shared.opts.cuda_compile_backend} fullgraph={shared.opts.cuda_compile_fullgraph} unet={shared.opts.cuda_compile} vae={shared.opts.cuda_compile_vae} upscaler={shared.opts.cuda_compile_upscaler}")
if shared.opts.cuda_compile_backend == 'stable-fast':
sd_model = compile_stablefast(sd_model)
else:
sd_model = compile_torch(sd_model)
return sd_model
modules/sd_vae.py
+10 -2
View File
@@ -91,6 +91,8 @@ def refresh_vae_list():
candidates += glob.iglob(path, recursive=True)
for filepath in candidates:
name = get_filename(filepath)
if name == 'VAE':
continue
if shared.backend == shared.Backend.ORIGINAL:
vae_dict[name] = filepath
else:
@@ -201,10 +203,16 @@ def load_vae_diffusers(model_file, vae_file=None, vae_source="unknown-source"):
if os.path.isfile(vae_file):
_pipeline, model_type = sd_models.detect_pipeline(model_file, 'vae')
diffusers_load_config = { "config_file": paths.sd_default_config if model_type != 'Stable Diffusion XL' else os.path.join(paths.sd_configs_path, 'sd_xl_base.yaml')}
vae = diffusers.AutoencoderKL.from_single_file(vae_file, **diffusers_load_config)
if os.path.getsize(vae_file) > 1310944880:
vae = diffusers.ConsistencyDecoderVAE.from_pretrained('openai/consistency-decoder', **diffusers_load_config) # consistency decoder does not have from single file, so we'll just download it once more
else:
vae = diffusers.AutoencoderKL.from_single_file(vae_file, **diffusers_load_config)
vae = vae.to(devices.dtype_vae)
else:
vae = diffusers.AutoencoderKL.from_pretrained(vae_file, **diffusers_load_config)
if 'consistency-decoder' in vae_file:
vae = diffusers.ConsistencyDecoderVAE.from_pretrained(vae_file, **diffusers_load_config)
else:
vae = diffusers.AutoencoderKL.from_pretrained(vae_file, **diffusers_load_config)
global loaded_vae_file # pylint: disable=global-statement
loaded_vae_file = os.path.basename(vae_file)
# shared.log.debug(f'Diffusers VAE config: {vae.config}')
modules/shared.py
+56 -36
View File
@@ -181,51 +181,69 @@ def temp_disable_extensions():
return disabled
def readfile(filename, silent=False):
def readfile(filename, silent=False, lock=False):
data = {}
lock_file = None
locked = False
try:
if not os.path.exists(filename):
return {}
with fasteners.InterProcessLock(f"{filename}.lock"):
with open(filename, "r", encoding="utf8") as file:
data = json.load(file)
if type(data) is str:
data = json.loads(data)
if not silent:
log.debug(f'Read: file="{filename}" len={len(data)}')
if lock:
lock_file = fasteners.InterProcessReaderWriterLock(f"{filename}.lock", logger=log)
locked = lock_file.acquire_read_lock(blocking=True, timeout=3)
with open(filename, "r", encoding="utf8") as file:
data = json.load(file)
if type(data) is str:
data = json.loads(data)
if not silent:
log.debug(f'Read: file="{filename}" json={len(data)} bytes={os.path.getsize(filename)}')
except Exception as e:
if not silent:
log.error(f'Reading failed: {filename} {e}')
return {}
finally:
if lock_file is not None:
lock_file.release_read_lock()
if locked and os.path.exists(f"{filename}.lock"):
os.remove(f"{filename}.lock")
return data
def writefile(data, filename, mode='w', silent=False):
lock = None
locked = False
def default(obj):
log.error(f"Saving: {filename} not a valid object: {obj}")
return str(obj)
try:
with fasteners.InterProcessLock(f"{filename}.lock"):
# skipkeys=True, ensure_ascii=True, check_circular=True, allow_nan=True
if type(data) == dict:
output = json.dumps(data, indent=2, default=default)
elif type(data) == list:
output = json.dumps(data, indent=2, default=default)
elif isinstance(data, object):
simple = {}
for k in data.__dict__:
if data.__dict__[k] is not None:
simple[k] = data.__dict__[k]
output = json.dumps(simple, indent=2, default=default)
else:
raise ValueError('not a valid object')
if not silent:
log.debug(f'Save: file="{filename}" len={len(output)}')
with open(filename, mode, encoding="utf8") as file:
file.write(output)
# skipkeys=True, ensure_ascii=True, check_circular=True, allow_nan=True
if type(data) == dict:
output = json.dumps(data, indent=2, default=default)
elif type(data) == list:
output = json.dumps(data, indent=2, default=default)
elif isinstance(data, object):
simple = {}
for k in data.__dict__:
if data.__dict__[k] is not None:
simple[k] = data.__dict__[k]
output = json.dumps(simple, indent=2, default=default)
else:
raise ValueError('not a valid object')
lock = fasteners.InterProcessReaderWriterLock(f"{filename}.lock", logger=log)
locked = lock.acquire_write_lock(blocking=True, timeout=3)
with open(filename, mode, encoding="utf8") as file:
file.write(output)
if not silent:
log.debug(f'Save: file="{filename}" json={len(data)} bytes={len(output)}')
except Exception as e:
log.error(f'Saving failed: {filename} {e}')
finally:
if lock is not None:
lock.release_read_lock()
if locked and os.path.exists(f"{filename}.lock"):
os.remove(f"{filename}.lock")
if devices.backend == "cpu":
@@ -282,16 +300,16 @@ options_templates.update(options_section(('cuda', "Compute Settings"), {
"other_sep": OptionInfo("<h2>Execution precision</h2>", "", gr.HTML),
"opt_channelslast": OptionInfo(False, "Use channels last as torch memory format "),
"cudnn_benchmark": OptionInfo(False, "Enable full-depth cuDNN benchmark feature"),
"torch_gc_threshold": OptionInfo(90, "VRAM usage threshold before running Torch GC to clear up VRAM", gr.Slider, {"minimum": 0, "maximum": 100, "step": 1}),
"torch_gc_threshold": OptionInfo(80 if devices.backend == "ipex" else 90, "VRAM usage threshold before running Torch GC to clear up VRAM", gr.Slider, {"minimum": 0, "maximum": 100, "step": 1}),
"cuda_compile_sep": OptionInfo("<h2>Model Compile</h2>", "", gr.HTML),
"cuda_compile": OptionInfo(True if cmd_opts.use_openvino else False, "Compile UNet"),
"cuda_compile_vae": OptionInfo(True if cmd_opts.use_openvino else False, "Compile VAE"),
"cuda_compile_upscaler": OptionInfo(True if cmd_opts.use_openvino else False, "Compile upscaler"),
"cuda_compile_backend": OptionInfo("openvino_fx" if cmd_opts.use_openvino else "none", "Model compile backend", gr.Radio, {"choices": ['none', 'inductor', 'cudagraphs', 'aot_ts_nvfuser', 'hidet', 'ipex', 'openvino_fx']}),
"cuda_compile_mode": OptionInfo("default", "Model compile mode", gr.Radio, {"choices": ['default', 'reduce-overhead', 'max-autotune']}),
"cuda_compile_backend": OptionInfo("openvino_fx" if cmd_opts.use_openvino else "none", "Model compile backend", gr.Radio, {"choices": ['none', 'inductor', 'cudagraphs', 'aot_ts_nvfuser', 'hidet', 'ipex', 'openvino_fx', 'stable-fast']}),
"cuda_compile_mode": OptionInfo("default", "Model compile mode", gr.Radio, {"choices": ['default', 'reduce-overhead', 'max-autotune', 'max-autotune-no-cudagraphs']}),
"cuda_compile_fullgraph": OptionInfo(False, "Model compile fullgraph"),
"cuda_compile_precompile": OptionInfo(False, "Model compile precompile"),
"cuda_compile_precompile": OptionInfo(False if cmd_opts.use_openvino else True, "Model compile precompile"),
"cuda_compile_verbose": OptionInfo(False, "Model compile verbose mode"),
"cuda_compile_errors": OptionInfo(True, "Model compile suppress errors"),
@@ -347,10 +365,9 @@ options_templates.update(options_section(('diffusers', "Diffusers Settings"), {
"diffusers_model_load_variant": OptionInfo("default", "Diffusers model loading variant", gr.Radio, {"choices": ['default', 'fp32', 'fp16']}),
"diffusers_vae_load_variant": OptionInfo("default", "Diffusers VAE loading variant", gr.Radio, {"choices": ['default', 'fp32', 'fp16']}),
"custom_diffusers_pipeline": OptionInfo('', 'Load custom Diffusers pipeline'),
"diffusers_lora_loader": OptionInfo("diffusers" if cmd_opts.use_openvino else "sequential apply", "Diffusers LoRA loading variant", gr.Radio, {"choices": ['diffusers', 'sequential apply', 'merge and apply']}),
"diffusers_eval": OptionInfo(True, "Force model eval"),
"diffusers_force_zeros": OptionInfo(True, "Force zeros for prompts when empty"),
"diffusers_aesthetics_score": OptionInfo(False, "Require aesthetics score"),
"diffusers_force_inpaint": OptionInfo(False, 'Diffusers force inpaint pipeline'),
"diffusers_pooled": OptionInfo("default", "Diffusers SDXL pooled embeds (experimental)", gr.Radio, {"choices": ['default', 'weighted']}),
}))
@@ -359,6 +376,7 @@ options_templates.update(options_section(('system-paths', "System Paths"), {
"models_dir": OptionInfo('models', "Base path where all models are stored", folder=True),
"ckpt_dir": OptionInfo(os.path.join(paths.models_path, 'Stable-diffusion'), "Folder with stable diffusion models", folder=True),
"diffusers_dir": OptionInfo(os.path.join(paths.models_path, 'Diffusers'), "Folder with Hugggingface models", folder=True),
"hfcache_dir": OptionInfo(os.path.join(os.path.expanduser('~'), '.cache', 'huggingface', 'hub'), "Folder for Hugggingface cache", folder=True),
"vae_dir": OptionInfo(os.path.join(paths.models_path, 'VAE'), "Folder with VAE files", folder=True),
"sd_lora": OptionInfo("", "Add LoRA to prompt", gr.Textbox, {"visible": False}),
"lora_dir": OptionInfo(os.path.join(paths.models_path, 'Lora'), "Folder with LoRA network(s)", folder=True),
@@ -401,6 +419,9 @@ options_templates.update(options_section(('saving-images', "Image Options"), {
"grid_save": OptionInfo(True, "Always save all generated image grids"),
"grid_format": OptionInfo('jpg', 'File format for grids', gr.Dropdown, {"choices": ["jpg", "png", "webp", "tiff", "jp2"]}),
"n_rows": OptionInfo(-1, "Grid row count", gr.Slider, {"minimum": -1, "maximum": 16, "step": 1}),
"grid_background": OptionInfo("#000000", "Grid background color", ui_components.FormColorPicker, {}),
"font": OptionInfo("", "Font file"),
"font_color": OptionInfo("#FFFFFF", "Font color", ui_components.FormColorPicker, {}),
"save_sep_options": OptionInfo("<h2>Intermediate Image Saving</h2>", "", gr.HTML),
"save_init_img": OptionInfo(False, "Save copy of img2img init images"),
@@ -452,7 +473,6 @@ options_templates.update(options_section(('ui', "User Interface"), {
"disable_weights_auto_swap": OptionInfo(True, "Do not change selected model when reading generation parameters"),
"send_seed": OptionInfo(True, "Send seed when sending prompt or image to other interface"),
"send_size": OptionInfo(True, "Send size when sending prompt or image to another interface"),
"font": OptionInfo("", "Font for image grids that have text"),
"keyedit_precision_attention": OptionInfo(0.1, "Ctrl+up/down precision when editing (attention:1.1)", gr.Slider, {"minimum": 0.01, "maximum": 0.2, "step": 0.001, "visible": False}),
"keyedit_precision_extra": OptionInfo(0.05, "Ctrl+up/down precision when editing <extra networks:0.9>", gr.Slider, {"minimum": 0.01, "maximum": 0.2, "step": 0.001, "visible": False}),
"keyedit_delimiters": OptionInfo(".,\/!?%^*;:{}=`~()", "Ctrl+up/down word delimiters", gr.Textbox, { "visible": False }), # pylint: disable=anomalous-backslash-in-string
@@ -510,9 +530,9 @@ options_templates.update(options_section(('sampler-params', "Sampler Settings"),
'uni_pc_variant': OptionInfo("bh1", "UniPC variant", gr.Radio, {"choices": ["bh1", "bh2", "vary_coeff"]}),
'uni_pc_skip_type': OptionInfo("time_uniform", "UniPC skip type", gr.Radio, {"choices": ["time_uniform", "time_quadratic", "logSNR"]}),
"ddim_discretize": OptionInfo('uniform', "DDIM discretize img2img", gr.Radio, {"choices": ['uniform', 'quad']}),
# TODO pad_cond_uncond implementation missing
# TODO pad_cond_uncond implementation missing for original backend
"pad_cond_uncond": OptionInfo(True, "Pad prompt and negative prompt to be same length", gr.Checkbox, {"visible": False}),
# TODO batch_cond-uncond implementation missing
# TODO batch_cond-uncond implementation missing for original backend
"batch_cond_uncond": OptionInfo(True, "Do conditional and unconditional denoising in one batch", gr.Checkbox, {"visible": False}),
}))
@@ -699,7 +719,7 @@ class Options:
log.debug(f'Created default config: {filename}')
self.save(filename)
return
self.data = readfile(filename)
self.data = readfile(filename, lock=True)
if self.data.get('quicksettings') is not None and self.data.get('quicksettings_list') is None:
self.data['quicksettings_list'] = [i.strip() for i in self.data.get('quicksettings').split(',')]
unknown_settings = []
modules/shared_items.py
+1
View File
@@ -43,6 +43,7 @@ def get_pipelines():
'Wuerstchen': getattr(diffusers, 'WuerstchenCombinedPipeline', None),
'Kandinsky 2.1': getattr(diffusers, 'KandinskyPipeline', None),
'Kandinsky 2.2': getattr(diffusers, 'KandinskyV22Pipeline', None),
# 'Kandinsky 3': getattr(diffusers, 'KandinskyV3Pipeline', None),
'DeepFloyd IF': getattr(diffusers, 'IFPipeline', None),
'Custom Diffusers Pipeline': getattr(diffusers, 'DiffusionPipeline', None),
# Segmind SSD-1B, Segmind Tiny
modules/theme.py
+1 -1
View File
@@ -64,7 +64,7 @@ def reload_gradio_theme(theme_name=None):
'font_mono':['IBM Plex Mono', 'ui-monospace', 'Consolas', 'monospace']
}
is_builtin = theme_name in list_builtin_themes()
modules.shared.log.info(f'Load UI theme: name="{theme_name}" style={modules.shared.opts.theme_style} base={"style.css" if is_builtin else "base.css"}')
modules.shared.log.info(f'Load UI theme: name="{theme_name}" style={modules.shared.opts.theme_style} base={"sdnext.css" if is_builtin else "base.css"}')
if is_builtin:
gradio_theme = gr.themes.Base(**default_font_params)
elif theme_name.startswith("gradio/"):
modules/ui.py
+4 -4
View File
@@ -421,7 +421,7 @@ def create_ui(startup_timer = None):
cfg_scale = gr.Slider(minimum=1.0, maximum=30.0, step=0.1, label='CFG scale', value=6.0, elem_id="txt2img_cfg_scale")
clip_skip = gr.Slider(label='CLIP skip', value=1, minimum=1, maximum=14, step=1, elem_id='txt2img_clip_skip', interactive=True)
with FormRow():
image_cfg_scale = gr.Slider(minimum=1.1, maximum=30.0, step=0.1, label='Secondary CFG scale', value=6.0, elem_id="txt2img_image_cfg_scale")
image_cfg_scale = gr.Slider(minimum=1.0, maximum=30.0, step=0.1, label='Secondary CFG scale', value=6.0, elem_id="txt2img_image_cfg_scale")
diffusers_guidance_rescale = gr.Slider(minimum=0.0, maximum=1.0, step=0.05, label='Guidance rescale', value=0.7, elem_id="txt2img_image_cfg_rescale")
with FormRow():
full_quality = gr.Checkbox(label='Full quality', value=True, elem_id="txt2img_full_quality")
@@ -711,8 +711,8 @@ def create_ui(startup_timer = None):
with gr.Accordion(open=False, label="Advanced", elem_classes=["small-accordion"], elem_id="img2img_advanced_group"):
with FormRow():
cfg_scale = gr.Slider(minimum=1.0, maximum=30.0, step=0.5, label='CFG scale', value=6.0, elem_id="img2img_cfg_scale")
image_cfg_scale = gr.Slider(minimum=0, maximum=30.0, step=0.05, label='Image CFG scale', value=1.5, elem_id="img2img_image_cfg_scale")
cfg_scale = gr.Slider(minimum=1.0, maximum=30.0, step=0.1, label='CFG scale', value=6.0, elem_id="img2img_cfg_scale")
image_cfg_scale = gr.Slider(minimum=1.0, maximum=30.0, step=0.15, label='Image CFG scale', value=1.5, elem_id="img2img_image_cfg_scale")
with FormRow():
clip_skip = gr.Slider(label='CLIP skip', value=1, minimum=1, maximum=4, step=1, elem_id='img2img_clip_skip', interactive=True)
diffusers_guidance_rescale = gr.Slider(minimum=0.0, maximum=1.0, step=0.05, label='Guidance rescale', value=0.7, elem_id="txt2img_image_cfg_rescale")
@@ -729,7 +729,7 @@ def create_ui(startup_timer = None):
with gr.Column():
inpainting_mask_invert = gr.Radio(label='Mask mode', choices=['Inpaint masked', 'Inpaint not masked'], value='Inpaint masked', type="index", elem_id="img2img_mask_mode")
with gr.Column():
inpainting_fill = gr.Radio(label='Masked content', choices=['fill', 'original', 'latent noise', 'latent nothing'], value='original', type="index", elem_id="img2img_inpainting_fill")
inpainting_fill = gr.Radio(label='Masked content', choices=['fill', 'original', 'noise', 'nothing'], value='original', type="index", elem_id="img2img_inpainting_fill")
with FormRow():
with gr.Column():
inpaint_full_res = gr.Radio(label="Inpaint area", choices=["Whole picture", "Only masked"], type="index", value="Whole picture", elem_id="img2img_inpaint_full_res")
modules/ui_common.py
+16 -9
View File
@@ -6,7 +6,7 @@ import platform
import subprocess
import gradio as gr
from modules import call_queue, shared
from modules.generation_parameters_copypaste import image_from_url_text
from modules.generation_parameters_copypaste import image_from_url_text, parse_generation_parameters
import modules.ui_symbols as symbols
import modules.images
import modules.script_callbacks
@@ -34,12 +34,19 @@ def plaintext_to_html(text):
def infotext_to_html(text):
res = '<p class="html_info">Prompt: ' + html.escape(text or '').replace('\n', '<br>') + '</p>'
sections = res.split('Steps:') # before and after prompt+negprompt'
if len(sections) > 1:
res = sections[0] + '<br>Steps: ' + sections[1].strip().replace(', ', ' | ')
res = res.replace('<br><br>', '<br>')
return res
res = parse_generation_parameters(text)
prompt = res.get('Prompt', None)
negative = res.get('Negative prompt', None)
res.pop('Prompt', None)
res.pop('Negative prompt', None)
params = [f'{k}: {v}' for k, v in res.items() if v is not None]
params = '| '.join(params)
code = f'''
<p><b>Prompt:</b> {prompt}</p>
<p><b>Negative:</b> {negative}</p>
<p><b>Parameters:</b> {params}</p>
'''
return code
def delete_files(js_data, images, _html_info, index):
@@ -232,7 +239,7 @@ def create_output_panel(tabname):
return result_gallery, generation_info, html_info, html_info_formatted, html_log
def create_refresh_button(refresh_component, refresh_method, refreshed_args, elem_id):
def create_refresh_button(refresh_component, refresh_method, refreshed_args, elem_id, visible: bool = True):
def refresh():
refresh_method()
@@ -242,7 +249,7 @@ def create_refresh_button(refresh_component, refresh_method, refreshed_args, ele
return gr.update(**(args or {}))
from modules.ui_components import ToolButton
refresh_button = ToolButton(value=symbols.refresh, elem_id=elem_id)
refresh_button = ToolButton(value=symbols.refresh, elem_id=elem_id, visible=visible)
refresh_button.click(fn=refresh, inputs=[], outputs=[refresh_component])
return refresh_button
modules/ui_extra_networks.py
+52 -42
View File
@@ -60,17 +60,6 @@ def listdir(path):
return dir_cache[path][1]
def register_page(page):
# registers extra networks page for the UI; recommend doing it in on_before_ui() callback for extensions
debug(f'EN register-page: {page}')
shared.extra_networks.append(page)
allowed_dirs.clear()
for pg in shared.extra_networks:
for folder in pg.allowed_directories_for_previews():
if folder not in allowed_dirs:
allowed_dirs.append(os.path.abspath(folder))
def init_api(app):
def fetch_file(filename: str = ""):
@@ -203,7 +192,7 @@ class ExtraNetworksPage:
os.remove(f)
elif img.width > 1024 or img.height > 1024 or os.path.getsize(f) > 65536:
img = img.convert('RGB')
img.thumbnail((512, 512), Image.HAMMING)
img.thumbnail((512, 512), Image.Resampling.HAMMING)
img.save(fn, quality=50)
img.close()
created += 1
@@ -253,8 +242,10 @@ class ExtraNetworksPage:
subdir = tgt[len(parentdir):].replace("\\", "/")
while subdir.startswith("/"):
subdir = subdir[1:]
if not self.is_empty(tgt):
# if not self.is_empty(tgt):
if not subdir.startswith("."):
subdirs[subdir] = 1
debug(f"Extra networks: page='{self.name}' subfolders={list(subdirs)}")
subdirs = OrderedDict(sorted(subdirs.items()))
if shared.backend == shared.Backend.DIFFUSERS and self.name == 'model':
subdirs['Reference'] = 1
@@ -279,7 +270,7 @@ class ExtraNetworksPage:
self.html = f"<div id='{tabname}_{self_name_id}_subdirs' class='extra-network-subdirs'>{subdirs_html}</div><div id='{tabname}_{self_name_id}_cards' class='extra-network-cards'>{self.html}</div>"
else:
return ''
shared.log.debug(f"Extra networks: page='{self.name}' items={len(self.items)} subdirs={len(subdirs)} tab={tabname} dirs={self.allowed_directories_for_previews()} list={self.list_time:.2f} desc={self.desc_time:.2f} info={self.info_time:.2f}")
shared.log.debug(f"Extra networks: page='{self.name}' items={len(self.items)} subfolders={len(subdirs)} tab={tabname} folders={self.allowed_directories_for_previews()} list={self.list_time:.2f} desc={self.desc_time:.2f} info={self.info_time:.2f}")
if len(self.missing_thumbs) > 0:
threading.Thread(target=self.create_thumb).start()
return self.html
@@ -329,10 +320,10 @@ class ExtraNetworksPage:
else:
files = listdir(os.path.dirname(path))
fn = os.path.splitext(path)[0]
preview_extensions = ["jpg", "jpeg", "png", "webp", "tiff", "jp2"]
for file in [f'{fn}{mid}{ext}' for ext in preview_extensions for mid in ['.thumb.', '.', '.preview.']]:
exts = ["jpg", "jpeg", "png", "webp", "tiff", "jp2"]
for file in [f'{fn}{mid}{ext}' for ext in exts for mid in ['.thumb.', '.', '.preview.']]:
if file in files:
if '.thumb.' not in file:
if 'Reference' not in file and '.thumb.' not in file:
self.missing_thumbs.append(file)
return file
return 'html/card-no-preview.png'
@@ -388,6 +379,17 @@ def initialize():
shared.extra_networks.clear()
def register_page(page: ExtraNetworksPage):
# registers extra networks page for the UI; recommend doing it in on_before_ui() callback for extensions
debug(f'EN register-page: {page}')
shared.extra_networks.append(page)
allowed_dirs.clear()
for pg in shared.extra_networks:
for folder in pg.allowed_directories_for_previews():
if folder not in allowed_dirs:
allowed_dirs.append(os.path.abspath(folder))
def register_pages():
from modules.ui_extra_networks_textual_inversion import ExtraNetworksPageTextualInversion
from modules.ui_extra_networks_hypernets import ExtraNetworksPageHypernetworks
@@ -401,18 +403,25 @@ def register_pages():
register_page(ExtraNetworksPageVAEs())
def get_pages():
def get_pages(title=None):
pages = []
if 'All' in shared.opts.extra_networks:
pages = shared.extra_networks
else:
titles = [page.title for page in shared.extra_networks]
for page in shared.opts.extra_networks:
if title is None:
for page in shared.opts.extra_networks:
try:
idx = titles.index(page)
pages.append(shared.extra_networks[idx])
except ValueError:
continue
else:
try:
idx = titles.index(page)
idx = titles.index(title)
pages.append(shared.extra_networks[idx])
except ValueError:
continue
pages.append(shared.extra_networks[idx])
pass
return pages
@@ -440,6 +449,7 @@ class ExtraNetworksUi:
self.details_components: list = []
self.last_item: dict = None
self.last_page: ExtraNetworksPage = None
self.state: gr.State = None
def create_ui(container, button_parent, tabname, skip_indexing = False):
@@ -505,18 +515,18 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
desc = gr.Textbox('', show_label=False, lines=8, placeholder="Extra network description...")
ui.details_components.append(desc)
with gr.Row():
btn_save_desc = gr.Button('Save', elem_classes=['small-button'])
btn_delete_desc = gr.Button('Delete', elem_classes=['small-button'])
btn_close_info = gr.Button('Close', elem_classes=['small-button'])
btn_close_info.click(fn=lambda: gr.update(visible=False), inputs=[], outputs=[ui.details])
btn_save_desc = gr.Button('Save', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_save_desc')
btn_delete_desc = gr.Button('Delete', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_delete_desc')
btn_close_desc = gr.Button('Close', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_close_desc')
btn_close_desc.click(fn=lambda: gr.update(visible=False), _js='refeshDetailsEN', inputs=[], outputs=[ui.details])
with gr.Tab('Model metadata'):
info = gr.JSON({}, show_label=False)
ui.details_components.append(info)
with gr.Row():
btn_save_info = gr.Button('Save', elem_classes=['small-button'])
btn_delete_info = gr.Button('Delete', elem_classes=['small-button'])
btn_close_info = gr.Button('Close', elem_classes=['small-button'])
btn_close_info.click(fn=lambda: gr.update(visible=False), inputs=[], outputs=[ui.details])
btn_save_info = gr.Button('Save', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_save_info')
btn_delete_info = gr.Button('Delete', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_delete_info')
btn_close_info = gr.Button('Close', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_close_info')
btn_close_info.click(fn=lambda: gr.update(visible=False), _js='refeshDetailsEN', inputs=[], outputs=[ui.details])
with gr.Tab('Embedded metadata'):
meta = gr.JSON({}, show_label=False)
ui.details_components.append(meta)
@@ -559,7 +569,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
tab.select(ui_tab_change, _js="getENActivePage", inputs=[ui.button_details], outputs=[ui.button_scan, ui.button_save, ui.button_model])
# ui.tabs.change(fn=ui_tab_change, inputs=[], outputs=[ui.button_scan, ui.button_save])
def fn_save_img():
def fn_save_img(image):
if ui.last_item is None or ui.last_item.local_preview is None:
return 'html/card-no-preview.png'
images = list(ui.gallery.temp_files) # gallery cannot be used as input component so looking at most recently registered temp files
@@ -572,10 +582,10 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
except Exception as e:
shared.log.error(f'Extra network error opening image: item={ui.last_item.name} {e}')
return 'html/card-no-preview.png'
fn_delete_img()
fn_delete_img(image)
if image.width > 512 or image.height > 512:
image = image.convert('RGB')
image.thumbnail((512, 512), Image.HAMMING)
image.thumbnail((512, 512), Image.Resampling.HAMMING)
try:
image.save(ui.last_item.local_preview, quality=50)
shared.log.debug(f'Extra network save image: item={ui.last_item.name} filename="{ui.last_item.local_preview}"')
@@ -583,7 +593,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
shared.log.error(f'Extra network save image: item={ui.last_item.name} filename="{ui.last_item.local_preview}" {e}')
return image
def fn_delete_img():
def fn_delete_img(_image):
preview_extensions = ["jpg", "jpeg", "png", "webp", "tiff", "jp2"]
fn = os.path.splitext(ui.last_item.filename)[0]
for file in [f'{fn}{mid}{ext}' for ext in preview_extensions for mid in ['.thumb.', '.preview.', '.']]:
@@ -633,12 +643,12 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
return ''
return info
btn_save_img.click(fn=fn_save_img, inputs=[], outputs=[img])
btn_delete_img.click(fn=fn_delete_img, inputs=[], outputs=[img])
btn_save_desc.click(fn=fn_save_desc, inputs=[desc], outputs=[desc])
btn_delete_desc.click(fn=fn_delete_desc, inputs=[desc], outputs=[desc])
btn_save_info.click(fn=fn_save_info, inputs=[info], outputs=[info])
btn_delete_info.click(fn=fn_delete_info, inputs=[info], outputs=[info])
btn_save_img.click(fn=fn_save_img, _js='closeDetailsEN', inputs=[img], outputs=[img])
btn_delete_img.click(fn=fn_delete_img, _js='closeDetailsEN', inputs=[img], outputs=[img])
btn_save_desc.click(fn=fn_save_desc, _js='closeDetailsEN', inputs=[desc], outputs=[desc])
btn_delete_desc.click(fn=fn_delete_desc, _js='closeDetailsEN', inputs=[desc], outputs=[desc])
btn_save_info.click(fn=fn_save_info, _js='closeDetailsEN', inputs=[info], outputs=[info])
btn_delete_info.click(fn=fn_delete_info, _js='closeDetailsEN', inputs=[info], outputs=[info])
def show_details(text, img, desc, info, meta, params):
page, item = get_item(state, params)
@@ -797,7 +807,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
shared.log.debug(f'Extra networks: {msg}')
return msg
dummy_state = gr.State(value=False) # pylint: disable=abstract-class-instantiated
dummy = gr.State(value=False) # pylint: disable=abstract-class-instantiated
button_parent.click(fn=toggle_visibility, inputs=[ui.visible], outputs=[ui.visible, container, button_parent])
ui.button_close.click(fn=toggle_visibility, inputs=[ui.visible], outputs=[ui.visible, container])
ui.button_sort.click(fn=ui_sort_cards, _js='sortExtraNetworks', inputs=[ui.search], outputs=[ui.description])
@@ -806,7 +816,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
ui.button_scan.click(fn=ui_scan_click, _js='getENActivePage', inputs=[ui.search], outputs=ui.pages)
ui.button_save.click(fn=ui_save_click, inputs=[], outputs=ui.details_components + [ui.details])
ui.button_quicksave.click(fn=ui_quicksave_click, _js="() => prompt('Prompt name', '')", inputs=[ui.search], outputs=[])
ui.button_details.click(show_details, _js="getCardDetails", inputs=ui.details_components + [dummy_state], outputs=ui.details_components + [ui.details])
ui.button_details.click(show_details, _js="getCardDetails", inputs=ui.details_components + [dummy], outputs=ui.details_components + [ui.details])
ui.state.change(state_change, inputs=[ui.state], outputs=[])
return ui
modules/ui_models.py
+360 -79
View File
@@ -1,15 +1,18 @@
import os
import json
import inspect
from datetime import datetime
import gradio as gr
from modules import sd_models, sd_vae, extras
from modules.ui_components import FormRow, ToolButton
from modules.ui_common import create_refresh_button
from modules.call_queue import wrap_gradio_gpu_call
from modules.shared import opts, log, req
from modules.shared import opts, log, req, readfile
import modules.errors
import modules.hashes
from modules.merging import merge_methods
from modules.merging.merge_utils import BETA_METHODS, TRIPLE_METHODS, interpolate
from modules.merging.merge_presets import BLOCK_WEIGHTS_PRESETS, SDXL_BLOCK_WEIGHTS_PRESETS
search_metadata_civit = None
@@ -34,7 +37,7 @@ def create_ui():
model_name = gr.Dropdown(sd_models.checkpoint_tiles(), label="Original model")
create_refresh_button(model_name, sd_models.list_models, lambda: {"choices": sd_models.checkpoint_tiles()}, "refresh_checkpoint_Z")
with gr.Row():
custom_name = gr.Textbox(label="New model name")
custom_name = gr.Textbox(label="Output model name")
with gr.Row():
precision = gr.Radio(choices=["fp32", "fp16", "bf16"], value="fp16", label="Model precision")
m_type = gr.Radio(choices=["disabled", "no-ema", "ema-only"], value="disabled", label="Model pruning methods")
@@ -69,64 +72,233 @@ def create_ui():
)
with gr.Tab(label="Merge"):
def sd_model_choices():
return ['None'] + sd_models.checkpoint_tiles()
with gr.Row(equal_height=False):
with gr.Column(variant='compact'):
with FormRow():
custom_name = gr.Textbox(label="New model name")
with FormRow():
def sd_model_choices():
return ['None'] + sd_models.checkpoint_tiles()
merge_mode = gr.Dropdown(choices=merge_methods.__all__, value="weighted_sum", label="Interpolation Method")
merge_mode_docs = gr.HTML(value=getattr(merge_methods, "weighted_sum").__doc__.replace("\n", "<br>"))
with FormRow():
primary_model_name = gr.Dropdown(sd_model_choices(), label="Primary model", value="None")
create_refresh_button(primary_model_name, sd_models.list_models, lambda: {"choices": sd_model_choices()}, "refresh_checkpoint_A")
secondary_model_name = gr.Dropdown(sd_model_choices(), label="Secondary model", value="None")
create_refresh_button(secondary_model_name, sd_models.list_models, lambda: {"choices": sd_model_choices()}, "refresh_checkpoint_B")
tertiary_model_name = gr.Dropdown(sd_model_choices(), label="Tertiary model", value="None")
create_refresh_button(tertiary_model_name, sd_models.list_models, lambda: {"choices": sd_model_choices()}, "refresh_checkpoint_C")
tertiary_model_name = gr.Dropdown(sd_model_choices(), label="Tertiary model", value="None", visible=False)
tertiary_refresh = create_refresh_button(tertiary_model_name, sd_models.list_models, lambda: {"choices": sd_model_choices()}, "refresh_checkpoint_C", visible=False)
with FormRow():
interp_method = gr.Radio(choices=["No interpolation", "Weighted sum", "Add difference"], value="Weighted sum", label="Interpolation Method")
interp_amount = gr.Slider(minimum=0.0, maximum=1.0, step=0.05, label='Interpolation ratio from Primary to Secondary', value=0.5)
with gr.Tabs() as tabs:
with gr.TabItem(label="Simple Merge", id=0):
with FormRow():
alpha = gr.Slider(minimum=0.0, maximum=1.0, step=0.05, label='Alpha Ratio', value=0.5)
beta = gr.Slider(minimum=0.0, maximum=1.0, step=0.05, label='Beta Ratio', value=None, visible=False)
with gr.TabItem(label="Preset Block Merge", id=1):
with FormRow():
sdxl = gr.Checkbox(label="SDXL")
with FormRow():
alpha_preset = gr.Dropdown(
choices=["None"] + list(BLOCK_WEIGHTS_PRESETS.keys()), value=None,
label="ALPHA Block Weight Preset", multiselect=True, max_choices=2)
alpha_preset_lambda = gr.Slider(minimum=0.0, maximum=1.0, step=0.05, label='Preset Interpolation Ratio', value=None, visible=False)
apply_preset = ToolButton('⇨', visible=True)
with FormRow():
beta_preset = gr.Dropdown(choices=["None"] + list(BLOCK_WEIGHTS_PRESETS.keys()), value=None, label="BETA Block Weight Preset", multiselect=True, max_choices=2, interactive=True, visible=False)
beta_preset_lambda = gr.Slider(minimum=0.0, maximum=1.0, step=0.05, label='Preset Interpolation Ratio', value=None, interactive=True, visible=False)
beta_apply_preset = ToolButton('⇨', interactive=True, visible=False)
with gr.TabItem(label="Manual Block Merge", id=2):
with FormRow():
alpha_label = gr.Markdown("# Alpha")
with FormRow():
alpha_base = gr.Textbox(value=None, label="Base", min_width=70, scale=1)
alpha_in_blocks = gr.Textbox(value=None, label="In Blocks", scale=15)
alpha_mid_block = gr.Textbox(value=None, label="Mid Block", min_width=80, scale=1)
alpha_out_blocks = gr.Textbox(value=None, label="Out Block", scale=15)
with FormRow():
beta_label = gr.Markdown("# Beta", visible=False)
with FormRow():
beta_base = gr.Textbox(value=None, label="Base", min_width=70, scale=1, interactive=True, visible=False)
beta_in_blocks = gr.Textbox(value=None, label="In Blocks", interactive=True, scale=15, visible=False)
beta_mid_block = gr.Textbox(value=None, label="Mid Block", min_width=80, interactive=True, scale=1, visible=False)
beta_out_blocks = gr.Textbox(value=None, label="Out Block", interactive=True, scale=15, visible=False)
with FormRow():
checkpoint_format = gr.Radio(choices=["ckpt", "safetensors"], value="safetensors", label="Model format")
with gr.Box():
save_as_half = gr.Radio(choices=["fp16", "fp32"], value="fp16", label="Model precision", type="index")
with FormRow():
config_source = gr.Radio(choices=["Primary", "Secondary", "Tertiary", "None"], value="Primary", label="Model configuration", type="index")
with FormRow():
bake_in_vae = gr.Dropdown(choices=["None"] + list(sd_vae.vae_dict), value="None", label="Bake in VAE")
create_refresh_button(bake_in_vae, sd_vae.refresh_vae_list, lambda: {"choices": ["None"] + list(sd_vae.vae_dict)}, "modelmerger_refresh_bake_in_vae")
with FormRow():
discard_weights = gr.Textbox(value="", label="Discard weights with matching name")
overwrite = gr.Checkbox(label="Overwrite model")
with FormRow():
save_metadata = gr.Checkbox(value=True, label="Save metadata")
with FormRow():
weights_clip = gr.Checkbox(label="Weights clip")
prune = gr.Checkbox(label="Prune", value=True, visible=False)
with FormRow():
re_basin = gr.Checkbox(label="ReBasin")
re_basin_iterations = gr.Slider(minimum=0, maximum=25, step=1, label='Number of ReBasin Iterations', value=None, visible=False)
with FormRow():
checkpoint_format = gr.Radio(choices=["ckpt", "safetensors"], value="safetensors", visible=False, label="Model format")
with FormRow():
precision = gr.Radio(choices=["fp16", "fp32"], value="fp16", label="Model precision")
with FormRow():
device = gr.Radio(choices=["cpu", "shuffle", "gpu"], value="cpu", label="Merge Device")
unload = gr.Checkbox(label="Unload Current Model from VRAM", value=False, visible=False)
with FormRow():
bake_in_vae = gr.Dropdown(choices=["None"] + list(sd_vae.vae_dict), value="None", interactive=True, label="Replace VAE")
create_refresh_button(bake_in_vae, sd_vae.refresh_vae_list,
lambda: {"choices": ["None"] + list(sd_vae.vae_dict)},
"modelmerger_refresh_bake_in_vae")
with gr.Row():
modelmerger_merge = gr.Button(value="Merge", variant='primary')
def modelmerger(*args):
try:
results = extras.run_modelmerger(*args)
except Exception as e:
modules.errors.display(e, 'model merge')
sd_models.list_models() # to remove the potentially missing models from the list
return [*[gr.Dropdown.update(choices=sd_models.checkpoint_tiles()) for _ in range(4)], f"Error merging checkpoints: {e}"]
return results
def modelmerger(dummy_component, # dummy function just to get argspec later
overwrite, # pylint: disable=unused-argument
primary_model_name, # pylint: disable=unused-argument
secondary_model_name, # pylint: disable=unused-argument
tertiary_model_name, # pylint: disable=unused-argument
merge_mode, # pylint: disable=unused-argument
alpha, # pylint: disable=unused-argument
beta, # pylint: disable=unused-argument
alpha_preset, # pylint: disable=unused-argument
alpha_preset_lambda, # pylint: disable=unused-argument
alpha_base, # pylint: disable=unused-argument
alpha_in_blocks, # pylint: disable=unused-argument
alpha_mid_block, # pylint: disable=unused-argument
alpha_out_blocks, # pylint: disable=unused-argument
beta_preset, # pylint: disable=unused-argument
beta_preset_lambda, # pylint: disable=unused-argument
beta_base, # pylint: disable=unused-argument
beta_in_blocks, # pylint: disable=unused-argument
beta_mid_block, # pylint: disable=unused-argument
beta_out_blocks, # pylint: disable=unused-argument
precision, # pylint: disable=unused-argument
custom_name, # pylint: disable=unused-argument
checkpoint_format, # pylint: disable=unused-argument
save_metadata, # pylint: disable=unused-argument
weights_clip, # pylint: disable=unused-argument
prune, # pylint: disable=unused-argument
re_basin, # pylint: disable=unused-argument
re_basin_iterations, # pylint: disable=unused-argument
device, # pylint: disable=unused-argument
unload, # pylint: disable=unused-argument
bake_in_vae): # pylint: disable=unused-argument
kwargs = {}
for x in inspect.getfullargspec(modelmerger)[0]:
kwargs[x] = locals()[x]
for key in list(kwargs.keys()):
if kwargs[key] in [None, "None", "", 0, []]:
del kwargs[key]
del kwargs['dummy_component']
if kwargs.get("custom_name", None) is None:
log.error('Merge: no output model specified')
return [*[gr.Dropdown.update(choices=sd_models.checkpoint_tiles()) for _ in range(4)], "No output model specified"]
elif kwargs.get("primary_model_name", None) is None or kwargs.get("secondary_model_name", None) is None:
log.error('Merge: no models selected')
return [*[gr.Dropdown.update(choices=sd_models.checkpoint_tiles()) for _ in range(4)], "No models selected"]
else:
log.debug(f'Merge start: {kwargs}')
try:
results = extras.run_modelmerger(dummy_component, **kwargs)
except Exception as e:
modules.errors.display(e, 'Merge')
sd_models.list_models() # to remove the potentially missing models from the list
return [*[gr.Dropdown.update(choices=sd_models.checkpoint_tiles()) for _ in range(4)], f"Error merging checkpoints: {e}"]
return results
def tertiary(mode):
if mode in TRIPLE_METHODS:
return [gr.update(visible=True) for _ in range(2)]
else:
return [gr.update(visible=False) for _ in range(2)]
def beta_visibility(mode):
if mode in BETA_METHODS:
return [gr.update(visible=True) for _ in range(9)]
else:
return [gr.update(visible=False) for _ in range(9)]
def show_iters(show):
if show:
return gr.Slider.update(value=5, visible=True)
else:
return gr.Slider.update(value=None, visible=False)
def show_help(mode):
doc = getattr(merge_methods, mode).__doc__.replace("\n", "<br>")
return gr.update(value=doc, visible=True)
def show_unload(device):
if device == "gpu":
return gr.update(visible=True)
else:
return gr.update(visible=False)
def preset_visiblility(x):
if len(x) == 2:
return gr.Slider.update(value=0.5, visible=True)
else:
return gr.Slider.update(value=None, visible=False)
def load_presets(presets, ratio):
for i, p in enumerate(presets):
presets[i] = BLOCK_WEIGHTS_PRESETS[p]
if len(presets) == 2:
preset = interpolate(presets, ratio)
else:
preset = presets[0]
preset = ['%.3f' % x if int(x) != x else str(x) for x in preset] # pylint: disable=consider-using-f-string
preset = [preset[0], ",".join(preset[1:13]), preset[13], ",".join(preset[14:])]
return [gr.update(value=x) for x in preset] + [gr.update(selected=2)]
def preset_choices(sdxl):
if sdxl:
return [gr.update(choices=["None"] + list(SDXL_BLOCK_WEIGHTS_PRESETS.keys())) for _ in range(2)]
else:
return [gr.update(choices=["None"] + list(BLOCK_WEIGHTS_PRESETS.keys())) for _ in range(2)]
device.change(fn=show_unload, inputs=device, outputs=unload)
merge_mode.change(fn=show_help, inputs=merge_mode, outputs=merge_mode_docs)
sdxl.change(fn=preset_choices, inputs=sdxl, outputs=[alpha_preset, beta_preset])
alpha_preset.change(fn=preset_visiblility, inputs=alpha_preset, outputs=alpha_preset_lambda)
beta_preset.change(fn=preset_visiblility, inputs=alpha_preset, outputs=beta_preset_lambda)
merge_mode.input(fn=tertiary, inputs=merge_mode, outputs=[tertiary_model_name, tertiary_refresh])
merge_mode.input(fn=beta_visibility, inputs=merge_mode, outputs=[beta, alpha_label, beta_label, beta_apply_preset, beta_preset, beta_base, beta_in_blocks, beta_mid_block, beta_out_blocks])
re_basin.change(fn=show_iters, inputs=re_basin, outputs=re_basin_iterations)
apply_preset.click(fn=load_presets, inputs=[alpha_preset, alpha_preset_lambda], outputs=[alpha_base, alpha_in_blocks, alpha_mid_block, alpha_out_blocks, tabs])
beta_apply_preset.click(fn=load_presets, inputs=[beta_preset, beta_preset_lambda], outputs=[beta_base, beta_in_blocks, beta_mid_block, beta_out_blocks, tabs])
modelmerger_merge.click(
fn=wrap_gradio_gpu_call(modelmerger, extra_outputs=lambda: [gr.update() for _ in range(4)]),
_js='modelmerger',
inputs=[
dummy_component,
overwrite,
primary_model_name,
secondary_model_name,
tertiary_model_name,
interp_method,
interp_amount,
save_as_half,
merge_mode,
alpha,
beta,
alpha_preset,
alpha_preset_lambda,
alpha_base,
alpha_in_blocks,
alpha_mid_block,
alpha_out_blocks,
beta_preset,
beta_preset_lambda,
beta_base,
beta_in_blocks,
beta_mid_block,
beta_out_blocks,
precision,
custom_name,
checkpoint_format,
config_source,
bake_in_vae,
discard_weights,
save_metadata,
weights_clip,
prune,
re_basin,
re_basin_iterations,
device,
unload,
bake_in_vae,
],
outputs=[
primary_model_name,
@@ -147,14 +319,14 @@ def create_ui():
model_checkhash_btn.click(fn=sd_models.update_model_hashes, inputs=[], outputs=[models_outcome])
with gr.Row():
model_table = gr.DataFrame(
value = None,
headers = model_headers,
label = 'Model data',
show_label = True,
interactive = False,
wrap = True,
overflow_row_behaviour = 'paginate',
max_rows = 50,
value=None,
headers=model_headers,
label='Model data',
show_label=True,
interactive=False,
wrap=True,
overflow_row_behaviour='paginate',
max_rows=50,
)
def list_models():
@@ -188,11 +360,7 @@ def create_ui():
def hf_search(keyword):
import huggingface_hub as hf
hf_api = hf.HfApi()
model_filter = hf.ModelFilter(
model_name=keyword,
# task='text-to-image',
library=['diffusers'],
)
model_filter = hf.ModelFilter(model_name=keyword, library=['diffusers'])
models = hf_api.list_models(filter=model_filter, full=True, limit=50, sort="downloads", direction=-1)
data.clear()
for model in models:
@@ -206,7 +374,7 @@ def create_ui():
def hf_download_model(hub_id: str, token, variant, revision, mirror, custom_pipeline):
from modules.modelloader import download_diffusers_model
download_diffusers_model(hub_id, cache_dir=opts.diffusers_dir, token=token, variant=variant, revision=revision, mirror=mirror, custom_pipeline=custom_pipeline)
from modules.sd_models import list_models # pylint: disable=W0621
from modules.sd_models import list_models # pylint: disable=W0621
list_models()
log.info(f'Diffuser model downloaded: model="{hub_id}"')
return f'Diffuser model downloaded: model="{hub_id}"'
@@ -214,20 +382,20 @@ def create_ui():
with gr.Column(scale=6):
gr.HTML('<h2>Search for models</h2>Select a model from the search results to download<br><br>')
with gr.Row():
hf_search_text = gr.Textbox('', label = 'Search models', placeholder='search huggingface models')
hf_search_text = gr.Textbox('', label='Search models', placeholder='search huggingface models')
hf_search_btn = ToolButton(value="🔍", label="Search")
with gr.Row():
with gr.Column(scale=2):
with gr.Row():
hf_selected = gr.Textbox('', label = 'Select model', placeholder='select model from search results or enter model name manually')
hf_selected = gr.Textbox('', label='Select model', placeholder='select model from search results or enter model name manually')
with gr.Column(scale=1):
with gr.Row():
hf_variant = gr.Textbox(opts.cuda_dtype.lower(), label = 'Specify model variant', placeholder='')
hf_revision = gr.Textbox('', label = 'Specify model revision', placeholder='')
hf_variant = gr.Textbox(opts.cuda_dtype.lower(), label='Specify model variant', placeholder='')
hf_revision = gr.Textbox('', label='Specify model revision', placeholder='')
with gr.Row():
hf_token = gr.Textbox('', label = 'Huggingface token', placeholder='optional access token for private or gated models')
hf_mirror = gr.Textbox('', label = 'Huggingface mirror', placeholder='optional mirror site for downloads')
hf_custom_pipeline = gr.Textbox('', label = 'Custom pipeline', placeholder='optional pipeline for downloads')
hf_token = gr.Textbox('', label='Huggingface token', placeholder='optional access token for private or gated models')
hf_mirror = gr.Textbox('', label='Huggingface mirror', placeholder='optional mirror site for downloads')
hf_custom_pipeline = gr.Textbox('', label='Custom pipeline', placeholder='optional pipeline for downloads')
with gr.Column(scale=1):
gr.HTML('<br>')
hf_download_model_btn = gr.Button(value="Download model", variant='primary')
@@ -235,7 +403,7 @@ def create_ui():
with gr.Row():
hf_headers = ['Name', 'Pipeline', 'Tags', 'Downloads', 'Updated', 'URL']
hf_types = ['str', 'str', 'str', 'number', 'date', 'markdown']
hf_results = gr.DataFrame(None, label = 'Search results', show_label = True, interactive = False, wrap = True, overflow_row_behaviour = 'paginate', max_rows = 10, headers = hf_headers, datatype = hf_types, type='array')
hf_results = gr.DataFrame(None, label='Search results', show_label=True, interactive=False, wrap=True, overflow_row_behaviour='paginate', max_rows=10, headers=hf_headers, datatype=hf_types, type='array')
hf_search_text.submit(fn=hf_search, inputs=[hf_search_text], outputs=[hf_results])
hf_search_btn.click(fn=hf_search, inputs=[hf_search_text], outputs=[hf_results])
@@ -283,7 +451,8 @@ def create_ui():
model['stats']['rating']
])
res = f'Search result: name={name} tag={tag or "none"} type={model_type} models={len(data1)}'
return res, gr.update(visible=len(data1) > 0, value=data1 if len(data1) > 0 else []), gr.update(visible=False, value=None), gr.update(visible=False, value=None)
return res, gr.update(visible=len(data1) > 0, value=data1 if len(data1) > 0 else []), gr.update(
visible=False, value=None), gr.update(visible=False, value=None)
def civit_select1(evt: gr.SelectData, in_data):
model_id = in_data[evt.index[0]][0]
@@ -294,13 +463,7 @@ def create_ui():
for d in model['modelVersions']:
if d.get('images') is not None and len(d['images']) > 0 and len(d['images'][0]['url']) > 0:
preview_img = d['images'][0]['url']
data2.append([
d['id'],
d['modelId'],
d['name'],
d['baseModel'],
d['createdAt'],
])
data2.append([d['id'], d['modelId'], d['name'], d['baseModel'], d['createdAt']])
log.debug(f'CivitAI select: model="{in_data[evt.index[0]]}" versions={len(data2)}')
return data2, None, preview_img
@@ -315,12 +478,7 @@ def create_ui():
for f in variant['files']:
try:
if os.path.splitext(f['name'])[1].lower() in ['.safetensors', '.ckpt', '.pt', '.pth', '.bin']:
data3.append([
f['name'],
round(f['sizeKB']),
json.dumps(f['metadata']),
f['downloadUrl'],
])
data3.append([f['name'], round(f['sizeKB']), json.dumps(f['metadata']), f['downloadUrl']])
except Exception:
pass
log.debug(f'CivitAI select: model="{in_data[evt.index[0]]}" files={len(data3)}')
@@ -340,7 +498,7 @@ def create_ui():
res = f"CivitAI model downloaded error: model={model_url} {e}"
log.error(res)
return res
from modules.sd_models import list_models # pylint: disable=W0621
from modules.sd_models import list_models # pylint: disable=W0621
list_models()
return res
@@ -392,7 +550,7 @@ def create_ui():
txt = '<br>'.join([r for r in res if len(r) > 0])
return txt
global search_metadata_civit # pylint: disable=global-statement
global search_metadata_civit # pylint: disable=global-statement
search_metadata_civit = civit_search_metadata
with gr.Row():
@@ -409,8 +567,8 @@ def create_ui():
civit_model_type = gr.Dropdown(label='Model type', choices=['SD 1.5', 'SD XL', 'LoRA', 'Other'], value='LoRA')
with gr.Column(scale=15):
with gr.Row():
civit_search_text = gr.Textbox('', label = 'Search models', placeholder='keyword')
civit_search_tag = gr.Textbox('', label = '', placeholder='tags')
civit_search_text = gr.Textbox('', label='Search models', placeholder='keyword')
civit_search_tag = gr.Textbox('', label='', placeholder='tags')
civit_search_btn = ToolButton(value="🔍", label="Search", interactive=False)
with gr.Row():
civit_search_res = gr.HTML('')
@@ -420,24 +578,33 @@ def create_ui():
civit_download_model_btn = gr.Button(value="Download", variant='primary')
gr.HTML('<span style="line-height: 2em">Select a model, model version and and model variant from the search results to download or enter model URL manually</span><br>')
with gr.Row():
civit_name = gr.Textbox('', label = 'Model name', placeholder='select model from search results', visible=True)
civit_selected = gr.Textbox('', label = 'Model URL', placeholder='select model from search results', visible=True)
civit_path = gr.Textbox('', label = 'Download path', placeholder='optional subfolder path where to save model', visible=True)
civit_name = gr.Textbox('', label='Model name', placeholder='select model from search results', visible=True)
civit_selected = gr.Textbox('', label='Model URL', placeholder='select model from search results', visible=True)
civit_path = gr.Textbox('', label='Download path', placeholder='optional subfolder path where to save model', visible=True)
with gr.Row():
gr.HTML('<h2>Search results</h2>')
with gr.Row():
civit_headers1 = ['ID', 'Name', 'Tags', 'Downloads', 'Rating']
civit_types1 = ['number', 'str', 'str', 'number', 'number']
civit_results1 = gr.DataFrame(value = None, label = None, show_label = False, interactive = False, wrap = True, overflow_row_behaviour = 'paginate', max_rows = 10, headers = civit_headers1, datatype = civit_types1, type='array', visible=False)
civit_results1 = gr.DataFrame(value=None, label=None, show_label=False, interactive=False,
wrap=True, overflow_row_behaviour='paginate', max_rows=10,
headers=civit_headers1, datatype=civit_types1, type='array',
visible=False)
with gr.Row():
with gr.Column():
civit_headers2 = ['ID', 'ModelID', 'Name', 'Base', 'Created', 'Preview']
civit_types2 = ['number', 'number', 'str', 'str', 'date', 'str']
civit_results2 = gr.DataFrame(value = None, label = 'Model versions', show_label = True, interactive = False, wrap = True, overflow_row_behaviour = 'paginate', max_rows = 10, headers = civit_headers2, datatype = civit_types2, type='array', visible=False)
civit_results2 = gr.DataFrame(value=None, label='Model versions', show_label=True,
interactive=False, wrap=True, overflow_row_behaviour='paginate',
max_rows=10, headers=civit_headers2, datatype=civit_types2,
type='array', visible=False)
with gr.Column():
civit_headers3 = ['Name', 'Size', 'Metadata', 'URL']
civit_types3 = ['str', 'number', 'str', 'str']
civit_results3 = gr.DataFrame(value = None, label = 'Model variants', show_label = True, interactive = False, wrap = True, overflow_row_behaviour = 'paginate', max_rows = 10, headers = civit_headers3, datatype = civit_types3, type='array', visible=False)
civit_results3 = gr.DataFrame(value=None, label='Model variants', show_label=True,
interactive=False, wrap=True, overflow_row_behaviour='paginate',
max_rows=10, headers=civit_headers3, datatype=civit_types3,
type='array', visible=False)
def is_visible(component):
visible = len(component) > 0 if component is not None else False
@@ -454,3 +621,117 @@ def create_ui():
civit_results3.change(fn=is_visible, inputs=[civit_results3], outputs=[civit_results3])
civit_download_model_btn.click(fn=civit_download_model, inputs=[civit_selected, civit_name, civit_path, civit_model_type, models_image], outputs=[models_outcome])
civit_previews_btn.click(fn=civit_search_metadata, inputs=[civit_previews_rehash, civit_previews_rehash], outputs=[models_outcome])
with gr.Tab(label="Update"):
with gr.Row():
gr.HTML('Fetch most recent information about all installed models<br>')
with gr.Row():
civit_update_btn = gr.Button(value="Update", variant='primary')
with gr.Row():
gr.HTML('<h2>Update scan results</h2>')
with gr.Row():
civit_headers4 = ['ID', 'File', 'Name', 'Versions', 'Current', 'Latest', 'Update']
civit_types4 = ['number', 'str', 'str', 'number', 'str', 'str', 'str']
civit_widths4 = ['10%', '25%', '25%', '5%', '10%', '10%', '15%']
civit_results4 = gr.DataFrame(value=None, label=None, show_label=False, interactive=False, wrap=True, overflow_row_behaviour='paginate',
row_count=20, max_rows=100, headers=civit_headers4, datatype=civit_types4, type='array', column_widths=civit_widths4)
with gr.Row():
gr.HTML('<h3>Select model from the list and download update if available</h3>')
with gr.Row():
civit_update_download_btn = gr.Button(value="Download", variant='primary', visible=False)
class CivitModel:
def __init__(self, name, fn, sha = None, meta = {}):
self.name = name
self.id = meta.get('id', 0)
self.fn = fn
self.sha = sha
self.meta = meta
self.versions = 0
self.vername = ''
self.latest = ''
self.latest_hashes = []
self.latest_name = ''
self.url = None
self.status = 'Not found'
def array(self):
return [self.id, self.fn, self.name, self.versions, self.vername, self.latest, self.status]
selected_model: CivitModel = None
update_data = []
def civit_update_metadata():
nonlocal update_data
log.debug('CivitAI update metadata: models')
from modules.ui_extra_networks import get_pages
from modules.modelloader import download_civit_meta
res = []
page: modules.ui_extra_networks.ExtraNetworksPage = get_pages('model')[0]
table_data = []
update_data.clear()
all_hashes = [(item.get('hash', None) or 'XXXXXXXX').upper()[:8] for item in page.list_items()]
for item in page.list_items():
model = CivitModel(name=item['name'], fn=item['filename'], sha=item.get('hash', None), meta=item.get('metadata', {}))
if model.sha is None or len(model.sha) == 0:
res.append(f'CivitAI skip search: name="{model.name}" hash=None')
else:
r = req(f'https://civitai.com/api/v1/model-versions/by-hash/{model.sha}')
res.append(f'CivitAI search: name="{model.name}" hash={model.sha} status={r.status_code}')
if r.status_code == 200:
d = r.json()
model.id = d['modelId']
download_civit_meta(model.fn, model.id)
fn = os.path.splitext(item['filename'])[0] + '.json'
model.meta = readfile(fn, silent=True)
model.name = model.meta.get('name', model.name)
model.versions = len(model.meta.get('modelVersions', []))
versions = model.meta.get('modelVersions', [])
if len(versions) > 0:
model.latest = versions[0].get('name', '')
model.latest_hashes.clear()
for v in versions[0].get('files', []):
for h in v.get('hashes', {}).values():
model.latest_hashes.append(h[:8].upper())
for ver in versions:
for f in ver.get('files', []):
for h in f.get('hashes', {}).values():
if h[:8].upper() == model.sha[:8].upper():
model.vername = ver.get('name', '')
model.url = f.get('downloadUrl', None)
model.latest_name = f.get('name', '')
if model.vername == model.latest:
model.status = 'Latest'
elif any(map(lambda v: v in model.latest_hashes, all_hashes)):
model.status = 'Downloaded'
else:
model.status = 'Available'
break
log.debug(res[-1])
update_data.append(model)
table_data.append(model.array())
yield gr.update(value=table_data), '<br>'.join([r for r in res if len(r) > 0])
return '<br>'.join([r for r in res if len(r) > 0])
def civit_update_select(evt: gr.SelectData, in_data):
nonlocal selected_model, update_data
try:
selected_model = [m for m in update_data if m.fn == in_data[evt.index[0]][1]][0]
except:
selected_model = None
if selected_model is None or selected_model.url is None or selected_model.status != 'Available':
return [gr.update(value='Model update not available'), gr.update(visible=False)]
else:
return [gr.update(), gr.update(visible=True)]
def civit_update_download():
if selected_model is None or selected_model.url is None or selected_model.status != 'Available':
return 'Model update not available'
if selected_model.latest_name is None or len(selected_model.latest_name) == 0:
model_name = f'{selected_model.name} {selected_model.latest}.safetensors'
else:
model_name = selected_model.latest_name
return civit_download_model(selected_model.url, model_name, model_path='', model_type='Model', image_url=None)
civit_update_btn.click(fn=civit_update_metadata, inputs=[], outputs=[civit_results4, models_outcome])
civit_results4.select(fn=civit_update_select, inputs=[civit_results4], outputs=[models_outcome, civit_update_download_btn])
civit_update_download_btn.click(fn=civit_update_download, inputs=[], outputs=[models_outcome])
modules/upscaler.py
+3 -15
View File
@@ -7,8 +7,8 @@ import modules.shared
from modules import modelloader
LANCZOS = (Image.Resampling.LANCZOS if hasattr(Image, 'Resampling') else Image.LANCZOS)
NEAREST = (Image.Resampling.NEAREST if hasattr(Image, 'Resampling') else Image.NEAREST)
LANCZOS = (Image.Resampling.LANCZOS if hasattr(Image, 'Resampling') else Image.Resampling.LANCZOS)
NEAREST = (Image.Resampling.NEAREST if hasattr(Image, 'Resampling') else Image.Resampling.NEAREST)
models = None
class Upscaler:
@@ -66,7 +66,6 @@ class Upscaler:
scalers.append(scaler)
loaded.append(file_name)
modules.shared.log.debug(f'Upscaler type={self.name} folder="{folder}" model="{model_name}" path="{file_name}"')
print(f'Upscaler type={self.name} folder="{folder}" model="{model_name}" path="{file_name}"')
def find_scalers(self):
scalers = []
@@ -226,20 +225,11 @@ def compile_upscaler(model, name=""):
modules.shared.log.info(f"Upscaler Compiling: {name} mode={modules.shared.opts.cuda_compile_backend}")
import logging
import torch._dynamo # pylint: disable=unused-import,redefined-outer-name
use_old_compiled_model_state = False
if modules.shared.opts.cuda_compile_backend == "openvino_fx":
from modules.intel.openvino import openvino_fx, openvino_clear_caches # pylint: disable=unused-import
from modules.sd_models import CompiledModelState
openvino_clear_caches()
from modules.intel.openvino import openvino_fx # pylint: disable=unused-import
torch._dynamo.eval_frame.check_if_dynamo_supported = lambda: True # pylint: disable=protected-access
if modules.shared.compiled_model_state is not None:
use_old_compiled_model_state = True
old_compiled_model_state = modules.shared.compiled_model_state
modules.shared.compiled_model_state = CompiledModelState()
log_level = logging.WARNING if modules.shared.opts.cuda_compile_verbose else logging.CRITICAL # pylint: disable=protected-access
if hasattr(torch, '_logging'):
torch._logging.set_logs(dynamo=log_level, aot=log_level, inductor=log_level) # pylint: disable=protected-access
@@ -248,8 +238,6 @@ def compile_upscaler(model, name=""):
torch._dynamo.config.suppress_errors = modules.shared.opts.cuda_compile_errors # pylint: disable=protected-access
model = torch.compile(model, mode=modules.shared.opts.cuda_compile_mode, backend=modules.shared.opts.cuda_compile_backend, fullgraph=modules.shared.opts.cuda_compile_fullgraph) # pylint: disable=attribute-defined-outside-init
if use_old_compiled_model_state:
modules.shared.compiled_model_state = old_compiled_model_state
modules.shared.log.info("Upscaler: Complilation done.")
except Exception as err:
modules.shared.log.warning(f"Model compile not supported: {err}")
requirements.txt
+6 -5
View File
@@ -33,6 +33,7 @@ rich
safetensors
scipy
tb_nightly
tensordict
toml
torchdiffeq
voluptuous
@@ -48,22 +49,22 @@ clip-interrogator==0.6.0
antlr4-python3-runtime==4.9.3
requests==2.31.0
tqdm==4.66.1
accelerate==0.20.3
accelerate==0.24.1
opencv-python-headless==4.7.0.72
diffusers==0.23.0
diffusers==0.23.1
einops==0.4.1
gradio==3.43.2
huggingface_hub==0.18.0
huggingface_hub==0.19.4
numexpr==2.8.4
numpy==1.24.4
numba==0.57.1
pandas==1.5.3
protobuf==3.20.3
pytorch_lightning==1.9.4
transformers==4.34.1
transformers==4.35.2
tomesd==0.1.3
urllib3==1.26.15
Pillow==9.5.0
Pillow==10.1.0
timm==0.9.7
pydantic==1.10.13
typing-extensions==4.8.0
scripts/xyz_grid.py
+45 -26
View File
@@ -1,4 +1,4 @@
# pylint: disable=unused-argument, attribute-defined-outside-init
# pylint: disable=unused-argument
import re
import csv
@@ -236,29 +236,29 @@ axis_options = [
AxisOption("Clip skip", int, apply_clip_skip),
AxisOption("Denoising strength", float, apply_field("denoising_strength")),
AxisOption("Prompt order", str_permutations, apply_order, fmt=format_value_join_list),
AxisOption("Model dictionary", str, apply_dict, fmt=format_value, cost=1.0, choices=lambda: ['None'] + list(sd_models.checkpoints_list)),
AxisOptionImg2Img("Image mask weight", float, apply_field("inpainting_mask_weight")),
AxisOption("[Postprocess] Upscaler", str, apply_upscaler, choices=lambda: [x.name for x in shared.sd_upscalers][1:]),
AxisOption("[Postprocess] Face restore", str, apply_face_restore, fmt=format_value),
AxisOptionImg2Img("Image mask weight", float, apply_field("inpainting_mask_weight")),
AxisOption("Model dictionary", str, apply_dict, fmt=format_value, cost=1.0, choices=lambda: ['None'] + list(sd_models.checkpoints_list)),
AxisOption("[Sampler] sigma min", float, apply_field("s_min")),
AxisOption("[Sampler] sigma max", float, apply_field("s_max")),
AxisOption("[Sampler] sigma tmin", float, apply_field("s_tmin")),
AxisOption("[Sampler] sigma tmax", float, apply_field("s_tmax")),
AxisOption("[Sampler] sigma Churn", float, apply_field("s_churn")),
AxisOption("[Sampler] sigma noise", float, apply_field("s_noise")),
AxisOption("[Sampler] eta", float, apply_field("eta")),
AxisOption("[Sampler] solver order", int, apply_setting("schedulers_solver_order")),
AxisOption("[Second pass] upscaler", str, apply_field("hr_upscaler"), choices=lambda: [*shared.latent_upscale_modes, *[x.name for x in shared.sd_upscalers]]),
AxisOption("[Second pass] sampler", str, apply_latent_sampler, fmt=format_value, confirm=confirm_samplers, choices=lambda: [x.name for x in sd_samplers.samplers]),
AxisOption("[Second pass] denoising Strength", float, apply_field("denoising_strength")),
AxisOption("[Second pass] hires steps", int, apply_field("hr_second_pass_steps")),
AxisOption("[Sampler] Sigma min", float, apply_field("s_min")),
AxisOption("[Sampler] Sigma max", float, apply_field("s_max")),
AxisOption("[Sampler] Sigma tmin", float, apply_field("s_tmin")),
AxisOption("[Sampler] Sigma tmax", float, apply_field("s_tmax")),
AxisOption("[Sampler] Sigma Churn", float, apply_field("s_churn")),
AxisOption("[Sampler] Sigma noise", float, apply_field("s_noise")),
AxisOption("[Sampler] ETA", float, apply_field("eta")),
AxisOption("[Sampler] Solver order", int, apply_setting("schedulers_solver_order")),
AxisOption("[Second pass] Upscaler", str, apply_field("hr_upscaler"), choices=lambda: [*shared.latent_upscale_modes, *[x.name for x in shared.sd_upscalers]]),
AxisOption("[Second pass] Sampler", str, apply_latent_sampler, fmt=format_value, confirm=confirm_samplers, choices=lambda: [x.name for x in sd_samplers.samplers]),
AxisOption("[Second pass] Denoising Strength", float, apply_field("denoising_strength")),
AxisOption("[Second pass] Hires steps", int, apply_field("hr_second_pass_steps")),
AxisOption("[Second pass] CFG scale", float, apply_field("image_cfg_scale")),
AxisOption("[Second pass] guidance rescale", float, apply_field("diffusers_guidance_rescale")),
AxisOption("[Refiner] model", str, apply_refiner, fmt=format_value, cost=1.0, choices=lambda: ['None'] + sorted(sd_models.checkpoints_list)),
AxisOption("[Refiner] refiner start", float, apply_field("refiner_start")),
AxisOption("[Refiner] refiner steps", float, apply_field("refiner_steps")),
AxisOption("[TOME] Token merging ratio (txt2img)", float, apply_override('token_merging_ratio')),
AxisOption("[TOME] Token merging ratio (hires)", float, apply_override('token_merging_ratio_hr')),
AxisOption("[Second pass] Guidance rescale", float, apply_field("diffusers_guidance_rescale")),
AxisOption("[Refiner] Model", str, apply_refiner, fmt=format_value, cost=1.0, choices=lambda: ['None'] + sorted(sd_models.checkpoints_list)),
AxisOption("[Refiner] Refiner start", float, apply_field("refiner_start")),
AxisOption("[Refiner] Refiner steps", float, apply_field("refiner_steps")),
AxisOption("[ToMe] Token merging ratio (txt2img)", float, apply_override('token_merging_ratio')),
AxisOption("[ToMe] Token merging ratio (hires)", float, apply_override('token_merging_ratio_hr')),
AxisOption("[FreeU] 1st stage backbone factor", float, apply_setting('freeu_b1')),
AxisOption("[FreeU] 2nd stage backbone factor", float, apply_setting('freeu_b2')),
AxisOption("[FreeU] 1st stage skip factor", float, apply_setting('freeu_s1')),
@@ -345,10 +345,10 @@ def draw_xyz_grid(p, xs, ys, zs, x_labels, y_labels, z_labels, cell, draw_legend
for i in range(z_count):
start_index = (i * len(xs) * len(ys)) + i
end_index = start_index + len(xs) * len(ys)
if not no_grid and images.check_grid_size(processed_result.images[start_index:end_index]):
if (not no_grid or include_sub_grids) and images.check_grid_size(processed_result.images[start_index:end_index]):
grid = images.image_grid(processed_result.images[start_index:end_index], rows=len(ys))
if draw_legend:
grid = images.draw_grid_annotations(grid, processed_result.images[start_index].size[0], processed_result.images[start_index].size[1], hor_texts, ver_texts, margin_size)
grid = images.draw_grid_annotations(grid, processed_result.images[start_index].size[0], processed_result.images[start_index].size[1], hor_texts, ver_texts, margin_size, title=title_texts[i])
processed_result.images.insert(i, grid)
processed_result.all_prompts.insert(i, processed_result.all_prompts[start_index])
processed_result.all_seeds.insert(i, processed_result.all_seeds[start_index])
@@ -357,7 +357,7 @@ def draw_xyz_grid(p, xs, ys, zs, x_labels, y_labels, z_labels, cell, draw_legend
if not no_grid and images.check_grid_size(processed_result.images[:z_count]):
z_grid = images.image_grid(processed_result.images[:z_count], rows=1)
if draw_legend:
z_grid = images.draw_grid_annotations(z_grid, sub_grid_size[0], sub_grid_size[1], title_texts, [[images.GridAnnotation()]])
z_grid = images.draw_grid_annotations(z_grid, sub_grid_size[0], sub_grid_size[1], [[images.GridAnnotation()] for _ in z_labels], [[images.GridAnnotation()]])
processed_result.images.insert(0, z_grid)
#processed_result.all_prompts.insert(0, processed_result.all_prompts[0])
#processed_result.all_seeds.insert(0, processed_result.all_seeds[0])
@@ -366,6 +366,14 @@ def draw_xyz_grid(p, xs, ys, zs, x_labels, y_labels, z_labels, cell, draw_legend
class SharedSettingsStackHelper(object):
vae = None
schedulers_solver_order = None
token_merging_ratio_hr = None
token_merging_ratio = None
sd_model_checkpoint = None
sd_model_dict = None
sd_vae_checkpoint = None
def __enter__(self):
#Save overridden settings so they can be restored later.
self.vae = shared.opts.sd_vae
@@ -398,6 +406,8 @@ re_range_count = re.compile(r"\s*([+-]?\s*\d+)\s*-\s*([+-]?\s*\d+)(?:\s*\[(\d+)\
re_range_count_float = re.compile(r"\s*([+-]?\s*\d+(?:.\d*)?)\s*-\s*([+-]?\s*\d+(?:.\d*)?)(?:\s*\[(\d+(?:.\d*)?)\s*])?\s*")
class Script(scripts.Script):
current_axis_options = []
def title(self):
return "X/Y/Z Grid"
@@ -699,9 +709,13 @@ class Script(scripts.Script):
z_count = len(zs)
processed.infotexts[:1+z_count] = grid_infotext[:1+z_count] # Set the grid infotexts to the real ones with extra_generation_params (1 main grid + z_count sub-grids)
if not include_lone_images:
processed.images = processed.images[:z_count+1] # Don't need sub-images anymore, drop from list:
# Don't need sub-images anymore, drop from list:
if no_grid and include_sub_grids:
processed.images = processed.images[:z_count] # we don't have the main grid image, and need zero additional sub-images
else:
processed.images = processed.images[:z_count+1] # we either have the main grid image, or need one sub-images
if shared.opts.grid_save: # Auto-save main and sub-grids:
grid_count = z_count + 1 if z_count > 1 else 1
grid_count = z_count + ( 1 if not no_grid and z_count > 1 else 0 )
for g in range(grid_count):
adj_g = g-1 if g > 0 else g
images.save_image(processed.images[g], p.outpath_grids, "xyz_grid", info=processed.infotexts[g], extension=shared.opts.grid_format, prompt=processed.all_prompts[adj_g], seed=processed.all_seeds[adj_g], grid=True, p=processed)
@@ -711,4 +725,9 @@ class Script(scripts.Script):
del processed.all_prompts[1]
del processed.all_seeds[1]
del processed.infotexts[1]
elif no_grid:
# del processed.images[0]
# del processed.all_prompts[0]
# del processed.all_seeds[0]
del processed.infotexts[0]
return processed
webui.py
+2
View File
@@ -247,6 +247,8 @@ def start_ui():
with open(cmd_opts.auth_file, 'r', encoding="utf8") as file:
for line in file.readlines():
gradio_auth_creds += [x.strip() for x in line.split(',') if x.strip()]
if len(gradio_auth_creds) > 0:
log.info(f'Authentication enabled: users={len(list(gradio_auth_creds))}')
global local_url # pylint: disable=global-statement
stdout = io.StringIO()