mirror of
https://github.com/vladmandic/automatic
synced 2026-09-20 01:31:13 +02:00
Merge remote-tracking branch 'upstream/dev' into Extended-Merging
This commit is contained in:
AI-Casanova
+29
-8
@@ -1,39 +1,60 @@
|
||||
# Change Log for SD.Next
|
||||
|
||||
## Update for 2023-11-10
|
||||
## Update for 2023-11-14
|
||||
|
||||
- **Diffusers**
|
||||
- **LCM** support for any *SD 1.5* or *SD-XL* model!
|
||||
- download [lcm-lora-sd15](https://huggingface.co/latent-consistency/lcm-lora-sdv1-5/tree/main) and/or [lcm-lora-sdxl](https://huggingface.co/latent-consistency/lcm-lora-sdxl/tree/main)
|
||||
- load for favorite *SD 1.5* or *SD-XL* model *(original LCM was SD 1.5 only, this is both)*
|
||||
- load **lcm lora**
|
||||
- load **lcm lora** *(note: lcm lora is processed differently than any other lora)*
|
||||
- set **sampler** to **LCM**
|
||||
- set number of steps to some low number, for SD-XL 6-7 steps is normally sufficient
|
||||
note: LCM scheduler does not support steps higher than 50
|
||||
- set cfg to 1 or 2
|
||||
note: LCM scheduler does not support steps higher than 50
|
||||
- set CFG to between 1 and 2
|
||||
- Add `cli/lcm-convert.py` script to convert any SD 1.5 or SD-XL model to LCM model
|
||||
by baking in LORA and uploading to Huggingface, thanks @Disty0
|
||||
- Support for [Stable Fast](https://github.com/chengzeyi/stable-fast) model compile on *Windows/Linux/WSL2* with *CUDA*
|
||||
See [Wiki:Benchmark](https://github.com/vladmandic/automatic/wiki/Benchmark) for details and comparisment
|
||||
of different backends, precision modes, advanced settings and compile modes
|
||||
*Hint*: **100+ it/s** is possible on *RTX4090* with no special tweaks
|
||||
- Add additional pipeline types for manual model loads when loading from `safetensors`
|
||||
- Updated logic for calculating **steps** when using base/hires/refiner workflows
|
||||
- Safe model offloading for non-standard models
|
||||
- Fix **DPM SDE** scheduler
|
||||
- Better support for SD 1.5 **inpainting** models
|
||||
- Add support for **OpenAI Consistency decoder VAE**
|
||||
- Enhance prompt parsing with long prompts and support for *BREAK* keyword
|
||||
Change-in-behavior: new line in prompt now means *BREAK*
|
||||
- Add alternative Lora loading algorithm, triggered if `SD_LORA_DIFFUSERS` is set
|
||||
- Update to `diffusers==0.23.0`
|
||||
- **Extra networks**
|
||||
- Use multi-threading for 5x load speedup
|
||||
- **General**:
|
||||
- Better Lora trigger words support
|
||||
- Auto refresh styles on change
|
||||
- **General**
|
||||
- Reworked parser when pasting previously generated images/prompts
|
||||
includes all `txt2img`, `img2img` and `override` params
|
||||
- Add refiner options to XYZ Grid
|
||||
- Reworked **model compile**
|
||||
- Support custom upscalers in subfolders
|
||||
- Add additional image info when loading image in process tab
|
||||
- Better file locking when sharing config and/or models between multiple instances
|
||||
- Handle custom API endpoints when using auth
|
||||
- Show logged in user in log when accessing via UI and/or API
|
||||
- Support `--ckpt none` to skip loading a model
|
||||
- **XYZ grid**
|
||||
- Add refiner options to XYZ Grid
|
||||
- Add option to create only subimages in XYZ grid, thanks @midcoastal
|
||||
- Allow custom font, background and text color in settings
|
||||
- **Fixes**
|
||||
- Fix `params.txt` saved before actual image
|
||||
- Fix inpaint
|
||||
- Fix manual grid image save
|
||||
- Fix img2img init image save
|
||||
- More uniform models paths
|
||||
- Safe scripts callback execution
|
||||
- Improve extension compatibility
|
||||
- Improve BF16 support
|
||||
- Improved extension compatibility
|
||||
- Improved BF16 support
|
||||
- Match previews for reference models with downloaded models
|
||||
|
||||
## Update for 2023-11-06
|
||||
|
||||
|
||||
@@ -37,6 +37,8 @@ All Individual features are not listed here, instead check [ChangeLog](CHANGELOG
|
||||
- Built in installer with automatic updates and dependency management
|
||||
- Modernized UI with theme support and number of built-in themes
|
||||
|
||||
<br><br>
|
||||
|
||||
## Backend support
|
||||
|
||||
**SD.Next** supports two main backends: *Original* and *Diffusers* which can be switched on-the-fly:
|
||||
@@ -44,9 +46,9 @@ All Individual features are not listed here, instead check [ChangeLog](CHANGELOG
|
||||
- **Original**: Based on [LDM](https://github.com/Stability-AI/stablediffusion) reference implementation and significantly expanded on by [A1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
|
||||
This is the default backend and it is fully compatible with all existing functionality and extensions
|
||||
It supports **SD 1.x** and **SD 2.x** models
|
||||
All other model types such as SD-XL, LCM, PixArt, Segmind, Kandinsky, etc. require backend **Diffusers**
|
||||
- **Diffusers**: Based on new [Huggingface Diffusers](https://huggingface.co/docs/diffusers/index) implementation
|
||||
It supports All models listed below
|
||||
It is also the *only backend* that supports **Stable Diffusion XL** model
|
||||
It supports *original* SD models as well as *all* models listed below
|
||||
See [wiki article](https://github.com/vladmandic/automatic/wiki/Diffusers) for more information
|
||||
|
||||
## Model support
|
||||
@@ -58,12 +60,19 @@ Additional models will be added as they become available and there is public int
|
||||
- [Segmind SSD-1B](https://huggingface.co/segmind/SSD-1B)
|
||||
- [LCM: Latent Consistency Models](https://github.com/openai/consistency_models)
|
||||
- [Kandinsky](https://github.com/ai-forever/Kandinsky-2) 2.1 and 2.2
|
||||
- [Pixart-α XL 2](https://github.com/PixArt-alpha/PixArt-alpha) Medium and Large
|
||||
- [PixArt-α XL 2](https://github.com/PixArt-alpha/PixArt-alpha) Medium and Large
|
||||
- [Warp Wuerstchen](https://huggingface.co/blog/wuertschen)
|
||||
- [Tsinghua UniDiffusion](https://github.com/thu-ml/unidiffuser)
|
||||
- [DeepFloyd IF](https://github.com/deep-floyd/IF) Medium and Large
|
||||
- [Segmind SD Distilled](https://huggingface.co/blog/sd_distillation) *(all variants)*
|
||||
|
||||
*Notes*:
|
||||
- Loading any model other than standard SD 1.x / SD 2.x requires use of backend **Diffusers**
|
||||
Loading any other models using **Original** backend is not supproted
|
||||
- Loading manually download model `.safetensors` files is supported for SD 1.x / SD 2.x / SD-XL models only
|
||||
For all other model types, use backend **Diffusers** and use built in Model downloader or
|
||||
select model from Networks -> Models -> Reference list in which case it will be auto-downloaded and loaded
|
||||
|
||||
## Platform support
|
||||
|
||||
- *nVidia* GPUs using **CUDA** libraries on both *Windows and Linux*
|
||||
@@ -88,8 +97,8 @@ Additional models will be added as they become available and there is public int
|
||||
- Server can run without virtual environment,
|
||||
but it is recommended to use it to avoid library version conflicts with other applications
|
||||
- **nVidia/CUDA** / **AMD/ROCm** / **Intel/OneAPI** are auto-detected if present and available,
|
||||
but for any other use case specify required parameter explicitly or wrong packages may be installed
|
||||
as installer will assume CPU-only environment
|
||||
For any other use case such as **DirectML**, **ONNX/Olive**, **OpenVINO** specify required parameter explicitly
|
||||
or wrong packages may be installed as installer will assume CPU-only environment
|
||||
- Full startup sequence is logged in `sdnext.log`, so if you encounter any issues, please check it first
|
||||
|
||||
### Run
|
||||
@@ -98,24 +107,47 @@ Once SD.Next is installed, simply run `webui.ps1` or `webui.bat` (*Windows*) or
|
||||
|
||||
Below is partial list of all available parameters, run `webui --help` for the full list:
|
||||
|
||||
Server options:
|
||||
--config CONFIG Use specific server configuration file, default: config.json
|
||||
--ui-config UI_CONFIG Use specific UI configuration file, default: ui-config.json
|
||||
--medvram Split model stages and keep only active part in VRAM, default: False
|
||||
--lowvram Split model components and keep only active part in VRAM, default: False
|
||||
--ckpt CKPT Path to model checkpoint to load immediately, default: None
|
||||
--vae VAE Path to VAE checkpoint to load immediately, default: None
|
||||
--data-dir DATA_DIR Base path where all user data is stored, default:
|
||||
--models-dir MODELS_DIR Base path where all models are stored, default: models
|
||||
--share Enable UI accessible through Gradio site, default: False
|
||||
--insecure Enable extensions tab regardless of other options, default: False
|
||||
--listen Launch web server using public IP address, default: False
|
||||
--auth AUTH Set access authentication like "user:pwd,user:pwd""
|
||||
--autolaunch Open the UI URL in the system's default browser upon launch
|
||||
--docs Mount Gradio docs at /docs, default: False
|
||||
--no-hashing Disable hashing of checkpoints, default: False
|
||||
--no-metadata Disable reading of metadata from models, default: False
|
||||
--no-download Disable download of default model, default: False
|
||||
--backend {original,diffusers} force model pipeline type
|
||||
|
||||
Setup options:
|
||||
--debug Run installer with debug logging, default: False
|
||||
--reset Reset main repository to latest version, default: False
|
||||
--upgrade Upgrade main repository to latest version, default: False
|
||||
--requirements Force re-check of requirements, default: False
|
||||
--quick Run with startup sequence only, default: False
|
||||
--use-directml Use DirectML if no compatible GPU is detected, default: False
|
||||
--use-openvino Use Intel OpenVINO backend, default: False
|
||||
--use-ipex Force use Intel OneAPI XPU backend, default: False
|
||||
--use-cuda Force use nVidia CUDA backend, default: False
|
||||
--use-rocm Force use AMD ROCm backend, default: False
|
||||
--skip-update Skip update of extensions and submodules, default: False
|
||||
--use-xformers Force use xFormers cross-optimization, default: False
|
||||
--skip-requirements Skips checking and installing requirements, default: False
|
||||
--skip-extensions Skips running individual extension installers, default: False
|
||||
--skip-git Skips running all GIT operations, default: False
|
||||
--skip-torch Skips running Torch checks, default: False
|
||||
--skip-all Skips running all checks, default: False
|
||||
--experimental Allow unsupported versions of libraries, default: False
|
||||
--reinstall Force reinstallation of all requirements, default: False
|
||||
--debug Run installer with debug logging, default: False
|
||||
--reset Reset main repository to latest version, default: False
|
||||
--upgrade Upgrade main repository to latest version, default: False
|
||||
--safe Run in safe mode with no user extensions
|
||||
|
||||
<br><br>
|
||||
|
||||
## Notes
|
||||
|
||||
@@ -126,7 +158,6 @@ SD.Next comes with several extensions pre-installed:
|
||||
- [ControlNet](https://github.com/Mikubill/sd-webui-controlnet)
|
||||
- [Agent Scheduler](https://github.com/ArtVentureX/sd-webui-agent-scheduler)
|
||||
- [Image Browser](https://github.com/AlUlkesh/stable-diffusion-webui-images-browser)
|
||||
- [Rembg Background Removal](https://github.com/AUTOMATIC1111/stable-diffusion-webui-rembg)
|
||||
|
||||
### **Collab**
|
||||
|
||||
@@ -143,10 +174,10 @@ The idea behind the fork is to enable latest technologies and advances in text-t
|
||||
|
||||
> *Sometimes this is not the same as "as simple as possible to use".*
|
||||
|
||||
If you are looking an amazing simple-to-use Stable Diffusion tool, I'd suggest [InvokeAI](https://invoke-ai.github.io/InvokeAI/) specifically due to its automated installer and ease of use.
|
||||
|
||||
General goals:
|
||||
|
||||
- Multi-model
|
||||
- Enable usage of as many as possible txt2img and img2img generative models
|
||||
- Cross-platform
|
||||
- Create uniform experience while automatically managing any platform specific differences
|
||||
- Performance
|
||||
|
||||
@@ -16,7 +16,7 @@ options = Map({
|
||||
'restore_faces': False,
|
||||
'prompt': 'photo of two dice on a table',
|
||||
'negative_prompt': 'foggy, blurry',
|
||||
'steps': 20,
|
||||
'steps': 50,
|
||||
'batch_size': 1,
|
||||
'n_iter': 1,
|
||||
'seed': -1,
|
||||
|
||||
@@ -64,6 +64,8 @@ class ExtraNetworkLora(extra_networks.ExtraNetwork):
|
||||
self.active = False
|
||||
|
||||
def deactivate(self, p):
|
||||
if shared.backend == shared.Backend.DIFFUSERS:
|
||||
shared.sd_model.unload_lora_weights()
|
||||
if not self.active and getattr(networks, "originals", None ) is not None:
|
||||
networks.originals.undo() # remove patches
|
||||
if networks.debug:
|
||||
|
||||
@@ -14,7 +14,7 @@ import network_norm
|
||||
import lora_convert
|
||||
import torch
|
||||
import diffusers.models.lora
|
||||
from modules import shared, devices, sd_models, errors, scripts, sd_hijack
|
||||
from modules import shared, devices, sd_models, sd_models_compile, errors, scripts, sd_hijack
|
||||
|
||||
|
||||
debug = os.environ.get('SD_LORA_DEBUG', None)
|
||||
@@ -74,6 +74,24 @@ def assign_network_names_to_compvis_modules(sd_model):
|
||||
sd_model.network_layer_mapping = network_layer_mapping
|
||||
|
||||
|
||||
def load_diffusers(name, network_on_disk):
|
||||
t0 = time.time()
|
||||
cached = lora_cache.get(name, None)
|
||||
# if debug:
|
||||
shared.log.debug(f'LoRA load: name={name} file={network_on_disk.filename} type=diffusers {"cached" if cached else ""}')
|
||||
if cached is not None:
|
||||
return cached
|
||||
if shared.backend != shared.Backend.DIFFUSERS:
|
||||
return None
|
||||
shared.sd_model.load_lora_weights(network_on_disk.filename)
|
||||
net = network.Network(name, network_on_disk)
|
||||
net.mtime = os.path.getmtime(network_on_disk.filename)
|
||||
lora_cache[name] = net
|
||||
t1 = time.time()
|
||||
timer['load'] += t1 - t0
|
||||
return net
|
||||
|
||||
|
||||
def load_network(name, network_on_disk):
|
||||
t0 = time.time()
|
||||
cached = lora_cache.get(name, None)
|
||||
@@ -128,11 +146,16 @@ def load_networks(names, te_multipliers=None, unet_multipliers=None, dyn_dims=No
|
||||
for i, name in enumerate(names):
|
||||
if shared.compiled_model_state.lora_model[i] != f"{name}:{te_multipliers[i] if te_multipliers else 1.0}":
|
||||
recompile_model = True
|
||||
shared.compiled_model_state.lora_model = []
|
||||
break
|
||||
else:
|
||||
recompile_model = True
|
||||
shared.compiled_model_state.lora_model = []
|
||||
shared.compiled_model_state.lora_model = []
|
||||
if recompile_model:
|
||||
if not shared.opts.openvino_disable_model_caching:
|
||||
shared.log.warning("LoRa: Disabling OpenVINO model caching")
|
||||
shared.opts.openvino_disable_model_caching = True
|
||||
shared.compiled_model_state.lora_compile = True
|
||||
sd_models.unload_model_weights(op='model')
|
||||
shared.opts.cuda_compile = False
|
||||
sd_models.reload_model_weights(op='model')
|
||||
@@ -142,9 +165,17 @@ def load_networks(names, te_multipliers=None, unet_multipliers=None, dyn_dims=No
|
||||
net = None
|
||||
if network_on_disk is not None:
|
||||
try:
|
||||
net = load_network(name, network_on_disk)
|
||||
if recompile_model:
|
||||
shared.compiled_model_state.lora_model.append(f"{name}:{te_multipliers[i] if te_multipliers else 1.0}")
|
||||
if shared.backend == shared.Backend.DIFFUSERS and (os.environ.get('SD_LORA_DIFFUSERS', None)
|
||||
or getattr(network_on_disk, 'shorthash', None) == 'aaebf6360f7d' # lcm sd15
|
||||
or getattr(network_on_disk, 'shorthash', None) == '3d18b05e4f56' # lcm sdxl
|
||||
or (shared.opts.cuda_compile and shared.opts.cuda_compile_backend == "openvino_fx")):
|
||||
net = load_diffusers(name, network_on_disk)
|
||||
else:
|
||||
net = load_network(name, network_on_disk)
|
||||
except Exception as e:
|
||||
shared.log.error(f"LoRA load failed: file={network_on_disk.filename}")
|
||||
shared.log.error(f"LoRA load failed: file={network_on_disk.filename} {e}")
|
||||
if debug:
|
||||
errors.display(e, f"LoRA load failed file={network_on_disk.filename}")
|
||||
continue
|
||||
@@ -170,7 +201,7 @@ def load_networks(names, te_multipliers=None, unet_multipliers=None, dyn_dims=No
|
||||
|
||||
if recompile_model:
|
||||
shared.log.info("LoRA recompiling model")
|
||||
sd_models.compile_diffusers(shared.sd_model)
|
||||
sd_models_compile.compile_diffusers(shared.sd_model)
|
||||
|
||||
|
||||
def network_restore_weights_from_backup(self: Union[torch.nn.Conv2d, torch.nn.Linear, torch.nn.GroupNorm, torch.nn.LayerNorm, torch.nn.MultiheadAttention, diffusers.models.lora.LoRACompatibleLinear, diffusers.models.lora.LoRACompatibleConv]):
|
||||
|
||||
@@ -30,7 +30,7 @@ script_callbacks.on_infotext_pasted(networks.infotext_pasted)
|
||||
|
||||
shared.options_templates.update(shared.options_section(('extra_networks', "Extra Networks"), {
|
||||
# "sd_lora": shared.OptionInfo("None", "Add network to prompt", gr.Dropdown, lambda: {"choices": ["None", *networks.available_networks], "visible": False}, refresh=networks.list_available_networks),
|
||||
"sd_lora": shared.OptionInfo("None", "Add network to prompt", gr.Dropdown, {"choices": ["None"]}),
|
||||
"sd_lora": shared.OptionInfo("None", "Add network to prompt", gr.Dropdown, {"choices": ["None"], "visible": False}),
|
||||
# "lora_show_all": shared.OptionInfo(False, "Always show all networks on the Lora page").info("otherwise, those detected as for incompatible version of Stable Diffusion will be hidden"),
|
||||
# "lora_hide_unknown_for_versions": shared.OptionInfo([], "Hide networks of unknown versions for model versions", gr.CheckboxGroup, {"choices": ["SD1", "SD2", "SDXL"]}),
|
||||
}))
|
||||
|
||||
@@ -33,24 +33,11 @@ class ExtraNetworksPageLora(ui_extra_networks.ExtraNetworksPage):
|
||||
if l.sd_version == network.SdVersion.SDXL:
|
||||
return None
|
||||
|
||||
# tags from model metedata
|
||||
possible_tags = l.metadata.get('ss_tag_frequency', {}) if l.metadata is not None else {}
|
||||
if isinstance(possible_tags, str):
|
||||
possible_tags = {}
|
||||
tags = {}
|
||||
for k, v in possible_tags.items():
|
||||
words = k.split('_', 1) if '_' in k else [v, k]
|
||||
words = [str(w).replace('.json', '') for w in words]
|
||||
if words[0] == '{}':
|
||||
words[0] = 0
|
||||
tags[' '.join(words[1:])] = words[0]
|
||||
|
||||
item = {
|
||||
"type": 'Lora',
|
||||
"name": name,
|
||||
"filename": l.filename,
|
||||
"hash": l.shorthash,
|
||||
"search_term": self.search_terms_from_path(l.filename) + ' '.join(tags.keys()),
|
||||
"preview": self.find_preview(l.filename),
|
||||
"prompt": json.dumps(f" <lora:{l.get_alias()}:{shared.opts.extra_networks_default_multiplier}>"),
|
||||
"local_preview": f"{path}.{shared.opts.samples_format}",
|
||||
@@ -59,16 +46,44 @@ class ExtraNetworksPageLora(ui_extra_networks.ExtraNetworksPage):
|
||||
"size": os.path.getsize(l.filename),
|
||||
}
|
||||
info = self.find_info(l.filename)
|
||||
item["info"] = info
|
||||
item["description"] = self.find_description(l.filename, info) # use existing info instead of double-read
|
||||
|
||||
# tags from user metadata
|
||||
possible_tags = info.get('tags', [])
|
||||
tags = {}
|
||||
possible_tags = l.metadata.get('ss_tag_frequency', {}) if l.metadata is not None else {} # tags from model metedata
|
||||
if isinstance(possible_tags, str):
|
||||
possible_tags = {}
|
||||
for k, v in possible_tags.items():
|
||||
words = k.split('_', 1) if '_' in k else [v, k]
|
||||
words = [str(w).replace('.json', '') for w in words]
|
||||
if words[0] == '{}':
|
||||
words[0] = 0
|
||||
tag = ' '.join(words[1:])
|
||||
tags[tag] = words[0]
|
||||
versions = info.get('modelVersions', []) # trigger words from info json
|
||||
for v in versions:
|
||||
possible_tags = v.get('trainedWords', [])
|
||||
if isinstance(possible_tags, list):
|
||||
for tag in possible_tags:
|
||||
if tag not in tags:
|
||||
tags[tag] = 0
|
||||
search = {}
|
||||
possible_tags = info.get('tags', []) # tags from info json
|
||||
if not isinstance(possible_tags, list):
|
||||
possible_tags = [v for v in possible_tags.values()]
|
||||
for v in possible_tags:
|
||||
tags[v] = 0
|
||||
item["tags"] = tags
|
||||
search[v] = 0
|
||||
if len(list(tags)) == 0:
|
||||
tags = search
|
||||
|
||||
bad_chars = [';', ':', '<', ">", "*", '?', '\'', '\"']
|
||||
clean_tags = {}
|
||||
for k, v in tags.items():
|
||||
tag = ''.join(i for i in k if not i in bad_chars)
|
||||
clean_tags[tag] = v
|
||||
|
||||
item["info"] = info
|
||||
item["description"] = self.find_description(l.filename, info) # use existing info instead of double-read
|
||||
item["tags"] = clean_tags
|
||||
item["search_term"] = f'{self.search_terms_from_path(l.filename)} {" ".join(tags.keys())} {" ".join(search.keys())}'
|
||||
|
||||
return item
|
||||
except Exception as e:
|
||||
|
||||
Submodule extensions-builtin/sd-webui-agent-scheduler updated: 8970f485b7...7a03f4e678
Submodule extensions-builtin/sd-webui-controlnet updated: 05ef0b1cd1...c1f3d6f850
+5
-10
@@ -17,32 +17,27 @@
|
||||
"Segmind SSD-1B": {
|
||||
"path": "segmind/SSD-1B",
|
||||
"desc": "The Segmind Stable Diffusion Model (SSD-1B) offers a compact, efficient, and distilled version of the SDXL model. At 50% smaller and 60% faster than Stable Diffusion XL (SDXL), it provides quick and seamless performance without sacrificing image quality.",
|
||||
"preview": "segmind--ssd-1b.jpg"
|
||||
"preview": "segmind--SSD-1B.jpg"
|
||||
},
|
||||
"Segmind Tiny": {
|
||||
"path": "segmind/tiny-sd",
|
||||
"desc": "Segmind's Tiny-SD offers a compact, efficient, and distilled version of Realistic Vision 4.0 and is up to 80% faster than SD1.5",
|
||||
"preview": "segmind--tiny-sd.jpg"
|
||||
},
|
||||
"LCM SD-XL": {
|
||||
"path": "latent-consistency/lcm-sdxl",
|
||||
"desc": "Latent Consistencey Models enable swift inference with minimal steps on any pre-trained LDMs, including Stable Diffusion. By distilling classifier-free guidance into the model's input, LCM can generate high-quality images in very short inference time. LCM can generate quality images in as few as 3-4 steps, making it blazingly fast.",
|
||||
"preview": "latent-consistency--lcm-sdxl.jpg"
|
||||
},
|
||||
"LCM SD-1.5 Dreamshaper 7": {
|
||||
"path": "SimianLuo/LCM_Dreamshaper_v7",
|
||||
"desc": "Latent Consistencey Models enable swift inference with minimal steps on any pre-trained LDMs, including Stable Diffusion. By distilling classifier-free guidance into the model's input, LCM can generate high-quality images in very short inference time. LCM can generate quality images in as few as 3-4 steps, making it blazingly fast.",
|
||||
"preview": "simianluo--lcm_dreamshaper_v7.jpg"
|
||||
"preview": "SimianLuo--LCM_Dreamshaper_v7.jpg"
|
||||
},
|
||||
"Pixart-α XL 2 Medium 512": {
|
||||
"path": "PixArt-alpha/PixArt-XL-2-512x512",
|
||||
"desc": "PixArt-α is a Transformer-based T2I diffusion model whose image generation quality is competitive with state-of-the-art image generators (e.g., Imagen, SDXL, and even Midjourney), and the training speed markedly surpasses existing large-scale T2I models. Extensive experiments demonstrate that PIXART-α excels in image quality, artistry, and semantic control. It can directly generate 512px images from text prompts within a single sampling process.",
|
||||
"preview": "pixart-alpha--pixart-xl-2-512x512.jpg"
|
||||
"preview": "PixArt-alpha--PixArt-XL-2-512x512.jpg"
|
||||
},
|
||||
"Pixart-α XL 2 Large 1024": {
|
||||
"path": "PixArt-alpha/PixArt-XL-2-1024-MS",
|
||||
"desc": "PixArt-α is a Transformer-based T2I diffusion model whose image generation quality is competitive with state-of-the-art image generators (e.g., Imagen, SDXL, and even Midjourney), and the training speed markedly surpasses existing large-scale T2I models. Extensive experiments demonstrate that PIXART-α excels in image quality, artistry, and semantic control. It can directly generate 1024px images from text prompts within a single sampling process.",
|
||||
"preview": "pixart-alpha--pixart-xl-2-1024-ms.jpg"
|
||||
"preview": "PixArt-alpha--PixArt-XL-2-1024-MS.jpg"
|
||||
},
|
||||
"Warp Wuerstchen": {
|
||||
"path": "warp-ai/wuerstchen",
|
||||
@@ -62,7 +57,7 @@
|
||||
"DeepFloyd IF Medium": {
|
||||
"path": "DeepFloyd/IF-I-M-v1.0",
|
||||
"desc": "DeepFloyd-IF is a pixel-based text-to-image triple-cascaded diffusion model, that can generate pictures with new state-of-the-art for photorealism and language understanding. The result is a highly efficient model that outperforms current state-of-the-art models, achieving a zero-shot FID-30K score of 6.66 on the COCO dataset. It is modular and composed of frozen text mode and three pixel cascaded diffusion modules, each designed to generate images of increasing resolution: 64x64, 256x256, and 1024x1024.",
|
||||
"preview": "deepfloyd--if-i-m-v1.0.jpg"
|
||||
"preview": "DeepFloyd--IF-I-M-v1.0.jpg"
|
||||
},
|
||||
"Tsinghua UniDiffuser": {
|
||||
"path": "thu-ml/unidiffuser-v1",
|
||||
|
||||
+31
-2
@@ -447,7 +447,36 @@ def check_torch():
|
||||
install('onnxruntime-openvino', 'onnxruntime-openvino', ignore=True) # TODO numpy version conflicts with tensorflow and doesn't support Python 3.11
|
||||
elif allow_openvino and args.use_openvino:
|
||||
log.info('Using OpenVINO')
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.1.0 torchvision==0.16.0 --index-url https://download.pytorch.org/whl/cpu')
|
||||
if "linux" in sys.platform:
|
||||
if sys.version_info[1] == 11:
|
||||
pytorch_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torch-2.1.0.dev20230820+cpu-cp311-cp311-linux_x86_64.whl'
|
||||
torchvision_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torchvision-0.16.0.dev20230820+cpu-cp311-cp311-linux_x86_64.whl'
|
||||
elif sys.version_info[1] == 10:
|
||||
pytorch_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torch-2.1.0.dev20230820+cpu-cp310-cp310-linux_x86_64.whl'
|
||||
torchvision_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torchvision-0.16.0.dev20230820+cpu-cp310-cp310-linux_x86_64.whl'
|
||||
elif sys.version_info[1] == 8:
|
||||
pytorch_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torch-2.1.0.dev20230820+cpu-cp38-cp38-linux_x86_64.whl'
|
||||
torchvision_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torchvision-0.16.0.dev20230820+cpu-cp38-cp38-linux_x86_64.whl'
|
||||
else:
|
||||
pytorch_pip = 'torch==2.1.0'
|
||||
torchvision_pip = 'torchvision==0.16.0 --index-url https://download.pytorch.org/whl/cpu'
|
||||
elif sys.platform == 'darwin':
|
||||
pytorch_pip = 'torch==2.1.0'
|
||||
torchvision_pip = 'torchvision==0.16.0 --index-url https://download.pytorch.org/whl/cpu'
|
||||
else:
|
||||
if sys.version_info[1] == 11:
|
||||
pytorch_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torch-2.1.0.dev20230820+cpu-cp311-cp311-win_amd64.whl'
|
||||
torchvision_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torchvision-0.16.0.dev20230820+cpu-cp311-cp311-win_amd64.whl'
|
||||
elif sys.version_info[1] == 10:
|
||||
pytorch_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torch-2.1.0.dev20230820+cpu-cp310-cp310-win_amd64.whl'
|
||||
torchvision_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torchvision-0.16.0.dev20230820+cpu-cp310-cp310-win_amd64.whl'
|
||||
elif sys.version_info[1] == 8:
|
||||
pytorch_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torch-2.1.0.dev20230820+cpu-cp38-cp38-win_amd64.whl'
|
||||
torchvision_pip = 'https://github.com/Disty0/automatic/releases/download/openvino_pre_release_pytorch/torchvision-0.16.0.dev20230820+cpu-cp38-cp38-win_amd64.whl'
|
||||
else:
|
||||
pytorch_pip = 'torch==2.1.0'
|
||||
torchvision_pip = 'torchvision==0.16.0 --index-url https://download.pytorch.org/whl/cpu'
|
||||
torch_command = os.environ.get('TORCH_COMMAND', f'{pytorch_pip} {torchvision_pip}')
|
||||
else:
|
||||
machine = platform.machine()
|
||||
if sys.platform == 'darwin':
|
||||
@@ -516,7 +545,7 @@ def check_torch():
|
||||
install('hidet', 'hidet')
|
||||
if args.use_openvino or opts.get('cuda_compile_backend', '') == 'openvino_fx':
|
||||
uninstall('openvino')
|
||||
install('openvino-nightly==2023.2.0.dev20231102', 'openvino-nightly')
|
||||
install('openvino-nightly==2023.3.0.dev20231114', 'openvino-nightly')
|
||||
install('onnxruntime-openvino', 'onnxruntime-openvino', ignore=True) # TODO numpy version conflicts with tensorflow and doesn't support Python 3.11
|
||||
os.environ.setdefault('PYTORCH_TRACING_MODE', 'TORCHFX')
|
||||
os.environ.setdefault('NEOReadDebugKeys', '1')
|
||||
|
||||
@@ -40,7 +40,7 @@ const setENState = (state) => {
|
||||
// methods
|
||||
|
||||
function showCardDetails(event) {
|
||||
console.log('showCardDetails', event)
|
||||
console.log('showCardDetails', event);
|
||||
const tabname = getENActiveTab();
|
||||
const btn = gradioApp().getElementById(`${tabname}_extra_details_btn`);
|
||||
btn.click();
|
||||
@@ -183,7 +183,7 @@ function sortExtraNetworks() {
|
||||
return `sort page ${pagename} cards ${num} by ${desc}`;
|
||||
}
|
||||
|
||||
function refreshExtraNetworks(tabname) {
|
||||
function refreshENInput(tabname) {
|
||||
log('refreshExtraNetworks', tabname, gradioApp().querySelector(`#${tabname}_extra_networks textarea`)?.value);
|
||||
gradioApp().querySelector(`#${tabname}_extra_networks textarea`)?.dispatchEvent(new Event('input'));
|
||||
}
|
||||
@@ -234,6 +234,27 @@ function quickSaveStyle() {
|
||||
if (btnSave) btnSave.click();
|
||||
}
|
||||
|
||||
let enDirty = false;
|
||||
function closeDetailsEN(args) {
|
||||
// log('closeDetailsEN');
|
||||
enDirty = true;
|
||||
const tabname = getENActiveTab();
|
||||
const btnClose = gradioApp().getElementById(`${tabname}_extra_details_close`);
|
||||
if (btnClose) setTimeout(() => btnClose.click(), 100);
|
||||
const btnRefresh = gradioApp().getElementById(`${tabname}_extra_refresh`);
|
||||
if (btnRefresh && enDirty) setTimeout(() => btnRefresh.click(), 100);
|
||||
return args;
|
||||
}
|
||||
|
||||
function refeshDetailsEN(args) {
|
||||
log(`refeshDetailsEN: ${enDirty}`);
|
||||
const tabname = getENActiveTab();
|
||||
const btnRefresh = gradioApp().getElementById(`${tabname}_extra_refresh`);
|
||||
if (btnRefresh && enDirty) setTimeout(() => btnRefresh.click(), 100);
|
||||
enDirty = false;
|
||||
return args;
|
||||
}
|
||||
|
||||
// init
|
||||
|
||||
function setupExtraNetworksForTab(tabname) {
|
||||
|
||||
@@ -128,7 +128,7 @@ def get_memory_stats():
|
||||
process = psutil.Process(os.getpid())
|
||||
res = process.memory_info()
|
||||
ram_total = 100 * res.rss / process.memory_percent()
|
||||
return f'used={gb(res.rss)} total={gb(ram_total)}'
|
||||
return f'{gb(res.rss)}/{gb(ram_total)}'
|
||||
|
||||
|
||||
def start_server(immediate=True, server=None):
|
||||
@@ -145,7 +145,7 @@ def start_server(immediate=True, server=None):
|
||||
if not immediate:
|
||||
time.sleep(3)
|
||||
if collected > 0:
|
||||
installer.log.debug(f'Memory {get_memory_stats()} Collected {collected}')
|
||||
installer.log.debug(f'Memory: {get_memory_stats()} collected={collected}')
|
||||
module_spec = importlib.util.spec_from_file_location('webui', 'webui.py')
|
||||
server = importlib.util.module_from_spec(module_spec)
|
||||
installer.log.debug(f'Starting module: {server}')
|
||||
@@ -233,7 +233,7 @@ if __name__ == "__main__":
|
||||
if round(time.time()) % 120 == 0:
|
||||
state = f'job="{instance.state.job}" {instance.state.job_no}/{instance.state.job_count}' if instance.state.job != '' or instance.state.job_no != 0 or instance.state.job_count != 0 else 'idle'
|
||||
uptime = round(time.time() - instance.state.server_start)
|
||||
installer.log.debug(f'Server alive={alive} jobs={instance.state.total_jobs} requests={requests} uptime={uptime} memory {get_memory_stats()} {state}')
|
||||
installer.log.debug(f'Server: alive={alive} jobs={instance.state.total_jobs} requests={requests} uptime={uptime} memory={get_memory_stats()} backend={instance.backend} {state}')
|
||||
if not alive:
|
||||
if uv is not None and uv.wants_restart:
|
||||
installer.log.info('Server restarting...')
|
||||
|
||||
|
Before Width: | Height: | Size: 93 KiB After Width: | Height: | Size: 93 KiB |
|
Before Width: | Height: | Size: 79 KiB After Width: | Height: | Size: 79 KiB |
|
Before Width: | Height: | Size: 58 KiB After Width: | Height: | Size: 58 KiB |
|
Before Width: | Height: | Size: 76 KiB After Width: | Height: | Size: 76 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 36 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 43 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 30 KiB |
|
Before Width: | Height: | Size: 114 KiB After Width: | Height: | Size: 114 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 42 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 29 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 27 KiB |
+6
-3
@@ -145,7 +145,7 @@ class Api:
|
||||
self.add_api_route("/sdapi/v1/reload-checkpoint", self.reloadapi, methods=["POST"])
|
||||
self.add_api_route("/sdapi/v1/scripts", self.get_scripts_list, methods=["GET"], response_model=models.ScriptsList)
|
||||
self.add_api_route("/sdapi/v1/script-info", self.get_script_info, methods=["GET"], response_model=List[models.ScriptInfo])
|
||||
self.add_api_route("/sdapi/v1/log", self.get_log_buffer, methods=["GET"], response_model=List) # bypass auth
|
||||
self.add_api_route("/sdapi/v1/log", self.get_log_buffer, methods=["GET"], response_model=List)
|
||||
self.add_api_route("/sdapi/v1/start", self.session_start, methods=["GET"])
|
||||
self.add_api_route("/sdapi/v1/motd", self.get_motd, methods=["GET"], response_model=str)
|
||||
self.add_api_route("/sdapi/v1/extra-networks", self.get_extra_networks, methods=["GET"], response_model=List[models.ExtraNetworkItem])
|
||||
@@ -153,11 +153,12 @@ class Api:
|
||||
self.default_script_arg_img2img = []
|
||||
|
||||
def add_api_route(self, path: str, endpoint, **kwargs):
|
||||
if shared.cmd_opts.auth or shared.cmd_opts.auth_file:
|
||||
if (shared.cmd_opts.auth or shared.cmd_opts.auth_file) and shared.cmd_opts.api_only:
|
||||
return self.app.add_api_route(path, endpoint, dependencies=[Depends(self.auth)], **kwargs)
|
||||
return self.app.add_api_route(path, endpoint, **kwargs)
|
||||
|
||||
def auth(self, credentials: HTTPBasicCredentials = Depends(HTTPBasic())):
|
||||
# this is only needed for api-only since otherwise auth is handled in gradio/routes.py
|
||||
if credentials.username in self.credentials:
|
||||
if compare_digest(credentials.password, self.credentials[credentials.username]):
|
||||
return True
|
||||
@@ -170,7 +171,9 @@ class Api:
|
||||
return lines
|
||||
|
||||
def session_start(self, req: Request, agent: Optional[str] = None):
|
||||
shared.log.info(f'Browser session: client={req.client.host} agent={agent}')
|
||||
token = req.cookies.get("access-token") or req.cookies.get("access-token-unsecure")
|
||||
user = self.app.tokens.get(token)
|
||||
shared.log.info(f'Browser session: user={user} client={req.client.host} agent={agent}')
|
||||
return {}
|
||||
|
||||
def get_motd(self):
|
||||
|
||||
@@ -211,6 +211,8 @@ def parse_generation_parameters(x: str):
|
||||
return res
|
||||
remaining = x[7:] if x.startswith('Prompt: ') else x
|
||||
remaining = x[11:] if x.startswith('parameters: ') else x
|
||||
if 'Steps: ' in remaining and 'Negative prompt: ' not in remaining:
|
||||
remaining = remaining.replace('Steps: ', 'Negative prompt: , Steps: ')
|
||||
prompt, remaining = remaining.strip().split('Negative prompt: ', maxsplit=1) if 'Negative prompt: ' in remaining else (remaining, '')
|
||||
res["Prompt"] = prompt.strip()
|
||||
negative, remaining = remaining.strip().split('Steps: ', maxsplit=1) if 'Steps: ' in remaining else (remaining, None)
|
||||
@@ -245,7 +247,6 @@ infotext_to_setting_name_mapping = [
|
||||
('VAE', 'sd_vae'),
|
||||
('Parser', 'prompt_attention'),
|
||||
('Color correction', 'img2img_color_correction'),
|
||||
('LoRA method', 'diffusers_lora_loader'),
|
||||
# Samplers
|
||||
('Sampler Eta', 'scheduler_eta'),
|
||||
('Sampler ENSD', 'eta_noise_seed_delta'),
|
||||
|
||||
+1
-1
@@ -16,7 +16,7 @@ def dump_cache():
|
||||
def cache(subsection):
|
||||
global cache_data # pylint: disable=global-statement
|
||||
if cache_data is None:
|
||||
cache_data = {} if not os.path.isfile(cache_filename) else shared.readfile(cache_filename)
|
||||
cache_data = {} if not os.path.isfile(cache_filename) else shared.readfile(cache_filename, lock=True)
|
||||
s = cache_data.get(subsection, {})
|
||||
cache_data[subsection] = s
|
||||
return s
|
||||
|
||||
+42
-23
@@ -56,7 +56,7 @@ def image_grid(imgs, batch_size=1, rows=None):
|
||||
params = script_callbacks.ImageGridLoopParams(imgs, cols, rows)
|
||||
script_callbacks.image_grid_callback(params)
|
||||
w, h = imgs[0].size
|
||||
grid = Image.new('RGB', size=(params.cols * w, params.rows * h), color='black')
|
||||
grid = Image.new('RGB', size=(params.cols * w, params.rows * h), color=shared.opts.grid_background)
|
||||
for i, img in enumerate(params.imgs):
|
||||
grid.paste(img, box=(i % params.cols * w, i // params.cols * h))
|
||||
return grid
|
||||
@@ -122,7 +122,7 @@ class GridAnnotation:
|
||||
self.size = None
|
||||
|
||||
|
||||
def draw_grid_annotations(im, width, height, hor_texts, ver_texts, margin=0):
|
||||
def draw_grid_annotations(im, width, height, hor_texts, ver_texts, margin=0, title=None):
|
||||
def wrap(drawing, text, font, line_length):
|
||||
lines = ['']
|
||||
for word in text.split():
|
||||
@@ -141,55 +141,63 @@ def draw_grid_annotations(im, width, height, hor_texts, ver_texts, margin=0):
|
||||
|
||||
def draw_texts(drawing: ImageDraw, draw_x, draw_y, lines, initial_fnt, initial_fontsize):
|
||||
for line in lines:
|
||||
fnt = initial_fnt
|
||||
font = initial_fnt
|
||||
fontsize = initial_fontsize
|
||||
while drawing.multiline_textsize(line.text, font=fnt)[0] > line.allowed_width and fontsize > 0:
|
||||
fontsize -= 1
|
||||
fnt = get_font(fontsize)
|
||||
drawing.multiline_text((draw_x, draw_y + line.size[1] / 2), line.text, font=fnt, fill=color_active if line.is_active else color_inactive, anchor="mm", align="center")
|
||||
while drawing.multiline_textsize(line.text, font=font)[0] > line.allowed_width and fontsize > 0:
|
||||
fontsize -= 2
|
||||
font = get_font(fontsize)
|
||||
drawing.multiline_text((draw_x, draw_y + line.size[1] / 2), line.text, font=font, fill=shared.opts.font_color if line.is_active else color_inactive, anchor="mm", align="center")
|
||||
if not line.is_active:
|
||||
drawing.line((draw_x - line.size[0] // 2, draw_y + line.size[1] // 2, draw_x + line.size[0] // 2, draw_y + line.size[1] // 2), fill=color_inactive, width=4)
|
||||
draw_y += line.size[1] + line_spacing
|
||||
|
||||
fontsize = (width + height) // 25
|
||||
line_spacing = fontsize // 2
|
||||
fnt = get_font(fontsize)
|
||||
color_active = (0, 0, 0)
|
||||
color_inactive = (153, 153, 153)
|
||||
font = get_font(fontsize)
|
||||
color_inactive = (127, 127, 127)
|
||||
pad_left = 0 if sum([sum([len(line.text) for line in lines]) for lines in ver_texts]) == 0 else width * 3 // 4
|
||||
cols = im.width // width
|
||||
rows = im.height // height
|
||||
assert cols == len(hor_texts), f'bad number of horizontal texts: {len(hor_texts)}; must be {cols}'
|
||||
assert rows == len(ver_texts), f'bad number of vertical texts: {len(ver_texts)}; must be {rows}'
|
||||
calc_img = Image.new("RGB", (1, 1), "white")
|
||||
calc_img = Image.new("RGB", (1, 1), shared.opts.grid_background)
|
||||
calc_d = ImageDraw.Draw(calc_img)
|
||||
for texts, allowed_width in zip(hor_texts + ver_texts, [width] * len(hor_texts) + [pad_left] * len(ver_texts)):
|
||||
title_texts = [title] if title else [[GridAnnotation()]]
|
||||
for texts, allowed_width in zip(hor_texts + ver_texts + title_texts, [width] * len(hor_texts) + [pad_left] * len(ver_texts) + [(width+margin)*cols]):
|
||||
items = [] + texts
|
||||
texts.clear()
|
||||
for line in items:
|
||||
wrapped = wrap(calc_d, line.text, fnt, allowed_width)
|
||||
wrapped = wrap(calc_d, line.text, font, allowed_width)
|
||||
texts += [GridAnnotation(x, line.is_active) for x in wrapped]
|
||||
for line in texts:
|
||||
bbox = calc_d.multiline_textbbox((0, 0), line.text, font=fnt)
|
||||
bbox = calc_d.multiline_textbbox((0, 0), line.text, font=font)
|
||||
line.size = (bbox[2] - bbox[0], bbox[3] - bbox[1])
|
||||
line.allowed_width = allowed_width
|
||||
hor_text_heights = [sum([line.size[1] + line_spacing for line in lines]) - line_spacing for lines in hor_texts]
|
||||
ver_text_heights = [sum([line.size[1] + line_spacing for line in lines]) - line_spacing * len(lines) for lines in ver_texts]
|
||||
pad_top = 0 if sum(hor_text_heights) == 0 else max(hor_text_heights) + line_spacing * 2
|
||||
result = Image.new("RGB", (im.width + pad_left + margin * (cols-1), im.height + pad_top + margin * (rows-1)), "white")
|
||||
title_pad = 0
|
||||
if title:
|
||||
title_text_heights = [sum([line.size[1] + line_spacing for line in lines]) - line_spacing for lines in title_texts] # pylint: disable=unsubscriptable-object
|
||||
title_pad = 0 if sum(title_text_heights) == 0 else max(title_text_heights) + line_spacing * 2
|
||||
result = Image.new("RGB", (im.width + pad_left + margin * (cols-1), im.height + pad_top + title_pad + margin * (rows-1)), shared.opts.grid_background)
|
||||
for row in range(rows):
|
||||
for col in range(cols):
|
||||
cell = im.crop((width * col, height * row, width * (col+1), height * (row+1)))
|
||||
result.paste(cell, (pad_left + (width + margin) * col, pad_top + (height + margin) * row))
|
||||
result.paste(cell, (pad_left + (width + margin) * col, pad_top + title_pad + (height + margin) * row))
|
||||
d = ImageDraw.Draw(result)
|
||||
if title:
|
||||
x = pad_left + ((width+margin)*cols) / 2
|
||||
y = title_pad / 2 - title_text_heights[0] / 2
|
||||
draw_texts(d, x, y, title_texts[0], font, fontsize)
|
||||
for col in range(cols):
|
||||
x = pad_left + (width + margin) * col + width / 2
|
||||
y = pad_top / 2 - hor_text_heights[col] / 2
|
||||
draw_texts(d, x, y, hor_texts[col], fnt, fontsize)
|
||||
y = (pad_top / 2 - hor_text_heights[col] / 2) + title_pad
|
||||
draw_texts(d, x, y, hor_texts[col], font, fontsize)
|
||||
for row in range(rows):
|
||||
x = pad_left / 2
|
||||
y = pad_top + (height + margin) * row + height / 2 - ver_text_heights[row] / 2
|
||||
draw_texts(d, x, y, ver_texts[row], fnt, fontsize)
|
||||
y = (pad_top + (height + margin) * row + height / 2 - ver_text_heights[row] / 2) + title_pad
|
||||
draw_texts(d, x, y, ver_texts[row], font, fontsize)
|
||||
return result
|
||||
|
||||
|
||||
@@ -204,6 +212,7 @@ def draw_prompt_matrix(im, width, height, all_prompts, margin=0):
|
||||
|
||||
|
||||
def resize_image(resize_mode, im, width, height, upscaler_name=None, output_type='image'):
|
||||
# shared.log.debug(f'Image resize: mode={resize_mode} resolution={width}x{height} upscaler={upscaler_name}')
|
||||
"""
|
||||
Resizes an image with the specified resize_mode, width, and height.
|
||||
Args:
|
||||
@@ -245,7 +254,7 @@ def resize_image(resize_mode, im, width, height, upscaler_name=None, output_type
|
||||
src_w = width if ratio > src_ratio else im.width * height // im.height
|
||||
src_h = height if ratio <= src_ratio else im.height * width // im.width
|
||||
resized = resize(im, src_w, src_h)
|
||||
res = Image.new("RGB", (width, height))
|
||||
res = Image.new(im.mode, (width, height))
|
||||
res.paste(resized, box=(width // 2 - src_w // 2, height // 2 - src_h // 2))
|
||||
else:
|
||||
ratio = width / height
|
||||
@@ -253,7 +262,7 @@ def resize_image(resize_mode, im, width, height, upscaler_name=None, output_type
|
||||
src_w = width if ratio < src_ratio else im.width * height // im.height
|
||||
src_h = height if ratio >= src_ratio else im.height * width // im.width
|
||||
resized = resize(im, src_w, src_h)
|
||||
res = Image.new("RGB", (width, height))
|
||||
res = Image.new(im.mode, (width, height))
|
||||
res.paste(resized, box=(width // 2 - src_w // 2, height // 2 - src_h // 2))
|
||||
if ratio < src_ratio:
|
||||
fill_height = height // 2 - src_h // 2
|
||||
@@ -541,6 +550,8 @@ def atomically_save_image():
|
||||
entry = { 'id': idx, 'filename': filename, 'time': datetime.datetime.now().isoformat(), 'info': exifinfo }
|
||||
entries.append(entry)
|
||||
shared.writefile(entries, fn, mode='w')
|
||||
with open(os.path.join(paths.data_path, "params.txt"), "w", encoding="utf8") as file:
|
||||
file.write(exifinfo)
|
||||
save_queue.task_done()
|
||||
|
||||
|
||||
@@ -626,7 +637,7 @@ def safe_decode_string(s: bytes):
|
||||
return None
|
||||
|
||||
|
||||
def read_info_from_image(image):
|
||||
def read_info_from_image(image: Image):
|
||||
items = image.info or {}
|
||||
geninfo = items.pop('parameters', None)
|
||||
if geninfo is None:
|
||||
@@ -678,6 +689,14 @@ Negative prompt: {json_info["uc"]}
|
||||
Steps: {json_info["steps"]}, Sampler: {sampler}, CFG scale: {json_info["scale"]}, Seed: {json_info["seed"]}, Size: {image.width}x{image.height}, Clip skip: 2, ENSD: 31337"""
|
||||
except Exception as e:
|
||||
errors.display(e, 'novelai image parser')
|
||||
|
||||
try:
|
||||
items['width'] = image.width
|
||||
items['height'] = image.height
|
||||
items['mode'] = image.mode
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return geninfo, items
|
||||
|
||||
|
||||
|
||||
@@ -33,10 +33,7 @@ def BUILD_MAP_UNPACK(self, inst):
|
||||
)
|
||||
tmp_torch = sys.modules["torch"]
|
||||
tmp_torch.BUILD_MAP_UNPACK_WITH_CALL = BUILD_MAP_UNPACK
|
||||
|
||||
compiled_cache = {}
|
||||
max_openvino_partitions = 0
|
||||
partitioned_modules = {}
|
||||
|
||||
DEFAULT_OPENVINO_PYTHON_CONFIG = MappingProxyType(
|
||||
{
|
||||
@@ -180,13 +177,6 @@ def execute_cached(compiled_model, *args):
|
||||
result = [torch.from_numpy(res[out]) for out in compiled_model.outputs]
|
||||
return result
|
||||
|
||||
def openvino_clear_caches():
|
||||
global partitioned_modules
|
||||
global compiled_cache
|
||||
|
||||
compiled_cache.clear()
|
||||
partitioned_modules.clear()
|
||||
|
||||
def openvino_compile(gm: GraphModule, *args, model_hash_str: str = None, file_name=""):
|
||||
core = Core()
|
||||
|
||||
@@ -279,21 +269,20 @@ def openvino_execute(gm: GraphModule, *args, executor_parameters=None, partition
|
||||
"use_python_fusion_cache",
|
||||
DEFAULT_OPENVINO_PYTHON_CONFIG["use_python_fusion_cache"],
|
||||
)
|
||||
global compiled_cache
|
||||
|
||||
model_hash_str = executor_parameters.get("model_hash_str", None)
|
||||
if model_hash_str is not None:
|
||||
model_hash_str = model_hash_str + str(partition_id)
|
||||
|
||||
if use_cache and (partition_id in compiled_cache):
|
||||
compiled = compiled_cache[partition_id]
|
||||
if use_cache and (partition_id in shared.compiled_model_state.compiled_cache):
|
||||
compiled = shared.compiled_model_state.compiled_cache[partition_id]
|
||||
else:
|
||||
if (shared.compiled_model_state.cn_model != [] and file_name is not None
|
||||
and os.path.isfile(file_name + ".xml") and os.path.isfile(file_name + ".bin")):
|
||||
compiled = openvino_compile_cached_model(file_name, *args)
|
||||
else:
|
||||
compiled = openvino_compile(gm, *args, model_hash_str=model_hash_str, file_name=file_name)
|
||||
compiled_cache[partition_id] = compiled
|
||||
shared.compiled_model_state.compiled_cache[partition_id] = compiled
|
||||
|
||||
flat_args, _ = tree_flatten(args)
|
||||
ov_inputs = [a.detach().cpu().numpy() for a in flat_args]
|
||||
@@ -308,8 +297,6 @@ def openvino_execute(gm: GraphModule, *args, executor_parameters=None, partition
|
||||
def openvino_execute_partitioned(gm: GraphModule, *args, executor_parameters=None, file_name=""):
|
||||
executor_parameters = executor_parameters or DEFAULT_OPENVINO_PYTHON_CONFIG
|
||||
|
||||
global partitioned_modules
|
||||
|
||||
use_python_fusion_cache = executor_parameters.get(
|
||||
"use_python_fusion_cache",
|
||||
DEFAULT_OPENVINO_PYTHON_CONFIG["use_python_fusion_cache"],
|
||||
@@ -323,11 +310,11 @@ def openvino_execute_partitioned(gm: GraphModule, *args, executor_parameters=Non
|
||||
else:
|
||||
signature = signature + "_" + str(idx) + ":" + type(input_data).__name__ + ":val(" + str(input_data) + ")"
|
||||
|
||||
if signature not in partitioned_modules:
|
||||
partitioned_modules[signature] = partition_graph(gm, use_python_fusion_cache=use_python_fusion_cache,
|
||||
if signature not in shared.compiled_model_state.partitioned_modules:
|
||||
shared.compiled_model_state.partitioned_modules[signature] = partition_graph(gm, use_python_fusion_cache=use_python_fusion_cache,
|
||||
model_hash_str=model_hash_str, file_name=file_name)
|
||||
|
||||
return partitioned_modules[signature](*args)
|
||||
return shared.compiled_model_state.partitioned_modules[signature](*args)
|
||||
|
||||
def partition_graph(gm: GraphModule, use_python_fusion_cache: bool, model_hash_str: str = None, file_name=""):
|
||||
global max_openvino_partitions
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import ssl
|
||||
import time
|
||||
import datetime
|
||||
import logging
|
||||
from asyncio.exceptions import CancelledError
|
||||
import anyio
|
||||
@@ -14,6 +13,7 @@ from fastapi.encoders import jsonable_encoder
|
||||
from installer import log
|
||||
import modules.errors as errors
|
||||
|
||||
|
||||
errors.install()
|
||||
|
||||
|
||||
@@ -42,11 +42,12 @@ def setup_middleware(app: FastAPI, cmd_opts):
|
||||
duration = str(round(time.time() - ts, 4))
|
||||
res.headers["X-Process-Time"] = duration
|
||||
endpoint = req.scope.get('path', 'err')
|
||||
token = req.cookies.get("access-token") or req.cookies.get("access-token-unsecure")
|
||||
if (cmd_opts.api_log or cmd_opts.api_only) and endpoint.startswith('/sdapi'):
|
||||
if endpoint.endswith('/sdapi/v1/log'):
|
||||
if '/sdapi/v1/log' in endpoint:
|
||||
return res
|
||||
log.info('API {t} {code} {prot}/{ver} {method} {endpoint} {cli} {duration}'.format( # pylint: disable=consider-using-f-string, logging-format-interpolation
|
||||
t = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f"),
|
||||
log.info('API {user} {code} {prot}/{ver} {method} {endpoint} {cli} {duration}'.format( # pylint: disable=consider-using-f-string, logging-format-interpolation
|
||||
user = app.tokens.get(token),
|
||||
code = res.status_code,
|
||||
ver = req.scope.get('http_version', '0.0'),
|
||||
cli = req.scope.get('client', ('0:0.0.0', 0))[0],
|
||||
|
||||
@@ -274,6 +274,8 @@ def load_diffusers_models(model_path: str, command_path: str = None, clear=True)
|
||||
try:
|
||||
if "--" not in folder:
|
||||
continue
|
||||
if folder.endswith("-prior"):
|
||||
continue
|
||||
_, name = folder.split("--", maxsplit=1)
|
||||
name = name.replace("--", "/")
|
||||
folder = os.path.join(place, folder)
|
||||
|
||||
+9
-14
@@ -58,7 +58,7 @@ def apply_color_correction(correction, original_image):
|
||||
return image
|
||||
|
||||
|
||||
def apply_overlay(image, paste_loc, index, overlays):
|
||||
def apply_overlay(image: Image, paste_loc, index, overlays):
|
||||
if overlays is None or index >= len(overlays):
|
||||
return image
|
||||
overlay = overlays[index]
|
||||
@@ -686,6 +686,7 @@ def process_images(p: StableDiffusionProcessing) -> Processed:
|
||||
p.override_settings.pop(k, None)
|
||||
for k in p.override_settings.keys():
|
||||
stored_opts[k] = shared.opts.data.get(k, None) or shared.opts.data_labels[k].default
|
||||
res = None
|
||||
try:
|
||||
# if no checkpoint override or the override checkpoint can't be found, remove override entry and load opts checkpoint
|
||||
if p.override_settings.get('sd_model_checkpoint', None) is not None and modules.sd_models.checkpoint_aliases.get(p.override_settings.get('sd_model_checkpoint')) is None:
|
||||
@@ -851,10 +852,6 @@ def process_images_inner(p: StableDiffusionProcessing) -> Processed:
|
||||
modules.extra_networks.activate(p, extra_network_data)
|
||||
if p.scripts is not None and isinstance(p.scripts, modules.scripts.ScriptRunner):
|
||||
p.scripts.process_batch(p, batch_number=n, prompts=p.prompts, seeds=p.seeds, subseeds=p.subseeds)
|
||||
if n == 0:
|
||||
with open(os.path.join(modules.paths.data_path, "params.txt"), "w", encoding="utf8") as file:
|
||||
processed = Processed(p, [], p.seed, "")
|
||||
file.write(processed.infotext(p, 0))
|
||||
step_multiplier = 1
|
||||
sampler_config = modules.sd_samplers.find_sampler_config(p.sampler_name)
|
||||
step_multiplier = 2 if sampler_config and sampler_config.options.get("second_order", False) else 1
|
||||
@@ -1214,7 +1211,6 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
|
||||
def init(self, all_prompts, all_seeds, all_subseeds):
|
||||
if shared.backend == shared.Backend.DIFFUSERS and self.image_mask is not None:
|
||||
shared.sd_model = modules.sd_models.set_diffuser_pipe(self.sd_model, modules.sd_models.DiffusersTaskType.INPAINTING)
|
||||
# self.sd_model.dtype = self.sd_model.unet.dtype
|
||||
elif shared.backend == shared.Backend.DIFFUSERS and self.image_mask is None:
|
||||
shared.sd_model = modules.sd_models.set_diffuser_pipe(self.sd_model, modules.sd_models.DiffusersTaskType.IMAGE_2_IMAGE)
|
||||
|
||||
@@ -1229,6 +1225,7 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
|
||||
else:
|
||||
self.ops.append('img2img')
|
||||
crop_region = None
|
||||
|
||||
image_mask = self.image_mask
|
||||
if image_mask is not None:
|
||||
if type(image_mask) == list:
|
||||
@@ -1254,6 +1251,7 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
|
||||
self.mask_for_overlay = Image.fromarray(np_mask)
|
||||
self.overlay_images = []
|
||||
latent_mask = self.latent_mask if self.latent_mask is not None else image_mask
|
||||
|
||||
add_color_corrections = shared.opts.img2img_color_correction and self.color_corrections is None
|
||||
if add_color_corrections:
|
||||
self.color_corrections = []
|
||||
@@ -1284,14 +1282,12 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
|
||||
if crop_region is not None:
|
||||
image = image.crop(crop_region)
|
||||
image = images.resize_image(3, image, self.width, self.height)
|
||||
if shared.backend == shared.Backend.DIFFUSERS:
|
||||
unprocessed.append(image)
|
||||
self.init_images = [image] # assign early for diffusers
|
||||
if image_mask is not None:
|
||||
if self.inpainting_fill != 1:
|
||||
image = modules.masking.fill(image, latent_mask)
|
||||
if image_mask is not None and self.inpainting_fill != 1:
|
||||
image = modules.masking.fill(image, latent_mask)
|
||||
if add_color_corrections:
|
||||
self.color_corrections.append(setup_color_correction(image))
|
||||
if shared.backend == shared.Backend.DIFFUSERS:
|
||||
unprocessed.append(image) # assign early for diffusers
|
||||
image = np.array(image).astype(np.float32) / 255.0
|
||||
image = np.moveaxis(image, 2, 0)
|
||||
imgs.append(image)
|
||||
@@ -1308,8 +1304,7 @@ class StableDiffusionProcessingImg2Img(StableDiffusionProcessing):
|
||||
else:
|
||||
raise RuntimeError(f"bad number of images passed: {len(imgs)}; expecting {self.batch_size} or less")
|
||||
if shared.backend == shared.Backend.DIFFUSERS:
|
||||
# we've already set self.init_images and self.mask and we dont need any more processing
|
||||
return
|
||||
return # we've already set self.init_images and self.mask and we dont need any more processing
|
||||
|
||||
image = torch.from_numpy(batch_images)
|
||||
image = 2. * image - 1.
|
||||
|
||||
@@ -31,10 +31,10 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
|
||||
p.height = tgt_height
|
||||
p.width = tgt_width
|
||||
hypertile_set(p)
|
||||
if getattr(p, 'mask', None) is not None:
|
||||
p.mask = images.resize_image(1, p.mask, tgt_width, tgt_height, upscaler_name=None)
|
||||
if getattr(p, 'mask_for_overlay', None) is not None:
|
||||
p.mask_for_overlay = images.resize_image(1, p.mask_for_overlay, tgt_width, tgt_height, upscaler_name=None)
|
||||
if getattr(p, 'mask', None) is not None and p.mask.size != (tgt_width, tgt_height):
|
||||
p.mask = images.resize_image(1, p.mask, tgt_width, tgt_height, upscaler_name=None)
|
||||
if getattr(p, 'mask_for_overlay', None) is not None and p.mask_for_overlay.size != (tgt_width, tgt_height):
|
||||
p.mask_for_overlay = images.resize_image(1, p.mask_for_overlay, tgt_width, tgt_height, upscaler_name=None)
|
||||
|
||||
def hires_resize(latents): # input=latents output=pil
|
||||
latent_upscaler = shared.latent_upscale_modes.get(p.hr_upscaler, None)
|
||||
@@ -98,7 +98,7 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
|
||||
model.vae.to(devices.device)
|
||||
latents.to(model.vae.device)
|
||||
|
||||
upcast = (model.vae.dtype == torch.float16) and model.vae.config.force_upcast and hasattr(model, 'upcast_vae')
|
||||
upcast = (model.vae.dtype == torch.float16) and getattr(model.vae.config, 'force_upcast', False) and hasattr(model, 'upcast_vae')
|
||||
if upcast: # this is done by diffusers automatically if output_type != 'latent'
|
||||
model.upcast_vae()
|
||||
latents = latents.to(next(iter(model.vae.post_quant_conv.parameters())).dtype)
|
||||
@@ -183,6 +183,8 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
|
||||
negative_prompts = [negative_prompts]
|
||||
while len(negative_prompts) < len(prompts):
|
||||
negative_prompts.append(negative_prompts[-1])
|
||||
while len(prompts) < len(negative_prompts):
|
||||
prompts.append(prompts[-1])
|
||||
if type(prompts_2) is str:
|
||||
prompts_2 = [prompts_2]
|
||||
if type(prompts_2) is list:
|
||||
@@ -213,7 +215,30 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
|
||||
p.mask = TF.to_pil_image(torch.ones_like(TF.to_tensor(p.init_images[0]))).convert("L")
|
||||
width = 8 * math.ceil(p.init_images[0].width / 8)
|
||||
height = 8 * math.ceil(p.init_images[0].height / 8)
|
||||
|
||||
# option-1: use images as inputs
|
||||
task_args = {"image": p.init_images, "mask_image": p.mask, "strength": p.denoising_strength, "height": height, "width": width}
|
||||
|
||||
""" # option-2: preprocess images into latents using diffusers
|
||||
vae_scale_factor = 2 ** (len(model.vae.config.block_out_channels) - 1)
|
||||
image_processor = diffusers.image_processor.VaeImageProcessor(vae_scale_factor=vae_scale_factor)
|
||||
mask_processor = diffusers.image_processor.VaeImageProcessor(vae_scale_factor=vae_scale_factor, do_normalize=False, do_binarize=True, do_convert_grayscale=True)
|
||||
init_image = image_processor.preprocess(p.init_images[0], width=width, height=height)
|
||||
mask_image = mask_processor.preprocess(p.mask, width=width, height=height)
|
||||
task_args = {"image": p.init_images, "mask_image": p.mask, "strength": p.denoising_strength, "height": height, "width": width}
|
||||
"""
|
||||
|
||||
""" # option-2: manually assemble masked image latents
|
||||
masked_image_latents = []
|
||||
mask_image = TF.to_tensor(p.mask)
|
||||
for init_image in p.init_images:
|
||||
init_image = TF.to_tensor(p.init_images[0])
|
||||
masked_image = init_image * (mask_image > 0.5)
|
||||
masked_image_latents.append(torch.cat([masked_image, mask_image], dim=0))
|
||||
masked_image_latents = torch.stack(masked_image_latents, dim=0).to(shared.device)
|
||||
task_args = {"image": p.init_images, "mask_image": mask_image, "masked_image_latents": masked_image_latents, "strength": p.denoising_strength, "height": height, "width": width}
|
||||
"""
|
||||
|
||||
if model.__class__.__name__ == 'LatentConsistencyModelPipeline' and hasattr(p, 'init_images') and len(p.init_images) > 0:
|
||||
init_latents = [vae_encode(image, model=shared.sd_model, full_quality=p.full_quality).squeeze(dim=0) for image in p.init_images]
|
||||
init_latent = torch.stack(init_latents, dim=0).to(shared.device)
|
||||
@@ -223,7 +248,6 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
|
||||
return task_args
|
||||
|
||||
def set_pipeline_args(model, prompts: list, negative_prompts: list, prompts_2: typing.Optional[list]=None, negative_prompts_2: typing.Optional[list]=None, desc:str='', **kwargs):
|
||||
|
||||
if hasattr(model, "set_progress_bar_config"):
|
||||
model.set_progress_bar_config(bar_format='Progress {rate_fmt}{postfix} {bar} {percentage:3.0f}% {n_fmt}/{total_fmt} {elapsed} {remaining} ' + '\x1b[38;5;71m' + desc, ncols=80, colour='#327fba')
|
||||
args = {}
|
||||
@@ -306,6 +330,8 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
|
||||
clean['image'] = type(clean['image'])
|
||||
if 'mask_image' in clean:
|
||||
clean['mask_image'] = type(clean['mask_image'])
|
||||
if 'masked_image_latents' in clean:
|
||||
clean['masked_image_latents'] = type(clean['masked_image_latents'])
|
||||
if 'prompt' in clean:
|
||||
clean['prompt'] = len(clean['prompt'])
|
||||
if 'negative_prompt' in clean:
|
||||
@@ -382,34 +408,52 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
|
||||
use_denoise_start = bool(is_img2img and p.refiner_start > 0 and p.refiner_start < 1)
|
||||
|
||||
def calculate_base_steps():
|
||||
steps = p.steps
|
||||
if use_refiner_start:
|
||||
steps = (p.steps // (1.0 - p.refiner_start)) if shared.sd_model_type == 'sdxl' else p.steps
|
||||
if is_img2img:
|
||||
if use_denoise_start and shared.sd_model_type == 'sdxl':
|
||||
steps = p.steps // (1 - p.refiner_start)
|
||||
else:
|
||||
steps = (p.steps // p.denoising_strength) + 1
|
||||
elif use_refiner_start and shared.sd_model_type == 'sdxl':
|
||||
steps = (p.steps // p.refiner_start) + 1
|
||||
else:
|
||||
steps = p.steps
|
||||
|
||||
if os.environ.get('SD_STEPS_DEBUG', None) is not None:
|
||||
shared.log.debug(f'Steps: type=base input={p.steps} output={steps} refiner={use_refiner_start}')
|
||||
return max(2, int(steps))
|
||||
|
||||
def calculate_hires_steps():
|
||||
# denoising strength is applied to steps by diffusers so this is no-op
|
||||
# steps = (p.hr_second_pass_steps * p.denoising_strength) if p.hr_second_pass_steps > 0 else (p.steps * p.denoising_strength)
|
||||
steps = p.hr_second_pass_steps if p.hr_second_pass_steps > 0 else p.steps
|
||||
if p.hr_second_pass_steps > 0:
|
||||
steps = (p.hr_second_pass_steps // p.denoising_strength) + 1
|
||||
else:
|
||||
steps = (p.steps // p.denoising_strength) + 1
|
||||
|
||||
if os.environ.get('SD_STEPS_DEBUG', None) is not None:
|
||||
shared.log.debug(f'Steps: type=hires input={p.hr_second_pass_steps} output={steps} denoise={p.denoising_strength}')
|
||||
return max(2, int(steps))
|
||||
|
||||
def calculate_refiner_steps():
|
||||
# diffusers apply additional math to refiner steps, but we leave numbers as-is without correction
|
||||
if p.refiner_start > 0 and p.refiner_start < 1:
|
||||
steps = ((1 - p.refiner_start) * p.refiner_steps) if p.refiner_steps > 0 else ((1 - p.refiner_start) * p.steps)
|
||||
if "StableDiffusionXL" in shared.sd_refiner.__class__.__name__:
|
||||
if p.refiner_start > 0 and p.refiner_start < 1:
|
||||
#steps = p.refiner_steps // (1 - p.refiner_start) # SDXL with denoise strenght
|
||||
steps = (p.refiner_steps // (1 - p.refiner_start) // 2) + 1
|
||||
else:
|
||||
steps = (p.refiner_steps // p.denoising_strength) + 1
|
||||
else:
|
||||
steps = (p.denoising_strength * p.refiner_steps) if p.refiner_steps > 0 else (p.denoising_strength * p.steps)
|
||||
#steps = p.refiner_steps # SD 1.5 with denoise strenght
|
||||
steps = (p.refiner_steps * 1.25) + 1
|
||||
|
||||
if os.environ.get('SD_STEPS_DEBUG', None) is not None:
|
||||
shared.log.debug(f'Steps: type=refiner input={p.refiner_steps} output={steps} start={p.refiner_start} denoise={p.denoising_strength}')
|
||||
return max(2, int(steps))
|
||||
|
||||
# pipeline type is set earlier in processing, but check for sanity
|
||||
if sd_models.get_diffusers_task(shared.sd_model) != sd_models.DiffusersTaskType.TEXT_2_IMAGE and len(getattr(p, 'init_images' ,[])) == 0: # reset pipeline
|
||||
shared.sd_model = sd_models.set_diffuser_pipe(shared.sd_model, sd_models.DiffusersTaskType.TEXT_2_IMAGE)
|
||||
if sd_models.get_diffusers_task(shared.sd_model) != sd_models.DiffusersTaskType.TEXT_2_IMAGE and len(getattr(p, 'init_images' ,[])) == 0:
|
||||
shared.sd_model = sd_models.set_diffuser_pipe(shared.sd_model, sd_models.DiffusersTaskType.TEXT_2_IMAGE) # reset pipeline
|
||||
if hasattr(shared.sd_model, 'unet') and hasattr(shared.sd_model.unet, 'config') and hasattr(shared.sd_model.unet.config, 'in_channels') and shared.sd_model.unet.config.in_channels == 9:
|
||||
shared.sd_model = sd_models.set_diffuser_pipe(shared.sd_model, sd_models.DiffusersTaskType.INPAINTING) # force pipeline
|
||||
if len(getattr(p, 'init_images' ,[])) == 0:
|
||||
p.init_images = [TF.to_pil_image(torch.rand((3, p.height, p.width)))]
|
||||
base_args = set_pipeline_args(
|
||||
model=shared.sd_model,
|
||||
prompts=prompts,
|
||||
|
||||
@@ -14,7 +14,7 @@ from typing import List
|
||||
import lark
|
||||
import torch
|
||||
from compel import Compel
|
||||
from modules.shared import opts, log
|
||||
from modules.shared import opts, log, backend, Backend
|
||||
|
||||
# a prompt like this: "fantasy landscape with a [mountain:lake:0.25] and [an oak:a christmas tree:0.75][ in foreground::0.6][ in background:0.25] [shoddy:masterful:0.5]"
|
||||
# will be represented with prompt_schedule like this (assuming steps=100):
|
||||
@@ -320,7 +320,10 @@ def parse_prompt_attention(text):
|
||||
whitespace = ''
|
||||
else:
|
||||
re_attention = re_attention_v1
|
||||
text = text.replace('\\n', ' ')
|
||||
if backend == Backend.DIFFUSERS:
|
||||
text = text.replace('\n', ' BREAK ')
|
||||
else:
|
||||
text = text.replace('\n', ' ')
|
||||
whitespace = ' '
|
||||
|
||||
def multiply_range(start_position, multiplier):
|
||||
|
||||
@@ -69,7 +69,7 @@ def encode_prompts(pipeline, prompts: list, negative_prompts: list, clip_skip: t
|
||||
negative_embeds = []
|
||||
negative_pooleds = []
|
||||
for i in range(len(prompts)):
|
||||
prompt_embed, positive_pooled, negative_embed, negative_pooled = get_weighted_text_embeddings_sdxl(pipeline,prompts[i], negative_prompts[i], clip_skip)
|
||||
prompt_embed, positive_pooled, negative_embed, negative_pooled = get_weighted_text_embeddings(pipeline,prompts[i], negative_prompts[i], clip_skip)
|
||||
prompt_embeds.append(prompt_embed)
|
||||
positive_pooleds.append(positive_pooled)
|
||||
negative_embeds.append(negative_embed)
|
||||
@@ -113,6 +113,7 @@ def prepare_embedding_providers(pipe, clip_skip):
|
||||
embeddings_providers.append(embedding)
|
||||
return embeddings_providers
|
||||
|
||||
|
||||
def pad_to_same_length(embeds):
|
||||
try: #SDXL
|
||||
empty_embed = shared.sd_model.encode_prompt("")
|
||||
@@ -127,7 +128,8 @@ def pad_to_same_length(embeds):
|
||||
embeds[i] = embed
|
||||
return embeds
|
||||
|
||||
def get_weighted_text_embeddings_sdxl(pipe, prompt: str = "", neg_prompt: str = "", clip_skip: int = None):
|
||||
|
||||
def get_weighted_text_embeddings(pipe, prompt: str = "", neg_prompt: str = "", clip_skip: int = None):
|
||||
prompt_2 = prompt.split("TE2:")[-1]
|
||||
neg_prompt_2 = neg_prompt.split("TE2:")[-1]
|
||||
prompt = prompt.split("TE2:")[0]
|
||||
@@ -152,8 +154,20 @@ def get_weighted_text_embeddings_sdxl(pipe, prompt: str = "", neg_prompt: str =
|
||||
negative_pooled_prompt_embeds = None
|
||||
|
||||
for i in range(len(embedding_providers)):
|
||||
embed, ptokens = embedding_providers[i].get_embeddings_for_weighted_prompt_fragments(text_batch=[positives[i]], fragment_weights_batch=[positive_weights[i]], device=pipe.device, should_return_tokens=True)
|
||||
prompt_embeds.append(embed)
|
||||
# add BREAK keyword that splits the prompt into multiple fragments
|
||||
text = positives[i]
|
||||
weights = positive_weights[i]
|
||||
text.append('BREAK')
|
||||
weights.append(-1)
|
||||
provider_embed = []
|
||||
while 'BREAK' in text:
|
||||
pos = text.index('BREAK')
|
||||
embed, ptokens = embedding_providers[i].get_embeddings_for_weighted_prompt_fragments(text_batch=[text[:pos]], fragment_weights_batch=[weights[:pos]], device=pipe.device, should_return_tokens=True)
|
||||
provider_embed.append(embed)
|
||||
text = text[pos+1:]
|
||||
weights = weights[pos+1:]
|
||||
prompt_embeds.append(torch.cat(provider_embed, dim=1))
|
||||
# negative prompt has no keywords
|
||||
embed, ntokens = embedding_providers[i].get_embeddings_for_weighted_prompt_fragments(text_batch=[negatives[i]], fragment_weights_batch=[negative_weights[i]],device=pipe.device, should_return_tokens=True)
|
||||
negative_prompt_embeds.append(embed)
|
||||
|
||||
|
||||
@@ -178,14 +178,14 @@ class StableDiffusionModelHijack:
|
||||
except Exception as err:
|
||||
shared.log.warning(f"IPEX Optimize not supported: {err}")
|
||||
|
||||
if opts.cuda_compile and opts.cuda_compile_backend != 'none' and shared.backend == shared.Backend.ORIGINAL:
|
||||
if (opts.cuda_compile or opts.cuda_compile_vae or opts.cuda_compile_upscaler) and shared.opts.cuda_compile_backend != 'none' and shared.backend == shared.Backend.ORIGINAL:
|
||||
try:
|
||||
import logging
|
||||
shared.log.info(f"Compiling pipeline={m.model.__class__.__name__} mode={opts.cuda_compile_backend}")
|
||||
import torch._dynamo # pylint: disable=unused-import,redefined-outer-name
|
||||
if shared.opts.cuda_compile_backend == "openvino_fx":
|
||||
torch._dynamo.reset() # pylint: disable=protected-access
|
||||
from modules.intel.openvino import openvino_fx, openvino_clear_caches # pylint: disable=unused-import
|
||||
from modules.intel.openvino import openvino_fx, openvino_clear_caches # pylint: disable=unused-import, no-name-in-module
|
||||
openvino_clear_caches()
|
||||
torch._dynamo.eval_frame.check_if_dynamo_supported = lambda: True # pylint: disable=protected-access
|
||||
log_level = logging.WARNING if opts.cuda_compile_verbose else logging.CRITICAL # pylint: disable=protected-access
|
||||
@@ -202,6 +202,9 @@ class StableDiffusionModelHijack:
|
||||
shared.log.info("Model complilation done.")
|
||||
except Exception as err:
|
||||
shared.log.warning(f"Model compile not supported: {err}")
|
||||
finally:
|
||||
from installer import setup_logging
|
||||
setup_logging()
|
||||
|
||||
self.optimization_method = apply_optimizations()
|
||||
self.clip = m.cond_stage_model
|
||||
|
||||
+34
-71
@@ -20,8 +20,7 @@ import tomesd
|
||||
from transformers import logging as transformers_logging
|
||||
import ldm.modules.midas as midas
|
||||
from ldm.util import instantiate_from_config
|
||||
from modules import paths, shared, shared_items, shared_state, modelloader, devices, script_callbacks, sd_vae, sd_disable_initialization, errors, hashes, sd_models_config
|
||||
from modules.sd_hijack_inpainting import do_inpainting_hijack
|
||||
from modules import paths, shared, shared_items, shared_state, modelloader, devices, script_callbacks, sd_vae, sd_disable_initialization, errors, hashes, sd_models_config, sd_models_compile, sd_hijack_inpainting
|
||||
from modules.timer import Timer
|
||||
from modules.memstats import memory_stats
|
||||
from modules.paths import models_path, script_path
|
||||
@@ -119,18 +118,6 @@ class CheckpointInfo:
|
||||
return self.shorthash
|
||||
|
||||
|
||||
#Used by OpenVINO, can be used with TensorRT or Olive
|
||||
class CompiledModelState:
|
||||
def __init__(self):
|
||||
self.first_pass = True
|
||||
self.height = 512
|
||||
self.width = 512
|
||||
self.batch_size = 1
|
||||
self.partition_id = 0
|
||||
self.cn_model = []
|
||||
self.lora_model = []
|
||||
|
||||
|
||||
class NoWatermark:
|
||||
def apply_watermark(self, img):
|
||||
return img
|
||||
@@ -340,7 +327,7 @@ def read_metadata_from_safetensors(filename):
|
||||
if not os.path.isfile(sd_metadata_file):
|
||||
sd_metadata = {}
|
||||
else:
|
||||
sd_metadata = shared.readfile(sd_metadata_file)
|
||||
sd_metadata = shared.readfile(sd_metadata_file, lock=True)
|
||||
res = sd_metadata.get(filename, None)
|
||||
if res is not None:
|
||||
return res
|
||||
@@ -594,7 +581,7 @@ model_data = ModelData()
|
||||
|
||||
def change_backend():
|
||||
shared.log.info(f'Backend changed: {shared.backend}')
|
||||
shared.log.warning('Server restart required to apply all changes')
|
||||
shared.log.warning('Full server restart required to apply all changes')
|
||||
if shared.backend == shared.Backend.ORIGINAL:
|
||||
change_from = shared.Backend.DIFFUSERS
|
||||
else:
|
||||
@@ -673,51 +660,6 @@ def detect_pipeline(f: str, op: str = 'model'):
|
||||
return pipeline, guess
|
||||
|
||||
|
||||
def compile_diffusers(sd_model):
|
||||
try:
|
||||
if shared.opts.ipex_optimize:
|
||||
import intel_extension_for_pytorch as ipex # pylint: disable=import-error, unused-import
|
||||
sd_model.unet.training = False
|
||||
sd_model.unet = ipex.optimize(sd_model.unet, dtype=devices.dtype_unet, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
|
||||
if hasattr(sd_model, 'vae'):
|
||||
sd_model.vae.training = False
|
||||
sd_model.vae = ipex.optimize(sd_model.vae, dtype=devices.dtype_vae, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
|
||||
if hasattr(sd_model, 'movq'):
|
||||
sd_model.movq.training = False
|
||||
sd_model.movq = ipex.optimize(sd_model.movq, dtype=devices.dtype_vae, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
|
||||
shared.log.info("Applied IPEX Optimize.")
|
||||
except Exception as err:
|
||||
shared.log.warning(f"IPEX Optimize not supported: {err}")
|
||||
|
||||
try:
|
||||
if shared.opts.cuda_compile and shared.opts.cuda_compile_backend != 'none':
|
||||
shared.log.info(f"Compiling pipeline={sd_model.__class__.__name__} shape={8 * sd_model.unet.config.sample_size} mode={shared.opts.cuda_compile_backend}")
|
||||
import torch._dynamo # pylint: disable=unused-import,redefined-outer-name
|
||||
if shared.opts.cuda_compile_backend == "openvino_fx":
|
||||
torch._dynamo.reset() # pylint: disable=protected-access
|
||||
from modules.intel.openvino import openvino_fx, openvino_clear_caches # pylint: disable=unused-import
|
||||
openvino_clear_caches()
|
||||
torch._dynamo.eval_frame.check_if_dynamo_supported = lambda: True # pylint: disable=protected-access
|
||||
if shared.compiled_model_state is None:
|
||||
shared.compiled_model_state = CompiledModelState()
|
||||
shared.compiled_model_state.first_pass = True if not shared.opts.cuda_compile_precompile else False
|
||||
log_level = logging.WARNING if shared.opts.cuda_compile_verbose else logging.CRITICAL # pylint: disable=protected-access
|
||||
if hasattr(torch, '_logging'):
|
||||
torch._logging.set_logs(dynamo=log_level, aot=log_level, inductor=log_level) # pylint: disable=protected-access
|
||||
torch._dynamo.config.verbose = shared.opts.cuda_compile_verbose # pylint: disable=protected-access
|
||||
torch._dynamo.config.suppress_errors = shared.opts.cuda_compile_errors # pylint: disable=protected-access
|
||||
sd_model.unet = torch.compile(sd_model.unet, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph) # pylint: disable=attribute-defined-outside-init
|
||||
if hasattr(sd_model, 'vae'):
|
||||
sd_model.vae.decode = torch.compile(sd_model.vae.decode, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph) # pylint: disable=attribute-defined-outside-init
|
||||
if hasattr(sd_model, 'movq'):
|
||||
sd_model.movq.decode = torch.compile(sd_model.movq.decode, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph) # pylint: disable=attribute-defined-outside-init
|
||||
if shared.opts.cuda_compile_precompile:
|
||||
sd_model("dummy prompt")
|
||||
shared.log.info("Complilation done.")
|
||||
except Exception as err:
|
||||
shared.log.warning(f"Model compile not supported: {err}")
|
||||
|
||||
|
||||
def set_diffuser_options(sd_model, vae, op: str):
|
||||
if sd_model is None:
|
||||
shared.log.warning(f'{op} is not loaded')
|
||||
@@ -774,10 +716,8 @@ def set_diffuser_options(sd_model, vae, op: str):
|
||||
sd_model.vae = vae
|
||||
if shared.opts.diffusers_vae_upcast != 'default':
|
||||
if shared.opts.diffusers_vae_upcast == 'true':
|
||||
# sd_model.vae.config["force_upcast"] = True
|
||||
sd_model.vae.config.force_upcast = True
|
||||
else:
|
||||
# sd_model.vae.config["force_upcast"] = False
|
||||
sd_model.vae.config.force_upcast = False
|
||||
if shared.opts.no_half_vae:
|
||||
devices.dtype_vae = torch.float32
|
||||
@@ -785,6 +725,18 @@ def set_diffuser_options(sd_model, vae, op: str):
|
||||
shared.log.debug(f'Setting {op} VAE: name={sd_vae.loaded_vae_file} upcast={sd_model.vae.config.get("force_upcast", None)}')
|
||||
if shared.opts.cross_attention_optimization == "xFormers" and hasattr(sd_model, 'enable_xformers_memory_efficient_attention'):
|
||||
sd_model.enable_xformers_memory_efficient_attention()
|
||||
|
||||
if shared.opts.diffusers_eval:
|
||||
if hasattr(sd_model, "unet"):
|
||||
sd_model.unet.requires_grad_(False)
|
||||
sd_model.unet.eval()
|
||||
if hasattr(sd_model, "vae"):
|
||||
sd_model.vae.requires_grad_(False)
|
||||
sd_model.vae.eval()
|
||||
if hasattr(sd_model, "text_encoder"):
|
||||
sd_model.text_encoder.requires_grad_(False)
|
||||
sd_model.text_encoder.eval()
|
||||
|
||||
if shared.opts.opt_channelslast and hasattr(sd_model, 'unet'):
|
||||
shared.log.debug(f'Setting {op}: enable channels last')
|
||||
sd_model.unet.to(memory_format=torch.channels_last)
|
||||
@@ -899,15 +851,26 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No
|
||||
if model_type.startswith('Stable Diffusion'):
|
||||
diffusers_load_config['force_zeros_for_empty_prompt '] = shared.opts.diffusers_force_zeros
|
||||
diffusers_load_config['requires_aesthetics_score'] = shared.opts.diffusers_aesthetics_score
|
||||
diffusers_load_config['config_files'] = {
|
||||
'v1': 'configs/v1-inference.yaml',
|
||||
'v2': 'configs/v2-inference-768-v.yaml',
|
||||
'xl': 'configs/sd_xl_base.yaml',
|
||||
'xl_refiner': 'configs/sd_xl_refiner.yaml',
|
||||
}
|
||||
if 'inpainting' in checkpoint_info.path.lower():
|
||||
diffusers_load_config['config_files'] = {
|
||||
'v1': 'configs/v1-inpainting-inference.yaml',
|
||||
'v2': 'configs/v2-inference-768-v.yaml',
|
||||
'xl': 'configs/sd_xl_base.yaml',
|
||||
'xl_refiner': 'configs/sd_xl_refiner.yaml',
|
||||
}
|
||||
else:
|
||||
diffusers_load_config['config_files'] = {
|
||||
'v1': 'configs/v1-inference.yaml',
|
||||
'v2': 'configs/v2-inference-768-v.yaml',
|
||||
'xl': 'configs/sd_xl_base.yaml',
|
||||
'xl_refiner': 'configs/sd_xl_refiner.yaml',
|
||||
}
|
||||
if hasattr(pipeline, 'from_single_file'):
|
||||
diffusers_load_config['use_safetensors'] = True
|
||||
sd_model = pipeline.from_single_file(checkpoint_info.path, **diffusers_load_config)
|
||||
if sd_model is not None and hasattr(sd_model, 'unet') and hasattr(sd_model.unet, 'config') and 'inpainting' in checkpoint_info.path.lower():
|
||||
shared.log.debug('Model patch: type=inpaint')
|
||||
sd_model.unet.config.in_channels = 9
|
||||
elif hasattr(pipeline, 'from_ckpt'):
|
||||
sd_model = pipeline.from_ckpt(checkpoint_info.path, **diffusers_load_config)
|
||||
else:
|
||||
@@ -961,7 +924,7 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No
|
||||
elif not getattr(sd_model, 'has_accelerate', False):
|
||||
sd_model.to(devices.device)
|
||||
|
||||
compile_diffusers(sd_model)
|
||||
sd_models_compile.compile_diffusers(sd_model)
|
||||
|
||||
if sd_model is None:
|
||||
shared.log.error('Diffuser model not loaded')
|
||||
@@ -1091,7 +1054,7 @@ def load_model(checkpoint_info=None, already_loaded_state_dict=None, timer=None,
|
||||
current_checkpoint_info = model_data.sd_refiner.sd_checkpoint_info
|
||||
unload_model_weights(op=op)
|
||||
|
||||
do_inpainting_hijack()
|
||||
sd_hijack_inpainting.do_inpainting_hijack()
|
||||
devices.set_cuda_params()
|
||||
if already_loaded_state_dict is not None:
|
||||
state_dict = already_loaded_state_dict
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
import time
|
||||
import logging
|
||||
import torch
|
||||
from modules import shared, devices
|
||||
from installer import setup_logging
|
||||
|
||||
|
||||
#Used by OpenVINO, can be used with TensorRT or Olive
|
||||
class CompiledModelState:
|
||||
def __init__(self):
|
||||
self.first_pass = True
|
||||
self.height = 512
|
||||
self.width = 512
|
||||
self.batch_size = 1
|
||||
self.partition_id = 0
|
||||
self.cn_model = []
|
||||
self.lora_model = []
|
||||
self.lora_compile = False
|
||||
self.compiled_cache = {}
|
||||
self.partitioned_modules = {}
|
||||
|
||||
|
||||
def optimize_ipex(sd_model):
|
||||
try:
|
||||
t0 = time.time()
|
||||
import intel_extension_for_pytorch as ipex # pylint: disable=import-error, unused-import
|
||||
sd_model.unet.training = False
|
||||
sd_model.unet = ipex.optimize(sd_model.unet, dtype=devices.dtype_unet, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
|
||||
if hasattr(sd_model, 'vae'):
|
||||
sd_model.vae.training = False
|
||||
sd_model.vae = ipex.optimize(sd_model.vae, dtype=devices.dtype_vae, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
|
||||
if hasattr(sd_model, 'movq'):
|
||||
sd_model.movq.training = False
|
||||
sd_model.movq = ipex.optimize(sd_model.movq, dtype=devices.dtype_vae, inplace=True, weights_prepack=False) # pylint: disable=attribute-defined-outside-init
|
||||
t1 = time.time()
|
||||
shared.log.info(f"Model compile: mode=IPEX-optimize time={t1-t0:.2f}")
|
||||
except Exception as e:
|
||||
shared.log.warning(f"Model compile: task=IPEX-optimize error: {e}")
|
||||
|
||||
|
||||
def optimize_openvino():
|
||||
try:
|
||||
from modules.intel.openvino import openvino_fx # pylint: disable=unused-import
|
||||
torch._dynamo.eval_frame.check_if_dynamo_supported = lambda: True # pylint: disable=protected-access
|
||||
if shared.compiled_model_state is None:
|
||||
shared.compiled_model_state = CompiledModelState()
|
||||
else:
|
||||
if not shared.compiled_model_state.lora_compile:
|
||||
shared.compiled_model_state.lora_compile = False
|
||||
shared.compiled_model_state.lora_model = []
|
||||
shared.compiled_model_state.compiled_cache.clear()
|
||||
shared.compiled_model_state.partitioned_modules.clear()
|
||||
shared.compiled_model_state.first_pass = True if not shared.opts.cuda_compile_precompile else False
|
||||
except Exception as e:
|
||||
shared.log.warning(f"Model compile: task=OpenVINO: {e}")
|
||||
|
||||
|
||||
def compile_stablefast(sd_model):
|
||||
try:
|
||||
import sfast.compilers.stable_diffusion_pipeline_compiler as sf
|
||||
except Exception as e:
|
||||
shared.log.warning(f'Model compile using stable-fast: {e}')
|
||||
return sd_model
|
||||
config = sf.CompilationConfig.Default()
|
||||
try:
|
||||
import xformers # pylint: disable=unused-import
|
||||
config.enable_xformers = True
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
import triton # pylint: disable=unused-import
|
||||
config.enable_triton = True
|
||||
except Exception:
|
||||
pass
|
||||
import warnings
|
||||
warnings.filterwarnings("ignore", category=torch.jit.TracerWarning)
|
||||
config.enable_cuda_graph = shared.opts.cuda_compile_fullgraph
|
||||
config.enable_jit_freeze = shared.opts.diffusers_eval
|
||||
try:
|
||||
t0 = time.time()
|
||||
sd_model = sf.compile(sd_model, config)
|
||||
setup_logging() # compile messes with logging so reset is needed
|
||||
if shared.opts.cuda_compile_precompile:
|
||||
sd_model("dummy prompt")
|
||||
t1 = time.time()
|
||||
shared.log.info(f"Model compile: task=Stable-fast config={config.__dict__} time={t1-t0:.2f}")
|
||||
except Exception as e:
|
||||
shared.log.info(f"Model compile: task=Stable-fast error: {e}")
|
||||
return sd_model
|
||||
|
||||
|
||||
def compile_torch(sd_model):
|
||||
try:
|
||||
import torch._dynamo # pylint: disable=unused-import,redefined-outer-name
|
||||
torch._dynamo.reset() # pylint: disable=protected-access
|
||||
shared.log.debug(f"Model compile available backends: {torch._dynamo.list_backends()}") # pylint: disable=protected-access
|
||||
if shared.opts.ipex_optimize:
|
||||
optimize_ipex(sd_model)
|
||||
if shared.opts.cuda_compile_backend == "openvino_fx":
|
||||
optimize_openvino()
|
||||
log_level = logging.WARNING if shared.opts.cuda_compile_verbose else logging.CRITICAL # pylint: disable=protected-access
|
||||
if hasattr(torch, '_logging'):
|
||||
torch._logging.set_logs(dynamo=log_level, aot=log_level, inductor=log_level) # pylint: disable=protected-access
|
||||
torch._dynamo.config.verbose = shared.opts.cuda_compile_verbose # pylint: disable=protected-access
|
||||
torch._dynamo.config.suppress_errors = shared.opts.cuda_compile_errors # pylint: disable=protected-access
|
||||
t0 = time.time()
|
||||
if shared.opts.cuda_compile:
|
||||
sd_model.unet = torch.compile(sd_model.unet, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph)
|
||||
if shared.opts.cuda_compile_vae:
|
||||
if hasattr(sd_model, 'vae'):
|
||||
sd_model.vae.decode = torch.compile(sd_model.vae.decode, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph)
|
||||
if hasattr(sd_model, 'movq'):
|
||||
sd_model.movq.decode = torch.compile(sd_model.movq.decode, mode=shared.opts.cuda_compile_mode, backend=shared.opts.cuda_compile_backend, fullgraph=shared.opts.cuda_compile_fullgraph)
|
||||
setup_logging() # compile messes with logging so reset is needed
|
||||
if shared.opts.cuda_compile_precompile:
|
||||
sd_model("dummy prompt")
|
||||
t1 = time.time()
|
||||
shared.log.info(f"Model compile: time={t1-t0:.2f}")
|
||||
except Exception as e:
|
||||
shared.log.warning(f"Model compile error: {e}")
|
||||
return sd_model
|
||||
|
||||
|
||||
def compile_diffusers(sd_model):
|
||||
if not (shared.opts.cuda_compile or shared.opts.cuda_compile_vae or shared.opts.cuda_compile_upscaler):
|
||||
return
|
||||
if not hasattr(sd_model, 'unet') or not hasattr(sd_model.unet, 'config'):
|
||||
shared.log.warning('Model compile enabled but model has no Unet')
|
||||
return
|
||||
if shared.opts.cuda_compile_backend == 'none':
|
||||
shared.log.warning('Model compile enabled but no backend specified')
|
||||
return
|
||||
size = 8*getattr(sd_model.unet.config, 'sample_size', 0)
|
||||
shared.log.info(f"Model compile: pipeline={sd_model.__class__.__name__} shape={size} mode={shared.opts.cuda_compile_mode} backend={shared.opts.cuda_compile_backend} fullgraph={shared.opts.cuda_compile_fullgraph} unet={shared.opts.cuda_compile} vae={shared.opts.cuda_compile_vae} upscaler={shared.opts.cuda_compile_upscaler}")
|
||||
if shared.opts.cuda_compile_backend == 'stable-fast':
|
||||
sd_model = compile_stablefast(sd_model)
|
||||
else:
|
||||
sd_model = compile_torch(sd_model)
|
||||
return sd_model
|
||||
+10
-2
@@ -91,6 +91,8 @@ def refresh_vae_list():
|
||||
candidates += glob.iglob(path, recursive=True)
|
||||
for filepath in candidates:
|
||||
name = get_filename(filepath)
|
||||
if name == 'VAE':
|
||||
continue
|
||||
if shared.backend == shared.Backend.ORIGINAL:
|
||||
vae_dict[name] = filepath
|
||||
else:
|
||||
@@ -201,10 +203,16 @@ def load_vae_diffusers(model_file, vae_file=None, vae_source="unknown-source"):
|
||||
if os.path.isfile(vae_file):
|
||||
_pipeline, model_type = sd_models.detect_pipeline(model_file, 'vae')
|
||||
diffusers_load_config = { "config_file": paths.sd_default_config if model_type != 'Stable Diffusion XL' else os.path.join(paths.sd_configs_path, 'sd_xl_base.yaml')}
|
||||
vae = diffusers.AutoencoderKL.from_single_file(vae_file, **diffusers_load_config)
|
||||
if os.path.getsize(vae_file) > 1310944880:
|
||||
vae = diffusers.ConsistencyDecoderVAE.from_pretrained('openai/consistency-decoder', **diffusers_load_config) # consistency decoder does not have from single file, so we'll just download it once more
|
||||
else:
|
||||
vae = diffusers.AutoencoderKL.from_single_file(vae_file, **diffusers_load_config)
|
||||
vae = vae.to(devices.dtype_vae)
|
||||
else:
|
||||
vae = diffusers.AutoencoderKL.from_pretrained(vae_file, **diffusers_load_config)
|
||||
if 'consistency-decoder' in vae_file:
|
||||
vae = diffusers.ConsistencyDecoderVAE.from_pretrained(vae_file, **diffusers_load_config)
|
||||
else:
|
||||
vae = diffusers.AutoencoderKL.from_pretrained(vae_file, **diffusers_load_config)
|
||||
global loaded_vae_file # pylint: disable=global-statement
|
||||
loaded_vae_file = os.path.basename(vae_file)
|
||||
# shared.log.debug(f'Diffusers VAE config: {vae.config}')
|
||||
|
||||
+54
-33
@@ -181,51 +181,69 @@ def temp_disable_extensions():
|
||||
return disabled
|
||||
|
||||
|
||||
def readfile(filename, silent=False):
|
||||
def readfile(filename, silent=False, lock=False):
|
||||
data = {}
|
||||
lock_file = None
|
||||
locked = False
|
||||
try:
|
||||
if not os.path.exists(filename):
|
||||
return {}
|
||||
with fasteners.InterProcessLock(f"{filename}.lock"):
|
||||
with open(filename, "r", encoding="utf8") as file:
|
||||
data = json.load(file)
|
||||
if type(data) is str:
|
||||
data = json.loads(data)
|
||||
if not silent:
|
||||
log.debug(f'Read: file="{filename}" len={len(data)}')
|
||||
if lock:
|
||||
lock_file = fasteners.InterProcessReaderWriterLock(f"{filename}.lock", logger=log)
|
||||
locked = lock_file.acquire_read_lock(blocking=True, timeout=3)
|
||||
with open(filename, "r", encoding="utf8") as file:
|
||||
data = json.load(file)
|
||||
if type(data) is str:
|
||||
data = json.loads(data)
|
||||
if not silent:
|
||||
log.debug(f'Read: file="{filename}" json={len(data)} bytes={os.path.getsize(filename)}')
|
||||
except Exception as e:
|
||||
if not silent:
|
||||
log.error(f'Reading failed: {filename} {e}')
|
||||
return {}
|
||||
finally:
|
||||
if lock_file is not None:
|
||||
lock_file.release_read_lock()
|
||||
if locked and os.path.exists(f"{filename}.lock"):
|
||||
os.remove(f"{filename}.lock")
|
||||
return data
|
||||
|
||||
|
||||
def writefile(data, filename, mode='w', silent=False):
|
||||
lock = None
|
||||
locked = False
|
||||
|
||||
def default(obj):
|
||||
log.error(f"Saving: {filename} not a valid object: {obj}")
|
||||
return str(obj)
|
||||
|
||||
try:
|
||||
with fasteners.InterProcessLock(f"{filename}.lock"):
|
||||
# skipkeys=True, ensure_ascii=True, check_circular=True, allow_nan=True
|
||||
if type(data) == dict:
|
||||
output = json.dumps(data, indent=2, default=default)
|
||||
elif type(data) == list:
|
||||
output = json.dumps(data, indent=2, default=default)
|
||||
elif isinstance(data, object):
|
||||
simple = {}
|
||||
for k in data.__dict__:
|
||||
if data.__dict__[k] is not None:
|
||||
simple[k] = data.__dict__[k]
|
||||
output = json.dumps(simple, indent=2, default=default)
|
||||
else:
|
||||
raise ValueError('not a valid object')
|
||||
if not silent:
|
||||
log.debug(f'Save: file="{filename}" len={len(output)}')
|
||||
with open(filename, mode, encoding="utf8") as file:
|
||||
file.write(output)
|
||||
# skipkeys=True, ensure_ascii=True, check_circular=True, allow_nan=True
|
||||
if type(data) == dict:
|
||||
output = json.dumps(data, indent=2, default=default)
|
||||
elif type(data) == list:
|
||||
output = json.dumps(data, indent=2, default=default)
|
||||
elif isinstance(data, object):
|
||||
simple = {}
|
||||
for k in data.__dict__:
|
||||
if data.__dict__[k] is not None:
|
||||
simple[k] = data.__dict__[k]
|
||||
output = json.dumps(simple, indent=2, default=default)
|
||||
else:
|
||||
raise ValueError('not a valid object')
|
||||
lock = fasteners.InterProcessReaderWriterLock(f"{filename}.lock", logger=log)
|
||||
locked = lock.acquire_write_lock(blocking=True, timeout=3)
|
||||
with open(filename, mode, encoding="utf8") as file:
|
||||
file.write(output)
|
||||
if not silent:
|
||||
log.debug(f'Save: file="{filename}" json={len(data)} bytes={len(output)}')
|
||||
except Exception as e:
|
||||
log.error(f'Saving failed: {filename} {e}')
|
||||
finally:
|
||||
if lock is not None:
|
||||
lock.release_read_lock()
|
||||
if locked and os.path.exists(f"{filename}.lock"):
|
||||
os.remove(f"{filename}.lock")
|
||||
|
||||
|
||||
if devices.backend == "cpu":
|
||||
@@ -285,12 +303,13 @@ options_templates.update(options_section(('cuda', "Compute Settings"), {
|
||||
"torch_gc_threshold": OptionInfo(90, "VRAM usage threshold before running Torch GC to clear up VRAM", gr.Slider, {"minimum": 0, "maximum": 100, "step": 1}),
|
||||
|
||||
"cuda_compile_sep": OptionInfo("<h2>Model Compile</h2>", "", gr.HTML),
|
||||
"cuda_compile": OptionInfo(True if cmd_opts.use_openvino else False, "Enable model compile"),
|
||||
"cuda_compile_upscaler": OptionInfo(True if cmd_opts.use_openvino else False, "Enable upscaler compile"),
|
||||
"cuda_compile_backend": OptionInfo("openvino_fx" if cmd_opts.use_openvino else "none", "Model compile backend", gr.Radio, {"choices": ['none', 'inductor', 'cudagraphs', 'aot_ts_nvfuser', 'hidet', 'ipex', 'openvino_fx']}),
|
||||
"cuda_compile": OptionInfo(True if cmd_opts.use_openvino else False, "Compile UNet"),
|
||||
"cuda_compile_vae": OptionInfo(True if cmd_opts.use_openvino else False, "Compile VAE"),
|
||||
"cuda_compile_upscaler": OptionInfo(True if cmd_opts.use_openvino else False, "Compile upscaler"),
|
||||
"cuda_compile_backend": OptionInfo("openvino_fx" if cmd_opts.use_openvino else "none", "Model compile backend", gr.Radio, {"choices": ['none', 'inductor', 'cudagraphs', 'aot_ts_nvfuser', 'hidet', 'ipex', 'openvino_fx', 'stable-fast']}),
|
||||
"cuda_compile_mode": OptionInfo("default", "Model compile mode", gr.Radio, {"choices": ['default', 'reduce-overhead', 'max-autotune']}),
|
||||
"cuda_compile_fullgraph": OptionInfo(False, "Model compile fullgraph"),
|
||||
"cuda_compile_precompile": OptionInfo(False, "Model compile precompile"),
|
||||
"cuda_compile_precompile": OptionInfo(False if cmd_opts.use_openvino else True, "Model compile precompile"),
|
||||
"cuda_compile_verbose": OptionInfo(False, "Model compile verbose mode"),
|
||||
"cuda_compile_errors": OptionInfo(True, "Model compile suppress errors"),
|
||||
|
||||
@@ -346,7 +365,7 @@ options_templates.update(options_section(('diffusers', "Diffusers Settings"), {
|
||||
"diffusers_model_load_variant": OptionInfo("default", "Diffusers model loading variant", gr.Radio, {"choices": ['default', 'fp32', 'fp16']}),
|
||||
"diffusers_vae_load_variant": OptionInfo("default", "Diffusers VAE loading variant", gr.Radio, {"choices": ['default', 'fp32', 'fp16']}),
|
||||
"custom_diffusers_pipeline": OptionInfo('', 'Load custom Diffusers pipeline'),
|
||||
"diffusers_lora_loader": OptionInfo("diffusers" if cmd_opts.use_openvino else "sequential apply", "Diffusers LoRA loading variant", gr.Radio, {"choices": ['diffusers', 'sequential apply', 'merge and apply']}),
|
||||
"diffusers_eval": OptionInfo(True, "Force model eval"),
|
||||
"diffusers_force_zeros": OptionInfo(True, "Force zeros for prompts when empty"),
|
||||
"diffusers_aesthetics_score": OptionInfo(False, "Require aesthetics score"),
|
||||
"diffusers_force_inpaint": OptionInfo(False, 'Diffusers force inpaint pipeline'),
|
||||
@@ -400,6 +419,9 @@ options_templates.update(options_section(('saving-images', "Image Options"), {
|
||||
"grid_save": OptionInfo(True, "Always save all generated image grids"),
|
||||
"grid_format": OptionInfo('jpg', 'File format for grids', gr.Dropdown, {"choices": ["jpg", "png", "webp", "tiff", "jp2"]}),
|
||||
"n_rows": OptionInfo(-1, "Grid row count", gr.Slider, {"minimum": -1, "maximum": 16, "step": 1}),
|
||||
"grid_background": OptionInfo("#000000", "Grid background color", ui_components.FormColorPicker, {}),
|
||||
"font": OptionInfo("", "Font file"),
|
||||
"font_color": OptionInfo("#FFFFFF", "Font color", ui_components.FormColorPicker, {}),
|
||||
|
||||
"save_sep_options": OptionInfo("<h2>Intermediate Image Saving</h2>", "", gr.HTML),
|
||||
"save_init_img": OptionInfo(False, "Save copy of img2img init images"),
|
||||
@@ -451,7 +473,6 @@ options_templates.update(options_section(('ui', "User Interface"), {
|
||||
"disable_weights_auto_swap": OptionInfo(True, "Do not change selected model when reading generation parameters"),
|
||||
"send_seed": OptionInfo(True, "Send seed when sending prompt or image to other interface"),
|
||||
"send_size": OptionInfo(True, "Send size when sending prompt or image to another interface"),
|
||||
"font": OptionInfo("", "Font for image grids that have text"),
|
||||
"keyedit_precision_attention": OptionInfo(0.1, "Ctrl+up/down precision when editing (attention:1.1)", gr.Slider, {"minimum": 0.01, "maximum": 0.2, "step": 0.001, "visible": False}),
|
||||
"keyedit_precision_extra": OptionInfo(0.05, "Ctrl+up/down precision when editing <extra networks:0.9>", gr.Slider, {"minimum": 0.01, "maximum": 0.2, "step": 0.001, "visible": False}),
|
||||
"keyedit_delimiters": OptionInfo(".,\/!?%^*;:{}=`~()", "Ctrl+up/down word delimiters", gr.Textbox, { "visible": False }), # pylint: disable=anomalous-backslash-in-string
|
||||
@@ -698,7 +719,7 @@ class Options:
|
||||
log.debug(f'Created default config: {filename}')
|
||||
self.save(filename)
|
||||
return
|
||||
self.data = readfile(filename)
|
||||
self.data = readfile(filename, lock=True)
|
||||
if self.data.get('quicksettings') is not None and self.data.get('quicksettings_list') is None:
|
||||
self.data['quicksettings_list'] = [i.strip() for i in self.data.get('quicksettings').split(',')]
|
||||
unknown_settings = []
|
||||
|
||||
+10
-7
@@ -421,7 +421,7 @@ def create_ui(startup_timer = None):
|
||||
cfg_scale = gr.Slider(minimum=1.0, maximum=30.0, step=0.1, label='CFG scale', value=6.0, elem_id="txt2img_cfg_scale")
|
||||
clip_skip = gr.Slider(label='CLIP skip', value=1, minimum=1, maximum=14, step=1, elem_id='txt2img_clip_skip', interactive=True)
|
||||
with FormRow():
|
||||
image_cfg_scale = gr.Slider(minimum=1.1, maximum=30.0, step=0.1, label='Secondary CFG scale', value=6.0, elem_id="txt2img_image_cfg_scale")
|
||||
image_cfg_scale = gr.Slider(minimum=1.0, maximum=30.0, step=0.1, label='Secondary CFG scale', value=6.0, elem_id="txt2img_image_cfg_scale")
|
||||
diffusers_guidance_rescale = gr.Slider(minimum=0.0, maximum=1.0, step=0.05, label='Guidance rescale', value=0.7, elem_id="txt2img_image_cfg_rescale")
|
||||
with FormRow():
|
||||
full_quality = gr.Checkbox(label='Full quality', value=True, elem_id="txt2img_full_quality")
|
||||
@@ -711,8 +711,8 @@ def create_ui(startup_timer = None):
|
||||
|
||||
with gr.Accordion(open=False, label="Advanced", elem_classes=["small-accordion"], elem_id="img2img_advanced_group"):
|
||||
with FormRow():
|
||||
cfg_scale = gr.Slider(minimum=1.0, maximum=30.0, step=0.5, label='CFG scale', value=6.0, elem_id="img2img_cfg_scale")
|
||||
image_cfg_scale = gr.Slider(minimum=0, maximum=30.0, step=0.05, label='Image CFG scale', value=1.5, elem_id="img2img_image_cfg_scale")
|
||||
cfg_scale = gr.Slider(minimum=1.0, maximum=30.0, step=0.1, label='CFG scale', value=6.0, elem_id="img2img_cfg_scale")
|
||||
image_cfg_scale = gr.Slider(minimum=1.0, maximum=30.0, step=0.15, label='Image CFG scale', value=1.5, elem_id="img2img_image_cfg_scale")
|
||||
with FormRow():
|
||||
clip_skip = gr.Slider(label='CLIP skip', value=1, minimum=1, maximum=4, step=1, elem_id='img2img_clip_skip', interactive=True)
|
||||
diffusers_guidance_rescale = gr.Slider(minimum=0.0, maximum=1.0, step=0.05, label='Guidance rescale', value=0.7, elem_id="txt2img_image_cfg_rescale")
|
||||
@@ -729,7 +729,7 @@ def create_ui(startup_timer = None):
|
||||
with gr.Column():
|
||||
inpainting_mask_invert = gr.Radio(label='Mask mode', choices=['Inpaint masked', 'Inpaint not masked'], value='Inpaint masked', type="index", elem_id="img2img_mask_mode")
|
||||
with gr.Column():
|
||||
inpainting_fill = gr.Radio(label='Masked content', choices=['fill', 'original', 'latent noise', 'latent nothing'], value='original', type="index", elem_id="img2img_inpainting_fill")
|
||||
inpainting_fill = gr.Radio(label='Masked content', choices=['fill', 'original', 'noise', 'nothing'], value='original', type="index", elem_id="img2img_inpainting_fill")
|
||||
with FormRow():
|
||||
with gr.Column():
|
||||
inpaint_full_res = gr.Radio(label="Inpaint area", choices=["Whole picture", "Only masked"], type="index", value="Whole picture", elem_id="img2img_inpaint_full_res")
|
||||
@@ -1239,11 +1239,14 @@ def webpath(fn):
|
||||
|
||||
|
||||
def html_head():
|
||||
script_js = os.path.join(script_path, "javascript", "script.js")
|
||||
head = f'<script type="text/javascript" src="{webpath(script_js)}"></script>\n'
|
||||
head = ''
|
||||
main = ['script.js']
|
||||
for js in main:
|
||||
script_js = os.path.join(script_path, "javascript", js)
|
||||
head += f'<script type="text/javascript" src="{webpath(script_js)}"></script>\n'
|
||||
added = []
|
||||
for script in modules.scripts.list_scripts("javascript", ".js"):
|
||||
if script.path == script_js:
|
||||
if script.filename in main:
|
||||
continue
|
||||
head += f'<script type="text/javascript" src="{webpath(script.path)}"></script>\n'
|
||||
added.append(script.path)
|
||||
|
||||
@@ -329,10 +329,10 @@ class ExtraNetworksPage:
|
||||
else:
|
||||
files = listdir(os.path.dirname(path))
|
||||
fn = os.path.splitext(path)[0]
|
||||
preview_extensions = ["jpg", "jpeg", "png", "webp", "tiff", "jp2"]
|
||||
for file in [f'{fn}{mid}{ext}' for ext in preview_extensions for mid in ['.thumb.', '.', '.preview.']]:
|
||||
exts = ["jpg", "jpeg", "png", "webp", "tiff", "jp2"]
|
||||
for file in [f'{fn}{mid}{ext}' for ext in exts for mid in ['.thumb.', '.', '.preview.']]:
|
||||
if file in files:
|
||||
if '.thumb.' not in file:
|
||||
if 'Reference' not in file and '.thumb.' not in file:
|
||||
self.missing_thumbs.append(file)
|
||||
return file
|
||||
return 'html/card-no-preview.png'
|
||||
@@ -440,6 +440,7 @@ class ExtraNetworksUi:
|
||||
self.details_components: list = []
|
||||
self.last_item: dict = None
|
||||
self.last_page: ExtraNetworksPage = None
|
||||
self.state: gr.State = None
|
||||
|
||||
|
||||
def create_ui(container, button_parent, tabname, skip_indexing = False):
|
||||
@@ -505,18 +506,18 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
|
||||
desc = gr.Textbox('', show_label=False, lines=8, placeholder="Extra network description...")
|
||||
ui.details_components.append(desc)
|
||||
with gr.Row():
|
||||
btn_save_desc = gr.Button('Save', elem_classes=['small-button'])
|
||||
btn_delete_desc = gr.Button('Delete', elem_classes=['small-button'])
|
||||
btn_close_info = gr.Button('Close', elem_classes=['small-button'])
|
||||
btn_close_info.click(fn=lambda: gr.update(visible=False), inputs=[], outputs=[ui.details])
|
||||
btn_save_desc = gr.Button('Save', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_save_desc')
|
||||
btn_delete_desc = gr.Button('Delete', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_delete_desc')
|
||||
btn_close_desc = gr.Button('Close', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_close_desc')
|
||||
btn_close_desc.click(fn=lambda: gr.update(visible=False), _js='refeshDetailsEN', inputs=[], outputs=[ui.details])
|
||||
with gr.Tab('Model metadata'):
|
||||
info = gr.JSON({}, show_label=False)
|
||||
ui.details_components.append(info)
|
||||
with gr.Row():
|
||||
btn_save_info = gr.Button('Save', elem_classes=['small-button'])
|
||||
btn_delete_info = gr.Button('Delete', elem_classes=['small-button'])
|
||||
btn_close_info = gr.Button('Close', elem_classes=['small-button'])
|
||||
btn_close_info.click(fn=lambda: gr.update(visible=False), inputs=[], outputs=[ui.details])
|
||||
btn_save_info = gr.Button('Save', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_save_info')
|
||||
btn_delete_info = gr.Button('Delete', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_delete_info')
|
||||
btn_close_info = gr.Button('Close', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_close_info')
|
||||
btn_close_info.click(fn=lambda: gr.update(visible=False), _js='refeshDetailsEN', inputs=[], outputs=[ui.details])
|
||||
with gr.Tab('Embedded metadata'):
|
||||
meta = gr.JSON({}, show_label=False)
|
||||
ui.details_components.append(meta)
|
||||
@@ -559,7 +560,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
|
||||
tab.select(ui_tab_change, _js="getENActivePage", inputs=[ui.button_details], outputs=[ui.button_scan, ui.button_save, ui.button_model])
|
||||
# ui.tabs.change(fn=ui_tab_change, inputs=[], outputs=[ui.button_scan, ui.button_save])
|
||||
|
||||
def fn_save_img():
|
||||
def fn_save_img(image):
|
||||
if ui.last_item is None or ui.last_item.local_preview is None:
|
||||
return 'html/card-no-preview.png'
|
||||
images = list(ui.gallery.temp_files) # gallery cannot be used as input component so looking at most recently registered temp files
|
||||
@@ -572,7 +573,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
|
||||
except Exception as e:
|
||||
shared.log.error(f'Extra network error opening image: item={ui.last_item.name} {e}')
|
||||
return 'html/card-no-preview.png'
|
||||
fn_delete_img()
|
||||
fn_delete_img(image)
|
||||
if image.width > 512 or image.height > 512:
|
||||
image = image.convert('RGB')
|
||||
image.thumbnail((512, 512), Image.HAMMING)
|
||||
@@ -583,7 +584,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
|
||||
shared.log.error(f'Extra network save image: item={ui.last_item.name} filename="{ui.last_item.local_preview}" {e}')
|
||||
return image
|
||||
|
||||
def fn_delete_img():
|
||||
def fn_delete_img(_image):
|
||||
preview_extensions = ["jpg", "jpeg", "png", "webp", "tiff", "jp2"]
|
||||
fn = os.path.splitext(ui.last_item.filename)[0]
|
||||
for file in [f'{fn}{mid}{ext}' for ext in preview_extensions for mid in ['.thumb.', '.preview.', '.']]:
|
||||
@@ -633,12 +634,12 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
|
||||
return ''
|
||||
return info
|
||||
|
||||
btn_save_img.click(fn=fn_save_img, inputs=[], outputs=[img])
|
||||
btn_delete_img.click(fn=fn_delete_img, inputs=[], outputs=[img])
|
||||
btn_save_desc.click(fn=fn_save_desc, inputs=[desc], outputs=[desc])
|
||||
btn_delete_desc.click(fn=fn_delete_desc, inputs=[desc], outputs=[desc])
|
||||
btn_save_info.click(fn=fn_save_info, inputs=[info], outputs=[info])
|
||||
btn_delete_info.click(fn=fn_delete_info, inputs=[info], outputs=[info])
|
||||
btn_save_img.click(fn=fn_save_img, _js='closeDetailsEN', inputs=[img], outputs=[img])
|
||||
btn_delete_img.click(fn=fn_delete_img, _js='closeDetailsEN', inputs=[img], outputs=[img])
|
||||
btn_save_desc.click(fn=fn_save_desc, _js='closeDetailsEN', inputs=[desc], outputs=[desc])
|
||||
btn_delete_desc.click(fn=fn_delete_desc, _js='closeDetailsEN', inputs=[desc], outputs=[desc])
|
||||
btn_save_info.click(fn=fn_save_info, _js='closeDetailsEN', inputs=[info], outputs=[info])
|
||||
btn_delete_info.click(fn=fn_delete_info, _js='closeDetailsEN', inputs=[info], outputs=[info])
|
||||
|
||||
def show_details(text, img, desc, info, meta, params):
|
||||
page, item = get_item(state, params)
|
||||
@@ -797,7 +798,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
|
||||
shared.log.debug(f'Extra networks: {msg}')
|
||||
return msg
|
||||
|
||||
dummy_state = gr.State(value=False) # pylint: disable=abstract-class-instantiated
|
||||
dummy = gr.State(value=False) # pylint: disable=abstract-class-instantiated
|
||||
button_parent.click(fn=toggle_visibility, inputs=[ui.visible], outputs=[ui.visible, container, button_parent])
|
||||
ui.button_close.click(fn=toggle_visibility, inputs=[ui.visible], outputs=[ui.visible, container])
|
||||
ui.button_sort.click(fn=ui_sort_cards, _js='sortExtraNetworks', inputs=[ui.search], outputs=[ui.description])
|
||||
@@ -806,7 +807,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
|
||||
ui.button_scan.click(fn=ui_scan_click, _js='getENActivePage', inputs=[ui.search], outputs=ui.pages)
|
||||
ui.button_save.click(fn=ui_save_click, inputs=[], outputs=ui.details_components + [ui.details])
|
||||
ui.button_quicksave.click(fn=ui_quicksave_click, _js="() => prompt('Prompt name', '')", inputs=[ui.search], outputs=[])
|
||||
ui.button_details.click(show_details, _js="getCardDetails", inputs=ui.details_components + [dummy_state], outputs=ui.details_components + [ui.details])
|
||||
ui.button_details.click(show_details, _js="getCardDetails", inputs=ui.details_components + [dummy], outputs=ui.details_components + [ui.details])
|
||||
ui.state.change(state_change, inputs=[ui.state], outputs=[])
|
||||
return ui
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ from collections import namedtuple
|
||||
from pathlib import Path
|
||||
import gradio as gr
|
||||
from PIL import Image, PngImagePlugin
|
||||
from modules import shared, errors
|
||||
from modules import shared, errors, paths
|
||||
|
||||
|
||||
Savedfile = namedtuple("Savedfile", ["name"])
|
||||
@@ -69,6 +69,10 @@ def pil_to_temp_file(self, img: Image, dir: str, format="png") -> str: # pylint:
|
||||
name = tmp.name
|
||||
img.save(name, pnginfo=(metadata if use_metadata else None))
|
||||
shared.log.debug(f'Saving temp: image="{name}"')
|
||||
params = ', '.join([f'{k}: {v}' for k, v in img.info.items()])
|
||||
params = params[12:] if params.startswith('parameters: ') else params
|
||||
with open(os.path.join(paths.data_path, "params.txt"), "w", encoding="utf8") as file:
|
||||
file.write(params)
|
||||
return name
|
||||
|
||||
|
||||
|
||||
+2
-13
@@ -66,7 +66,6 @@ class Upscaler:
|
||||
scalers.append(scaler)
|
||||
loaded.append(file_name)
|
||||
modules.shared.log.debug(f'Upscaler type={self.name} folder="{folder}" model="{model_name}" path="{file_name}"')
|
||||
print(f'Upscaler type={self.name} folder="{folder}" model="{model_name}" path="{file_name}"')
|
||||
|
||||
def find_scalers(self):
|
||||
scalers = []
|
||||
@@ -226,20 +225,12 @@ def compile_upscaler(model, name=""):
|
||||
modules.shared.log.info(f"Upscaler Compiling: {name} mode={modules.shared.opts.cuda_compile_backend}")
|
||||
import logging
|
||||
import torch._dynamo # pylint: disable=unused-import,redefined-outer-name
|
||||
use_old_compiled_model_state = False
|
||||
|
||||
if modules.shared.opts.cuda_compile_backend == "openvino_fx":
|
||||
from modules.intel.openvino import openvino_fx, openvino_clear_caches # pylint: disable=unused-import
|
||||
from modules.sd_models import CompiledModelState
|
||||
|
||||
openvino_clear_caches()
|
||||
from modules.intel.openvino import openvino_fx # pylint: disable=unused-import
|
||||
from modules.sd_models_compile import CompiledModelState
|
||||
torch._dynamo.eval_frame.check_if_dynamo_supported = lambda: True # pylint: disable=protected-access
|
||||
|
||||
if modules.shared.compiled_model_state is not None:
|
||||
use_old_compiled_model_state = True
|
||||
old_compiled_model_state = modules.shared.compiled_model_state
|
||||
modules.shared.compiled_model_state = CompiledModelState()
|
||||
|
||||
log_level = logging.WARNING if modules.shared.opts.cuda_compile_verbose else logging.CRITICAL # pylint: disable=protected-access
|
||||
if hasattr(torch, '_logging'):
|
||||
torch._logging.set_logs(dynamo=log_level, aot=log_level, inductor=log_level) # pylint: disable=protected-access
|
||||
@@ -248,8 +239,6 @@ def compile_upscaler(model, name=""):
|
||||
torch._dynamo.config.suppress_errors = modules.shared.opts.cuda_compile_errors # pylint: disable=protected-access
|
||||
model = torch.compile(model, mode=modules.shared.opts.cuda_compile_mode, backend=modules.shared.opts.cuda_compile_backend, fullgraph=modules.shared.opts.cuda_compile_fullgraph) # pylint: disable=attribute-defined-outside-init
|
||||
|
||||
if use_old_compiled_model_state:
|
||||
modules.shared.compiled_model_state = old_compiled_model_state
|
||||
modules.shared.log.info("Upscaler: Complilation done.")
|
||||
except Exception as err:
|
||||
modules.shared.log.warning(f"Model compile not supported: {err}")
|
||||
|
||||
+2
-2
@@ -54,14 +54,14 @@ opencv-python-headless==4.7.0.72
|
||||
diffusers==0.23.0
|
||||
einops==0.4.1
|
||||
gradio==3.43.2
|
||||
huggingface_hub==0.18.0
|
||||
huggingface_hub==0.19.2
|
||||
numexpr==2.8.4
|
||||
numpy==1.24.4
|
||||
numba==0.57.1
|
||||
pandas==1.5.3
|
||||
protobuf==3.20.3
|
||||
pytorch_lightning==1.9.4
|
||||
transformers==4.34.1
|
||||
transformers==4.35.1
|
||||
tomesd==0.1.3
|
||||
urllib3==1.26.15
|
||||
Pillow==9.5.0
|
||||
|
||||
+45
-26
@@ -1,4 +1,4 @@
|
||||
# pylint: disable=unused-argument, attribute-defined-outside-init
|
||||
# pylint: disable=unused-argument
|
||||
|
||||
import re
|
||||
import csv
|
||||
@@ -236,29 +236,29 @@ axis_options = [
|
||||
AxisOption("Clip skip", int, apply_clip_skip),
|
||||
AxisOption("Denoising strength", float, apply_field("denoising_strength")),
|
||||
AxisOption("Prompt order", str_permutations, apply_order, fmt=format_value_join_list),
|
||||
AxisOption("Model dictionary", str, apply_dict, fmt=format_value, cost=1.0, choices=lambda: ['None'] + list(sd_models.checkpoints_list)),
|
||||
AxisOptionImg2Img("Image mask weight", float, apply_field("inpainting_mask_weight")),
|
||||
AxisOption("[Postprocess] Upscaler", str, apply_upscaler, choices=lambda: [x.name for x in shared.sd_upscalers][1:]),
|
||||
AxisOption("[Postprocess] Face restore", str, apply_face_restore, fmt=format_value),
|
||||
AxisOptionImg2Img("Image mask weight", float, apply_field("inpainting_mask_weight")),
|
||||
AxisOption("Model dictionary", str, apply_dict, fmt=format_value, cost=1.0, choices=lambda: ['None'] + list(sd_models.checkpoints_list)),
|
||||
AxisOption("[Sampler] sigma min", float, apply_field("s_min")),
|
||||
AxisOption("[Sampler] sigma max", float, apply_field("s_max")),
|
||||
AxisOption("[Sampler] sigma tmin", float, apply_field("s_tmin")),
|
||||
AxisOption("[Sampler] sigma tmax", float, apply_field("s_tmax")),
|
||||
AxisOption("[Sampler] sigma Churn", float, apply_field("s_churn")),
|
||||
AxisOption("[Sampler] sigma noise", float, apply_field("s_noise")),
|
||||
AxisOption("[Sampler] eta", float, apply_field("eta")),
|
||||
AxisOption("[Sampler] solver order", int, apply_setting("schedulers_solver_order")),
|
||||
AxisOption("[Second pass] upscaler", str, apply_field("hr_upscaler"), choices=lambda: [*shared.latent_upscale_modes, *[x.name for x in shared.sd_upscalers]]),
|
||||
AxisOption("[Second pass] sampler", str, apply_latent_sampler, fmt=format_value, confirm=confirm_samplers, choices=lambda: [x.name for x in sd_samplers.samplers]),
|
||||
AxisOption("[Second pass] denoising Strength", float, apply_field("denoising_strength")),
|
||||
AxisOption("[Second pass] hires steps", int, apply_field("hr_second_pass_steps")),
|
||||
AxisOption("[Sampler] Sigma min", float, apply_field("s_min")),
|
||||
AxisOption("[Sampler] Sigma max", float, apply_field("s_max")),
|
||||
AxisOption("[Sampler] Sigma tmin", float, apply_field("s_tmin")),
|
||||
AxisOption("[Sampler] Sigma tmax", float, apply_field("s_tmax")),
|
||||
AxisOption("[Sampler] Sigma Churn", float, apply_field("s_churn")),
|
||||
AxisOption("[Sampler] Sigma noise", float, apply_field("s_noise")),
|
||||
AxisOption("[Sampler] ETA", float, apply_field("eta")),
|
||||
AxisOption("[Sampler] Solver order", int, apply_setting("schedulers_solver_order")),
|
||||
AxisOption("[Second pass] Upscaler", str, apply_field("hr_upscaler"), choices=lambda: [*shared.latent_upscale_modes, *[x.name for x in shared.sd_upscalers]]),
|
||||
AxisOption("[Second pass] Sampler", str, apply_latent_sampler, fmt=format_value, confirm=confirm_samplers, choices=lambda: [x.name for x in sd_samplers.samplers]),
|
||||
AxisOption("[Second pass] Denoising Strength", float, apply_field("denoising_strength")),
|
||||
AxisOption("[Second pass] Hires steps", int, apply_field("hr_second_pass_steps")),
|
||||
AxisOption("[Second pass] CFG scale", float, apply_field("image_cfg_scale")),
|
||||
AxisOption("[Second pass] guidance rescale", float, apply_field("diffusers_guidance_rescale")),
|
||||
AxisOption("[Refiner] model", str, apply_refiner, fmt=format_value, cost=1.0, choices=lambda: sorted(sd_models.checkpoints_list)),
|
||||
AxisOption("[Refiner] refiner start", float, apply_field("refiner_start")),
|
||||
AxisOption("[Refiner] refiner steps", float, apply_field("refiner_steps")),
|
||||
AxisOption("[TOME] Token merging ratio (txt2img)", float, apply_override('token_merging_ratio')),
|
||||
AxisOption("[TOME] Token merging ratio (hires)", float, apply_override('token_merging_ratio_hr')),
|
||||
AxisOption("[Second pass] Guidance rescale", float, apply_field("diffusers_guidance_rescale")),
|
||||
AxisOption("[Refiner] Model", str, apply_refiner, fmt=format_value, cost=1.0, choices=lambda: ['None'] + sorted(sd_models.checkpoints_list)),
|
||||
AxisOption("[Refiner] Refiner start", float, apply_field("refiner_start")),
|
||||
AxisOption("[Refiner] Refiner steps", float, apply_field("refiner_steps")),
|
||||
AxisOption("[ToMe] Token merging ratio (txt2img)", float, apply_override('token_merging_ratio')),
|
||||
AxisOption("[ToMe] Token merging ratio (hires)", float, apply_override('token_merging_ratio_hr')),
|
||||
AxisOption("[FreeU] 1st stage backbone factor", float, apply_setting('freeu_b1')),
|
||||
AxisOption("[FreeU] 2nd stage backbone factor", float, apply_setting('freeu_b2')),
|
||||
AxisOption("[FreeU] 1st stage skip factor", float, apply_setting('freeu_s1')),
|
||||
@@ -345,10 +345,10 @@ def draw_xyz_grid(p, xs, ys, zs, x_labels, y_labels, z_labels, cell, draw_legend
|
||||
for i in range(z_count):
|
||||
start_index = (i * len(xs) * len(ys)) + i
|
||||
end_index = start_index + len(xs) * len(ys)
|
||||
if not no_grid and images.check_grid_size(processed_result.images[start_index:end_index]):
|
||||
if (not no_grid or include_sub_grids) and images.check_grid_size(processed_result.images[start_index:end_index]):
|
||||
grid = images.image_grid(processed_result.images[start_index:end_index], rows=len(ys))
|
||||
if draw_legend:
|
||||
grid = images.draw_grid_annotations(grid, processed_result.images[start_index].size[0], processed_result.images[start_index].size[1], hor_texts, ver_texts, margin_size)
|
||||
grid = images.draw_grid_annotations(grid, processed_result.images[start_index].size[0], processed_result.images[start_index].size[1], hor_texts, ver_texts, margin_size, title=title_texts[i])
|
||||
processed_result.images.insert(i, grid)
|
||||
processed_result.all_prompts.insert(i, processed_result.all_prompts[start_index])
|
||||
processed_result.all_seeds.insert(i, processed_result.all_seeds[start_index])
|
||||
@@ -357,7 +357,7 @@ def draw_xyz_grid(p, xs, ys, zs, x_labels, y_labels, z_labels, cell, draw_legend
|
||||
if not no_grid and images.check_grid_size(processed_result.images[:z_count]):
|
||||
z_grid = images.image_grid(processed_result.images[:z_count], rows=1)
|
||||
if draw_legend:
|
||||
z_grid = images.draw_grid_annotations(z_grid, sub_grid_size[0], sub_grid_size[1], title_texts, [[images.GridAnnotation()]])
|
||||
z_grid = images.draw_grid_annotations(z_grid, sub_grid_size[0], sub_grid_size[1], [[images.GridAnnotation()] for _ in z_labels], [[images.GridAnnotation()]])
|
||||
processed_result.images.insert(0, z_grid)
|
||||
#processed_result.all_prompts.insert(0, processed_result.all_prompts[0])
|
||||
#processed_result.all_seeds.insert(0, processed_result.all_seeds[0])
|
||||
@@ -366,6 +366,14 @@ def draw_xyz_grid(p, xs, ys, zs, x_labels, y_labels, z_labels, cell, draw_legend
|
||||
|
||||
|
||||
class SharedSettingsStackHelper(object):
|
||||
vae = None
|
||||
schedulers_solver_order = None
|
||||
token_merging_ratio_hr = None
|
||||
token_merging_ratio = None
|
||||
sd_model_checkpoint = None
|
||||
sd_model_dict = None
|
||||
sd_vae_checkpoint = None
|
||||
|
||||
def __enter__(self):
|
||||
#Save overridden settings so they can be restored later.
|
||||
self.vae = shared.opts.sd_vae
|
||||
@@ -398,6 +406,8 @@ re_range_count = re.compile(r"\s*([+-]?\s*\d+)\s*-\s*([+-]?\s*\d+)(?:\s*\[(\d+)\
|
||||
re_range_count_float = re.compile(r"\s*([+-]?\s*\d+(?:.\d*)?)\s*-\s*([+-]?\s*\d+(?:.\d*)?)(?:\s*\[(\d+(?:.\d*)?)\s*])?\s*")
|
||||
|
||||
class Script(scripts.Script):
|
||||
current_axis_options = []
|
||||
|
||||
def title(self):
|
||||
return "X/Y/Z Grid"
|
||||
|
||||
@@ -699,9 +709,13 @@ class Script(scripts.Script):
|
||||
z_count = len(zs)
|
||||
processed.infotexts[:1+z_count] = grid_infotext[:1+z_count] # Set the grid infotexts to the real ones with extra_generation_params (1 main grid + z_count sub-grids)
|
||||
if not include_lone_images:
|
||||
processed.images = processed.images[:z_count+1] # Don't need sub-images anymore, drop from list:
|
||||
# Don't need sub-images anymore, drop from list:
|
||||
if no_grid and include_sub_grids:
|
||||
processed.images = processed.images[:z_count] # we don't have the main grid image, and need zero additional sub-images
|
||||
else:
|
||||
processed.images = processed.images[:z_count+1] # we either have the main grid image, or need one sub-images
|
||||
if shared.opts.grid_save: # Auto-save main and sub-grids:
|
||||
grid_count = z_count + 1 if z_count > 1 else 1
|
||||
grid_count = z_count + ( 1 if not no_grid and z_count > 1 else 0 )
|
||||
for g in range(grid_count):
|
||||
adj_g = g-1 if g > 0 else g
|
||||
images.save_image(processed.images[g], p.outpath_grids, "xyz_grid", info=processed.infotexts[g], extension=shared.opts.grid_format, prompt=processed.all_prompts[adj_g], seed=processed.all_seeds[adj_g], grid=True, p=processed)
|
||||
@@ -711,4 +725,9 @@ class Script(scripts.Script):
|
||||
del processed.all_prompts[1]
|
||||
del processed.all_seeds[1]
|
||||
del processed.infotexts[1]
|
||||
elif no_grid:
|
||||
# del processed.images[0]
|
||||
# del processed.all_prompts[0]
|
||||
# del processed.all_seeds[0]
|
||||
del processed.infotexts[0]
|
||||
return processed
|
||||
|
||||
@@ -44,6 +44,7 @@ except Exception:
|
||||
pass
|
||||
|
||||
state = shared.state
|
||||
backend = shared.backend
|
||||
if not modules.loader.initialized:
|
||||
timer.startup.record("libraries")
|
||||
log.setLevel(logging.DEBUG if cmd_opts.debug else logging.INFO)
|
||||
@@ -246,6 +247,8 @@ def start_ui():
|
||||
with open(cmd_opts.auth_file, 'r', encoding="utf8") as file:
|
||||
for line in file.readlines():
|
||||
gradio_auth_creds += [x.strip() for x in line.split(',') if x.strip()]
|
||||
if len(gradio_auth_creds) > 0:
|
||||
log.info(f'Authentication enabled: {gradio_auth_creds}')
|
||||
|
||||
global local_url # pylint: disable=global-statement
|
||||
stdout = io.StringIO()
|
||||
@@ -274,7 +277,7 @@ def start_ui():
|
||||
shared.log.info(f'API Docs: {local_url[:-1]}/docs') # pylint: disable=unsubscriptable-object
|
||||
if share_url is not None:
|
||||
shared.log.info(f'Share URL: {share_url}')
|
||||
shared.log.debug(f'Gradio registered functions: {len(shared.demo.fns)}')
|
||||
shared.log.debug(f'Gradio functions: registered={len(shared.demo.fns)}')
|
||||
shared.demo.server.wants_restart = False
|
||||
setup_middleware(app, cmd_opts)
|
||||
|
||||
|
||||
+1
-1
Submodule wiki updated: cd040c02e4...6f0a39edad
Reference in New Issue
Block a user