diff --git a/CHANGELOG.md b/CHANGELOG.md index 9765b46b8..7ade3aaa9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,11 +1,11 @@ # Change Log for SD.Next -## Update for 2023-10-30 +## Update for 2023-10-31 *Note*: Pending release of `diffusers==0.22.0` Another pretty big release, this time with focus on -new models, new backends and optimizations and tons of fixes +new models, new backends and optimizations - plus quite a few fixes Also, [Wiki](https://github.com/vladmandic/automatic/wiki) has been updated with new content, so check it out! Some highlights: [OpenVINO](https://github.com/vladmandic/automatic/wiki/OpenVINO), [IntelArc](https://github.com/vladmandic/automatic/wiki/Intel-ARC), [DirectML](https://github.com/vladmandic/automatic/wiki/DirectML), [ONNX/Olive>](https://github.com/vladmandic/automatic/wiki/ONNX-Runtime) @@ -14,23 +14,28 @@ Some highlights: [OpenVINO](https://github.com/vladmandic/automatic/wiki/OpenVIN - new model type: [Segmind SSD-1B](https://huggingface.co/segmind/SSD-1B) its a *distilled* model, this time 50% smaller and faster version of SD-XL! (and quality does not suffer, its just more optimized) - test shows batch-size:4 with 1k images used less than 6.5GB of VRAM + test shows batch-size:4 with 1k images at full quality used less than 6.5GB of VRAM + and for further optimization, you can use built-in **TAESD** decoder, + which results in batch-size:16 with 1k images using 7.9GB of VRAM download using built-in **Huggingface** downloader: `segmind/SSD-1B` - new model type: [LCM: Latent Consistency Models](https://github.com/openai/consistency_models) near-instant generate in a as little as 3 steps! combined with OpenVINO, generate on CPU takes less than 10 seconds: + and absolute beast when combined with **HyperTile** and **TAESD** decoder resulting in **28 FPS** + (on RTX4090 for batch 16x16 at 512px) download using built-in **Huggingface** downloader: `SimianLuo/LCM_Dreamshaper_v7` - support for **Custom pipelines**, thanks @disty0 download using built-in **Huggingface** downloader think of them as plugins for diffusers not unlike original extensions that modify behavior of `ldm` backend list of community pipelines: - - new custom pipeline: `Disty0/zero123plus-pipeline` + - new custom pipeline: `Disty0/zero123plus-pipeline`, thanks @disty0 generate 4 output images with different camera positions: front, side, top, back! for more details, see - - new backend: **ONNX/Olive** (experimental) - for details, see WiKi + - new backend: **ONNX/Olive** *(experimental)*, thanks @lshqqytiger + for details, see [WiKi](https://github.com/vladmandic/automatic/wiki/ONNX-Runtime) - extend support for [Free-U](https://github.com/ChenyangSi/FreeU) improve generations quality at no cost (other than finding params that work for you) + - updated `diffusers` to 0.22.0 - **General** - add **Lora OFT** support, thanks @antis0007 and @ai-casanova - **Upscalers** @@ -45,6 +50,9 @@ Some highlights: [OpenVINO](https://github.com/vladmandic/automatic/wiki/OpenVIN - remove external clone of items in `/repositories` - switch core font in default theme to **noto-sans** previously default font was simply *system-ui*, but it lead to too much variations between browsers and platforms + - update **openvino**, thanks @disty0 + - update **ipex**, thanks @disty0 + - update **directml**, @lshqqytiger - **Fixes** - fix **freeu** for backend original and add it to xyz grid - fix loading diffuser models in huggingface format from non-standard location @@ -65,14 +73,12 @@ Some highlights: [OpenVINO](https://github.com/vladmandic/automatic/wiki/OpenVIN - fix prompts-from-file saving incorrect metadata - fix before-hires step - fix diffusers switch from invalid model - - **directml** and **ipex** updates - force second requirements check on startup - remove **lyco**, multiple_tqdm - enhance extension compatibility for exensions directly importing codeformers - enhance extension compatibility for exensions directly accessing processing params - **css** fixes - clearly mark external themes in ui - - update `openvino`, thanks @disty0 - update `typing-extensions` ## Update for 2023-10-17 diff --git a/javascript/sdnext.css b/javascript/sdnext.css index 0be48f25e..ee6841ca5 100644 --- a/javascript/sdnext.css +++ b/javascript/sdnext.css @@ -47,11 +47,11 @@ tr { border-bottom: none !important; padding: 0.1em 0.5em !important; } /* custom gradio elements */ .accordion-compact { padding: 8px 0px 4px 0px !important; } .settings-accordion > div { flex-flow: wrap; } -.small-accordion .form { min-width: var(--left-column) !important; width: max-content; } +.small-accordion .form { min-width: var(--left-column) !important; max-width: max-content; } .small-accordion .label-wrap .icon { margin-right: 1.6em; margin-left: 0.6em; color: var(--button-primary-border-color); } .small-accordion .label-wrap { padding: 16px 0px 8px 0px; margin: 0; border-top: 2px solid var(--button-secondary-border-color); } .small-accordion { width: fit-content !important; padding-left: 0 !important; } -.extension-script { max-width: 50%; } +.extension-script { max-width: 48vw; } button.custom-button{ border-radius: var(--button-large-radius); padding: var(--button-large-padding); font-weight: var(--button-large-text-weight); border: var(--button-border-width) solid var(--button-secondary-border-color); background: var(--button-secondary-background-fill); color: var(--button-secondary-text-color); font-size: var(--button-large-text-size); display: inline-flex; justify-content: center; align-items: center; transition: var(--button-transition); box-shadow: var(--button-shadow); text-align: center; } diff --git a/modules/processing.py b/modules/processing.py index bf098664a..f02425a93 100644 --- a/modules/processing.py +++ b/modules/processing.py @@ -80,7 +80,7 @@ def create_binary_mask(image): return image -def images_tensor_to_samples(image, approximation=None, model=None): +def images_tensor_to_samples(image, approximation=None, model=None): # pylint: disable=unused-argument if model is None: model = shared.sd_model model.first_stage_model.to(devices.dtype_vae) @@ -481,7 +481,7 @@ def decode_first_stage(model, x, full_quality=True): else: x_sample = torch.zeros((len(x), 3, x.shape[2] * 8, x.shape[3] * 8), dtype=devices.dtype_vae, device=devices.device) for i in range(len(x_sample)): - x_sample[i] = (modules.taesd.sd_vae_taesd.decode(x[i]) * 2.0) - 1.0 + x_sample[i] = modules.taesd.sd_vae_taesd.decode(x[i]) except Exception as e: x_sample = x shared.log.error(f'Decode VAE: {e}') diff --git a/modules/processing_diffusers.py b/modules/processing_diffusers.py index bf402971c..566635194 100644 --- a/modules/processing_diffusers.py +++ b/modules/processing_diffusers.py @@ -116,7 +116,7 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro return [] decoded = torch.zeros((len(latents), 3, latents.shape[2] * 8, latents.shape[3] * 8), dtype=devices.dtype_vae, device=devices.device) for i in range(len(output.images)): - decoded[i] = (sd_vae_taesd.decode(latents[i]) * 2.0) - 1.0 + decoded[i] = sd_vae_taesd.decode(latents[i]) return decoded def taesd_vae_encode(image): diff --git a/modules/sd_samplers_common.py b/modules/sd_samplers_common.py index 75198b832..d44136aa6 100644 --- a/modules/sd_samplers_common.py +++ b/modules/sd_samplers_common.py @@ -31,7 +31,6 @@ def setup_img2img_steps(p, steps=None): def single_sample_to_image(sample, approximation=None): - if approximation is None: approximation = approximation_indexes.get(shared.opts.show_progress_type, None) if approximation is None: diff --git a/modules/shared.py b/modules/shared.py index 523536387..5d4937f4d 100644 --- a/modules/shared.py +++ b/modules/shared.py @@ -927,10 +927,13 @@ class Shared(sys.modules[__name__].__class__): # this class is here to provide s model_type = 'sdxl' elif "StableDiffusion" in self.sd_model.__class__.__name__: model_type = 'sd' + elif "LatentConsistencyModel" in self.sd_model.__class__.__name__: + model_type = 'sd' # lcm is compatible with sd elif "Kandinsky" in self.sd_model.__class__.__name__: model_type = 'kandinsky' else: model_type = self.sd_model.__class__.__name__ + print('HERE', model_type) except Exception: model_type = 'unknown' return model_type diff --git a/modules/taesd/sd_vae_taesd.py b/modules/taesd/sd_vae_taesd.py index 8ad4f6f21..28b2eb4e3 100644 --- a/modules/taesd/sd_vae_taesd.py +++ b/modules/taesd/sd_vae_taesd.py @@ -60,6 +60,7 @@ def decode(latents): vae.to(devices.device, devices.dtype_vae) enc = latents.unsqueeze(0).to(devices.device, devices.dtype_vae) image = vae.decoder(enc).clamp(0, 1).detach() + image = 2.0 * image - 1.0 return image[0]