post merge cleanup

CHANGELOG.md

@@ -25,6 +25,8 @@ Another big release, highlights being:
 - Brand new **Intelligent masking**, manual or automatic  
   Using ML models (*LAMA* object removal, *REMBG* background removal, *SAM* segmentation, etc.) and with live previews  
   Granular blur, erode and dilate controls  
+- Massive work integrating latest advances with **OpenVINO**, **IPEX** and **Olive**  
+- New models and pipelines: *Mixture Tiling, SAG, InstaFlow, BlipDiffusion*  
 
 Plus welcome additions to **UI performance, usability and accessibility** and flexibility of deployment  
 And it also includes fixes for all reported issues so far  
@@ -126,15 +128,15 @@ As of this release, default backend is set to **diffusers** as its more feature
 - [FreeInit](https://tianxingwu.github.io/pages/FreeInit/) for **AnimateDiff**
   - greatly improves temporal consistency of generated outputs  
   - all options are available in animateddiff script  
-- [SalesForce BlipDiffusion](https://huggingface.co/docs/diffusers/api/pipelines/blip_diffusion)
+- [SalesForce BlipDiffusion](https://huggingface.co/docs/diffusers/api/pipelines/blip_diffusion)  
   - model can be used to place subject in a different context  
   - requires input image  
-  - last word in prompt and negative prompt will be used as source and target subjects
+  - last word in prompt and negative prompt will be used as source and target subjects  
   - sampler must be set to default before loading the model  
 - [InstaFlow](https://github.com/gnobitab/InstaFlow)  
   - another take on super-fast image generation in a single step  
-  - set sampler:default steps:1
-  - load from networks -> models -> reference
+  - set *sampler:default, steps:1, cfg-scale:0*  
+  - load from networks -> models -> reference  
 - **Improvements**  
   - **ui**  
     - check version and **update** SD.Next via UI  
@@ -244,6 +246,9 @@ As of this release, default backend is set to **diffusers** as its more feature
   - remove `OPENVINO_TORCH_BACKEND_DEVICE` env variable  
   - reduce system memory usage after compile  
   - fix cache loading with multiple models  
+- **Olive** support, thanks @lshqqytiger
+  - fully merged in in [wiki](https://github.com/vladmandic/automatic/wiki/ONNX-Olive)  , see wiki for details, thanks @lshqqytiger  
+  - as a highlight, 4-5 it/s using DirectML on AMD GPU translates to 23-25 it/s using ONNX/Olive!  
 - **fixes**  
   - civitai model download: enable downloads of embeddings
   - ipadapter: allow changing of model/image on-the-fly  

modules/sd_models.py

@@ -790,7 +790,14 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No
         shared.log.debug(f'Diffusers loading: path="{checkpoint_info.path}"')
         pipeline, model_type = detect_pipeline(checkpoint_info.path, op)
         if os.path.isdir(checkpoint_info.path):
-            if model_type in ['InstaFlow'] or 'ONNX' in model_type: # forced pipeline
+            if model_type in ['InstaFlow']: # forced pipeline
+                try:
+                    pipeline = diffusers.utils.get_class_from_dynamic_module('instaflow_one_step', module_file='pipeline.py')
+                    sd_model = pipeline.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
+                except Exception as e:
+                    shared.log.error(f'Diffusers Failed loading {op}: {checkpoint_info.path} {e}')
+                    return
+            elif 'ONNX' in model_type: # forced pipeline
                 sd_model = pipeline.from_pretrained(checkpoint_info.path, cache_dir=shared.opts.diffusers_dir, **diffusers_load_config)
             else:
                 err1, err2, err3 = None, None, None

modules/shared_items.py

@@ -56,12 +56,9 @@ def get_pipelines():
         'ONNX Stable Diffusion XL': getattr(diffusers, 'OnnxStableDiffusionXLPipeline', None),
         'ONNX Stable Diffusion XL Img2Img': getattr(diffusers, 'OnnxStableDiffusionXLImg2ImgPipeline', None),
         'Custom Diffusers Pipeline': getattr(diffusers, 'DiffusionPipeline', None),
+        'InstaFlow': getattr(diffusers, 'StableDiffusionPipeline', None) # dynamically redefined and loaded in sd_models.load_diffuser
         # Segmind SSD-1B, Segmind Tiny
     }
-    try:
-        pipelines['InstaFlow'] = diffusers.utils.get_class_from_dynamic_module('instaflow_one_step', module_file='pipeline.py')
-    except Exception:
-        pipelines['InstaFlow'] = getattr(diffusers, 'StableDiffusionPipeline', None)
 
     for k, v in pipelines.items():
         if k != 'Autodetect' and v is None: