Merge branch 'master' of https://github.com/vladmandic/automatic

2026-09-19 17:24:32 +02:00 · 2023-08-01 02:03:51 +09:00
parent d711880aa9 8ffaea76ba
commit 8787f36b2c
13 changed files with 106 additions and 81 deletions
@@ -1,6 +1,8 @@
 {
+  "MD004": false,
  "MD012": false,
  "MD013": false,
+  "MD032": false,
  "MD033": false,
  "MD036": false,
  "MD041": false
@@ -1,6 +1,17 @@
 # Change Log for SD.Next

-## Update for 07/26/2023
+## Update for 2023-07-30
+
+Smaller release, but IMO worth a post...
+
+- diffusers:
+  - sd-xl lora's are now supported!
+  - memory optimizations: Enhanced sequential CPU offloading, model CPU offload, FP16 VAE
+    - significant impact if running SD-XL (for example, but applies to any model) with only 8GB VRAM
+  - update packages
+- minor bugfixes
+
+## Update for 2023-07-26

 This is a big one, new models, new diffusers, new features and updated UI...

@@ -12,7 +23,7 @@ First, **SD-XL 1.0** is released and yes, SD.Next supports it out of the box!
 Also fresh is new **Kandinsky 2.2** model that does look quite nice:

 - [Kandinsky Decoder](https://huggingface.co/kandinsky-community/kandinsky-2-2-decoder)
- [Kandinsky Prior](kandinsky-community/kandinsky-2-2-prior)
+- [Kandinsky Prior](https://huggingface.co/kandinsky-community/kandinsky-2-2-prior)

 Actual changelog is:

@@ -60,7 +71,7 @@ Actual changelog is:
    this now fully obsoletes `fallback_sampler` and `force_latent_sampler`  


-## Update for 07/18/2023
+## Update for 2023-07-18

 While we're waiting for official SD-XL release, here's another update with some fixes and enhancements...

@@ -90,7 +101,7 @@ While we're waiting for official SD-XL release, here's another update with some
    download using model downloader: `thu-ml/unidiffuser-v1`  
    and set resolution to 512x512  

-## Update for 07/14/2023
+## Update for 2023-07-14

 Trying to unify settings for both original and diffusers backend without introducing duplicates...

@@ -107,7 +118,7 @@ Trying to unify settings for both original and diffusers backend without introdu
 - reorganized settings -> samplers to show which section refers to which backend  
 - added diffusers **lmsd** sampler  

-## Update for 07/13/2023
+## Update for 2023-07-13

 Another big one, but now improvements to both **diffusers** and **original** backends as well plus ability to dynamically switch between them!

@@ -136,7 +147,7 @@ Another big one, but now improvements to both **diffusers** and **original** bac
 big thanks to @huggingface team for great communication, support and fixing all the reported issues asap!


-## Update for 07/10/2023
+## Update for 2023-07-10

 Service release with some fixes and enhancements:

@@ -158,7 +169,7 @@ Service release with some fixes and enhancements:
  - fix startup issue with incorrect config  
  - installer should always check requirements on upgrades

-## Update for 07/08/2023
+## Update for 2023-07-08

 This is a massive update which has been baking in a `dev` branch for a while now

@@ -179,7 +190,7 @@ Release also contains number of smaller updates:
  speeds up app start if you have a lot of extra networks and you want to build them manually when needed  
 - extra network ui tweaks  

-## Update for 07/01/2023
+## Update for 2023-07-01

 Small quality-of-life updates and bugfixes:

@@ -191,7 +202,7 @@ Small quality-of-life updates and bugfixes:
 - folder `Show` button is present only when working on localhost,  
  otherwise its replaced with `Copy` that places image URLs on clipboard so they can be used in other apps

-## Update for 06/30/2023
+## Update for 2023-06-30

 A bit bigger update this time, but contained to specific areas...

@@ -213,7 +224,7 @@ A bit bigger update this time, but contained to specific areas...
 - **extensions sort** by trending  
 - add requirements check for training  

-## Update for 06/26/2023
+## Update for 2023-06-26

 - new training tab interface  
  - redesigned preprocess, train embedding, train hypernetwork  
@@ -222,7 +233,7 @@ A bit bigger update this time, but contained to specific areas...
  - new model verify functionality  
 - lot of ipex specific fixes/optimizations, thanks @disty0  

-## Update for 06/20/2023
+## Update for 2023-06-20

 This one is less relevant for standard users, but pretty major if you're running an actual server  
 But even if not, it still includes bunch of cumulative fixes since last release - and going by number of new issues, this is probably the most stable release so far...
@@ -237,7 +248,7 @@ But even if not, it still includes bunch of cumulative fixes since last release
  - ring buffer with api endpoint `/sdapi/v1/log`  
  - real-time status and load endpoint `/sdapi/v1/system-info/status`

-## Update for 06/14/2023
+## Update for 2023-06-14

 Second stage of a jumbo merge from upstream plus few minor changes...

@@ -252,7 +263,7 @@ Second stage of a jumbo merge from upstream plus few minor changes...
  - broken/reverted:  
    sub-quadratic optimization changes  

-## Update for 06/13/2023
+## Update for 2023-06-13

 Just a day later and one *bigger update*...
 Both some **new functionality** as well as **massive merges** from upstream  
@@ -270,7 +281,7 @@ Both some **new functionality** as well as **massive merges** from upstream
  this was a massive merge, hopefully this does not cause any regressions  
  and still a bit more pending...

-## Update for 06/12/2023
+## Update for 2023-06-12

 - updated ui labels and hints to improve clarity and provide some extra info  
  this is 1st stage of the process, more to come...  
@@ -291,7 +302,7 @@ Both some **new functionality** as well as **massive merges** from upstream
  results? who am i to judge :)


-## Update for 06/05/2023
+## Update for 2023-06-05

 Few new features and extra handling for broken extensions  
 that caused my phone to go crazy with notifications over the weekend...
@@ -311,7 +322,7 @@ that caused my phone to go crazy with notifications over the weekend...
 - some work on bfloat16 which nobody really should be using, but why not 🙂


-## Update for 06/02/2023
+## Update for 2023-06-02

 Some quality-of-life improvements while working on larger stuff in the background...

@@ -328,7 +339,7 @@ Some quality-of-life improvements while working on larger stuff in the backgroun
  if you do have `xformers` selected as your desired cross-optimization method, then they will be used  
  reason is that a lot of libaries try to blindly import xformers even if they are not selected or not functional  

-## Update for 05/30/2023
+## Update for 2023-05-30

 Another bigger one...And more to come in the next few days...

@@ -344,7 +355,7 @@ Another bigger one...And more to come in the next few days...
 - updated lora, swinir, scunet and ldsr code from upstream  
 - start of merge from a1111 v1.3  

-## Update for 05/26/2023
+## Update for 2023-05-26

 Some quality-of-life improvements...

@@ -372,7 +383,7 @@ other than that - first stage of [Diffusers](https://github.com/huggingface/diff
 i don't recommend anyone to try it (and dont even think reporting issues for it)  
 but if anyone wants to contribute, take a look at [project page](https://github.com/users/vladmandic/projects/1/views/1)

-## Update for 05/23/2023
+## Update for 2023-05-23

 Major internal work with perhaps not that much user-facing to show for it ;)

@@ -398,7 +409,7 @@ Major internal work with perhaps not that much user-facing to show for it ;)
 - fixes...amazing how many issues were introduced by porting a1111 v1.20 code without adding almost no new functionality  
  next one is v1.30 (still in dev) which does bring a lot of new features  

-## Update for 05/17/2023
+## Update for 2023-05-17

 This is a massive one due to huge number of changes,  
 but hopefully it will go ok...
@@ -419,7 +430,7 @@ but hopefully it will go ok...
 - port *all* upstream changes from [A1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui)  
  up to today - commit hash `89f9faa`  

-## Update for 05/15/2023
+## Update for 2023-05-15

 - major work on **prompt parsing**
  this can cause some differences in results compared to what you're used to, but its all about fixes & improvements
@@ -431,7 +442,7 @@ but hopefully it will go ok...
 - add option in settings to force different **latent sampler** instead of using primary only
 - add **interrupt/skip** capabilities to process images

-## Update for 05/13/2023
+## Update for 2023-05-13

 This is mostly about optimizations...

@@ -448,7 +459,7 @@ This is mostly about optimizations...
 - allow per-extension updates in extension manager  
 - option to reset configuration in settings  

-## Update for 05/11/2023
+## Update for 2023-05-11

 - brand new **extension manager**  
  this is pretty much a complete rewrite, so new issues are possible
@@ -459,7 +470,7 @@ This is mostly about optimizations...
 - auto-recover & re-attempt when `--upgrade` is requested but fails
 - check for duplicate extensions  

-## Update for 05/08/2023
+## Update for 2023-05-08

 Back online with few updates:

@@ -470,7 +481,7 @@ Back online with few updates:
 - configurable order of scripts in txt2img and img2img  
  for both, see sections in ui-> settings -> user interface

-## Update for 05/04/2023
+## Update for 2023-05-04

 Again, few days later...

@@ -487,7 +498,7 @@ Again, few days later...
 - better logging  
  this includes new `--debug` flag for more verbose logging when troubleshooting  

-## Update for 05/01/2023
+## Update for 2023-05-01

 Been a bit quieter for last few days as changes were quite significant, but finally here we are...

@@ -506,7 +517,7 @@ Been a bit quieter for last few days as changes were quite significant, but fina

 To take advantage of new out-of-the-box tunings, its recommended to delete your `config.json` so new defaults are applied. Its not necessary, but otherwise you may need to play with UI Settings to get the best of Intel ARC, TorchML, ROCm or Apple M1/M2.

-## Update for 04/27/2023
+## Update for 2023-04-27

 a bit shorter list as:

@@ -527,7 +538,7 @@ what's new:
 - set defaults for Apple M1/M2  
  if you have issues, you may want to start with a fresh install so configuration can be created from scratch

-## Update for 04/25/2023
+## Update for 2023-04-25

 - update process image -> info
 - add VAE info to metadata
@@ -540,7 +551,7 @@ what's new:
 - validate script args for always-on scripts  
  fixes: deforum with controlnet  

-## Update for 04/24/2023
+## Update for 2023-04-24

 - identify race condition where generate locks up while fetching preview
 - add pulldowns to x/y/z script
@@ -552,7 +563,7 @@ what's new:
 - fix setup not installing `tensorflow` dependencies
 - update default git flags to reduce number of warnings

-## Update for 04/23/2023
+## Update for 2023-04-23

 - fix VAE dtype  
  should fix most issues with NaN or black images  
@@ -567,7 +578,7 @@ what's new:
 - mark tensorflow as optional
 - add additional image name templates

-## Update for 04/22/2023
+## Update for 2023-04-22

 - autodetect which system libs should be installed  
  this is a first pass of autoconfig for **nVidia** vs **AMD** environments  
@@ -579,7 +590,7 @@ what's new:

 *expect shorter updates for the next few days as i'll be partially ooo*

-## Update for 04/20/2023
+## Update for 2023-04-20

 - full CUDA tuning section in UI Settings
 - improve exif/pnginfo metadata parsing  
@@ -593,7 +604,7 @@ what's new:

 note: if you previously had command line optimizations such as --no-half, those are now ignored and moved to ui settings

-## Update for 04/19/2023
+## Update for 2023-04-19

 - fix live preview
 - fix model merge
@@ -605,7 +616,7 @@ note: if you previously had command line optimizations such as --no-half, those
 - created initial merge backlog from pending prs on a1111 repo  
  see #258 for details

-## Update for 04/18/2023
+## Update for 2023-04-18

 - reconnect ui to active session on browser restart  
  this is one of most frequently asked for items, finally figured it out  
@@ -625,7 +636,7 @@ note: if you previously had command line optimizations such as --no-half, those
 - update README.md
 - update TODO.md

-## Update for 04/17/2023
+## Update for 2023-04-17

 - **themes** are now dynamic and discovered from list of available gradio themes on huggingface  
  its quite a list of 30+ supported themes so far  
@@ -636,7 +647,7 @@ note: if you previously had command line optimizations such as --no-half, those
 - updated github integration
 - make model download as optional if no models found

-## Update for 04/16/2023
+## Update for 2023-04-16

 - support for ui themes! to to *settings* -> *user interface* -> "ui theme*
  includes 12 predefined themes
@@ -648,7 +659,7 @@ note: if you previously had command line optimizations such as --no-half, those
 - install bugfixes, hopefully new installer is now ok  \
  i really want to focus on features and not troubleshooting installer

-## Update for 04/15/2023
+## Update for 2023-04-15

 - update default values
 - remove `ui-config.json` from repo, its not fully under user control
@@ -663,7 +674,7 @@ note: if you previously had command line optimizations such as --no-half, those
 - fix git update issues
 - update github templates

-## Update for 04/14/2023
+## Update for 2023-04-14

 - handle duplicate extensions
 - redo exception handler
@@ -675,7 +686,7 @@ note: if you previously had command line optimizations such as --no-half, those
 - pin transformers version to last known good version
 - fix extension loader

-## Update for 04/12/2023
+## Update for 2023-04-12

 This has been pending for a while, but finally uploaded some massive changes

@@ -572,7 +572,10 @@
  {"id":"","label":"Enable model CPU offload","localized":"","hint":"Transferring of entire models to the GPU, negligible impact on inference time while still providing some memory savings. Use with Enable Attention slicing for additional memory savings"},
  {"id":"","label":"Enable VAE slicing","localized":"","hint":"Decodes batch latents one image at a time with limited VRAM. Small performance boost in VAE decode on multi-image batches. Use with Enable Attention slicing"},
  {"id":"","label":"Enable VAE tiling","localized":"","hint":"Divide large images into overlapping tiles with limited VRAM. Might result in a minor increase in processing time. Use with Enable Attention Slicing"},
-  {"id":"","label":"Enable attention slicing","localized":"","hint":"Performs attention computation in steps instead of all at once. 10% slower inference times. Greatly reduces memory usage. Best used, period"}
+  {"id":"","label":"Enable attention slicing","localized":"","hint":"Performs attention computation in steps instead of all at once. 10% slower inference times. Greatly reduces memory usage. Best used, period"},
+  {"id":"","label":"Diffusers model loading variant","localized":"","hint":""},
+  {"id":"","label":"Diffusers VAE loading variant","localized":"","hint":""}
+
 ],
 "scripts": [
  {"id":"","label":"Script","localized":"","hint":""},
@@ -572,7 +572,9 @@
  {"id":"","label":"Enable model CPU offload","localized":"모델 CPU 오프로드 활성화","hint":"전체 모델을 GPU로 옮긴다. 여전히 GPU 메모리 사용률을 약간 낮춰주지만 생성 속도에는 무시할 수 있는 정도의 영향을 준다. 추가적인 메모리 절약을 위해서는 어텐션 슬라이싱과 함께 사용한다."},
  {"id":"","label":"Enable VAE slicing","localized":"VAE 슬라이싱 활성화","hint":"Decodes batch latents one image at a time with limited VRAM. 여러 이미지를 다룰 때 VAE 해독 시 작은 성능 향상이 있다. 어텐션 슬라이싱과 함께 사용한다."},
  {"id":"","label":"Enable VAE tiling","localized":"VAE 타일링 활성화","hint":"Divide large images into overlapping tiles with limited VRAM. 생성 시간이 약간 늘어날 수 있다. 어텐션 슬라이싱과 함께 사용한다."},
-  {"id":"","label":"Enable attention slicing","localized":"어텐션 슬라이싱 활성화","hint":"Performs attention computation in steps instead of all at once. 생성이 10% 느려지지만 메모리 사용률을 매우 크게 줄여준다."}
+  {"id":"","label":"Enable attention slicing","localized":"어텐션 슬라이싱 활성화","hint":"Performs attention computation in steps instead of all at once. 생성이 10% 느려지지만 메모리 사용률을 매우 크게 줄여준다."},
+  {"id":"","label":"Diffusers model loading variant","localized":"","hint":""},
+  {"id":"","label":"Diffusers VAE loading variant","localized":"","hint":""}
 ],
 "scripts": [
  {"id":"","label":"Script","localized":"스크립트","hint":""},
@@ -10,6 +10,7 @@ lora_state = { # TODO Lora state for Diffusers
 def unload_diffusers_lora():
    try:
        pipe = shared.sd_model
+        pipe.unload_lora_weights()
        lora_state['active'] = False
        lora_state['loaded'] = 0
        pipe._remove_text_encoder_monkey_patch() # pylint: disable=W0212
@@ -24,7 +25,7 @@ def unload_diffusers_lora():
 def load_diffusers_lora(name, lora, strength = 1.0):
    try:
        pipe = shared.sd_model
-        pipe.load_lora_weights(lora.filename, cache_dir=shared.opts.diffusers_dir, local_files_only=True)
+        pipe.load_lora_weights(lora.filename, cache_dir=shared.opts.diffusers_dir, local_files_only=True, lora_scale=strength)
        lora_state['active'] = True
        lora_state['loaded'] += 1
        lora_state['multiplier'] = strength
@@ -36,7 +36,7 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
    def vae_decode(latents, model, output_type='np'):
        if hasattr(model, 'vae') and torch.is_tensor(latents):
            shared.log.debug(f'Diffusers VAE decode: name={model.vae.config.get("_name_or_path", "default")} dtype={model.vae.dtype} upcast={model.vae.config.get("force_upcast", None)}')
-            if shared.opts.diffusers_move_unet:
+            if shared.opts.diffusers_move_unet and not model.has_accelerate:
                shared.log.debug('Diffusers: Moving UNet to CPU')
                unet_device = model.unet.device
                model.unet.to(devices.cpu)
@@ -44,7 +44,7 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
            latents.to(model.vae.device)
            decoded = model.vae.decode(latents / model.vae.config.scaling_factor, return_dict=False)[0]
            imgs = model.image_processor.postprocess(decoded, output_type=output_type)
-            if shared.opts.diffusers_move_unet:
+            if shared.opts.diffusers_move_unet and not model.has_accelerate:
                model.unet.to(unet_device)
            return imgs
        else:
@@ -56,7 +56,7 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
        pipeline = model
        signature = inspect.signature(type(pipeline).__call__)
        possible = signature.parameters.keys()
-        generator_device = 'cpu' if shared.opts.diffusers_generator_device == "cpu" else shared.device
+        generator_device = devices.cpu if shared.opts.diffusers_generator_device == "cpu" else shared.device
        generator = [torch.Generator(generator_device).manual_seed(s) for s in seeds]
        if 'prompt' in possible:
            if hasattr(model, 'text_encoder') and 'prompt_embeds' in possible:
@@ -134,7 +134,7 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
    if shared.state.interrupted or shared.state.skipped:
        return results

-    if shared.opts.diffusers_move_base:
+    if shared.opts.diffusers_move_base and not shared.sd_model.has_accelerate:
        shared.sd_model.to(devices.device)

    refiner_enabled = shared.sd_refiner is not None and p.enable_hr
@@ -159,6 +159,9 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
    if shared.sd_refiner is None or not p.enable_hr:
        output.images = vae_decode(output.images, shared.sd_model)

+    if lora_state['active']:
+        unload_diffusers_lora()
+
    if refiner_enabled:
        for i in range(len(output.images)):
            if shared.opts.save and not p.do_not_save_samples and shared.opts.save_images_before_refiner and hasattr(shared.sd_model, 'vae'):
@@ -168,9 +171,9 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
                for i in range(len(decoded)):
                    images.save_image(decoded[i], path=p.outpath_samples, basename="", seed=seeds[i], prompt=prompts[i], extension=shared.opts.samples_format, info=info, p=p, suffix="-before-refiner")

-        if shared.opts.diffusers_move_base:
+        if (shared.opts.diffusers_move_base or shared.cmd_opts.medvram or shared.opts.diffusers_model_cpu_offload) and not (shared.cmd_opts.lowvram or shared.opts.diffusers_seq_cpu_offload):
            shared.log.debug('Diffusers: Moving base model to CPU')
-            shared.sd_model.to('cpu')
+            shared.sd_model.to(devices.cpu)
            devices.torch_gc()

        if (not hasattr(shared.sd_refiner.scheduler, 'name')) or (shared.sd_refiner.scheduler.name != p.latent_sampler) and (p.sampler_name != 'Default'):
@@ -182,7 +185,7 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
        if shared.state.interrupted or shared.state.skipped:
            return results

-        if shared.opts.diffusers_move_refiner:
+        if shared.opts.diffusers_move_refiner and not shared.sd_refiner.has_accelerate:
            shared.sd_refiner.to(devices.device)
        p.ops.append('refine')
        for i in range(len(output.images)):
@@ -205,16 +208,15 @@ def process_diffusers(p: StableDiffusionProcessing, seeds, prompts, negative_pro
                refiner_images = vae_decode(refiner_output.images, shared.sd_refiner)
                results.append(refiner_images[0])

-        if shared.opts.diffusers_move_refiner:
+        if shared.opts.diffusers_move_refiner and not shared.sd_refiner.has_accelerate:
            shared.log.debug('Diffusers: Moving refiner model to CPU')
-            shared.sd_refiner.to('cpu')
+            shared.sd_refiner.to(devices.cpu)
    else:
        results = output.images

    if p.is_hr_pass:
        shared.log.warning('Diffusers not implemented: hires fix')

-    if lora_state['active']:
-        unload_diffusers_lora()
+

    return results
@@ -548,8 +548,13 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No
        "load_connected_pipeline": True  # always load end-to-end / connected pipelines
        # "use_safetensors": True,  # TODO(PVP) - we can't enable this for all checkpoints just yet
    }
-    if devices.dtype == torch.float16:
-        diffusers_load_config['variant'] = 'fp16'
+    if shared.opts.diffusers_model_load_variant == 'default':
+        if devices.dtype == torch.float16:
+            diffusers_load_config['variant'] = 'fp16'
+    elif shared.opts.diffusers_model_load_variant == 'fp32':
+        pass
+    else:
+        diffusers_load_config['variant'] = shared.opts.diffusers_model_load_variant

    if shared.opts.data.get('sd_model_checkpoint', '') == 'model.ckpt' or shared.opts.data.get('sd_model_checkpoint', '') == '':
        shared.opts.data['sd_model_checkpoint'] = "runwayml/stable-diffusion-v1-5"
@@ -659,18 +664,10 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No
                sd_model.enable_model_cpu_offload()
                sd_model.has_accelerate = True
        if hasattr(sd_model, "enable_sequential_cpu_offload"):
-            if shared.opts.diffusers_seq_cpu_offload:
+            if shared.cmd_opts.lowvram or shared.opts.diffusers_seq_cpu_offload:
+                shared.log.debug(f'Diffusers {op}: enable sequential CPU offload')
                sd_model.enable_sequential_cpu_offload(device=devices.device)
                sd_model.has_accelerate = True
-                shared.log.debug(f'Diffusers {op}: enable sequential CPU offload')
-        if sd_model.has_accelerate and (shared.opts.diffusers_move_base or shared.opts.diffusers_move_refiner or shared.opts.diffusers_move_unet):
-            shared.log.warning("Moving models to CPU is not compatible with sequential CPU offload")
-            shared.log.debug('Disabled moving base model to CPU')
-            shared.log.debug('Disabled moving refiner model to CPU')
-            shared.log.debug('Disabled moving UNet to CPU')
-            shared.opts.diffusers_move_base=False
-            shared.opts.diffusers_move_refiner=False
-            shared.opts.diffusers_move_unet=False
        if hasattr(sd_model, "enable_vae_slicing"):
            if shared.cmd_opts.lowvram or shared.opts.diffusers_vae_slicing:
                shared.log.debug(f'Diffusers {op}: enable VAE slicing')
@@ -721,7 +718,7 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No
                        shared.opts.diffusers_move_base=True
                        shared.opts.diffusers_move_refiner=True
                    shared.log.debug('Moving base model to CPU')
-                    model_data.sd_model.to("cpu")
+                    model_data.sd_model.to(devices.cpu)
                    devices.torch_gc(force=True)
                    sd_model.to(devices.device)
                    base_sent_to_cpu=True
@@ -753,9 +750,9 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No
        sd_model.sd_model_hash = checkpoint_info.hash # pylint: disable=attribute-defined-outside-init
        if hasattr(sd_model, "set_progress_bar_config"):
            sd_model.set_progress_bar_config(bar_format='Progress {rate_fmt}{postfix} {bar} {percentage:3.0f}% {n_fmt}/{total_fmt} {elapsed} {remaining}', ncols=80, colour='#327fba')
-        if op == 'refiner' and shared.opts.diffusers_move_refiner:
+        if op == 'refiner' and shared.opts.diffusers_move_refiner and not sd_model.has_accelerate:
            shared.log.debug('Moving refiner model to CPU')
-            sd_model.to("cpu")
+            sd_model.to(devices.cpu)
        elif not sd_model.has_accelerate:
            # In offload modes, accelerate will move models around.
            sd_model.to(devices.device)
@@ -943,12 +940,12 @@ def reload_model_weights(sd_model=None, info=None, reuse_dict=False, op='model')
        current_checkpoint_info = getattr(sd_model, 'sd_checkpoint_info', None)
        if current_checkpoint_info is not None and checkpoint_info is not None and current_checkpoint_info.filename == checkpoint_info.filename:
            return
-        if shared.backend == shared.Backend.ORIGINAL or not sd_model.has_accelerate:
+        if not sd_model.has_accelerate:
            if shared.cmd_opts.lowvram or shared.cmd_opts.medvram:
                lowvram.send_everything_to_cpu()
            else:
                sd_model.to(devices.cpu)
-    if reuse_dict or (shared.opts.model_reuse_dict and sd_model is not None):
+    if (reuse_dict or (shared.opts.model_reuse_dict and sd_model is not None)) and not sd_model.has_accelerate:
        shared.log.info('Reusing previous model dictionary')
        sd_hijack.model_hijack.undo_hijack(sd_model)
    else:
@@ -980,7 +977,7 @@ def reload_model_weights(sd_model=None, info=None, reuse_dict=False, op='model')
        timer.record("hijack")
        script_callbacks.model_loaded_callback(sd_model)
        timer.record("callbacks")
-        if not shared.cmd_opts.lowvram and not shared.cmd_opts.medvram and (shared.backend == shared.Backend.ORIGINAL or not sd_model.has_accelerate):
+        if not shared.cmd_opts.lowvram and not shared.cmd_opts.medvram and not sd_model.has_accelerate:
            sd_model.to(devices.device)
            timer.record("device")
    shared.log.info(f"Weights loaded in {timer.summary()}")
@@ -990,7 +987,7 @@ def unload_model_weights(op='model'):
    from modules import sd_hijack
    if op == 'model' or op == 'dict':
        if model_data.sd_model:
-            if shared.backend == shared.Backend.ORIGINAL or not model_data.sd_model.has_accelerate:
+            if not model_data.sd_model.has_accelerate:
                model_data.sd_model.to(devices.cpu)
            if shared.backend == shared.Backend.ORIGINAL:
                sd_hijack.model_hijack.undo_hijack(model_data.sd_model)
@@ -998,7 +995,7 @@ def unload_model_weights(op='model'):
            shared.log.debug(f'Weights unloaded {op}: {memory_stats()}')
    else:
        if model_data.sd_refiner:
-            if shared.backend == shared.Backend.ORIGINAL or not model_data.sd_refiner.has_accelerate:
+            if not model_data.sd_refiner.has_accelerate:
                model_data.sd_refiner.to(devices.cpu)
            if shared.backend == shared.Backend.ORIGINAL:
                sd_hijack.model_hijack.undo_hijack(model_data.sd_refiner)
@@ -181,8 +181,13 @@ def load_vae_diffusers(_model, vae_file=None, vae_source="from unknown source"):
        "torch_dtype": devices.dtype_vae,
        "use_safetensors": True,
    }
-    if devices.dtype_vae == torch.float16:
-        diffusers_load_config['variant'] = 'fp16'
+    if shared.opts.diffusers_vae_load_variant == 'default':
+        if devices.dtype_vae == torch.float16:
+            diffusers_load_config['variant'] = 'fp16'
+    elif shared.opts.diffusers_vae_load_variant == 'fp32':
+        pass
+    else:
+        diffusers_load_config['variant'] = shared.opts.diffusers_vae_load_variant

    if shared.opts.diffusers_vae_upcast != 'default':
        diffusers_load_config['force_upcast'] = True if shared.opts.diffusers_vae_upcast == 'true' else False
@@ -232,7 +237,7 @@ def reload_vae_weights(sd_model=None, vae_file=unspecified):
        vae_source = "from function argument"
    if loaded_vae_file == vae_file:
        return
-    if shared.backend == shared.Backend.ORIGINAL or not sd_model.has_accelerate:
+    if not sd_model.has_accelerate:
        if shared.cmd_opts.lowvram or shared.cmd_opts.medvram:
            lowvram.send_everything_to_cpu()
        else:
@@ -246,7 +251,7 @@ def reload_vae_weights(sd_model=None, vae_file=unspecified):
        sd_hijack.model_hijack.hijack(sd_model)
        script_callbacks.model_loaded_callback(sd_model)

-    if not shared.cmd_opts.lowvram and not shared.cmd_opts.medvram and (shared.backend == shared.Backend.ORIGINAL or not sd_model.has_accelerate):
+    if not shared.cmd_opts.lowvram and not shared.cmd_opts.medvram and not sd_model.has_accelerate:
        sd_model.to(devices.device)
    shared.log.info(f"VAE weights loaded: {vae_file}")
    return sd_model
@@ -408,6 +408,8 @@ options_templates.update(options_section(('diffusers', "Diffusers Settings"), {
    "diffusers_vae_slicing": OptionInfo(True, "Enable VAE slicing"),
    "diffusers_vae_tiling": OptionInfo(False, "Enable VAE tiling"),
    "diffusers_attention_slicing": OptionInfo(False, "Enable attention slicing"),
+    "diffusers_model_load_variant": OptionInfo("default", "Diffusers model loading variant", gr.Radio, lambda: {"choices": ['default', 'fp32', 'fp16']}),
+    "diffusers_vae_load_variant": OptionInfo("default", "Diffusers VAE loading variant", gr.Radio, lambda: {"choices": ['default', 'fp32', 'fp16']}),
    # "diffusers_force_zeros": OptionInfo(False, "Force zeros for prompts when empty"),
    # "diffusers_aesthetics_score": OptionInfo(6.0, "Require aesthetic score", gr.Slider, {"minimum": 0, "maximum": 10, "step": 0.1}),
 }))
@@ -49,7 +49,7 @@ requests==2.31.0
 tqdm==4.65.0
 accelerate==0.20.3
 opencv-python==4.7.0.72
-diffusers==0.19.0
+diffusers==0.19.3
 einops==0.4.1
 gradio==3.32.0
 numexpr==2.8.4