extend use of move_model

2026-09-20 01:31:13 +02:00 · 2024-02-27 07:46:28 -05:00
parent 3488dd2cf6
commit 36dac2af9c
10 changed files with 110 additions and 69 deletions
@@ -19,7 +19,8 @@
  - remove obsolete textual inversion training code
  - remove obsolete hypernetworks training code
 - **Fixes**
-  - improve model offload compatibility
+  - improve model cpu offload compatibility
+  - improve model sequential offload compatibility
  - fix extra networks refresh
  - fix sdp memory attention in backend original
  - fix autodetect v2.1 models
@@ -56,19 +56,6 @@ For screenshots and informations on other available themes, see [Themes Wiki](ht
  Supports **SD 1.x** and **SD 2.x** models  
  All other model types such as *SD-XL, LCM, PixArt, Segmind, Kandinsky, etc.* require backend **Diffusers**  

-## Control
-
-**SD.Next** comes with built-in control for all types of text2image, image2image, video2video and batch processing
-
-*Control interface*:  
-![Screenshot-Control](html/screenshot-control.jpg)
-
-*Control processors*:  
-![Screenshot-Process](html/screenshot-processors.jpg)
-
-*Masking*:
-![Screenshot-Mask](html/screenshot-mask.jpg)
-
 ## Model support

 Additional models will be added as they become available and there is public interest in them
@@ -110,7 +97,6 @@ Also supported are modifiers such as:
 *InstantID*:  
 ![Screenshot-InstantID](html/screenshot-instantid.jpg)

-
 > [!IMPORTANT]
 > - Loading any model other than standard SD 1.x / SD 2.x requires use of backend **Diffusers**  
 > - Loading any other models using **Original** backend is not supported  
@@ -151,51 +137,89 @@ Also supported are modifiers such as:

 Once SD.Next is installed, simply run `webui.ps1` or `webui.bat` (*Windows*) or `webui.sh` (*Linux or MacOS*)

-Below is partial list of all available parameters, run `webui --help` for the full list:
+List of available parameters, run `webui --help` for the full & up-to-date list:

    Server options:
-      --config CONFIG                  Use specific server configuration file, default: config.json
-      --ui-config UI_CONFIG            Use specific UI configuration file, default: ui-config.json
-      --medvram                        Split model stages and keep only active part in VRAM, default: False
-      --lowvram                        Split model components and keep only active part in VRAM, default: False
-      --ckpt CKPT                      Path to model checkpoint to load immediately, default: None
-      --vae VAE                        Path to VAE checkpoint to load immediately, default: None
-      --data-dir DATA_DIR              Base path where all user data is stored, default:
-      --models-dir MODELS_DIR          Base path where all models are stored, default: models
-      --share                          Enable UI accessible through Gradio site, default: False
-      --insecure                       Enable extensions tab regardless of other options, default: False
-      --listen                         Launch web server using public IP address, default: False
-      --auth AUTH                      Set access authentication like "user:pwd,user:pwd""
-      --autolaunch                     Open the UI URL in the system's default browser upon launch
-      --docs                           Mount API docs, default: False
-      --no-hashing                     Disable hashing of checkpoints, default: False
-      --no-metadata                    Disable reading of metadata from models, default: False
-      --backend {original,diffusers}   force model pipeline type
+      --config CONFIG                                    Use specific server configuration file, default: config.json
+      --ui-config UI_CONFIG                              Use specific UI configuration file, default: ui-config.json
+      --medvram                                          Split model stages and keep only active part in VRAM, default: False
+      --lowvram                                          Split model components and keep only active part in VRAM, default: False
+      --ckpt CKPT                                        Path to model checkpoint to load immediately, default: None
+      --vae VAE                                          Path to VAE checkpoint to load immediately, default: None
+      --data-dir DATA_DIR                                Base path where all user data is stored, default:
+      --models-dir MODELS_DIR                            Base path where all models are stored, default: models
+      --allow-code                                       Allow custom script execution, default: False
+      --share                                            Enable UI accessible through Gradio site, default: False
+      --insecure                                         Enable extensions tab regardless of other options, default: False
+      --use-cpu USE_CPU [USE_CPU ...]                    Force use CPU for specified modules, default: []
+      --listen                                           Launch web server using public IP address, default: False
+      --port PORT                                        Launch web server with given server port, default: 7860
+      --freeze                                           Disable editing settings
+      --auth AUTH                                        Set access authentication like "user:pwd,user:pwd""
+      --auth-file AUTH_FILE                              Set access authentication using file, default: None
+      --autolaunch                                       Open the UI URL in the system's default browser upon launch
+      --docs                                             Mount API docs, default: False
+      --api-only                                         Run in API only mode without starting UI
+      --api-log                                          Enable logging of all API requests, default: False
+      --device-id DEVICE_ID                              Select the default CUDA device to use, default: None
+      --cors-origins CORS_ORIGINS                        Allowed CORS origins as comma-separated list, default: None
+      --cors-regex CORS_REGEX                            Allowed CORS origins as regular expression, default: None
+      --tls-keyfile TLS_KEYFILE                          Enable TLS and specify key file, default: None
+      --tls-certfile TLS_CERTFILE                        Enable TLS and specify cert file, default: None
+      --tls-selfsign                                     Enable TLS with self-signed certificates, default: False
+      --server-name SERVER_NAME                          Sets hostname of server, default: None
+      --no-hashing                                       Disable hashing of checkpoints, default: False
+      --no-metadata                                      Disable reading of metadata from models, default: False
+      --disable-queue                                    Disable queues, default: False
+      --subpath SUBPATH                                  Customize the URL subpath for usage with reverse proxy
+      --backend {original,diffusers}                     force model pipeline type
+      --allowed-paths ALLOWED_PATHS [ALLOWED_PATHS ...]  add additional paths to paths allowed for web access

    Setup options:
-      --debug                          Run installer with debug logging, default: False
-      --reset                          Reset main repository to latest version, default: False
-      --upgrade                        Upgrade main repository to latest version, default: False
-      --requirements                   Force re-check of requirements, default: False
-      --quick                          Run with startup sequence only, default: False
-      --use-directml                   Use DirectML if no compatible GPU is detected, default: False
-      --use-openvino                   Use Intel OpenVINO backend, default: False
-      --use-ipex                       Force use Intel OneAPI XPU backend, default: False
-      --use-cuda                       Force use nVidia CUDA backend, default: False
-      --use-rocm                       Force use AMD ROCm backend, default: False
-      --use-xformers                   Force use xFormers cross-optimization, default: False
-      --skip-requirements              Skips checking and installing requirements, default: False
-      --skip-extensions                Skips running individual extension installers, default: False
-      --skip-git                       Skips running all GIT operations, default: False
-      --skip-torch                     Skips running Torch checks, default: False
-      --skip-all                       Skips running all checks, default: False
-      --experimental                   Allow unsupported versions of libraries, default: False
-      --reinstall                      Force reinstallation of all requirements, default: False
-      --safe                           Run in safe mode with no user extensions
+      --reset                                            Reset main repository to latest version, default: False
+      --upgrade                                          Upgrade main repository to latest version, default: False
+      --requirements                                     Force re-check of requirements, default: False
+      --quick                                            Bypass version checks, default: False
+      --use-directml                                     Use DirectML if no compatible GPU is detected, default: False
+      --use-openvino                                     Use Intel OpenVINO backend, default: False
+      --use-ipex                                         Force use Intel OneAPI XPU backend, default: False
+      --use-cuda                                         Force use nVidia CUDA backend, default: False
+      --use-rocm                                         Force use AMD ROCm backend, default: False
+      --use-zluda                                        Force use ZLUDA, AMD GPUs only, default: False
+      --use-xformers                                     Force use xFormers cross-optimization, default: False
+      --skip-requirements                                Skips checking and installing requirements, default: False
+      --skip-extensions                                  Skips running individual extension installers, default: False
+      --skip-git                                         Skips running all GIT operations, default: False
+      --skip-torch                                       Skips running Torch checks, default: False
+      --skip-all                                         Skips running all checks, default: False
+      --skip-env                                         Skips setting of env variables during startup, default: False
+      --experimental                                     Allow unsupported versions of libraries, default: False
+      --reinstall                                        Force reinstallation of all requirements, default: False
+      --test                                             Run test only and exit
+      --version                                          Print version information
+      --ignore                                           Ignore any errors and attempt to continue
+      --safe                                             Run in safe mode with no user extensions

+    Logging options:
+      --log LOG                                          Set log file, default: None
+      --debug                                            Run installer with debug logging, default: False
+      --profile                                          Run profiler, default: False

 ## Notes

+### Control
+
+**SD.Next** comes with built-in control for all types of text2image, image2image, video2video and batch processing
+
+*Control interface*:  
+![Screenshot-Control](html/screenshot-control.jpg)
+
+*Control processors*:  
+![Screenshot-Process](html/screenshot-processors.jpg)
+
+*Masking*:
+![Screenshot-Mask](html/screenshot-mask.jpg)
+
 ### **Extensions**

 SD.Next comes with several extensions pre-installed:
@@ -1028,8 +1028,6 @@ def check_timestamp():

 def add_args(parser):
    group = parser.add_argument_group('Setup options')
-    group.add_argument("--log", type=str, default=os.environ.get("SD_LOG", None), help="Set log file, default: %(default)s")
-    group.add_argument('--debug', default = os.environ.get("SD_DEBUG",False), action='store_true', help = "Run installer with debug logging, default: %(default)s")
    group.add_argument('--reset', default = os.environ.get("SD_RESET",False), action='store_true', help = "Reset main repository to latest version, default: %(default)s")
    group.add_argument('--upgrade', default = os.environ.get("SD_UPGRADE",False), action='store_true', help = "Upgrade main repository to latest version, default: %(default)s")
    group.add_argument('--requirements', default = os.environ.get("SD_REQUIREMENTS",False), action='store_true', help = "Force re-check of requirements, default: %(default)s")
@@ -1039,6 +1037,7 @@ def add_args(parser):
    group.add_argument("--use-ipex", default = os.environ.get("SD_USEIPEX",False), action='store_true', help="Force use Intel OneAPI XPU backend, default: %(default)s")
    group.add_argument("--use-cuda", default = os.environ.get("SD_USECUDA",False), action='store_true', help="Force use nVidia CUDA backend, default: %(default)s")
    group.add_argument("--use-rocm", default = os.environ.get("SD_USEROCM",False), action='store_true', help="Force use AMD ROCm backend, default: %(default)s")
+    group.add_argument('--use-zluda', default=os.environ.get("SD_USEZLUDA", False), action='store_true', help = "Force use ZLUDA, AMD GPUs only, default: %(default)s")
    group.add_argument("--use-xformers", default = os.environ.get("SD_USEXFORMERS",False), action='store_true', help="Force use xFormers cross-optimization, default: %(default)s")
    group.add_argument('--skip-requirements', default = os.environ.get("SD_SKIPREQUIREMENTS",False), action='store_true', help = "Skips checking and installing requirements, default: %(default)s")
    group.add_argument('--skip-extensions', default = os.environ.get("SD_SKIPEXTENSION",False), action='store_true', help = "Skips running individual extension installers, default: %(default)s")
@@ -1053,6 +1052,13 @@ def add_args(parser):
    group.add_argument('--ignore', default = os.environ.get("SD_IGNORE",False), action='store_true', help = "Ignore any errors and attempt to continue")
    group.add_argument('--safe', default = os.environ.get("SD_SAFE",False), action='store_true', help = "Run in safe mode with no user extensions")

+    group = parser.add_argument_group('Logging options')
+    group.add_argument("--log", type=str, default=os.environ.get("SD_LOG", None), help="Set log file, default: %(default)s")
+    group.add_argument('--debug', default = os.environ.get("SD_DEBUG",False), action='store_true', help = "Run installer with debug logging, default: %(default)s")
+    group.add_argument("--profile", default=os.environ.get("SD_PROFILE", False), action='store_true', help="Run profiler, default: %(default)s")
+    group.add_argument('--docs', default=os.environ.get("SD_DOCS", False), action='store_true', help = "Mount API docs, default: %(default)s")
+    group.add_argument("--api-log", default=os.environ.get("SD_APILOG", False), action='store_true', help="Enable logging of all API requests, default: %(default)s")
+

 def parse_args(parser):
    # command line args
@@ -27,7 +27,6 @@ group.add_argument("--auth-file", type=str, default=os.environ.get("SD_AUTHFILE"
 group.add_argument("--autolaunch", default=os.environ.get("SD_AUTOLAUNCH", False), action='store_true', help="Open the UI URL in the system's default browser upon launch")
 group.add_argument('--docs', default=os.environ.get("SD_DOCS", False), action='store_true', help = "Mount API docs, default: %(default)s")
 group.add_argument('--api-only', default=os.environ.get("SD_APIONLY", False), action='store_true', help = "Run in API only mode without starting UI")
-group.add_argument("--api-log", default=os.environ.get("SD_APILOG", False), action='store_true', help="Enable logging of all API requests, default: %(default)s")
 group.add_argument("--device-id", type=str, default=os.environ.get("SD_DEVICEID", None), help="Select the default CUDA device to use, default: %(default)s")
 group.add_argument("--cors-origins", type=str, default=os.environ.get("SD_CORSORIGINS", None), help="Allowed CORS origins as comma-separated list, default: %(default)s")
 group.add_argument("--cors-regex", type=str, default=os.environ.get("SD_CORSREGEX", None), help="Allowed CORS origins as regular expression, default: %(default)s")
@@ -4,7 +4,7 @@ from typing import Union
 from diffusers import StableDiffusionPipeline, StableDiffusionXLPipeline, ControlNetModel, StableDiffusionControlNetPipeline, StableDiffusionXLControlNetPipeline
 from modules.control.units import detect
 from modules.shared import log, opts, listdir
-from modules import errors
+from modules import errors, sd_models


 what = 'ControlNet'
@@ -193,7 +193,8 @@ class ControlNetPipeline():
                scheduler=pipeline.scheduler,
                feature_extractor=getattr(pipeline, 'feature_extractor', None),
                controlnet=controlnet, # can be a list
-            ).to(pipeline.device)
+            )
+            sd_models.move_model(self.pipeline, pipeline.device)
        elif detect.is_sd15(pipeline):
            self.pipeline = StableDiffusionControlNetPipeline(
                vae=pipeline.vae,
@@ -205,7 +206,8 @@ class ControlNetPipeline():
                requires_safety_checker=False,
                safety_checker=None,
                controlnet=controlnet, # can be a list
-            ).to(pipeline.device)
+            )
+            sd_models.move_model(self.pipeline, pipeline.device)
        else:
            log.error(f'Control {what} pipeline: class={pipeline.__class__.__name__} unsupported model type')
            return
@@ -4,6 +4,7 @@ import diffusers.utils
 from diffusers import StableDiffusionPipeline, StableDiffusionXLPipeline
 from modules.shared import log, opts
 from modules.control.units import detect
+from modules import sd_models


 what = 'Reference'
@@ -34,7 +35,8 @@ class ReferencePipeline():
                unet=pipeline.unet,
                scheduler=pipeline.scheduler,
                feature_extractor=getattr(pipeline, 'feature_extractor', None),
-            ).to(pipeline.device)
+            )
+            sd_models.move_model(self.pipeline, pipeline.device)
        elif detect.is_sd15(pipeline):
            cls = diffusers.utils.get_class_from_dynamic_module('stable_diffusion_reference', module_file='pipeline.py')
            self.pipeline = cls(
@@ -46,7 +48,8 @@ class ReferencePipeline():
                feature_extractor=getattr(pipeline, 'feature_extractor', None),
                requires_safety_checker=False,
                safety_checker=None,
-            ).to(pipeline.device)
+            )
+            sd_models.move_model(self.pipeline, pipeline.device)
        else:
            log.error(f'Control {what} pipeline: class={pipeline.__class__.__name__} unsupported model type')
            return
@@ -3,7 +3,7 @@ import time
 from typing import Union
 from diffusers import StableDiffusionPipeline, StableDiffusionXLPipeline, T2IAdapter, MultiAdapter, StableDiffusionAdapterPipeline, StableDiffusionXLAdapterPipeline # pylint: disable=unused-import
 from modules.shared import log
-from modules import errors
+from modules import errors, sd_models
 from modules.control.units import detect


@@ -128,7 +128,8 @@ class AdapterPipeline():
                scheduler=pipeline.scheduler,
                feature_extractor=getattr(pipeline, 'feature_extractor', None),
                adapter=adapter,
-            ).to(pipeline.device)
+            )
+            sd_models.move_model(self.pipeline, pipeline.device)
        elif detect.is_sd15(pipeline):
            self.pipeline = StableDiffusionAdapterPipeline(
                vae=pipeline.vae,
@@ -140,7 +141,8 @@ class AdapterPipeline():
                requires_safety_checker=False,
                safety_checker=None,
                adapter=adapter,
-            ).to(pipeline.device)
+            )
+            sd_models.move_model(self.pipeline, pipeline.device)
        else:
            log.error(f'Control {what} pipeline: class={pipeline.__class__.__name__} unsupported model type')
            return
@@ -3,7 +3,7 @@ import time
 from typing import Union
 from diffusers import StableDiffusionPipeline, StableDiffusionXLPipeline
 from modules.shared import log, opts, listdir
-from modules import errors
+from modules import errors, sd_models
 from modules.control.units.xs_model import ControlNetXSModel
 from modules.control.units.xs_pipe import StableDiffusionControlNetXSPipeline, StableDiffusionXLControlNetXSPipeline
 from modules.control.units import detect
@@ -126,7 +126,8 @@ class ControlNetXSPipeline():
                scheduler=pipeline.scheduler,
                # feature_extractor=getattr(pipeline, 'feature_extractor', None),
                controlnet=controlnet, # can be a list
-            ).to(pipeline.device)
+            )
+            sd_models.move_model(self.pipeline, pipeline.device)
        elif detect.is_sd15(pipeline):
            self.pipeline = StableDiffusionControlNetXSPipeline(
                vae=pipeline.vae,
@@ -138,7 +139,8 @@ class ControlNetXSPipeline():
                requires_safety_checker=False,
                safety_checker=None,
                controlnet=controlnet, # can be a list
-            ).to(pipeline.device)
+            )
+            sd_models.move_model(self.pipeline, pipeline.device)
        else:
            log.error(f'Control {what} pipeline: class={pipeline.__class__.__name__} unsupported model type')
            return
@@ -1130,7 +1130,8 @@ def switch_pipe(cls: diffusers.DiffusionPipeline, pipeline: diffusers.DiffusionP
                unet=pipeline.unet,
                scheduler=pipeline.scheduler,
                feature_extractor=getattr(pipeline, 'feature_extractor', None),
-            ).to(pipeline.device)
+            )
+            move_model(new_pipe, pipeline.device)
            switch_mode = 'sdxl'
        elif 'tokenizer' in possible and hasattr(pipeline, 'tokenizer'):
            new_pipe = cls(
@@ -1142,7 +1143,8 @@ def switch_pipe(cls: diffusers.DiffusionPipeline, pipeline: diffusers.DiffusionP
                feature_extractor=getattr(pipeline, 'feature_extractor', None),
                requires_safety_checker=False,
                safety_checker=None,
-            ).to(pipeline.device)
+            )
+            move_model(new_pipe, pipeline.device)
            switch_mode = 'sd'
        else:
            shared.log.error(f'Pipeline switch error: {pipeline.__class__.__name__} unrecognized')