Public/automatic
1
0
Fork 0
mirror of https://github.com/vladmandic/automatic synced 2026-09-19 17:24:32 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'dev' into SD3-parsing

Browse Source
This commit is contained in:
AI-Casanova
2024-06-19 23:47:17 -05:00
committed by GitHub
parent e3009753e3 eb6e3c337c
commit 1fcd378bd1
43 changed files with 442 additions and 279 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+40 -12
View File
@@ -3,44 +3,72 @@
## Pending
- Diffusers==0.30.0
- https://github.com/huggingface/diffusers/issues/8546
- https://github.com/huggingface/diffusers/pull/8566
- https://github.com/huggingface/diffusers/pull/8584
## Update for 2024-06-16
## Update for 2024-06-19
### Improvements: SD3
### Highlights for 2024-06-19
- enable taesd preview and non-full quality mode
- enable base LoRA support
- simplified loading of model in single-file safetensors format
loading sd3 can now be performed fully offline
- add support for nncf compressed weights, thanks @Disty0!
- add support for sampler shift for Euler FlowMatch
Following zero-day **SD3** release, a week later here's a refresh with more than a few improvements.
But there's more than SD3:
- support for quantized **T5** text encoder in all models that use T5: FP4/FP8/FP16/INT8 (SD3, PixArt-Σ, etc)
- support for **PixArt-Sigma** in small/medium/large variants
- support for **HunyuanDiT 1.1**
- (finally) new release of **Torch-DirectML**
### Model Improvements
- **SD3**: enable tiny-VAE (TAESD) preview and non-full quality mode
- SD3: enable base LoRA support
- SD3: add support for FP4 quantized T5 text encoder
simply select in *settings -> model -> text encoder*
- SD3: add support for INT8 quantized T5 text encoder, thanks @Disty0!
- SD3: enable cpu-offloading for T5 text encoder, thanks @Disty0!
- SD3: simplified loading of model in single-file safetensors format
model load can now be performed fully offline
- SD3: add support for NNCF compressed weights, thanks @Disty0!
- SD3: add support for sampler shift for Euler FlowMatch
see *settings -> samplers*, also available as param in xyz grid
higher shift means model will spend more time on structure and less on details
- SD3: add support for selecting T5 text encoder variant in XYZ grid
- **Pixart-Σ**: Add *small* (512px) and *large* (2k) variations, in addition to existing *medium* (1k)
- Pixart-Σ: Add support for 4/8bit quantized t5 text encoder
*note* by default pixart-Σ uses full fp16 t5 encoder with large memory footprint
simply select in *settings -> model -> text encoder* before or after model load
- **HunyuanDiT**: support for model version 1.1
### Improvements: General
- support FP4 quantized T5 text encoder, in addtion to existing FP8 and FP16
- support for T5 text-encoder loader in **all** models that use T5
*example*: load FP8 quantized T5 text-encoder into PixArt Sigma
*example*: load FP4 or FP8 quantized T5 text-encoder into PixArt Sigma or Stable Cascade!
- support for `torch-directml` **0.2.2**, thanks @lshqqytiger!
*note*: new directml is finally based on modern `torch` 2.3.1!
- extra networks: info display now contains link to source url if model if its known
works for civitai and huggingface models
- improved google.colab support
- css tweaks for standardui
- css tweaks for modernui
### Fixes
- fix unsaturated outputs, force apply vae config on model load
- fix hidiffusion handling of non-square aspect ratios, thanks @ShenZhang-Shin!
- fix control second pass resize
- fix api face-hires
- fix **hunyuandit** set attention processor
- fix hunyuandit set attention processor
- fix civitai download without name
- fix compatibility with latest adetailer
- fix invalid sampler warning
- fix starting from non git repo
- fix control api negative prompt handling
- fix saving style without name provided
- fix t2i-color adapter
- fix sdxl "has been incorrectly initialized"
- fix api face-hires
- fix api ip-adapter
- cleanup image metadata
- restructure api examples: `cli/api-*`
- handle theme fallback when invalid theme is specified
- remove obsolete training code leftovers
cli/api-json.py
Executable
+52
View File
@@ -0,0 +1,52 @@
#!/usr/bin/env python
# curl -vX POST http://localhost:7860/sdapi/v1/txt2img --header "Content-Type: application/json" -d @3261.json
import os
import json
import logging
import argparse
import requests
import urllib3
sd_url = os.environ.get('SDAPI_URL', "http://127.0.0.1:7860")
sd_username = os.environ.get('SDAPI_USR', None)
sd_password = os.environ.get('SDAPI_PWD', None)
options = {
"save_images": True,
"send_images": True,
}
logging.basicConfig(level = logging.INFO, format = '%(asctime)s %(levelname)s: %(message)s')
log = logging.getLogger(__name__)
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
def auth():
if sd_username is not None and sd_password is not None:
return requests.auth.HTTPBasicAuth(sd_username, sd_password)
return None
def post(endpoint: str, payload: dict = None):
if 'sdapi' not in endpoint:
endpoint = f'sdapi/v1/{endpoint}'
if 'http' not in endpoint:
endpoint = f'{sd_url}/{endpoint}'
req = requests.post(endpoint, json = payload, timeout=300, verify=False, auth=auth())
return { 'error': req.status_code, 'reason': req.reason, 'url': req.url } if req.status_code != 200 else req.json()
if __name__ == "__main__":
parser = argparse.ArgumentParser(description = 'api-txt2img')
parser.add_argument('endpoint', nargs=1, help='endpoint')
parser.add_argument('json', nargs=1, help='json data or file')
args = parser.parse_args()
log.info(f'api-json: {args}')
if os.path.isfile(args.json[0]):
with open(args.json[0], 'r', encoding='ascii') as f:
dct = json.load(f) # TODO fails with b64 encoded images inside json due to string encoding
else:
dct = json.loads(args.json[0])
res = post(endpoint=args.endpoint[0], payload=dct)
print(res)
cli/image-encode.py
Executable
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env python
import io
import os
import sys
import base64
from PIL import Image
from rich import print # pylint: disable=redefined-builtin
def encode(file: str):
image = Image.open(file) if os.path.exists(file) else None
print(f'Input: file={file} image={image}')
if image is None:
return None
if image.mode != 'RGB':
image = image.convert('RGB')
with io.BytesIO() as stream:
image.save(stream, 'JPEG')
image.close()
values = stream.getvalue()
encoded = base64.b64encode(values).decode()
return encoded
if __name__ == "__main__":
sys.argv.pop(0)
fn = sys.argv[0] if len(sys.argv) > 0 else ''
b64 = encode(fn)
print('=== BEGIN ===')
print(f'{b64}')
print('=== END ===')
extensions-builtin/Lora/network_overrides.py
+12 -3
View File
@@ -1,7 +1,7 @@
from modules import shared
maybe_diffusers = [
maybe_diffusers = [ # forced if lora_maybe_diffusers is enabled
'aaebf6360f7d', # sd15-lcm
'3d18b05e4f56', # sdxl-lcm
'b71dcb732467', # sdxl-tcd
@@ -19,14 +19,23 @@ maybe_diffusers = [
'8cca3706050b', # hyper-sdxl-1step
]
force_diffusers = [
force_diffusers = [ # forced always
'816d0eed49fd', # flash-sdxl
'c2ec22757b46', # flash-sd15
]
force_models = [ # forced always
'sd3',
]
force_classes = [ # forced always
]
def check_override(shorthash=''):
force = False
force = force or (shared.sd_model_type == 'sd3') # TODO sd3 forced diffusers for lora load
force = force or (shared.sd_model_type in force_models)
force = force or (shared.sd_model.__class__.__name__ in force_classes)
if len(shorthash) < 4:
return force
force = force or (any(x.startswith(shorthash) for x in maybe_diffusers) if shared.opts.lora_maybe_diffusers else False)
extensions-builtin/Lora/networks.py
+6
View File
@@ -49,6 +49,7 @@ def assign_network_names_to_compvis_modules(sd_model):
network_layer_mapping = {}
if shared.native:
if not hasattr(shared.sd_model, 'text_encoder') or not hasattr(shared.sd_model, 'unet'):
sd_model.network_layer_mapping = {}
return
for name, module in shared.sd_model.text_encoder.named_modules():
prefix = "lora_te1_" if shared.sd_model_type == "sdxl" else "lora_te_"
@@ -66,6 +67,7 @@ def assign_network_names_to_compvis_modules(sd_model):
module.network_layer_name = network_name
else:
if not hasattr(shared.sd_model, 'cond_stage_model'):
sd_model.network_layer_mapping = {}
return
for name, module in shared.sd_model.cond_stage_model.wrapped.named_modules():
network_name = name.replace(".", "_")
@@ -87,10 +89,14 @@ def load_diffusers(name, network_on_disk, lora_scale=1.0) -> network.Network:
return cached
if not shared.native:
return None
if not hasattr(shared.sd_model, 'load_lora_weights'):
shared.log.error(f"LoRA load failed: class={shared.sd_model.__class__} does not implement load lora")
return None
try:
shared.sd_model.load_lora_weights(network_on_disk.filename)
except Exception as e:
errors.display(e, "LoRA")
return None
if shared.opts.lora_fuse_diffusers:
shared.sd_model.fuse_lora(lora_scale=lora_scale)
net = network.Network(name, network_on_disk)
extensions-builtin/Lora/ui_extra_networks_lora.py
+1 -1
View File
@@ -102,7 +102,7 @@ class ExtraNetworksPageLora(ui_extra_networks.ExtraNetworksPage):
return item
except Exception as e:
shared.log.debug(f"Extra networks error: type=lora file={name} {e}")
shared.log.debug(f"Networks error: type=lora file={name} {e}")
from modules import errors
errors.display('e', 'Lora')
return None
extensions-builtin/sdnext-modernui
+1 -1
Submodule extensions-builtin/sdnext-modernui updated: 285743a83f...dae2c67d82
html/locale_en.json
+1 -1
View File
@@ -230,7 +230,7 @@
{"id":"","label":"Control Options","localized":"","hint":"Settings related the Control tab"},
{"id":"","label":"Training","localized":"","hint":"Settings related to model training configuration and directories"},
{"id":"","label":"Interrogate","localized":"","hint":"Settings related to interrogation configuration"},
{"id":"","label":"Extra Networks","localized":"","hint":"Settings related to extra networks user interface, extra networks multiplier defaults, and configuration"},
{"id":"","label":"Networks","localized":"","hint":"Settings related to networks user interface, networks multiplier defaults, and configuration"},
{"id":"","label":"Licenses","localized":"","hint":"View licenses of all additional included libraries"},
{"id":"","label":"Show all pages","localized":"","hint":"Show all settings pages"}
],
html/locale_ko.json
+1 -1
View File
@@ -48,7 +48,7 @@
{"id":"","label":"Interrogate\nDeepBooru","localized":"DeepBooru 모델 사용","hint":"DeepBooru 모델을 사용해 이미지에서 설명을 추출한다."}
],
"extra networks": [
{"id":"","label":"Extra networks tab order","localized":"엑스트라 네트워크 탭 순서","hint":"Comma-separated list of tab names; tabs listed here will appear in the extra networks UI first and in order lsited"},
{"id":"","label":"Networks tab order","localized":"엑스트라 네트워크 탭 순서","hint":"Comma-separated list of tab names; tabs listed here will appear in the extra networks UI first and in order lsited"},
{"id":"","label":"UI position","localized":"UI 위치","hint":""},
{"id":"","label":"UI height (%)","localized":"UI 높이 (%)","hint":""},
{"id":"","label":"UI sidebar width (%)","localized":"UI 사이드바 너비 (%)","hint":""},
html/reference.json
+19 -4
View File
@@ -160,15 +160,30 @@
"preview": "PixArt-alpha--PixArt-XL-2-1024-MS.jpg",
"extras": "width: 1024, height: 1024, sampler: Default, cfg_scale: 2.0"
},
"Pixart-Σ": {
"path": "PixArt-alpha/PixArt-Sigma-XL-2-1024-MS",
"Pixart-Σ Small": {
"path": "huggingface/PixArt-alpha/PixArt-Sigma-XL-2-512-MS",
"desc": "PixArt-Σ, a Diffusion Transformer model (DiT) capable of directly generating images at 4K resolution. PixArt-Σ represents a significant advancement over its predecessor, PixArt-α, offering images of markedly higher fidelity and improved alignment with text prompts.",
"preview": "PixArt-alpha--pixart_sigma_sdxlvae_T5_diffusers.jpg",
"skip": true,
"extras": "width: 512, height: 512, sampler: Default, cfg_scale: 2.0"
},
"Pixart-Σ Medium": {
"path": "huggingface/PixArt-alpha/PixArt-Sigma-XL-2-1024-MS",
"desc": "PixArt-Σ, a Diffusion Transformer model (DiT) capable of directly generating images at 4K resolution. PixArt-Σ represents a significant advancement over its predecessor, PixArt-α, offering images of markedly higher fidelity and improved alignment with text prompts.",
"preview": "PixArt-alpha--pixart_sigma_sdxlvae_T5_diffusers.jpg",
"skip": true,
"extras": "width: 1024, height: 1024, sampler: Default, cfg_scale: 2.0"
},
"Pixart-Σ Large": {
"path": "huggingface/PixArt-alpha/PixArt-Sigma-XL-2-2K-MS",
"desc": "PixArt-Σ, a Diffusion Transformer model (DiT) capable of directly generating images at 4K resolution. PixArt-Σ represents a significant advancement over its predecessor, PixArt-α, offering images of markedly higher fidelity and improved alignment with text prompts.",
"preview": "PixArt-alpha--pixart_sigma_sdxlvae_T5_diffusers.jpg",
"skip": true,
"extras": "width: 1024, height: 1024, sampler: Default, cfg_scale: 2.0"
},
"Tencent HunyuanDiT": {
"path": "Tencent-Hunyuan/HunyuanDiT-Diffusers",
"Tencent HunyuanDiT 1.1": {
"path": "Tencent-Hunyuan/HunyuanDiT-v1.1-Diffusers",
"desc": "Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding.",
"preview": "Tencent-Hunyuan-HunyuanDiT.jpg",
"extras": "width: 1024, height: 1024, sampler: Default, cfg_scale: 2.0"
installer.py
+24 -10
View File
@@ -275,9 +275,12 @@ def install(package, friendly: str = None, ignore: bool = False, reinstall: bool
# execute git command
@lru_cache()
def git(arg: str, folder: str = None, ignore: bool = False):
def git(arg: str, folder: str = None, ignore: bool = False, optional: bool = False):
if args.skip_git:
return ''
if optional:
if 'google.colab' in sys.modules:
return ''
git_cmd = os.environ.get('GIT', "git")
if git_cmd != "git":
git_cmd = os.path.abspath(git_cmd)
@@ -306,7 +309,7 @@ def branch(folder=None):
return None
branches = []
try:
b = git('branch --show-current', folder)
b = git('branch --show-current', folder, optional=True)
if b == '':
branches = git('branch', folder).split('\n')
if len(branches) > 0:
@@ -315,7 +318,7 @@ def branch(folder=None):
b = branches[1].strip()
log.debug(f'Git detached head detected: folder="{folder}" reattach={b}')
except Exception:
b = git('git rev-parse --abbrev-ref HEAD', folder)
b = git('git rev-parse --abbrev-ref HEAD', folder, optional=True)
if 'main' in b:
b = 'main'
elif 'master' in b:
@@ -323,7 +326,7 @@ def branch(folder=None):
else:
b = b.split('\n')[0].replace('*', '').strip()
log.debug(f'Submodule: {folder} / {b}')
git(f'checkout {b}', folder, ignore=True)
git(f'checkout {b}', folder, ignore=True, optional=True)
return b
@@ -396,6 +399,12 @@ def check_python(supported_minors=[9, 10, 11, 12], reason=None):
if args.quick:
return
log.info(f'Python version={platform.python_version()} platform={platform.system()} bin="{sys.executable}" venv="{sys.prefix}"')
if int(sys.version_info.major) == 3 and int(sys.version_info.minor) == 12 and int(sys.version_info.minor) > 3: # TODO python 3.12.4 or higher cause a mess with pydantic
log.error(f"Incompatible Python version: {sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro} required 3.12.3 or lower")
if reason is not None:
log.error(reason)
if not args.ignore:
sys.exit(1)
if not (int(sys.version_info.major) == 3 and int(sys.version_info.minor) in supported_minors):
log.error(f"Incompatible Python version: {sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro} required 3.{supported_minors}")
if reason is not None:
@@ -1035,19 +1044,24 @@ def get_version(force=False):
def check_ui(ver):
if ver is None or 'branch' not in ver or 'ui' not in ver or ver['branch'] == ver['ui']:
return
log.debug(f'Branch mismatch: sdnext={ver["branch"]} ui={ver["ui"]}')
def same(ver):
core = ver['branch'] if ver is not None and 'branch' in ver else 'unknown'
ui = ver['ui'] if ver is not None and 'ui' in ver else 'unknown'
return core == ui or (core == 'master' and ui == 'main')
if not same(ver):
log.debug(f'Branch mismatch: sdnext={ver["branch"]} ui={ver["ui"]}')
cwd = os.getcwd()
try:
os.chdir('extensions-builtin/sdnext-modernui')
git('checkout ' + ver['branch'], ignore=True)
target = 'dev' if 'dev' in ver['branch'] else 'main'
git('checkout ' + target, ignore=True, optional=True)
os.chdir(cwd)
ver = get_version(force=True)
if ver['branch'] == ver['ui']:
if not same(ver):
log.debug(f'Branch synchronized: {ver["branch"]}')
else:
log.debug(f'Branch synch failed: sdnext={ver["branch"]} ui={ver["ui"]}')
log.debug(f'Branch sync failed: sdnext={ver["branch"]} ui={ver["ui"]}')
except Exception as e:
log.debug(f'Branch switch: {e}')
os.chdir(cwd)
javascript/base.css
+1
View File
@@ -17,6 +17,7 @@
.tooltip-show { opacity: 0.9; }
.tooltip-left { right: unset; left: 1em; }
.toolbutton-selected { background: var(--background-fill-primary) !important; }
.input-accordion-checkbox { display: none; }
/* live preview */
.progressDiv { position: relative; height: 20px; background: #b4c0cc; margin-bottom: -3px; }
javascript/inputAccordion.js
+10 -21
View File
@@ -10,13 +10,10 @@ function setupAccordion(accordion) {
const extra = gradioApp().querySelector(`#${accordion.id}-extra`);
const span = labelWrap.querySelector('span');
let linked = true;
const isOpen = () => labelWrap.classList.contains('open');
const observerAccordionOpen = new MutationObserver((mutations) => {
mutations.forEach((mutationRecord) => {
accordion.classList.toggle('input-accordion-open', isOpen());
if (linked) {
accordion.visibleCheckbox.checked = isOpen();
accordion.onVisibleCheckboxChange();
@@ -24,15 +21,9 @@ function setupAccordion(accordion) {
});
});
observerAccordionOpen.observe(labelWrap, { attributes: true, attributeFilter: ['class'] });
if (extra) {
labelWrap.insertBefore(extra, labelWrap.lastElementChild);
}
if (extra) labelWrap.insertBefore(extra, labelWrap.lastElementChild);
accordion.onChecked = (checked) => {
if (isOpen() !== checked) {
labelWrap.click();
}
if (isOpen() !== checked) labelWrap.click();
};
const visibleCheckbox = document.createElement('INPUT');
@@ -41,13 +32,9 @@ function setupAccordion(accordion) {
visibleCheckbox.id = `${accordion.id}-visible-checkbox`;
visibleCheckbox.className = `${gradioCheckbox.className} input-accordion-checkbox`;
span.insertBefore(visibleCheckbox, span.firstChild);
accordion.visibleCheckbox = visibleCheckbox;
accordion.onVisibleCheckboxChange = () => {
if (linked && isOpen() !== visibleCheckbox.checked) {
labelWrap.click();
}
if (linked && isOpen() !== visibleCheckbox.checked) labelWrap.click();
gradioCheckbox.checked = visibleCheckbox.checked;
updateInput(gradioCheckbox);
};
@@ -59,8 +46,10 @@ function setupAccordion(accordion) {
visibleCheckbox.addEventListener('input', accordion.onVisibleCheckboxChange);
}
onUiLoaded(() => {
for (const accordion of gradioApp().querySelectorAll('.input-accordion')) {
setupAccordion(accordion);
}
});
// onUiLoaded(() => {
// for (const accordion of gradioApp().querySelectorAll('.input-accordion')) setupAccordion(accordion);
// });
function initAccordions() {
for (const accordion of gradioApp().querySelectorAll('.input-accordion')) setupAccordion(accordion);
}
javascript/sdnext.css
+1 -1
View File
@@ -240,7 +240,7 @@ table.settings-value-table td { padding: 0.4em; border: 1px solid #ccc; max-widt
.extra-details > div { overflow-y: auto; min-height: 40vh; max-height: 80vh; align-self: flex-start; }
.extra-details td:first-child { font-weight: bold; vertical-align: top; }
.extra-details .gradio-image { max-height: 50vh; }
.input-accordion-checkbox { display: none !important; }
/* specific elements */
#modelmerger_interp_description { margin-top: 1em; margin-bottom: 1em; }
javascript/startup.js
+1
View File
@@ -12,6 +12,7 @@ async function initStartup() {
initLogMonitor();
initContextMenu();
initDragDrop();
initAccordions();
initSettings();
initImageViewer();
initGallery();
javascript/ui.js
+1
View File
@@ -424,6 +424,7 @@ function selectVAE(name) {
}
function selectReference(name) {
log(`Select reference: ${name}`);
desiredCheckpointName = name;
gradioApp().getElementById('change_reference').click();
}
modules/api/models.py
+2 -2
View File
@@ -152,8 +152,8 @@ class ItemIPAdapter(BaseModel):
adapter: str = Field(title="Adapter", default="Base", description="")
images: List[str] = Field(title="Image", default=[], description="")
masks: Optional[List[str]] = Field(title="Mask", default=[], description="")
scale: float = Field(title="Scale", default=0.5, gt=0, le=1, description="")
start: float = Field(title="Start", default=0.0, gt=0, le=1, description="")
scale: float = Field(title="Scale", default=0.5, ge=0, le=1, description="")
start: float = Field(title="Start", default=0.0, ge=0, le=1, description="")
end: float = Field(title="End", default=1.0, gt=0, le=1, description="")
class ItemFace(BaseModel):
modules/control/run.py
+2 -2
View File
@@ -55,7 +55,7 @@ def control_set(kwargs):
def control_run(units: List[unit.Unit] = [], inputs: List[Image.Image] = [], inits: List[Image.Image] = [], mask: Image.Image = None, unit_type: str = None, is_generator: bool = True,
input_type: int = 0,
prompt: str = '', negative: str = '', styles: List[str] = [],
prompt: str = '', negative_prompt: str = '', styles: List[str] = [],
steps: int = 20, sampler_index: int = None,
seed: int = -1, subseed: int = -1, subseed_strength: float = 0, seed_resize_from_h: int = -1, seed_resize_from_w: int = -1,
cfg_scale: float = 6.0, clip_skip: float = 1.0, image_cfg_scale: float = 6.0, diffusers_guidance_rescale: float = 0.7, pag_scale: float = 0.0, pag_adaptive: float = 0.5, cfg_end: float = 1.0,
@@ -94,7 +94,7 @@ def control_run(units: List[unit.Unit] = [], inputs: List[Image.Image] = [], ini
p = StableDiffusionProcessingControl(
prompt = prompt,
negative_prompt = negative,
negative_prompt = negative_prompt,
styles = styles,
steps = steps,
n_iter = batch_count,
modules/control/units/controlnet.py
+11 -2
View File
@@ -172,10 +172,19 @@ class ControlNet():
self.load_safetensors(model_path)
else:
self.model = ControlNetModel.from_pretrained(model_path, **self.load_config)
if self.device is not None:
self.model.to(self.device)
if self.dtype is not None:
self.model.to(self.dtype)
if "ControlNet" in opts.nncf_compress_weights:
try:
log.debug(f'Control {what} model NNCF Compress: id="{model_id}"')
from installer import install
install('nncf==2.7.0', quiet=True)
from modules.sd_models_compile import nncf_compress_model
self.model = nncf_compress_model(self.model)
except Exception as e:
log.error(f'Control {what} model NNCF Compression failed: id="{model_id}" error={e}')
if self.device is not None:
self.model.to(self.device)
t1 = time.time()
self.model_id = model_id
log.debug(f'Control {what} model loaded: id="{model_id}" path="{model_path}" time={t1-t0:.2f}')
modules/control/units/t2iadapter.py
+2 -2
View File
@@ -74,7 +74,7 @@ class Adapter():
self.model_id: str = model_id
self.device = device
self.dtype = dtype
self.load_config = { 'cache_dir': cache_dir }
self.load_config = { 'cache_dir': cache_dir, 'use_safetensors': False }
if load_config is not None:
self.load_config.update(load_config)
if model_id is not None:
@@ -101,7 +101,7 @@ class Adapter():
log.error(f'Control {what} model load failed: id="{model_id}" error=unknown model id')
return
log.debug(f'Control {what} model loading: id="{model_id}" path="{model_path}"')
if model_path.endswith('.pth') or model_path.endswith('.pt') or model_path.endswith('.safetensors'):
if model_path.endswith('.pth') or model_path.endswith('.pt') or model_path.endswith('.safetensors') or model_path.endswith('.bin'):
from huggingface_hub import hf_hub_download
parts = model_path.split('/')
repo_id = f'{parts[0]}/{parts[1]}'
modules/devices.py
+1 -1
View File
@@ -175,7 +175,7 @@ def set_cuda_sync_mode(mode):
return
try:
import ctypes
log.info(f'Set cuda synch: mode={mode}')
log.info(f'Set cuda sync: mode={mode}')
torch.cuda.set_device(torch.device(get_optimal_device_name()))
ctypes.CDLL('libcudart.so').cudaSetDeviceFlags({'auto': 0, 'spin': 1, 'yield': 2, 'block': 4}[mode])
except Exception:
modules/model_pixart.py
+30
View File
@@ -0,0 +1,30 @@
import diffusers
def load_pixart(checkpoint_info, diffusers_load_config={}):
from modules import shared, devices, modelloader, model_t5
modelloader.hf_login()
# shared.opts.data['cuda_dtype'] = 'FP32' # override
# shared.opts.data['diffusers_model_cpu_offload'] = True # override
# devices.set_cuda_params()
fn = checkpoint_info.path.replace('huggingface/', '')
t5 = model_t5.load_t5(shared.opts.sd_text_encoder, cache_dir=shared.opts.diffusers_dir)
transformer = diffusers.PixArtTransformer2DModel.from_pretrained(
fn,
subfolder = 'transformer',
cache_dir = shared.opts.diffusers_dir,
**diffusers_load_config,
)
transformer.to(devices.device)
kwargs = { 'transformer': transformer }
if t5 is not None:
kwargs['text_encoder'] = t5
diffusers_load_config.pop('variant', None)
pipe = diffusers.PixArtSigmaPipeline.from_pretrained(
'PixArt-alpha/PixArt-Sigma-XL-2-1024-MS',
cache_dir = shared.opts.diffusers_dir,
**kwargs,
**diffusers_load_config,
)
devices.torch_gc()
return pipe
modules/model_sd3.py
+1 -55
View File
@@ -1,14 +1,7 @@
import os
import warnings
import torch
import diffusers
import transformers
import rich.traceback
rich.traceback.install()
warnings.filterwarnings(action="ignore", category=FutureWarning)
loggedin = False
def load_sd3(fn=None, cache_dir=None, config=None):
@@ -48,7 +41,7 @@ def load_sd3(fn=None, cache_dir=None, config=None):
),
'text_encoder_3': None,
}
elif fn_size < 1e10: # if model is below 10gb it does not have te4
elif fn_size < 1e10: # if model is below 10gb it does not have te3
kwargs = {
'text_encoder_3': None,
}
@@ -69,50 +62,3 @@ def load_sd3(fn=None, cache_dir=None, config=None):
diffusers.pipelines.auto_pipeline.AUTO_IMAGE2IMAGE_PIPELINES_MAPPING["stable-diffusion-3"] = diffusers.StableDiffusion3Img2ImgPipeline
devices.torch_gc()
return pipe
def load_t5(pipe, module, te3=None, cache_dir=None):
from modules import devices, modelloader
repo_id = 'stabilityai/stable-diffusion-3-medium-diffusers'
if pipe is None or not hasattr(pipe, module):
return pipe
if 'fp16' in te3.lower():
modelloader.hf_login()
t5 = transformers.T5EncoderModel.from_pretrained(
repo_id,
subfolder='text_encoder_3',
# torch_dtype=dtype,
cache_dir=cache_dir,
torch_dtype=pipe.text_encoder.dtype,
)
setattr(pipe, module, t5)
elif 'fp8' in te3.lower():
modelloader.hf_login()
from installer import install
install('bitsandbytes', quiet=True)
quantization_config = transformers.BitsAndBytesConfig(load_in_8bit=True)
t5 = transformers.T5EncoderModel.from_pretrained(
repo_id,
subfolder='text_encoder_3',
quantization_config=quantization_config,
cache_dir=cache_dir,
torch_dtype=pipe.text_encoder.dtype,
)
setattr(pipe, module, t5)
"""
if hasattr(pipe, 'remove_all_hooks'):
pipe.remove_all_hooks()
nn = getattr(pipe, module)
import accelerate
accelerate.hooks.remove_hook_from_module(nn, recurse=True)
nn.to(device=devices.device)
"""
else:
setattr(pipe, module, None)
if getattr(pipe, 'text_encoder_3', None) is not None and getattr(pipe, 'tokenizer_3', None) is None: # not needed anymore
pipe.tokenizer_3 = transformers.T5TokenizerFast.from_pretrained(
repo_id,
subfolder='tokenizer_3',
cache_dir=cache_dir,
)
devices.torch_gc()
modules/model_t5.py
+77
View File
@@ -0,0 +1,77 @@
import transformers
def load_t5(t5=None, cache_dir=None):
from modules import devices, modelloader
repo_id = 'stabilityai/stable-diffusion-3-medium-diffusers'
if 'fp16' in t5.lower():
modelloader.hf_login()
t5 = transformers.T5EncoderModel.from_pretrained(
repo_id,
subfolder='text_encoder_3',
# torch_dtype=dtype,
cache_dir=cache_dir,
torch_dtype=devices.dtype,
)
elif 'fp4' in t5.lower():
modelloader.hf_login()
from installer import install
install('bitsandbytes', quiet=True)
quantization_config = transformers.BitsAndBytesConfig(load_in_4bit=True)
t5 = transformers.T5EncoderModel.from_pretrained(
repo_id,
subfolder='text_encoder_3',
quantization_config=quantization_config,
cache_dir=cache_dir,
torch_dtype=devices.dtype,
)
elif 'fp8' in t5.lower():
modelloader.hf_login()
from installer import install
install('bitsandbytes', quiet=True)
quantization_config = transformers.BitsAndBytesConfig(load_in_8bit=True)
t5 = transformers.T5EncoderModel.from_pretrained(
repo_id,
subfolder='text_encoder_3',
quantization_config=quantization_config,
cache_dir=cache_dir,
torch_dtype=devices.dtype,
)
elif 'int8' in t5.lower():
modelloader.hf_login()
from installer import install
install('nncf==2.7.0', quiet=True)
from modules.sd_models_compile import nncf_compress_model
from modules.sd_hijack import NNCF_T5DenseGatedActDense # T5DenseGatedActDense uses fp32
t5 = transformers.T5EncoderModel.from_pretrained(
repo_id,
subfolder='text_encoder_3',
cache_dir=cache_dir,
torch_dtype=devices.dtype,
)
for i in range(len(t5.encoder.block)):
t5.encoder.block[i].layer[1].DenseReluDense = NNCF_T5DenseGatedActDense(
t5.encoder.block[i].layer[1].DenseReluDense
)
t5 = nncf_compress_model(t5)
else:
t5 = None
return t5
def set_t5(pipe, module, t5=None, cache_dir=None):
from modules import devices, shared
if pipe is None or not hasattr(pipe, module):
return pipe
t5 = load_t5(t5=t5, cache_dir=cache_dir)
setattr(pipe, module, t5)
if shared.cmd_opts.lowvram or shared.opts.diffusers_seq_cpu_offload:
from accelerate import cpu_offload
getattr(pipe, module).to("cpu")
cpu_offload(getattr(pipe, module), devices.device, offload_buffers=len(getattr(pipe, module)._parameters) > 0) # pylint: disable=protected-access
elif shared.cmd_opts.medvram or shared.opts.diffusers_model_cpu_offload:
if not hasattr(pipe, "_all_hooks") or len(pipe._all_hooks) == 0: # pylint: disable=protected-access
pipe.enable_model_cpu_offload(device=devices.device)
else:
pipe.maybe_free_model_hooks()
devices.torch_gc()
modules/modelloader.py
+4 -1
View File
@@ -204,7 +204,6 @@ def download_diffusers_model(hub_id: str, cache_dir: str = None, download_config
shared.log.debug(f'Diffusers downloading: id="{hub_id}" args={download_config}')
token = token or shared.opts.huggingface_token
if token is not None and len(token) > 2:
shared.log.debug(f"Diffusers authentication: {token}")
hf_login(token)
pipeline_dir = None
@@ -318,6 +317,10 @@ def get_reference_opts(name: str, quiet=False):
if k == name or model_name == name:
model_opts = v
break
model_name = model_name.replace('huggingface/', '')
if k == name or model_name == name:
model_opts = v
break
if not model_opts:
# shared.log.error(f'Reference: model="{name}" not found')
return {}
modules/pag/pipe_sdxl.py
+3 -3
View File
@@ -446,6 +446,7 @@ class StableDiffusionXLPAGPipeline(
feature_extractor: CLIPImageProcessor = None,
force_zeros_for_empty_prompt: bool = True,
add_watermarker: Optional[bool] = None,
requires_aesthetics_score: Optional[bool] = None, # todo: patch SDXLPAG pipeline
):
super().__init__()
@@ -461,12 +462,11 @@ class StableDiffusionXLPAGPipeline(
feature_extractor=feature_extractor,
)
self.register_to_config(force_zeros_for_empty_prompt=force_zeros_for_empty_prompt)
self.register_to_config(requires_aesthetics_score=requires_aesthetics_score)
self.vae_scale_factor = 2 ** (len(self.vae.config.block_out_channels) - 1)
self.image_processor = VaeImageProcessor(vae_scale_factor=self.vae_scale_factor)
self.default_sample_size = self.unet.config.sample_size
add_watermarker = add_watermarker if add_watermarker is not None else is_invisible_watermark_available()
add_watermarker = False
if add_watermarker:
self.watermark = StableDiffusionXLWatermarker()
modules/processing_diffusers.py
-1
View File
@@ -105,7 +105,6 @@ def process_diffusers(p: processing.StableDiffusionProcessing):
desc='Base',
)
shared.state.sampling_steps = base_args.get('prior_num_inference_steps', None) or base_args.get('num_inference_steps', None) or p.steps
p.extra_generation_params['Pipeline'] = shared.sd_model.__class__.__name__
if shared.opts.scheduler_eta is not None and shared.opts.scheduler_eta > 0 and shared.opts.scheduler_eta < 1:
p.extra_generation_params["Sampler Eta"] = shared.opts.scheduler_eta
output = None
modules/processing_info.py
+16 -4
View File
@@ -63,6 +63,10 @@ def create_infotext(p: StableDiffusionProcessing, all_prompts=None, all_seeds=No
"Comment": comment,
"Operations": '; '.join(ops).replace('"', '') if len(p.ops) > 0 else 'none',
}
# native
if shared.native:
args['Pipeline'] = shared.sd_model.__class__.__name__
args['T5'] = None if (not shared.opts.add_model_name_to_info or shared.opts.sd_text_encoder is None or shared.opts.sd_text_encoder == 'None') else shared.opts.sd_text_encoder
if 'txt2img' in p.ops:
args["Variation seed"] = all_subseeds[index] if p.subseed_strength > 0 else None
args["Variation strength"] = p.subseed_strength if p.subseed_strength > 0 else None
@@ -143,12 +147,20 @@ def create_infotext(p: StableDiffusionProcessing, all_prompts=None, all_seeds=No
args['Sampler sigma uncond'] = shared.opts.s_churn if shared.opts.s_churn != shared.opts.data_labels.get('s_churn').default else None
args['Sampler sigma noise'] = shared.opts.s_noise if shared.opts.s_noise != shared.opts.data_labels.get('s_noise').default else None
args['Sampler sigma tmin'] = shared.opts.s_tmin if shared.opts.s_tmin != shared.opts.data_labels.get('s_tmin').default else None
# tome
args['ToMe'] = shared.opts.tome_ratio if shared.opts.tome_ratio != 0 else None
args['ToDo'] = shared.opts.todo_ratio if shared.opts.todo_ratio != 0 else None
# tome/todo
if shared.opts.token_merging_method == 'ToMe':
args['ToMe'] = shared.opts.tome_ratio if shared.opts.tome_ratio != 0 else None
else:
args['ToDo'] = shared.opts.todo_ratio if shared.opts.todo_ratio != 0 else None
args.update(p.extra_generation_params)
params_text = ", ".join([k if k == v else f'{k}: {generation_parameters_copypaste.quote(v)}' for k, v in args.items() if v is not None])
for k, v in args.copy().items():
if v is None:
del args[k]
if isinstance(v, str):
if len(v) == 0 or v == '0x0':
del args[k]
params_text = ", ".join([k if k == v else f'{k}: {generation_parameters_copypaste.quote(v)}' for k, v in args.items()])
negative_prompt_text = f"\nNegative prompt: {all_negative_prompts[index]}" if all_negative_prompts[index] else ""
infotext = f"{all_prompts[index]}{negative_prompt_text}\n{params_text}".strip()
return infotext
modules/prompt_parser_diffusers.py
+1 -1
View File
@@ -144,7 +144,7 @@ def get_tokens(msg, prompt):
except Exception:
tokens.append(f'UNK_{i}')
token_count = len(ids) - int(has_bos_token) - int(has_eos_token)
shared.log.trace(f'Prompt tokenizer: type={msg} tokens={token_count} {tokens}')
debug(f'Prompt tokenizer: type={msg} tokens={token_count} {tokens}')
def encode_prompts(pipe, p, prompts: list, negative_prompts: list, steps: int, clip_skip: typing.Optional[int] = None):
modules/sd_models.py
+38 -55
View File
@@ -202,11 +202,17 @@ def get_closet_checkpoint_match(search_string):
if checkpoint_info is not None:
return checkpoint_info
found = sorted([info for info in checkpoints_list.values() if search_string in info.title], key=lambda x: len(x.title))
if found:
if found and len(found) > 0:
return found[0]
found = sorted([info for info in checkpoints_list.values() if search_string.split(' ')[0] in info.title], key=lambda x: len(x.title))
if found:
if found and len(found) > 0:
return found[0]
for v in shared.reference_models.values():
if search_string in v['path'] or os.path.basename(search_string) in v['path']:
model_name = search_string.replace('huggingface/', '')
checkpoint_info = CheckpointInfo(v['path']) # create a virutal model info
checkpoint_info.type = 'huggingface'
return checkpoint_info
return None
@@ -565,34 +571,20 @@ def detect_pipeline(f: str, op: str = 'model', warning=True, quiet=False):
# elif size < 0: # unknown
# guess = 'Stable Diffusion 2B'
elif size >= 5791 and size <= 5799: # 5795
if not shared.native:
warn(f'Model detected as SD-XL refiner model, but attempting to load using backend=original: {op}={f} size={size} MB')
if op == 'model':
warn(f'Model detected as SD-XL refiner model, but attempting to load a base model: {op}={f} size={size} MB')
guess = 'Stable Diffusion XL Refiner'
elif (size >= 6611 and size <= 7220): # 6617, HassakuXL is 6776, monkrenRealisticINT_v10 is 7217
if not shared.native:
warn(f'Model detected as SD-XL base model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Stable Diffusion XL'
elif size >= 3361 and size <= 3369: # 3368
if not shared.native:
warn(f'Model detected as SD upscale model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Stable Diffusion Upscale'
elif size >= 4891 and size <= 4899: # 4897
if not shared.native:
warn(f'Model detected as SD XL inpaint model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Stable Diffusion XL Inpaint'
elif size >= 9791 and size <= 9799: # 9794
if not shared.native:
warn(f'Model detected as SD XL instruct pix2pix model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Stable Diffusion XL Instruct'
elif size > 3138 and size < 3142: #3140
if not shared.native:
warn(f'Model detected as Segmind Vega model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Stable Diffusion XL'
elif size > 5692 and size < 5698 or size > 4134 and size < 4138:
if not shared.native:
warn(f'Model detected as Stable Diffusion 3 model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Stable Diffusion 3'
# guess by name
"""
@@ -602,34 +594,20 @@ def detect_pipeline(f: str, op: str = 'model', warning=True, quiet=False):
guess = 'Latent Consistency Model'
"""
if 'instaflow' in f.lower():
if not shared.native:
warn(f'Model detected as InstaFlow model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'InstaFlow'
if 'segmoe' in f.lower():
if not shared.native:
warn(f'Model detected as SegMoE model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'SegMoE'
if 'hunyuandit' in f.lower():
if not shared.native:
warn(f'Model detected as Tenecent HunyuanDiT model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'HunyuanDiT'
if 'pixart-xl' in f.lower():
if not shared.native:
warn(f'Model detected as PixArt Alpha model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'PixArt-Alpha'
if 'stable-diffusion-3' in f.lower():
if not shared.native:
warn(f'Model detected as Stable Diffusion 3 model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'Stable Diffusion 3'
if 'stable-cascade' in f.lower() or 'stablecascade' in f.lower() or 'wuerstchen3' in f.lower():
if not shared.native:
warn(f'Model detected as Stable Cascade model, but attempting to load using backend=original: {op}={f} size={size} MB')
if devices.dtype == torch.float16:
warn('Stable Cascade does not support Float16')
guess = 'Stable Cascade'
if 'pixart-sigma' in f.lower():
if not shared.native:
warn(f'Model detected as PixArt-Sigma model, but attempting to load using backend=original: {op}={f} size={size} MB')
guess = 'PixArt-Sigma'
# switch for specific variant
if guess == 'Stable Diffusion' and 'inpaint' in f.lower():
@@ -675,15 +653,10 @@ def copy_diffuser_options(new_pipe, orig_pipe):
new_pipe.is_sd1 = getattr(orig_pipe, 'is_sd1', True)
def set_diffuser_options(sd_model, vae = None, op: str = 'model'):
def set_diffuser_options(sd_model, vae = None, op: str = 'model', offload=True):
if sd_model is None:
shared.log.warning(f'{op} is not loaded')
return
if (shared.opts.diffusers_model_cpu_offload or shared.cmd_opts.medvram) and (shared.opts.diffusers_seq_cpu_offload or shared.cmd_opts.lowvram):
shared.log.warning(f'Setting {op}: Model CPU offload and Sequential CPU offload are not compatible')
shared.log.debug(f'Setting {op}: disabling model CPU offload')
shared.opts.diffusers_model_cpu_offload=False
shared.cmd_opts.medvram=False
if hasattr(sd_model, "watermark"):
sd_model.watermark = NoWatermark()
@@ -739,6 +712,20 @@ def set_diffuser_options(sd_model, vae = None, op: str = 'model'):
shared.log.debug(f'Setting {op}: enable channels last')
sd_model.unet.to(memory_format=torch.channels_last)
if offload:
set_diffuser_offload(sd_model, op)
def set_diffuser_offload(sd_model, op: str = 'model'):
if sd_model is None:
shared.log.warning(f'{op} is not loaded')
return
if (shared.opts.diffusers_model_cpu_offload or shared.cmd_opts.medvram) and (shared.opts.diffusers_seq_cpu_offload or shared.cmd_opts.lowvram):
shared.log.warning(f'Setting {op}: Model CPU offload and Sequential CPU offload are not compatible')
shared.log.debug(f'Setting {op}: disabling model CPU offload')
shared.opts.diffusers_model_cpu_offload=False
shared.cmd_opts.medvram=False
if not (hasattr(sd_model, "has_accelerate") and sd_model.has_accelerate):
sd_model.has_accelerate = False
if hasattr(sd_model, "enable_model_cpu_offload"):
if shared.cmd_opts.medvram or shared.opts.diffusers_model_cpu_offload:
shared.log.debug(f'Setting {op}: enable model CPU offload')
@@ -996,14 +983,8 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No
return
elif model_type in ['PixArt-Sigma']: # forced pipeline
try:
# shared.opts.data['cuda_dtype'] = 'FP32' # override
# shared.opts.data['diffusers_model_cpu_offload'] = True # override
devices.set_cuda_params()
sd_model = diffusers.PixArtSigmaPipeline.from_pretrained(
checkpoint_info.path,
use_safetensors=True,
cache_dir=shared.opts.diffusers_dir,
**diffusers_load_config)
from modules.model_pixart import load_pixart
sd_model = load_pixart(checkpoint_info, diffusers_load_config)
except Exception as e:
shared.log.error(f'Diffusers Failed loading {op}: {checkpoint_info.path} {e}')
if debug_load:
@@ -1161,7 +1142,12 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No
from modules.prompt_parser_diffusers import insert_parser_highjack
insert_parser_highjack(sd_model.__class__.__name__)
set_diffuser_options(sd_model, vae, op)
set_diffuser_options(sd_model, vae, op, offload=False)
if shared.opts.nncf_compress_weights and not (shared.opts.cuda_compile and shared.opts.cuda_compile_backend == "openvino_fx"):
sd_model = sd_models_compile.nncf_compress_weights(sd_model) # run this before move model so it can be compressed in CPU
timer.record("options")
set_diffuser_offload(sd_model, op)
if op == 'model':
sd_vae.apply_vae_config(shared.sd_model.sd_checkpoint_info.filename, vae_file, sd_model)
if op == 'refiner' and shared.opts.diffusers_move_refiner:
@@ -1176,9 +1162,6 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No
if shared.opts.ipex_optimize:
sd_model = sd_models_compile.ipex_optimize(sd_model)
if shared.opts.nncf_compress_weights and not (shared.opts.cuda_compile and shared.opts.cuda_compile_backend == "openvino_fx"):
sd_model = sd_models_compile.nncf_compress_weights(sd_model)
if (shared.opts.cuda_compile and shared.opts.cuda_compile_backend != 'none'):
sd_model = sd_models_compile.compile_diffusers(sd_model)
timer.record("compile")
@@ -1531,18 +1514,18 @@ def load_model(checkpoint_info=None, already_loaded_state_dict=None, timer=None,
def reload_text_encoder(initial=False):
if initial and (shared.opts.sd_te3 is None or shared.opts.sd_te3 == 'None'):
if initial and (shared.opts.sd_text_encoder is None or shared.opts.sd_text_encoder == 'None'):
return # dont unload
signature = inspect.signature(shared.sd_model.__class__.__init__, follow_wrapped=True, eval_str=True).parameters
t5 = [k for k, v in signature.items() if 'T5EncoderModel' in str(v)]
if len(t5) > 0:
from modules.model_sd3 import load_t5
shared.log.debug(f'Load: t5={shared.opts.sd_te3} module="{t5[0]}"')
load_t5(pipe=shared.sd_model, module=t5[0], te3=shared.opts.sd_te3, cache_dir=shared.opts.diffusers_dir)
from modules.model_t5 import set_t5
shared.log.debug(f'Load: t5={shared.opts.sd_text_encoder} module="{t5[0]}"')
set_t5(pipe=shared.sd_model, module=t5[0], t5=shared.opts.sd_text_encoder, cache_dir=shared.opts.diffusers_dir)
elif hasattr(shared.sd_model, 'text_encoder_3'):
from modules.model_sd3 import load_t5
shared.log.debug(f'Load: t5={shared.opts.sd_te3} module="text_encoder_3"')
load_t5(pipe=shared.sd_model, module='text_encoder_3', te3=shared.opts.sd_te3, cache_dir=shared.opts.diffusers_dir)
from modules.model_t5 import set_t5
shared.log.debug(f'Load: t5={shared.opts.sd_text_encoder} module="text_encoder_3"')
set_t5(pipe=shared.sd_model, module='text_encoder_3', t5=shared.opts.sd_text_encoder, cache_dir=shared.opts.diffusers_dir)
def reload_model_weights(sd_model=None, info=None, reuse_dict=False, op='model', force=False):
modules/sd_models_compile.py
+21 -16
View File
@@ -114,27 +114,32 @@ def ipex_optimize(sd_model):
shared.log.warning(f"IPEX Optimize: error: {e}")
return sd_model
def nncf_send_to_device(model):
for child in model.children():
if child.__class__.__name__ == "WeightsDecompressor":
child.scale = child.scale.to(devices.device)
child.zero_point = child.zero_point.to(devices.device)
nncf_send_to_device(child)
def nncf_compress_model(model):
import nncf
model.eval()
backup_embeddings = None
if hasattr(model, "get_input_embeddings"):
backup_embeddings = copy.deepcopy(model.get_input_embeddings())
model = nncf.compress_weights(model)
nncf_send_to_device(model)
if hasattr(model, "set_input_embeddings") and backup_embeddings is not None:
model.set_input_embeddings(backup_embeddings)
devices.torch_gc(force=True)
return model
def nncf_compress_weights(sd_model):
try:
t0 = time.time()
if sd_model.device.type == "meta":
shared.log.warning("Compress Weights is not compatible with Sequential CPU offload")
return sd_model
from installer import install
install('nncf==2.7.0', quiet=True)
def nncf_compress_model(model):
return_device = model.device
model.eval()
backup_embeddings = None
if hasattr(model, "get_input_embeddings"):
backup_embeddings = copy.deepcopy(model.get_input_embeddings())
model = nncf.compress_weights(model.to(devices.device)).to(return_device)
if hasattr(model, "set_input_embeddings") and backup_embeddings is not None:
model.set_input_embeddings(backup_embeddings)
devices.torch_gc(force=True)
return model
import nncf
shared.compiled_model_state = CompiledModelState()
shared.compiled_model_state.is_compiled = True
modules/shared.py
+4 -4
View File
@@ -391,7 +391,7 @@ options_templates.update(options_section(('sd', "Execution & Models"), {
"sd_model_refiner": OptionInfo('None', "Refiner model", gr.Dropdown, lambda: {"choices": ['None'] + list_checkpoint_tiles()}, refresh=refresh_checkpoints),
"sd_vae": OptionInfo("Automatic", "VAE model", gr.Dropdown, lambda: {"choices": shared_items.sd_vae_items()}, refresh=shared_items.refresh_vae_list),
"sd_unet": OptionInfo("None", "UNET model", gr.Dropdown, lambda: {"choices": shared_items.sd_unet_items()}, refresh=shared_items.refresh_unet_list),
"sd_te3": OptionInfo('None', "Text encoder model", gr.Dropdown, lambda: {"choices": ['None', 'T5 FP8', 'T5 FP16']}),
"sd_text_encoder": OptionInfo('None', "Text encoder model", gr.Dropdown, lambda: {"choices": ['None', 'T5 FP4', 'T5 FP8', 'T5 INT8', 'T5 FP16']}),
"sd_checkpoint_autoload": OptionInfo(True, "Model autoload on start"),
"sd_model_dict": OptionInfo('None', "Use separate base dict", gr.Dropdown, lambda: {"choices": ['None'] + list_checkpoint_tiles()}, refresh=refresh_checkpoints),
"stream_load": OptionInfo(False, "Load models using stream loading method", gr.Checkbox, {"visible": not native }),
@@ -449,7 +449,7 @@ options_templates.update(options_section(('cuda', "Compute Settings"), {
"deep_cache_interval": OptionInfo(3, "DeepCache cache interval", gr.Slider, {"minimum": 1, "maximum": 10, "step": 1}),
"nncf_sep": OptionInfo("<h2>Model Compress</h2>", "", gr.HTML),
"nncf_compress_weights": OptionInfo([], "Compress Model weights with NNCF", gr.CheckboxGroup, {"choices": ["Model", "VAE", "Text Encoder"], "visible": native}),
"nncf_compress_weights": OptionInfo([], "Compress Model weights with NNCF", gr.CheckboxGroup, {"choices": ["Model", "VAE", "Text Encoder", "ControlNet"], "visible": native}),
"ipex_sep": OptionInfo("<h2>IPEX</h2>", "", gr.HTML, {"visible": devices.backend == "ipex"}),
"ipex_optimize": OptionInfo([], "IPEX Optimize for Intel GPUs", gr.CheckboxGroup, {"choices": ["Model", "VAE", "Text Encoder", "Upscaler"], "visible": devices.backend == "ipex"}),
@@ -806,9 +806,9 @@ options_templates.update(options_section(('interrogate', "Interrogate"), {
"deepbooru_filter_tags": OptionInfo("", "Filter out tags from deepbooru output"),
}))
options_templates.update(options_section(('extra_networks', "Extra Networks"), {
options_templates.update(options_section(('extra_networks', "Networks"), {
"extra_networks_sep1": OptionInfo("<h2>Extra networks UI</h2>", "", gr.HTML),
"extra_networks": OptionInfo(["All"], "Extra networks", gr.Dropdown, lambda: {"multiselect":True, "choices": ['All'] + [en.title for en in extra_networks]}),
"extra_networks": OptionInfo(["All"], "Networks", gr.Dropdown, lambda: {"multiselect":True, "choices": ['All'] + [en.title for en in extra_networks]}),
"extra_networks_sort": OptionInfo("Default", "Sort order", gr.Dropdown, {"choices": ['Default', 'Name [A-Z]', 'Name [Z-A]', 'Date [Newest]', 'Date [Oldest]', 'Size [Largest]', 'Size [Smallest]']}),
"extra_networks_view": OptionInfo("gallery", "UI view", gr.Radio, {"choices": ["gallery", "list"]}),
"extra_networks_card_cover": OptionInfo("sidebar", "UI position", gr.Radio, {"choices": ["cover", "inline", "sidebar"]}),
modules/styles.py
+1 -1
View File
@@ -328,7 +328,7 @@ class StyleDatabase:
"preview": "",
}
keepcharacters = (' ','.','_')
fn = "".join(c for c in name if c.isalnum() or c in keepcharacters).rstrip()
fn = "".join(c for c in name if c.isalnum() or c in keepcharacters).strip()
fn = os.path.join(path, fn + ".json")
try:
with open(fn, 'w', encoding='utf-8') as f:
modules/ui_extra_networks.py
+11 -10
View File
@@ -71,7 +71,7 @@ def init_api(app):
metadata = page.metadata.get(item, 'none')
if metadata is None:
metadata = ''
# shared.log.debug(f"Extra networks metadata: page='{page}' item={item} len={len(metadata)}")
# shared.log.debug(f"Networks metadata: page='{page}' item={item} len={len(metadata)}")
return JSONResponse({"metadata": metadata})
def get_info(page: str = "", item: str = ""):
@@ -84,7 +84,7 @@ def init_api(app):
info = page.find_info(item['filename'])
if info is None:
info = {}
# shared.log.debug(f"Extra networks info: page='{page.name}' item={item['name']} len={len(info)}")
# shared.log.debug(f"Networks info: page='{page.name}' item={item['name']} len={len(info)}")
return JSONResponse({"info": info})
def get_desc(page: str = "", item: str = ""):
@@ -97,7 +97,7 @@ def init_api(app):
desc = page.find_description(item['filename'])
if desc is None:
desc = ''
# shared.log.debug(f"Extra networks desc: page='{page.name}' item={item['name']} len={len(desc)}")
# shared.log.debug(f"Networks desc: page='{page.name}' item={item['name']} len={len(desc)}")
return JSONResponse({"description": desc})
app.add_api_route("/sd_extra_networks/thumb", fetch_file, methods=["GET"])
@@ -186,7 +186,7 @@ class ExtraNetworksPage:
except Exception as e:
shared.log.warning(f'Extra network error creating thumbnail: {f} {e}')
if created > 0:
shared.log.info(f"Extra network thumbnails: {self.name} created={created}")
shared.log.info(f"Network thumbnails: {self.name} created={created}")
self.missing_thumbs.clear()
def create_items(self, tabname):
@@ -235,7 +235,7 @@ class ExtraNetworksPage:
continue
# if not self.is_empty(tgt):
subdirs[subdir] = 1
debug(f"Extra networks: page='{self.name}' subfolders={list(subdirs)}")
debug(f"Networks: page='{self.name}' subfolders={list(subdirs)}")
subdirs = OrderedDict(sorted(subdirs.items()))
if self.name == 'model':
subdirs['Reference'] = 1
@@ -272,7 +272,7 @@ class ExtraNetworksPage:
self.html += ''.join(htmls)
self.page_time = time.time()
self.html = f"<div id='~tabname_{self_name_id}_subdirs' class='extra-network-subdirs'>{subdirs_html}</div><div id='~tabname_{self_name_id}_cards' class='extra-network-cards'>{self.html}</div>"
shared.log.debug(f"Extra networks: page='{self.name}' items={len(self.items)} subfolders={len(subdirs)} tab={tabname} folders={self.allowed_directories_for_previews()} list={self.list_time:.2f} thumb={self.preview_time:.2f} desc={self.desc_time:.2f} info={self.info_time:.2f} workers={shared.max_workers} sort={shared.opts.extra_networks_sort}")
shared.log.debug(f"Networks: page='{self.name}' items={len(self.items)} subfolders={len(subdirs)} tab={tabname} folders={self.allowed_directories_for_previews()} list={self.list_time:.2f} thumb={self.preview_time:.2f} desc={self.desc_time:.2f} info={self.info_time:.2f} workers={shared.max_workers} sort={shared.opts.extra_networks_sort}")
if len(self.missing_thumbs) > 0:
threading.Thread(target=self.create_thumb).start()
return self.patch(self.html, tabname)
@@ -570,7 +570,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
with gr.Group(elem_id=f"{tabname}_extra_details_tabs", visible=False) as ui.details_tabs:
with gr.Tabs():
with gr.Tab('Description', elem_classes=['extra-details-tabs']):
desc = gr.Textbox('', show_label=False, lines=8, placeholder="Extra network description...")
desc = gr.Textbox('', show_label=False, lines=8, placeholder="Network description...")
ui.details_components.append(desc)
with gr.Row():
btn_save_desc = gr.Button('Save', elem_classes=['small-button'], elem_id=f'{tabname}_extra_details_save_desc')
@@ -895,7 +895,8 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
return res
def ui_quicksave_click(name):
if name is None:
if name is None or len(name) < 1:
shared.log.warning("Network quick save style: no name provided")
return
fn = os.path.join(paths.data_path, "params.txt")
if os.path.exists(fn):
@@ -915,9 +916,9 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
}
shared.writefile(item, fn, silent=True)
if len(prompt) > 0:
shared.log.debug(f"Extra network quick save style: item={name} filename='{fn}'")
shared.log.debug(f"Network quick save style: item={name} filename='{fn}'")
else:
shared.log.warning(f"Extra network quick save model: item={name} filename='{fn}' prompt is empty")
shared.log.warning(f"Network quick save model: item={name} filename='{fn}' prompt is empty")
def ui_sort_cards(sort_order):
if shared.opts.extra_networks_sort != sort_order:
modules/ui_extra_networks_checkpoints.py
+1 -1
View File
@@ -64,7 +64,7 @@ class ExtraNetworksPageCheckpoints(ui_extra_networks.ExtraNetworksPage):
record["info"] = self.find_info(checkpoint.filename)
record["description"] = self.find_description(checkpoint.filename, record["info"])
except Exception as e:
shared.log.debug(f"Extra networks error: type=model file={name} {e}")
shared.log.debug(f"Networks error: type=model file={name} {e}")
return record
def list_items(self):
modules/ui_extra_networks_hypernets.py
+1 -1
View File
@@ -27,7 +27,7 @@ class ExtraNetworksPageHypernetworks(ui_extra_networks.ExtraNetworksPage):
"size": os.path.getsize(path),
}
except Exception as e:
shared.log.debug(f"Extra networks error: type=hypernetwork file={path} {e}")
shared.log.debug(f"Networks error: type=hypernetwork file={path} {e}")
def allowed_directories_for_previews(self):
return [shared.opts.hypernetwork_dir]
modules/ui_extra_networks_styles.py
+2 -1
View File
@@ -93,11 +93,12 @@ class ExtraNetworksPageStyles(ui_extra_networks.ExtraNetworksPage):
"size": os.path.getsize(style.filename),
}
except Exception as e:
shared.log.debug(f"Extra networks error: type=style file={k} {e}")
shared.log.debug(f"Networks error: type=style file={k} {e}")
return item
def list_items(self):
items = [self.create_item(k) for k in list(shared.prompt_styles.styles)]
items = [item for item in items if item is not None]
self.update_all_previews(items)
return items
modules/ui_extra_networks_textual_inversion.py
+1 -1
View File
@@ -37,7 +37,7 @@ class ExtraNetworksPageTextualInversion(ui_extra_networks.ExtraNetworksPage):
record["info"] = self.find_info(embedding.filename)
record["description"] = self.find_description(embedding.filename, record["info"])
except Exception as e:
shared.log.debug(f"Extra networks error: type=embedding file={embedding.filename} {e}")
shared.log.debug(f"Networks error: type=embedding file={embedding.filename} {e}")
return record
def list_items(self):
modules/ui_extra_networks_vae.py
+1 -1
View File
@@ -31,7 +31,7 @@ class ExtraNetworksPageVAEs(ui_extra_networks.ExtraNetworksPage):
record["description"] = self.find_description(filename, record["info"])
yield record
except Exception as e:
shared.log.debug(f"Extra networks error: type=vae file={filename} {e}")
shared.log.debug(f"Networks error: type=vae file={filename} {e}")
def allowed_directories_for_previews(self):
return [v for v in [shared.opts.vae_dir] if v is not None]
modules/ui_prompt_styles.py
-57
View File
@@ -46,60 +46,3 @@ def refresh_styles():
class UiPromptStyles:
def __init__(self, tabname, main_ui_prompt, main_ui_negative_prompt): # pylint: disable=unused-argument
self.dropdown = gr.Dropdown(label="Styles", elem_id=f"{tabname}_styles", choices=[style.name for style in shared.prompt_styles.styles.values()], value=[], multiselect=True)
"""
def __init__(self, tabname, main_ui_prompt, main_ui_negative_prompt):
self.tabname = tabname
with gr.Row(elem_id=f"{tabname}_styles_row"):
self.dropdown = gr.Dropdown(label="Styles", show_label=False, elem_id=f"{tabname}_styles", choices=list(shared.prompt_styles.styles), value=[], multiselect=True, tooltip="Styles")
edit_button = ui_components.ToolButton(value=styles_edit_symbol, elem_id=f"{tabname}_styles_edit_button", tooltip="Edit styles")
with gr.Box(elem_id=f"{tabname}_styles_dialog", elem_classes="popup-dialog") as styles_dialog:
with gr.Row():
self.selection = gr.Dropdown(label="Styles", elem_id=f"{tabname}_styles_edit_select", choices=list(shared.prompt_styles.styles), value=[], allow_custom_value=True, info="Styles allow you to add custom text to prompt. Use the {prompt} token in style text, and it will be replaced with user's prompt when applying style. Otherwise, style's text will be added to the end of the prompt.")
ui_common.create_refresh_button([self.dropdown, self.selection], shared.prompt_styles.reload, lambda: {"choices": list(shared.prompt_styles.styles)}, f"refresh_{tabname}_styles")
self.materialize = ui_components.ToolButton(value=styles_materialize_symbol, elem_id=f"{tabname}_style_apply", tooltip="Apply all selected styles from the style selction dropdown in main UI to the prompt.")
with gr.Row():
self.prompt = gr.Textbox(label="Prompt", show_label=True, elem_id=f"{tabname}_edit_style_prompt", lines=3)
with gr.Row():
self.neg_prompt = gr.Textbox(label="Negative prompt", show_label=True, elem_id=f"{tabname}_edit_style_neg_prompt", lines=3)
with gr.Row():
self.save = gr.Button('Save', variant='primary', elem_id=f'{tabname}_edit_style_save', visible=False)
self.delete = gr.Button('Delete', variant='primary', elem_id=f'{tabname}_edit_style_delete', visible=False)
self.close = gr.Button('Close', variant='secondary', elem_id=f'{tabname}_edit_style_close')
self.selection.change(
fn=select_style,
inputs=[self.selection],
outputs=[self.prompt, self.neg_prompt, self.delete, self.save],
show_progress=False,
)
self.save.click(
fn=save_style,
inputs=[self.selection, self.prompt, self.neg_prompt],
outputs=[self.delete],
show_progress=False,
).then(refresh_styles, outputs=[self.dropdown, self.selection], show_progress=False)
self.delete.click(
fn=delete_style,
_js='function(name){ if(name == "") return ""; return confirm("Delete style " + name + "?") ? name : ""; }',
inputs=[self.selection],
outputs=[self.selection, self.prompt, self.neg_prompt],
show_progress=False,
).then(refresh_styles, outputs=[self.dropdown, self.selection], show_progress=False)
self.materialize.click(
fn=materialize_styles,
inputs=[main_ui_prompt, main_ui_negative_prompt, self.dropdown],
outputs=[main_ui_prompt, main_ui_negative_prompt, self.dropdown],
show_progress=False,
).then(fn=None, _js="function(){update_"+tabname+"_tokens(); closePopup();}", show_progress=False)
ui_common.setup_dialog(button_show=edit_button, dialog=styles_dialog, button_close=self.close)
"""
scripts/xyz_grid.py
+6
View File
@@ -138,6 +138,11 @@ def apply_vae(p, x, xs):
sd_vae.reload_vae_weights(shared.sd_model, vae_file=find_vae(x))
def apply_te(p, x, xs):
shared.opts.data["sd_text_encoder"] = x
sd_models.reload_text_encoder()
def apply_styles(p: processing.StableDiffusionProcessingTxt2Img, x: str, _):
p.styles.extend(x.split(','))
@@ -230,6 +235,7 @@ axis_options = [
AxisOption("Prompt S/R", str, apply_prompt, fmt=format_value),
AxisOption("Model", str, apply_checkpoint, fmt=format_value, cost=1.0, choices=lambda: sorted(sd_models.checkpoints_list)),
AxisOption("VAE", str, apply_vae, cost=0.7, choices=lambda: ['None'] + list(sd_vae.vae_dict)),
AxisOption("Text encoder", str, apply_te, cost=0.7, choices=lambda: ['None', 'T5 FP4', 'T5 FP8', 'T5 FP16']),
AxisOption("Styles", str, apply_styles, choices=lambda: [s.name for s in shared.prompt_styles.styles.values()]),
AxisOption("Seed", int, apply_field("seed")),
AxisOption("Steps", int, apply_field("steps")),
webui.py
+1 -1
View File
@@ -168,7 +168,7 @@ def load_model():
thread_refiner.join()
shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: modules.sd_models.reload_model_weights(op='model')), call=False)
shared.opts.onchange("sd_model_refiner", wrap_queued_call(lambda: modules.sd_models.reload_model_weights(op='refiner')), call=False)
shared.opts.onchange("sd_te3", wrap_queued_call(lambda: modules.sd_models.reload_text_encoder()), call=False)
shared.opts.onchange("sd_text_encoder", wrap_queued_call(lambda: modules.sd_models.reload_text_encoder()), call=False)
shared.opts.onchange("sd_model_dict", wrap_queued_call(lambda: modules.sd_models.reload_model_weights(op='dict')), call=False)
shared.opts.onchange("sd_vae", wrap_queued_call(lambda: modules.sd_vae.reload_vae_weights()), call=False)
shared.opts.onchange("sd_backend", wrap_queued_call(lambda: modules.sd_models.change_backend()), call=False)
wiki
+1 -1
Submodule wiki updated: 4e01da914a...c5c9e89981