diff --git a/CHANGELOG.md b/CHANGELOG.md index 1e16f894a..57d977666 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,22 +1,34 @@ # Change Log for SD.Next -## Update for 07/17/2023 +## Update for 07/18/2023 +While we're waiting for official SD-XL release, here's another update with some fixes and enhancements... + +- **global** + - image save: option to add invisible image watermark to all your generated images + disabled by default, can be enabled in settings -> image options + watermark information will be shown when loading image such as in process image tab + also additional cli utility `/cli/image-watermark.py` to read/write/strip watermarks from images + - batch processing: fix metadata saving, also allow to drag&drop images for batch processing + - ui configuration: you can modify all ui default values from settings as usual, + but only values that are non-default will be written to `ui-config.json` + - startup: add cmd flag to skip all `torch` checks + - startup: force requirements check on each server start + there are too many misbehaving extensions that change system requirements + - internal: safe handling of all config file read/write operations + this allows sdnext to run in fully shared environments and prevents any possible configuration corruptions - **diffusers**: + - sd-xl: remove image watermarks autocreated by 0.9 model - vae: enable loading of external vae, documented in diffusers wiki and mix&match continues, you can even use sd-xl vae with sd 1.5 models! - samplers: add concept of *default* sampler to avoid needing to tweak settings for primary or second pass note that sampler details will be printed in log when running in debug level - - samplers: allow overriding of sampler beta values in settings - - refiner: fix refiner applying only to first image in batch - - refiner: allow using direct latents or processed output in refiner + - samplers: allow overriding of sampler beta values in settings + - refiner: fix refiner applying only to first image in batch + - refiner: allow using direct latents or processed output in refiner - model: basic support for one more model: [UniDiffuser](https://github.com/thu-ml/unidiffuser) download using model downloader: `thu-ml/unidiffuser-v1` - and set resolution to 512x512 -- **other** - - add cmd flag to skip all torch checks - - force requirements check on each start - there are too many misbehaving extensions that change system requirements + and set resolution to 512x512 ## Update for 07/14/2023 diff --git a/README.md b/README.md index faea08e25..92a2a6969 100644 --- a/README.md +++ b/README.md @@ -15,16 +15,21 @@ -This project started as a fork from [Automatic1111 WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui/) and it grew significantly since then, but although it diverged considerably, any substantial features to original work is ported to this repository as well. +This project started as a fork from [Automatic1111 WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui/) and it grew significantly since then, +but although it diverged considerably, any substantial features to original work is ported to this repository as well. ## Top-10 Differentiators All Individual features are not listed here, instead check [Changelog](CHANGELOG.md) for full list of changes. -- Optimized processingwith latest **torch** developments +- Optimized processing with latest **torch** developments Including built-in support for `torch.compile` -- Support for multiple backends: `diffusers` as well as standard `ldm` backend +- Support for multiple backends! + **original** and **diffusers** +- Support for multiple diffusion models! + Stable Diffusion, SD-XL, Kandinsky, DeepFloyd IF, etc. - Fully multiplatform with platform specific autodetection and tuning performed on install + Windows / Linux / MacOS with CPU / nVidia / AMD / IntelArc / DirectML - Improved prompt parser - Enhanced *Lora*/*Locon*/*Lyco* code supporting latest trends in training - Built-in queue management @@ -35,6 +40,17 @@ All Individual features are not listed here, instead check [Changelog](CHANGELOG - Built in installer with automatic updates and dependency management - Modernized UI (still based on Gradio) with theme support +## Backend support + +**SD.Next** supports two main backends: *Original* and *Diffusers* which can be switched on-the-fly: + +- **Original**: Based on [LDM](https://github.com/Stability-AI/stablediffusion) reference implementation and significantly expanded on by [A1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui) + This is the default backend and it is fully compatible with all existing functionality and extensions +- **Diffusers**: Based on new [Huggingface Diffusers](https://huggingface.co/docs/diffusers/index) implementation + It is also the only backend that supports **Stable Diffusion XL** model + Support for legacy workflows and extensions is limited, but its being expanded + See [wiki article](https://github.com/vladmandic/automatic/wiki/Diffusers) for more information + ## Model support Additional models will be added as they become available and there is public interest in them @@ -64,7 +80,7 @@ Additional models will be added as they become available and there is public int 3. Run launcher `webui.bat` or `webui.sh`: - Platform specific wrapper scripts For Windows, Linux and OSX - - Starts `sdnext.py` in a Python virtual environment (`venv`) + - Starts `launch.py` in a Python virtual environment (`venv`) - Uses `install.py` to handle all actual requirements and dependencies diff --git a/cli/image-watermark.py b/cli/image-watermark.py index a3bed1090..4b3396c17 100755 --- a/cli/image-watermark.py +++ b/cli/image-watermark.py @@ -52,17 +52,17 @@ def get_watermark(image, params): data = np.asarray(image) decoder = WatermarkDecoder(options.type, params.length) decoded = decoder.decode(data, options.method) - try: - s = str(decoded, 'UTF-8').replace('\x00', '') - except Exception: - s = '' - return s + wm = decoded.decode(encoding='ascii', errors='ignore') + return wm def set_watermark(image, params): data = np.asarray(image) encoder = WatermarkEncoder() - encoder.set_watermark(options.type, params.wm.encode('utf-8')) + length = params.length // 8 + text = f"{params.wm:<{length}}"[:length] + bytearr = text.encode(encoding='ascii', errors='ignore') + encoder.set_watermark(options.type, bytearr) encoded = encoder.encode(data, options.method) image = Image.fromarray(encoded) return image @@ -83,8 +83,8 @@ def watermark(params, file): exif = get_exif(image) if params.command == 'read': + fn = params.input wm = get_watermark(image, params) - log.info({ 'file': file, 'watermark': wm, 'exif': exif, 'resolution': f'{image.width}x{image.height}' }) elif params.command == 'write': metadata = b'' if params.strip else set_exif(exif) @@ -95,17 +95,18 @@ def watermark(params, file): image.save(fn, exif=metadata) if params.verify: + image = Image.open(fn) data = np.asarray(image) decoder = WatermarkDecoder(options.type, params.length) decoded = decoder.decode(data, options.method) - if decoded.startswith(b'\xff'): - wm = '' - else: - wm = str(decoded, 'UTF-8').replace('\x00', '') + wm = decoded.decode(encoding='ascii', errors='ignore') else: wm = params.wm - log.info({ 'file': fn, 'watermark': wm, 'exif': None if params.strip else exif, 'resolution': f'{image.width}x{image.height}' }) + log.info({ 'file': fn }) + log.info({ 'resolution': f'{image.width}x{image.height}' }) + log.info({ 'watermark': wm }) + log.info({ 'exif': None if params.strip else exif }) if __name__ == '__main__': @@ -114,11 +115,11 @@ if __name__ == '__main__': parser.add_argument('--wm', type=str, required=False, default='sdnext', help='watermark string') parser.add_argument('--strip', default=False, action='store_true', help = "strip existing exif data") parser.add_argument('--verify', default=False, action='store_true', help = "verify watermark during write") - parser.add_argument('--length', type=int, default=16, help="watermark length in bits") + parser.add_argument('--length', type=int, default=32, help="watermark length in bits") parser.add_argument('--output', type=str, required=False, default='', help='folder to store images, default is overwrite in-place') parser.add_argument('input', type=str, nargs='*') args = parser.parse_args() - log.info({ 'watermark args': vars(args), 'options': options }) + # log.info({ 'watermark args': vars(args), 'options': options }) for arg in args.input: if os.path.isfile(arg): watermark(args, arg) diff --git a/installer.py b/installer.py index 0bcedfd01..dfe1781be 100644 --- a/installer.py +++ b/installer.py @@ -147,9 +147,13 @@ def installed(package, friendly: str = None): version = pkg_resources.get_distribution(p[0]).version # log.debug(f"Package version found: {p[0]} {version}") if len(p) > 1: - ok = ok and version == p[1] - if not ok: - log.warning(f"Package wrong version: {p[0]} {version} required {p[1]}") + exact = version == p[1] + ok = ok and (exact or args.experimental) + if not exact: + if args.experimental: + log.warning(f"Package allowing experimental: {p[0]} {version} required {p[1]}") + else: + log.warning(f"Package wrong version: {p[0]} {version} required {p[1]}") else: log.debug(f"Package version not found: {p[0]}") return ok diff --git a/javascript/setHints.js b/javascript/setHints.js index b83c88761..368a79801 100644 --- a/javascript/setHints.js +++ b/javascript/setHints.js @@ -63,6 +63,7 @@ async function setHints() { let localized = 0; let hints = 0; locale.finished = true; + const t0 = performance.now(); for (const el of elements) { const found = locale.data.find((l) => l.label === el.textContent.trim()); if (found?.localized?.length > 0) { @@ -82,8 +83,9 @@ async function setHints() { } } } + const t1 = performance.now(); console.log('setHints', { - type: locale.type, elements: elements.length, localized, hints, data: locale.data.length, + type: locale.type, elements: elements.length, localized, hints, data: locale.data.length, time: t1 - t0, }); // validateHints(elements, locale.data) } diff --git a/javascript/style.css b/javascript/style.css index 7bf2dcefd..4032742c4 100644 --- a/javascript/style.css +++ b/javascript/style.css @@ -27,6 +27,7 @@ div.gradio-html.min{ min-height: 0; } .gradio-slider input[type="number"]{ width: 6em; } .hidden { display: none; } footer { display: none; } +td { border-bottom: none !important; } /* general styled components */ .gradio-button.tool{ max-width: 1em; min-width: 1em !important; align-self: end; font-size: 1.4em } diff --git a/modules/api/api.py b/modules/api/api.py index a580af3f7..d4d443378 100644 --- a/modules/api/api.py +++ b/modules/api/api.py @@ -143,7 +143,7 @@ class Api: self.add_api_route("/sdapi/v1/reload-checkpoint", self.reloadapi, methods=["POST"]) self.add_api_route("/sdapi/v1/scripts", self.get_scripts_list, methods=["GET"], response_model=models.ScriptsList) self.add_api_route("/sdapi/v1/script-info", self.get_script_info, methods=["GET"], response_model=List[models.ScriptInfo]) - self.add_api_route("/sdapi/v1/log", self.get_log_buffer, methods=["GET"], response_model=List) + self.app.add_api_route("/sdapi/v1/log", self.get_log_buffer, methods=["GET"], response_model=List) # bypass auth self.default_script_arg_txt2img = [] self.default_script_arg_img2img = [] diff --git a/modules/hashes.py b/modules/hashes.py index 3dd9bdb6c..76953c6bd 100644 --- a/modules/hashes.py +++ b/modules/hashes.py @@ -1,7 +1,5 @@ import hashlib -import json import os.path -import filelock from rich import progress from modules import shared from modules.paths import data_path @@ -11,20 +9,16 @@ cache_data = None def dump_cache(): - with filelock.FileLock(f"{cache_filename}.lock"): - with open(cache_filename, "w", encoding="utf8") as file: - json.dump(cache_data, file, indent=4) + shared.writefile(cache_data, cache_filename) def cache(subsection): global cache_data # pylint: disable=global-statement if cache_data is None: - with filelock.FileLock(f"{cache_filename}.lock"): - if not os.path.isfile(cache_filename): - cache_data = {} - else: - with open(cache_filename, "r", encoding="utf8") as file: - cache_data = json.load(file) + if not os.path.isfile(cache_filename): + cache_data = {} + else: + cache_data = shared.readfile(cache_filename) s = cache_data.get(subsection, {}) cache_data[subsection] = s return s diff --git a/modules/images.py b/modules/images.py index 0218a2436..c99b227c8 100644 --- a/modules/images.py +++ b/modules/images.py @@ -443,6 +443,8 @@ def atomically_save_image(): except Exception: shared.log.warning(f'Unknown image format: {extension}') image_format = 'JPEG' + if shared.opts.image_watermark_enabled: + image = set_watermark(image, shared.opts.image_watermark) shared.log.debug(f'Saving image: {image_format} {fn} {image.size}') # actual save exifinfo = (exifinfo or "") if shared.opts.image_metadata else "" @@ -483,14 +485,8 @@ def atomically_save_image(): with open(os.path.join(paths.data_path, "params.txt"), "w", encoding="utf8") as file: file.write(exifinfo) if shared.opts.save_log_fn != '' and len(exifinfo) > 0: - try: - with open(os.path.join(paths.data_path, shared.opts.save_log_fn), mode='a+', encoding='utf-8') as f: - entry = { 'filename': filename, 'time': datetime.datetime.now().isoformat(), 'info': exifinfo } - json.dump(entry, f) - f.write(os.linesep) - shared.log.debug(f'Log file updated: {os.path.join(paths.data_path, shared.opts.save_log_fn)}') - except Exception as e: - shared.log.warning(f'Failed to save log file: {shared.opts.save_log_fn} {e}') + entry = { 'filename': filename, 'time': datetime.datetime.now().isoformat(), 'info': exifinfo } + shared.writefile(entry, os.path.join(paths.data_path, shared.opts.save_log_fn), mode='a+') save_queue.task_done() @@ -643,6 +639,10 @@ def read_info_from_image(image): items[ExifTags.TAGS[key]] = val elif val is not None and key in ExifTags.GPSTAGS: items[ExifTags.GPSTAGS[key]] = val + wm = get_watermark(image) + if wm != '': + # geninfo += f' Watermark: {wm}' + items['watermark'] = wm for key, val in items.items(): if isinstance(val, bytes): # decode bytestring @@ -696,3 +696,40 @@ def flatten(img, bgcolor): background.paste(img, mask=img) img = background return img.convert('RGB') + + +def set_watermark(image, watermark): + from imwatermark import WatermarkEncoder + wm_type = 'bytes' + wm_method = 'dwtDctSvd' + wm_length = 32 + length = wm_length // 8 + info = image.info + data = np.asarray(image) + encoder = WatermarkEncoder() + text = f"{watermark:<{length}}"[:length] + bytearr = text.encode(encoding='ascii', errors='ignore') + try: + encoder.set_watermark(wm_type, bytearr) + encoded = encoder.encode(data, wm_method) + image = Image.fromarray(encoded) + image.info = info + shared.log.debug(f'Set watermark: {watermark} method={wm_method} bits={wm_length}') + except Exception as e: + shared.log.warning(f'Set watermark error: {watermark} method={wm_method} bits={wm_length} {e}') + return image + + +def get_watermark(image): + from imwatermark import WatermarkDecoder + wm_type = 'bytes' + wm_method = 'dwtDctSvd' + wm_length = 32 + data = np.asarray(image) + decoder = WatermarkDecoder(wm_type, wm_length) + try: + decoded = decoder.decode(data, wm_method) + wm = decoded.decode(encoding='ascii', errors='ignore') + except Exception: + wm = '' + return wm diff --git a/modules/modelloader.py b/modules/modelloader.py index 682c30106..7a444aa3c 100644 --- a/modules/modelloader.py +++ b/modules/modelloader.py @@ -1,7 +1,6 @@ import os import shutil import importlib -import json from typing import Dict from urllib.parse import urlparse @@ -37,7 +36,7 @@ def download_diffusers_model(hub_id: str, cache_dir: str = None, download_config hf.login(token) pipeline_dir = DiffusionPipeline.download(hub_id, **download_config) try: - model_info_dict = hf.model_info(hub_id).cardData # TODO HF-Hub cardData invalid property + model_info_dict = hf.model_info(hub_id).cardData # pylint: disable=no-member # TODO Diffusers is this real error? except Exception: model_info_dict = None # some checkpoints need to be downloaded as "hidden" as they just serve as pre- or post-pipelines of other pipelines @@ -47,8 +46,7 @@ def download_diffusers_model(hub_id: str, cache_dir: str = None, download_config # mark prior as hidden with open(os.path.join(download_dir, "hidden"), "w", encoding="utf-8") as f: f.write("True") - with open(os.path.join(pipeline_dir, "model_info.json"), "w", encoding="utf-8") as json_file: - json.dump(model_info_dict, json_file) + shared.writefile(model_info_dict, os.path.join(pipeline_dir, "model_info.json")) return pipeline_dir diff --git a/modules/sd_models.py b/modules/sd_models.py index 8f6c037e5..f81715d41 100644 --- a/modules/sd_models.py +++ b/modules/sd_models.py @@ -7,7 +7,6 @@ import threading from os import mkdir from urllib import request from enum import Enum -import filelock from rich import progress # pylint: disable=redefined-builtin import torch import safetensors.torch @@ -80,12 +79,7 @@ class CheckpointInfo: self.model_name = repo[0]['name'] if os.path.isfile(repo[0]['model_info']): file_path = repo[0]['model_info'] - with open(file_path, "r", encoding="utf-8") as json_file: - try: - self.model_info = json.load(json_file) - except Exception as e: - shared.log.error(f'Error loading model info: {json_file} {e}') - self.model_info = {} + self.model_info = shared.readfile(file_path, silent=True) self.shorthash = self.sha256[0:10] if self.sha256 else None self.title = self.name if self.shorthash is None else f'{self.name} [{self.shorthash}]' @@ -116,6 +110,11 @@ class CheckpointInfo: return self.shorthash +class NoWatermark: + def apply_watermark(self, img): + return img + + def setup_model(): if not os.path.exists(model_path): os.makedirs(model_path) @@ -276,20 +275,11 @@ def get_state_dict_from_checkpoint(pl_sd): def write_metadata(): - def default(obj): - shared.log.debug(f"Model metadata not a valid object: {obj}") - return str(obj) - global sd_metadata_pending # pylint: disable=global-statement if sd_metadata_pending == 0: shared.log.debug(f"Model metadata: {sd_metadata_file} no changes") return - with filelock.FileLock(f"{sd_metadata_file}.lock"): - try: - with open(sd_metadata_file, "w", encoding="utf8") as file: - json.dump(sd_metadata, file, indent=4, skipkeys=True, ensure_ascii=True, check_circular=True, allow_nan=True, default=default) - except Exception as e: - shared.log.error(f"Model metadata save error: {sd_metadata_file} {e}") + shared.writefile(sd_metadata, sd_metadata_file) shared.log.info(f"Model metadata saved: {sd_metadata_file} {sd_metadata_pending}") sd_metadata_pending = 0 @@ -297,15 +287,10 @@ def write_metadata(): def read_metadata_from_safetensors(filename): global sd_metadata # pylint: disable=global-statement if sd_metadata is None: - with filelock.FileLock(f"{sd_metadata_file}.lock"): - if not os.path.isfile(sd_metadata_file): - sd_metadata = {} - else: - try: - with open(sd_metadata_file, "r", encoding="utf8") as file: - sd_metadata = json.load(file) - except Exception: - sd_metadata = {} + if not os.path.isfile(sd_metadata_file): + sd_metadata = {} + else: + sd_metadata = shared.readfile(sd_metadata_file) res = sd_metadata.get(filename, None) if res is not None: return res @@ -703,6 +688,9 @@ def load_diffuser(checkpoint_info=None, already_loaded_state_dict=None, timer=No elif "Kandinsky" in sd_model.__class__.__name__: sd_model.scheduler.name = 'DDIM' + if hasattr(sd_model, "watermark"): + sd_model.watermark = NoWatermark() + # Prior pipelines if hasattr(checkpoint_info, 'model_info') and checkpoint_info.model_info is not None and "prior" in checkpoint_info.model_info: prior_id = checkpoint_info.model_info["prior"] diff --git a/modules/shared.py b/modules/shared.py index 854191493..fb726d6e7 100644 --- a/modules/shared.py +++ b/modules/shared.py @@ -9,6 +9,7 @@ from enum import Enum import gradio as gr import tqdm import requests +import fasteners from modules import errors, ui_components, shared_items, cmd_args from modules.paths_internal import models_path, script_path, data_path, sd_configs_path, sd_default_config, sd_model_file, default_sd_model_file, extensions_dir, extensions_builtin_dir # pylint: disable=W0611 from modules.dml import directml_hijack_init, directml_override_opts @@ -298,6 +299,38 @@ def refresh_themes(): log.error('Exception refreshing UI themes') +def readfile(filename, silent=False): + data = {} + try: + if not os.path.exists(filename): + return {} + with fasteners.InterProcessLock(f"{filename}.lock"): + with open(filename, "r", encoding="utf8") as file: + data = json.load(file) + if not silent: + log.debug(f'Reading: {filename} len={len(data)}') + except Exception as e: + log.error(f'Reading failed: {filename} {e}') + return data + + +def writefile(data, filename, mode='w'): + + def default(obj): + log.error(f"Saving: {filename} not a valid object: {obj}") + return str(obj) + + try: + with fasteners.InterProcessLock(f"{filename}.lock"): + # skipkeys=True, ensure_ascii=True, check_circular=True, allow_nan=True + output = json.dumps(data, indent=2, default=default) + log.debug(f'Saving: {filename} len={len(output)}') + with open(filename, mode, encoding="utf8") as file: + file.write(output) + except Exception as e: + log.error(f'Saving failed: {filename} {e}') + + if devices.backend == "cpu": cross_attention_optimization_default = "Doggettx's" elif devices.backend == "mps": @@ -405,6 +438,8 @@ options_templates.update(options_section(('saving-images', "Image Options"), { "samples_save": OptionInfo(True, "Always save all generated images"), "samples_format": OptionInfo('jpg', 'File format for generated images', gr.Dropdown, lambda: {"choices": ["jpg", "png", "webp", "tiff", "jp2"]}), "image_metadata": OptionInfo(True, "Include metadata in saved images"), + "image_watermark_enabled": OptionInfo(False, "Include watermark in saved images"), + "image_watermark": OptionInfo('', "Image watermark string"), "samples_filename_pattern": OptionInfo("[seq]-[prompt_words]", "Images filename pattern", component_args=hide_dirs), "directories_max_prompt_words": OptionInfo(8, "Max prompt words for [prompt_words] pattern", gr.Slider, {"minimum": 1, "maximum": 99, "step": 1, **hide_dirs}), "save_images_add_number": OptionInfo(True, "Add number to filename when saving", component_args=hide_dirs), @@ -662,8 +697,13 @@ class Options: if cmd_opts.freeze: log.warning(f'Settings saving is disabled: {filename}') return - with open(filename, "w", encoding="utf8") as file: - json.dump(self.data, file, indent=4) + try: + output = json.dumps(self.data, indent=2) + log.debug(f'Saving settings: {filename} len={len(output)}') + with open(filename, "w", encoding="utf8") as file: + file.write(output) + except Exception as e: + log.error(f'Saving settings failed: {filename} {e}') def same_type(self, x, y): if x is None or y is None: @@ -677,8 +717,7 @@ class Options: log.debug(f'Created default config: {filename}') self.save(filename) return - with open(filename, "r", encoding="utf8") as file: - self.data = json.load(file) + self.data = readfile(filename) if self.data.get('quicksettings') is not None and self.data.get('quicksettings_list') is None: self.data['quicksettings_list'] = [i.strip() for i in self.data.get('quicksettings').split(',')] bad_settings = 0 diff --git a/modules/textual_inversion/textual_inversion.py b/modules/textual_inversion/textual_inversion.py index 8529caa0f..ee4ae45f0 100644 --- a/modules/textual_inversion/textual_inversion.py +++ b/modules/textual_inversion/textual_inversion.py @@ -150,7 +150,6 @@ class EmbeddingDatabase: text_inv_tokens = [t for t in text_inv_tokens if not (len(t.split("_")) > 1 and t.split("_")[-1].isdigit())] except Exception: text_inv_tokens = [] - pass def load_from_file(self, path, filename): name, ext = os.path.splitext(filename) @@ -378,6 +377,8 @@ def train_embedding(id_task, embedding_name, learn_rate, batch_size, gradient_st filename = os.path.join(shared.opts.embeddings_dir, f'{embedding_name}.pt') + if log_directory == '': + log_directory = f"{os.path.join(shared.cmd_opts.data_dir, 'train/log/embeddings')}" log_directory = os.path.join(log_directory, embedding_name) unload = shared.opts.unload_models_when_training diff --git a/modules/ui.py b/modules/ui.py index 8121dd298..876998147 100644 --- a/modules/ui.py +++ b/modules/ui.py @@ -1161,10 +1161,8 @@ def create_ui(startup_timer = None): ) startup_timer.record("ui-defaults") - loadsave.dump_defaults() demo.ui_loadsave = loadsave - return demo diff --git a/modules/ui_loadsave.py b/modules/ui_loadsave.py index 1ce4af25d..b6a20a92c 100644 --- a/modules/ui_loadsave.py +++ b/modules/ui_loadsave.py @@ -1,8 +1,5 @@ -import json import os - import gradio as gr - from modules import errors from modules.ui_components import ToolButton @@ -12,20 +9,14 @@ class UiLoadsave: def __init__(self, filename): self.filename = filename - self.ui_settings = {} self.component_mapping = {} - self.error_loading = False self.finalized_ui = False - self.ui_defaults_view = None - self.ui_defaults_apply = None - self.ui_defaults_review = None - self.ui_defaults_restore = None - try: - if os.path.exists(self.filename): - self.ui_settings = self.read_from_file() - except Exception as e: - self.error_loading = True - errors.display(e, "loading settings") + self.ui_defaults_view = None # button + self.ui_defaults_apply = None # button + self.ui_defaults_review = None # button + self.ui_defaults_restore = None # button + self.ui_defaults = {} + self.ui_settings = self.read_from_file() def add_component(self, path, x): """adds component to the registry of tracked components""" @@ -37,8 +28,10 @@ class UiLoadsave: if getattr(obj, 'do_not_save_to_config', False): return saved_value = self.ui_settings.get(key, None) + self.ui_defaults[key] = getattr(obj, field) if saved_value is None: - self.ui_settings[key] = getattr(obj, field) + # self.ui_settings[key] = getattr(obj, field) + pass elif condition and not condition(saved_value): pass else: @@ -97,47 +90,53 @@ class UiLoadsave: self.add_component(f"{path}/{x.value}", x) def read_from_file(self): - if os.path.exists(self.filename): - with open(self.filename, "r", encoding="utf8") as file: - return json.load(file) - else: - return {} + from modules.shared import readfile + return readfile(self.filename) def write_to_file(self, current_ui_settings): - with open(self.filename, "w", encoding="utf8") as file: - json.dump(current_ui_settings, file, indent=4) + from modules.shared import writefile + writefile(current_ui_settings, self.filename) def dump_defaults(self): - """saves default values to a file unless tjhe file is present and there was an error loading default values at start""" - if self.error_loading and os.path.exists(self.filename): + """saves default values to a file unless the file is present and there was an error loading default values at start""" + if os.path.exists(self.filename): return self.write_to_file(self.ui_settings) - def iter_changes(self, current_ui_settings, values): + def iter_changes(self, values): + from modules.shared import log """ given a dictionary with defaults from a file and current values from gradio elements, returns an iterator over tuples of values that are not the same between the file and the current; tuple contents are: path, old value, new value """ - for (path, component), new_value in zip(self.component_mapping.items(), values): - old_value = current_ui_settings.get(path) + # for (path, component), new_value in zip(self.component_mapping.items(), values): + for i, name in enumerate(self.component_mapping): + component = self.component_mapping[name] choices = getattr(component, 'choices', None) + new_value = values[i] if isinstance(new_value, int) and choices: if new_value >= len(choices): continue new_value = choices[new_value] - if new_value == old_value: + old_value = self.ui_settings.get(name, None) + default_value = self.ui_defaults.get(name, '') + if old_value == new_value: continue - if old_value is None and new_value == '' or new_value == []: + if old_value is None and (new_value == '' or new_value == []): continue - yield path, old_value, new_value + if (new_value == default_value) and (old_value is None): + continue + log.debug(f'Settings: name={name} component={component} old={old_value} default={default_value} new={new_value}') + yield name, old_value, new_value, default_value + return [] def ui_view(self, *values): - text = [""] - for path, old_value, new_value in self.iter_changes(self.read_from_file(), values): + text = ['
PathOld valueNew value
'] + for path, old_value, new_value, default_value in self.iter_changes(values): if old_value is None: old_value = "None" - text.append(f"") + text.append(f"") if len(text) == 1: text.append("") text.append("") @@ -146,7 +145,7 @@ class UiLoadsave: def ui_apply(self, *values): num_changed = 0 current_ui_settings = self.read_from_file() - for path, _, new_value in self.iter_changes(current_ui_settings.copy(), values): + for path, _, new_value, _ in self.iter_changes(values): num_changed += 1 current_ui_settings[path] = new_value if num_changed == 0: diff --git a/modules/ui_models.py b/modules/ui_models.py index bd8eb2cbf..d1afb8183 100644 --- a/modules/ui_models.py +++ b/modules/ui_models.py @@ -31,7 +31,7 @@ def create_ui(): with gr.Row(): custom_name = gr.Textbox(label="New model name") with gr.Row(): - precision = gr.Radio(choices=["fp32", "fp16", "bf16"], value="fp32", label="Model precision") + precision = gr.Radio(choices=["fp32", "fp16", "bf16"], value="fp16", label="Model precision") m_type = gr.Radio(choices=["disabled", "no-ema", "ema-only"], value="disabled", label="Model pruning methods") with gr.Row(): checkpoint_formats = gr.CheckboxGroup(choices=["ckpt", "safetensors"], value=["safetensors"], label="Model Format") diff --git a/modules/ui_train.py b/modules/ui_train.py index 209d21fba..d5f4970db 100644 --- a/modules/ui_train.py +++ b/modules/ui_train.py @@ -188,7 +188,7 @@ def create_ui(txt2img_preview_params): ti_save_every = gr.Number(label='Create interim embeddings', value=500, precision=0) ti_save_image_with_stored_embedding = gr.Checkbox(label='Save images with embedding in PNG chunks', value=True) ti_preview_from_txt2img = gr.Checkbox(label='Use current settings for previews', value=False) - ti_log_directory = gr.Textbox(label='Log directory', placeholder="Path to directory where to write outputs", value=f"{os.path.join(shared.cmd_opts.data_dir, 'train/log/embeddings')}") + ti_log_directory = gr.Textbox(label='Log directory', placeholder="Defaults to train/log/embedding", value="") ti_stop.click(fn=lambda: shared.state.interrupt(), inputs=[], outputs=[]) diff --git a/requirements.txt b/requirements.txt index b1cb94aab..6d6673631 100644 --- a/requirements.txt +++ b/requirements.txt @@ -42,10 +42,10 @@ yapf scikit-image basicsr compel +fasteners pyarrow==11.0.0 typing-extensions==4.7.1 antlr4-python3-runtime==4.9.3 -pydantic==1.10.11 requests==2.31.0 tqdm==4.65.0 accelerate==0.20.3 @@ -64,3 +64,4 @@ tomesd==0.1.3 urllib3==1.26.15 Pillow==9.5.0 timm==0.6.13 +pydantic==1.10.11 diff --git a/webui.py b/webui.py index 6edb35cb8..72e878c40 100644 --- a/webui.py +++ b/webui.py @@ -2,6 +2,7 @@ from __future__ import annotations import os import re import sys +import glob import signal import asyncio import logging @@ -159,6 +160,11 @@ def initialize(): # make the program just exit at ctrl+c without waiting for anything def sigint_handler(_sig, _frame): log.info('Exiting') + try: + for f in glob.glob("*.lock"): + os.remove(f) + except Exception: + pass sys.exit(0) signal.signal(signal.SIGINT, sigint_handler)
VariableUser valueNew valueDefault value
{path}{old_value}{new_value}
{path}{old_value}{new_value}{default_value}
No changes