add file logging

2026-09-18 16:54:33 +02:00 · 2023-01-26 10:27:38 -05:00
parent 8dec75d99e
commit 40f11ed1fe
9 changed files with 306 additions and 111 deletions
@@ -1,80 +1,160 @@
-# Stable Diffusion - Automatic
+# Stable Diffusion web UI
+A browser interface based on Gradio library for Stable Diffusion.

-*Heavily opinionated custom fork of* <https://github.com/AUTOMATIC1111/stable-diffusion-webui>  
+![](screenshot.png)

-![](ui-screenshot.jpg)
+## Features
+[Detailed feature showcase with images](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features):
+- Original txt2img and img2img modes
+- One click install and run script (but you still must install python and git)
+- Outpainting
+- Inpainting
+- Color Sketch
+- Prompt Matrix
+- Stable Diffusion Upscale
+- Attention, specify parts of text that the model should pay more attention to
+    - a man in a ((tuxedo)) - will pay more attention to tuxedo
+    - a man in a (tuxedo:1.21) - alternative syntax
+    - select text and press ctrl+up or ctrl+down to automatically adjust attention to selected text (code contributed by anonymous user)
+- Loopback, run img2img processing multiple times
+- X/Y plot, a way to draw a 2 dimensional plot of images with different parameters
+- Textual Inversion
+    - have as many embeddings as you want and use any names you like for them
+    - use multiple embeddings with different numbers of vectors per token
+    - works with half precision floating point numbers
+    - train embeddings on 8GB (also reports of 6GB working)
+- Extras tab with:
+    - GFPGAN, neural network that fixes faces
+    - CodeFormer, face restoration tool as an alternative to GFPGAN
+    - RealESRGAN, neural network upscaler
+    - ESRGAN, neural network upscaler with a lot of third party models
+    - SwinIR and Swin2SR([see here](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/2092)), neural network upscalers
+    - LDSR, Latent diffusion super resolution upscaling
+- Resizing aspect ratio options
+- Sampling method selection
+    - Adjust sampler eta values (noise multiplier)
+    - More advanced noise setting options
+- Interrupt processing at any time
+- 4GB video card support (also reports of 2GB working)
+- Correct seeds for batches
+- Live prompt token length validation
+- Generation parameters
+     - parameters you used to generate images are saved with that image
+     - in PNG chunks for PNG, in EXIF for JPEG
+     - can drag the image to PNG info tab to restore generation parameters and automatically copy them into UI
+     - can be disabled in settings
+     - drag and drop an image/text-parameters to promptbox
+- Read Generation Parameters Button, loads parameters in promptbox to UI
+- Settings page
+- Running arbitrary python code from UI (must run with --allow-code to enable)
+- Mouseover hints for most UI elements
+- Possible to change defaults/mix/max/step values for UI elements via text config
+- Tiling support, a checkbox to create images that can be tiled like textures
+- Progress bar and live image generation preview
+    - Can use a separate neural network to produce previews with almost none VRAM or compute requirement
+- Negative prompt, an extra text field that allows you to list what you don't want to see in generated image
+- Styles, a way to save part of prompt and easily apply them via dropdown later
+- Variations, a way to generate same image but with tiny differences
+- Seed resizing, a way to generate same image but at slightly different resolution
+- CLIP interrogator, a button that tries to guess prompt from an image
+- Prompt Editing, a way to change prompt mid-generation, say to start making a watermelon and switch to anime girl midway
+- Batch Processing, process a group of files using img2img
+- Img2img Alternative, reverse Euler method of cross attention control
+- Highres Fix, a convenience option to produce high resolution pictures in one click without usual distortions
+- Reloading checkpoints on the fly
+- Checkpoint Merger, a tab that allows you to merge up to 3 checkpoints into one
+- [Custom scripts](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Custom-Scripts) with many extensions from community
+- [Composable-Diffusion](https://energy-based-model.github.io/Compositional-Visual-Generation-with-Composable-Diffusion-Models/), a way to use multiple prompts at once
+     - separate prompts using uppercase `AND`
+     - also supports weights for prompts: `a cat :1.2 AND a dog AND a penguin :2.2`
+- No token limit for prompts (original stable diffusion lets you use up to 75 tokens)
+- DeepDanbooru integration, creates danbooru style tags for anime prompts
+- [xformers](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Xformers), major speed increase for select cards: (add --xformers to commandline args)
+- via extension: [History tab](https://github.com/yfszzx/stable-diffusion-webui-images-browser): view, direct and delete images conveniently within the UI
+- Generate forever option
+- Training tab
+     - hypernetworks and embeddings options
+     - Preprocessing images: cropping, mirroring, autotagging using BLIP or deepdanbooru (for anime)
+- Clip skip
+- Hypernetworks
+- Loras (same as Hypernetworks but more pretty)
+- A sparate UI where you can choose, with preview, which embeddings, hypernetworks or Loras to add to your prompt. 
+- Can select to load a different VAE from settings screen
+- Estimated completion time in progress bar
+- API
+- Support for dedicated [inpainting model](https://github.com/runwayml/stable-diffusion#inpainting-with-stable-diffusion) by RunwayML.
+- via extension: [Aesthetic Gradients](https://github.com/AUTOMATIC1111/stable-diffusion-webui-aesthetic-gradients), a way to generate images with a specific aesthetic by using clip images embeds (implementation of [https://github.com/vicgalle/stable-diffusion-aesthetic-gradients](https://github.com/vicgalle/stable-diffusion-aesthetic-gradients))
+- [Stable Diffusion 2.0](https://github.com/Stability-AI/stablediffusion) support - see [wiki](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#stable-diffusion-20) for instructions
+- [Alt-Diffusion](https://arxiv.org/abs/2211.06679) support - see [wiki](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#alt-diffusion) for instructions
+- Now without any bad letters!
+- Load checkpoints in safetensors format
+- Eased resolution restriction: generated image's domension must be a multiple of 8 rather than 64
+- Now with a license!
+- Reorder elements in the UI from settings screen
+- 

-<br>
+## Installation and Running
+Make sure the required [dependencies](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Dependencies) are met and follow the instructions available for both [NVidia](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-NVidia-GPUs) (recommended) and [AMD](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-AMD-GPUs) GPUs.

-## Notes
+Alternatively, use online services (like Google Colab):

-Fork is as close as up-to-date with origin as time allows  
-All code changes are merged upstream whenever possible  
+- [List of Online Services](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Online-Services)

-Fork adds extra functionality:
- Ships with additional **extensions**  
-  e.g. `System Info`, `Steps Animation`, etc.  
- Ships with set of **CLI** tools that rely on *SD API* for execution:  
-  e.g. `generate`, `train`, `bench`, etc.  
-  [Full list](<cli/>)
+### Automatic Installation on Windows
+1. Install [Python 3.10.6](https://www.python.org/downloads/windows/), checking "Add Python to PATH"
+2. Install [git](https://git-scm.com/download/win).
+3. Download the stable-diffusion-webui repository, for example by running `git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git`.
+4. Place stable diffusion checkpoint (`model.ckpt`) in the `models/Stable-diffusion` directory (see [dependencies](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Dependencies) for where to get it).
+5. Run `webui-user.bat` from Windows Explorer as normal, non-administrator, user.

-Simplified start script: `automatic.sh`  
-*Existing `webui.sh` still exists for backward compatibility, fresh installs to auto-install dependencies, etc.*  
+### Automatic Installation on Linux
+1. Install the dependencies:
+```bash
+# Debian-based:
+sudo apt install wget git python3 python3-venv
+# Red Hat-based:
+sudo dnf install wget git python3
+# Arch-based:
+sudo pacman -S wget git python3
+```
+2. To install in `/home/$(whoami)/stable-diffusion-webui/`, run:
+```bash
+bash <(wget -qO- https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh)
+```

-> ./automatic.sh  
+### Installation on Apple Silicon

- Start in default mode with optimizations enabled
+Find the instructions [here](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Installation-on-Apple-Silicon).

-> ./automatic.sh env  
+## Contributing
+Here's how to add code to this repo: [Contributing](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Contributing)

- Print env info and exit  
-  Example:
+## Documentation
+The documentation was moved from this README over to the project's [wiki](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki).

-      Version: c07487a Tue Jan 24 08:04:31 2023 -0500
-      Platform: Ubuntu 22.04.1 LTS 5.15.79.1-microsoft-standard-WSL2 x86_64
-      Python 3.10.6
-      Torch: 2.0.0.dev20230118+cu118 CUDA: 11.8 cuDNN: 8700 GPU: NVIDIA GeForce RTX 3060 Arch: (8, 6)
+## Credits
+Licenses for borrowed code can be found in `Settings -> Licenses` screen, and also in `html/licenses.html` file.

-> ./automatic.sh install  
-
- Install requirements and exit
-
-> ./automatic.sh public  
-
- Start with listen on public IP with authentication enabled
-
-> ./automatic.sh clean  
-
- Start with all optimizations disabled  
-  Use this for troubleshooting  
-
-<br>
-
-## Differences
-
-Fork does differ in few things:
- Drops compatibility with `python` **3.7** and requires **3.9**  
- Updated **Python** libraries to latest known compatible versions  
-  e.g. `accelerate`, `transformers`, `numpy`, etc.  
- Includes opinionated **System** and **Options** configuration  
-  e.g. `samplers`, `upscalers`, etc.  
- Includes reskinned **UI**  
-  Black and orange dark theme with fixed width options panels and larger previews  
- Includes **SD2** configuration files  
- Uses simplified folder structure  
-  e.g. `/train`, `/outputs/*`  
- Modified training templates  
-
-Only Python library which is not auto-updated is `PyTorch` itself as that is very system specific  
-For some Torch optimizations notes, see Wiki
-
-Fork is compatible with regular **PyTorch 1.13** as well as pre-release of **PyTorch 2.0**  
-See [Wiki](https://github.com/vladmandic/automatic/wiki) for **Torch** optimization notes
-
-
-<br>
-
-## Docs
-
-Everything is in [Wiki](https://github.com/vladmandic/automatic/wiki)  
-Except my current [TODO](TODO.md)  
+- Stable Diffusion - https://github.com/CompVis/stable-diffusion, https://github.com/CompVis/taming-transformers
+- k-diffusion - https://github.com/crowsonkb/k-diffusion.git
+- GFPGAN - https://github.com/TencentARC/GFPGAN.git
+- CodeFormer - https://github.com/sczhou/CodeFormer
+- ESRGAN - https://github.com/xinntao/ESRGAN
+- SwinIR - https://github.com/JingyunLiang/SwinIR
+- Swin2SR - https://github.com/mv-lab/swin2sr
+- LDSR - https://github.com/Hafiidz/latent-diffusion
+- MiDaS - https://github.com/isl-org/MiDaS
+- Ideas for optimizations - https://github.com/basujindal/stable-diffusion
+- Cross Attention layer optimization - Doggettx - https://github.com/Doggettx/stable-diffusion, original idea for prompt editing.
+- Cross Attention layer optimization - InvokeAI, lstein - https://github.com/invoke-ai/InvokeAI (originally http://github.com/lstein/stable-diffusion)
+- Sub-quadratic Cross Attention layer optimization - Alex Birch (https://github.com/Birch-san/diffusers/pull/1), Amin Rezaei (https://github.com/AminRezaei0x443/memory-efficient-attention)
+- Textual Inversion - Rinon Gal - https://github.com/rinongal/textual_inversion (we're not using his code, but we are using his ideas).
+- Idea for SD upscale - https://github.com/jquesnelle/txt2imghd
+- Noise generation for outpainting mk2 - https://github.com/parlance-zz/g-diffuser-bot
+- CLIP interrogator idea and borrowing some code - https://github.com/pharmapsychotic/clip-interrogator
+- Idea for Composable Diffusion - https://github.com/energy-based-model/Compositional-Visual-Generation-with-Composable-Diffusion-Models-PyTorch
+- xformers - https://github.com/facebookresearch/xformers
+- DeepDanbooru - interrogator for anime diffusers https://github.com/KichangKim/DeepDanbooru
+- Security advice - RyotaK
+- Initial Gradio script - posted on 4chan by an Anonymous user. Thank you Anonymous user.
+- (You)
@@ -276,6 +276,7 @@ def process_file(f: str, dst: str = None):
    else:
        log.debug({ 'no body': f })

+    image.close()

 def process_images(src: str, dst: str, args = None):
    params.src = src
@@ -311,9 +312,3 @@ if __name__ == '__main__':
            for root, _sub_dirs, files in os.walk(loc):
                for f in files:
                    process_file(os.path.join(root, f), dst)
-
-"""
- add interrogate on save
- write final stats
- create final grid
-"""
@@ -5,11 +5,20 @@ generic helper methods

 import logging

-
-logging.basicConfig(level = logging.INFO, format = '%(asctime)s %(levelname)s: %(message)s')
+log_format = '%(asctime)s %(levelname)s: %(message)s'
+logging.basicConfig(level = logging.INFO, format = log_format)
 log = logging.getLogger("sd")


+def set_logfile(logfile):
+    fh = logging.FileHandler(logfile)
+    formatter = logging.Formatter(log_format)
+    fh.setLevel(log.getEffectiveLevel())
+    fh.setFormatter(formatter)
+    log.addHandler(fh)
+    log.info({ 'log file': logfile })
+
+
 class Map(dict):
    def __init__(self, *args, **kwargs):
        super(Map, self).__init__(*args, **kwargs)
@@ -24,7 +24,7 @@ import filetype
 from PIL import Image

 sys.path.append(os.path.join(os.path.dirname(__file__), 'modules'))
-from modules.util import Map, log
+from modules.util import Map, log, set_logfile
 from modules.losschart import plot
 from modules.ffmpeg import extract
 from modules.lossrate import gen_loss_rate_str
@@ -183,9 +183,16 @@ async def preprocess(params):


 async def check(params):
-    log.info({ 'checking server options' })
    global options # pylint: disable=global-statement
    options = await get('/sdapi/v1/options')
+    global cmdflags # pylint: disable=global-statement
+    cmdflags = await get('/sdapi/v1/cmd-flags')
+
+    logdir = os.path.abspath(os.path.join(cmdflags['embeddings_dir'], '../train/log', params.name))
+    logfile = os.path.abspath(os.path.join(cmdflags['embeddings_dir'], '../train/log', params.name + '.log'))
+    set_logfile(logfile)
+
+    log.info({ 'checking server options' })

    options['training_xattention_optimizations'] = False
    options['training_image_repeats_per_epoch'] = 1
@@ -206,8 +213,6 @@ async def check(params):
            options['sd_model_checkpoint'] = found[0]

    log.debug({ 'check embedding': params.name })
-    global cmdflags # pylint: disable=global-statement
-    cmdflags = await get('/sdapi/v1/cmd-flags')

    lst = os.path.join(cmdflags['embeddings_dir'])
    log.debug({ 'embeddings folder': lst })
@@ -221,7 +226,6 @@ async def check(params):
            log.error({ 'embedding exists': match.name })
            await close()
            exit()
-    logdir = os.path.abspath(os.path.join(cmdflags['embeddings_dir'], '../train/log', params.name))
    f = os.path.join(logdir, 'train.csv')
    if os.path.isfile(f):
        if params.overwrite:
@@ -237,6 +241,7 @@ async def check(params):

    log.debug({ 'options': 'update' })
    await post('/sdapi/v1/options', options)
+
    return


@@ -148,10 +148,7 @@ def run_extension_installer(extension_dir):
        env = os.environ.copy()
        env['PYTHONPATH'] = os.path.abspath(".")

-        stdout = run(f'"{python}" "{path_installer}"', errdesc=f"Error running install.py for extension {extension_dir}", custom_env=env)
-        if stdout is not None:
-            print(stdout)
-
+        print(run(f'"{python}" "{path_installer}"', errdesc=f"Error running install.py for extension {extension_dir}", custom_env=env))
    except Exception as e:
        print(e, file=sys.stderr)

@@ -220,8 +217,8 @@ def prepare_environment():

    commit = commit_hash()

-    # print(f"Python {sys.version}")
-    # print(f"Commit hash: {commit}")
+    print(f"Python {sys.version}")
+    print(f"Commit hash: {commit}")
    
    if reinstall_torch or not is_installed("torch") or not is_installed("torchvision"):
        run(f'"{python}" -m {torch_command}', "Installing torch and torchvision", "Couldn't install torch")
@@ -270,7 +267,7 @@ def prepare_environment():

    if update_check:
        version_check(commit)
-
+    
    if "--exit" in sys.argv:
        print("Exiting because of --exit argument")
        exit(0)
@@ -1,30 +1,33 @@
+blendmodes
 accelerate
 basicsr
-clean-fid
-einops
-fastapi
-filetype
-font-roboto
+fairscale==0.4.4
 fonts
+font-roboto
 gfpgan
-gradio
+gradio==3.16.2
 invisible-watermark
-jsonmerge
-kornia
-lark
-mediapipe
 numpy
 omegaconf
 opencv-contrib-python
+requests
 piexif
 Pillow
-pytorch_lightning
+pytorch_lightning==1.7.7
 realesrgan
-resize-right
-safetensors
-scikit-image
-timm
+scikit-image>=0.19
+timm==0.4.12
+transformers==4.19.2
 torch
+einops
+jsonmerge
+clean-fid
+resize-right
 torchdiffeq
+kornia
+lark
+inflection
+GitPython
 torchsde
-transformers
+safetensors
+psutil
@@ -1,17 +1,30 @@
-accelerate==0.15.0
-basicsr==1.4.2
 blendmodes==2022
+transformers==4.19.2
+accelerate==0.12.0
+basicsr==1.4.2
 gfpgan==1.3.8
-GitPython==3.1.27
 gradio==3.16.2
-numpy==1.23.5
-omegaconf==2.2.3
+numpy==1.23.3
 Pillow==9.4.0
-protobuf==3.20.3
-pytorch_lightning==1.7.7
 realesrgan==0.3.0
+torch
+omegaconf==2.2.3
+pytorch_lightning==1.7.6
 scikit-image==0.19.2
+fonts
+font-roboto
 timm==0.6.7
+fairscale==0.4.9
+piexif==1.1.3
+einops==0.4.1
+jsonmerge==1.8.0
+clean-fid==0.1.29
+resize-right==0.0.2
 torchdiffeq==0.2.3
+kornia==0.6.7
+lark==1.1.2
+inflection==0.5.1
+GitPython==3.1.27
 torchsde==0.2.5
-transformers==4.25.1
+safetensors==0.2.7
+httpcore<=0.15
@@ -0,0 +1,19 @@
+#!/bin/bash
+####################################################################
+#                          macOS defaults                          #
+# Please modify webui-user.sh to change these instead of this file #
+####################################################################
+
+if [[ -x "$(command -v python3.10)" ]]
+then
+    python_cmd="python3.10"
+fi
+
+export install_dir="$HOME"
+export COMMANDLINE_ARGS="--skip-torch-cuda-test --no-half --use-cpu interrogate"
+export TORCH_COMMAND="pip install torch==1.12.1 torchvision==0.13.1"
+export K_DIFFUSION_REPO="https://github.com/brkirch/k-diffusion.git"
+export K_DIFFUSION_COMMIT_HASH="51c9778f269cedb55a4d88c79c0246d35bdadb71"
+export PYTORCH_ENABLE_MPS_FALLBACK=1
+
+####################################################################
@@ -0,0 +1,74 @@
+@echo off
+
+if not defined PYTHON (set PYTHON=python)
+if not defined VENV_DIR (set "VENV_DIR=%~dp0%venv")
+
+set ERROR_REPORTING=FALSE
+
+mkdir tmp 2>NUL
+
+%PYTHON% -c "" >tmp/stdout.txt 2>tmp/stderr.txt
+if %ERRORLEVEL% == 0 goto :start_venv
+echo Couldn't launch python
+goto :show_stdout_stderr
+
+:start_venv
+if ["%VENV_DIR%"] == ["-"] goto :skip_venv
+
+dir "%VENV_DIR%\Scripts\Python.exe" >tmp/stdout.txt 2>tmp/stderr.txt
+if %ERRORLEVEL% == 0 goto :activate_venv
+
+for /f "delims=" %%i in ('CALL %PYTHON% -c "import sys; print(sys.executable)"') do set PYTHON_FULLNAME="%%i"
+echo Creating venv in directory %VENV_DIR% using python %PYTHON_FULLNAME%
+%PYTHON_FULLNAME% -m venv "%VENV_DIR%" >tmp/stdout.txt 2>tmp/stderr.txt
+if %ERRORLEVEL% == 0 goto :activate_venv
+echo Unable to create venv in directory "%VENV_DIR%"
+goto :show_stdout_stderr
+
+:activate_venv
+set PYTHON="%VENV_DIR%\Scripts\Python.exe"
+echo venv %PYTHON%
+if [%ACCELERATE%] == ["True"] goto :accelerate
+goto :launch
+
+:skip_venv
+
+:accelerate
+echo "Checking for accelerate"
+set ACCELERATE="%VENV_DIR%\Scripts\accelerate.exe"
+if EXIST %ACCELERATE% goto :accelerate_launch
+
+:launch
+%PYTHON% launch.py %*
+pause
+exit /b
+
+:accelerate_launch
+echo "Accelerating"
+%ACCELERATE% launch --num_cpu_threads_per_process=6 launch.py
+pause
+exit /b
+
+:show_stdout_stderr
+
+echo.
+echo exit code: %errorlevel%
+
+for /f %%i in ("tmp\stdout.txt") do set size=%%~zi
+if %size% equ 0 goto :show_stderr
+echo.
+echo stdout:
+type tmp\stdout.txt
+
+:show_stderr
+for /f %%i in ("tmp\stderr.txt") do set size=%%~zi
+if %size% equ 0 goto :show_stderr
+echo.
+echo stderr:
+type tmp\stderr.txt
+
+:endofscript
+
+echo.
+echo Launch unsuccessful. Exiting.
+pause