mirror of
https://github.com/vladmandic/automatic
synced 2026-09-19 09:14:35 +02:00
Vladimir Mandic
GitHub
@@ -41,6 +41,7 @@ tunableop_results*.csv
|
||||
/*.txt
|
||||
/*.mp3
|
||||
/*.lnk
|
||||
/*.swp
|
||||
!webui.bat
|
||||
!webui.sh
|
||||
!package.json
|
||||
|
||||
+5
-13
@@ -1,15 +1,6 @@
|
||||
# To use:
|
||||
#
|
||||
# pre-commit run -a
|
||||
#
|
||||
# Or:
|
||||
#
|
||||
# pre-commit install # (runs every time you commit in git)
|
||||
#
|
||||
# To update this file:
|
||||
#
|
||||
# pre-commit autoupdate
|
||||
#
|
||||
# To use: pre-commit run -a
|
||||
# Or: pre-commit install # (runs every time you commit in git)
|
||||
# To update this file: pre-commit autoupdate
|
||||
# See https://github.com/pre-commit/pre-commit
|
||||
|
||||
ci:
|
||||
@@ -19,7 +10,7 @@ ci:
|
||||
repos:
|
||||
# Standard hooks
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v5.0.0
|
||||
rev: v6.0.0
|
||||
hooks:
|
||||
- id: check-added-large-files
|
||||
- id: check-case-conflict
|
||||
@@ -35,6 +26,7 @@ repos:
|
||||
- id: check-json
|
||||
- id: check-toml
|
||||
- id: check-xml
|
||||
- id: debug-statements
|
||||
- id: end-of-file-fixer
|
||||
- id: mixed-line-ending
|
||||
- id: check-executables-have-shebangs
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
analyse-fallback-blocks=no
|
||||
clear-cache-post-run=no
|
||||
extension-pkg-allow-list=
|
||||
prefer-stubs=yes
|
||||
extension-pkg-whitelist=
|
||||
fail-on=
|
||||
fail-under=10
|
||||
@@ -44,6 +45,7 @@ ignore-paths=/usr/lib/.*$,
|
||||
pipelines/omnigen2,
|
||||
pipelines/segmoe,
|
||||
pipelines/xomni,
|
||||
pipelines/chrono,
|
||||
scripts/consistory,
|
||||
scripts/ctrlx,
|
||||
scripts/daam,
|
||||
@@ -69,11 +71,11 @@ ignore-patterns=.*test*.py$,
|
||||
.*_model_arch.py*,
|
||||
.*_model_arch_v2.py$,
|
||||
ignored-modules=
|
||||
jobs=0
|
||||
jobs=8
|
||||
limit-inference-results=100
|
||||
load-plugins=
|
||||
persistent=yes
|
||||
py-version=3.9
|
||||
persistent=no
|
||||
py-version=3.10
|
||||
recursive=no
|
||||
source-roots=
|
||||
unsafe-load-any-extension=no
|
||||
@@ -207,6 +209,8 @@ disable=abstract-method,
|
||||
unnecessary-lambda-assigment,
|
||||
unnecessary-lambda,
|
||||
unused-wildcard-import,
|
||||
unpacking-non-sequence,
|
||||
unsubscriptable-object,
|
||||
useless-return,
|
||||
use-dict-literal,
|
||||
use-symbolic-message-instead,
|
||||
|
||||
@@ -24,6 +24,7 @@ exclude = [
|
||||
"pipelines/hdm",
|
||||
"pipelines/segmoe",
|
||||
"pipelines/xomni",
|
||||
"pipelines/chrono",
|
||||
|
||||
"scripts/lbm",
|
||||
"scripts/daam",
|
||||
|
||||
+109
@@ -1,5 +1,114 @@
|
||||
# Change Log for SD.Next
|
||||
|
||||
## Update for 2025-10-31
|
||||
|
||||
### Highlights for 2025-10-31
|
||||
|
||||
Less than 2 weeks since last release, here's a service-pack style update with a lot of fixes and improvements:
|
||||
- Reorganization of **Reference Models** into *Base, Quantized, Distilled and Community* sections for easier navigation
|
||||
and introduction of optimized **pre-quantized** variants for many popular models - use this as your quick start!
|
||||
- New models:
|
||||
**HunyuanImage 2.1** capable of 2K images natively, **HunyuanImage 3.0** large unified multimodal autoregressive model,
|
||||
**ChronoEdit** that re-purposes temporal consistency of generation for image editing
|
||||
**Pony 7** based on AuraFlow architecture, **Kandinsky 5** 10s video models
|
||||
- New **offline mode** to use previously downloaded models without internet connection
|
||||
- Optimizations to **WAN-2.2** given its popularity
|
||||
plus addition of native **VAE Upscaler** and optimized **pre-quantized** variants
|
||||
- New SOTA model loader using **Run:ai streamer**
|
||||
- Updates to `rocm` and `xpu` backends
|
||||
- Fixes, fixes, fixes... too many to list here!
|
||||
|
||||
|
||||
|
||||
[ReadMe](https://github.com/vladmandic/automatic/blob/master/README.md) | [ChangeLog](https://github.com/vladmandic/automatic/blob/master/CHANGELOG.md) | [Docs](https://vladmandic.github.io/sdnext-docs/) | [WiKi](https://github.com/vladmandic/automatic/wiki) | [Discord](https://discord.com/invite/sd-next-federal-batch-inspectors-1101998836328697867) | [Sponsor](https://github.com/sponsors/vladmandic)
|
||||
|
||||
### Details for 2025-10-31
|
||||
|
||||
- **Reference** networks section is now split into actual *Base* models plus:
|
||||
- **Quantized**: pre-quantized variants of the base models using SDNQ-SVD quantization for optimal quality and smallest possible resource usage
|
||||
examples: *FLUX.1-Dev/Krea/Kontext/Schnell, Qwen-Image/Edit/2509, Chroma1-HD, WAN-2.2-A44B, etc.*
|
||||
*note*: pre-quantized *WAN-2.2-14B* is also available in video models and runs with only 12GB VRAM!
|
||||
- **Distilled**: distilled variants of base models
|
||||
examples: *Turbo, Lightning, Lite, SRPO, Distill, Pruning, etc.*
|
||||
- **Community**: community highlights
|
||||
examples: *Tempest, Juggernaut, Illustrious, Pony, NoobAI, etc.*
|
||||
and all reference models have new preview images, thanks @liutyi
|
||||
- **Models Reference**
|
||||
- [Tencent HunyuanImage 2.1](https://huggingface.co/tencent/HunyuanImage-2.1) in *full*, *distilled* and *refiner* variants
|
||||
*HunyuanImage-2.1* is a large (51GB) T2I model capable of natively generating 2K images and uses Qwen2.5 + T5 text-encoders and 32x VAE
|
||||
- [Tencent HunyuanImage 3.0](https://huggingface.co/tencent/HunyuanImage-3.0) in [pre-quant](https://huggingface.co/Disty0/HunyuanImage3-SDNQ-uint4-svd-r32) only variant due to massive size
|
||||
*HunyuanImage 3.0* is very large at 47GB pre-quantized (oherwise its 157GB) that unifies multimodal understanding and generation within an autoregressive framework
|
||||
- [nVidia ChronoEdit](https://huggingface.co/nvidia/ChronoEdit-14B-Diffusers)
|
||||
*ChronoEdit* is a 14B image editing model based on *WAN*
|
||||
this model reframes image editing as a video generation task, using input and edited images as start/end frames to leverage pretrained video models with temporal consistency
|
||||
to extend temporal consistency for image editing, set *settings -> model options -> chrono temporal steps* to desired number of temporaly reasoning steps
|
||||
- [Kandinsky 5 Lite 10s](https://huggingface.co/ai-forever/Kandinsky-5.0-T2V-Lite-sft-10s-Diffusers') in *SFT, CFG-distilled and Steps-distilled* variants
|
||||
second series of models in *Kandinsky5* series is T2V model optimized for 10sec videos and uses Qwen2.5 text encoder
|
||||
- [Pony 7](https://huggingface.co/purplesmartai/pony-v7-base)
|
||||
Pony 7 steps in a different direction from previous Pony models and is based on AuraFlow architecture and UMT5 encoder
|
||||
- **Models Auxiliary**
|
||||
- [Qwen 3-VL](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct) VLM for interrogate and prompt enhance, thanks @CalamitousFelicitousness
|
||||
this includes *2B, 4B and 8B* variants
|
||||
- [WAN Asymettric Upscale](https://huggingface.co/spacepxl/Wan2.1-VAE-upscale2x)
|
||||
available as general purpose upscaler that can be used during standard workflow or process tab
|
||||
available as VAE for compatible video models: *WAN-2.x-14B, SkyReels-v2* models
|
||||
- [Apple DepthPro](https://huggingface.co/apple/DepthPro) controlnet processor, thanks @nolbert82
|
||||
- [LibreFlux controlnet](https://huggingface.co/neuralvfx/LibreFlux-ControlNet) segmentation controlnet for FLUX.1
|
||||
- **Features**
|
||||
- **offline mode**: enable in *settings -> hugginface*
|
||||
enables fully offline mode where previously downloaded models can be used as-is
|
||||
*note*: must be enabled only after all packages have been installed and model has been run online at least once
|
||||
- **model load**: SOTA method using nVidia's [Run:ai streamer](https://github.com/run-ai/runai-model-streamer)
|
||||
enable in *settings -> model options -> runai streamer*
|
||||
applies to *diffusers, transformers and sdnq* loaders, note this is linux-only feature
|
||||
*experimental* but shows significant model load speedups, 20-40% depending on model and hardware
|
||||
- **Backend**
|
||||
- switch to `torch==2.9` for *ipex, rocm and openvino*
|
||||
- switch to `rocm==7.0` for nightlies
|
||||
- log `triton` availability on startup
|
||||
- add `xpu` stats in gpu monitor
|
||||
- **Other**
|
||||
- improved **SDNQ SVD** and low-bit matmul performance
|
||||
- reduce RAM usage on model load using **SDNQ SVD**
|
||||
- change default **schedulers** for sdxl
|
||||
- warn on `python==3.9` end-of-life and `python==3.10` not actively supported
|
||||
- **scheduler** add base and max shift parameters for flow-matching samplers
|
||||
- enhance `--optional` flag to pre-install optional packages
|
||||
- add `[lora]` to recognized filename patterns
|
||||
- when using **shared-t5** *(default)*, it will load standard or pre-quant depending on model
|
||||
- enhanced LoRA support for **Wan-2.2-14B**
|
||||
- log available attention mechanisms on startup
|
||||
- support for switching back-and-forth **t2i** and **t2v** for *wan-2.x* models
|
||||
- control `api` cache controlnets
|
||||
- additional model modules **deduplication** for both normal and pre-quant models: *umt5, qwen25-vl*
|
||||
- **Fixes**
|
||||
- startup error with `--profile` enabled if using `--skip`
|
||||
- restore orig init image for each batch sequence
|
||||
- fix modernui hints layout
|
||||
- fix `wan-2.2-a14b` stage selection
|
||||
- fix `wan-2.2-5b` vae decode
|
||||
- disabling live preview should not disable progress updates
|
||||
- video tab create `params.txt` with metadata
|
||||
- fix full-screen image-viewer toolbar actions with control tab
|
||||
- improve filename sanitization
|
||||
- lora auto-detect low/high stage if not specified
|
||||
- lora disable fuse on partially applied network
|
||||
- fix networks display with extended characters, thanks @awsr
|
||||
- installer handle different `opencv` package variants
|
||||
- fix using pre-quantized shared-t5
|
||||
- fix `wan-2.2-14b-vace` single-stage exectution
|
||||
- fix `wan-2.2-5b` tiled vae decode
|
||||
- fix `controlnet` loading with quantization
|
||||
- video use pre-quantized text-encoder if selected model is pre-quantized
|
||||
- handle sparse `controlnet` models
|
||||
- catch `xet` warnings
|
||||
- avoid unnecessary pipe variant switching
|
||||
- validate pipelines on import
|
||||
- fix `nudenet` process tab operations
|
||||
- `controlnet` input validation
|
||||
- log metadata keys that cannot be applied
|
||||
- fix `framepack` with image input
|
||||
|
||||
## Update for 2025-10-18
|
||||
|
||||
- **Models**
|
||||
|
||||
@@ -1,71 +1,85 @@
|
||||
# TODO
|
||||
|
||||
Main ToDo list can be found at [GitHub projects](https://github.com/users/vladmandic/projects)
|
||||
## Project Board
|
||||
|
||||
## Future Candidates
|
||||
- <https://github.com/users/vladmandic/projects>
|
||||
|
||||
- Transformers unified cache handler
|
||||
- Remote TE
|
||||
- [Canvas](https://konvajs.org/)
|
||||
## Internal
|
||||
|
||||
- UI: New inpaint/outpaint interface
|
||||
[Kanvas](https://github.com/vladmandic/kanvas)
|
||||
- Deploy: Create executable for SD.Next
|
||||
- Feature: Integrate natural language imagesearch
|
||||
[ImageDB](https://github.com/vladmandic/imagedb)
|
||||
- Feature: Transformers unified cache handler
|
||||
- Feature: Remote Text-Encoder support
|
||||
- Refactor: [Modular pipelines and guiders](https://github.com/huggingface/diffusers/issues/11915)
|
||||
- Refactor: Sampler options
|
||||
- Refactor: move sampler options to settings to config
|
||||
- Refactor: [GGUF](https://huggingface.co/docs/diffusers/main/en/quantization/gguf)
|
||||
- Feature: LoRA add OMI format support for SD35/FLUX.1
|
||||
- Video Core: API
|
||||
- Video LTX: TeaCache and others, API, Conditioning preprocess Video: LTX API
|
||||
|
||||
### Under Consideration
|
||||
|
||||
- [Inf-DiT](https://github.com/zai-org/Inf-DiT)
|
||||
- [DiffSynth Studio](https://github.com/modelscope/DiffSynth-Studio)
|
||||
- [IPAdapter negative guidance](https://github.com/huggingface/diffusers/discussions/7167)
|
||||
- [IPAdapter composition](https://huggingface.co/ostris/ip-composition-adapter)
|
||||
- [STG](https://github.com/huggingface/diffusers/blob/main/examples/community/README.md#spatiotemporal-skip-guidance)
|
||||
- [SmoothCache](https://github.com/huggingface/diffusers/issues/11135)
|
||||
- [MagCache](https://github.com/lllyasviel/FramePack/pull/673/files)
|
||||
- [Dream0 guidance](https://huggingface.co/ByteDance/DreamO)
|
||||
- [ByteDance OneReward](https://github.com/bytedance/OneReward)
|
||||
- [ByteDance USO](https://github.com/bytedance/USO)
|
||||
- Remove: `CodeFormer`
|
||||
- Remove: `GFPGAN`
|
||||
- ModernUI: Lite vs Expert mode
|
||||
- Refactor: remove `CodeFormer`
|
||||
- Refactor: remove `GFPGAN`
|
||||
- UI: Lite vs Expert mode
|
||||
- Video tab: add full API support
|
||||
- Control tab: add overrides handling
|
||||
- Engine: TensorRT acceleration
|
||||
|
||||
### New models
|
||||
## Features
|
||||
|
||||
- [Ovi](https://github.com/character-ai/Ovi)
|
||||
- [Kandinsky 5](https://github.com/ai-forever/Kandinsky-5)
|
||||
- [IPAdapter composition](https://huggingface.co/ostris/ip-composition-adapter)
|
||||
- [IPAdapter negative guidance](https://github.com/huggingface/diffusers/discussions/7167)
|
||||
- [MagCache](https://github.com/lllyasviel/FramePack/pull/673/files)
|
||||
- [SmoothCache](https://github.com/huggingface/diffusers/issues/11135)
|
||||
- [STG](https://github.com/huggingface/diffusers/blob/main/examples/community/README.md#spatiotemporal-skip-guidance)
|
||||
- [Video Inpaint Pipeline](https://github.com/huggingface/diffusers/pull/12506)
|
||||
|
||||
### New models / Pipelines
|
||||
|
||||
TODO: *Prioritize*!
|
||||
|
||||
- [Bria FIBO](https://huggingface.co/briaai/FIBO)
|
||||
- [Bytedance Lynx](https://github.com/bytedance/lynx)
|
||||
- [HunyuanImage](https://huggingface.co/tencent/HunyuanImage-2.1)
|
||||
- [Phantom HuMo](https://github.com/Phantom-video/Phantom)
|
||||
- [Lumina-DiMOO](https://huggingface.co/Alpha-VLLM/Lumina-DiMOO)
|
||||
- [Wan2.2-Animate-14B](https://huggingface.co/Wan-AI/Wan2.2-Animate-14B)
|
||||
- [Magi](https://github.com/SandAI-org/MAGI-1)(https://github.com/huggingface/diffusers/pull/11713)
|
||||
- [SEVA](https://github.com/huggingface/diffusers/pull/11440)
|
||||
- [Ming](https://github.com/inclusionAI/Ming)
|
||||
- [Liquid](https://github.com/FoundationVision/Liquid)
|
||||
- [Step1X](https://github.com/stepfun-ai/Step1X-Edit)
|
||||
- [LucyEdit](https://github.com/huggingface/diffusers/pull/12340)
|
||||
- [SD3 UltraEdit](https://github.com/HaozheZhao/UltraEdit)
|
||||
- [WAN2GP](https://github.com/deepbeepmeep/Wan2GP)
|
||||
- [SelfForcing](https://github.com/guandeh17/Self-Forcing)
|
||||
- [ByteDance OneReward](https://github.com/bytedance/OneReward)
|
||||
- [ByteDance USO](https://github.com/bytedance/USO)
|
||||
- [Chroma1 Radiance](https://huggingface.co/lodestones/Chroma1-Radiance)
|
||||
- [DiffSynth Studio](https://github.com/modelscope/DiffSynth-Studio)
|
||||
- [DiffusionForcing](https://github.com/kwsong0113/diffusion-forcing-transformer)
|
||||
- [LanDiff](https://github.com/landiff/landiff)
|
||||
- [HunyuanCustom](https://github.com/Tencent-Hunyuan/HunyuanCustom)
|
||||
- [Dream0 guidance](https://huggingface.co/ByteDance/DreamO)
|
||||
- [HunyuanAvatar](https://huggingface.co/tencent/HunyuanVideo-Avatar)
|
||||
- [WAN-CausVid](https://huggingface.co/lightx2v/Wan2.1-T2V-14B-CausVid)
|
||||
- [HunyuanCustom](https://github.com/Tencent-Hunyuan/HunyuanCustom)
|
||||
- [Inf-DiT](https://github.com/zai-org/Inf-DiT)
|
||||
- [Krea Realtime Video](https://huggingface.co/krea/krea-realtime-video)
|
||||
- [LanDiff](https://github.com/landiff/landiff)
|
||||
- [Liquid](https://github.com/FoundationVision/Liquid)
|
||||
- [LongCat-Video](https://huggingface.co/meituan-longcat/LongCat-Video)
|
||||
- [LucyEdit](https://github.com/huggingface/diffusers/pull/12340)
|
||||
- [Lumina-DiMOO](https://huggingface.co/Alpha-VLLM/Lumina-DiMOO)
|
||||
- [Magi](https://github.com/SandAI-org/MAGI-1)(https://github.com/huggingface/diffusers/pull/11713)
|
||||
- [Ming](https://github.com/inclusionAI/Ming)
|
||||
- [MUG-V 10B](https://huggingface.co/MUG-V/MUG-V-inference)
|
||||
- [Ovi](https://github.com/character-ai/Ovi)
|
||||
- [Phantom HuMo](https://github.com/Phantom-video/Phantom)
|
||||
- [SD3 UltraEdit](https://github.com/HaozheZhao/UltraEdit)
|
||||
- [SelfForcing](https://github.com/guandeh17/Self-Forcing)
|
||||
- [SEVA](https://github.com/huggingface/diffusers/pull/11440)
|
||||
- [Step1X](https://github.com/stepfun-ai/Step1X-Edit)
|
||||
- [Wan-2.2 Animate](https://github.com/huggingface/diffusers/pull/12526)
|
||||
- [Wan-2.2 S2V](https://github.com/huggingface/diffusers/pull/12258)
|
||||
- [WAN-CausVid-Plus t2v](https://github.com/goatWu/CausVid-Plus/)
|
||||
- [WAN-CausVid](https://huggingface.co/lightx2v/Wan2.1-T2V-14B-CausVid)
|
||||
- [WAN-StepDistill](https://huggingface.co/lightx2v/Wan2.1-T2V-14B-StepDistill-CfgDistill)
|
||||
- [Wan2.2-Animate-14B](https://huggingface.co/Wan-AI/Wan2.2-Animate-14B)
|
||||
- [WAN2GP](https://github.com/deepbeepmeep/Wan2GP)
|
||||
|
||||
## Code TODO
|
||||
|
||||
> pnpm lint | grep W0511 | awk -F'TODO ' '{print "- "$NF}' | sed 's/ (fixme)//g' | sort
|
||||
> npm run todo
|
||||
|
||||
- control: support scripts via api
|
||||
- fc: autodetect distilled based on model
|
||||
- fc: autodetect tensor format based on model
|
||||
- hypertile: vae breaks when using non-standard sizes
|
||||
- install: enable ROCm for windows when available
|
||||
- install: switch to pytorch source when it becomes available
|
||||
- loader: load receipe
|
||||
- loader: save receipe
|
||||
- lora: add other quantization types
|
||||
@@ -75,5 +89,6 @@ Main ToDo list can be found at [GitHub projects](https://github.com/users/vladma
|
||||
- model load: implement model in-memory caching
|
||||
- modernui: monkey-patch for missing tabs.select event
|
||||
- modules/lora/lora_extract.py:188:9: W0511: TODO: lora: support pre-quantized flux
|
||||
- modules/modular_guiders.py:65:58: W0511: TODO: guiders
|
||||
- processing: remove duplicate mask params
|
||||
- resize image: enable full VAE mode for resize-latent
|
||||
|
||||
+26
-4
@@ -1,4 +1,5 @@
|
||||
#!/usr/bin/env python
|
||||
# example: api-control.py --prompt "anime girl" --control "Canny:Canny:1.0:0.1:0.9:/home/vlado/generative/Samples/anime1.jpg,None:Depth:0.9:0.0:1.0:/home/vlado/generative/Samples/anime1.jpg" --hires --detailer --output /tmp/anime.jpg
|
||||
import os
|
||||
import io
|
||||
import time
|
||||
@@ -70,10 +71,12 @@ def generate(args): # pylint: disable=redefined-outer-name
|
||||
options['negative_prompt'] = args.negative
|
||||
options['steps'] = int(args.steps)
|
||||
options['seed'] = int(args.seed)
|
||||
options['sampler_name'] = args.sampler
|
||||
if args.sampler is not None:
|
||||
options['sampler_name'] = args.sampler
|
||||
|
||||
if args.control is not None:
|
||||
options['unit_type'] = args.type
|
||||
if args.type is not None:
|
||||
options['unit_type'] = args.type
|
||||
options['control'] = []
|
||||
for control in args.control.split(','):
|
||||
u = control.split(':')
|
||||
@@ -86,7 +89,9 @@ def generate(args): # pylint: disable=redefined-outer-name
|
||||
'strength': float(u[2].strip()) if len(u) > 2 else 1.0,
|
||||
'start': float(u[3].strip()) if len(u) > 3 else 0.0,
|
||||
'end': float(u[4].strip()) if len(u) > 4 else 1.0,
|
||||
'override': encode(u[5].strip()) if len(u) > 5 else None,
|
||||
})
|
||||
log.info(f'added control: {options["control"]}')
|
||||
|
||||
if args.ipadapter is not None:
|
||||
options['ip_adapter'] = []
|
||||
@@ -109,6 +114,20 @@ def generate(args): # pylint: disable=redefined-outer-name
|
||||
if args.mask is not None:
|
||||
options['mask'] = encode(args.mask)
|
||||
|
||||
if args.detailer:
|
||||
options['detailer_enabled'] = True
|
||||
|
||||
if args.hires:
|
||||
options['enable_hr'] = True
|
||||
options['hr_force'] = True
|
||||
|
||||
if args.upscaler is not None:
|
||||
options['enable_hr'] = True
|
||||
options['hr_force'] = True
|
||||
options['hr_scale'] = 2
|
||||
options['hr_resize_mode'] = 1
|
||||
options['hr_upscaler'] = args.upscaler
|
||||
|
||||
data = post('/sdapi/v1/control', options)
|
||||
t1 = time.time()
|
||||
if 'info' in data:
|
||||
@@ -141,13 +160,16 @@ if __name__ == "__main__":
|
||||
parser.add_argument('--negative', required=False, default='', help='negative prompt text')
|
||||
parser.add_argument('--steps', required=False, default=20, help='number of steps')
|
||||
parser.add_argument('--seed', required=False, default=-1, help='initial seed')
|
||||
parser.add_argument('--sampler', required=False, default='UniPC', help='sampler name')
|
||||
parser.add_argument('--sampler', required=False, default=None, help='sampler name')
|
||||
parser.add_argument('--output', required=False, default=None, help='output image file')
|
||||
parser.add_argument('--processed', required=False, default=None, help='processed output file')
|
||||
parser.add_argument('--model', required=False, help='model name')
|
||||
parser.add_argument('--type', required=False, help='control type')
|
||||
parser.add_argument('--type', required=False, default="controlnet", help='control type')
|
||||
parser.add_argument('--control', required=False, help='control units')
|
||||
parser.add_argument('--ipadapter', required=False, help='ipadapter units')
|
||||
parser.add_argument('--detailer', required=False, default=False, action='store_true', help='force detailer')
|
||||
parser.add_argument('--hires', required=False, default=False, action='store_true', help='force hires')
|
||||
parser.add_argument('--upscaler', required=False, default=None, help='upscaler name')
|
||||
args = parser.parse_args()
|
||||
log.info(f'api-control: {args}')
|
||||
generate(args)
|
||||
|
||||
+8
-8
@@ -7,12 +7,12 @@ from rich import print # pylint: disable=redefined-builtin
|
||||
if __name__ == "__main__":
|
||||
sys.argv.pop(0)
|
||||
keyword = sys.argv[0] if len(sys.argv) > 0 else ''
|
||||
hf.logging.set_verbosity_info()
|
||||
hf_api = hf.HfApi()
|
||||
model_filter = hf.ModelFilter(
|
||||
model_name=keyword,
|
||||
# task='text-to-image',
|
||||
library=['diffusers'],
|
||||
)
|
||||
res = hf_api.list_models(filter=model_filter, full=True, limit=50, sort="downloads", direction=-1)
|
||||
models = [{ 'name': m.id, 'downloads': m.downloads, 'mtime': m.lastModified, 'url': f'https://huggingface.co/{m.id}', 'pipeline': m.pipeline_tag, 'tags': m.tags } for m in res]
|
||||
print(models)
|
||||
res = hf_api.list_models(model_name=keyword, full=True, limit=100, sort="downloads", direction=-1)
|
||||
res = sorted(res, key=lambda x: x.id)
|
||||
for m in res:
|
||||
meta = hf_api.model_info(m.id, files_metadata=True)
|
||||
m.files = [f.rfilename for f in meta.siblings if f.rfilename.endswith('.bin') or f.rfilename.endswith('.safetensors')]
|
||||
m.size = sum([f.size for f in meta.siblings]) / 1024 / 1024 / 1024 # in GB
|
||||
print({ 'name': m.id, 'files': len(m.files), 'size': m.size, 'downloads': m.downloads, 'mtime': m.lastModified, 'url': f'https://huggingface.co/{m.id}', 'pipeline': m.pipeline_tag })
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"_class_name": "EulerDiscreteScheduler",
|
||||
"_diffusers_version": "0.19.0.dev0",
|
||||
"_class_name": "EulerAncestralDiscreteScheduler",
|
||||
"_diffusers_version": "0.35.1",
|
||||
"beta_end": 0.012,
|
||||
"beta_schedule": "scaled_linear",
|
||||
"beta_start": 0.00085,
|
||||
@@ -8,11 +8,12 @@
|
||||
"interpolation_type": "linear",
|
||||
"num_train_timesteps": 1000,
|
||||
"prediction_type": "epsilon",
|
||||
"rescale_betas_zero_snr": false,
|
||||
"sample_max_value": 1.0,
|
||||
"set_alpha_to_one": false,
|
||||
"skip_prk_steps": true,
|
||||
"steps_offset": 1,
|
||||
"timestep_spacing": "leading",
|
||||
"timestep_spacing": "trailing",
|
||||
"trained_betas": null,
|
||||
"use_karras_sigmas": false
|
||||
}
|
||||
|
||||
Submodule extensions-builtin/sdnext-modernui updated: 8f6427aa03...eea7fc5d94
+10
-9
@@ -197,13 +197,16 @@
|
||||
{"id":"","label":"Init Image Same As Control","localized":"","reload":"","hint":"Will additionally treat any image placed into the Control input window as a source for img2img type tasks, an image to modify for example."},
|
||||
{"id":"","label":"Separate Init Image","localized":"","reload":"","hint":"Creates an additional window next to Control input labeled Init input, so you can have a separate image for both Control operations and an init source."},
|
||||
{"id":"","label":"Override settings","localized":"","reload":"","hint":"If generation parameters deviate from your system settings override settings populated with those settings to override your system configuration for this workflow"},
|
||||
{"id":"","label":"sigma method","localized":"","reload":"","hint":"Controls how noise levels (sigmas) are distributed across diffusion steps. Options:\n- default: the model default\n- karras: smoother noise schedule, higher quality with fewer steps\n- beta: based on beta schedule values\n- exponential: exponential decay of noise\n- lambdas: experimental, balances signal-to-noise\n- flowmatch: tuned for flow-matching models"},
|
||||
{"id":"","label":"timestep spacing","localized":"","reload":"","hint":"Determines how timesteps are spaced across the diffusion process. Options:\n- default: the model default\n- leading: creates evenly spaced steps\n- linspace: includes the first and last steps and evenly selects the remaining intermediate steps\n- trailing: only includes the last step and evenly selects the remaining intermediate steps starting from the end"},
|
||||
{"id":"","label":"beta schedule","localized":"","reload":"","hint":"Defines how beta (noise strength per step) grows. Options:\n- default: the model default\n- linear: evenly decays noise per step\n- scaled: squared version of linear, used only by Stable Diffusion\n- cosine: smoother decay, often better results with fewer steps\n- sigmoid: sharp transition, experimental"},
|
||||
{"id":"","label":"prediction method","localized":"","reload":"","hint":"Defines what the model predicts at each step. Options:\n- default: the model default\n- epsilon: noise (most common for Stable Diffusion)\n- sample: direct denoised image prediction, also called as x0 prediction\n- v_prediction: velocity prediction, used by CosXL and NoobAI VPred models\n- flow_prediction: used with newer flow-matching models like SD3 and Flux"},
|
||||
{"id":"","label":"sigma method","localized":"","reload":"","hint":"Controls how noise levels (sigmas) are distributed across diffusion steps. Options:<br>- default: the model default<br>- karras: smoother noise schedule, higher quality with fewer steps<br>- beta: based on beta schedule values<br>- exponential: exponential decay of noise<br>- lambdas: experimental, balances signal-to-noise<br>- flowmatch: tuned for flow-matching models"},
|
||||
{"id":"","label":"timestep spacing","localized":"","reload":"","hint":"Determines how timesteps are spaced across the diffusion process. Options:<br>- default: the model default<br>- leading: creates evenly spaced steps<br>- linspace: includes the first and last steps and evenly selects the remaining intermediate steps<br>- trailing: only includes the last step and evenly selects the remaining intermediate steps starting from the end"},
|
||||
{"id":"","label":"beta schedule","localized":"","reload":"","hint":"Defines how beta (noise strength per step) grows. Options:<br>- default: the model default<br>- linear: evenly decays noise per step<br>- scaled: squared version of linear, used only by Stable Diffusion<br>- cosine: smoother decay, often better results with fewer steps<br>- sigmoid: sharp transition, experimental"},
|
||||
{"id":"","label":"prediction method","localized":"","reload":"","hint":"Defines what the model predicts at each step. Options:<br>- default: the model default<br>- epsilon: noise (most common for Stable Diffusion)<br>- sample: direct denoised image prediction, also called as x0 prediction<br>- v_prediction: velocity prediction, used by CosXL and NoobAI VPred models<br>- flow_prediction: used with newer flow-matching models like SD3 and Flux"},
|
||||
{"id":"","label":"sampler order","localized":"","reload":"","hint":"Order of solver updates in the sampler. Higher order improves stability/accuracy but increases compute cost."},
|
||||
{"id":"","label":"flow shift","localized":"","reload":"","hint":"Adjustment for flow-based samplers. Shifts noise distribution during generation, useful for fine-tuning balance between detail and consistency."},
|
||||
{"id":"","label":"resize mode","localized":"","reload":"","hint":"Defines how the input is resized or adapted in second-pass refinement:\n- none: no resizing, keep original resolution\n- fixed: force resize to target resolution (may distort)\n- crop: center-crop to fit target while keeping aspect ratio\n- fill: resize to fit and pad empty space with borders\n- outpaint: extend canvas beyond image borders\n- context aware: smart resize that blends or adapts surrounding areas"}
|
||||
{"id":"","label":"flow shift","localized":"","reload":"","hint":"Shift value for flowmatching models. Controls the distribution of denoising steps.<br><br>Values:<br>- >1.0: allocate more steps to early denoising (better structure)<br>-<1.0: allocate more steps to late denoising (better fine details)<br>- 1.0: balanced schedule<br><br>Most flowmatching models use the value of 3 as default. Effectively inactive if dynamic shift is enabled."},
|
||||
{"id":"","label":"dynamic","localized":"","reload":"","hint":"Dynamic shifting automatically adjusts the denoising schedule based on your image resolution.<br><br>The scheduler interpolates between base_shift and max_shift based on actual image resolution.<br><br>Enabling disables static Flow shift."},
|
||||
{"id":"","label":"base shift","localized":"","reload":"","hint":"Minimum shift value for low resolutions when using dynamic shifting."},
|
||||
{"id":"","label":"max shift","localized":"","reload":"","hint":"Maximum shift value for high resolutions when using dynamic shifting."},
|
||||
{"id":"","label":"resize mode","localized":"","reload":"","hint":"Defines how the input is resized or adapted in second-pass refinement:<br>- none: no resizing, keep original resolution<br>- fixed: force resize to target resolution (may distort)<br>- crop: center-crop to fit target while keeping aspect ratio<br>- fill: resize to fit and pad empty space with borders<br>- outpaint: extend canvas beyond image borders<br>- context aware: smart resize that blends or adapts surrounding areas"}
|
||||
],
|
||||
"other": [
|
||||
{"id":"","label":"Install","localized":"","reload":"","hint":"Install"},
|
||||
@@ -277,7 +280,7 @@
|
||||
{"id":"","label":"Final denoising strength","localized":"","reload":"","hint":"The denoising strength for the final loop of each image in the batch"},
|
||||
{"id":"","label":"Denoising strength curve","localized":"","reload":"","hint":"The denoising curve controls the rate of denoising strength change each loop. Aggressive: Most of the change will happen towards the start of the loops. Linear: Change will be constant through all loops. Lazy: Most of the change will happen towards the end of the loops"},
|
||||
{"id":"","label":"Tile overlap","localized":"","reload":"","hint":"For SD upscale, how much overlap in pixels should there be between tiles. Tiles overlap so that when they are merged back into one picture, there is no clearly visible seam"},
|
||||
{"id":"","label":"ACI: Color to Mask","localized":"","reload":"","hint":"Pick the color you want to mask and inpaint. Click on the color in the image to automatically select it.\n Advised to use images like green screens to get precise results."},
|
||||
{"id":"","label":"ACI: Color to Mask","localized":"","reload":"","hint":"Pick the color you want to mask and inpaint. Click on the color in the image to automatically select it.<br> Advised to use images like green screens to get precise results."},
|
||||
{"id":"","label":"ACI: Color Tolerance","localized":"","reload":"","hint":"Adjust the tolerance to include similar colors in the mask. Lower values = mask only very similar colors. Higher = values mask a wider range of similar colors."},
|
||||
{"id":"","label":"ACI: Mask Erode","localized":"","reload":"","hint":"Adjust padding to apply a inside offset to the mask. (Recommended value = 2 to remove leftovers at edges)"},
|
||||
{"id":"","label":"ACI: Mask Blur","localized":"","reload":"","hint":"Adjust blur to apply a smooth transition between image and inpainted area. (Recommended value = 0 for sharpness)"},
|
||||
@@ -651,7 +654,6 @@
|
||||
{"id":"","label":"dropdown","localized":"","reload":"","hint":"dropdown"},
|
||||
{"id":"","label":"duration","localized":"","reload":"","hint":"duration"},
|
||||
{"id":"","label":"dwpose","localized":"","reload":"","hint":"dwpose"},
|
||||
{"id":"","label":"dynamic","localized":"","reload":"","hint":"dynamic"},
|
||||
{"id":"","label":"dynamic attention slicing rate in gb","localized":"","reload":"","hint":"dynamic attention slicing rate in gb"},
|
||||
{"id":"","label":"dynamic attention trigger rate in gb","localized":"","reload":"","hint":"dynamic attention trigger rate in gb"},
|
||||
{"id":"","label":"edge","localized":"","reload":"","hint":"edge"},
|
||||
@@ -1070,7 +1072,6 @@
|
||||
{"id":"","label":"same latent","localized":"","reload":"","hint":"same latent"},
|
||||
{"id":"","label":"sample","localized":"","reload":"","hint":"sample"},
|
||||
{"id":"","label":"sampler","localized":"","reload":"","hint":"sampler"},
|
||||
{"id":"","label":"sampler dynamic shift","localized":"","reload":"","hint":"sampler dynamic shift"},
|
||||
{"id":"","label":"sampler shift","localized":"","reload":"","hint":"sampler shift"},
|
||||
{"id":"","label":"sana: use complex human instructions","localized":"","reload":"","hint":"sana: use complex human instructions"},
|
||||
{"id":"","label":"saturation","localized":"","reload":"","hint":"saturation"},
|
||||
|
||||
+335
-54
@@ -1,37 +1,4 @@
|
||||
{
|
||||
"Tempest-by-Vlad XL": {
|
||||
"path": "tempestByVlad_baseV01.safetensors@https://civitai.com/api/download/models/1301775",
|
||||
"preview": "tempestByVlad_baseV01.jpg",
|
||||
"desc": "Flexible SDXL model with custom encoder and finetuned for larger landscape resolutions with high details and high contrast.",
|
||||
"extras": ""
|
||||
},
|
||||
"Tempest-by-Vlad XL Hyper": {
|
||||
"path": "tempestByVlad_hyperV01.safetensors@https://civitai.com/api/download/models/1343512",
|
||||
"preview": "tempestByVlad_hyperV01.jpg",
|
||||
"desc": "Custom distilled variant with goal to get as-normal-as-possible model that works with low steps and guidance-free",
|
||||
"extras": ""
|
||||
},
|
||||
|
||||
"Juggernaut XL XI": {
|
||||
"path": "juggernautXL_juggXIByRundiffusion.safetensors@https://civitai.com/api/download/models/782002",
|
||||
"preview": "juggernautXL_juggXIByRundiffusion.jpg",
|
||||
"desc": "Showcase finetuned model based on Stable diffusion XL",
|
||||
"extras": "sampler: DEIS, steps: 20, cfg_scale: 6.0"
|
||||
},
|
||||
"Juggernaut XL XI Lightning": {
|
||||
"path": "juggernautXL_juggXILightningByRD.safetensors@https://civitai.com/api/download/models/920957",
|
||||
"preview": "juggernautXL_juggXILightningByRD.jpg",
|
||||
"desc": "Showcase finetuned model based on Stable diffusion XL",
|
||||
"extras": "sampler: DPM SDE, steps: 6, cfg_scale: 2.0"
|
||||
},
|
||||
"Juggernaut SD Reborn": {
|
||||
"original": true,
|
||||
"path": "juggernaut_reborn.safetensors@https://civitai.com/api/download/models/274039",
|
||||
"preview": "juggernaut_reborn.jpg",
|
||||
"desc": "Showcase finetuned model based on Stable diffusion 1.5",
|
||||
"extras": "width: 512, height: 512, sampler: DEIS, steps: 20, cfg_scale: 6.0"
|
||||
},
|
||||
|
||||
"RunwayML StableDiffusion 1.5": {
|
||||
"original": true,
|
||||
"path": "v1-5-pruned-fp16-emaonly.safetensors@https://huggingface.co/Aptronym/SDNext/resolve/main/Reference/v1-5-pruned-fp16-emaonly.safetensors?download=true",
|
||||
@@ -56,6 +23,8 @@
|
||||
"preview": "stabilityai--stable-diffusion-2-1.jpg",
|
||||
"skip": true,
|
||||
"variant": "fp16",
|
||||
"size": 2.58,
|
||||
"date": "2022 December",
|
||||
"desc": "This stable-diffusion-2 model is resumed from stable-diffusion-2-base (512-base-ema.ckpt) and trained for 150k steps using a v-objective on the same dataset. Resumed for another 140k steps on 768x768 images",
|
||||
"extras": "width: 768, height: 768, sampler: DEIS, steps: 20, cfg_scale: 6.0"
|
||||
},
|
||||
@@ -75,6 +44,7 @@
|
||||
"desc": "SDXL-Turbo is a fast generative text-to-image model that can synthesize photorealistic images from a text prompt in a 1-4 steps.",
|
||||
"skip": true,
|
||||
"variant": "fp16",
|
||||
"tags": "distilled",
|
||||
"extras": "steps: 4, cfg_scale: 0.0"
|
||||
},
|
||||
"StabilityAI Stable Cascade": {
|
||||
@@ -95,9 +65,10 @@
|
||||
"preview": "stabilityai--stable-cascade-lite.jpg",
|
||||
"extras": "sampler: Default, cfg_scale: 4.0, image_cfg_scale: 1.0",
|
||||
"size": 4.97,
|
||||
"tags": "distilled",
|
||||
"date": "2024 February"
|
||||
},
|
||||
"StabilityAI Stable Diffusion 3 Medium": {
|
||||
"StabilityAI Stable Diffusion 3.0 Medium": {
|
||||
"path": "stabilityai/stable-diffusion-3-medium-diffusers",
|
||||
"skip": true,
|
||||
"variant": "fp16",
|
||||
@@ -133,6 +104,7 @@
|
||||
"variant": "fp16",
|
||||
"desc": "Stable Diffusion 3.5 Large Turbo is a Multimodal Diffusion Transformer (MMDiT) text-to-image model with Adversarial Diffusion Distillation (ADD) that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency, with a focus on fewer inference steps.",
|
||||
"preview": "stabilityai--stable-diffusion-3_5-large-turbo.jpg",
|
||||
"tags": "distilled",
|
||||
"extras": "sampler: Default, cfg_scale: 7.0"
|
||||
},
|
||||
|
||||
@@ -176,6 +148,7 @@
|
||||
"path": "vladmandic/flux.1-dev-SRPO",
|
||||
"preview": "vladmandic--flux.1-dev-SRPO.jpg",
|
||||
"desc": "FLUX.1 Dev SRPO is Tencent trained with specific technique: Directly Aligning the Full Diffusion Trajectory with Fine-Grained Human Preference",
|
||||
"tags": "distilled",
|
||||
"skip": true,
|
||||
"extras": "sampler: Default, cfg_scale: 4.5"
|
||||
},
|
||||
@@ -214,6 +187,7 @@
|
||||
"skip": true,
|
||||
"extras": "steps: 8",
|
||||
"size": 56.1,
|
||||
"tags": "distilled",
|
||||
"date": "2025 August"
|
||||
},
|
||||
"Qwen-Image-Distill": {
|
||||
@@ -223,6 +197,7 @@
|
||||
"skip": true,
|
||||
"extras": "steps: 15",
|
||||
"size": 56.1,
|
||||
"tags": "distilled",
|
||||
"date": "2025 August"
|
||||
},
|
||||
"Qwen-Image-Lightning-Edit": {
|
||||
@@ -232,6 +207,7 @@
|
||||
"skip": true,
|
||||
"extras": "steps: 8",
|
||||
"size": 56.1,
|
||||
"tags": "distilled",
|
||||
"date": "2025 August"
|
||||
},
|
||||
"Qwen-Image Pruning-12B": {
|
||||
@@ -240,6 +216,7 @@
|
||||
"preview": "OPPOer--Qwen-Image-Pruning.jpg",
|
||||
"desc": "This open-source project is based on Qwen-Image and has attempted model pruning, removing 20 layers while retaining the weights of 40 layers, resulting in a model size of 12B parameters.",
|
||||
"skip": true,
|
||||
"tags": "distilled",
|
||||
"date": "2025 Ocotober"
|
||||
},
|
||||
"Qwen-Image-Edit Pruning-13B": {
|
||||
@@ -248,6 +225,7 @@
|
||||
"preview": "OPPOer--Qwen-Image-Edit-Pruning.jpg",
|
||||
"desc": "This open-source project is based on Qwen-Image-Edit and has attempted model pruning, removing 20 layers while retaining the weights of 40 layers, resulting in a model size of 13.6B parameters.",
|
||||
"skip": true,
|
||||
"tags": "distilled",
|
||||
"date": "2025 Ocotober"
|
||||
},
|
||||
"Qwen-Image-Edit-2509 Pruning-13B": {
|
||||
@@ -256,6 +234,7 @@
|
||||
"preview": "OPPOer--Qwen-Image-Edit-2509-Pruning.jpg",
|
||||
"desc": "This open-source project is based on Qwen-Image-Edit and has attempted model pruning, removing 20 layers while retaining the weights of 40 layers, resulting in a model size of 13.6B parameters.",
|
||||
"skip": true,
|
||||
"tags": "distilled",
|
||||
"date": "2025 Ocotober"
|
||||
},
|
||||
|
||||
@@ -284,6 +263,7 @@
|
||||
"skip": true,
|
||||
"extras": "",
|
||||
"size": 26.84,
|
||||
"tags": "distilled",
|
||||
"date": "2025 July"
|
||||
},
|
||||
"lodestones Chroma1 v50 Preview Annealed": {
|
||||
@@ -419,6 +399,7 @@
|
||||
"preview": "SDXL-Flash_Mini.jpg",
|
||||
"desc": "Introducing the new fast model SDXL Flash (Mini), we learned that all fast XL models work fast, but the quality decreases, and we also made a fast model, but it is not as fast as LCM, Turbo, Lightning and Hyper, but the quality is higher.",
|
||||
"extras": "width: 2048, height: 1024, sampler: DEIS, steps: 40, cfg_scale: 6.0",
|
||||
"tags": "distilled",
|
||||
"experimental": true
|
||||
},
|
||||
|
||||
@@ -442,6 +423,7 @@
|
||||
"path": "Efficient-Large-Model/Sana_Sprint_1.6B_1024px_diffusers",
|
||||
"desc": "SANA-Sprint is an ultra-efficient diffusion model for text-to-image (T2I) generation, reducing inference steps from 20 to 1-4 while achieving state-of-the-art performance.",
|
||||
"preview": "Efficient-Large-Model--Sana15_Sprint_1600M_1024px_diffusers.jpg",
|
||||
"tags": "distilled",
|
||||
"skip": true
|
||||
},
|
||||
"NVLabs Sana 1.0 1.6B 4k": {
|
||||
@@ -476,6 +458,13 @@
|
||||
"size": 7.51,
|
||||
"date": "2024 November"
|
||||
},
|
||||
"nVidia ChronoEdit": {
|
||||
"path": "nvidia/ChronoEdit-14B-Diffusers",
|
||||
"preview": "nvidia--ChronoEdit-14B-Diffusers.jpg",
|
||||
"desc": "ChronoEdit reframes image editing as a video generation task, using input and edited images as start/end frames to leverage pretrained video models with temporal consistency.",
|
||||
"skip": true,
|
||||
"extras": ""
|
||||
},
|
||||
"nVidia Cosmos-Predict2 T2I 2B": {
|
||||
"path": "nvidia/Cosmos-Predict2-2B-Text2Image",
|
||||
"desc": "Cosmos-Predict2: A family of highly performant pre-trained world foundation models purpose-built for generating physics-aware images, videos and world states for physical AI development.",
|
||||
@@ -555,6 +544,7 @@
|
||||
"skip": true,
|
||||
"extras": "sampler: Default, cfg_scale: 9.0",
|
||||
"size": 8.72,
|
||||
"tags": "distilled",
|
||||
"date": "2023 October"
|
||||
},
|
||||
"Segmind Tiny": {
|
||||
@@ -563,6 +553,7 @@
|
||||
"desc": "Segmind's Tiny-SD offers a compact, efficient, and distilled version of Realistic Vision 4.0 and is up to 80% faster than SD1.5",
|
||||
"extras": "width: 512, height: 512, sampler: Default, cfg_scale: 9.0",
|
||||
"size": 1.03,
|
||||
"tags": "distilled",
|
||||
"date": "2023 July"
|
||||
},
|
||||
"Segmind SegMoE SD 4x2": {
|
||||
@@ -615,6 +606,34 @@
|
||||
"date": "2024 April"
|
||||
},
|
||||
|
||||
"Tencent HunyuanImage 2.1": {
|
||||
"path": "hunyuanvideo-community/HunyuanImage-2.1-Diffusers",
|
||||
"desc": "HunyuanImage-2.1, a highly efficient text-to-image model that is capable of generating 2K (2048 × 2048) resolution images.",
|
||||
"preview": "hunyuanvideo-community--HunyuanImage-2.1-Diffusers.jpg",
|
||||
"extras": "",
|
||||
"skip": true,
|
||||
"size": 0,
|
||||
"date": "2025 August"
|
||||
},
|
||||
"Tencent HunyuanImage 2.1 Distilled": {
|
||||
"path": "hunyuanvideo-community/HunyuanImage-2.1-Distilled-Diffusers",
|
||||
"desc": "HunyuanImage-2.1, a highly efficient text-to-image model that is capable of generating 2K (2048 × 2048) resolution images.",
|
||||
"preview": "hunyuanvideo-community--HunyuanImage-2.1-Distilled-Diffusers.jpg",
|
||||
"extras": "",
|
||||
"tags": "distilled",
|
||||
"skip": true,
|
||||
"size": 0,
|
||||
"date": "2025 August"
|
||||
},
|
||||
"Tencent HunyuanImage 2.1 Refiner": {
|
||||
"path": "hunyuanvideo-community/HunyuanImage-2.1-Refiner-Diffusers",
|
||||
"desc": "HunyuanImage-2.1, a highly efficient text-to-image model that is capable of generating 2K (2048 × 2048) resolution images.",
|
||||
"preview": "hunyuanvideo-community--HunyuanImage-2.1-Diffusers.jpg",
|
||||
"extras": "",
|
||||
"skip": true,
|
||||
"size": 0,
|
||||
"date": "2025 August"
|
||||
},
|
||||
"Tencent HunyuanDiT 1.2": {
|
||||
"path": "Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers",
|
||||
"desc": "Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding.",
|
||||
@@ -627,6 +646,7 @@
|
||||
"path": "Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers-Distilled",
|
||||
"desc": "Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding.",
|
||||
"preview": "Tencent-Hunyuan--HunyuanDiT-v1.2-Diffusers-Distilled.jpg",
|
||||
"tags": "distilled",
|
||||
"extras": "sampler: Default, cfg_scale: 2.0"
|
||||
},
|
||||
"Tencent HunyuanDiT 1.1": {
|
||||
@@ -639,6 +659,7 @@
|
||||
"path": "Tencent-Hunyuan/HunyuanDiT-v1.1-Diffusers-Distilled",
|
||||
"desc": "Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding.",
|
||||
"preview": "Tencent-Hunyuan--HunyuanDiT-v1.1-Diffusers-Distilled.jpg",
|
||||
"tags": "distilled",
|
||||
"extras": "sampler: Default, cfg_scale: 2.0"
|
||||
},
|
||||
|
||||
@@ -698,7 +719,7 @@
|
||||
"HiDream-E1.1": {
|
||||
"path": "HiDream-ai/HiDream-E1-1",
|
||||
"desc": "HiDream-E1 is an image editing model built on HiDream-I1.",
|
||||
"preview": "HiDream-ai--HiDream-E1-Full.jpg",
|
||||
"preview": "HiDream-ai--HiDream-E1-1.jpg",
|
||||
"skip": true,
|
||||
"extras": "sampler: Default"
|
||||
},
|
||||
@@ -792,25 +813,6 @@
|
||||
"date": "2024 October"
|
||||
},
|
||||
|
||||
"ShuttleAI Shuttle 3.0 Diffusion": {
|
||||
"path": "shuttleai/shuttle-3-diffusion",
|
||||
"desc": "Shuttle uses Flux.1 Schnell as its base. It can produce images similar to Flux Dev or Pro in just 4 steps, and it is licensed under Apache 2. The model was partially de-distilled during training. When used beyond 10 steps, it enters refiner mode enhancing image details without altering the composition",
|
||||
"preview": "shuttleai--shuttle-3-diffusion.jpg",
|
||||
"skip": true
|
||||
},
|
||||
"ShuttleAI Shuttle 3.1 Aesthetic": {
|
||||
"path": "shuttleai/shuttle-3.1-aesthetic",
|
||||
"desc": "Shuttle uses Flux.1 Schnell as its base. It can produce images similar to Flux Dev or Pro in just 4 steps, and it is licensed under Apache 2. The model was partially de-distilled during training. When used beyond 10 steps, it enters refiner mode enhancing image details without altering the composition",
|
||||
"preview": "shuttleai--shuttle-3_1-aestetic.jpg",
|
||||
"skip": true
|
||||
},
|
||||
"ShuttleAI Shuttle Jaguar": {
|
||||
"path": "shuttleai/shuttle-jaguar",
|
||||
"desc": "Shuttle uses Flux.1 Schnell as its base. It can produce images similar to Flux Dev or Pro in just 4 steps, and it is licensed under Apache 2. The model was partially de-distilled during training. When used beyond 10 steps, it enters refiner mode enhancing image details without altering the composition",
|
||||
"preview": "shuttleai--shuttle-jaguar.jpg",
|
||||
"skip": true
|
||||
},
|
||||
|
||||
"Bria 3.2": {
|
||||
"path": "briaai/BRIA-3.2",
|
||||
"desc": "Bria 3.2 is the next-generation commercial-ready text-to-image model. With just 4 billion parameters, it provides exceptional aesthetics and text rendering, evaluated to provide on par results to leading open-source models, and outperforming other licensed models.",
|
||||
@@ -909,6 +911,285 @@
|
||||
"extras": "sampler: Default",
|
||||
"size": 15.48,
|
||||
"date": "2023 April"
|
||||
},
|
||||
|
||||
"FLUX.1-Dev sdnq-svd-uint4": {
|
||||
"path": "Disty0/FLUX.1-dev-SDNQ-uint4-svd-r32",
|
||||
"preview": "Disty0--FLUX.1-dev-SDNQ-uint4-svd-r32.jpg",
|
||||
"desc": "Quantization of black-forest-labs/FLUX.1-dev using SDNQ: sdnq-svd 4-bit uint with svd rank 32",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"size": 12.60,
|
||||
"date": "2025 October",
|
||||
"extras": ""
|
||||
},
|
||||
"FLUX.1-Schnell sdnq-svd-uint4": {
|
||||
"path": "Disty0/FLUX.1-schnell-SDNQ-uint4-svd-r32",
|
||||
"preview": "Disty0--FLUX.1-dev-SDNQ-uint4-svd-r32.jpg",
|
||||
"desc": "Quantization of black-forest-labs/FLUX.1-schnell using SDNQ: sdnq-svd 4-bit uint with svd rank 32",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"size": 12.60,
|
||||
"date": "2025 October",
|
||||
"extras": ""
|
||||
},
|
||||
"FLUX.1-Dev Krea sdnq-svd-uint4": {
|
||||
"path": "Disty0/FLUX.1-Krea-dev-SDNQ-uint4-svd-r32",
|
||||
"preview": "Disty0--FLUX.1-dev-SDNQ-uint4-svd-r32.jpg",
|
||||
"desc": "Quantization of black-forest-labs/FLUX.1-Krea-dev using SDNQ: sdnq-svd 4-bit uint with svd rank 32",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"size": 12.60,
|
||||
"date": "2025 October",
|
||||
"extras": ""
|
||||
},
|
||||
"FLUX.1-Dev Kontext sdnq-svd-uint4": {
|
||||
"path": "Disty0/FLUX.1-Kontext-dev-SDNQ-uint4-svd-r32",
|
||||
"preview": "Disty0--FLUX.1-dev-SDNQ-uint4-svd-r32.jpg",
|
||||
"desc": "Quantization of black-forest-labs/FLUX.1-Kontext-dev using SDNQ: sdnq-svd 4-bit uint with svd rank 32",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"size": 12.60,
|
||||
"date": "2025 October",
|
||||
"extras": ""
|
||||
},
|
||||
"Chroma1-HD sdnq-svd-uint4": {
|
||||
"path": "Disty0/Chroma1-HD-SDNQ-uint4-svd-r32",
|
||||
"preview": "Disty0--Chroma1-HD-SDNQ-uint4-svd-r32.jpg",
|
||||
"desc": "Quantization of lodestones/Chroma1-HD using SDNQ: sdnq-svd 4-bit uint with svd rank 32",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"size": 11.89,
|
||||
"date": "2025 October",
|
||||
"extras": ""
|
||||
},
|
||||
"Wan-AI Wan2.2 A14B T2I sdnq-svd-uint4": {
|
||||
"path": "Disty0/Wan2.2-T2V-A14B-SDNQ-uint4-svd-r32",
|
||||
"preview": "Wan-AI--Wan2.2-T2V-A14B-Diffusers.jpg",
|
||||
"desc": "Quantization of black-forest-labs/FLUX.1-dev using SDNQ: sdnq-svd 4-bit uint with svd rank 32",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"date": "2025 October",
|
||||
"size": 23.54,
|
||||
"extras": ""
|
||||
},
|
||||
"Wan-AI Wan2.2 A14B I2I sdnq-svd-uint4": {
|
||||
"path": "Disty0/Wan2.2-I2V-A14B-SDNQ-uint4-svd-r32",
|
||||
"preview": "Wan-AI--Wan2.2-T2V-A14B-Diffusers.jpg",
|
||||
"desc": "Quantization of Laxhar/noobai-XL-1.1 using SDNQ: sdnq-svd 4-bit uint with svd rank 128",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"date": "2025 October",
|
||||
"size": 23.55,
|
||||
"extras": ""
|
||||
},
|
||||
"Qwen-Image sdnq-svd-uint4": {
|
||||
"path": "Disty0/Qwen-Image-SDNQ-uint4-svd-r32",
|
||||
"preview": "Qwen--Qwen-Image.jpg",
|
||||
"desc": "Quantization of Qwen/Qwen-Image using SDNQ: sdnq-svd 4-bit uint with svd rank 32",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"date": "2025 October",
|
||||
"size": 16.09,
|
||||
"extras": ""
|
||||
},
|
||||
"Qwen-Image-Edit sdnq-svd-uint4": {
|
||||
"path": "Disty0/Qwen-Image-Edit-SDNQ-uint4-svd-r32",
|
||||
"preview": "Qwen--Qwen-Image-Edit.jpg",
|
||||
"desc": "Quantization of Qwen/Qwen-Image-Edit using SDNQ: sdnq-svd 4-bit uint with svd rank 32",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"date": "2025 October",
|
||||
"size": 16.10,
|
||||
"extras": ""
|
||||
},
|
||||
"Qwen-Image-Edit-2509 sdnq-svd-uint4": {
|
||||
"path": "Disty0/Qwen-Image-Edit-2509-SDNQ-uint4-svd-r32",
|
||||
"preview": "Qwen--Qwen-Image-Edit-2509.jpg",
|
||||
"desc": "Quantization of Qwen/Qwen-Image-Edit-2509 using SDNQ: sdnq-svd 4-bit uint with svd rank 32",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"date": "2025 October",
|
||||
"size": 16.10,
|
||||
"extras": ""
|
||||
},
|
||||
"nVidia ChronoEdit sdnq-svd-uint4": {
|
||||
"path": "Disty0/ChronoEdit-14B-SDNQ-uint4-svd-r32",
|
||||
"preview": "nvidia--ChronoEdit-14B-Diffusers.jpg",
|
||||
"desc": "Quantization of nvidia/ChronoEdit-14B-Diffusers using SDNQ: sdnq-svd 4-bit uint with svd rank 32.",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"date": "2025 October",
|
||||
"size": 18.10,
|
||||
"extras": ""
|
||||
},
|
||||
"Tencent HunyuanImage 3.0 sdnq-svd-uint4": {
|
||||
"path": "Disty0/HunyuanImage3-SDNQ-uint4-svd-r32",
|
||||
"desc": "Quantization of tencent/HunyuanImage-3.0 using SDNQ: sdnq-svd 4-bit uint with svd rank 32.",
|
||||
"preview": "Disty0--HunyuanImage3-SDNQ-uint4-svd-r32.jpg",
|
||||
"extras": "",
|
||||
"skip": true,
|
||||
"tags": "quantized",
|
||||
"size": 57.06,
|
||||
"date": "2025 September"
|
||||
},
|
||||
"Tempest-by-Vlad XL sdnq-svd-uint4": {
|
||||
"path": "vladmandic/tempestByVlad_baseV01-SDNQ-uint4-svd",
|
||||
"preview": "tempestByVlad_baseV01.jpg",
|
||||
"desc": "Quantization of vladmandic/tempestByVlad_baseV01 using SDNQ: sdnq-svd 4-bit uint with svd rank 128",
|
||||
"tags": "quantized",
|
||||
"size": 3.37,
|
||||
"date": "2025 October",
|
||||
"extras": ""
|
||||
},
|
||||
"NoobAI-XL v1.1 epsilon sdnq-svd-uint4": {
|
||||
"path": "Disty0/NoobAI-XL-v1.1-SDNQ-uint4-svd-r128",
|
||||
"preview": "Disty0--NoobAI-XL-v1.1-SDNQ-uint4-svd-r128.jpg",
|
||||
"desc": "Quantization of Laxhar/noobai-XL-1.1 using SDNQ: sdnq-svd 4-bit uint with svd rank 128",
|
||||
"tags": "quantized",
|
||||
"size": 3.37,
|
||||
"date": "2025 October",
|
||||
"extras": ""
|
||||
},
|
||||
"NoobAI-XL v1.0 v-pred sdnq-svd-uint4": {
|
||||
"path": "Disty0/NoobAI-XL-Vpred-v1.0-SDNQ-uint4-svd-r128",
|
||||
"preview": "Disty0--NoobAI-XL-Vpred-v1.0-SDNQ-uint4-svd-r128.jpg",
|
||||
"desc": "Quantization of Laxhar/noobai-XL-Vpred-1.0 using SDNQ: sdnq-svd 4-bit uint with svd rank 128",
|
||||
"tags": "quantized",
|
||||
"size": 3.37,
|
||||
"date": "2025 October",
|
||||
"extras": ""
|
||||
},
|
||||
|
||||
"Tempest-by-Vlad XL": {
|
||||
"path": "tempestByVlad_baseV01.safetensors@https://civitai.com/api/download/models/1301775",
|
||||
"preview": "tempestByVlad_baseV01.jpg",
|
||||
"desc": "Flexible SDXL model with custom encoder and finetuned for larger landscape resolutions with high details and high contrast.",
|
||||
"tags": "community",
|
||||
"size": 6.94,
|
||||
"date": "2025 January",
|
||||
"extras": ""
|
||||
},
|
||||
"Tempest-by-Vlad XL Hyper": {
|
||||
"path": "tempestByVlad_hyperV01.safetensors@https://civitai.com/api/download/models/1343512",
|
||||
"preview": "tempestByVlad_hyperV01.jpg",
|
||||
"desc": "Custom distilled variant with goal to get as-normal-as-possible model that works with low steps and guidance-free",
|
||||
"tags": "community",
|
||||
"size": 6.94,
|
||||
"date": "2025 January",
|
||||
"extras": ""
|
||||
},
|
||||
"Juggernaut XL XI": {
|
||||
"path": "juggernautXL_juggXIByRundiffusion.safetensors@https://civitai.com/api/download/models/782002",
|
||||
"preview": "juggernautXL_juggXIByRundiffusion.jpg",
|
||||
"desc": "Showcase finetuned model based on Stable diffusion XL",
|
||||
"date": "2024 August",
|
||||
"size": 6.94,
|
||||
"tags": "community",
|
||||
"extras": "sampler: DEIS, steps: 20, cfg_scale: 6.0"
|
||||
},
|
||||
"Juggernaut XL XI Lightning": {
|
||||
"path": "juggernautXL_juggXILightningByRD.safetensors@https://civitai.com/api/download/models/920957",
|
||||
"preview": "juggernautXL_juggXILightningByRD.jpg",
|
||||
"desc": "Showcase finetuned model based on Stable diffusion XL",
|
||||
"date": "2024 August",
|
||||
"size": 6.94,
|
||||
"tags": "community",
|
||||
"extras": "sampler: DPM SDE, steps: 6, cfg_scale: 2.0"
|
||||
},
|
||||
"Juggernaut SD Reborn": {
|
||||
"original": true,
|
||||
"path": "juggernaut_reborn.safetensors@https://civitai.com/api/download/models/274039",
|
||||
"preview": "juggernaut_reborn.jpg",
|
||||
"desc": "Showcase finetuned model based on Stable diffusion 1.5",
|
||||
"date": "2023 December",
|
||||
"size": 2.28,
|
||||
"tags": "community",
|
||||
"extras": "width: 512, height: 512, sampler: DEIS, steps: 20, cfg_scale: 6.0"
|
||||
},
|
||||
"WAI Illustrious XL v15": {
|
||||
"path": "waiIllustriousSDXL_v150.safetensors@https://civitai.com/api/download/models/2167369",
|
||||
"preview": "waiIllustriousSDXL_v150.jpg",
|
||||
"desc": "",
|
||||
"tags": "community",
|
||||
"size": 6.94,
|
||||
"date": "2025 August",
|
||||
"extras": ""
|
||||
},
|
||||
"Pony Realism XL v2.3": {
|
||||
"path": "ponyRealism_V23.safetensors@https://civitai.com/api/download/models/2167369",
|
||||
"preview": "ponyRealism_V23.jpg",
|
||||
"desc": "",
|
||||
"tags": "community",
|
||||
"size": 6.94,
|
||||
"date": "2025 May",
|
||||
"extras": ""
|
||||
},
|
||||
"NoobAI XL 1.0 V-Pred": {
|
||||
"path": "noobaiXLNAIXL_vPred10Version.safetensors@https://huggingface.co/Laxhar/noobai-XL-Vpred-1.0/resolve/main/NoobAI-XL-Vpred-v1.0.safetensors",
|
||||
"preview": "noobaiXLNAIXL_vPred10Version.jpg",
|
||||
"desc": "",
|
||||
"tags": "community",
|
||||
"size": 6.94,
|
||||
"date": "2024 December",
|
||||
"extras": ""
|
||||
},
|
||||
"NoobAI XL 1.1 Epsilon": {
|
||||
"path": "noobaiXLNAIXL_epsilonPred11Version.safetensors@https://huggingface.co/Laxhar/noobai-XL-1.1/resolve/main/NoobAI-XL-v1.1.safetensors",
|
||||
"preview": "noobaiXLNAIXL_epsilonPred11Version.jpg",
|
||||
"desc": "",
|
||||
"tags": "community",
|
||||
"size": 6.94,
|
||||
"date": "2024 November",
|
||||
"extras": ""
|
||||
},
|
||||
"WAI-Ani-Pony XL v14": {
|
||||
"path": "waiANIPONYXL_v140.safetensors.safetensors@https://civitai.com/api/download/models/1767402",
|
||||
"preview": "waiANIPONYXL_v140.jpg",
|
||||
"desc": "",
|
||||
"tags": "community",
|
||||
"size": 6.94,
|
||||
"date": "2025 May",
|
||||
"extras": ""
|
||||
},
|
||||
"Tiwaz CenKreChro": {
|
||||
"path": "Tiwaz/CenKreChro",
|
||||
"preview": "Tiwaz--CenKreChro.jpg",
|
||||
"skip": true,
|
||||
"desc": "Based Centerfold Flux 5, trying to merge in Chroma and Krea.",
|
||||
"extras": "",
|
||||
"tags": "community",
|
||||
"date": "2025 September"
|
||||
},
|
||||
"purplesmartai Pony 7": {
|
||||
"path": "purplesmartai/pony-v7-base",
|
||||
"preview": "purplesmartai--pony-v7-base.jpg",
|
||||
"skip": true,
|
||||
"desc": "Pony V7 is a versatile character generation model based on AuraFlow architecture. It supports a wide range of styles and species types (humanoid, anthro, feral, and more) and handles character interactions through natural language prompts.",
|
||||
"extras": "",
|
||||
"tags": "community",
|
||||
"date": "October September"
|
||||
},
|
||||
"ShuttleAI Shuttle 3.0 Diffusion": {
|
||||
"path": "shuttleai/shuttle-3-diffusion",
|
||||
"desc": "Shuttle uses Flux.1 Schnell as its base. It can produce images similar to Flux Dev or Pro in just 4 steps, and it is licensed under Apache 2. The model was partially de-distilled during training. When used beyond 10 steps, it enters refiner mode enhancing image details without altering the composition",
|
||||
"preview": "shuttleai--shuttle-3-diffusion.jpg",
|
||||
"tags": "community",
|
||||
"skip": true
|
||||
},
|
||||
"ShuttleAI Shuttle 3.1 Aesthetic": {
|
||||
"path": "shuttleai/shuttle-3.1-aesthetic",
|
||||
"desc": "Shuttle uses Flux.1 Schnell as its base. It can produce images similar to Flux Dev or Pro in just 4 steps, and it is licensed under Apache 2. The model was partially de-distilled during training. When used beyond 10 steps, it enters refiner mode enhancing image details without altering the composition",
|
||||
"preview": "shuttleai--shuttle-3_1-aestetic.jpg",
|
||||
"tags": "community",
|
||||
"skip": true
|
||||
},
|
||||
"ShuttleAI Shuttle Jaguar": {
|
||||
"path": "shuttleai/shuttle-jaguar",
|
||||
"desc": "Shuttle uses Flux.1 Schnell as its base. It can produce images similar to Flux Dev or Pro in just 4 steps, and it is licensed under Apache 2. The model was partially de-distilled during training. When used beyond 10 steps, it enters refiner mode enhancing image details without altering the composition",
|
||||
"preview": "shuttleai--shuttle-jaguar.jpg",
|
||||
"tags": "community",
|
||||
"skip": true
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
+61
-39
@@ -241,7 +241,10 @@ def setup_logging():
|
||||
log.addHandler(fh)
|
||||
global log_rolled # pylint: disable=global-statement
|
||||
if not log_rolled and args.debug and not args.log:
|
||||
fh.doRollover()
|
||||
try:
|
||||
fh.doRollover()
|
||||
except Exception:
|
||||
pass
|
||||
log_rolled = True
|
||||
|
||||
rb = RingBuffer(100) # 100 entries default in log ring buffer
|
||||
@@ -386,6 +389,9 @@ def pip(arg: str, ignore: bool = False, quiet: bool = True, uv = True):
|
||||
t_start = time.time()
|
||||
originalArg = arg
|
||||
arg = arg.replace('>=', '==')
|
||||
if opts.get('offline_mode', False):
|
||||
log.warning('Offline mode enabled')
|
||||
return 'offline'
|
||||
package = arg.replace("install", "").replace("--upgrade", "").replace("--no-deps", "").replace("--force-reinstall", "").replace(" ", " ").strip()
|
||||
uv = uv and args.uv and not package.startswith('git+')
|
||||
pipCmd = "uv pip" if uv else "pip"
|
||||
@@ -586,8 +592,10 @@ def check_python(supported_minors=[], experimental_minors=[], reason=None):
|
||||
sys.exit(1)
|
||||
if int(sys.version_info.minor) == 12:
|
||||
os.environ.setdefault('SETUPTOOLS_USE_DISTUTILS', 'local') # hack for python 3.11 setuptools
|
||||
if int(sys.version_info.minor) == 10:
|
||||
log.warning(f"Python: version={platform.python_version()} is not actively supported")
|
||||
if int(sys.version_info.minor) == 9:
|
||||
log.warning("Python 3.9 support is scheduled to be removed")
|
||||
log.warning(f"Python: version={platform.python_version()} is end-of-life")
|
||||
if not args.skip_git:
|
||||
git_cmd = os.environ.get('GIT', "git")
|
||||
if shutil.which(git_cmd) is None:
|
||||
@@ -605,10 +613,7 @@ def check_diffusers():
|
||||
t_start = time.time()
|
||||
if args.skip_all:
|
||||
return
|
||||
if args.skip_git:
|
||||
install('diffusers')
|
||||
return
|
||||
sha = '23ebbb4bc81a17ebea17cb7cb94f301199e49a7f' # diffusers commit hash
|
||||
sha = '9f3c0fdcd859905c2c13ec47f10eb0250d2576ac' # diffusers commit hash
|
||||
# if args.use_rocm or args.use_zluda or args.use_directml:
|
||||
# sha = '043ab2520f6a19fce78e6e060a68dbc947edb9f9' # lock diffusers versions for now
|
||||
pkg = pkg_resources.working_set.by_key.get('diffusers', None)
|
||||
@@ -620,6 +625,8 @@ def check_diffusers():
|
||||
else:
|
||||
log.info(f'Diffusers update: current={pkg.version} hash={cur} target={sha}')
|
||||
pip('uninstall --yes diffusers', ignore=True, quiet=True, uv=False)
|
||||
if args.skip_git:
|
||||
log.warning('Git: marked as not available but required for diffusers installation')
|
||||
pip(f'install --upgrade git+https://github.com/huggingface/diffusers@{sha}', ignore=False, quiet=True, uv=False)
|
||||
global diffusers_commit # pylint: disable=global-statement
|
||||
diffusers_commit = sha
|
||||
@@ -756,28 +763,24 @@ def install_rocm_zluda():
|
||||
except Exception as e:
|
||||
log.warning(f'Failed to load ZLUDA: {e}')
|
||||
else:
|
||||
#check_python(supported_minors=[10, 11, 12, 13], reason='ROCm backend requires a Python version between 3.10 and 3.13')
|
||||
|
||||
#check_python(supported_minors=[10, 11, 12, 13, 14], reason='ROCm backend requires a Python version between 3.10 and 3.13')
|
||||
if args.use_nightly:
|
||||
if rocm.version is None or float(rocm.version) >= 6.4: # assume the latest if version check fails
|
||||
if rocm.version is None or float(rocm.version) >= 7.0: # assume the latest if version check fails
|
||||
torch_command = os.environ.get('TORCH_COMMAND', '--upgrade --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/rocm7.0')
|
||||
else: # oldest rocm version on nightly is 6.4
|
||||
torch_command = os.environ.get('TORCH_COMMAND', '--upgrade --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/rocm6.4')
|
||||
else: # oldest rocm version on nightly is 6.3
|
||||
torch_command = os.environ.get('TORCH_COMMAND', '--upgrade --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/rocm6.3')
|
||||
else:
|
||||
if rocm.version is None or float(rocm.version) >= 6.4: # assume the latest if version check fails
|
||||
# Torch 2.8 with ROCm has common segfaults, memory access violations and accuracy issues
|
||||
#torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.8.0+rocm6.4 torchvision==0.23.0+rocm6.4 --index-url https://download.pytorch.org/whl/rocm6.4')
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.7.1+rocm6.3 torchvision==0.22.1+rocm6.3 --index-url https://download.pytorch.org/whl/rocm6.3')
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.9.0+rocm6.4 torchvision==0.24.0+rocm6.4 --index-url https://download.pytorch.org/whl/rocm6.4')
|
||||
elif rocm.version == "6.3":
|
||||
#torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.8.0+rocm6.3 torchvision==0.23.0+rocm6.3 --index-url https://download.pytorch.org/whl/rocm6.3')
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.7.1+rocm6.3 torchvision==0.22.1+rocm6.3 --index-url https://download.pytorch.org/whl/rocm6.3')
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.9.0+rocm6.3 torchvision==0.24.0+rocm6.3 --index-url https://download.pytorch.org/whl/rocm6.3')
|
||||
elif rocm.version == "6.2":
|
||||
# use rocm 6.2.4 instead of 6.2 as torch==2.7.1+rocm6.2 doesn't exists
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.7.1+rocm6.2.4 torchvision==0.22.1+rocm6.2.4 --index-url https://download.pytorch.org/whl/rocm6.2.4')
|
||||
elif rocm.version == "6.1":
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.6.0+rocm6.1 torchvision==0.21.0+rocm6.1 --index-url https://download.pytorch.org/whl/rocm6.1')
|
||||
else:
|
||||
# lock to 2.4.1 instead of 2.5.1 for performance reasons there are no support for torch 2.6.0 for rocm 6.0
|
||||
# lock to 2.4.1 instead of 2.5.1 for performance reasons there are no support for torch 2.6 for rocm 6.0
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.4.1+rocm6.0 torchvision==0.19.1+rocm6.0 --index-url https://download.pytorch.org/whl/rocm6.0')
|
||||
if float(rocm.version) < 6.0:
|
||||
log.warning(f"ROCm: unsupported version={rocm.version}")
|
||||
@@ -797,15 +800,14 @@ def install_rocm_zluda():
|
||||
|
||||
def install_ipex():
|
||||
t_start = time.time()
|
||||
#check_python(supported_minors=[10, 11, 12, 13], reason='IPEX backend requires a Python version between 3.10 and 3.13')
|
||||
#check_python(supported_minors=[10, 11, 12, 13, 14], reason='IPEX backend requires a Python version between 3.10 and 3.13')
|
||||
args.use_ipex = True # pylint: disable=attribute-defined-outside-init
|
||||
log.info('IPEX: Intel OneAPI toolkit detected')
|
||||
|
||||
if args.use_nightly:
|
||||
torch_command = os.environ.get('TORCH_COMMAND', '--upgrade --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/xpu')
|
||||
else:
|
||||
# torch 2.8 segfaults with torch.compile: https://github.com/pytorch/pytorch/issues/159974
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.7.1+xpu torchvision==0.22.1+xpu --index-url https://download.pytorch.org/whl/xpu')
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.9.0+xpu torchvision==0.24.0+xpu --index-url https://download.pytorch.org/whl/xpu')
|
||||
|
||||
ts('ipex', t_start)
|
||||
return torch_command
|
||||
@@ -818,9 +820,9 @@ def install_openvino():
|
||||
|
||||
#check_python(supported_minors=[10, 11, 12, 13], reason='OpenVINO backend requires a Python version between 3.10 and 3.13')
|
||||
if sys.platform == 'darwin':
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.8.0 torchvision==0.23.0')
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.9.0 torchvision==0.24.0')
|
||||
else:
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.8.0+cpu torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cpu')
|
||||
torch_command = os.environ.get('TORCH_COMMAND', 'torch==2.9.0+cpu torchvision==0.24.0 --index-url https://download.pytorch.org/whl/cpu')
|
||||
|
||||
if not (args.skip_all or args.skip_requirements):
|
||||
install(os.environ.get('OPENVINO_COMMAND', 'openvino==2025.3.0'), 'openvino')
|
||||
@@ -855,7 +857,7 @@ def install_torch_addons():
|
||||
if len(opts.get('torchao_quantization', [])):
|
||||
install('torchao==0.10.0', 'torchao')
|
||||
if opts.get('samples_format', 'jpg') == 'jxl' or opts.get('grid_format', 'jpg') == 'jxl':
|
||||
install('pillow-jxl-plugin==1.3.4', 'pillow-jxl-plugin')
|
||||
install('pillow-jxl-plugin==1.3.5', 'pillow-jxl-plugin')
|
||||
if not args.experimental:
|
||||
uninstall('wandb', quiet=True)
|
||||
uninstall('pynvml', quiet=True)
|
||||
@@ -1261,6 +1263,13 @@ def install_pydantic():
|
||||
reload('pydantic', '1.10.21')
|
||||
|
||||
|
||||
def install_opencv():
|
||||
install('opencv-python==4.12.0.88', ignore=True, quiet=True)
|
||||
install('opencv-python-headless==4.12.0.88', ignore=True, quiet=True)
|
||||
install('opencv-contrib-python==4.12.0.88', ignore=True, quiet=True)
|
||||
install('opencv-contrib-python-headless==4.12.0.88', ignore=True, quiet=True)
|
||||
|
||||
|
||||
def install_insightface():
|
||||
install('git+https://github.com/deepinsight/insightface@29b6cd65aa0e9ae3b6602de3c52e9d8949c8ee86#subdirectory=python-package', 'insightface') # insightface==0.7.3 with patches
|
||||
if args.new:
|
||||
@@ -1275,18 +1284,23 @@ def install_insightface():
|
||||
def install_optional():
|
||||
t_start = time.time()
|
||||
log.info('Installing optional requirements...')
|
||||
install('--no-build-isolation git+https://github.com/Disty0/BasicSR@23c1fb6f5c559ef5ce7ad657f2fa56e41b121754', 'basicsr')
|
||||
install('--no-build-isolation git+https://github.com/Disty0/GFPGAN@ae0f7e44fafe0ef4716f3c10067f8f379b74c21c', 'gfpgan')
|
||||
install('clean-fid', quiet=True)
|
||||
install('pillow-jxl-plugin==1.3.4', ignore=True, quiet=True)
|
||||
install('optimum-quanto==0.2.7', ignore=True, quiet=True)
|
||||
install('torchao==0.10.0', ignore=True, quiet=True)
|
||||
install('bitsandbytes==0.47.0', ignore=True, quiet=True)
|
||||
install('nvidia-ml-py', ignore=True, quiet=True)
|
||||
install('ultralytics==8.3.40', ignore=True, quiet=True)
|
||||
install('Cython', ignore=True, quiet=True)
|
||||
install('--no-build-isolation git+https://github.com/Disty0/BasicSR@23c1fb6f5c559ef5ce7ad657f2fa56e41b121754', 'basicsr', ignore=True, quiet=True)
|
||||
install('--no-build-isolation git+https://github.com/Disty0/GFPGAN@ae0f7e44fafe0ef4716f3c10067f8f379b74c21c', 'gfpgan', ignore=True, quiet=True)
|
||||
install('av', ignore=True, quiet=True)
|
||||
install('gguf', ignore=True)
|
||||
install('beautifulsoup4', ignore=True, quiet=True)
|
||||
install('bitsandbytes==0.47.0', ignore=True, quiet=True)
|
||||
install('clean-fid', ignore=True, quiet=True)
|
||||
install('clip_interrogator==0.6.0', ignore=True, quiet=True)
|
||||
install('Cython', ignore=True, quiet=True)
|
||||
install('gguf', ignore=True, quiet=True)
|
||||
install('git+https://github.com/tencent-ailab/IP-Adapter.git', 'ip_adapter', ignore=True, quiet=True)
|
||||
install('hf_transfer', ignore=True, quiet=True)
|
||||
install('hf_xet', ignore=True, quiet=True)
|
||||
install('nvidia-ml-py', ignore=True, quiet=True)
|
||||
install('optimum-quanto==0.2.7', ignore=True, quiet=True)
|
||||
install('pillow-jxl-plugin==1.3.5', ignore=True, quiet=True)
|
||||
install('torchao==0.10.0', ignore=True, quiet=True)
|
||||
install('ultralytics==8.3.40', ignore=True, quiet=True)
|
||||
try:
|
||||
import gguf
|
||||
scripts_dir = os.path.join(os.path.dirname(gguf.__file__), '..', 'scripts')
|
||||
@@ -1299,11 +1313,11 @@ def install_optional():
|
||||
|
||||
def install_requirements():
|
||||
t_start = time.time()
|
||||
if args.skip_requirements and not args.requirements:
|
||||
return
|
||||
if args.profile:
|
||||
pr = cProfile.Profile()
|
||||
pr.enable()
|
||||
if args.skip_requirements and not args.requirements:
|
||||
return
|
||||
if int(sys.version_info.minor) >= 13:
|
||||
install('audioop-lts')
|
||||
if not installed('diffusers', quiet=True): # diffusers are not installed, so run initial installation
|
||||
@@ -1324,6 +1338,7 @@ def install_requirements():
|
||||
if not installed(line, quiet=True):
|
||||
_res = install(line)
|
||||
install_pydantic()
|
||||
install_opencv()
|
||||
if args.profile:
|
||||
pr.disable()
|
||||
print_profile(pr, 'Requirements')
|
||||
@@ -1356,6 +1371,8 @@ def set_environment():
|
||||
os.environ.setdefault('UV_INDEX_STRATEGY', 'unsafe-any-match')
|
||||
os.environ.setdefault('UV_NO_BUILD_ISOLATION', '1')
|
||||
os.environ.setdefault('UVICORN_TIMEOUT_KEEP_ALIVE', '60')
|
||||
os.environ.setdefault('RUNAI_STREAMER_CHUNK_BYTESIZE', '2097152')
|
||||
os.environ.setdefault('RUNAI_STREAMER_MEMORY_LIMIT', '-1')
|
||||
allocator = f'garbage_collection_threshold:{opts.get("torch_gc_threshold", 80)/100:0.2f},max_split_size_mb:512'
|
||||
if opts.get("torch_malloc", "native") == 'cudaMallocAsync':
|
||||
allocator += ',backend:cudaMallocAsync'
|
||||
@@ -1501,12 +1518,17 @@ def check_venv():
|
||||
|
||||
|
||||
# check version of the main repo and optionally upgrade it
|
||||
def check_version(offline=False, reset=True): # pylint: disable=unused-argument
|
||||
def check_version(reset=True): # pylint: disable=unused-argument
|
||||
if opts.get('offline_mode', False):
|
||||
log.warning('Offline mode enabled')
|
||||
args.skip_git = True # pylint: disable=attribute-defined-outside-init
|
||||
args.skip_all = True # pylint: disable=attribute-defined-outside-init
|
||||
return
|
||||
t_start = time.time()
|
||||
if args.skip_all:
|
||||
return
|
||||
if not os.path.exists('.git'):
|
||||
log.warning('Not a git repository, all git operations are disabled')
|
||||
log.warning('Not a git repository')
|
||||
args.skip_git = True # pylint: disable=attribute-defined-outside-init
|
||||
ver = get_version()
|
||||
log.info(f'Version: {print_dict(ver)}')
|
||||
@@ -1543,7 +1565,7 @@ def check_version(offline=False, reset=True): # pylint: disable=unused-argument
|
||||
else:
|
||||
log.warning('Repository: retrying upgrade...')
|
||||
git_reset()
|
||||
check_version(offline=offline, reset=False)
|
||||
check_version(reset=False)
|
||||
else:
|
||||
dt = commits["commit"]["commit"]["author"]["date"]
|
||||
commit = commits["commit"]["sha"][:8]
|
||||
|
||||
@@ -144,7 +144,19 @@ async function filterExtraNetworksForTab(searchTerm) {
|
||||
} else if (searchTerm === 'reference/') {
|
||||
cards.forEach((elem) => elem.style.display = elem.dataset.name
|
||||
.toLowerCase()
|
||||
.includes('reference/') ? '' : 'none');
|
||||
.includes('reference/') && elem.dataset.tags === '' ? '' : 'none');
|
||||
} else if (searchTerm === 'distilled/') {
|
||||
cards.forEach((elem) => elem.style.display = elem.dataset.tags
|
||||
.toLowerCase()
|
||||
.includes('distilled') ? '' : 'none');
|
||||
} else if (searchTerm === 'community/') {
|
||||
cards.forEach((elem) => elem.style.display = elem.dataset.tags
|
||||
.toLowerCase()
|
||||
.includes('community') ? '' : 'none');
|
||||
} else if (searchTerm === 'quantized/') {
|
||||
cards.forEach((elem) => elem.style.display = elem.dataset.tags
|
||||
.toLowerCase()
|
||||
.includes('quantized') ? '' : 'none');
|
||||
} else if (searchTerm === 'local/') {
|
||||
cards.forEach((elem) => elem.style.display = elem.dataset.name
|
||||
.toLowerCase()
|
||||
|
||||
@@ -52,12 +52,15 @@ function modalImageSwitch(offset) {
|
||||
}
|
||||
|
||||
function modalSaveImage(event) {
|
||||
if (gradioApp().getElementById('tab_txt2img').style.display !== 'none') gradioApp().getElementById('save_txt2img').click();
|
||||
else if (gradioApp().getElementById('tab_img2img').style.display !== 'none') gradioApp().getElementById('save_img2img').click();
|
||||
else if (gradioApp().getElementById('tab_process').style.display !== 'none') gradioApp().getElementById('save_extras').click();
|
||||
const tabName = getENActiveTab();
|
||||
const saveBtn = gradioApp().getElementById(`save_${tabName}`);
|
||||
log('modalSaveImage', tabName, saveBtn);
|
||||
if (saveBtn) saveBtn.click();
|
||||
modalImageSwitch(0);
|
||||
}
|
||||
|
||||
function modalKeyHandler(event) {
|
||||
log('modalKeyHandler', event.key);
|
||||
switch (event.key) {
|
||||
case 's':
|
||||
modalSaveImage();
|
||||
@@ -158,6 +161,7 @@ function modalZoomToggle(event) {
|
||||
const modalImage = gradioApp().getElementById('modalImage');
|
||||
modalZoomSet(modalImage, !modalImage.classList.contains('modalImageFullscreen'));
|
||||
event.stopPropagation();
|
||||
modalImageSwitch(0);
|
||||
}
|
||||
|
||||
function modalTileToggle(event) {
|
||||
@@ -172,12 +176,15 @@ function modalTileToggle(event) {
|
||||
modal.style.setProperty('background-image', `url(${modalImage.src})`);
|
||||
}
|
||||
event.stopPropagation();
|
||||
modalImageSwitch(0);
|
||||
}
|
||||
|
||||
function modalResetInstance(event) {
|
||||
const modalImage = document.getElementById('modalImage');
|
||||
previewInstance.dispose();
|
||||
previewInstance = panzoom(modalImage, { zoomSpeed: 0.05, minZoom: 0.1, maxZoom: 5.0, filterKey: (/* e, dx, dy, dz */) => true });
|
||||
event.stopPropagation();
|
||||
modalImageSwitch(0);
|
||||
}
|
||||
|
||||
function modalToggleParams(event) {
|
||||
@@ -188,6 +195,7 @@ function modalToggleParams(event) {
|
||||
modalExif.style.display = 'none';
|
||||
}
|
||||
event.stopPropagation();
|
||||
modalImageSwitch(0);
|
||||
}
|
||||
|
||||
function galleryClickEventHandler(event) {
|
||||
|
||||
@@ -132,7 +132,7 @@ function requestProgress(id_task, progressEl, galleryEl, atEnd = null, onProgres
|
||||
};
|
||||
|
||||
const start = (id_task, id_live_preview) => { // eslint-disable-line no-shadow
|
||||
if (!opts.live_previews_enable || opts.live_preview_refresh_period === 0 || opts.show_progress_every_n_steps === 0) return;
|
||||
if (opts.live_preview_refresh_period === 0) return;
|
||||
const request_id = document.hidden ? -1 : id_live_preview;
|
||||
|
||||
const onProgressHandler = (res) => {
|
||||
|
||||
@@ -170,7 +170,7 @@ async function initModels() {
|
||||
if (en.classList.contains('hide')) gradioApp().getElementById('txt2img_extra_networks_btn').click();
|
||||
const repeat = setInterval(() => {
|
||||
const buttons = Array.from(gradioApp().querySelectorAll('#txt2img_model_subdirs > button')) || [];
|
||||
const reference = buttons.find((b) => b.innerText === 'Reference');
|
||||
const reference = buttons.find((b) => (b.innerText === 'Reference') || (b.innerText === 'Distilled') || (b.innerText === 'Community') || (b.innerText === 'Quantized'));
|
||||
if (reference) {
|
||||
clearInterval(repeat);
|
||||
reference.click();
|
||||
|
||||
@@ -297,10 +297,9 @@ def main():
|
||||
installer.log.info(f'Installer time: {init_summary()}')
|
||||
get_custom_args()
|
||||
|
||||
uv, instance = start_server(immediate=True, server=None)
|
||||
if installer.restart_required:
|
||||
installer.log.warning('Restart is recommended due to packages updates...')
|
||||
|
||||
uv, instance = start_server(immediate=True, server=None)
|
||||
t_server = time.time()
|
||||
t_monitor = time.time()
|
||||
while True:
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 29 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 36 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 34 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 32 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 37 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 74 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 68 KiB After Width: | Height: | Size: 65 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 80 KiB After Width: | Height: | Size: 70 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 74 KiB After Width: | Height: | Size: 70 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 70 KiB After Width: | Height: | Size: 67 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 76 KiB After Width: | Height: | Size: 75 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 77 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 69 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 77 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 54 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 49 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 61 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 38 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 75 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 54 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 84 KiB |
+26
-38
@@ -37,28 +37,6 @@ ReqControl = models.create_model_from_signature(
|
||||
if not hasattr(ReqControl, "__config__"):
|
||||
ReqControl.__config__ = models.DummyConfig
|
||||
|
||||
"""
|
||||
ReqControl = models.PydanticModelGenerator(
|
||||
"StableDiffusionProcessingControl",
|
||||
StableDiffusionProcessingControl,
|
||||
[
|
||||
{"key": "sampler_index", "type": Union[int, str], "default": 0},
|
||||
{"key": "sampler_name", "type": str, "default": "Default"},
|
||||
{"key": "script_name", "type": Optional[str], "default": ""},
|
||||
{"key": "script_args", "type": list, "default": []},
|
||||
{"key": "send_images", "type": bool, "default": True},
|
||||
{"key": "save_images", "type": bool, "default": False},
|
||||
{"key": "alwayson_scripts", "type": dict, "default": {}},
|
||||
{"key": "ip_adapter", "type": Optional[List[models.ItemIPAdapter]], "default": None, "exclude": True},
|
||||
{"key": "face", "type": Optional[models.ItemFace], "default": None, "exclude": True},
|
||||
{"key": "control", "type": Optional[List[ItemControl]], "default": [], "exclude": True},
|
||||
{"key": "extra", "type": Optional[dict], "default": {}, "exclude": True},
|
||||
]
|
||||
).generate_model()
|
||||
if not hasattr(ReqControl, "__config__"):
|
||||
ReqControl.__config__ = models.DummyConfig
|
||||
"""
|
||||
|
||||
|
||||
class ResControl(BaseModel):
|
||||
images: List[str] = Field(default=None, title="Images", description="")
|
||||
@@ -71,6 +49,7 @@ class APIControl():
|
||||
def __init__(self, queue_lock: Lock):
|
||||
self.queue_lock = queue_lock
|
||||
self.default_script_arg = []
|
||||
self.units = []
|
||||
|
||||
def sanitize_args(self, args: dict):
|
||||
args = vars(args)
|
||||
@@ -142,29 +121,38 @@ class APIControl():
|
||||
from modules.control.unit import Unit, unit_types
|
||||
req.units = []
|
||||
if req.unit_type is None:
|
||||
return req.control
|
||||
req.unit_type = 'controlnet'
|
||||
if req.unit_type not in unit_types:
|
||||
shared.log.error(f'Control uknown unit type: type={req.unit_type} available={unit_types}')
|
||||
return req.control
|
||||
for u in req.control:
|
||||
unit = Unit(
|
||||
enabled = True,
|
||||
unit_type = req.unit_type,
|
||||
model_id = u.model,
|
||||
process_id = u.process,
|
||||
strength = u.strength,
|
||||
start = u.start,
|
||||
end = u.end,
|
||||
)
|
||||
return
|
||||
for i in range(len(req.control)):
|
||||
u = req.control[i]
|
||||
if (len(self.units) > i) and (self.units[i].process_id == u.process) and (self.units[i].model_id == u.model):
|
||||
unit = self.units[i]
|
||||
unit.enabled = True
|
||||
unit.strength = u.strength
|
||||
unit.start = u.start
|
||||
unit.end = u.end
|
||||
else:
|
||||
unit = Unit(
|
||||
enabled = True,
|
||||
unit_type = req.unit_type,
|
||||
model_id = u.model,
|
||||
process_id = u.process,
|
||||
strength = u.strength,
|
||||
start = u.start,
|
||||
end = u.end,
|
||||
)
|
||||
if u.override is not None:
|
||||
unit.override = helpers.decode_base64_to_image(u.override)
|
||||
req.units.append(unit)
|
||||
return req.control
|
||||
self.units = req.units
|
||||
del req.control
|
||||
|
||||
def post_control(self, req: ReqControl):
|
||||
self.prepare_face_module(req)
|
||||
orig_control = self.prepare_control(req)
|
||||
del req.control
|
||||
requested = req.control
|
||||
self.prepare_control(req)
|
||||
|
||||
# prepare args
|
||||
args = req.copy(update={ # Override __init__ params
|
||||
@@ -203,5 +191,5 @@ class APIControl():
|
||||
b64images = list(map(helpers.encode_pil_to_base64, output_images)) if send_images else []
|
||||
b64processed = list(map(helpers.encode_pil_to_base64, output_processed)) if send_images else []
|
||||
self.sanitize_b64(req)
|
||||
req.units = orig_control
|
||||
req.units = requested
|
||||
return ResControl(images=b64images, processed=b64processed, params=vars(req), info=output_info)
|
||||
|
||||
@@ -20,6 +20,9 @@ def get_gpu_status():
|
||||
elif 'amd' in device.lower():
|
||||
from modules.api import rocm_smi
|
||||
return rocm_smi.get_rocm_smi()
|
||||
elif 'arc' in device.lower():
|
||||
from modules.api import xpu_smi
|
||||
return xpu_smi.get_xpu_smi()
|
||||
return []
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
try:
|
||||
from installer import log
|
||||
except Exception:
|
||||
import logging
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def get_xpu_smi():
|
||||
try:
|
||||
import torch
|
||||
from modules.memstats import ram_stats
|
||||
|
||||
devices = []
|
||||
mem = torch.xpu.memory_stats()
|
||||
ram = ram_stats()
|
||||
cap = torch.xpu.get_device_capability()
|
||||
prop = torch.xpu.get_device_properties()
|
||||
load = {
|
||||
'gpu': 0, # no interface to get gpu load
|
||||
'memory': mem['active_bytes.all.allocated'] // (1024**3), # no interface to get gpu memory so use torch instead
|
||||
}
|
||||
total = prop.total_memory // (1024**2)
|
||||
data = {
|
||||
'Version': cap['version'],
|
||||
'Driver': prop.driver_version,
|
||||
'Platform': prop.platform_name,
|
||||
'ID': hex(prop.device_id).removeprefix("0x"),
|
||||
'Compute Units': prop.max_compute_units,
|
||||
"VRAM usage": f'{round(100 * load["memory"] / total)}% | {load["memory"]} MB used | {total - load["memory"]} MB free | {total} MB total',
|
||||
"RAM usage": f'{round(100 * ram["used"] / ram["total"])}% | {round(1024 * ram["used"])} MB used | {round(1024 * ram["free"])} MB free | {round(1024 * ram["total"])} MB total',
|
||||
}
|
||||
chart = [load["memory"], load["gpu"]]
|
||||
devices.append({
|
||||
'name': torch.xpu.get_device_name(),
|
||||
'data': data,
|
||||
'chart': chart,
|
||||
})
|
||||
return devices
|
||||
except Exception as e:
|
||||
log.error(f'XPU SMI: {e}')
|
||||
return []
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
from rich import print as rprint
|
||||
for gpu in get_xpu_smi():
|
||||
rprint(gpu)
|
||||
@@ -108,7 +108,7 @@ def search_civitai(
|
||||
global models # pylint: disable=global-statement
|
||||
import requests
|
||||
from urllib.parse import urlencode
|
||||
install('bs4') # Ensure BeautifulSoup is installed
|
||||
install('beautifulsoup4')
|
||||
|
||||
if len(query) == 0:
|
||||
log.error('CivitAI: empty query')
|
||||
|
||||
@@ -27,7 +27,7 @@ class DepthAnythingDetector:
|
||||
PrepareForNet()])
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path: str, cache_dir: str) -> str:
|
||||
def from_pretrained(cls, pretrained_model_or_path: str, cache_dir: str, local_files_only=False) -> str:
|
||||
from modules.control.proc.depth_anything.dpt import DPT_DINOv2
|
||||
import huggingface_hub as hf
|
||||
model = (
|
||||
@@ -40,7 +40,7 @@ class DepthAnythingDetector:
|
||||
.to(devices.device)
|
||||
.eval()
|
||||
)
|
||||
model_path = hf.hf_hub_download(repo_id=pretrained_model_or_path, filename="pytorch_model.bin", cache_dir=cache_dir)
|
||||
model_path = hf.hf_hub_download(repo_id=pretrained_model_or_path, filename="pytorch_model.bin", cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
model_dict = torch.load(model_path)
|
||||
model.load_state_dict(model_dict)
|
||||
return cls(model)
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
import cv2
|
||||
import torch
|
||||
import torch.nn.functional as F
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from modules import devices, masking
|
||||
from modules.shared import opts
|
||||
|
||||
|
||||
class DepthProDetector:
|
||||
"""Apple DepthPro detector (aligned with Depth Anything style)."""
|
||||
|
||||
def __init__(self, model, processor):
|
||||
self.model = model
|
||||
self.processor = processor
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path: str = "apple/DepthPro-hf", cache_dir: str = None, local_files_only = False) -> "DepthProDetector":
|
||||
from transformers import AutoImageProcessor, DepthProForDepthEstimation
|
||||
|
||||
processor = AutoImageProcessor.from_pretrained(pretrained_model_or_path, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
model = DepthProForDepthEstimation.from_pretrained(
|
||||
pretrained_model_or_path,
|
||||
cache_dir=cache_dir,
|
||||
local_files_only=local_files_only,
|
||||
).to(devices.device).eval()
|
||||
return cls(model, processor)
|
||||
|
||||
def __call__(self, image, color_map: str = "none", output_type: str = "pil"):
|
||||
self.model.to(devices.device)
|
||||
if isinstance(image, Image.Image):
|
||||
image = np.array(image)
|
||||
h, w = image.shape[:2]
|
||||
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
|
||||
pil_image = Image.fromarray(image_rgb)
|
||||
|
||||
inputs = self.processor(images=pil_image, return_tensors="pt")
|
||||
inputs = {k: v.to(devices.device) if isinstance(v, torch.Tensor) else v for k, v in inputs.items()}
|
||||
|
||||
with devices.inference_context():
|
||||
outputs = self.model(**inputs)
|
||||
results = self.processor.post_process_depth_estimation(outputs, target_sizes=[(h, w)])
|
||||
depth_tensor = results[0]["predicted_depth"].to(devices.device, dtype=torch.float32)
|
||||
|
||||
if opts.control_move_processor:
|
||||
self.model.to("cpu")
|
||||
|
||||
depth_tensor = F.interpolate(depth_tensor[None, None], size=(h, w), mode="bilinear", align_corners=False)[0, 0]
|
||||
depth_tensor = 1.0 / torch.clamp(depth_tensor, min=1e-6)
|
||||
depth_tensor -= depth_tensor.min()
|
||||
depth_max = depth_tensor.max()
|
||||
if depth_max > 0:
|
||||
depth_tensor /= depth_max
|
||||
depth = (depth_tensor * 255.0).clamp(0, 255).to(torch.uint8).cpu().numpy()
|
||||
|
||||
if color_map != "none":
|
||||
colormap_key = color_map if color_map in masking.COLORMAP else "inferno"
|
||||
depth = cv2.applyColorMap(depth, masking.COLORMAP.index(colormap_key))[:, :, ::-1]
|
||||
if output_type == "pil":
|
||||
mode = "RGB" if depth.ndim == 3 else "L"
|
||||
depth = Image.fromarray(depth, mode=mode)
|
||||
return depth
|
||||
@@ -51,7 +51,7 @@ def check_dependencies():
|
||||
status = [installed(p, reload=False, quiet=True) for p in packages]
|
||||
debug(f'DWPose required={packages} status={status}')
|
||||
if not all(status):
|
||||
log.info(f'Installing DWPose dependencies: {packages}')
|
||||
log.info(f'Installing dependencies: for=dwpose packages={packages}')
|
||||
cmd = 'install --upgrade --no-deps --force-reinstall '
|
||||
pkgs = ' '.join(packages)
|
||||
pip(cmd + pkgs, ignore=False, quiet=True, uv=False)
|
||||
|
||||
@@ -60,12 +60,12 @@ class HEDdetector:
|
||||
self.model = model
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, cache_dir=None):
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, cache_dir=None, local_files_only=False):
|
||||
filename = filename or "ControlNetHED.pth"
|
||||
if os.path.isdir(pretrained_model_or_path):
|
||||
model_path = os.path.join(pretrained_model_or_path, filename)
|
||||
else:
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
model = ControlNetHED_Apache2()
|
||||
model.load_state_dict(torch.load(model_path, map_location='cpu'))
|
||||
model.float().eval()
|
||||
|
||||
@@ -20,13 +20,13 @@ class LeresDetector:
|
||||
self.pix2pixmodel = pix2pixmodel
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, pix2pix_filename=None, cache_dir=None):
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, pix2pix_filename=None, cache_dir=None, local_files_only=False):
|
||||
filename = filename or "res101.pth"
|
||||
pix2pix_filename = pix2pix_filename or "latest_net_G.pth"
|
||||
if os.path.isdir(pretrained_model_or_path):
|
||||
model_path = os.path.join(pretrained_model_or_path, filename)
|
||||
else:
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
checkpoint = torch.load(model_path, map_location=torch.device('cpu'))
|
||||
model = RelDepthModel(backbone='resnext101')
|
||||
model.load_state_dict(strip_prefix_if_present(checkpoint['depth_model'], "module."), strict=True)
|
||||
@@ -34,7 +34,7 @@ class LeresDetector:
|
||||
if os.path.isdir(pretrained_model_or_path):
|
||||
model_path = os.path.join(pretrained_model_or_path, pix2pix_filename)
|
||||
else:
|
||||
model_path = hf_hub_download(pretrained_model_or_path, pix2pix_filename, cache_dir=cache_dir)
|
||||
model_path = hf_hub_download(pretrained_model_or_path, pix2pix_filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
opt = TestOptions().parse()
|
||||
if not torch.cuda.is_available():
|
||||
opt.gpu_ids = [] # cpu mode
|
||||
|
||||
@@ -95,7 +95,7 @@ class LineartDetector:
|
||||
self.model_coarse = coarse_model
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, coarse_filename=None, cache_dir=None):
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, coarse_filename=None, cache_dir=None, local_files_only=False):
|
||||
filename = filename or "sk_model.pth"
|
||||
coarse_filename = coarse_filename or "sk_model2.pth"
|
||||
|
||||
@@ -103,8 +103,8 @@ class LineartDetector:
|
||||
model_path = os.path.join(pretrained_model_or_path, filename)
|
||||
coarse_model_path = os.path.join(pretrained_model_or_path, coarse_filename)
|
||||
else:
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
|
||||
coarse_model_path = hf_hub_download(pretrained_model_or_path, coarse_filename, cache_dir=cache_dir)
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
coarse_model_path = hf_hub_download(pretrained_model_or_path, coarse_filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
|
||||
model = Generator(3, 1, 3)
|
||||
model.load_state_dict(torch.load(model_path, map_location=torch.device('cpu')))
|
||||
|
||||
@@ -117,12 +117,12 @@ class LineartAnimeDetector:
|
||||
self.model = model
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, cache_dir=None):
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, cache_dir=None, local_files_only=False):
|
||||
filename = filename or "netG.pth"
|
||||
if os.path.isdir(pretrained_model_or_path):
|
||||
model_path = os.path.join(pretrained_model_or_path, filename)
|
||||
else:
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
norm_layer = functools.partial(nn.InstanceNorm2d, affine=False, track_running_stats=False)
|
||||
net = UnetGenerator(3, 1, 8, 64, norm_layer=norm_layer, use_dropout=False)
|
||||
ckpt = torch.load(model_path)
|
||||
|
||||
@@ -17,7 +17,7 @@ class MidasDetector:
|
||||
self.model = model
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path, model_type="dpt_hybrid", filename=None, cache_dir=None):
|
||||
def from_pretrained(cls, pretrained_model_or_path, model_type="dpt_hybrid", filename=None, cache_dir=None, local_files_only=False):
|
||||
if pretrained_model_or_path == "lllyasviel/ControlNet":
|
||||
filename = filename or "annotator/ckpts/dpt_hybrid-midas-501f0c75.pt"
|
||||
else:
|
||||
@@ -25,7 +25,7 @@ class MidasDetector:
|
||||
if os.path.isdir(pretrained_model_or_path):
|
||||
model_path = os.path.join(pretrained_model_or_path, filename)
|
||||
else:
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
model = MiDaSInference(model_type=model_type, model_path=model_path)
|
||||
return cls(model)
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ class MLSDdetector:
|
||||
self.model = model
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, cache_dir=None):
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, cache_dir=None, local_files_only=False):
|
||||
if pretrained_model_or_path == "lllyasviel/ControlNet":
|
||||
filename = filename or "annotator/ckpts/mlsd_large_512_fp32.pth"
|
||||
else:
|
||||
@@ -24,7 +24,7 @@ class MLSDdetector:
|
||||
if os.path.isdir(pretrained_model_or_path):
|
||||
model_path = os.path.join(pretrained_model_or_path, filename)
|
||||
else:
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
model = MobileV2_MLSD_Large()
|
||||
model.load_state_dict(torch.load(model_path), strict=True)
|
||||
model.eval()
|
||||
|
||||
@@ -33,12 +33,12 @@ class NormalBaeDetector:
|
||||
self.norm = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, cache_dir=None):
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, cache_dir=None, local_files_only=False):
|
||||
filename = filename or "scannet.pt"
|
||||
if os.path.isdir(pretrained_model_or_path):
|
||||
model_path = os.path.join(pretrained_model_or_path, filename)
|
||||
else:
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
args = types.SimpleNamespace()
|
||||
args.mode = 'client'
|
||||
args.architecture = 'BN'
|
||||
|
||||
@@ -76,7 +76,7 @@ class OpenposeDetector:
|
||||
self.face_estimation = face_estimation
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, hand_filename=None, face_filename=None, cache_dir=None):
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, hand_filename=None, face_filename=None, cache_dir=None, local_files_only=False):
|
||||
|
||||
if pretrained_model_or_path == "lllyasviel/ControlNet":
|
||||
filename = filename or "annotator/ckpts/body_pose_model.pth"
|
||||
@@ -96,9 +96,9 @@ class OpenposeDetector:
|
||||
hand_model_path = os.path.join(pretrained_model_or_path, hand_filename)
|
||||
face_model_path = os.path.join(face_pretrained_model_or_path, face_filename)
|
||||
else:
|
||||
body_model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
|
||||
hand_model_path = hf_hub_download(pretrained_model_or_path, hand_filename, cache_dir=cache_dir)
|
||||
face_model_path = hf_hub_download(face_pretrained_model_or_path, face_filename, cache_dir=cache_dir)
|
||||
body_model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
hand_model_path = hf_hub_download(pretrained_model_or_path, hand_filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
face_model_path = hf_hub_download(face_pretrained_model_or_path, face_filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
|
||||
body_estimation = Body(body_model_path)
|
||||
hand_estimation = Hand(hand_model_path)
|
||||
|
||||
@@ -16,12 +16,12 @@ class PidiNetDetector:
|
||||
self.model = model
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, cache_dir=None):
|
||||
def from_pretrained(cls, pretrained_model_or_path, filename=None, cache_dir=None, local_files_only=False):
|
||||
filename = filename or "table5_pidinet.pth"
|
||||
if os.path.isdir(pretrained_model_or_path):
|
||||
model_path = os.path.join(pretrained_model_or_path, filename)
|
||||
else:
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
model = pidinet()
|
||||
model.load_state_dict({k.replace('module.', ''): v for k, v in torch.load(model_path)['state_dict'].items()})
|
||||
model.eval()
|
||||
|
||||
@@ -23,12 +23,12 @@ class SamDetector:
|
||||
self.model = mask_generator
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, model_path, filename, model_type, cache_dir=None):
|
||||
def from_pretrained(cls, model_path, filename, model_type, cache_dir=None, local_files_only=False):
|
||||
"""
|
||||
Possible model_type : vit_h, vit_l, vit_b, vit_t
|
||||
download weights from https://github.com/facebookresearch/segment-anything
|
||||
"""
|
||||
model_path = hf_hub_download(model_path, filename, cache_dir=cache_dir)
|
||||
model_path = hf_hub_download(model_path, filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
sam = sam_model_registry[model_type](checkpoint=model_path)
|
||||
sam.to(devices.device)
|
||||
mask_generator = SamAutomaticMaskGenerator(sam)
|
||||
|
||||
@@ -20,12 +20,12 @@ class ZoeDetector:
|
||||
self.model = model
|
||||
|
||||
@classmethod
|
||||
def from_pretrained(cls, pretrained_model_or_path, model_type="zoedepth", filename=None, cache_dir=None):
|
||||
def from_pretrained(cls, pretrained_model_or_path, model_type="zoedepth", filename=None, cache_dir=None, local_files_only=False):
|
||||
filename = filename or "ZoeD_M12_N.pt"
|
||||
if os.path.isdir(pretrained_model_or_path):
|
||||
model_path = os.path.join(pretrained_model_or_path, filename)
|
||||
else:
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir)
|
||||
model_path = hf_hub_download(pretrained_model_or_path, filename, cache_dir=cache_dir, local_files_only=local_files_only)
|
||||
if model_type == "zoedepth":
|
||||
model_cls = ZoeDepth
|
||||
elif model_type == "zoedepth_nk":
|
||||
|
||||
@@ -34,6 +34,7 @@ processors = [
|
||||
'DPT Depth Hybrid',
|
||||
'GLPN Depth',
|
||||
'Depth Anything',
|
||||
'Depth Pro',
|
||||
]
|
||||
|
||||
|
||||
|
||||
@@ -39,6 +39,7 @@ config = {
|
||||
'DPT Depth Hybrid': {'class': None, 'checkpoint': False, 'params': {}},
|
||||
'GLPN Depth': {'class': None, 'checkpoint': False, 'params': {}},
|
||||
'Depth Anything': {'class': None, 'checkpoint': True, 'load_config': {'pretrained_model_or_path': 'LiheYoung/depth_anything_vitl14' }, 'params': { 'color_map': 'inferno' }},
|
||||
'Depth Pro': {'class': None, 'checkpoint': True, 'load_config': {'pretrained_model_or_path': 'apple/DepthPro-hf'}, 'params': {'color_map': 'inferno'}},
|
||||
# 'Midas Depth Large': {'class': MidasDetector, 'checkpoint': True, 'params': {'bg_th': 0.1, 'depth_and_normal': False}, 'load_config': {'pretrained_model_or_path': 'Intel/dpt-large', 'model_type': "dpt_large", 'filename': ''}},
|
||||
# 'Zoe Depth Zoe': {'class': ZoeDetector, 'checkpoint': True, 'params': {}},
|
||||
# 'Zoe Depth NK': {'class': ZoeDetector, 'checkpoint': True, 'params': {}, 'load_config': {'pretrained_model_or_path': 'halffried/gyre_zoedepth', 'filename': 'ZoeD_M12_NK.safetensors', 'model_type': "zoedepth_nk"}},
|
||||
@@ -67,6 +68,7 @@ def delay_load_config():
|
||||
from modules.control.proc.dpt import DPTDetector
|
||||
from modules.control.proc.glpn import GLPNDetector
|
||||
from modules.control.proc.depth_anything import DepthAnythingDetector
|
||||
from modules.control.proc.depth_pro import DepthProDetector
|
||||
config = {
|
||||
# placeholder
|
||||
'None': {},
|
||||
@@ -95,6 +97,7 @@ def delay_load_config():
|
||||
'DPT Depth Hybrid': {'class': DPTDetector, 'checkpoint': False, 'params': {}},
|
||||
'GLPN Depth': {'class': GLPNDetector, 'checkpoint': False, 'params': {}},
|
||||
'Depth Anything': {'class': DepthAnythingDetector, 'checkpoint': True, 'load_config': {'pretrained_model_or_path': 'LiheYoung/depth_anything_vitl14' }, 'params': { 'color_map': 'inferno' }},
|
||||
'Depth Pro': {'class': DepthProDetector, 'checkpoint': True, 'load_config': {'pretrained_model_or_path': 'apple/DepthPro-hf'}, 'params': {'color_map': 'inferno'}},
|
||||
# 'Midas Depth Large': {'class': MidasDetector, 'checkpoint': True, 'params': {'bg_th': 0.1, 'depth_and_normal': False}, 'load_config': {'pretrained_model_or_path': 'Intel/dpt-large', 'model_type': "dpt_large", 'filename': ''}},
|
||||
# 'Zoe Depth Zoe': {'class': ZoeDetector, 'checkpoint': True, 'params': {}},
|
||||
# 'Zoe Depth NK': {'class': ZoeDetector, 'checkpoint': True, 'params': {}, 'load_config': {'pretrained_model_or_path': 'halffried/gyre_zoedepth', 'filename': 'ZoeD_M12_NK.safetensors', 'model_type': "zoedepth_nk"}},
|
||||
@@ -155,6 +158,7 @@ def update_settings(*settings):
|
||||
update(['Marigold Depth', 'params', 'denoising_steps'], settings[25])
|
||||
update(['Marigold Depth', 'params', 'ensemble_size'], settings[26])
|
||||
update(['Depth Anything', 'params', 'color_map'], settings[27])
|
||||
update(['Depth Pro', 'params', 'color_map'], settings[28])
|
||||
|
||||
|
||||
class Processor():
|
||||
@@ -177,9 +181,15 @@ class Processor():
|
||||
self.model = None
|
||||
self.processor_id = processor_id
|
||||
devices.torch_gc(force=True, reason='processor')
|
||||
# self.override = None
|
||||
# devices.torch_gc()
|
||||
self.load_config = { 'cache_dir': cache_dir }
|
||||
from modules.shared import opts
|
||||
if opts.offline_mode:
|
||||
self.load_config["local_files_only"] = True
|
||||
os.environ['HF_HUB_OFFLINE'] = '1'
|
||||
else:
|
||||
os.environ.pop('HF_HUB_OFFLINE', None)
|
||||
os.unsetenv('HF_HUB_OFFLINE')
|
||||
|
||||
|
||||
def config(self, processor_id = None):
|
||||
if processor_id is not None:
|
||||
|
||||
+26
-6
@@ -53,9 +53,22 @@ def is_unified_model():
|
||||
return shared.sd_model.__class__.__name__ in unified_models
|
||||
|
||||
|
||||
def set_pipe(p, has_models, unit_type, selected_models, active_model, active_strength, control_conditioning, control_guidance_start, control_guidance_end, inits=None):
|
||||
def has_inputs(inputs):
|
||||
current = inputs or []
|
||||
current = current if isinstance(current, list) else [current]
|
||||
current = [input for input in current if input is not None]
|
||||
if current is None or len(current) == 0:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def set_pipe(p, has_models, unit_type, selected_models, active_model, active_strength, active_units, control_conditioning, control_guidance_start, control_guidance_end, inits=None, inputs=None):
|
||||
global pipe, instance # pylint: disable=global-statement
|
||||
pipe = None
|
||||
if has_models and not has_inputs(inits) and not has_inputs(inputs):
|
||||
if not any(has_inputs(u.override) for u in active_units if u.enabled): # check overrides
|
||||
shared.log.error('Control: no input images')
|
||||
return pipe
|
||||
if has_models:
|
||||
p.ops.append('control')
|
||||
p.extra_generation_params["Control type"] = unit_type # overriden later with pretty-print
|
||||
@@ -132,6 +145,7 @@ def check_active(p, unit_type, units):
|
||||
active_strength: List[float] = [] # strength factors for all active models
|
||||
active_start: List[float] = [] # start step for all active models
|
||||
active_end: List[float] = [] # end step for all active models
|
||||
active_units: List[unit.Unit] = [] # all active units
|
||||
num_units = 0
|
||||
for u in units:
|
||||
if u.type != unit_type:
|
||||
@@ -151,6 +165,7 @@ def check_active(p, unit_type, units):
|
||||
active_model.append(u.adapter)
|
||||
active_strength.append(float(u.strength))
|
||||
p.adapter_conditioning_factor = u.factor
|
||||
active_units.append(u)
|
||||
shared.log.debug(f'Control T2I-Adapter unit: i={num_units} process="{u.process.processor_id}" model="{u.adapter.model_id}" strength={u.strength} factor={u.factor}')
|
||||
elif unit_type == 'controlnet' and (u.controlnet.model is not None or is_unified_model()):
|
||||
active_process.append(u.process)
|
||||
@@ -159,6 +174,7 @@ def check_active(p, unit_type, units):
|
||||
active_start.append(float(u.start))
|
||||
active_end.append(float(u.end))
|
||||
p.guess_mode = u.guess
|
||||
active_units.append(u)
|
||||
if isinstance(u.mode, str):
|
||||
if not hasattr(p, 'control_mode'):
|
||||
p.control_mode = []
|
||||
@@ -173,11 +189,13 @@ def check_active(p, unit_type, units):
|
||||
active_strength.append(float(u.strength))
|
||||
active_start.append(float(u.start))
|
||||
active_end.append(float(u.end))
|
||||
active_units.append(u)
|
||||
shared.log.debug(f'Control ControlNet-XS unit: i={num_units} process={u.process.processor_id} model={u.controlnet.model_id} strength={u.strength} guess={u.guess} start={u.start} end={u.end}')
|
||||
elif unit_type == 'lite' and u.controlnet.model is not None:
|
||||
active_process.append(u.process)
|
||||
active_model.append(u.controlnet)
|
||||
active_strength.append(float(u.strength))
|
||||
active_units.append(u)
|
||||
shared.log.debug(f'Control ControlLLite unit: i={num_units} process={u.process.processor_id} model={u.controlnet.model_id} strength={u.strength} guess={u.guess} start={u.start} end={u.end}')
|
||||
elif unit_type == 'reference':
|
||||
p.override = u.override
|
||||
@@ -185,14 +203,16 @@ def check_active(p, unit_type, units):
|
||||
p.query_weight = float(u.query_weight)
|
||||
p.adain_weight = float(u.adain_weight)
|
||||
p.fidelity = u.fidelity
|
||||
active_units.append(u)
|
||||
shared.log.debug('Control Reference unit')
|
||||
else:
|
||||
if u.process.processor_id is not None:
|
||||
active_process.append(u.process)
|
||||
active_units.append(u)
|
||||
shared.log.debug(f'Control process unit: i={num_units} process={u.process.processor_id}')
|
||||
active_strength.append(float(u.strength))
|
||||
debug_log(f'Control active: process={len(active_process)} model={len(active_model)}')
|
||||
return active_process, active_model, active_strength, active_start, active_end
|
||||
return active_process, active_model, active_strength, active_start, active_end, active_units
|
||||
|
||||
|
||||
def check_enabled(p, unit_type, units, active_model, active_strength, active_start, active_end):
|
||||
@@ -258,7 +278,7 @@ def control_run(state: str = '', # pylint: disable=keyword-arg-before-vararg
|
||||
guidance_name: str = 'Default', guidance_scale: float = 6.0, guidance_rescale: float = 0.0, guidance_start: float = 0.0, guidance_stop: float = 1.0,
|
||||
cfg_scale: float = 6.0, clip_skip: float = 1.0, image_cfg_scale: float = 6.0, diffusers_guidance_rescale: float = 0.7, pag_scale: float = 0.0, pag_adaptive: float = 0.5, cfg_end: float = 1.0,
|
||||
vae_type: str = 'Full', tiling: bool = False, hidiffusion: bool = False,
|
||||
detailer_enabled: bool = True, detailer_prompt: str = '', detailer_negative: str = '', detailer_steps: int = 10, detailer_strength: float = 0.3, detailer_resolution: int = 1024,
|
||||
detailer_enabled: bool = False, detailer_prompt: str = '', detailer_negative: str = '', detailer_steps: int = 10, detailer_strength: float = 0.3, detailer_resolution: int = 1024,
|
||||
hdr_mode: int = 0, hdr_brightness: float = 0, hdr_color: float = 0, hdr_sharpen: float = 0, hdr_clamp: bool = False, hdr_boundary: float = 4.0, hdr_threshold: float = 0.95,
|
||||
hdr_maximize: bool = False, hdr_max_center: float = 0.6, hdr_max_boundary: float = 1.0, hdr_color_picker: str = None, hdr_tint_ratio: float = 0,
|
||||
resize_mode_before: int = 0, resize_name_before: str = 'None', resize_context_before: str = 'None', width_before: int = 512, height_before: int = 512, scale_by_before: float = 1.0, selected_scale_tab_before: int = 0,
|
||||
@@ -384,8 +404,8 @@ def control_run(state: str = '', # pylint: disable=keyword-arg-before-vararg
|
||||
outpath_grids=shared.opts.outdir_grids or shared.opts.outdir_control_grids,
|
||||
# overrides
|
||||
override_settings=extra
|
||||
|
||||
)
|
||||
|
||||
p.state = state
|
||||
p.is_tile = False
|
||||
p.init_control = inits or []
|
||||
@@ -419,7 +439,7 @@ def control_run(state: str = '', # pylint: disable=keyword-arg-before-vararg
|
||||
return [], '', '', 'Error: model not loaded'
|
||||
|
||||
unit_type = unit_type.strip().lower() if unit_type is not None else ''
|
||||
active_process, active_model, active_strength, active_start, active_end = check_active(p, unit_type, units)
|
||||
active_process, active_model, active_strength, active_start, active_end, active_units = check_active(p, unit_type, units)
|
||||
has_models, selected_models, control_conditioning, control_guidance_start, control_guidance_end = check_enabled(p, unit_type, units, active_model, active_strength, active_start, active_end)
|
||||
|
||||
image_txt = ''
|
||||
@@ -429,7 +449,7 @@ def control_run(state: str = '', # pylint: disable=keyword-arg-before-vararg
|
||||
if is_unified_model():
|
||||
p.init_images = inputs
|
||||
|
||||
pipe = set_pipe(p, has_models, unit_type, selected_models, active_model, active_strength, control_conditioning, control_guidance_start, control_guidance_end, inits)
|
||||
pipe = set_pipe(p, has_models, unit_type, selected_models, active_model, active_strength, active_units, control_conditioning, control_guidance_start, control_guidance_end, inits, inputs)
|
||||
debug_log(f'Control pipeline: class={pipe.__class__.__name__} args={vars(p)}')
|
||||
status = True
|
||||
frame = None
|
||||
|
||||
@@ -31,7 +31,7 @@ class Unit(): # mashup of gradio controls and mapping to actual implementation c
|
||||
self.choices = ['default']
|
||||
|
||||
def __str__(self):
|
||||
return f'Unit(index={self.index} enabled={self.enabled} type="{self.type}" strength={self.strength} start={self.start} end={self.end}{self.process}{self.controlnet})'
|
||||
return f'Unit(index={self.index} enabled={self.enabled} type="{self.type}" strength={self.strength} start={self.start} end={self.end}{self.process}{self.controlnet} override={self.override})'
|
||||
|
||||
def __init__(self,
|
||||
# values
|
||||
|
||||
@@ -89,7 +89,8 @@ predefined_f1 = {
|
||||
"Shakker-Labs Depth F1": 'Shakker-Labs/FLUX.1-dev-ControlNet-Depth',
|
||||
"XLabs-AI Canny F1": 'XLabs-AI/flux-controlnet-canny-diffusers',
|
||||
"XLabs-AI Depth F1": 'XLabs-AI/flux-controlnet-depth-diffusers',
|
||||
"XLabs-AI HED F1": 'XLabs-AI/flux-controlnet-hed-diffusers'
|
||||
"XLabs-AI HED F1": 'XLabs-AI/flux-controlnet-hed-diffusers',
|
||||
"LibreFlux Segment F1": 'neuralvfx/LibreFlux-ControlNet',
|
||||
}
|
||||
predefined_sd3 = {
|
||||
"StabilityAI Canny SD35": 'diffusers-internal-dev/sd35-controlnet-canny-8b',
|
||||
@@ -119,6 +120,15 @@ variants = {
|
||||
'NoobAI SoftEdge XL': 'fp16',
|
||||
'TTPlanet Tile Realistic XL': 'fp16',
|
||||
}
|
||||
|
||||
subfolders = {
|
||||
"LibreFlux Segment F1": 'controlnet',
|
||||
}
|
||||
|
||||
remote_code = {
|
||||
"LibreFlux Segment F1": True,
|
||||
}
|
||||
|
||||
models = {}
|
||||
all_models = {}
|
||||
all_models.update(predefined_sd15)
|
||||
@@ -205,6 +215,12 @@ class ControlNet():
|
||||
self.load_config = { 'cache_dir': cache_dir }
|
||||
if load_config is not None:
|
||||
self.load_config.update(load_config)
|
||||
if opts.offline_mode:
|
||||
self.load_config["local_files_only"] = True
|
||||
os.environ['HF_HUB_OFFLINE'] = '1'
|
||||
else:
|
||||
os.environ.pop('HF_HUB_OFFLINE', None)
|
||||
os.unsetenv('HF_HUB_OFFLINE')
|
||||
if model_id is not None:
|
||||
self.load()
|
||||
|
||||
@@ -279,7 +295,7 @@ class ControlNet():
|
||||
self.load_config['original_config_file '] = config_path
|
||||
self.model = cls.from_single_file(model_path, config=config, **self.load_config)
|
||||
|
||||
def load(self, model_id: str = None, force: bool = True) -> str:
|
||||
def load(self, model_id: str = None, force: bool = False) -> str:
|
||||
with load_lock:
|
||||
try:
|
||||
t0 = time.time()
|
||||
@@ -320,6 +336,10 @@ class ControlNet():
|
||||
self.load_config['use_safetensors'] = True
|
||||
if variants.get(model_id, None) is not None:
|
||||
kwargs['variant'] = variants[model_id]
|
||||
if subfolders.get(model_id, None) is not None:
|
||||
kwargs['subfolder'] = subfolders[model_id]
|
||||
if remote_code.get(model_id, None) is not None:
|
||||
kwargs['trust_remote_code'] = remote_code[model_id]
|
||||
try:
|
||||
self.model = cls.from_pretrained(model_path, **self.load_config, **kwargs)
|
||||
except Exception as e:
|
||||
@@ -332,6 +352,8 @@ class ControlNet():
|
||||
self.model.offload_never = True
|
||||
if self.dtype is not None:
|
||||
self.model.to(self.dtype)
|
||||
if self.device is not None:
|
||||
self.model.to_empty(device=self.device) # model could be sparse
|
||||
if "Control" in opts.sdnq_quantize_weights:
|
||||
try:
|
||||
log.debug(f'Control {what} model SDNQ Compress: id="{model_id}"')
|
||||
@@ -356,7 +378,7 @@ class ControlNet():
|
||||
except Exception as e:
|
||||
log.error(f'Control {what} model Torch AO: id="{model_id}" {e}')
|
||||
if self.device is not None:
|
||||
self.model.to(self.device)
|
||||
sd_models.move_model(self.model, self.device)
|
||||
if "Control" in opts.cuda_compile:
|
||||
try:
|
||||
from modules.sd_models_compile import compile_torch
|
||||
@@ -365,9 +387,9 @@ class ControlNet():
|
||||
log.warning(f"Control compile error: {e}")
|
||||
t1 = time.time()
|
||||
self.model_id = model_id
|
||||
log.info(f'Control {what} model loaded: id="{model_id}" path="{model_path}" cls={cls.__name__} time={t1-t0:.2f}')
|
||||
log.info(f'Control {what} model loaded: id="{self.model_id}" path="{model_path}" cls={cls.__name__} time={t1-t0:.2f}')
|
||||
state.end(jobid)
|
||||
return f'{what} loaded model: {model_id}'
|
||||
return f'{what} loaded model: {self.model_id}'
|
||||
except Exception as e:
|
||||
log.error(f'Control {what} model load: id="{model_id}" {e}')
|
||||
errors.display(e, f'Control {what} load')
|
||||
@@ -507,7 +529,7 @@ class ControlNetPipeline():
|
||||
if opts.diffusers_offload_mode == 'none':
|
||||
sd_models.move_model(self.pipeline, devices.device)
|
||||
sd_models.clear_caches()
|
||||
sd_models.set_diffuser_offload(self.pipeline, 'model')
|
||||
sd_models.set_diffuser_offload(self.pipeline, 'model', force=True)
|
||||
|
||||
t1 = time.time()
|
||||
debug_log(f'Control {what} pipeline: class={self.pipeline.__class__.__name__} time={t1-t0:.2f}')
|
||||
|
||||
@@ -108,8 +108,15 @@ class ControlLLLite():
|
||||
self.model = ControlNetLLLite(model_path)
|
||||
else:
|
||||
import huggingface_hub as hf
|
||||
offline_config = {}
|
||||
if opts.offline_mode:
|
||||
offline_config["local_files_only"] = True
|
||||
os.environ['HF_HUB_OFFLINE'] = '1'
|
||||
else:
|
||||
os.environ.pop('HF_HUB_OFFLINE', None)
|
||||
os.unsetenv('HF_HUB_OFFLINE')
|
||||
folder, filename = os.path.split(model_path)
|
||||
model_path = hf.hf_hub_download(repo_id=folder, filename=f'{filename}.safetensors', cache_dir=cache_dir)
|
||||
model_path = hf.hf_hub_download(repo_id=folder, filename=f'{filename}.safetensors', cache_dir=cache_dir, **offline_config)
|
||||
self.model = ControlNetLLLite(model_path)
|
||||
if self.device is not None:
|
||||
self.model.to(self.device)
|
||||
|
||||
@@ -2,7 +2,7 @@ import os
|
||||
import time
|
||||
from typing import Union
|
||||
import threading
|
||||
from diffusers import pipelines, StableDiffusionPipeline, StableDiffusionXLPipeline, T2IAdapter, MultiAdapter, StableDiffusionAdapterPipeline, StableDiffusionXLAdapterPipeline # pylint: disable=unused-import
|
||||
from diffusers import StableDiffusionPipeline, StableDiffusionXLPipeline, T2IAdapter, MultiAdapter, StableDiffusionAdapterPipeline, StableDiffusionXLAdapterPipeline # pylint: disable=unused-import
|
||||
from installer import log
|
||||
from modules import errors, sd_models
|
||||
from modules.control.units import detect
|
||||
@@ -104,6 +104,13 @@ class Adapter():
|
||||
return
|
||||
model_path, model_args = all_models[model_id]
|
||||
self.load_config.update(model_args)
|
||||
from modules.shared import opts
|
||||
if opts.offline_mode:
|
||||
self.load_config["local_files_only"] = True
|
||||
os.environ['HF_HUB_OFFLINE'] = '1'
|
||||
else:
|
||||
os.environ.pop('HF_HUB_OFFLINE', None)
|
||||
os.unsetenv('HF_HUB_OFFLINE')
|
||||
if model_path is None:
|
||||
log.error(f'Control {what} model load failed: id="{model_id}" error=unknown model id')
|
||||
return
|
||||
@@ -168,6 +175,7 @@ class AdapterPipeline():
|
||||
adapter=adapter,
|
||||
)
|
||||
sd_models.move_model(self.pipeline, pipeline.device)
|
||||
sd_models.apply_balanced_offload(self.pipeline, force=True)
|
||||
elif detect.is_sd15(pipeline):
|
||||
self.pipeline = StableDiffusionAdapterPipeline(
|
||||
vae=pipeline.vae,
|
||||
@@ -181,6 +189,7 @@ class AdapterPipeline():
|
||||
adapter=adapter,
|
||||
)
|
||||
sd_models.move_model(self.pipeline, pipeline.device)
|
||||
sd_models.apply_balanced_offload(self.pipeline, force=True)
|
||||
else:
|
||||
log.error(f'Control {what} pipeline: class={pipeline.__class__.__name__} unsupported model type')
|
||||
return
|
||||
|
||||
@@ -100,6 +100,12 @@ class ControlNetXS():
|
||||
# log.debug(f'Control {what} model: id="{model_id}" path="{model_path}" already loaded')
|
||||
return
|
||||
self.load_config['time_embedding_mix'] = time_embedding_mix
|
||||
if opts.offline_mode:
|
||||
self.load_config["local_files_only"] = True
|
||||
os.environ['HF_HUB_OFFLINE'] = '1'
|
||||
else:
|
||||
os.environ.pop('HF_HUB_OFFLINE', None)
|
||||
os.unsetenv('HF_HUB_OFFLINE')
|
||||
log.debug(f'Control {what} model loading: id="{model_id}" path="{model_path}" {self.load_config}')
|
||||
if model_path.endswith('.safetensors'):
|
||||
self.model = ControlNetXSModel.from_single_file(model_path, **self.load_config)
|
||||
@@ -140,6 +146,7 @@ class ControlNetXSPipeline():
|
||||
controlnet=controlnet, # can be a list
|
||||
)
|
||||
sd_models.move_model(self.pipeline, pipeline.device)
|
||||
sd_models.apply_balanced_offload(self.pipeline, force=True)
|
||||
elif detect.is_sd15(pipeline):
|
||||
self.pipeline = StableDiffusionControlNetXSPipeline(
|
||||
vae=pipeline.vae,
|
||||
@@ -153,6 +160,7 @@ class ControlNetXSPipeline():
|
||||
controlnet=controlnet, # can be a list
|
||||
)
|
||||
sd_models.move_model(self.pipeline, pipeline.device)
|
||||
sd_models.apply_balanced_offload(self.pipeline, force=True)
|
||||
else:
|
||||
log.error(f'Control {what} pipeline: class={pipeline.__class__.__name__} unsupported model type')
|
||||
return
|
||||
|
||||
@@ -55,7 +55,7 @@ def nms(x, t, s):
|
||||
for f in [f1, f2, f3, f4]:
|
||||
np.putmask(y, cv2.dilate(x, kernel=f) == x, x)
|
||||
z = np.zeros_like(y, dtype=np.uint8)
|
||||
z[y > t] = 255
|
||||
z[y > t] = 255 # pylint: disable=unsupported-assignment-operation
|
||||
return z
|
||||
|
||||
def min_max_norm(x):
|
||||
|
||||
+43
-8
@@ -21,6 +21,7 @@ cpu = torch.device("cpu")
|
||||
|
||||
fp16_ok = None # set once by test_fp16
|
||||
bf16_ok = None # set once by test_bf16
|
||||
triton_ok = None # set once by test_triton
|
||||
|
||||
backend = None # set by get_backend
|
||||
device = None # set by get_optimal_device
|
||||
@@ -64,12 +65,10 @@ def has_zluda() -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def has_triton() -> bool:
|
||||
try:
|
||||
from torch.utils._triton import has_triton as torch_has_triton
|
||||
return torch_has_triton()
|
||||
except Exception:
|
||||
return False
|
||||
def has_triton(early:bool=False) -> bool:
|
||||
if triton_ok is not None:
|
||||
return triton_ok
|
||||
return test_triton(early=early)
|
||||
|
||||
|
||||
def get_backend(shared_cmd_opts):
|
||||
@@ -382,6 +381,36 @@ def test_bf16():
|
||||
return bf16_ok
|
||||
|
||||
|
||||
def test_triton(early: bool = False):
|
||||
global triton_ok # pylint: disable=global-statement
|
||||
if triton_ok is not None and early:
|
||||
return triton_ok
|
||||
t0 = time.time()
|
||||
try:
|
||||
from torch.utils._triton import has_triton as torch_has_triton
|
||||
if torch_has_triton():
|
||||
if early:
|
||||
return True
|
||||
def test_triton_func(a,b,c):
|
||||
return a * b + c
|
||||
test_triton_func = torch.compile(test_triton_func, fullgraph=True)
|
||||
test_triton_func(torch.randn(32, device=device), torch.randn(32, device=device), torch.randn(32, device=device))
|
||||
triton_ok = True
|
||||
else:
|
||||
triton_ok = False
|
||||
except Exception as e:
|
||||
triton_ok = False
|
||||
log.warning(f"Triton test fail: {e}")
|
||||
from modules import errors
|
||||
errors.display(e, 'Triton')
|
||||
t1 = time.time()
|
||||
fn = f'{sys._getframe(2).f_code.co_name}:{sys._getframe(1).f_code.co_name}' # pylint: disable=protected-access
|
||||
log.debug(f'Triton: pass={triton_ok} fn={fn} time={t1-t0:.2f}')
|
||||
if not triton_ok and opts is not None:
|
||||
opts.sdnq_dequantize_compile = False
|
||||
return triton_ok
|
||||
|
||||
|
||||
def set_cudnn_params():
|
||||
if not cuda_ok:
|
||||
return
|
||||
@@ -542,6 +571,7 @@ def set_sdpa_params():
|
||||
except Exception as err:
|
||||
log.error(f'Torch attention: type="sage attention" {err}')
|
||||
|
||||
|
||||
from importlib.metadata import version
|
||||
try:
|
||||
flash = version('flash-attn')
|
||||
@@ -551,7 +581,11 @@ def set_sdpa_params():
|
||||
sage = version('sageattention')
|
||||
except Exception:
|
||||
sage = False
|
||||
log.info(f'Torch attention: flashattn={flash} sageattention={sage}')
|
||||
log.debug(f'Torch attention installed: flashattn={flash} sageattention={sage}')
|
||||
|
||||
from diffusers.models import attention_dispatch as a
|
||||
log.debug(f'Torch attention status: flash={a._CAN_USE_FLASH_ATTN} flash3={a._CAN_USE_FLASH_ATTN_3} aiter={a._CAN_USE_AITER_ATTN} sage={a._CAN_USE_SAGE_ATTN} flex={a._CAN_USE_FLEX_ATTN} npu={a._CAN_USE_NPU_ATTN} xla={a._CAN_USE_XLA_ATTN} xformers={a._CAN_USE_XFORMERS_ATTN}') # pylint: disable=protected-access
|
||||
|
||||
except Exception as e:
|
||||
log.warning(f'Torch SDPA: {e}')
|
||||
|
||||
@@ -614,6 +648,7 @@ def set_cuda_params():
|
||||
set_cudnn_params()
|
||||
set_sdpa_params()
|
||||
set_dtype()
|
||||
test_triton()
|
||||
if backend == 'openvino':
|
||||
from modules.intel.openvino import get_device as get_raw_openvino_device
|
||||
device_name = get_raw_openvino_device()
|
||||
@@ -624,7 +659,7 @@ def set_cuda_params():
|
||||
tunable = [torch.cuda.tunable.is_enabled(), torch.cuda.tunable.tuning_is_enabled()]
|
||||
except Exception:
|
||||
tunable = [False, False]
|
||||
log.info(f'Torch parameters: backend={backend} device={device_name} config={opts.cuda_dtype} dtype={dtype} context={inference_context.__name__} nohalf={opts.no_half} nohalfvae={opts.no_half_vae} upcast={opts.upcast_sampling} deterministic={opts.cudnn_deterministic} tunable={tunable} fp16={"pass" if fp16_ok else "fail"} bf16={"pass" if bf16_ok else "fail"} optimization="{opts.cross_attention_optimization}"')
|
||||
log.info(f'Torch parameters: backend={backend} device={device_name} config={opts.cuda_dtype} dtype={dtype} context={inference_context.__name__} nohalf={opts.no_half} nohalfvae={opts.no_half_vae} upcast={opts.upcast_sampling} deterministic={opts.cudnn_deterministic} tunable={tunable} fp16={"pass" if fp16_ok else "fail"} bf16={"pass" if bf16_ok else "fail"} triton={"pass" if triton_ok else "fail"} optimization="{opts.cross_attention_optimization}"')
|
||||
|
||||
|
||||
def randn(seed, shape=None):
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
# https://github.com/somanchiu/ReSwapper/blob/GAN/Image.py
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
|
||||
### https://github.com/somanchiu/ReSwapper/blob/GAN/Image.py
|
||||
|
||||
input_std = 255.0
|
||||
input_mean = 0.0
|
||||
|
||||
@@ -38,7 +37,7 @@ def blend_swapped_image(swapped_face, target_image, M):
|
||||
warped_face = cv2.warpAffine(swapped_face, M_inv, (w, h),borderValue=0.0)
|
||||
img_white = np.full((swapped_face.shape[0], swapped_face.shape[1]), 255, dtype=np.float32)
|
||||
img_mask = cv2.warpAffine(img_white, M_inv, (w, h), borderValue=0.0)
|
||||
img_mask[img_mask > 20] = 255
|
||||
img_mask[img_mask > 20] = 255 # pylint: disable=unsupported-assignment-operation
|
||||
mask_h_inds, mask_w_inds = np.where(img_mask == 255)
|
||||
if len(mask_h_inds) > 0 and len(mask_w_inds) > 0: # safety check
|
||||
mask_h = np.max(mask_h_inds) - np.min(mask_h_inds)
|
||||
|
||||
@@ -21,7 +21,7 @@ class ReqFramepack(BaseModel):
|
||||
resolution: Optional[int] = Field(default=640, title="Resolution", description="Resolution of the image")
|
||||
duration: Optional[float] = Field(default=4, title="Duration", description="Duration of the video in seconds")
|
||||
latent_ws: Optional[int] = Field(default=9, title="Latent window size", description="Size of the latent window")
|
||||
steps: Optional[int] = Field(default=25, title="Steps", description="Number of steps for the model")
|
||||
steps: Optional[int] = Field(default=25, title="Video steps", description="Number of steps for the video generation")
|
||||
cfg_scale: Optional[float] = Field(default=1.0, title="CFG scale", description="CFG scale for the model")
|
||||
cfg_distilled: Optional[float] = Field(default=10.0, title="Distilled CFG scale", description="Distilled CFG scale for the model")
|
||||
cfg_rescale: Optional[float] = Field(default=0.0, title="CFG re-scale", description="CFG re-scale for the model")
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import os
|
||||
import time
|
||||
from modules import shared, devices, errors, sd_models, sd_checkpoint, model_quant
|
||||
|
||||
@@ -128,23 +129,31 @@ def load_model(variant:str=None, pipeline:str=None, text_encoder:str=None, text_
|
||||
sd_models.hf_auth_check(model["text_encoder"]["repo"])
|
||||
sd_models.hf_auth_check(model["text_encoder_2"]["repo"])
|
||||
|
||||
offline_config = {}
|
||||
if shared.opts.offline_mode:
|
||||
offline_config["local_files_only"] = True
|
||||
os.environ['HF_HUB_OFFLINE'] = '1'
|
||||
else:
|
||||
os.environ.pop('HF_HUB_OFFLINE', None)
|
||||
os.unsetenv('HF_HUB_OFFLINE')
|
||||
|
||||
shared.log.debug(f'FramePack load: module=llm {model["text_encoder"]}')
|
||||
load_args, quant_args = model_quant.get_dit_args({}, module='TE', device_map=True)
|
||||
text_encoder = LlamaModel.from_pretrained(model["text_encoder"]["repo"], subfolder=model["text_encoder"]["subfolder"], cache_dir=shared.opts.hfcache_dir, **load_args, **quant_args)
|
||||
tokenizer = LlamaTokenizerFast.from_pretrained(model["tokenizer"]["repo"], subfolder=model["tokenizer"]["subfolder"], cache_dir=shared.opts.hfcache_dir)
|
||||
text_encoder = LlamaModel.from_pretrained(model["text_encoder"]["repo"], subfolder=model["text_encoder"]["subfolder"], cache_dir=shared.opts.hfcache_dir, **load_args, **quant_args, **offline_config)
|
||||
tokenizer = LlamaTokenizerFast.from_pretrained(model["tokenizer"]["repo"], subfolder=model["tokenizer"]["subfolder"], cache_dir=shared.opts.hfcache_dir, **offline_config)
|
||||
text_encoder.requires_grad_(False)
|
||||
text_encoder.eval()
|
||||
sd_models.move_model(text_encoder, devices.cpu)
|
||||
|
||||
shared.log.debug(f'FramePack load: module=te {model["text_encoder_2"]}')
|
||||
text_encoder_2 = CLIPTextModel.from_pretrained(model["text_encoder_2"]["repo"], subfolder=model["text_encoder_2"]["subfolder"], torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir)
|
||||
tokenizer_2 = CLIPTokenizer.from_pretrained(model["pipeline"]["repo"], subfolder='tokenizer_2', cache_dir=shared.opts.hfcache_dir)
|
||||
text_encoder_2 = CLIPTextModel.from_pretrained(model["text_encoder_2"]["repo"], subfolder=model["text_encoder_2"]["subfolder"], torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir, **offline_config)
|
||||
tokenizer_2 = CLIPTokenizer.from_pretrained(model["pipeline"]["repo"], subfolder='tokenizer_2', cache_dir=shared.opts.hfcache_dir, **offline_config)
|
||||
text_encoder_2.requires_grad_(False)
|
||||
text_encoder_2.eval()
|
||||
sd_models.move_model(text_encoder_2, devices.cpu)
|
||||
|
||||
shared.log.debug(f'FramePack load: module=vae {model["vae"]}')
|
||||
vae = AutoencoderKLHunyuanVideo.from_pretrained(model["vae"]["repo"], subfolder=model["vae"]["subfolder"], torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir)
|
||||
vae = AutoencoderKLHunyuanVideo.from_pretrained(model["vae"]["repo"], subfolder=model["vae"]["subfolder"], torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir, **offline_config)
|
||||
vae.requires_grad_(False)
|
||||
vae.eval()
|
||||
vae.enable_slicing()
|
||||
@@ -152,8 +161,8 @@ def load_model(variant:str=None, pipeline:str=None, text_encoder:str=None, text_
|
||||
sd_models.move_model(vae, devices.cpu)
|
||||
|
||||
shared.log.debug(f'FramePack load: module=encoder {model["feature_extractor"]} model={model["image_encoder"]}')
|
||||
feature_extractor = SiglipImageProcessor.from_pretrained(model["feature_extractor"]["repo"], subfolder=model["feature_extractor"]["subfolder"], cache_dir=shared.opts.hfcache_dir)
|
||||
image_encoder = SiglipVisionModel.from_pretrained(model["image_encoder"]["repo"], subfolder=model["image_encoder"]["subfolder"], torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir)
|
||||
feature_extractor = SiglipImageProcessor.from_pretrained(model["feature_extractor"]["repo"], subfolder=model["feature_extractor"]["subfolder"], cache_dir=shared.opts.hfcache_dir, **offline_config)
|
||||
image_encoder = SiglipVisionModel.from_pretrained(model["image_encoder"]["repo"], subfolder=model["image_encoder"]["subfolder"], torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir, **offline_config)
|
||||
image_encoder.requires_grad_(False)
|
||||
image_encoder.eval()
|
||||
sd_models.move_model(image_encoder, devices.cpu)
|
||||
@@ -161,7 +170,7 @@ def load_model(variant:str=None, pipeline:str=None, text_encoder:str=None, text_
|
||||
shared.log.debug(f'FramePack load: module=transformer {model["transformer"]}')
|
||||
dit_repo = model["transformer"]["repo"]
|
||||
load_args, quant_args = model_quant.get_dit_args({}, module='Model', device_map=True)
|
||||
transformer = HunyuanVideoTransformer3DModelPacked.from_pretrained(dit_repo, subfolder=model["transformer"]["subfolder"], cache_dir=shared.opts.hfcache_dir, **load_args, **quant_args)
|
||||
transformer = HunyuanVideoTransformer3DModelPacked.from_pretrained(dit_repo, subfolder=model["transformer"]["subfolder"], cache_dir=shared.opts.hfcache_dir, **load_args, **quant_args, **offline_config)
|
||||
transformer.high_quality_fp32_output_for_inference = False
|
||||
transformer.requires_grad_(False)
|
||||
transformer.eval()
|
||||
|
||||
@@ -308,12 +308,26 @@ def worker(
|
||||
if is_last_section:
|
||||
break
|
||||
|
||||
total_generated_frames, _video_filename = save_video(history_pixels, mp4_fps, mp4_codec, mp4_opt, mp4_ext, mp4_sf, mp4_video, mp4_frames, mp4_interpolate, pbar=pbar, stream=stream, metadata=metadata)
|
||||
total_generated_frames, _video_filename = save_video(
|
||||
None,
|
||||
history_pixels,
|
||||
mp4_fps,
|
||||
mp4_codec,
|
||||
mp4_opt,
|
||||
mp4_ext,
|
||||
mp4_sf,
|
||||
mp4_video,
|
||||
mp4_frames,
|
||||
mp4_interpolate,
|
||||
pbar=pbar,
|
||||
stream=stream,
|
||||
metadata=metadata,
|
||||
)
|
||||
|
||||
except AssertionError:
|
||||
shared.log.info('FramePack: interrupted')
|
||||
if shared.opts.keep_incomplete:
|
||||
save_video(history_pixels, mp4_fps, mp4_codec, mp4_opt, mp4_ext, mp4_sf, mp4_video, mp4_frames, mp4_interpolate=0, stream=stream, metadata=metadata)
|
||||
save_video(None, history_pixels, mp4_fps, mp4_codec, mp4_opt, mp4_ext, mp4_sf, mp4_video, mp4_frames, mp4_interpolate=0, stream=stream, metadata=metadata)
|
||||
except Exception as e:
|
||||
shared.log.error(f'FramePack: {e}')
|
||||
errors.display(e, 'FramePack')
|
||||
|
||||
@@ -5,6 +5,7 @@ import threading
|
||||
import numpy as np
|
||||
import torch
|
||||
import gradio as gr
|
||||
from PIL import Image
|
||||
from modules import shared, processing, timer, paths, extra_networks, progress, ui_video_vlm
|
||||
from modules.video_models.video_utils import check_av
|
||||
from modules.framepack import framepack_install # pylint: disable=wrong-import-order
|
||||
@@ -27,6 +28,8 @@ def prepare_image(image, resolution):
|
||||
(416, 960), (448, 864), (480, 832), (512, 768), (544, 704), (576, 672), (608, 640),
|
||||
(640, 608), (672, 576), (704, 544), (768, 512), (832, 480), (864, 448), (960, 416),
|
||||
]
|
||||
if isinstance(image, Image.Image):
|
||||
image = np.array(image)
|
||||
h, w, _c = image.shape
|
||||
min_metric = float('inf')
|
||||
scale_factor = resolution / 640.0
|
||||
|
||||
@@ -1,15 +1,14 @@
|
||||
import os
|
||||
import cv2
|
||||
import json
|
||||
import random
|
||||
import glob
|
||||
import datetime
|
||||
import torch
|
||||
import einops
|
||||
import cv2
|
||||
import numpy as np
|
||||
import datetime
|
||||
import torchvision
|
||||
import safetensors.torch as sf
|
||||
from PIL import Image
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
|
||||
|
||||
def min_resize(x, m):
|
||||
@@ -30,7 +29,7 @@ def min_resize(x, m):
|
||||
|
||||
|
||||
def d_resize(x, y):
|
||||
H, W, C = y.shape
|
||||
H, W, _C = y.shape
|
||||
new_min = min(H, W)
|
||||
raw_min = min(x.shape[0], x.shape[1])
|
||||
if new_min < raw_min:
|
||||
@@ -50,7 +49,7 @@ def resize_and_center_crop(image, target_width, target_height):
|
||||
scale_factor = max(target_width / original_width, target_height / original_height)
|
||||
resized_width = int(round(original_width * scale_factor))
|
||||
resized_height = int(round(original_height * scale_factor))
|
||||
resized_image = pil_image.resize((resized_width, resized_height), Image.LANCZOS)
|
||||
resized_image = pil_image.resize((resized_width, resized_height), Image.Resampling.LANCZOS)
|
||||
left = (resized_width - target_width) / 2
|
||||
top = (resized_height - target_height) / 2
|
||||
right = (resized_width + target_width) / 2
|
||||
@@ -60,7 +59,7 @@ def resize_and_center_crop(image, target_width, target_height):
|
||||
|
||||
|
||||
def resize_and_center_crop_pytorch(image, target_width, target_height):
|
||||
B, C, H, W = image.shape
|
||||
_B, _C, H, W = image.shape
|
||||
|
||||
if H == target_height and W == target_width:
|
||||
return image
|
||||
@@ -83,7 +82,7 @@ def resize_without_crop(image, target_width, target_height):
|
||||
return image
|
||||
|
||||
pil_image = Image.fromarray(image)
|
||||
resized_image = pil_image.resize((target_width, target_height), Image.LANCZOS)
|
||||
resized_image = pil_image.resize((target_width, target_height), Image.Resampling.LANCZOS)
|
||||
return np.array(resized_image)
|
||||
|
||||
|
||||
@@ -188,7 +187,7 @@ def supress_lower_channels(m, k, alpha=0.01):
|
||||
|
||||
def freeze_module(m):
|
||||
if not hasattr(m, '_forward_inside_frozen_module'):
|
||||
m._forward_inside_frozen_module = m.forward
|
||||
m._forward_inside_frozen_module = m.forward # pylint: disable=protected-access
|
||||
m.requires_grad_(False)
|
||||
m.forward = torch.no_grad()(m.forward)
|
||||
return m
|
||||
@@ -243,7 +242,7 @@ def soft_append_bcthw(history, current, overlap=0):
|
||||
|
||||
|
||||
def save_bcthw_as_mp4(x, output_filename, fps=10, crf=0):
|
||||
b, c, t, h, w = x.shape
|
||||
b, _c, _t, _h, _w = x.shape
|
||||
|
||||
per_row = b
|
||||
for p in [6, 5, 4, 3, 2]:
|
||||
@@ -297,8 +296,6 @@ def add_tensors_with_padding(tensor1, tensor2):
|
||||
|
||||
|
||||
def visualize_txt_as_img(width, height, text, font_path='font/DejaVuSans.ttf', size=18):
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
|
||||
txt = Image.new("RGB", (width, height), color="white")
|
||||
draw = ImageDraw.Draw(txt)
|
||||
font = ImageFont.truetype(font_path, size=size)
|
||||
|
||||
@@ -3,7 +3,6 @@ import io
|
||||
import os
|
||||
from PIL import Image
|
||||
import gradio as gr
|
||||
from modules.paths import params_path
|
||||
from modules import shared, gr_tempdir, script_callbacks, images
|
||||
from modules.infotext import parse, mapping, quote, unquote # pylint: disable=unused-import
|
||||
|
||||
@@ -204,6 +203,7 @@ def create_override_settings_dict(text_pairs):
|
||||
def connect_paste(button, local_paste_fields, input_comp, override_settings_component, tabname):
|
||||
|
||||
def paste_func(prompt):
|
||||
from modules.paths import params_path
|
||||
if prompt is None or len(prompt.strip()) == 0:
|
||||
if os.path.exists(params_path):
|
||||
with open(params_path, "r", encoding="utf8") as file:
|
||||
@@ -252,7 +252,8 @@ def connect_paste(button, local_paste_fields, input_comp, override_settings_comp
|
||||
val = valtype(v)
|
||||
res.append(gr.update(value=val))
|
||||
applied[key] = val
|
||||
except Exception:
|
||||
except Exception as e:
|
||||
shared.log.error(f'Paste param: key="{key}" value="{v}" error="{e}"')
|
||||
res.append(gr.update())
|
||||
list_applied = [{k: v} for k, v in applied.items() if not callable(v) and not callable(k)]
|
||||
shared.log.debug(f"Prompt restore: apply={list_applied} skip={skipped}")
|
||||
|
||||
@@ -157,6 +157,10 @@ def save_image(image,
|
||||
if image is None:
|
||||
shared.log.warning('Image is none')
|
||||
return None, None, None
|
||||
if isinstance(image, list):
|
||||
if len(image) > 1:
|
||||
shared.log.warning(f'Save: images={image} multiple images provided only the first one will be saved')
|
||||
image = image[0]
|
||||
if not check_grid_size([image]):
|
||||
return None, None, None
|
||||
if path is None or path == '': # set default path to avoid errors when functions are triggered manually or via api and param is not set
|
||||
|
||||
@@ -13,7 +13,11 @@ def check_grid_size(imgs):
|
||||
return False
|
||||
mp = 0
|
||||
for img in imgs:
|
||||
mp += img.width * img.height if img is not None else 0
|
||||
if isinstance(img, list):
|
||||
for im in img:
|
||||
mp += im.width * im.height if im is not None else 0
|
||||
else:
|
||||
mp += img.width * img.height if img is not None else 0
|
||||
mp = round(mp / 1000000)
|
||||
ok = mp <= shared.opts.img_max_size_mp
|
||||
if not ok:
|
||||
|
||||
+32
-15
@@ -1,6 +1,7 @@
|
||||
import re
|
||||
import os
|
||||
import time
|
||||
import unicodedata
|
||||
import uuid
|
||||
import string
|
||||
import hashlib
|
||||
@@ -23,8 +24,8 @@ NOTHING = object()
|
||||
|
||||
class FilenameGenerator:
|
||||
replacements = {
|
||||
'width': lambda self: self.image.width,
|
||||
'height': lambda self: self.image.height,
|
||||
'width': lambda self: self.width,
|
||||
'height': lambda self: self.height,
|
||||
'batch_number': lambda self: self.batch_number,
|
||||
'iter_number': lambda self: self.iter_number,
|
||||
'num': lambda self: NOTHING if self.p.n_iter == 1 and self.p.batch_size == 1 else self.p.iteration * self.p.batch_size + self.p.batch_index + 1,
|
||||
@@ -32,8 +33,8 @@ class FilenameGenerator:
|
||||
'date': lambda self: datetime.datetime.now().strftime('%Y-%m-%d'),
|
||||
'datetime': lambda self, *args: self.datetime(*args), # accepts formats: [datetime], [datetime<Format>], [datetime<Format><Time Zone>]
|
||||
'hasprompt': lambda self, *args: self.hasprompt(*args), # accepts formats:[hasprompt<prompt1|default><prompt2>..]
|
||||
'hash': lambda self: self.image_hash(),
|
||||
'image_hash': lambda self: self.image_hash(),
|
||||
'hash': lambda self: self.image_hash() if self.image is not None else '',
|
||||
'image_hash': lambda self: self.image_hash() if self.image is not None else '',
|
||||
'timestamp': lambda self: getattr(self.p, "job_timestamp", shared.state.job_timestamp),
|
||||
'epoch': lambda self: int(time.time()),
|
||||
'job_timestamp': lambda self: getattr(self.p, "job_timestamp", shared.state.job_timestamp),
|
||||
@@ -44,6 +45,8 @@ class FilenameGenerator:
|
||||
'model_type': lambda self: shared.sd_model_type if shared.sd_loaded else '',
|
||||
'model_hash': lambda self: shared.sd_model.sd_checkpoint_info.shorthash if shared.sd_loaded and getattr(shared.sd_model, 'sd_checkpoint_info', None) is not None else '',
|
||||
|
||||
'lora': lambda self: self.p and getattr(self.p, 'extra_generation_params', {}).get('LoRA networks', ''),
|
||||
|
||||
'prompt': lambda self: self.prompt_full(),
|
||||
'prompt_no_styles': lambda self: self.prompt_no_style(),
|
||||
'prompt_words': lambda self: self.prompt_words(),
|
||||
@@ -61,27 +64,29 @@ class FilenameGenerator:
|
||||
}
|
||||
default_time_format = '%Y%m%d%H%M%S'
|
||||
|
||||
def __init__(self, p, seed, prompt, image, grid=False):
|
||||
def __init__(self, p, seed, prompt, image=None, grid=False, width=None, height=None):
|
||||
if p is None:
|
||||
debug('Filename generator init skip')
|
||||
else:
|
||||
debug(f'Filename generator init: {seed} {prompt}')
|
||||
debug(f'Filename generator init: seed={seed} prompt="{prompt}"')
|
||||
self.p = p
|
||||
if seed is not None and int(seed) > 0:
|
||||
self.seed = seed
|
||||
elif p is not None and hasattr(p, 'all_seeds'):
|
||||
self.seed = p.all_seeds[0]
|
||||
elif p is not None and hasattr(p, 'seeds'):
|
||||
self.seed = p.seeds[0]
|
||||
elif p is not None and getattr(p, 'all_seeds', None) is not None and len(p.all_seeds) > 0:
|
||||
self.seed = p.all_seeds[0] if int(p.all_seeds[0]) > 0 else 0
|
||||
elif p is not None and getattr(p, 'seeds', None) is not None and len(p.seeds) > 0:
|
||||
self.seed = p.seeds[0] if int(p.seeds[0]) > 0 else 0
|
||||
else:
|
||||
self.seed = p.seed if p is not None else 0
|
||||
self.seed = p.seed if p is not None and getattr(p, 'seed', 0) > 0 else 0
|
||||
if prompt is not None:
|
||||
self.prompt = prompt
|
||||
else:
|
||||
self.prompt = p.prompt if p is not None else ''
|
||||
self.prompt = p.prompt if p is not None and getattr(p, 'prompt', '') != '' else ''
|
||||
if isinstance(self.prompt, list):
|
||||
self.prompt = ' '.join(self.prompt)
|
||||
self.image = image
|
||||
self.image = image[0] if isinstance(image, list) and len(image) > 0 else image
|
||||
self.width = width if width is not None else (self.image.width if self.image is not None else (p.width if p is not None else 0))
|
||||
self.height = height if height is not None else (self.image.height if self.image is not None else (p.height if p is not None else 0))
|
||||
if not grid:
|
||||
self.batch_number = NOTHING if self.p is None or getattr(self.p, 'batch_size', 1) == 1 else (self.p.batch_index + 1 if hasattr(self.p, 'batch_index') else NOTHING)
|
||||
self.iter_number = NOTHING if self.p is None or getattr(self.p, 'n_iter', 1) == 1 else (self.p.iteration + 1 if hasattr(self.p, 'iteration') else NOTHING)
|
||||
@@ -162,14 +167,25 @@ class FilenameGenerator:
|
||||
return sanitized
|
||||
|
||||
def sanitize(self, filename):
|
||||
invalid_chars = '\'"|?*\n\t\r' # <https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file>
|
||||
# starting reference: <https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file>
|
||||
invalid_chars = (
|
||||
"#<>\"'`" # ASCII quote and backtick
|
||||
"’‚‛\u2018\u2019\u201B" # smart single quotes and variants # noqa: RUF001
|
||||
"\u02BB" # modifier letter turned comma
|
||||
"\u201C\u201D\u201F" # smart double quotes and variants
|
||||
"|?*^%$\u00A0\u2013\u2014\n\t\r" # pipes, wildcards, percent, currency, NBSP, dashes, control chars
|
||||
)
|
||||
invalid_folder = ':'
|
||||
invalid_files = ['CON', 'PRN', 'AUX', 'NUL', 'NULL', 'COM0', 'COM1', 'LPT0', 'LPT1']
|
||||
invalid_prefix = ', '
|
||||
invalid_suffix = '.,_ '
|
||||
fn, ext = os.path.splitext(filename)
|
||||
fn, ext = os.path.splitext(unicodedata.normalize('NFKC', filename))
|
||||
fn = fn.strip()
|
||||
ext = ext.strip()
|
||||
parts = Path(fn).parts
|
||||
newparts = []
|
||||
# for ch in filename:
|
||||
# print(repr(ch), hex(ord(ch)), unicodedata.name(ch, 'UNKNOWN'), ch in invalid_chars)
|
||||
for i, part in enumerate(parts):
|
||||
part = part.translate({ ord(x): '_' for x in invalid_chars })
|
||||
if i > 0 or (len(part) >= 2 and part[1] != invalid_folder): # skip drive, otherwise remove
|
||||
@@ -179,6 +195,7 @@ class FilenameGenerator:
|
||||
[part := part.replace(word, '_') for word in invalid_files] # pylint: disable=expression-not-assigned
|
||||
newparts.append(part)
|
||||
fn = str(Path(*newparts))
|
||||
fn = fn.replace(' ', ' ').strip()
|
||||
max_length = max(256 - len(ext), os.statvfs(__file__).f_namemax - 32 if hasattr(os, 'statvfs') else 256 - len(ext))
|
||||
while len(os.path.abspath(fn)) > max_length:
|
||||
fn = fn[:-1]
|
||||
|
||||
@@ -87,6 +87,7 @@ def parse(infotext):
|
||||
|
||||
mapping = [
|
||||
# Models
|
||||
('Model', 'sd_model_checkpoint'),
|
||||
('Model hash', 'sd_model_checkpoint'),
|
||||
('Refiner', 'sd_model_refiner'),
|
||||
('VAE', 'sd_vae'),
|
||||
|
||||
@@ -108,12 +108,8 @@ def ipex_init(): # pylint: disable=too-many-statements
|
||||
torch.cuda.BoolStorage = torch.xpu.BoolStorage
|
||||
torch.cuda.ComplexFloatStorage = torch.xpu.ComplexFloatStorage
|
||||
torch.cuda.ComplexDoubleStorage = torch.xpu.ComplexDoubleStorage
|
||||
|
||||
torch._C._cuda_getCurrentRawStream = ipex._C._getCurrentRawStream
|
||||
ipex._C._DeviceProperties.multi_processor_count = ipex._C._DeviceProperties.gpu_subslice_count
|
||||
ipex._C._DeviceProperties.major = 12
|
||||
ipex._C._DeviceProperties.minor = 1
|
||||
ipex._C._DeviceProperties.L2_cache_size = 16*1024*1024 # A770 and A750
|
||||
if has_ipex:
|
||||
torch._C._cuda_getCurrentRawStream = ipex._C._getCurrentRawStream
|
||||
else:
|
||||
torch.cuda._initialization_lock = torch.xpu._initialization_lock
|
||||
torch.cuda._initialized = torch.xpu._initialized
|
||||
@@ -123,12 +119,7 @@ def ipex_init(): # pylint: disable=too-many-statements
|
||||
torch.cuda._tls = torch.xpu._tls
|
||||
torch.cuda.threading = torch.xpu.threading
|
||||
torch.cuda.traceback = torch.xpu.traceback
|
||||
|
||||
torch._C._cuda_getCurrentRawStream = torch._C._xpu_getCurrentRawStream
|
||||
torch._C._XpuDeviceProperties.multi_processor_count = torch._C._XpuDeviceProperties.gpu_subslice_count
|
||||
torch._C._XpuDeviceProperties.major = 12
|
||||
torch._C._XpuDeviceProperties.minor = 1
|
||||
torch._C._XpuDeviceProperties.L2_cache_size = 16*1024*1024 # A770 and A750
|
||||
|
||||
if torch_version[0] < 2 or (torch_version[0] == 2 and torch_version[1] < 5):
|
||||
torch.cuda.os = torch.xpu.os
|
||||
@@ -207,9 +198,6 @@ def ipex_init(): # pylint: disable=too-many-statements
|
||||
torch.version.cuda = "12.1"
|
||||
torch.cuda.get_arch_list = getattr(torch.xpu, "get_arch_list", lambda: ["pvc", "dg2", "ats-m150"])
|
||||
torch.cuda.get_device_capability = lambda *args, **kwargs: (12,1)
|
||||
torch.cuda.get_device_properties.major = 12
|
||||
torch.cuda.get_device_properties.minor = 1
|
||||
torch.cuda.get_device_properties.L2_cache_size = 16*1024*1024 # A770 and A750
|
||||
torch.cuda.ipc_collect = lambda *args, **kwargs: None
|
||||
torch.cuda.utilization = lambda *args, **kwargs: 0
|
||||
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
|
||||
|
||||
def mb_to_byte(mb: int) -> int:
|
||||
return mb * 1024*1024
|
||||
|
||||
unknown_cache_size = 2
|
||||
cache_size_dict = {
|
||||
0x0000: mb_to_byte(unknown_cache_size),
|
||||
0xE212: mb_to_byte(4), # Arc Pro B50 / Xe2
|
||||
0xE211: mb_to_byte(18), # Arc Pro B60 / Xe2
|
||||
0xE20B: mb_to_byte(18), # Arc B580 / Xe2
|
||||
0xE20C: mb_to_byte(18), # Arc B570 / Xe2
|
||||
0x64A0: mb_to_byte(4), # Arc 130V Mobile / Arc 140V Mobile / Lunar Lake / Xe2
|
||||
0x6420: mb_to_byte(unknown_cache_size*2), # (?) (EU: 64/56) / Lunar Lake / Xe2
|
||||
0x64B0: mb_to_byte(unknown_cache_size), # (?) (EU: 32) / Lunar Lake / Xe2
|
||||
0x7D51: mb_to_byte(4), # Arc 130T Mobile / Arc 140T Mobile / Arrow Lake-H / Xe-LPG
|
||||
0x7D67: mb_to_byte(unknown_cache_size*2), # (?) (EU: 64/48/32) / Arrow Lake-S / Xe-LPG
|
||||
0x7D41: mb_to_byte(unknown_cache_size*2), # (?) (EU: 64) / Arrow Lake-U / Xe-LPG
|
||||
0x7DD5: mb_to_byte(unknown_cache_size*2), # (?) (EU: 128/112) / Meteor Lake / Xe-LPG
|
||||
0x7D45: mb_to_byte(unknown_cache_size), # (?) (EU: 64/48) / Meteor Lake / Xe-LPG
|
||||
0x7D40: mb_to_byte(unknown_cache_size), # (?) (EU: 64/48) / Meteor Lake / Xe-LPG
|
||||
0x7D55: mb_to_byte(unknown_cache_size*2), # (?) (EU: 128/112) / Meteor Lake / Xe-LPG
|
||||
0x0BD5: mb_to_byte(408), # Max 1550 / Xe-HPC
|
||||
0x0BDA: mb_to_byte(204) , # Max 1100 / Xe-HPC
|
||||
0x56C0: mb_to_byte(16), # Flex 170 / Xe-HPG
|
||||
0x56C1: mb_to_byte(4), # Flex 140 / Xe-HPG
|
||||
0x5690: mb_to_byte(16), # Arc A770M / Xe-HPG
|
||||
0x5691: mb_to_byte(12), # Arc A730M / Xe-HPG
|
||||
0x5696: mb_to_byte(8), # Arc A570M / Xe-HPG
|
||||
0x5692: mb_to_byte(8), # Arc A550M / Xe-HPG
|
||||
0x5697: mb_to_byte(8), # Arc A530M / Xe-HPG
|
||||
0x5693: mb_to_byte(4), # Arc A370M / Xe-HPG
|
||||
0x5694: mb_to_byte(4), # Arc A350M / Xe-HPG
|
||||
0x56A0: mb_to_byte(16), # Arc A770 / Xe-HPG
|
||||
0x56A1: mb_to_byte(16), # Arc A750 / Xe-HPG
|
||||
0x56A2: mb_to_byte(8), # Arc A580 / Xe-HPG
|
||||
0x56A5: mb_to_byte(4), # Arc A380 / Xe-HPG
|
||||
0x56A6: mb_to_byte(4), # Arc A310 / Xe-HPG
|
||||
0x56B3: mb_to_byte(12), # Arc Pro A60 / Xe-HPG
|
||||
0x56B2: mb_to_byte(8), # Arc Pro A60M / Xe-HPG
|
||||
0x56B1: mb_to_byte(4), # Arc Pro A40/A50 / Xe-HPG
|
||||
0x56B0: mb_to_byte(4), # Arc Pro A30M / Xe-HPG
|
||||
0x56BA: mb_to_byte(unknown_cache_size*2), # Arc A380E / Xe-HPG
|
||||
0x56BC: mb_to_byte(unknown_cache_size*2), # Arc A370E / Xe-HPG
|
||||
0x56BD: mb_to_byte(unknown_cache_size*2), # Arc A350E / Xe-HPG
|
||||
0x56BB: mb_to_byte(unknown_cache_size*2), # Arc A310E / Xe-HPG
|
||||
}
|
||||
@@ -5,6 +5,7 @@ import torch
|
||||
import numpy as np
|
||||
from modules import devices
|
||||
|
||||
from .device_prop import cache_size_dict
|
||||
|
||||
torch_version = torch.__version__[:4]
|
||||
if torch_version[-1] not in {"0", "1", "2", "3", "4", "5", "6", "7", "8", "9"}:
|
||||
@@ -310,6 +311,28 @@ def torch_cuda_set_device(device):
|
||||
torch.xpu.set_device(device)
|
||||
|
||||
|
||||
@wraps(torch.cuda.get_device_properties)
|
||||
def get_device_properties(device=None):
|
||||
device_prop = torch.xpu.get_device_properties(device)
|
||||
new_keys = {
|
||||
"major": 12,
|
||||
"minor": 1,
|
||||
"multi_processor_count": device_prop.gpu_subslice_count,
|
||||
}
|
||||
if not hasattr(device_prop, "L2_cache_size"):
|
||||
new_keys["L2_cache_size"] = cache_size_dict.get(getattr(device_prop, "device_id", 0x56A0), cache_size_dict[0x0000])
|
||||
return DeviceProperties(device_prop, new_keys)
|
||||
|
||||
|
||||
class DeviceProperties():
|
||||
def __init__(self, device_prop, new_keys):
|
||||
for key in dir(device_prop):
|
||||
if not key.startswith("__"):
|
||||
setattr(self, key, getattr(device_prop, key))
|
||||
for key, value in new_keys.items():
|
||||
setattr(self, key, value)
|
||||
|
||||
|
||||
# torch.Generator has to be a class for isinstance checks
|
||||
original_torch_Generator = torch.Generator
|
||||
class torch_Generator(original_torch_Generator):
|
||||
@@ -343,6 +366,7 @@ def ipex_hijacks():
|
||||
torch.cuda.synchronize = torch_cuda_synchronize
|
||||
torch.cuda.device = torch_cuda_device
|
||||
torch.cuda.set_device = torch_cuda_set_device
|
||||
torch.cuda.get_device_properties = get_device_properties
|
||||
|
||||
torch.Generator = torch_Generator
|
||||
torch._C.Generator = torch_Generator
|
||||
|
||||
+14
-10
@@ -14,27 +14,27 @@ processor = None
|
||||
model = None
|
||||
loaded: str = None
|
||||
quant_args = None
|
||||
vlm_default = "Alibaba Qwen 2.5 VL 4B"
|
||||
vlm_default = "Alibaba Qwen 2.5 VL 3B"
|
||||
vlm_models = {
|
||||
"Google Gemma 3 4B": "google/gemma-3-4b-it",
|
||||
"Google Gemma 3n E2B": "google/gemma-3n-E2B-it", # 1.5GB
|
||||
"Google Gemma 3n E4B": "google/gemma-3n-E4B-it", # 1.5GB
|
||||
"Alibaba Qwen 2.0 VL 2B": "Qwen/Qwen2-VL-2B-Instruct",
|
||||
"Alibaba Qwen 2.5 Omni 3B": "Qwen/Qwen2.5-Omni-3B",
|
||||
"Alibaba Qwen 2.5 VL 4B": "Qwen/Qwen2.5-VL-3B-Instruct",
|
||||
"Alibaba Qwen 2.5 VL 3B": "Qwen/Qwen2.5-VL-3B-Instruct",
|
||||
"Alibaba Qwen 3 VL 2B": "Qwen/Qwen3-VL-2B-Instruct",
|
||||
"Alibaba Qwen 3 VL 2B Thinking": "Qwen/Qwen3-VL-2B-Thinking",
|
||||
"Alibaba Qwen 3 VL 4B": "Qwen/Qwen3-VL-4B-Instruct",
|
||||
"Alibaba Qwen 3 VL 4B Thinking": "Qwen/Qwen3-VL-4B-Thinking",
|
||||
"Alibaba Qwen 3 VL 8B": "Qwen/Qwen3-VL-8B-Instruct",
|
||||
"Alibaba Qwen 3 VL 8B Thinking": "Qwen/Qwen3-VL-8B-Thinking",
|
||||
"Huggingface Smol VL2 0.5B": "HuggingFaceTB/SmolVLM-500M-Instruct",
|
||||
"Huggingface Smol VL2 2B": "HuggingFaceTB/SmolVLM-Instruct",
|
||||
"Apple FastVLM 0.5B": "apple/FastVLM-0.5B",
|
||||
"Apple FastVLM 1.5B": "apple/FastVLM-1.5B",
|
||||
"Apple FastVLM 7B": "apple/FastVLM-7B",
|
||||
# "Microsoft Florence 2 Base": "microsoft/Florence-2-base-ft", # 0.5GB
|
||||
# "Microsoft Florence 2 Large": "microsoft/Florence-2-large-ft", # 1.5GB
|
||||
"Microsoft Florence 2 Base": "florence-community/Florence-2-base-ft", # 0.5GB
|
||||
"Microsoft Florence 2 Large": "florence-community/Florence-2-large-ft", # 1.5GB
|
||||
#"MiaoshouAI PromptGen 1.5 Base": "MiaoshouAI/Florence-2-base-PromptGen-v1.5@c06a5f02cc6071a5d65ee5d294cf3732d3097540", # 1.1GB
|
||||
#"MiaoshouAI PromptGen 1.5 Large": "MiaoshouAI/Florence-2-large-PromptGen-v1.5@28a42440e39c9c32b83f7ae74ec2b3d1540404f0", # 3.3GB
|
||||
#"MiaoshouAI PromptGen 2.0 Base": "MiaoshouAI/Florence-2-base-PromptGen-v2.0", # 1.1GB
|
||||
#"MiaoshouAI PromptGen 2.0 Large": "MiaoshouAI/Florence-2-large-PromptGen-v2.0", # 3.3GB
|
||||
"MiaoshouAI PromptGen 1.5 Base": "Disty0/Florence-2-base-PromptGen-v1.5", # 0.5GB
|
||||
"MiaoshouAI PromptGen 1.5 Large": "Disty0/Florence-2-large-PromptGen-v1.5", # 1.5GB
|
||||
"MiaoshouAI PromptGen 2.0 Base": "Disty0/Florence-2-base-PromptGen-v2.0", # 0.5GB
|
||||
@@ -181,10 +181,14 @@ def qwen(question: str, image: Image.Image, repo: str = None, system_prompt: str
|
||||
if model is None or loaded != repo:
|
||||
shared.log.debug(f'Interrogate load: vlm="{repo}"')
|
||||
model = None
|
||||
if '2.5' in repo:
|
||||
if 'Qwen3-VL' in repo or 'Qwen3VL' in repo:
|
||||
cls_name = transformers.Qwen3VLForConditionalGeneration
|
||||
elif 'Qwen2.5-VL' in repo or 'Qwen2_5_VL' in repo:
|
||||
cls_name = transformers.Qwen2_5_VLForConditionalGeneration
|
||||
else:
|
||||
elif 'Qwen2-VL' in repo or 'Qwen2VL' in repo:
|
||||
cls_name = transformers.Qwen2VLForConditionalGeneration
|
||||
else:
|
||||
cls_name = transformers.AutoModelForCausalLM
|
||||
model = cls_name.from_pretrained(
|
||||
repo,
|
||||
torch_dtype=devices.dtype,
|
||||
|
||||
@@ -190,14 +190,15 @@ def load_image_encoder(pipe: diffusers.DiffusionPipeline, adapter_names: list[st
|
||||
if pipe.image_encoder is None or clip_loaded != f'{clip_repo}/{clip_subfolder}':
|
||||
jobid = shared.state.begin('Load encoder')
|
||||
try:
|
||||
offline_config = { 'local_files_only': True } if shared.opts.offline_mode else {}
|
||||
if shared.sd_model_type == 'sd3':
|
||||
image_encoder = transformers.SiglipVisionModel.from_pretrained(clip_repo, torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir)
|
||||
image_encoder = transformers.SiglipVisionModel.from_pretrained(clip_repo, torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir, **offline_config)
|
||||
else:
|
||||
if clip_subfolder is None:
|
||||
image_encoder = transformers.CLIPVisionModelWithProjection.from_pretrained(clip_repo, torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir, use_safetensors=True)
|
||||
image_encoder = transformers.CLIPVisionModelWithProjection.from_pretrained(clip_repo, torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir, use_safetensors=True, **offline_config)
|
||||
shared.log.debug(f'IP adapter load: encoder="{clip_repo}" cls={pipe.image_encoder.__class__.__name__}')
|
||||
else:
|
||||
image_encoder = transformers.CLIPVisionModelWithProjection.from_pretrained(clip_repo, subfolder=clip_subfolder, torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir, use_safetensors=True)
|
||||
image_encoder = transformers.CLIPVisionModelWithProjection.from_pretrained(clip_repo, subfolder=clip_subfolder, torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir, use_safetensors=True, **offline_config)
|
||||
shared.log.debug(f'IP adapter load: encoder="{clip_repo}/{clip_subfolder}" cls={pipe.image_encoder.__class__.__name__}')
|
||||
sd_models.clear_caches()
|
||||
image_encoder = model_quant.do_post_load_quant(image_encoder, allow=True)
|
||||
@@ -220,8 +221,9 @@ def load_feature_extractor(pipe):
|
||||
if pipe.feature_extractor is None:
|
||||
try:
|
||||
jobid = shared.state.begin('Load extractor')
|
||||
offline_config = { 'local_files_only': True } if shared.opts.offline_mode else {}
|
||||
if shared.sd_model_type == 'sd3':
|
||||
feature_extractor = transformers.SiglipImageProcessor.from_pretrained(SIGLIP_ID, torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir)
|
||||
feature_extractor = transformers.SiglipImageProcessor.from_pretrained(SIGLIP_ID, torch_dtype=devices.dtype, cache_dir=shared.opts.hfcache_dir, **offline_config)
|
||||
else:
|
||||
feature_extractor = transformers.CLIPImageProcessor()
|
||||
if hasattr(pipe, 'register_modules'):
|
||||
@@ -343,6 +345,8 @@ def apply(pipe, p: processing.StableDiffusionProcessing, adapter_names=[], adapt
|
||||
kwargs['weight_name'] = names if len(names) > 1 else names[0]
|
||||
if len(revisions) > 0:
|
||||
kwargs['revision'] = revisions[0]
|
||||
if shared.opts.offline_mode:
|
||||
kwargs["local_files_only"] = True
|
||||
pipe.load_ip_adapter(repos, **kwargs)
|
||||
adapters_loaded = names
|
||||
if hasattr(p, 'ip_adapter_layers'):
|
||||
|
||||
+8
-3
@@ -129,6 +129,8 @@ import tqdm as tqdm_lib # pylint: disable=C0411
|
||||
from tqdm.rich import tqdm # pylint: disable=W0611,C0411
|
||||
|
||||
try:
|
||||
logging.getLogger("diffusers.guiders").setLevel(logging.ERROR)
|
||||
logging.getLogger("diffusers.loaders.single_file").setLevel(logging.ERROR)
|
||||
import diffusers.utils.import_utils # pylint: disable=W0611,C0411
|
||||
diffusers.utils.import_utils._k_diffusion_available = True # pylint: disable=protected-access # monkey-patch since we use k-diffusion from git
|
||||
diffusers.utils.import_utils._k_diffusion_version = '0.0.12' # pylint: disable=protected-access
|
||||
@@ -136,7 +138,6 @@ try:
|
||||
import diffusers # pylint: disable=W0611,C0411
|
||||
import diffusers.loaders.single_file # pylint: disable=W0611,C0411
|
||||
diffusers.loaders.single_file.logging.tqdm = partial(tqdm, unit='C')
|
||||
logging.getLogger("diffusers.loaders.single_file").setLevel(logging.ERROR)
|
||||
timer.startup.record("diffusers")
|
||||
except Exception as e:
|
||||
errors.log.error(f'Loader: diffusers=={diffusers.__version__ if "diffusers" in sys.modules else None} {e}')
|
||||
@@ -144,6 +145,7 @@ except Exception as e:
|
||||
sys.exit(1)
|
||||
|
||||
import huggingface_hub # pylint: disable=W0611,C0411
|
||||
logging.getLogger("huggingface_hub.file_download").setLevel(logging.ERROR)
|
||||
timer.startup.record("hfhub")
|
||||
|
||||
try:
|
||||
@@ -154,6 +156,9 @@ from PIL import Image # pylint: disable=W0611,C0411
|
||||
timer.startup.record("pillow")
|
||||
|
||||
|
||||
import cv2 # pylint: disable=W0611,C0411
|
||||
timer.startup.record("cv2")
|
||||
|
||||
class _tqdm_cls():
|
||||
def __call__(self, *args, **kwargs):
|
||||
bar_format = 'Progress {rate_fmt}{postfix} {bar} {percentage:3.0f}% {n_fmt}/{total_fmt} {elapsed} {remaining} ' + '\x1b[38;5;71m' + '{desc}' + '\x1b[0m'
|
||||
@@ -184,7 +189,7 @@ def get_packages():
|
||||
try:
|
||||
import math
|
||||
cores = os.cpu_count()
|
||||
affinity = len(os.sched_getaffinity(0))
|
||||
affinity = len(os.sched_getaffinity(0)) # pylint: disable=no-member
|
||||
threads = torch.get_num_threads()
|
||||
if threads < (affinity / 2):
|
||||
torch.set_num_threads(math.floor(affinity / 2))
|
||||
@@ -224,4 +229,4 @@ class VersionString(str): # support both string and tuple for version check
|
||||
|
||||
torch.__version__ = VersionString(torch.__version__)
|
||||
errors.log.info(f'Torch: torch=={torch.__version__} torchvision=={torchvision.__version__}')
|
||||
errors.log.info(f'Packages: diffusers=={diffusers.__version__} transformers=={transformers.__version__} accelerate=={accelerate.__version__} gradio=={gradio.__version__} pydantic=={pydantic.__version__} numpy=={np.__version__}')
|
||||
errors.log.info(f'Packages: diffusers=={diffusers.__version__} transformers=={transformers.__version__} accelerate=={accelerate.__version__} gradio=={gradio.__version__} pydantic=={pydantic.__version__} numpy=={np.__version__} cv2=={cv2.__version__}')
|
||||
|
||||
@@ -132,6 +132,13 @@ def parse(p, params_list, step=0):
|
||||
lora_module.append('transformer_2')
|
||||
if params.named.get('module', None) is not None:
|
||||
lora_module.append(params.named['module'].lower())
|
||||
|
||||
if len(lora_module) == 0 and shared.sd_loaded:
|
||||
if hasattr(shared.sd_model, 'transformer') and (shared.sd_model.transformer is not None) and hasattr(shared.sd_model, 'transformer_2') and (shared.sd_model.transformer_2 is None):
|
||||
lora_module.append('transformer')
|
||||
if hasattr(shared.sd_model, 'transformer') and (shared.sd_model.transformer is None) and hasattr(shared.sd_model, 'transformer_2') and (shared.sd_model.transformer_2 is not None):
|
||||
lora_module.append('transformer_2')
|
||||
|
||||
lora_modules.append(lora_module)
|
||||
|
||||
return names, te_multipliers, unet_multipliers, dyn_dims, lora_modules
|
||||
|
||||
@@ -61,6 +61,7 @@ def load_diffusers(name: str, network_on_disk: network.NetworkOnDisk, lora_scale
|
||||
try:
|
||||
if lora_module is not None and isinstance(lora_module, list) and len(lora_module) > 0:
|
||||
name = load_per_module(sd_model, network_on_disk.filename, adapter_name=name, lora_modules=lora_module)
|
||||
sd_model._lora_partial = True # pylint: disable=protected-access
|
||||
else:
|
||||
sd_model.load_lora_weights(network_on_disk.filename, adapter_name=name)
|
||||
except Exception as e:
|
||||
|
||||
@@ -13,6 +13,9 @@ available_network_aliases = {}
|
||||
forbidden_network_aliases = {}
|
||||
available_network_hash_lookup = {}
|
||||
dump_lora_keys = os.environ.get('SD_LORA_DUMP', None) is not None
|
||||
exclude_errors = [
|
||||
"'ChronoEditTransformer3DModel'",
|
||||
]
|
||||
|
||||
|
||||
def lora_dump(lora, dct):
|
||||
@@ -285,7 +288,8 @@ def network_load(names, te_multipliers=None, unet_multipliers=None, dyn_dims=Non
|
||||
shared.log.trace(f'Network load: type=LoRA active={sd_model.get_active_adapters()}')
|
||||
sd_model.set_adapters(adapter_names=lora_diffusers.diffuser_loaded, adapter_weights=lora_diffusers.diffuser_scales)
|
||||
except Exception as e:
|
||||
shared.log.error(f'Network load: type=LoRA action=set {e}')
|
||||
if str(e) not in exclude_errors:
|
||||
shared.log.error(f'Network load: type=LoRA action=strength {str(e)}')
|
||||
if l.debug:
|
||||
errors.display(e, 'LoRA')
|
||||
try:
|
||||
@@ -294,7 +298,7 @@ def network_load(names, te_multipliers=None, unet_multipliers=None, dyn_dims=Non
|
||||
sd_model.unload_lora_weights()
|
||||
l.timer.activate += time.time() - t1
|
||||
except Exception as e:
|
||||
shared.log.error(f'Network load: type=LoRA action=fuse {e}')
|
||||
shared.log.error(f'Network load: type=LoRA action=fuse {str(e)}')
|
||||
if l.debug:
|
||||
errors.display(e, 'LoRA')
|
||||
|
||||
|
||||
@@ -31,16 +31,19 @@ force_models_diffusers = [ # forced always
|
||||
'h1',
|
||||
'kandinsky',
|
||||
'hunyuandit',
|
||||
'hunyuanimage',
|
||||
'auraflow',
|
||||
'lumina2',
|
||||
'qwen',
|
||||
'bria',
|
||||
'flite',
|
||||
'cosmos',
|
||||
'chrono',
|
||||
# video models
|
||||
'hunyuanvideo',
|
||||
'cogvideo',
|
||||
'wanai',
|
||||
'chrono',
|
||||
'ltxvideo',
|
||||
'mochivideo',
|
||||
'allegrovideo',
|
||||
@@ -77,4 +80,8 @@ def disable_fuse():
|
||||
return True
|
||||
if hasattr(shared.sd_model, 'transformer') and hasattr(shared.sd_model.transformer, 'quantization_config'):
|
||||
return True
|
||||
if hasattr(shared.sd_model, 'transformer_2') and hasattr(shared.sd_model.transformer_2, 'quantization_config'):
|
||||
return True
|
||||
if hasattr(shared.sd_model, '_lora_partial'):
|
||||
return True
|
||||
return shared.sd_model_type in fuse_ignore
|
||||
|
||||
@@ -89,6 +89,8 @@ def run_ltx(task_id,
|
||||
shared.state.job_count = 1
|
||||
|
||||
p = processing.StableDiffusionProcessingVideo(
|
||||
video_engine=engine,
|
||||
video_model=model,
|
||||
prompt=prompt,
|
||||
negative_prompt=negative,
|
||||
styles=styles,
|
||||
@@ -247,6 +249,7 @@ def run_ltx(task_id,
|
||||
timer.process.add('offload', t11 - t10)
|
||||
|
||||
num_frames, video_file = save_video(
|
||||
p=p,
|
||||
pixels=frames,
|
||||
mp4_fps=mp4_fps,
|
||||
mp4_codec=mp4_codec,
|
||||
|
||||
@@ -14,7 +14,7 @@ def create_ui(prompt, negative, styles, overrides, init_image, init_strength, la
|
||||
with gr.Row():
|
||||
generate = gr.Button('Generate', elem_id="ltx_generate_btn", variant='primary', visible=False)
|
||||
with gr.Row():
|
||||
ltx_models = [m.name for m in models['LTX Video']]
|
||||
ltx_models = [m.name for m in models['LTX Video']] if 'LTX Video' in models else ['None']
|
||||
model = gr.Dropdown(label='LTX model', choices=ltx_models, value=ltx_models[0])
|
||||
with gr.Accordion(open=False, label="Condition", elem_id='ltx_condition_accordion'):
|
||||
with gr.Tabs():
|
||||
|
||||
+1
-1
@@ -339,7 +339,7 @@ def outpaint(input_image: Image.Image, outpaint_type: str = 'Edge'):
|
||||
debug(f'Run outpaint: fn={fn}') # pylint: disable=protected-access
|
||||
image = cv2.cvtColor(np.array(input_image), cv2.COLOR_RGB2BGR)
|
||||
h0, w0 = image.shape[:2]
|
||||
empty = (image == 0).all(axis=2)
|
||||
empty = (image == 0).all(axis=2) # pylint: disable=no-member
|
||||
y0, x0 = np.where(~empty) # non empty
|
||||
x1, x2 = min(x0), max(x0)
|
||||
y1, y2 = min(y0), max(y0)
|
||||
|
||||
@@ -564,7 +564,7 @@ def sdnq_quantize_model(model, op=None, sd_model=None, do_gc: bool = True, weigh
|
||||
if do_gc:
|
||||
devices.torch_gc(force=True, reason='sdnq')
|
||||
|
||||
log.debug(f'Quantization: module="{op if op is not None else model.__class__}" type=sdnq mode=post dtype={weights_dtype} matmul={shared.opts.sdnq_use_quantized_matmul} group_size={shared.opts.sdnq_quantize_weights_group_size} svd_rank={shared.opts.sdnq_svd_rank} svd_steps={shared.opts.sdnq_svd_steps} use_svd={shared.opts.sdnq_use_svd} quant_conv={shared.opts.sdnq_quantize_conv_layers} matmul_conv={shared.opts.sdnq_use_quantized_matmul_conv} dequantize_fp32={shared.opts.sdnq_dequantize_fp32} quantize_with_gpu={shared.opts.sdnq_quantize_with_gpu} quantization_device={quantization_device} return_device={return_device} device_map={shared.opts.device_map} offload_mode={shared.opts.diffusers_offload_mode} non_blocking={shared.opts.diffusers_offload_nonblocking} modules_to_not_convert={modules_to_not_convert} modules_dtype_dict={modules_dtype_dict}')
|
||||
log.debug(f'Quantization: module="{op if op is not None else model.__class__}" type=sdnq mode=post dtype={weights_dtype} matmul={shared.opts.sdnq_use_quantized_matmul} svd={shared.opts.sdnq_use_svd}:group={shared.opts.sdnq_quantize_weights_group_size}:rank={shared.opts.sdnq_svd_rank}:steps={shared.opts.sdnq_svd_steps} quant_conv={shared.opts.sdnq_quantize_conv_layers} matmul_conv={shared.opts.sdnq_use_quantized_matmul_conv} fp32={shared.opts.sdnq_dequantize_fp32} gpu={shared.opts.sdnq_quantize_with_gpu} device={quantization_device} return={return_device} map={shared.opts.device_map} non_blocking={shared.opts.diffusers_offload_nonblocking} modules_skip={modules_to_not_convert} modules_dtype={modules_dtype_dict}')
|
||||
return model
|
||||
|
||||
|
||||
@@ -572,7 +572,7 @@ def sdnq_quantize_weights(sd_model):
|
||||
try:
|
||||
t0 = time.time()
|
||||
from modules import shared, devices, sd_models
|
||||
log.debug(f"Quantization: type=SDNQ modules={shared.opts.sdnq_quantize_weights} dtype={shared.opts.sdnq_quantize_weights_mode} dtype_te={shared.opts.sdnq_quantize_weights_mode_te} matmul={shared.opts.sdnq_use_quantized_matmul} svd_rank={shared.opts.sdnq_svd_rank} svd_steps={shared.opts.sdnq_svd_steps} use_svd={shared.opts.sdnq_use_svd} group_size={shared.opts.sdnq_quantize_weights_group_size} quant_conv={shared.opts.sdnq_quantize_conv_layers} matmul_conv={shared.opts.sdnq_use_quantized_matmul_conv} quantize_with_gpu={shared.opts.sdnq_quantize_with_gpu} dequantize_fp32={shared.opts.sdnq_dequantize_fp32} pre_forward={shared.opts.diffusers_offload_pre}")
|
||||
log.debug(f"Quantization: type=SDNQ modules={shared.opts.sdnq_quantize_weights} dtype={shared.opts.sdnq_quantize_weights_mode} dtype_te={shared.opts.sdnq_quantize_weights_mode_te} offload={shared.opts.diffusers_offload_mode} pre_forward={shared.opts.diffusers_offload_pre}")
|
||||
global quant_last_model_name, quant_last_model_device # pylint: disable=global-statement
|
||||
|
||||
sd_model = sd_models.apply_function_to_model(sd_model, sdnq_quantize_model, shared.opts.sdnq_quantize_weights, op="sdnq")
|
||||
|
||||
@@ -78,8 +78,14 @@ def get_model_type(pipe):
|
||||
# hybrid models
|
||||
elif 'Wan' in name:
|
||||
model_type = 'wanai'
|
||||
if 'ChronoEdit' in name:
|
||||
model_type = 'chrono'
|
||||
elif 'HDM-xut' in name:
|
||||
model_type = 'hdm'
|
||||
elif 'HunyuanImage3' in name:
|
||||
model_type = 'hunyuanimage3'
|
||||
elif 'HunyuanImage' in name:
|
||||
model_type = 'hunyuanimage'
|
||||
else:
|
||||
model_type = name
|
||||
return model_type
|
||||
|
||||
+16
-25
@@ -20,6 +20,8 @@ pbar = None
|
||||
|
||||
|
||||
def hf_login(token=None):
|
||||
if shared.opts.offline_mode:
|
||||
return False
|
||||
global loggedin # pylint: disable=global-statement
|
||||
token = token or shared.opts.huggingface_token
|
||||
token = token.replace("\n", "").replace("\r", "").strip() if token is not None else None
|
||||
@@ -38,7 +40,8 @@ def hf_login(token=None):
|
||||
if loggedin != token:
|
||||
stdout = io.StringIO()
|
||||
try:
|
||||
hf.logout()
|
||||
with contextlib.redirect_stdout(stdout):
|
||||
hf.logout()
|
||||
except Exception:
|
||||
pass
|
||||
with contextlib.redirect_stdout(stdout):
|
||||
@@ -76,35 +79,23 @@ def download_diffusers_model(hub_id: str, cache_dir: str = None, download_config
|
||||
download_config["mirror"] = mirror
|
||||
if custom_pipeline is not None and len(custom_pipeline) > 0:
|
||||
download_config["custom_pipeline"] = custom_pipeline
|
||||
shared.log.debug(f'Diffusers downloading: id="{hub_id}" args={download_config}')
|
||||
shared.log.debug(f'HF download: id="{hub_id}" args={download_config}')
|
||||
token = token or shared.opts.huggingface_token
|
||||
if token is not None and len(token) > 2:
|
||||
hf_login(token)
|
||||
pipeline_dir = None
|
||||
|
||||
ok = False
|
||||
err = None
|
||||
if not ok:
|
||||
try:
|
||||
pipeline_dir = DiffusionPipeline.download(hub_id, **download_config)
|
||||
ok = True
|
||||
except Exception as e:
|
||||
err = e
|
||||
ok = False
|
||||
debug(f'Diffusers download error: id="{hub_id}" {e}')
|
||||
if not ok and 'Repository Not Found' not in str(err):
|
||||
try:
|
||||
download_config.pop('load_connected_pipeline', None)
|
||||
download_config.pop('variant', None)
|
||||
pipeline_dir = hf.snapshot_download(hub_id, **download_config)
|
||||
except Exception as e:
|
||||
debug(f'Diffusers download error: id="{hub_id}" {e}')
|
||||
if 'gated' in str(e):
|
||||
shared.log.error(f'Diffusers download error: id="{hub_id}" model access requires login')
|
||||
shared.state.end(jobid)
|
||||
return None
|
||||
try:
|
||||
download_config.pop('load_connected_pipeline', None)
|
||||
download_config.pop('variant', None)
|
||||
pipeline_dir = hf.snapshot_download(hub_id, **download_config)
|
||||
except Exception as e:
|
||||
debug(f'HF download error: id="{hub_id}" {e}')
|
||||
if 'gated' in str(e):
|
||||
shared.log.error(f'HF download error: id="{hub_id}" model access requires login')
|
||||
shared.state.end(jobid)
|
||||
return None
|
||||
if pipeline_dir is None:
|
||||
shared.log.error(f'Diffusers download error: id="{hub_id}" {err}')
|
||||
shared.log.error(f'HF download error: id="{hub_id}" no data')
|
||||
shared.state.end(jobid)
|
||||
return None
|
||||
try:
|
||||
|
||||
@@ -26,6 +26,11 @@ def is_compatible(diffusion_pipeline: diffusers.DiffusionPipeline) -> bool:
|
||||
return compatible
|
||||
|
||||
|
||||
def is_guider(diffusion_pipeline: diffusers.DiffusionPipeline) -> bool:
|
||||
guider = getattr(diffusion_pipeline, 'guider', None)
|
||||
return guider is not None
|
||||
|
||||
|
||||
def convert_to_modular(diffusion_pipeline: diffusers.DiffusionPipeline) -> diffusers.ModularPipeline:
|
||||
modular_pipe = None
|
||||
try:
|
||||
|
||||
@@ -32,16 +32,24 @@ def set_guider(p: processing.StableDiffusionProcessing):
|
||||
if guidance_name == 'Default':
|
||||
if hasattr(shared.sd_model, 'default_guider'):
|
||||
guider_info = shared.sd_model.default_guider
|
||||
guider_cls = guider_info.type_hint if hasattr(guider_info, 'type_hint') else type(guider_info)
|
||||
shared.sd_model.update_components(guider=guider_info)
|
||||
else:
|
||||
elif hasattr(shared.sd_model, 'get_component_spec'):
|
||||
guider_info = shared.sd_model.get_component_spec("guider")
|
||||
guider_cls = guider_info.type_hint if hasattr(guider_info, 'type_hint') else type(guider_info)
|
||||
shared.sd_model.default_guider = guider_info
|
||||
guider_cls = guider_info.type_hint
|
||||
elif hasattr(shared.sd_model, 'guider') and hasattr(shared.sd_model.guider, 'config'):
|
||||
guider_info = shared.sd_model.guider
|
||||
guider_cls = type(shared.sd_model.guider)
|
||||
# shared.sd_model.default_guider = guider_info
|
||||
else:
|
||||
guider_info = None
|
||||
guider_cls = None
|
||||
if guider_info is not None and guider_cls is not None and guider_info.config is not None:
|
||||
guider_args = {k: v for k, v in guider_info.config.items() if not k.startswith('_') and v is not None}
|
||||
else:
|
||||
guider_args = {}
|
||||
shared.log.info(f'Guider: name={guidance_name} cls={guider_cls.__name__} args={guider_args}')
|
||||
shared.log.info(f'Guider: name={guidance_name} cls={guider_cls.__name__ if guider_cls is not None else None} args={guider_args}')
|
||||
return
|
||||
if guidance_name == 'None':
|
||||
shared.sd_model.update_components(guider=None) # breaks the pipeline
|
||||
|
||||
@@ -55,7 +55,7 @@ def edge_detect_for_pixelart(image: PipelineImageInput, image_weight: float = 1.
|
||||
block_height = height // block_size
|
||||
block_width = width // block_size
|
||||
|
||||
min_pool = -torch.nn.functional.max_pool2d(-new_image, block_size, 1, block_size//2, 1, False, False)
|
||||
min_pool = 0 - torch.nn.functional.max_pool2d(-new_image, block_size, 1, block_size//2, 1, False, False)
|
||||
min_pool = min_pool[:, :, :height, :width]
|
||||
|
||||
greyscale = (new_image[:,0,:,:] * 0.299).add_(new_image[:,1,:,:], alpha=0.587).add_(new_image[:,2,:,:], alpha=0.114)
|
||||
|
||||
@@ -163,6 +163,9 @@ def process_images(p: StableDiffusionProcessing) -> Processed:
|
||||
if p.override_settings.get('sd_vae', None) == 'TAESD':
|
||||
p.vae_type = 'Tiny'
|
||||
p.override_settings.pop('sd_vae', None)
|
||||
if p.override_settings.get('sd_vae', None) == 'REPA-E':
|
||||
p.vae_type = 'Repa'
|
||||
p.override_settings.pop('sd_vae', None)
|
||||
if p.override_settings.get('Hires upscaler', None) is not None:
|
||||
p.enable_hr = True
|
||||
if len(p.override_settings.keys()) > 0:
|
||||
@@ -282,6 +285,11 @@ def process_samples(p: StableDiffusionProcessing, samples):
|
||||
sample = validate_sample(sample)
|
||||
image = Image.fromarray(sample)
|
||||
|
||||
if isinstance(image, list):
|
||||
if len(image) > 1:
|
||||
shared.log.warning(f'Processing: images={image} contains multiple images using first one only')
|
||||
image = image[0]
|
||||
|
||||
if not shared.state.interrupted and not shared.state.skipped:
|
||||
|
||||
if p.restore_faces:
|
||||
@@ -391,7 +399,9 @@ def process_images_inner(p: StableDiffusionProcessing) -> Processed:
|
||||
if not hasattr(p, 'skip_init'):
|
||||
p.init(p.all_prompts, p.all_seeds, p.all_subseeds)
|
||||
debug(f'Processing inner: args={vars(p)}')
|
||||
p.iter_init_images = p.init_images # required so we use same starting non-processed images for each batch sequence
|
||||
for n in range(p.n_iter):
|
||||
p.init_images = p.iter_init_images
|
||||
if p.n_iter > 1:
|
||||
shared.log.debug(f'Processing: batch={n+1} total={p.n_iter} progress={(n+1)/p.n_iter:.2f}')
|
||||
shared.state.batch_no = n + 1
|
||||
@@ -428,6 +438,7 @@ def process_images_inner(p: StableDiffusionProcessing) -> Processed:
|
||||
output_images.append(script_image)
|
||||
infotexts.append(script_infotext)
|
||||
|
||||
# main processing
|
||||
if samples is None:
|
||||
from modules.processing_diffusers import process_diffusers
|
||||
samples = process_diffusers(p)
|
||||
|
||||
@@ -123,11 +123,11 @@ def task_specific_kwargs(p, model):
|
||||
}
|
||||
|
||||
# model specific args
|
||||
if 'QwenImageEdit' in model_cls and (p.init_images is None or len(p.init_images) == 0):
|
||||
if ('QwenImageEdit' in model_cls) and (p.init_images is None or len(p.init_images) == 0):
|
||||
task_args['image'] = [Image.new('RGB', (p.width, p.height), (0, 0, 0))] # monkey-patch so qwen-image-edit pipeline does not error-out on t2i
|
||||
if 'QwenImageEditPlusPipeline' in model_cls and p.init_control is not None and len(p.init_control) > 0:
|
||||
if ('QwenImageEditPlusPipeline' in model_cls) and (p.init_control is not None) and (len(p.init_control) > 0):
|
||||
task_args['image'] += p.init_control
|
||||
if 'LatentConsistencyModelPipeline' in model_cls and len(p.init_images) > 0:
|
||||
if ('LatentConsistencyModelPipeline' in model_cls) and (len(p.init_images) > 0):
|
||||
p.ops.append('lcm')
|
||||
init_latents = [processing_vae.vae_encode(image, model=shared.sd_model, vae_type=p.vae_type).squeeze(dim=0) for image in p.init_images]
|
||||
init_latent = torch.stack(init_latents, dim=0).to(shared.device)
|
||||
@@ -138,6 +138,13 @@ def task_specific_kwargs(p, model):
|
||||
'width': p.width,
|
||||
'height': p.height,
|
||||
}
|
||||
if ('WanImageToVideoPipeline' in model_cls) or ('ChronoEditPipeline' in model_cls):
|
||||
if (p.init_images is not None) and (len(p.init_images) > 0):
|
||||
task_args['image'] = p.init_images[0]
|
||||
else:
|
||||
task_args['image'] = Image.new('RGB', (p.width, p.height), (0, 0, 0)) # monkey-patch so wan-i2i pipeline does not error-out on t2i
|
||||
if ('WanVACEPipeline' in model_cls) and (p.init_images is not None) and (len(p.init_images) > 0):
|
||||
task_args['reference_images'] = p.init_images
|
||||
if 'BlipDiffusionPipeline' in model_cls:
|
||||
if len(p.init_images) == 0:
|
||||
shared.log.error('BLiP diffusion requires init image')
|
||||
@@ -148,10 +155,6 @@ def task_specific_kwargs(p, model):
|
||||
'target_subject_category': getattr(p, 'prompt', '').split()[-1],
|
||||
'output_type': 'pil',
|
||||
}
|
||||
if ('WanImageToVideoPipeline' in model_cls) and (p.init_images is not None) and (len(p.init_images) > 0):
|
||||
task_args['image'] = p.init_images[0]
|
||||
if ('WanVACEPipeline' in model_cls) and (p.init_images is not None) and (len(p.init_images) > 0):
|
||||
task_args['reference_images'] = p.init_images
|
||||
|
||||
if debug_enabled:
|
||||
debug_log(f'Process task specific args: {task_args}')
|
||||
|
||||
@@ -93,15 +93,24 @@ def diffusers_callback(pipe, step: int = 0, timestep: int = 0, kwargs: dict = {}
|
||||
if step != getattr(pipe, 'num_timesteps', 0):
|
||||
kwargs = processing_correction.correction_callback(p, timestep, kwargs, initial=step == 0)
|
||||
kwargs = prompt_callback(step, kwargs) # monkey patch for diffusers callback issues
|
||||
if step == int(getattr(pipe, 'num_timesteps', 100) * p.cfg_end) and 'prompt_embeds' in kwargs and 'negative_prompt_embeds' in kwargs:
|
||||
|
||||
if step == 0:
|
||||
pipe._cfg_end_applied = False # pylint: disable=protected-access
|
||||
|
||||
cfg_end = getattr(p, "cfg_end", 1.0) or 1.0
|
||||
total_steps = getattr(pipe, "num_timesteps", 0)
|
||||
target_step = int(total_steps * cfg_end) if total_steps else 0
|
||||
if (cfg_end < 1.0) and not getattr(pipe, "_cfg_end_applied", False) and (step >= target_step):
|
||||
pipe._cfg_end_applied = True # pylint: disable=protected-access
|
||||
if "PAG" in shared.sd_model.__class__.__name__:
|
||||
pipe._guidance_scale = 1.001 if pipe._guidance_scale > 1 else pipe._guidance_scale # pylint: disable=protected-access
|
||||
pipe._pag_scale = 0.001 # pylint: disable=protected-access
|
||||
else:
|
||||
pipe._guidance_scale = 0.0 # pylint: disable=protected-access
|
||||
for key in {"prompt_embeds", "negative_prompt_embeds", "add_text_embeds", "add_time_ids"} & set(kwargs):
|
||||
if kwargs[key] is not None:
|
||||
kwargs[key] = kwargs[key].chunk(2)[-1]
|
||||
for key in ["prompt_embeds", "negative_prompt_embeds", "add_text_embeds", "add_time_ids"]:
|
||||
tensor = kwargs.get(key, None)
|
||||
if tensor is not None and hasattr(tensor, "chunk") and tensor.shape[0] % 2 == 0:
|
||||
kwargs[key] = tensor.chunk(2)[-1]
|
||||
try:
|
||||
current_noise_pred = kwargs.get("noise_pred", None)
|
||||
if current_noise_pred is None:
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user