automatic/TODO.md

TODO

Features

Assigned

• Chat-based interface, @vladmandic

• Control tab verify overrides handling, @vladmandic

• Reimplement llama remover for Kanvas, @vladmandic

• Detailer postprocessing, @CalamitousFelicitousness

• Cloud providers, @CalamitousFelicitousness

• Video processing add full API support, @CalamitousFelicitousness

Unassigned

• Processing -> Video capabilities
  • RIFE in processing
  • SeedVR2 in processing
• Video model loader: Add video models to Reference
• UI Lite vs Expert mode
• TensorRT acceleration
• Auto handle scheduler prediction_type
• Cache models in memory
• JSON image metadata
• Integrate natural language image search: ImageDB
• Unify huggingface and diffusers model folders
• Refactor GGUF

OnHold

• Implement pruna, @vladmandic, pending support for transformers 5.5
• LoRA add OMI format support for SD35/FLUX.1, on-hold
• Remote Text-Encoder support, sidelined for the moment
• Multi-user support
• Settings profile manager

Modular

Pending finalization of modular pipelines implementation and development of compatibility layer

• Switch to modular pipelines
• Feature: Transformers unified cache handler
• Refactor: Modular pipelines and guiders
• MagCache
• SmoothCache
• STG
• TextKVCache, @vladmandic

New models / Pipelines

TODO: Investigate which models are diffusers-compatible and prioritize!

Image

• JoyAI-Image-Edit (pr in-progress)
• nVidia Cosmos-Predict-2.5 (in diffusers)
• nVidia Cosmos-Transfer-2.5 (in diffusers)
• Tencent HY-WU (transformers-compatible)

Video

• HY-OmniWeaving
• OpenMOSS MOVA
• Wan2.2-Animate
• Wan2.1-T2V-14B-CausVid
• Wan2.1-StepDistill-CfgDistill
• Krea Realtime Video
• MAGI-1
• MUG-V 10B
• Ovi
• LucyEdit
• HunyuanVideo-Avatar
• Sana I2V
• Wan-2.2 S2V
• Meituan LongCat-Video
• Phantom HuMo
• CausVid-Plus
• LivePortrait
• SandAI Magi
• inclusionAI Ming
• HummingbirdXT
• DiffusionForcing
• ByteDance Lynx
• LanDiff

Other/Unsorted

• GPEN Face Restoration
• ByteDance DreamO
  • Unified image customization framework combining face identity preservation, virtual try-on, style transfer, etc.
  • Created: 2025-05 | Updated: 2025-08 | Stars: 1,700
• ControlNeXt
  • Lightweight controllable generation framework for images and videos (SD1.5, SDXL, SVD) that uses up to 90% fewer trainable parameters than ControlNet
  • Created: 2024-08 | Updated: 2024-08 | Stars: 1,600
• ByteDance USO
  • Unified model for both style-transfer and subject-driven image generation from one or two reference images
  • Created: 2025-08 | Updated: 2025-09 | Stars: 1,200
• TwinFlow
  • Distillation technique that converts large image generation models into 1–2 step generators without requiring a separate teacher model
  • Created: 2025-12 | Updated: 2026-02 | Stars: 506
• FlashFace
  • Zero-shot face personalization method that generates images of a specific person from one or a few reference photos
  • Created: 2024-03 | Updated: 2024-05 | Stars: 436
• DiffSynth-Engine
  • Alternative to diffusers library that unlocks some diffsynth specific capabilities
  • Created: 2024-05 | Updated: 2026-03 | Stars: 393
• MS-Diffusion
  • Multi-subject image personalization framework that uses layout guidance to place multiple reference subjects in a single generated image without identity confusion
  • Created: 2024-04 | Updated: 2025-07 | Stars: 309
• RamTorch
  • Alternative memory management and offloading library
  • Created: 2025-09 | Updated: 2026-04 | Stars: 266
• UniRef
  • Unified segmentation model that handles referring image segmentation and few-shot segmentation
  • Created: 2023-04 | Updated: 2025-04 | Stars: 238
• FreeFuse
  • Training-free method to combine multiple subject LoRAs in one image generation without conflicts, by automatically routing each LoRA's influence to its target spatial region.
  • Created: 2026-01 | Updated: 2026-03 | Stars: 178
• mmgp
  • Alternative memory management and offloading library
  • Created: 2024-03 | Updated: 2026-02 | Stars: 175
• ReNO
  • Inference-time technique that improves one-step text-to-image models by iteratively optimizing the initial noise using reward model signals, boosting prompt accuracy in 20–50 seconds
  • Created: 2024-06 | Updated: 2025-09 | Stars: 166
• RegionE
  • Speeds up instruction-based image editing by skipping redundant computation in image regions that are not being changed.
  • Created: 2025-10 | Updated: 2026-02 | Stars: 98
• Make-It-Count
  • Method that reliably generates the exact number of objects requested by tracking instance identities during denoising
  • Created: 2024-04 | Updated: 2025-04 | Stars: 96
• FaceClip
  • Identity-preserving image generation model that jointly encodes a face and a text prompt into a shared embedding to produce portraits matching both the subject's appearance and the scene description
  • Created: 2025-04 | Updated: 2025-04 | Likes: 88
• T5Gemma Adapter
  • Experiment that replaces the SDXL text encoder with a T5Gemma LLM via a trained adapter for richer prompt understanding
  • Created: 2025-07 | Updated: 2025-10 | Stars: 67
• Sonic Inpaint
  • Image inpainting method that optimizes for better masked-region filling
  • Created: 2025-11 | Updated: 2026-01 | Stars: 23
• SEVA
  • Model that generates novel-view images of a scene from a single input photo.
  • Created: 2025-04 | Updated: 2025-06 | Stars: N/A (draft PR)
• Bria FIBO RMBG
  • Background removal model trained on Bria FIBO dataset
  • Created: 2025-08 | Updated: 2025-09 | Stars: N/A (private model)

Rejected

• Mugen (sdxl with flux vae experiment, not clean)
• Liquid (autoregressive, not clean)

Code TODO

npm run todo

installer.py:TODO rocm: switch to pytorch source when it becomes available
modules/control/run.py:TODO modernui: monkey-patch for missing tabs.select event
modules/history.py:TODO: apply metadata, preview, load/save
modules/image/resize.py:TODO resize image: enable full VAE mode for resize-latent
modules/lora/lora_load.py:TODO lora: add t5 key support for sd35/f1
modules/masking.py:TODO: additional masking algorithms
modules/modular_guiders.py:TODO: guiders
modules/processing_class.py:TODO processing: remove duplicate mask params
modules/sd_hijack_hypertile.py:TODO hypertile: vae breaks when using non-standard sizes
modules/sd_models.py:TODO model load: implement model in-memory caching
modules/sd_samplers_diffusers.py:TODO enso-required
modules/sd_unet.py:TODO model load: force-reloading entire model as loading transformers only leads to massive memory usage
modules/transformer_cache.py:TODO fc: autodetect distilled based on model
modules/transformer_cache.py:TODO fc: autodetect tensor format based on model
modules/ui_models_load.py:TODO loader: load receipe
modules/ui_models_load.py:TODO loader: save receipe