calc folder size

Signed-off-by: Vladimir Mandic <mandic00@live.com>
2026-08-26 15:16:01 +02:00 · 2025-08-13 12:23:29 -04:00
parent 532e89b4fa
commit 8e9244939b
7 changed files with 71 additions and 50 deletions
@@ -1,18 +1,18 @@
 # Change Log for SD.Next

-## Update for 2025-08-12
+## Update for 2025-08-13

-### Highlights for 2025-08-12
+### Highlights for 2025-08-13

 Several new models: [Qwen-Image](https://qwenlm.github.io/blog/qwen-image/) (plus *Lightning* variant) and [FLUX.1-Krea-Dev](https://www.krea.ai/blog/flux-krea-open-source-release)  
-Several updated models: [Chroma](https://huggingface.co/lodestones/Chroma), [SkyReels-V2](https://huggingface.co/Skywork/SkyReels-V2-DF-14B-720P-Diffusers) and [Wan-VACE](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B-diffusers)  
+Several updated models: [Chroma](https://huggingface.co/lodestones/Chroma), [SkyReels-V2](https://huggingface.co/Skywork/SkyReels-V2-DF-14B-720P-Diffusers), [Wan-VACE](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B-diffusers), [HunyuanDiT](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers-Distilled)  
 Plus continuing with major **UI** work, we have new embedded **Docs/Wiki** search, redesigned real-time **hints**, built-in **GPU monitor**, **CivitAI** integration and more!  
-On the compute side, new profiles for high-vram GPUs, offloading improvements and support for new `torch` release    
+On the compute side, new profiles for high-vram GPUs, offloading improvements, support for new `torch` release and improved quality when using low-bit quantization!      
 And (*as always*) many bugfixes and improvements to existing features!  

 [ReadMe](https://github.com/vladmandic/automatic/blob/master/README.md) | [ChangeLog](https://github.com/vladmandic/automatic/blob/master/CHANGELOG.md) | [Docs](https://vladmandic.github.io/sdnext-docs/) | [WiKi](https://github.com/vladmandic/automatic/wiki) | [Discord](https://discord.com/invite/sd-next-federal-batch-inspectors-1101998836328697867)

-### Details for 2025-08-12
+### Details for 2025-08-13

 - **Models**  
  - [Qwen-Image](https://qwenlm.github.io/blog/qwen-image/)  
@@ -38,6 +38,8 @@ And (*as always*) many bugfixes and improvements to existing features!
  - [Wan-VACE](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B-diffusers)  
    basic support for *Wan 2.1 VACE 1.3B* and *14B* variants  
    optimized support with granular guidance control will follow soon  
+  - [HunyuanDiT-Distilled](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers-Distilled)  
+    variant of HunyuanDiT with reduced steps and improved performance  
 **Torch**  
  - Set default to `torch==2.8.0` for *CUDA, ROCm and OpenVINO*  
  - Add support for `torch==2.9.0-nightly`  
@@ -98,10 +100,10 @@ And (*as always*) many bugfixes and improvements to existing features!
  - **Nunchaku** support for *FLUX.1-Fill* and *FLUX.1-Depth* models  
  - update requirements/packages  
  - use model vae scale-factor for image width/heigt calculations  
-  - **SDNQ** add modules_dtype_dict to quantize *Qwen Image* with mixed dtype  
- **Other**
-  - **prompt enhance** add `allura-org/Gemma-3-Glitter-4B`, `Qwen/Qwen3-4B-Instruct-2507`, `Qwen/Qwen2.5-VL-3B-Instruct` model support  
-  - **prompt enhance** improve system prompt  
+  - **SDNQ** add `modules_dtype_dict` to quantize *Qwen Image* with mixed dtype  
+  - **prompt enhance**
+    add `allura-org/Gemma-3-Glitter-4B`, `Qwen/Qwen3-4B-Instruct-2507`, `Qwen/Qwen2.5-VL-3B-Instruct` model support  
+    improve system prompt  
  - **schedulers** add **Flash FlowMatch**  
 - **API**
  - add `/sdapi/v1/checkpoint` POST endpoint to simply load a model  
@@ -131,6 +133,7 @@ And (*as always*) many bugfixes and improvements to existing features!
  - fix `nudenet` api  
  - fix global state tracking  
  - fix ui tab detection for networks  
+  - unified stat size/mtime calls  
  - reapply offloading on ipadapter load  
  - api set default script-name  
  - avoid forced gc and rely on thresholds  
@@ -2,11 +2,16 @@

 Main ToDo list can be found at [GitHub projects](https://github.com/users/vladmandic/projects)

+## Current Candidates
+
+- `HF_ENABLE_PARALLEL_LOADING`
+
 ## Future Candidates

 - Unified `CLIPTextModelWithProjection` loader  
 - [Modular pipelines and guiders](https://github.com/huggingface/diffusers/issues/11915)  
 - Refactor: Sampler options  
+- Refactor: [GGUF](https://huggingface.co/docs/diffusers/main/en/quantization/gguf)  
 - Feature: Diffusers [group offloading](https://github.com/vladmandic/sdnext/issues/4049)  
 - Feature: Common repo for `T5` and `CLiP`  
 - Feature: LoRA add OMI format support for SD35/FLUX.1  
@@ -9,7 +9,7 @@ from fastapi.responses import JSONResponse
 from starlette.websockets import WebSocket, WebSocketState
 from pydantic import BaseModel, Field # pylint: disable=no-name-in-module
 from PIL import Image
-from modules import shared, images, files_cache
+from modules import shared, images, files_cache, modelstats


 debug = shared.log.debug if os.environ.get('SD_BROWSER_DEBUG', None) is not None else lambda *args, **kwargs: None
@@ -76,7 +76,7 @@ def register_api(app: FastAPI): # register api
    def get_video_thumbnail(filepath):
        from modules.video import get_video_params
        try:
-            stat = os.stat(filepath)
+            stat_size, stat_mtime = modelstats.stat(filepath)
            frames, fps, duration, width, height, codec, frame = get_video_params(filepath, capture=True)
            h = shared.opts.extra_networks_card_size
            w = shared.opts.extra_networks_card_size if shared.opts.browser_fixed_width else width * h // height
@@ -91,8 +91,8 @@ def register_api(app: FastAPI): # register api
                'data': data_url,
                'width': width,
                'height': height,
-                'size': stat.st_size,
-                'mtime': stat.st_mtime,
+                'size': stat_size,
+                'mtime': stat_mtime,
            }
            return content
        except Exception as e:
@@ -101,7 +101,7 @@ def register_api(app: FastAPI): # register api

    def get_image_thumbnail(filepath):
        try:
-            stat = os.stat(filepath)
+            stat_size, stat_mtime = modelstats.stat(filepath)
            image = Image.open(filepath)
            geninfo, _items = images.read_info_from_image(image)
            h = shared.opts.extra_networks_card_size
@@ -118,8 +118,8 @@ def register_api(app: FastAPI): # register api
                'data': data_url,
                'width': width,
                'height': height,
-                'size': stat.st_size,
-                'mtime': stat.st_mtime,
+                'size': stat_size,
+                'mtime': stat_mtime,
            }
            return content
        except Exception as e:
@@ -4,6 +4,29 @@ import torch
 from modules import shared, sd_models


+def walk(folder: str):
+    files = []
+    for root, _, filenames in os.walk(folder):
+        for filename in filenames:
+            files.append(os.path.join(root, filename))
+    return files
+
+
+def stat(fn: str):
+    if fn is None or len(fn) == 0 or not os.path.exists(fn):
+        return 0, None
+    fs_stat = os.stat(fn)
+    mtime = datetime.fromtimestamp(fs_stat.st_mtime).replace(microsecond=0)
+    if os.path.isfile(fn):
+        size = round(fs_stat.st_size)
+    elif os.path.isdir(fn):
+        size = round(sum(stat(fn)[0] for fn in walk(fn)))
+    else:
+        size = 0
+    print('HERE', fn, os.path.isfile(fn), os.path.isdir(fn), size, mtime)
+    return size, mtime
+
+
 class Module():
    name: str = ''
    cls: str = None
@@ -60,11 +83,7 @@ class Model():
            self.name = self.info.name or self.name
            self.hash = self.info.shorthash or ''
            self.meta = self.info.metadata or {}
-            if os.path.exists(self.info.filename):
-                stat = os.stat(self.info.filename)
-                self.mtime = datetime.fromtimestamp(stat.st_mtime).replace(microsecond=0)
-                if os.path.isfile(self.info.filename):
-                    self.size = round(stat.st_size)
+            self.size, self.mtime = stat(self.info.filename)

    def __repr__(self):
        return f'model="{self.name}" type={self.type} class={self.cls} size={self.size} mtime="{self.mtime}" modules={self.modules}'
@@ -8,7 +8,6 @@ import html
 import base64
 import urllib.parse
 import threading
-from datetime import datetime
 from types import SimpleNamespace
 from pathlib import Path
 from html.parser import HTMLParser
@@ -16,9 +15,7 @@ from collections import OrderedDict
 import gradio as gr
 from PIL import Image
 from starlette.responses import FileResponse, JSONResponse
-from modules import paths, shared, files_cache, errors, infotext
-from modules.ui_components import ToolButton
-import modules.ui_symbols as symbols
+from modules import paths, shared, files_cache, errors, infotext, ui_symbols, ui_components, modelstats


 allowed_dirs = []
@@ -612,7 +609,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
        return is_visible, gr.update(visible=is_visible), gr.update(variant=("secondary-down" if is_visible else "secondary"))

    with ui.details:
-        details_close = ToolButton(symbols.close, elem_id=f"{tabname}_extra_details_close", elem_classes=['extra-details-close'])
+        details_close = ui_components.ToolButton(ui_symbols.close, elem_id=f"{tabname}_extra_details_close", elem_classes=['extra-details-close'])
        details_close.click(fn=lambda: gr.update(visible=False), inputs=[], outputs=[ui.details])
        with gr.Row():
            with gr.Column(scale=1):
@@ -665,14 +662,14 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
            model_visible = page in ['Model']
            return [gr.update(visible=scan_visible), gr.update(visible=save_visible), gr.update(visible=model_visible)]

-        ui.button_refresh = ToolButton(symbols.refresh, elem_id=f"{tabname}_extra_refresh")
-        ui.button_scan = ToolButton(symbols.scan, elem_id=f"{tabname}_extra_scan", visible=True)
-        ui.button_quicksave = ToolButton(symbols.book, elem_id=f"{tabname}_extra_quicksave", visible=False)
-        ui.button_save = ToolButton(symbols.book, elem_id=f"{tabname}_extra_save", visible=False)
-        ui.button_sort = ToolButton(symbols.sort, elem_id=f"{tabname}_extra_sort", visible=True)
-        ui.button_view = ToolButton(symbols.view, elem_id=f"{tabname}_extra_view", visible=True)
-        ui.button_close = ToolButton(symbols.close, elem_id=f"{tabname}_extra_close", visible=True)
-        ui.button_model = ToolButton(symbols.refine, elem_id=f"{tabname}_extra_model", visible=True)
+        ui.button_refresh = ui_components.ToolButton(ui_symbols.refresh, elem_id=f"{tabname}_extra_refresh")
+        ui.button_scan = ui_components.ToolButton(ui_symbols.scan, elem_id=f"{tabname}_extra_scan", visible=True)
+        ui.button_quicksave = ui_components.ToolButton(ui_symbols.book, elem_id=f"{tabname}_extra_quicksave", visible=False)
+        ui.button_save = ui_components.ToolButton(ui_symbols.book, elem_id=f"{tabname}_extra_save", visible=False)
+        ui.button_sort = ui_components.ToolButton(ui_symbols.sort, elem_id=f"{tabname}_extra_sort", visible=True)
+        ui.button_view = ui_components.ToolButton(ui_symbols.view, elem_id=f"{tabname}_extra_view", visible=True)
+        ui.button_close = ui_components.ToolButton(ui_symbols.close, elem_id=f"{tabname}_extra_close", visible=True)
+        ui.button_model = ui_components.ToolButton(ui_symbols.refine, elem_id=f"{tabname}_extra_model", visible=True)
        ui.search = gr.Textbox('', show_label=False, elem_id=f"{tabname}_extra_search", placeholder="Search...", elem_classes="textbox", lines=2, container=False)
        ui.description = gr.Textbox('', show_label=False, elem_id=f"{tabname}_description", elem_classes=["textbox", "extra-description"], lines=2, interactive=False, container=False)

@@ -802,7 +799,7 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
        is_valid = (item is not None) and hasattr(item, 'name') and hasattr(item, 'filename')

        if is_valid:
-            stat = os.stat(item.filename) if os.path.exists(item.filename) else None
+            stat_size, stat_mtime = modelstats.stat(item.filename)
            desc = item.description
            fullinfo = shared.readfile(os.path.splitext(item.filename)[0] + '.json', silent=True)
            if 'modelVersions' in fullinfo: # sanitize massive objects
@@ -892,8 +889,8 @@ def create_ui(container, button_parent, tabname, skip_indexing = False):
                    <tr><td>Alias</td><td>{getattr(item, 'alias', 'N/A')}</td></tr>
                    <tr><td>Filename</td><td>{item.filename}</td></tr>
                    <tr><td>Hash</td><td>{getattr(item, 'hash', 'N/A')}</td></tr>
-                    <tr><td>Size</td><td>{round(stat.st_size/1024/1024, 2) if stat is not None else 'N/A'} MB</td></tr>
-                    <tr><td>Last modified</td><td>{datetime.fromtimestamp(stat.st_mtime) if stat is not None else 'N/A'}</td></tr>
+                    <tr><td>Size</td><td>{round(stat_size/1024/1024, 2)} MB</td></tr>
+                    <tr><td>Last modified</td><td>{stat_mtime}</td></tr>
                    <tr><td>Source URL</td><td>{url}</td></tr>
                    <tr><td style="border-top: 1px solid var(--button-primary-border-color);"></td><td></td></tr>
                    {lora}
@@ -1,9 +1,8 @@
 import os
-from datetime import datetime
 from urllib.parse import unquote
 import gradio as gr
 from PIL import Image
-from modules import shared, ui_symbols, ui_common, images, video
+from modules import shared, ui_symbols, ui_common, images, video, modelstats
 from modules.ui_components import ToolButton


@@ -12,7 +11,7 @@ def read_media(fn):
    if not os.path.isfile(fn):
        shared.log.error(f'Gallery not found: file="{fn}"')
        return [[], None, '', '', f'Media not found: {fn}']
-    stat = os.stat(fn)
+    stat_size, stat_mtime = modelstats.stat(fn)
    if fn.lower().endswith('.mp4'):
        frames, fps, duration, w, h, codec, _frame = video.get_video_params(fn)
        geninfo = ''
@@ -22,8 +21,8 @@ def read_media(fn):
            | Frames <b>{frames:,}</b>
            | FPS <b>{fps:.2f}</b>
            | Duration <b>{duration:.2f}</b>
-            | Size <b>{stat.st_size:,}</b>
-            | Modified <b>{datetime.fromtimestamp(stat.st_mtime)}</b></p><br>
+            | Size <b>{stat_size:,}</b>
+            | Modified <b>{stat_mtime}</b></p><br>
            '''
        return [gr.update(visible=False, value=[]), gr.update(visible=True, value=fn), geninfo, geninfo, log]
    else:
@@ -34,8 +33,8 @@ def read_media(fn):
            <p>Image <b>{image.width} x {image.height}</b>
            | Format <b>{image.format}</b>
            | Mode <b>{image.mode}</b>
-            | Size <b>{stat.st_size:,}</b>
-            | Modified <b>{datetime.fromtimestamp(stat.st_mtime)}</b></p><br>
+            | Size <b>{stat_size:,}</b>
+            | Modified <b>{stat_mtime}</b></p><br>
            '''
        return [gr.update(visible=True, value=[image]), gr.update(visible=False), geninfo, geninfo, log]

@@ -1,8 +1,7 @@
 import os
 import inspect
-from datetime import datetime
 import gradio as gr
-from modules import errors, sd_models, sd_vae, extras, sd_samplers, ui_symbols
+from modules import errors, sd_models, sd_vae, extras, sd_samplers, ui_symbols, modelstats
 from modules.ui_components import ToolButton
 from modules.ui_common import create_refresh_button
 from modules.call_queue import wrap_gradio_gpu_call
@@ -57,7 +56,6 @@ def create_ui():
                    return html.format(tbody=tbody)

                def analyze():
-                    from modules import modelstats
                    model = modelstats.analyze()
                    if model is None:
                        return ["Model not loaded", {}]
@@ -93,10 +91,10 @@ def create_ui():
                    for row in rows:
                        try:
                            f = row.filename
-                            stat = os.stat(row.filename)
+                            stat_size, stat_mtime = modelstats.stat(f)
                            if os.path.isfile(f):
                                typ = os.path.splitext(f)[1][1:]
-                                size = f"{round(stat.st_size / 1024 / 1024 / 1024, 3)} gb"
+                                size = f"{round(stat_size / 1024 / 1024 / 1024, 3)} gb"
                            elif os.path.isdir(f):
                                typ = 'diffusers'
                                size = 'folder'
@@ -117,7 +115,7 @@ def create_ui():
                                    <td>{pipeline.__name__ if pipeline else '(unknown)'}</td>
                                    <td>{row.shorthash}</td>
                                    <td>{size}</td>
-                                    <td>{datetime.fromtimestamp(stat.st_mtime).replace(microsecond=0)}</td>
+                                    <td>{stat_mtime}</td>
                                </tr>
                            """
                        except Exception as e: