pin correct tokenizers

Signed-off-by: Vladimir Mandic <mandic00@live.com>

CHANGELOG.md

@@ -1,8 +1,8 @@
 # Change Log for SD.Next
 
-## Update for 2026-07-05
+## Update for 2026-07-06
 
-### Highlights for 2026-07-05
+### Highlights for 2026-07-06
 
 Service-pack update with number of fixes and quality-of-life improvements  
 Plus few new models: **Krea 2**, **Boogu**, **Photoroom PRXPixel**, **FLUX.2 Klein KV** and some new community models  
@@ -11,7 +11,7 @@ Also couple of *experimental* features: see below for details...
 
 [Home](https://vladmandic.github.io/sdnext/) | [ChangeLog](https://github.com/vladmandic/automatic/blob/master/CHANGELOG.md) | [Docs](https://vladmandic.github.io/sdnext-docs/) | [Discord](https://discord.com/invite/sd-next-federal-batch-inspectors-1101998836328697867) | [Sponsor](https://github.com/sponsors/vladmandic)  
 
-### Details for 2026-07-05
+### Details for 2026-07-06
 
 - **Models**
   - [Krea 2](https://www.krea.ai/blog/krea-2-image-model) in *base* and *turbo* (distilled) variants  
@@ -44,6 +44,7 @@ Also couple of *experimental* features: see below for details...
     auto-select download path  
     improved search  
     validate downloads  
+  - improved custom **Text Encoder** loader  
 - **UI**
   - dynamic visibility of image controls
   - improve main panel positioning: *portrait/landscape*

installer.py

@@ -583,7 +583,7 @@ def check_transformers():
     else:
         # target_transformers = '4.57.6'
         target_transformers = None
-        target_tokenizers = '0.23.1'
+        target_tokenizers = '0.22.2'
     if target_transformers is not None:
         # Pinned release version (e.g. DirectML)
         if args.reinstall or (pkg_transformers is None) or ((pkg_transformers.version != target_transformers) or (pkg_tokenizers is None) or ((pkg_tokenizers.version != target_tokenizers) and (not args.experimental))):

modules/loader.py

@@ -157,6 +157,7 @@ except Exception as e:
     report(f'pydantic=={pydantic.__version__ if "pydantic" in sys.modules else None}', e)
 timer.startup.record("pydantic")
 
+import tokenizers # pylint: disable=W0611,C0411
 try:
     # transformers==5.x has different dependency stack so switching between v4 and v5 becomes very painful
     # this temporarily disables dependency version checks so we can use either v4 or v5 until we drop support for v4
@@ -250,6 +251,7 @@ def get_packages():
         "torchvision": torchvision.__version__,
         "diffusers": diffusers.__version__,
         "transformers": transformers.__version__,
+        "tokenizers": tokenizers.__version__,
         "accelerate": accelerate.__version__,
         "hub": huggingface_hub.__version__,
         "gradio": gradio.__version__,
@@ -288,5 +290,5 @@ class VersionString(str): # support both string and tuple for version check
 
 
 torch.__version__ = VersionString(torch.__version__)
-log.info(f'Torch: torch=={torch.__version__} torchvision=={torchvision.__version__}')
-log.info(f'Packages: diffusers=={diffusers.__version__} transformers=={transformers.__version__} accelerate=={accelerate.__version__} gradio=={gradio.__version__} pydantic=={pydantic.__version__} numpy=={np.__version__} cv2=={cv2.__version__} onnxruntime=={ort.__version__ if ort is not None else None}')
+packages_lst = [f'{pkg}=={ver}' for pkg, ver in get_packages().items()]
+log.info(f'Packages: {" ".join(packages_lst)}')