a90d85ddfd
feat(ui): add dynamic task selection based on VLM model
...
- Rename "Predefined question" to "Task"
- Task dropdown updates choices when model changes
- Prompt placeholder updates based on selected task
- Model-specific tasks: Florence-2 gets detection tasks, Moondream gets point/detect
2025-12-05 00:00:25 +00:00
4df6aa7944
fix(ui): set prefill text to empty by default
2025-12-05 00:00:25 +00:00
0d88fcd396
feat(ui): add prefill and thinking controls to Caption tab
...
Add minimal UI controls to expose new VQA functionality:
- Prefill Text input for guiding VLM responses
- Thinking Mode checkbox for reasoning models
- Keep Thinking Trace checkbox for output retention
- Keep Prefill checkbox for output retention
- Annotated Image output panel for detection visualization
- Updated button handlers to pass new parameters
2025-12-05 00:00:24 +00:00
c2810dfee2
fix(api): update VQA API endpoint for tuple return format
...
Update interrogate API endpoint to handle the new (text, image)
tuple return format from VQA interrogate function.
2025-12-05 00:00:24 +00:00
27fa48cc99
feat(vqa): major VQA handler refactor with prefill, thinking, and visualization
...
Comprehensive overhaul of the VQA interrogation system including:
- Prefill text support for guiding VLM responses
- Thinking mode support with tag cleanup/retention
- Dynamic prompt/task selection based on model type
- Bounding box visualization for detection results
- Debug infrastructure (SD_VQA_DEBUG env var)
- New model support: MiMo-VL, Nidum Gemma, Allura Gemma
- Model-specific prompt lists (Florence, Moondream)
2025-12-05 00:00:24 +00:00
0a322c0faf
feat(vqa): add Moondream 3 Preview handler
...
Add support for Moondream 3 Preview VLM with:
- Text query, caption, point, and detect capabilities
- Bounding box visualization for object detection
- Max pixels setting for resolution control
- Device offloading support
2025-12-05 00:00:24 +00:00
c024c0c9c6
feat(settings): add VLM prefill and thinking retention options
...
Add new VLM configuration options:
- interrogate_vlm_keep_prefill: Keep prefill text in output
- interrogate_vlm_keep_thinking: Keep reasoning trace in output
Also adjust defaults:
- Change interrogate_clip_flavor_count: 16 -> 1024 with updated range
- Change interrogate_vlm_prompt default to first item ("Use Prompt")
2025-12-05 00:00:24 +00:00
85cd222793
fix(vqa): sort CLiP analysis results and add text output
...
Improvements to the OpenCLIP interrogation:
- Sort all ranking dicts by similarity score (descending)
- Add format_category() helper for text formatting
- Add formatted text output for CLIP labels textbox
- Return additional text update in analyze_image()
2025-12-02 21:48:09 +00:00
eb832a4850
fix(vqa): respect offload setting in JoyCaption, add max_pixels
...
Two fixes for the JoyCaption handler:
- Only offload model if shared.opts.interrogate_offload is True
- Add max_pixels=1024*1024 to AutoProcessor for consistent image handling
2025-12-02 21:46:09 +00:00
766cb49928
feat(ui): add vision and reasoning symbols, fix dropdown fonts
...
Add new Font Awesome symbols for model capability indicators:
- vision symbol (eye icon) for vision-capable VLM models
- reasoning symbol (lightbulb icon) for thinking/reasoning models
Also fix dropdown font styling by adding NotoSans font-family.
2025-12-02 21:43:13 +00:00
d3a2f6c7ed
fix loading local prequant models
...
Signed-off-by: vladmandic <mandic00@live.com >
2025-12-02 20:53:19 +01:00
0ad40d2b8b
lint
...
Signed-off-by: vladmandic <mandic00@live.com >
2025-12-02 12:25:04 +01:00
39bced0987
Merge branch 'dev' of https://github.com/vladmandic/sdnext into dev
2025-12-02 10:40:31 +01:00
903d47f9e6
add zimage and f2 to lora overrides
...
Signed-off-by: vladmandic <mandic00@live.com >
2025-12-02 10:40:27 +01:00
3b4f909862
Merge pull request #4436 from CalamitousFelicitousness/runai-update
...
Update runai-model-streamer logging integration
2025-12-02 03:59:38 -05:00
1673380b94
Merge pull request #4430 from awsr/fix_show_progress
...
show_progress requires "full", "minimal", or "hidden"
2025-12-02 03:50:34 -05:00
de3ebf470d
Merge pull request #4428 from awsr/revert-for-now
...
Revert changes that require at least Python version 3.12
2025-12-02 03:49:20 -05:00
55c089ae48
Update runai-model-streamer logging integration
...
- Remove stdout redirect monkeypatch (fixed in runai v0.15.1 via PR #97 )
- Add RUNAI_STREAMER_LOG_LEVEL controlled by SD_LOAD_DEBUG
- Add one-time runai config log when hijack is activated
- Add `loader=runai|default` to model loading logs
- Remove per-file logging clutter from sd_hijack_safetensors.py
2025-12-02 02:01:51 +00:00
7aa1bfdc70
Add get_modules_to_not_convert from transformers v5
2025-12-02 01:01:51 +03:00
d9bc31e7da
Cleanup
2025-11-29 01:46:04 +03:00
01a0f6b356
Warn and disable quantized matmul if triton is not available
2025-11-29 01:34:54 +03:00
3e52009a4f
SDNQ assert Triton for quantized matmul
2025-11-29 00:54:19 +03:00
aaef4992c3
SDNQ fix svd + fp8 tw and fp16 mm
2025-11-28 22:31:09 +03:00
00d3be3dba
show_progress requires "full", "minimal", or "hidden"
2025-11-28 02:27:19 -08:00
328eb95851
Revert changes due to min Python version being 3.12
2025-11-27 14:57:51 -08:00
62b959a229
Fix importlib.metadata
2025-11-27 14:47:56 -08:00
a46f32b354
pull sdnq version from .common
2025-11-28 01:10:05 +03:00
55cf627ac6
add version to sdnq
2025-11-28 00:45:24 +03:00
368eb3103a
cleanup
2025-11-27 18:40:15 +03:00
73e4d1e379
Pass torch_dtype to sdnq loader
2025-11-27 18:37:35 +03:00
7b2a8e3f87
cleanup
2025-11-27 18:26:14 +03:00
ff4c254930
Auto handle tied weights with new transformers
2025-11-27 18:24:55 +03:00
9dd537072c
Fix import path for SDNQ options and handle Qwen models in load_sdnq_model
2025-11-27 14:53:03 +00:00
f7b0bbb6e1
Complete Z-Image support
2025-11-27 17:25:36 +03:00
e56dfc6a95
Merge pull request #4424 from CalamitousFelicitousness/z-image
...
Add Z-Image model support
2025-11-27 16:59:07 +03:00
4faddd7e45
Add Z-Image model support
2025-11-27 13:54:03 +00:00
131c51918b
SDNQ fix model_ oader
2025-11-27 14:51:45 +03:00
ed6f977218
SDNQ fix z_image matmul
2025-11-27 14:19:29 +03:00
16c429711c
update lumina and z_image keys
2025-11-26 23:22:44 +03:00
679060bd00
SDNQ add lumina and z_image keys
2025-11-26 22:51:15 +03:00
48b5d56ba4
Enable or disable quantized matmul on pre-quant models
2025-11-26 21:08:15 +03:00
6d9555fc3c
cleanup
...
Signed-off-by: vladmandic <mandic00@live.com >
2025-11-26 10:52:37 -05:00
ad6458aa7b
add flux2
...
Signed-off-by: vladmandic <mandic00@live.com >
2025-11-26 15:47:17 +00:00
70b96daa63
cleanup
2025-11-25 23:02:01 +03:00
c1a7424c76
ipex fix flux2 and cleanup
2025-11-25 22:51:39 +03:00
da0df35106
fix typo
2025-11-25 21:58:53 +03:00
da3c439059
SDNQ fix _tied_weights_keys is dict case
2025-11-25 19:37:46 +03:00
aeb71d172e
SDNQ add Flux2Transformer2DModel keys
2025-11-25 19:22:02 +03:00
81e44ca90f
Merge branch 'dev' of https://github.com/vladmandic/sdnext into dev
2025-11-25 10:36:18 -05:00
0da06b642a
fix networks refresh
...
Signed-off-by: vladmandic <mandic00@live.com >
2025-11-25 10:35:47 -05:00
CalamitousFelicitousness
vladmandic
Disty0
awsr