sdnext enables tf32 globally; a math-backend dispatch fallback (gqa,
some mask shapes) degrades the fp32 reference to tf32 precision,
measured 4.2e-4 vs 5.0e-7 relative to fp64. the default mem-efficient
backend is unaffected.
Recommendations must hold on any gpu, so every verdict now comes from
measuring the configuration the advice points to.
- run benchmarks in the webui's configured dtype, --dtype overrides
- bench the prep mode matching the environment: compiled, verified
dynamic=false workaround, or eager; label each with the webui change
the numbers correspond to
- add a prep column for the q/k/v quantization cost outside the
kernel; eager prep measured 1.9 ms vs 0.2 ms fused at sdxl,
inverting the quantized matmul verdict
- compare Use Quantized MatMul against the unquantized sdnq row so
quantization pays for its own prep
- star the lowest-error config within 5% of the fastest sdnq time;
the same rule drives the worth-enabling verdict vs torch sdpa
- drop the (auto, default) suffix; document the settings mapping and
star rule in the notes
- wrap correctness and shape tables in panels to separate sections
The pv matmul recommendation trusted the float8 hardware probe alone,
so gpus where fp8 pv compiles but runs slower than unquantized pv got
a recommendation that degrades generation speed and error. Measured on
an rtx 2000 ada, fp8 pv lands at x0.93 of the sdpa baseline while int8
qk alone reaches x1.26.
- add fp8 pv and fp8 qk bench configs, gated on the eager float8
probe so unsupported gpus keep clean tables
- include fp8 pv in the video presets since the pv stage matters most
at long sequence lengths
- recommend a quantized pv type only when the fastest measured pv
variant beats plain int8 qk by at least 5%, otherwise recommend auto
and name the variants that measured no gain
- cite the measured fp8 qk ratio in the matmul type reason instead of
the static compile claim
The environment panel reported triton as missing and float8 as
unsupported on ada gpus. The triton lookup used the distribution name,
which varies by platform (triton-windows on windows), and the fp8 probe
ran through the torch.compile input prep, so a broken compile toolchain
read as a hardware limitation with the actual error discarded.
- read the module version for the triton line instead of the
distribution name
- probe float8 with eager input prep so the result reflects gpu
capability; show the failure reason when genuinely unsupported
- probe the compiled input prep separately and cold, before any eager
kernel run: inductor lowers part of the dynamic-shape prep to a cpu
helper kernel, so a missing host c++ compiler fails every sdnq
attention call at generation, and a prior eager run in the same
process masks the cold-start failure the webui hits
- on failure, verify the SDNQ_COMPILE_KWARGS dynamic:false workaround
live and report it in the environment panel and recommendation notes
- benchmark with eager input prep when the compiled path is broken so
kernel numbers still fill in
- replace the unicode star marker with ascii: rich's legacy windows
renderer crashes with a cp1252 UnicodeEncodeError on piped output
Benchmark and validate sdnq attention on the local gpu: correctness
checks for mask, causal, gqa and padding code paths, float8 support
probe, per-shape speed and error comparison against torch sdpa and
sageattention, and measured recommendations for the settings section.
Tables fill in live with per-config compile/warmup/timing progress,
current settings are compared against recommended values, and --save
writes a plain-text transcript of the results. --config-timeout turns
stuck compiles into failed rows; sd15 skips its hadamard configs since
compiling hadamard with a non pow2 head dim currently hangs inductor.
- shape presets follow real model geometries: sd15, sdxl, anima,
flux2 klein, wan22 a14b, ltx 2.3, plus a masked joint-attention case
- runs the shipped kernel from modules/sdnq/kernels/triton_atten.py
directly, with stock sdpa restored for baselines and references
- Remove remote manifest fetch from startup dropdown population;
list only locally available tag files via files_cache.list_files()
- Rename CLI utilities to tags-fetch, tags-prune, tags-manifest
- Replace create/destroy mirror div with persistent offscreen mirror
for caret position measurement
- Add ResizeObserver to reposition dropdown on textarea resize
- Add autocomplete-mirror classname to measurement div
- Remove superfluous SimpleNamespace import in cli/api-caption.py, use Map instead
- Drop _ prefix from internal helper functions in modules/api/caption.py
- Move DeepDanbooru model path to top-level models folder instead of nesting under CLIP
- Rename shadowing import in waifudiffusion batch to avoid F823/E0606
- Fix import order in cli/api-caption.py (stdlib before third-party)
- Rename local variable shadowing function name in cli/api-caption.py
- Remove unnecessary global statement in devices.bypass_sdpa_hijacks
- Add use_safetensors=True to all 16 model from_pretrained calls to
avoid downloading redundant .bin files alongside safetensors
- Add device property to JoyTag VisionModel so move_model can relocate
it to CUDA (fixes 'ViT object has no attribute device')
- Fix Pix2Struct dtype mismatch by casting float inputs to model dtype
while preserving integer tensor types
- Patch AutoConfig.register with exist_ok=True during Ovis loading to
handle duplicate aimv2 registration on model reload
- Detect Qwen VL fine-tune architecture from config model_type instead
of repo name, fixing ToriiGate and similar third-party fine-tunes
- Change UI default task from Short Caption to Normal Caption, and
preserve it on model switch instead of resetting to Use Prompt
- Add dual-prefill testing across 5 VQA test methods using a shared
_check_prefill helper
- Fix pre-existing ruff W605 in strip_think_xml_tags docstring
- Remove caption_openclip_min_length from settings, API models, endpoints, and UI
(clip_interrogator library has no min_length support; parameter was never functional)
- Split vlm_prompts_florence into base Florence prompts and PromptGen-only prompts
(GENERATE_TAGS, Analyze, Mixed Caption require MiaoshouAI PromptGen fine-tune)
- Add 'promptgen' category to /vqa/prompts API endpoint
- Fix gaze detection: move DETECT_GAZE check before generic 'detect ' prefix
to prevent "Detect Gaze" matching as detect target="Gaze"
- Update test suite: remove min_length tests, fix min_flavors to use mode='best',
add acceptance-only notes, fix thinking trace detection, improve bracket/OCR tests,
split Florence/PromptGen test coverage
Update cli/test-caption-api.py:
- Update test structure for new caption API endpoints
- Fix Moondream gaze detection test prompt to use 'Detect Gaze'
instead of 'Where is the person looking?' to match handler trigger
- Improve test result categorization and tracking
- Rename cli/api-interrogate.py to cli/api-caption.py
- Update cli/options.py, cli/process.py for new module paths
- Update cli/test-tagger.py for caption module imports
- Update cli/api-interrogate.py to use /sdapi/v1/tagger for DeepBooru
- Handle tagger response format (scores dict or tags string)
- Remove DeepBooru test from interrogate endpoint tests
- Update API model descriptions to reference tagger for anime tagging
Add model architecture coverage tests:
- VQA model family detection for 19 architectures
- Florence special prompts test (<OD>, <OCR>, <CAPTION>, etc.)
- Moondream detection features test
- VQA architecture capabilities test
- Tagger model types and WD version comparison tests
Improve test validation:
- Add is_meaningful_answer() to reject responses like "."
- Verify parameters have actual effect (not just accepted)
- Show actual output traces in PASS/FAIL messages
- Fix prefill tests to verify keep_prefill behavior
Add configurable timeout:
- Default timeout increased to 300s for slow models
- Add --timeout CLI argument for customization
Other improvements:
- Add JoyCaption to recognized model families
- Reduce BLIP models to avoid reloading large models
- Better detection result validation for annotated images
Comprehensive test script for all Caption API endpoints:
- GET/POST /sdapi/v1/interrogate (OpenCLiP/DeepBooru)
- POST /sdapi/v1/vqa (VLM captioning)
- GET /sdapi/v1/vqa/models, /sdapi/v1/vqa/prompts
- POST /sdapi/v1/tagger
- GET /sdapi/v1/tagger/models
Usage: python cli/test-caption-api.py [--url URL] [--image PATH]
CalamitousFelicitousness
Vladimir Mandic
QualiaRain
awsr