mirror of
https://github.com/vladmandic/automatic
synced 2026-08-30 17:11:00 +02:00
CalamitousFelicitousness
cf031b869c
Benchmark weight dequantization alongside attention: eager vs compiled dequantization, measured standalone and through the full linear forward, plus the quantized matmul forward, across int/uint 8-6-4-2 and the pre-quantized float formats (float8_e4m3fn, float8_e4m3fn_sdnq, float4_e2m1fn group 16) at krea 2 layer geometry, against a bf16 nn.Linear baseline and a true-fp32 output reference. Each row carries measured storage size and one-shot quantize time; an svd/hadamard variants table measures rotation and low-rank costs on top of the base dtypes. A combined block section measures complete configurations (weights dtype x matmul path x attention) end to end through a dit-style transformer block, with output error at depth one and four against an fp32 reference block, because component speedups and errors do not compose multiplicatively. - recommendations weigh error against speed: a faster option is rejected when it multiplies measured output error beyond 2x, and reason strings cite both numbers; notes include measured size/error and speed/error frontiers - robustness: extreme-activation stress rows in the correctness matrix (pass on finite output), max-token-error columns beside norm error, and non-finite outputs labeled as verdicts; norm metrics alone hide token-level corruption on outlier-heavy inputs - probe compiled weight dequant for e4m3 and e5m2 storage and report the fp8 compile gate status; on ampere e5m2 compiles while e4m3 does not - new attention presets: sdxl-cross (cross-attention), qwen3-te (causal gqa text encoder), wan22-cfg (batched cfg video); cross-attention correctness check - --sections, --dequant-dtypes, --dequant-variants and --block-configs selectors, --json structured results output