Autotune sweeps and kernel compiles run inside the first forward pass at a
new shape and can take minutes with no indication in the log or UI. Add
listeners on the triton autotuning and compilation knobs plus a wrap of the
per-candidate benchmark: a sweep draws a console progress bar over its
candidates and mirrors the count in the live progress text, the completion
line records the kernel, shape, duration and compile share, and standalone
compiles over 1s are logged.
- fix timer_sdnq reading bench_time from the python wrapper functions
instead of the autotuner kernel objects, which left the two matmul
autotune timers permanently empty
- restore the pre-tuning progress text across chained sweeps, so an
abandoned sweep cannot strand its own tuning label in the UI
The audio waveform rides as an attribute on the decoded sample list, but
the batch script hooks rewrap samples into a plain list before the
capture ran, so joint audio-video models muxed silent files whenever a
script runner was attached. Capture the attribute before the hooks run.
The keep-current-scheduler sentinel leaked into user-facing surfaces:
infotexts and filenames recorded Sampler: None, restoring such an
infotext pushed an invalid value into the dropdown, and the API
samplers list never offered Default at all.
- infotext always records the sampler, mapping a stray None to Default
- parse maps legacy Sampler: None infotexts to Default
- api samplers list leads with Default; the remaining excluded config
keys are shared templates, not samplers
Dynamo tracks a lifetime recompile counter per compiled function that
freed models leave climbing while their graphs and guards die, and the
compiled dequant runs fullgraph, so crossing the accumulated limit is a
hard FailOnRecompileLimitHit instead of an eager fallback; enough model
or quant switches in one process got there. unload_model_weights now
calls reset_compile_caches when the compiled dequant is active, dropping
the dead graphs and the counters in the same sweep as the unload gc.
Raised limits only move the wall; the reset removes it.
- scoped to the model unload branch: the reset is global and must only
run when the graphs' owner is being discarded
- regression test trips the wall under a lowered limit and recovers
through the same helper the unload path calls
NATIVE_DISPATCH is the documented registration surface for per-arch
native loaders and is read cross-module by the fidelity analyzer, so
the private marker signaled the opposite of its role and enforced
nothing.
safe_open exposes keys() but implements no __iter__, so the dict idiom applied
in d11a619a6 raises TypeError on every non-streamer load of a pre-quantized
checkpoint. The threaded method reaches the same loop, since load_threaded
delegates one file at a time.
Cached networks are shared objects, and network_load overwrote their
multipliers before network_deactivate ran, so fuse-mode removal recomputed
the subtraction delta with the new values: a strength edit froze at its
first applied value and a later removal left residue in the model weights.
network_load now stages the values on the net and network_activate promotes
them, so the removal pass always subtracts the delta that was applied.
Backup mode restores from stored tensors and was unaffected.
Backup-mode apply and restore installed fresh Parameters. Matmul kernel
selection is sensitive to operand placement, so the first load/remove cycle
shifted otherwise deterministic renders once per process even though every
weight restored byte-exact: bit-identical inputs entered the first post-cycle
unet forward and a different output left it. Copying into the existing
parameter keeps each touched module on its load-time allocation and drops the
per-layer transient of holding old and new weights side by side.
- assign_weight writes weight and bias installs in place when shape, dtype
and device match; quantized fallback layers keep their rebuild path
- regression test pins storage stability across the activate walk
Under a pressed balanced offload, dispatched modules hold meta tensors
whose data lives in the accelerate offload map. The factor path raised
trying to move a meta svd tensor and aborted activation mid-pass; the
legacy requantize path silently skipped those layers. Both left the
model with a partially applied network.
Rebuild the offload state with apply_balanced_offload(force) at
activate and deactivate entry: modules come back real on cpu with
hooks intact and the execution device unchanged, so both paths see
usable tensors and the next forward re-onloads under the watermark.
network_load seeded net.dyn_dim with extra_networks_default_multiplier
when no dyn_dims list was passed, so a float multiplier landed where
consumers expect a rank and slice with it. The prompt path always builds
a per-network list of ints or None, which is why the crash never fired
from the UI; any direct network_load caller hits it in both
rebuild_conventional and the sdnq factor path.
create_module built each up/down module with the default constructor,
which kaiming-initializes the parameter, then copied the stored weight
over the whole thing. The init is thrown away every time and costs about
four times the copy: 22.1ms per module against 2.5ms, or 5.8s against
0.7s over a 264-module lora, on every load.
skip_init constructs on meta and materializes uninitialized, so the copy
still fully defines the parameter. Dtype, device and values are
unchanged, including the fp32 upcast of bf16 files that the copy performs.
Vladimir Mandic
Dity0
CalamitousFelicitousness