Group hooks on the no-stream path and the on-demand hook return a
component to cpu through a device copy, so a memory-mapped text encoder
sits in memory twice, as the mapped file its never-run vision tower
keeps alive and as the copies, and every encode pays a device-to-host
transfer of unchanged weights. The engine now records the cpu tensors
at onload and hands them back at offload; a component moved by any
other path still takes the copy.
A checkpoint-name key survives unload, so a reload of the same checkpoint
never printed the per-module stats block again, even with a different
quantization. Each component now carries its own reported stamp: a task
switch rebuilds the pipe around the same modules and stays quiet, while a
reload or a component swap brings new modules and reports them.
CalamitousFelicitousness
Vladimir Mandic