Replace the six closure hijacks stacked in devices.set_sdpa_params with
a registry of declarative backends and one router installed in their
place. Each backend declares the constraints its closure carried as a
predicate, a priority matching its old stacking position, and a prepare
step that imports and configures the implementation; the router walks
the prepared entries by priority and hands declined calls to the
terminal backend (dynamic, flex) or the original sdpa, so fallback is
the router's job rather than each closure's.
- parity held: gates transcribed literally, the same kernel kwargs,
enable_gqa passed to the original only when set, torch_info keeps the
last prepared backend, the dynamic pin still set
- a backend enabled on a platform without it warns instead of silently
doing nothing
- the legacy set_* entry points are gone; devices.py installs the router
- test/test-attention-router.py checks every override subset against the
old stacking order, gate parity over 16,000 shape cases, dispatch,
terminal handoff and prepare isolation, offline
Model switches kept most of the previous model resident, and the next
load could stall in kernel reclaim while the freed memory was still held.
- strip group offload hooks in disable_offload so the meta move at unload
actually frees component weights; hook removal resolves wrapper
components that carry hooks on the inner model
- flush the torch pinned host cache in torch_gc so freed streaming
buffers return to the OS instead of staying cached in-process
- skip the pipe-level accelerator move for group-managed pipes: the
offload engine already placed every component, and the move only
dragged on-demand components to the accelerator for the trailing
eviction to undo
triton_version was initialized to False instead of None, causing
the `is None` checks to be skipped and torch_info to report triton=false
even when triton is functional
- Rename shadowing import in waifudiffusion batch to avoid F823/E0606
- Fix import order in cli/api-caption.py (stdlib before third-party)
- Rename local variable shadowing function name in cli/api-caption.py
- Remove unnecessary global statement in devices.bypass_sdpa_hijacks
- Add parse_florence_detections() and format_florence_response() to
vqa_detection for handling Florence-2 detection output formats
- Add bypass_sdpa_hijacks() context manager to devices.py for models
incompatible with SageAttention or other SDPA hijacks
- Add OpenCLIP model offload support when caption_offload is enabled
CalamitousFelicitousness
Vladimir Mandic
Dity0
awsr
Seunghoon Lee