Public/automatic
1
0
Fork 0
mirror of https://github.com/vladmandic/automatic synced 2026-09-19 09:14:35 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs(offload): describe the group offload blocks setting

Browse Source 
The blocks count is clamped to one under streams and leaf level never
reads it; the hint says so, and the type hint names the setting it
refers to.
This commit is contained in:
CalamitousFelicitousness
2026-09-16 02:13:28 +01:00
parent 09e861157f
commit c3fc8560cb
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ui/locale/locale_en.json
+2 -1
View File
@@ -624,7 +624,8 @@
{"id":"","label":"Generic","localized":"","hint":"","ui":"video"},
{"id":"","label":"Google GenAI","localized":"","hint":"","ui":"settings_model_options"},
{"id":"","label":"Group Offload","localized":"","hint":"Offloads components in groups of layers rather than as a whole, so only the layers in use occupy VRAM.<br>Lets a single component larger than the card run, at the cost of transferring weights throughout every step.<br><br>Applies only when <b><i>Model offload mode</i></b> is <b>group</b>.","ui":"settings_offload"},
{"id":"","label":"Group offload type","localized":"","hint":"Granularity used by <b>group</b> offload.<br>- <b>leaf_level</b>: offloads at the smallest module level; maximum memory savings, slower<br>- <b>block_level</b>: offloads groups of transformer blocks (size set by <b><i>Offload blocks</i></b>); faster with less savings<br>This setting applies to the parts of the model that run at every step. Components used once per generation, such as text encoders, always offload at <b>leaf_level</b>. The VAE is handled separately: it waits in system memory and loads as a whole when encoding or decoding.<br>Anything named in <b><i>Modules to never offload</i></b> or <b><i>Model types not to offload</i></b> stays in VRAM instead.<br><br>Applies only when <b><i>Model offload mode</i></b> is <b>group</b>.<br><br>Default is <b>leaf_level</b>.","ui":"settings_offload"},
{"id":"","label":"Group offload type","localized":"","hint":"Granularity used by <b>group</b> offload.<br>- <b>leaf_level</b>: offloads at the smallest module level; maximum memory savings, slower<br>- <b>block_level</b>: offloads groups of transformer blocks (size set by <b><i>Group offload blocks</i></b>, one block when <b><i>Prefetch with streams</i></b> is enabled); faster with less savings<br>This setting applies to the parts of the model that run at every step. Components used once per generation, such as text encoders, always offload at <b>leaf_level</b>. The VAE is handled separately: it waits in system memory and loads as a whole when encoding or decoding.<br>Anything named in <b><i>Modules to never offload</i></b> or <b><i>Model types not to offload</i></b> stays in VRAM instead.<br><br>Applies only when <b><i>Model offload mode</i></b> is <b>group</b>.<br><br>Default is <b>leaf_level</b>.","ui":"settings_offload"},
{"id":"","label":"Group offload blocks","localized":"","hint":"Number of transformer blocks moved together as one group on <b>block_level</b> group offload. Larger groups mean fewer, larger transfers and more weights resident in VRAM at once.<br>Ignored when <b><i>Prefetch with streams</i></b> is enabled, which runs one block per group, and on <b>leaf_level</b>, which has no blocks. Components used once per generation always offload at <b>leaf_level</b> and never read this value.<br><br>Applies only when <b><i>Model offload mode</i></b> is <b>group</b>.<br><br>Default is <b>1</b>.","ui":"settings_offload"},
{"id":"","label":"Grid Options","localized":"","hint":"","ui":"settings_saving-images"},
{"id":"","label":"Grids","localized":"","hint":"","ui":"settings_saving-paths"},
{"id":"","label":"Guider","localized":"","hint":"","ui":"txt2img"},