mirror of
https://github.com/vladmandic/automatic
synced 2026-09-18 16:54:33 +02:00
feat(lora): default the select-mode ramp to 0
At 1.5 the ramp hands nearly every layer to the style network mid-generation, which on few-step models overwrites the forming subject before identity sets. Alpha 0 freezes the schedule into a static per-layer split: the subject keeps its layers for the whole generation and style keeps the layers where it is more salient. Nonzero values remain the scheduled handover toward the second network. - locale hints describe both regimes; the mode hint no longer implies the shift is always on
This commit is contained in:
@@ -711,7 +711,7 @@ def create_settings(cmd_opts):
|
||||
"lora_stack_sep": OptionInfo("<h2>Stacking options</h2>", "", gr.HTML),
|
||||
"lora_stack_mode": OptionInfo("sum", "LoRA stack mode", gr.Dropdown, {"choices": ["sum", "ties", "dare_ties", "dare_linear", "magnitude_prune", "klora", "estlora"]}),
|
||||
"lora_stack_density": OptionInfo(0.5, "LoRA stack density", gr.Slider, {"minimum": 0.05, "maximum": 1.0, "step": 0.05}),
|
||||
"lora_stack_alpha": OptionInfo(1.5, "LoRA stack ramp", gr.Slider, {"minimum": 0.0, "maximum": 3.0, "step": 0.1}),
|
||||
"lora_stack_alpha": OptionInfo(0.0, "LoRA stack ramp", gr.Slider, {"minimum": 0.0, "maximum": 3.0, "step": 0.1}),
|
||||
"lora_stack_discrepancy": OptionInfo(0.5, "LoRA stack discrepancy", gr.Slider, {"minimum": 0.0, "maximum": 1.0, "step": 0.05}),
|
||||
|
||||
"lora_meta_sep": OptionInfo("<h2>Metadata</h2>", "", gr.HTML),
|
||||
|
||||
@@ -867,9 +867,9 @@
|
||||
{"id":"","label":"LoRA quantized host rank","localized":"","hint":"Maximum rank used to carry adapter types that are not natively low-rank (LoKR, LoHA, OFT, DoRA) alongside the quantized weights instead of merging them in.<br>Higher values retain more of the adapter at proportionally more memory. Plain LoRA files are carried exactly at their own rank.<br><br>Applies only to SDNQ models quantized below 8 bits, where merging erases most of the adapter; at 8 bits and above merging retains it and hosting is skipped.<br><br><b>0</b> disables hosting and merges every adapter into the quantized weights.<br><br>Default is <b>256</b>.","ui":"settings_lora"},
|
||||
{"id":"","label":"LoRA quantized host calibration","localized":"","hint":"Collects per-channel activation statistics from the model's own generations and uses them to focus hosted-adapter truncation on the channels with the strongest activations.<br>Statistics accumulate in the background on models quantized below 8 bits, persist per checkpoint, and raise delivered adapter fidelity at the same <b><i>LoRA quantized host rank</i></b>, most at low ranks.<br><br>Capture is skipped while the model is compiled; previously cached statistics still apply.<br><br>Enabled by default.","ui":"settings_lora"},
|
||||
{"id":"","label":"LoRA quantized host cache","localized":"","hint":"Disk space in GB for caching computed hosting factors.<br>A cached set skips the truncation math on the next load; least recently used entries are evicted once the budget is exceeded.<br><br><b>0</b> disables the cache.<br><br>Default is <b>10</b>.","ui":"settings_lora"},
|
||||
{"id":"","label":"LoRA stack mode","localized":"","hint":"How multiple networks targeting the same layer are combined:<br>- <b>sum</b>: adds all contributions<br>- <b>ties</b>: keeps each network's strongest elements and merges only where signs agree<br>- <b>dare_ties</b>: randomly drops elements, rescales the survivors, then merges where signs agree<br>- <b>dare_linear</b>: randomly drops elements, rescales the survivors and sums<br>- <b>magnitude_prune</b>: keeps each network's strongest elements and sums<br>- <b>klora</b> / <b>estlora</b>: assign each layer to one of exactly two networks, the first in the prompt as subject and the second as style, shifting from subject toward style over the sampling steps<br><br>Each layer is given to a single network at a time, so a subject and a style that both need sustained strength can end up under-applied. For reliable blending of two strong networks, <b>sum</b>, <b>ties</b> and <b>dare_ties</b> apply every network throughout and combine more fully.<br><br>Kept fractions are set by <b><i>LoRA stack density</i></b>; the subject-to-style shift by <b><i>LoRA stack ramp</i></b> and <b><i>LoRA stack discrepancy</i></b>.<br><br>Applies to the native load path; other load methods and text encoder networks always combine as <b>sum</b>. Selection modes fall back to <b>sum</b> unless exactly two networks are loaded, or when model compile is active.<br><br>Default is <b>sum</b>.","ui":"settings_lora"},
|
||||
{"id":"","label":"LoRA stack mode","localized":"","hint":"How multiple networks targeting the same layer are combined:<br>- <b>sum</b>: adds all contributions<br>- <b>ties</b>: keeps each network's strongest elements and merges only where signs agree<br>- <b>dare_ties</b>: randomly drops elements, rescales the survivors, then merges where signs agree<br>- <b>dare_linear</b>: randomly drops elements, rescales the survivors and sums<br>- <b>magnitude_prune</b>: keeps each network's strongest elements and sums<br>- <b>klora</b> / <b>estlora</b>: assign each layer to one of exactly two networks, the first in the prompt as subject and the second as style; <b><i>LoRA stack ramp</i></b> optionally shifts layers toward style over the sampling steps<br><br>Each layer is given to a single network at a time, so a subject and a style that both need sustained strength can end up under-applied. For reliable blending of two strong networks, <b>sum</b>, <b>ties</b> and <b>dare_ties</b> apply every network throughout and combine more fully.<br><br>Kept fractions are set by <b><i>LoRA stack density</i></b>; the subject-to-style shift by <b><i>LoRA stack ramp</i></b> and <b><i>LoRA stack discrepancy</i></b>.<br><br>Applies to the native load path; other load methods and text encoder networks always combine as <b>sum</b>. Selection modes fall back to <b>sum</b> unless exactly two networks are loaded, or when model compile is active.<br><br>Default is <b>sum</b>.","ui":"settings_lora"},
|
||||
{"id":"","label":"LoRA stack density","localized":"","hint":"Fraction of elements each network keeps under the <b>ties</b>, <b>dare_ties</b>, <b>dare_linear</b> and <b>magnitude_prune</b> stack modes.<br>Lower values keep only the strongest contributions and reduce interference between networks at the cost of per-network detail. The dare variants drop at random and rescale the survivors to preserve expected strength.<br><br>Default is <b>0.5</b>.","ui":"settings_lora"},
|
||||
{"id":"","label":"LoRA stack ramp","localized":"","hint":"Slope of the subject-to-style shift across the sampling steps in the <b>klora</b> and <b>estlora</b> stack modes.<br>Higher values shift layers to the style network earlier and more broadly; lower values keep the subject network dominant for longer.<br><br><b>0</b> keeps the balance fixed for the whole generation.<br><br>Default is <b>1.5</b>.","ui":"settings_lora"},
|
||||
{"id":"","label":"LoRA stack ramp","localized":"","hint":"Slope of the subject-to-style shift across the sampling steps in the <b>klora</b> and <b>estlora</b> stack modes.<br><br><b>0</b> keeps the layer assignment fixed for the whole generation: each layer stays with the network that is more salient there, which preserves the subject while the style keeps its own layers. Higher values hand layers to the style network progressively, ending in a style takeover; on few-step models the handover happens early enough to override the subject.<br><br>Default is <b>0</b>.","ui":"settings_lora"},
|
||||
{"id":"","label":"LoRA stack discrepancy","localized":"","hint":"Stand-in for the measured style separation the <b>estlora</b> stack mode would otherwise derive from data.<br>Higher values keep layers with the subject network longer; lower values let the style network take layers earlier.<br><br>Layer scores are balanced by each network's overall strength, so a louder network does not take layers on magnitude alone.<br><br>Applies only when <b><i>LoRA stack mode</i></b> is <b>estlora</b>.<br><br>Default is <b>0.5</b>.","ui":"settings_lora"},
|
||||
{"id":"","label":"LoRA auto-apply tags","localized":"","hint":"Automatically add trigger words/tags from LoRA metadata to your prompt.<br>Set to the number of tags to auto-apply, e.g., 3 = add top 3 trigger tags.<br>Set to 0 to disable, -1 to add all available tags.","ui":"settings_lora"},
|
||||
{"id":"","label":"LoRA memory cache","localized":"","hint":"How many LoRAs to keep in network for future use before requiring reloading from storage","ui":"settings_lora"},
|
||||
|
||||
Reference in New Issue
Block a user