common : enable default speculative config

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp
This commit is contained in:
Georgi Gerganov
2026-09-04 10:59:53 +03:00
parent d646c9d155
commit 7c2e0dea43
10 changed files with 196 additions and 25 deletions
+4 -1
View File
@@ -26,7 +26,7 @@ ctx-size = 0
mmap = 1
kv-unified = 1
parallel = 4
spec-default = 1
no-spec-type = ngram-mod
[Qwen3.5-4B]
hf = unsloth/Qwen3.5-4B-GGUF:Q4_K_M
@@ -52,6 +52,9 @@ chat-template-kwargs = {"reasoning_effort": "high"}
The preset will be loaded similarly to the `--models-preset` option. Therefore, you can also override certain params via CLI arguments:
> [!NOTE]
> A default speculative decoding is enabled by default for the server and CLI.
```sh
# Force temp = 0.1, overriding the preset value
llama-cli -hf username/my-preset --temp 0.1
+5 -3
View File
@@ -226,10 +226,12 @@ Use exactly one of these options:
```
--spec-type [none|draft-simple|draft-eagle3|draft-dflash|draft-dspark|draft-mtp|ngram-cache|ngram-simple|ngram-map-k|ngram-map-k4v|ngram-mod]
comma-separated list of types of speculative decoding to use
(default: none)
(default: server/CLI default config, none for other tools)
(env: LLAMA_ARG_SPEC_TYPE)
--spec-default use default speculative decoding config
(enables ngram-mod)
--no-spec-type [none|draft-simple|draft-eagle3|draft-dflash|draft-dspark|draft-mtp|ngram-cache|ngram-simple|ngram-map-k|ngram-map-k4v|ngram-mod]
comma-separated list of types of speculative decoding to remove
(env: LLAMA_ARG_NO_SPEC_TYPE)
--spec-default DEPRECATED: default speculative decoding is enabled by default
```
### Draft Model Parameters