mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-20 01:31:31 +02:00
common : enable default speculative config
Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp
This commit is contained in:
+4
-1
@@ -26,7 +26,7 @@ ctx-size = 0
|
||||
mmap = 1
|
||||
kv-unified = 1
|
||||
parallel = 4
|
||||
spec-default = 1
|
||||
no-spec-type = ngram-mod
|
||||
|
||||
[Qwen3.5-4B]
|
||||
hf = unsloth/Qwen3.5-4B-GGUF:Q4_K_M
|
||||
@@ -52,6 +52,9 @@ chat-template-kwargs = {"reasoning_effort": "high"}
|
||||
|
||||
The preset will be loaded similarly to the `--models-preset` option. Therefore, you can also override certain params via CLI arguments:
|
||||
|
||||
> [!NOTE]
|
||||
> A default speculative decoding is enabled by default for the server and CLI.
|
||||
|
||||
```sh
|
||||
# Force temp = 0.1, overriding the preset value
|
||||
llama-cli -hf username/my-preset --temp 0.1
|
||||
|
||||
+5
-3
@@ -226,10 +226,12 @@ Use exactly one of these options:
|
||||
```
|
||||
--spec-type [none|draft-simple|draft-eagle3|draft-dflash|draft-dspark|draft-mtp|ngram-cache|ngram-simple|ngram-map-k|ngram-map-k4v|ngram-mod]
|
||||
comma-separated list of types of speculative decoding to use
|
||||
(default: none)
|
||||
(default: server/CLI default config, none for other tools)
|
||||
(env: LLAMA_ARG_SPEC_TYPE)
|
||||
--spec-default use default speculative decoding config
|
||||
(enables ngram-mod)
|
||||
--no-spec-type [none|draft-simple|draft-eagle3|draft-dflash|draft-dspark|draft-mtp|ngram-cache|ngram-simple|ngram-map-k|ngram-map-k4v|ngram-mod]
|
||||
comma-separated list of types of speculative decoding to remove
|
||||
(env: LLAMA_ARG_NO_SPEC_TYPE)
|
||||
--spec-default DEPRECATED: default speculative decoding is enabled by default
|
||||
```
|
||||
|
||||
### Draft Model Parameters
|
||||
|
||||
Reference in New Issue
Block a user