mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-08-25 22:21:14 +02:00

Files

T

History

John-Henry Lim 681c29d36a mtmd: add --mmproj-device argument (#23255 )

* feat: add --mmproj-device arg & backwards compatible MTMD_BACKEND_DEVICE env var

* feat: load mmproj device backend immediately, add -mmdev shortflag

* fix: its a pointer now get the name

* clean up

* gen docs

* nits

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>

2026-08-20 18:45:37 +02:00

CMakeLists.txt

mtmd: support Qwen3-TTS (note: breaking change to llama-tts binary) (#26254 )

2026-08-04 17:26:15 +02:00

README.md

mtmd: support pocket-tts (#26871 )

2026-08-11 14:18:30 +02:00

tts.cpp

mtmd: add --mmproj-device argument (#23255 )

2026-08-20 18:45:37 +02:00

README.md

llama.cpp TTS

This is a tool to demonstrate audio generation capability in llama.cpp via libmtmd. It was added via PR #26254

Note: this tool used to serve as a demo for OuteTTS, but it was converted to a more model-agnostic tool.

Common usage

Simple usage:

llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF -p "Hello world" --output out.wav

Common params:

Sampling params such as --top-k, --top-p, --temp, etc.
-n <number_of_frames> limits the output length, e.g. -n 500. Note that how many milliseconds each frame represents varies by model
Core inference params such as -ngl, -b, -ub, etc.

Qwen3-TTS

Available params:

--tts-lang can be zh, en, de, it, pt, es, ja, ko, fr, ru (default: en)
--tts-speaker-file should point to a speaker reference audio file (wav, mp3)

Example usage:

llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF \
    -p "Hello world" \
    --tts-lang english \
    --tts-speaker-file speaker.mp3 \
    --output out.wav

Pocket TTS

Available params:

--tts-speaker-file should point to a speaker reference audio file (wav, mp3). It is required, the model produces almost no audio without it
Note: lang is not used, the language is a property of the weights

Example usage:

llama-tts -m pocket-tts.gguf \
    -mm mmproj-pocket-tts.gguf \
    -p "Hello world" \
    --tts-speaker-file speaker.mp3 \
    --output out.wav

Note for GGUF conversion:

The upstream repository holds one complete model per language under languages/, next to a set of shared files at the root. Convert one of the languages/<name> directories, not the root directory:

python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --outfile pocket-tts.gguf
python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --mmproj --outfile mmproj-pocket-tts.gguf