Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-19 17:25:07 +02:00
Code Issues Packages Projects Releases Wiki Activity

refactor: handle GGML_VK_VISIBLE_DEVICES at the Python level (#2179)

Browse Source 
All C++ handling code currently:
- build a comma-separated list from the info_vulkan array
- if GGML_VK_VISIBLE_DEVICES isn't set
  - set GGML_VK_VISIBLE_DEVICES to the list

Once set, GGML_VK_VISIBLE_DEVICES affects the whole process. So this
can be done in the same way at the Python level, before all loading
functions.

Caveat: load_model had the default `inputs.vulkan_info = "0"`,
so the default GPU would be "0" only when loading a text model.
This commit is contained in:
Wagner Bruna
2026-05-02 12:10:29 -03:00
committed by GitHub
parent 42ce63fd3b
commit 25fab4113e
8 changed files with 3 additions and 120 deletions
Download Patch File Download Diff File Expand all files Collapse all files
otherarch/tts_adapter.cpp
-16
View File
@@ -478,7 +478,6 @@ static llama_context * cts_ctx = nullptr; //codes to speech
static TTS_VER ttsver = TTS_VER_2;
static int ttsdebugmode = 0;
static bool tts_is_quiet = false;
static std::string ttsvulkandeviceenv;
static std::string last_generated_audio = "";
static std::string last_generation_settings_prompt = ""; //for caching purposes to fix ST bug
static int last_generation_settings_speaker_seed;
@@ -511,27 +510,12 @@ bool ttstype_load_model(const tts_load_model_inputs inputs)
tts_is_quiet = inputs.quiet;
tts_executable_path = inputs.executable_path;
//duplicated from expose.cpp
std::string vulkan_info_raw = inputs.vulkan_info;
std::string vulkan_info_str = "";
for (size_t i = 0; i < vulkan_info_raw.length(); ++i) {
vulkan_info_str += vulkan_info_raw[i];
if (i < vulkan_info_raw.length() - 1) {
vulkan_info_str += ",";
}
}
const char* existingenv = getenv("GGML_VK_VISIBLE_DEVICES");
std::vector<ggml_backend_dev_t> devices_override;
std::string dev_override_str = inputs.devices_override;
if(dev_override_str!="")
{
devices_override = kcpp_parse_device_list(dev_override_str);
}
if(!existingenv && vulkan_info_str!="")
{
ttsvulkandeviceenv = "GGML_VK_VISIBLE_DEVICES="+vulkan_info_str;
putenv((char*)ttsvulkandeviceenv.c_str());
}
llama_backend_init();