Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-19 09:15:18 +02:00
Code Issues Packages Projects Releases Wiki Activity

refactor: handle GGML_VK_VISIBLE_DEVICES at the Python level (#2179)

Browse Source 
All C++ handling code currently:
- build a comma-separated list from the info_vulkan array
- if GGML_VK_VISIBLE_DEVICES isn't set
  - set GGML_VK_VISIBLE_DEVICES to the list

Once set, GGML_VK_VISIBLE_DEVICES affects the whole process. So this
can be done in the same way at the Python level, before all loading
functions.

Caveat: load_model had the default `inputs.vulkan_info = "0"`,
so the default GPU would be "0" only when loading a text model.
This commit is contained in:
Wagner Bruna
2026-05-02 12:10:29 -03:00
committed by GitHub
parent 42ce63fd3b
commit 25fab4113e
8 changed files with 3 additions and 120 deletions
Download Patch File Download Diff File Expand all files Collapse all files
otherarch/whispercpp/whisper_adapter.cpp
-18
View File
@@ -59,28 +59,10 @@ static std::string output_txt(struct whisper_context * ctx) {
void cb_log_disable(enum ggml_log_level , const char * , void * ) { }
static std::string whispervulkandeviceenv;
bool whispertype_load_model(const whisper_load_model_inputs inputs)
{
whisper_is_quiet = inputs.quiet;
//duplicated from expose.cpp
std::string vulkan_info_raw = inputs.vulkan_info;
std::string vulkan_info_str = "";
for (size_t i = 0; i < vulkan_info_raw.length(); ++i) {
vulkan_info_str += vulkan_info_raw[i];
if (i < vulkan_info_raw.length() - 1) {
vulkan_info_str += ",";
}
}
const char* existingenv = getenv("GGML_VK_VISIBLE_DEVICES");
if(!existingenv && vulkan_info_str!="")
{
whispervulkandeviceenv = "GGML_VK_VISIBLE_DEVICES="+vulkan_info_str;
putenv((char*)whispervulkandeviceenv.c_str());
}
std::string modelfile = inputs.model_filename;
printf("\nLoading Whisper Model: %s",modelfile.c_str());