Public/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-20 01:31:31 +02:00
Code Issues Packages Projects Releases Wiki Activity

server: fix LRU hang on multiple requests same model

Browse Source
This commit is contained in:
Xuan Son Nguyen
2026-09-07 12:35:51 +02:00
parent 5202104b59
commit f2dc72b866
3 changed files with 83 additions and 88 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/server/server-models.h
+4
View File
@@ -216,6 +216,10 @@ private:
// not thread-safe, caller must hold mutex
void add_model(server_model_meta && meta);
// ask the monitoring thread to stop a running instance
// not thread-safe, caller must hold mutex
void request_stop(const std::string & name);
// notify SSE clients
void notify_sse(const std::string & event, const std::string & model_id, const json & data = nullptr);