Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-20 01:31:42 +02:00
Code Issues Packages Projects Releases Wiki Activity

gracefully handle bad grammar instead of crashing

Browse Source
This commit is contained in:
Concedo
2026-03-23 17:00:53 +08:00
parent ef854f002e
commit 993925ba96
3 changed files with 17 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
gpttype_adapter.cpp
+4
View File
@@ -789,6 +789,10 @@ static speculative_draft_result speculative_decoding_eval_chunk(llama_context *
// KCPP SAMPLING FUNCTIONS
void sample_softmax(llama_token_data_array * cur_p, bool do_sort=true) {
if(!(cur_p->size > 0))
{
throw std::runtime_error("No valid candidates during sampling. Current request aborted!");
}
GGML_ASSERT(cur_p->size > 0);
// Sort the logits in descending order
if (!cur_p->sorted && do_sort) {