Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-08-29 08:30:51 +02:00
Code Issues Packages Projects Releases Wiki Activity
5,293 Commits 3 Branches 253 Tags
e39b8aab8b4ece253d9fe088e356ac2f6d34da50
Commit Graph

5293 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Concedo e39b8aab8b improvements to auto layer calcs 2024-07-29 18:51:10 +08:00
Concedo f289fb494a bump size of some payload arr sequences from 16 to 24 2024-07-28 20:29:39 +08:00
Concedo e47477fd4d don't build rope factors from https://github.com/ggerganov/llama.cpp/pull/8676 for CLBlast as it segfaults v1.71.1 2024-07-28 17:27:09 +08:00
Concedo edbdfbced2 Revert "cu11 build threads" 
This reverts commit c3aa259907a77b19bb5c94015de61b8178b9d283. (+2 squashed commit)

Squashed commit:

[bf2f7e7c] missing include

[c3aa2599] cu11 build threads
 2024-07-28 16:46:10 +08:00
Concedo 0029e36f50 fix for older phi3 models without swa 2024-07-28 12:13:38 +08:00
Concedo 01afb28a63 not working 2024-07-28 11:43:10 +08:00
R0CKSTAR e54c35e4fb feat: Support Moore Threads GPU (#8383) 
* Update doc for MUSA

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>

* Add GGML_MUSA in Makefile

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>

* Add GGML_MUSA in CMake

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>

* CUDA => MUSA

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>

* MUSA adds support for __vsubss4

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>

* Fix CI build failure

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>

---------

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2024-07-28 01:41:25 +02:00
Concedo ba5babb876 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/nix/apps.nix
#	.devops/tools.sh
#	Makefile
#	README.md
#	docs/backend/SYCL.md
#	docs/build.md
#	examples/CMakeLists.txt
#	ggml/include/ggml.h
#	src/llama-vocab.cpp
#	tests/test-backend-ops.cpp
#	tests/test-chat-template.cpp
#	tests/test-sampling.cpp
 2024-07-27 23:15:54 +08:00
Georgi Gerganov 5e2727fe03 scripts : sync vulkan-shaders (#0) 2024-07-27 18:08:47 +03:00
Georgi Gerganov 56f20aa25d scripts : sync ggml-aarch64 sources 2024-07-27 18:07:33 +03:00
Georgi Gerganov 345c8c0c87 ggml : add missing semicolon (#0) 
ggml-ci
 2024-07-27 17:43:44 +03:00
Georgi Gerganov ae7985cd7b sync : ggml 
ggml-ci
 2024-07-27 17:43:44 +03:00
Mahesh Madhav a05ca93697 ggml : loop tiling optimizations for scalar path (ggml/898) 
Apply a loop tiling technique to the generic path, which provides
performance upside for ISAs with enough registers to take advantage
of it. Also helps the compiler optimize this path.
 2024-07-27 17:43:44 +03:00
Ivan Filipov 9f77d899b7 ggml: add support for float16 input tensors in pooling operations (ggml/895) 
* Add support for float16 tensors in 1d pooling operations

* Add support for float16 input tensors in 2d pooling operations

* code cleanup

remove unnecessary casting during srow ptr initialization

---------

Co-authored-by: vanaka11 <vanaka1189@gmail.com>
 2024-07-27 17:43:44 +03:00
Tony Wasserka 203b7f1531 vulkan : initialize vk_buffer_struct members to VK_NULL_HANDLE (ggml/893) 
This prevents invalid frees when destroying a partially initialized
vk_buffer_struct. For example, this could happen in ggml_vk_create_buffer
when running out of device memory.

Co-authored-by: Tony Wasserka <neobrain@users.noreply.github.com>
 2024-07-27 17:43:44 +03:00
Borislav Stanimirov d2b851bfa1 cmake : only enable GGML_NATIVE and x86 flags if not crosscompiling (ggml/885) 2024-07-27 17:43:44 +03:00
Daniel Bevenius c12b6e8ee7 ggml : remove unnecessary UNUSED macro call (ggml/880) 
This commit removes an UNUSED macro call that is not needed as the
variable n0 is used in the code and will not produce a warning.

Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
 2024-07-27 17:43:44 +03:00
Concedo eaa702852d increased padding, it is still way too little but whatever 2024-07-27 22:32:13 +08:00
Jeffrey Morgan b5e95468b1 llama : add support for llama 3.1 rope scaling factors (#8676) 
* Add llama 3.1 rope scaling factors to llama conversion and inference

This commit generates the rope factors on conversion and adds them to the resulting model as a tensor. At inference time, these factors are passed to the `ggml_rope_ext` rope oepration, improving results for context windows above 8192

* Update convert_hf_to_gguf.py

Co-authored-by: compilade <git@compilade.net>

* address comments

* address comments

* Update src/llama.cpp

Co-authored-by: compilade <git@compilade.net>

* Update convert_hf_to_gguf.py

Co-authored-by: compilade <git@compilade.net>

---------

Co-authored-by: compilade <git@compilade.net>
 2024-07-27 15:03:45 +03:00
Georgi Gerganov 92090eca21 llama : add function for model-based max number of graph nodes (#8622) 
* llama : model-based max number of graph nodes

ggml-ci

* llama : disable 405B max_nodes path due to lack of complaints

ggml-ci
 2024-07-27 14:59:29 +03:00
Daniel Bevenius 9d03d085dd common : add --no-warmup option for main/llama-cli (#8712) 
This commit adds a --no-warmup option for llama-cli.

The motivation for this is that it can be convenient to skip the
warmup llama_decode call when debugging.

Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
 2024-07-27 13:45:02 +03:00
wangshuai09 bfb4c74981 cann: Fix Multi-NPU execution error (#8710) 
* cann: fix multi-npu exec error

* cann: update comment  for ggml_backend_cann_supports_buft
 2024-07-27 16:36:44 +08:00
Concedo 729eb1e552 no fast forward for empty prompt 2024-07-27 16:29:35 +08:00
slaren 2b1f616b20 ggml : reduce hash table reset cost (#8698) 
* ggml : reduce hash table reset cost

* fix unreachable code warnings after GGML_ASSERT(false)

* GGML_ASSERT(false) -> GGML_ABORT("fatal error")

* GGML_ABORT use format string
 2024-07-27 04:41:55 +02:00
Concedo 4531ab5465 refactor some fields 2024-07-27 00:04:29 +08:00
Judd 01245f5b16 llama : fix order of parameters (#8706) 
usage of `aclrtGetMemInfo` is correct:

https://www.hiascend.com/doc_center/source/zh/canncommercial/63RC2/inferapplicationdev/aclcppdevg/aclcppdevg_03_0103.html

Co-authored-by: Judd <foldl@boxvest.com>
 2024-07-26 11:38:12 +03:00
Yaiko 01aec4a631 server : add Speech Recognition & Synthesis to UI (#8679) 
* server : add Speech Recognition & Synthesis to UI

* server : add Speech Recognition & Synthesis to UI (fixes)
 2024-07-26 00:10:16 +02:00
Xuan Son Nguyen 41cd47caab examples : export-lora : fix issue with quantized base models (#8687) 2024-07-25 23:49:39 +02:00
DavidKorczynski 49ce0ab6d4 ggml: handle ggml_init failure to fix NULL pointer deref (#8692) 
`ggml_init` can fail if no unused context is found. In that case, a NULL-pointer deref will happen later in the code during a call to `ggml_set_on_alloc`.

This fixes it by bailing out if no context is found.
 2024-07-25 23:23:05 +02:00
Georgi Gerganov 4226a8d10e llama : fix build + fix fabs compile warnings (#8683) 
ggml-ci
 2024-07-25 19:57:31 +03:00
Andreas (Andi) Kunar bf5a81df37 ggml : fix build on Windows with Snapdragon X (#8531) 
* Improvements for Windows with Snapdragon X

* Revert "Improvements for Windows with Snapdragon X"

This reverts commit bf21397ae5ea7c73d3494db3b91505599909227d.

* Improvements for Windows with Snapdragon X

* WOA build clarifications

* WIndows on ARM build clarifications

* cmake build for Windows clarifications

* Update docs/build.md

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: AndreasKunar <andreaskmsn.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-07-25 19:01:00 +03:00
Georgi Gerganov 88954f7fbd tests : fix printfs (#8068) 2024-07-25 18:58:04 +03:00
Concedo 9f2076b4b3 fix rocminfo error 2024-07-25 22:23:36 +08:00
Chen Xi ed67bcb24f [SYCL] fix multi-gpu issue on sycl (#8554) 
---------

Signed-off-by: Chen Xi <xi2chen@intel.com>
Co-authored-by: Meng, Hengyu <hengyu.meng@intel.com>
 2024-07-25 19:45:18 +08:00
Georgi Gerganov eddcb5238b ggml : add and use ggml_cpu_has_llamafile() (#8664) 2024-07-25 12:37:42 +03:00
Xuan Son Nguyen be6d7c0791 examples : remove finetune and train-text-from-scratch (#8669) 
* examples : remove finetune and train-text-from-scratch

* fix build

* update help message

* fix small typo for export-lora
 2024-07-25 10:39:04 +02:00
Ujjawal Panchal 4b0eff3df5 docs : Quantum -> Quantized (#8666) 
* docfix: imatrix readme, quantum models -> quantized models.

* docfix: server readme: quantum models -> quantized models.
 2024-07-25 11:13:27 +03:00
Fan Shupei 8a4bad50a8 llama: use sliding window for phi3 (#8627) 
* use sliding window for phi3

* fix typo, "data_swa" -> "data"

* [conver_hf_to_gguf.py] add phi3 sliding window
 2024-07-25 10:21:09 +03:00
Concedo a84f7c5d81 revert num old cpu for ci v1.71 2024-07-25 13:24:34 +08:00
Concedo 57a98ba308 fixed dict loading 2024-07-25 11:41:05 +08:00
Concedo 0024d9d682 fixed order of selection 2024-07-25 11:15:30 +08:00
MorganRO8 68504f0970 readme : update games list (#8673) 
Added link to game I made that depends on llama
 2024-07-24 19:48:00 +03:00
Concedo d1f7832d21 adjusted layer estimation 2024-07-24 22:51:02 +08:00
Concedo cca2fa9a6c Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/llama-cli-intel.Dockerfile
#	.devops/llama-server-intel.Dockerfile
#	README.md
#	ggml/src/CMakeLists.txt
#	tests/test-chat-template.cpp
 2024-07-24 21:57:50 +08:00
Concedo e28c42d7f7 adjusted layer estimation 2024-07-24 21:54:49 +08:00
Joe Todd f19bf99c01 Build Llama SYCL Intel with static libs (#8668) 
Ensure SYCL CI builds both static & dynamic libs for testing purposes

Signed-off-by: Joe Todd <joe.todd@codeplay.com>
 2024-07-24 14:36:00 +01:00
Thorsten Sommer 3a7ac5300a readme : update UI list [no ci] (#8505) 2024-07-24 15:52:30 +03:00
Concedo b7fc8e644a fix broken template, updated lite 2024-07-24 20:47:05 +08:00
Xuan Son Nguyen 96952e7181 llama : fix llama_chat_format_single for mistral (#8657) 
* fix `llama_chat_format_single` for mistral

* fix typo

* use printf
 2024-07-24 13:48:46 +02:00
Joe Todd 79167d9e49 Re-add erroneously removed -fsycl from GGML_EXTRA_LIBS (#8667) 2024-07-24 11:55:26 +01:00