Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-19 17:25:07 +02:00
Code Issues Packages Projects Releases Wiki Activity
6,097 Commits 3 Branches 254 Tags
d595a80abc8caabdfd427dfa77f94d8ea3ec921f
Commit Graph

6097 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
luoyu-intel d08c20edde [SYCL] Fix the sub group size of Intel (#8106) 
* use warp_size macro for all sycl kernels

* fix mask of permute_sub_group_by_xor

* fix rms_norm with correct warp number

* fix rms_norm_f32/group_norm_f32

* move norm to norm.cpp file

* fix quantize bug

* fix mmvq's batch size
 2024-07-02 10:16:00 +08:00
Xuan Son Nguyen 5fac350b9c Fix gemma2 tokenizer convert (#8244) 
* fix gemma2 tokenizer convert

* remove scores

* improve code, fix new line issue
 2024-07-02 01:07:23 +02:00
Johannes Gäßler cb5fad4c6c CUDA: refactor and optimize IQ MMVQ (#8215) 
* CUDA: refactor and optimize IQ MMVQ

* uint -> uint32_t

* __dp4a -> ggml_cuda_dp4a

* remove MIN_CC_DP4A checks

* change default

* try CI fix
 2024-07-01 20:39:06 +02:00
Mateusz Charytoniuk dae57a1ebc readme: add Paddler to the list of projects (#8239) 2024-07-01 20:13:22 +03:00
Xuan Son Nguyen 49122a873f gemma2: add sliding window mask (#8227) 
* gemma2: add sliding window mask

* fix data_swa uninitialized

* better naming

* add co-author

Co-authored-by: Arlo Phoenix <arlo-phoenix@users.noreply.github.com>

* replace list with single tensor

* update

* llama : minor styling

* convert : add sanity check for query_pre_attn_scalar

* fix small typo in README

---------

Co-authored-by: Arlo Phoenix <arlo-phoenix@users.noreply.github.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-07-01 18:48:34 +02:00
Roni 0ddeff1023 readme : update tool list (#8209) 
* Added gppm to Tool list in README

* Update README.md

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-07-01 15:48:16 +03:00
Michael Francis 3840b6f593 nix : enable curl (#8043) 
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-07-01 14:47:04 +03:00
Georgi Gerganov 257f8e41e2 nix : remove OpenCL remnants (#8235) 
* nix : remove OpenCL remnants

* minor : remove parentheses
 2024-07-01 14:46:18 +03:00
iacore 694c59cb42 Document BERT support. (#8205) 
* Update README.md

document BERT support

* Update README.md
 2024-07-01 13:40:58 +02:00
zhentaoyu 197fe6c1d7 [SYCL] Update SYCL-Rope op and Refactor (#8157) 
* align with rope.cu and move sycl-op to a single file
 2024-07-01 19:39:06 +08:00
Georgi Gerganov d0a7145ba9 flake.lock: Update (#8218) 2024-06-30 16:09:34 -07:00
Xuan Son Nguyen 9ef0780062 Fix new line issue with chat template, disable template when in-prefix/suffix is set (#8203) 
* preserve new line llama_chat_format_single

* disable chat template if in-prefix/suffix is set

* remove redundant change
 2024-06-30 20:27:13 +02:00
bebopkim 7499a6bd4b Resolve make: *** No rule to make target ggml-metal.m', needed by ggml-metal.o'. Stop error on macOS Metal (#957) v1.69 2024-06-30 16:05:03 +08:00
Concedo b1f9c97a0b Merge branch 'upstream' into concedo_experimental 2024-06-30 11:50:40 +08:00
Concedo 8a07ce306a edit sampler order warning 2024-06-30 11:46:09 +08:00
Andrei 1c5eba6f8e llama: Add attention and final logit soft-capping, update scaling factor to Gemma2 (#8197) 
* Add attention and final logit softcapping.

* fix

* Add custom add_ functions

* Disable flash attention for Gemma2

* Update src/llama.cpp

Co-authored-by: slaren <slarengh@gmail.com>

* Add default value for attention and final logit softcap value

* Add custom kq scaling from Gemma2Attention

* Remove custom pre attention scaling and use computed value instead.

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-06-29 23:44:08 -04:00
Concedo 18df56b8cf add tensor split gui input for vulkan 2024-06-30 11:16:48 +08:00
Concedo 02f92f6ecc Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/full-cuda.Dockerfile
#	.devops/full-rocm.Dockerfile
#	.devops/llama-cli-cuda.Dockerfile
#	.devops/llama-cli-rocm.Dockerfile
#	.devops/llama-cli-vulkan.Dockerfile
#	.devops/llama-cpp-cuda.srpm.spec
#	.devops/llama-server-cuda.Dockerfile
#	.devops/llama-server-rocm.Dockerfile
#	.devops/llama-server-vulkan.Dockerfile
#	.github/workflows/build.yml
#	.github/workflows/docker.yml
#	CMakeLists.txt
#	Makefile
#	README.md
#	examples/llama.android/llama/src/main/cpp/CMakeLists.txt
#	flake.lock
#	ggml/CMakeLists.txt
#	ggml/src/CMakeLists.txt
#	grammars/README.md
#	scripts/sync-ggml-am.sh
#	scripts/sync-ggml.last
#	tests/test-chat-template.cpp
#	tests/test-grammar-integration.cpp
#	tests/test-json-schema-to-grammar.cpp
 2024-06-30 10:59:42 +08:00
henk717 8421243c6d Chat Adapters (#956) 
* Give the CI builds a recognizable AVX1 name

* Chat Adapters
 2024-06-30 10:28:43 +08:00
Concedo d120c55e12 try to fix build errors (+1 squashed commits) 
Squashed commits:

[27c28292] try fix build errors
 2024-06-29 23:11:00 +08:00
Concedo 9c10486204 merge the file structure refactor, testing 2024-06-29 12:14:38 +08:00
Nexesenex a5a32b9179 Add Cuda Graphs in CMakeList (#947) 
original LCPP PR 3242 by Slaren
 2024-06-29 10:10:51 +08:00
Xuan Son Nguyen 72272b83a3 fix code typo in llama-cli (#8198) 2024-06-29 00:14:20 +02:00
Olivier Chafik 8748d8ac6f json: attempt to skip slow tests when running under emulator (#8189) 2024-06-28 18:02:05 +01:00
Concedo 2857fb52ba remove contribute template 2024-06-28 23:47:03 +08:00
Nexesenex 1203a56f6b Add Cublas12 dlls to .gitignore (#946) 2024-06-28 23:41:44 +08:00
Xuan Son Nguyen 26a39bbd6b Add MiniCPM, Deepseek V2 chat template + clean up llama_chat_apply_template_internal (#8172) 
* tmp_contains

* minicpm chat template

* add DeepSeek Lite template

* change deepseek-lite to deepseek2

* correct code comment

* correct code from master branch
 2024-06-28 15:11:44 +02:00
Sigbjørn Skjæret 38373cfbab Add SPM infill support (#8016) 
* add --spm-infill option

* support --spm-infill

* support --spm-infill
 2024-06-28 12:53:43 +02:00
slaren b851b3fba0 cmake : allow user to override default options (#8178) 2024-06-28 12:37:45 +02:00
Concedo 5e671c2162 ignore utf decode errors 2024-06-28 16:39:48 +08:00
Olivier Chafik 139cc621e9 json: restore default additionalProperties to false, fix some pattern escapes (#8180) 
* json: expand ESCAPED_IN_REGEXPS_BUT_NOT_IN_LITERALS charset

* json: revert default of additionalProperties to false

* Update README.md
 2024-06-28 09:26:45 +01:00
Concedo 3e6a9673ef add runpod to readme 2024-06-28 14:47:11 +08:00
pculliton e57dc62057 llama: Add support for Gemma2ForCausalLM (#8156) 
* Inference support for Gemma 2 model family

* Update convert-hf-to-gguf.py, constants, and tensor mappings

* cleanup

* format fix

* Fix special token vocab bug

* Don't add space prefix

* fix deleted lines

* Update src/llama.cpp

Co-authored-by: slaren <slarengh@gmail.com>

* Add model type names

* Add control vector

* Fix model type identification

---------

Co-authored-by: Andrei Betlen <abetlen@gmail.com>
Co-authored-by: slaren <slarengh@gmail.com>
 2024-06-27 21:00:43 -07:00
Xuan Son Nguyen a27aa50ab7 Add missing items in makefile (#8177) 2024-06-28 02:19:11 +02:00
Olivier Chafik cb0b06a8a6 json: update grammars/README w/ examples & note about additionalProperties (#8132) 
* json: update grammars/README

* mention broken prefixItems

* add mention to llama-gbnf-validator

* json: explicit type: object for nested items object in cli example
 2024-06-27 22:08:42 +01:00
loonerin 558f44bf83 CI: fix release build (Ubuntu+Mac) (#8170) 
* CI: fix release build (Ubuntu)

PR #8006 changes defaults to build shared libs. However, CI for releases
expects static builds.

* CI: fix release build (Mac)

---------

Co-authored-by: loonerin <loonerin@users.noreply.github.com>
 2024-06-27 21:01:23 +02:00
slaren 8172ee9da9 cmake : fix deprecated option names not working (#8171) 
* cmake : fix deprecated option names not working

* remove LlAMA_OPENMP
 2024-06-27 20:04:39 +02:00
Xuan Son Nguyen 16791b8f0b Add chatml fallback for cpp llama_chat_apply_template (#8160) 
* add chatml fallback for cpp `llama_chat_apply_template`

* remove redundant code
 2024-06-27 18:14:19 +02:00
Georgi Gerganov ab3679112d flake.lock: Update (#8071) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/e9ee548d90ff586a6471b4ae80ae9cfcbceb3420?narHash=sha256-4Zu0RYRcAY/VWuu6awwq4opuiD//ahpc2aFHg2CWqFY%3D' (2024-06-13)
  → 'github:NixOS/nixpkgs/d603719ec6e294f034936c0d0dc06f689d91b6c3?narHash=sha256-k3JqJrkdoYwE3fHE6xGDY676AYmyh4U2Zw%2B0Bwe5DLU%3D' (2024-06-20)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Philip Taron <philip.taron@gmail.com>
 2024-06-27 08:37:29 -07:00
jukofyork 97877eb10b Control vector loading fixes (#8137) 
* Fixed leak in llama_control_vector_load_one() and allow llama_control_vector_load() to grow

* refactored `llama_control_vector_load_one()`

* allow multiple directions for same layer in same file

* llama_control_vector_load_one() and llama_control_vector_load() now break on error

* removed unnecessary ggml_free() call
 2024-06-27 16:48:07 +02:00
Raj Hammeer Singh Hada 387952651a Delete examples/llama.android/llama/CMakeLists.txt (#8165) 
* Delete examples/llama.android/llama/CMakeLists.txt

https://github.com/ggerganov/llama.cpp/pull/8145#issuecomment-2194534244

This file is not being used for building on Android. `llama.cpp/examples/llama.android/llama/src/main/cpp/CMakeLists.txt` is being used instead.

* Update CMakeLists.txt

Pick local llama.cpp files instead of fetching content from git
 2024-06-27 16:39:29 +02:00
Sigbjørn Skjæret 6030c61281 Add Qwen2MoE 57B-A14B model identifier (#8158) 
* Add Qwen2MoE 57B-A14B

* Add Qwen2MoE 57B-A14B
 2024-06-27 16:27:41 +02:00
Johannes Gäßler 85a267daaa CUDA: fix MMQ stream-k for --split-mode row (#8167) 2024-06-27 16:26:05 +02:00
Concedo 176923f58b test wip cmake 2024-06-27 21:53:44 +08:00
Concedo 1801594972 allow forced positive prompt 2024-06-27 20:21:17 +08:00
kustaaya f675b20a3b Added support for Viking pre-tokenizer (#8135) 
Co-authored-by: kustaaya <kustaaya@protonmail.com>
 2024-06-27 10:58:54 +02:00
Concedo e433afb261 updated lite 2024-06-27 15:51:34 +08:00
Sigbjørn Skjæret 911e35bb8b llama : fix CodeLlama FIM token checks (#8144) 
* account for space prefix character

* use find instead
 2024-06-27 10:46:41 +03:00
Concedo 4f369b0a0a update colab 2024-06-27 15:41:06 +08:00
Concedo 11f0643fa4 fix pyinstallers 2024-06-27 15:19:44 +08:00