Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-03 03:21:24 +02:00
Code Issues Packages Projects Releases Wiki Activity
10,692 Commits 3 Branches 253 Tags
fd0d0cab03375df419aac94de70bf96bd93e0614
Commit Graph

10692 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Concedo 3ba348c4ab Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	README.md
 2024-09-12 11:18:13 +08:00
Concedo 5b658ab6d4 updated lite 2024-09-12 10:47:47 +08:00
Xinpeng Dou df4b7945ae cann: Fix error when running a non-exist op (#9424) 2024-09-12 09:02:35 +08:00
Faisal Zaghloul 449ccfb6f5 Add Jais to list of supported models (#9439) 
Co-authored-by: fmz <quic_fzaghlou@quic.com>
 2024-09-12 02:29:53 +02:00
slaren 1b28061400 llama : skip token bounds check when evaluating embeddings (#9437) 2024-09-11 17:52:13 +02:00
Pavel Zloi 8db003a19d py : support converting local models (#7547) 
* Support of converting local models added to convert-hf-to-gguf-update.py

* Description fixed

* shutil added to imports
 2024-09-11 15:29:51 +03:00
Concedo 13394368b6 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	examples/embedding/embedding.cpp
#	examples/infill/infill.cpp
#	examples/perplexity/perplexity.cpp
#	flake.lock
#	src/llama-sampling.cpp
 2024-09-11 20:27:53 +08:00
Concedo 7bdac9bc44 prevent shifting on rwkv 2024-09-11 20:22:45 +08:00
Xuan Son Nguyen 0996c5597f llava : correct args for minicpmv-cli (#9429) 2024-09-11 12:59:13 +02:00
Xuan Son Nguyen 5bb2c5dbd2 files : remove accidentally added lora_test submodule (#9430) 2024-09-11 13:02:09 +03:00
Farbod Bijary 67155ab7f5 feat: Implements retrying logic for downloading models using --model-url flag (#9255) 
* feat: Implements retrying logic for downloading models using --model-url flag

* Update common/common.cpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* Update common/common.cpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* apply comments

* implements a retry function to avoid duplication

* fix editorconfig

* change function name

---------

Co-authored-by: farbod <farbod.bjary82@gmail.com>
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
Co-authored-by: slaren <slarengh@gmail.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2024-09-11 11:22:37 +02:00
Johannes Gäßler 5af118efda CUDA: fix --split-mode row race condition (#9413) 2024-09-11 10:22:40 +02:00
Georgi Gerganov d2b496bff4 batched-bench : remove unused code (#9305) 2024-09-11 10:03:54 +03:00
R0CKSTAR b34e023480 musa: remove Clang builtins mapping (#9421) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2024-09-11 03:46:55 +02:00
Alberto Cabrera Pérez 51b6038636 sycl : update support conditions (#9394) 
* sycl : update support condition to im2col

Signed-off-by: Alberto Cabrera <alberto.cabrera@codeplay.com>

* Added TODO to remind supporting FP32 im2col

---------

Signed-off-by: Alberto Cabrera <alberto.cabrera@codeplay.com>
 2024-09-11 08:53:42 +08:00
Georgi Gerganov cb9c933eb2 flake.lock: Update (#9360) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/af510d4a62d071ea13925ce41c95e3dec816c01d?narHash=sha256-ODYRm8zHfLTH3soTFWE452ydPYz2iTvr9T8ftDMUQ3E%3D' (2024-08-30)
  → 'github:hercules-ci/flake-parts/567b938d64d4b4112ee253b9274472dc3a346eb6?narHash=sha256-%2Bebgonl3NbiKD2UD0x4BszCZQ6sTfL4xioaM49o5B3Y%3D' (2024-09-01)
• Updated input 'flake-parts/nixpkgs-lib':
    'https://github.com/NixOS/nixpkgs/archive/a5d394176e64ab29c852d03346c1fc9b0b7d33eb.tar.gz?narHash=sha256-uFf2QeW7eAHlYXuDktm9c25OxOyCoUOQmh5SZ9amE5Q%3D' (2024-08-01)
  → 'https://github.com/NixOS/nixpkgs/archive/356624c12086a18f2ea2825fed34523d60ccc4e3.tar.gz?narHash=sha256-Ss8QWLXdr2JCBPcYChJhz4xJm%2Bh/xjl4G0c0XlP6a74%3D' (2024-09-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/71e91c409d1e654808b2621f28a327acfdad8dc2?narHash=sha256-GnR7/ibgIH1vhoy8cYdmXE6iyZqKqFxQSVkFgosBh6w%3D' (2024-08-28)
  → 'github:NixOS/nixpkgs/574d1eac1c200690e27b8eb4e24887f8df7ac27c?narHash=sha256-v3rIhsJBOMLR8e/RNWxr828tB%2BWywYIoajrZKFM%2B0Gg%3D' (2024-09-06)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-09-10 15:46:59 -07:00
Xuan Son Nguyen 6cd4e03444 arg : bring back missing ifdef (#9411) 
* arg : bring back missing ifdef

* replace with llama_supports_gpu_offload
 2024-09-10 22:41:29 +02:00
matteo 8d300bd35f enable --special arg for llama-server (#9419) 
Co-authored-by: matteo serva <matteo.serva@gmail.com>
 2024-09-10 22:40:59 +02:00
slaren 49006c67b4 llama : move random seed generation to the samplers (#9398) 
* llama_sampler_penalties : clamp penalty_last_n to zero
 2024-09-10 18:04:25 +02:00
Concedo a947558e0e Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	Makefile
#	README.md
#	common/CMakeLists.txt
#	common/common.cpp
#	common/common.h
#	examples/embedding/embedding.cpp
#	examples/imatrix/imatrix.cpp
#	examples/infill/infill.cpp
#	examples/parallel/parallel.cpp
#	examples/perplexity/perplexity.cpp
#	examples/rpc/README.md
#	examples/save-load-state/save-load-state.cpp
#	examples/server/README.md
#	examples/speculative/speculative.cpp
#	tests/test-sampling.cpp
 2024-09-10 16:39:23 +08:00
Concedo eee67281be move kcpp params out 2024-09-10 16:30:12 +08:00
Georgi Gerganov 00ba2ff781 metal : fix compile warning with GGML_METAL_NDEBUG (#0) 2024-09-10 10:17:43 +03:00
Daniel Bevenius 83008b7cfe llama : update llm_build_copy_mask_state comment [no ci] (#9385) 
This commit updates the comment, which seems to contain a typo or be an
outdated comment, in the copy_mask_state function changing the variable
n_rs to n_kv.

I believe this change is correct and what the comment wants to
convey is to copy the states that are not going to be used in the
upcoming processing, which are the tokens states from n_seqs up to
the number of possible token states n_kv.
 2024-09-10 10:03:21 +03:00
Molly Sophia 0b4ac75772 RWKV v6: Add time_mix_decay_w1/w2 in quant exclusion list (#9387) 
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
 2024-09-10 10:02:30 +03:00
slaren fb3f249815 make : do not run llama-gen-docs when building (#9399) 2024-09-10 09:23:33 +03:00
Xuan Son Nguyen bfe76d4a17 common : move arg parser code to arg.cpp (#9388) 
* common : move arg parser to arg.cpp

* better categorize args

* add cmake

* missing climits

* missing cstdarg

* common : more explicit includes

* fix build

* refactor gpt_params_parse

* update server readme

* fix test

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-09-09 23:36:09 +02:00
Radoslav Gerganov 293bebe077 rpc : fix segfault with nkvo (#9389) 
* rpc : fix nkvo

* rpc : buf_size must not be static

ref: #9337

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-09-09 18:40:10 +03:00
Prashant Vithule 5fac4d5764 ggml : vector length agnostic SVE support (#9290) 
* Implemented vector length agnostic SVE using switch case for 512-bit, 256-bit, 128-bit vector lengths

* Implemented vector length agnostic SVE using switch case for 512-bit, 256-bit, 128-bit vector lengths

* Removed WhiteSpaces

* ggml : style changes + fix 512-bit nb loop check

- fix local scope in switch cases
- consistent predicate names
- empty lines when necessary
- opening braces, spaces
- const-correctness
- add asserts

* Update ggml/src/ggml-quants.c

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-09-09 18:37:18 +03:00
slaren 5fb5e24811 llama : minor sampling refactor (2) (#9386) 2024-09-09 17:10:46 +02:00
Georgi Gerganov 38ca6f644b readme : update hot topics 2024-09-09 15:51:37 +03:00
Concedo fc7fe2e7a0 allow rwkv6 to run although its broken 2024-09-09 20:50:58 +08:00
Johannes Gäßler 8e6e2fbe14 CUDA: fix variable name conflict for Windows build (#9382) 2024-09-09 14:22:53 +02:00
Antonis Makropoulos 5ed087573e readme : add LLMUnity to UI projects (#9381) 
* add LLMUnity to UI projects

* add newline to examples/rpc/README.md to fix editorconfig-checker unit test
 2024-09-09 14:21:38 +03:00
Concedo b63158005f All samplers moved to kcpp side 2024-09-09 18:14:11 +08:00
Concedo 12fd16bfd4 Merge commit 'df270ef74596da8f1178f08991f4c51f18c9ee82' into concedo_experimental 
# Conflicts:
#	Makefile
#	common/CMakeLists.txt
#	common/common.h
#	common/sampling.cpp
#	common/sampling.h
#	examples/infill/infill.cpp
#	examples/llama-bench/llama-bench.cpp
#	examples/quantize-stats/quantize-stats.cpp
#	examples/server/server.cpp
#	include/llama.h
#	src/llama-sampling.cpp
#	src/llama-sampling.h
#	src/llama.cpp
#	tests/test-grammar-integration.cpp
#	tests/test-grammar-parser.cpp
#	tests/test-json-schema-to-grammar.cpp
#	tests/test-llama-grammar.cpp
#	tests/test-sampling.cpp
 2024-09-09 17:10:08 +08:00
Radoslav Gerganov 54f376d0b9 rpc : update README [no ci] (#9320) 
Update README with instructions how to offload model layers to both
local and remote devices
 2024-09-09 11:04:39 +03:00
Dan Johansson b2e89a3274 Arm AArch64: Documentation updates (#9321) 
* Arm AArch64: Documentation updates

* Update docs/build.md to include information on how to enable the Arm optimized gemm/gemv kernels

* Update examples/quantize/README.md with information on the Q4_0_4_4, Q4_0_4_8 and Q4_0_8_8 formats

* Add newline to the end of docs/build.md
 2024-09-09 10:02:45 +03:00
Concedo 70cdb55cc9 Merge commit '947538acb8617756a092042ff7e58db18dde05ec' into concedo_experimental 
# Conflicts:
#	.github/workflows/build.yml
#	.github/workflows/docker.yml
#	CMakePresets.json
#	examples/llama-bench/llama-bench.cpp
#	ggml/CMakeLists.txt
#	ggml/src/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tests/test-quantize-fns.cpp
 2024-09-09 11:26:34 +08:00
Concedo 27bbdf7d2a added link for novita AI, added legacy warning for old GGML models 2024-09-09 11:19:32 +08:00
Markus Tavenrath daa9623ab0 Overlap cmdbuffer creation and cmdbuffer execution in Vulkan backend by submitting smaller cmdbuffers early. (#9118) 
* Overlap cmdbuffer creation and cmdbuffer execution in Vulkan backend by submitting smaller cmdbuffers early.

* fix compile issues

* Fix issues where the last submit wasn't executed or handled properly.

* remove trailing whitespace

* Repair GGML_VULKAN_CHECK_RESULTS

* Increase submit counter only if actual work has been submitted and increase submit count to 100.

* Fix some nodes are not checked with GGML_VULKAN_CHECK_RESULTS enabled.
 2024-09-08 21:43:48 +02:00
Georgi Gerganov e079bffb66 cuda : fix FA Q src index (1 -> 0) (#9374) 2024-09-08 22:01:02 +03:00
Xuan Son Nguyen 3f7ccfd649 common : bring back missing args, add env var duplication check (#9375) 
* common : bring back missing args

* move duplication check to test-arg-parser

* add check for duplicated env var

* correct default values
 2024-09-08 18:08:55 +02:00
slaren a249843d89 common : restore --n-gpu-layers (#9371) 2024-09-08 16:44:42 +02:00
slaren 19f4a7b296 llama : refactor samplers internal implementation (#9370) 2024-09-08 15:52:07 +02:00
Neo Zhang Jianyu 2a358fb0c4 [SYCL] add check malloc result on device (#9346) 
* add check malloc result on device

* update for review comments, check all malloc_device() result

---------

Co-authored-by: arthw <14088817+arthw@users.noreply.github.com>
 2024-09-08 19:05:29 +08:00
slaren eae597182c llama : sanitize tokens in the upper bound (#9359) 2024-09-08 12:41:51 +02:00
Concedo c78690737c fix for DRY segfault on unicode character substring tokenization 2024-09-08 18:25:00 +08:00
Xuan Son Nguyen 00b02bb249 imatrix : fix arg parser for imatrix (#9366) 
* imatrix : fix arg parser

* beautify printing first arg
 2024-09-08 12:12:17 +02:00
Georgi Gerganov a876861455 metal : update support condition for im2col + fix warning (#0) 2024-09-08 11:05:55 +03:00
Georgi Gerganov 385decbd63 sync : ggml 2024-09-08 11:05:55 +03:00