Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-18 16:55:14 +02:00
Code Issues Packages Projects Releases Wiki Activity
6,386 Commits 3 Branches 254 Tags
4c4ce5e808546ec9ace14dd3a75b4b88e25ffe9d
Commit Graph

6386 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Concedo 4c4ce5e808 rewritten checkpoint 1 - before coopmat 2024-12-13 16:55:23 +08:00
Concedo 4548d893ee better way to handle termux compatibility (+2 squashed commit) 
Squashed commit:

[301986f11] better way to handle termux compatibility

[16b03b225] updated lite
 2024-12-11 15:05:01 +08:00
DontEatOreo f6aebb1449 Expand nix section (#1255) 
* examples/nix_example.md: fix markdownlint complaints

* examples/nix_example.md: add home manager section

* examples/nix_example.md: add hardware support section

* examples/nix_example.md: rewrite CUDA section

---------

Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
 2024-12-10 22:58:48 +08:00
Concedo 00a686fc72 fixed fast forwarding context corruption after abort during prompt processing 2024-12-10 22:37:40 +08:00
Concedo a11bba5893 cleanup, fix native build for arm (+28 squashed commit) 
Squashed commit:

[d1f6a4154] bundle library

[947ab84b7] undo

[0f9aba8d8] test

[e9ac93873] test

[920438202] test

[1c6d98804] Revert "quick test"

This reverts commit acf8ec8940.

[acf8ec894] quick test

[6a9937233] undo

[5a263a5bd] test

[ddfd82bca] test

[0b30e45da] test

[c3bfece55] messed up

[2a4b37fe0] Revert "test"

This reverts commit 80a1fcaeaf.

[80a1fcaea] test

[e2aa7d944] test

[264d80200] test

[f5b123173] undo

[1ffacc484] test

[63c0be926] undo

[510e0377e] ofast try fix

[4ac199b20] try fix sigill

[1bc987ba2] try fix illegal instruction

[7697252b1] edit

[f87087b28] check gcc ver

[e9dfe2cef] try using qemu to do the pyinstaller

[b411192db] revert

[25b5301e5] try using qemu to do the pyinstaller

[58038cddc] try using qemu to do the pyinstaller
 2024-12-10 19:42:23 +08:00
Robert Ormandi 86a1934978 metal : Extend how Llama.cpp locates metal resources (#10676) 
* metal : Extend how Llama.cpp locates metal resources (#10675)

  * It searches the resource file in the directory where the current
    binary is located as well.
  * Resolves symbolic links.

Rationale:

When we plug this dependency into a Bazel build and run it in the
context of Bazel (e.g. testing):

  * the execution directory is often very different from where the files
    are located and no direct control over this (Bazel sandboxing),
  * the Bazel sandbox often use symbolic links to make files available.

With this patch, we can have the resource file added to the target,
can build and run tests in the context of Bazel.

* Update ggml/src/ggml-metal/ggml-metal.m

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* Update ggml/src/ggml-metal/ggml-metal.m

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-12-07 09:55:01 +02:00
Sukriti Sharma 784a14aa49 convert : add support for Roberta embeddings (#10695) 2024-12-07 09:02:14 +02:00
Georgi Gerganov c5ede3849f convert : add custom attention mapping 2024-12-06 21:33:49 +02:00
Xuan Son Nguyen f162d45a21 common : bring back --no-warmup to server (#10686) 2024-12-06 13:29:05 +01:00
Xuan Son Nguyen 6c5bc0625f server : (refactoring) do not rely on JSON internally (#10643) 
* server : (refactoring) reduce usage of json internally

* move all response types to struct

* wip [no ci]

* many fixes

* add virtual function

* fix index

* minor style fix

* add std::move

* refactor handle_completions_generic

* add virtual functions

* remove server.hpp

* clarify server_sent_event RFC specs

* apply review comments

* fix model_alias and completion_probabilities

* small clean up

* remove virtual for to_json_oai_compat()

* naming oai_compat --> oaicompat

* fix unwanted recursive call

* update docs
 2024-12-06 11:14:32 +01:00
Concedo e9d2332dd8 improved tool calls and whisper 2024-12-06 14:34:31 +08:00
Plamen Minev 7736837d62 fix(server) : not show alert when DONE is received (#10674) 2024-12-05 22:36:41 +01:00
Jeff Bolz c9c6e01dae vulkan: Add VK_NV_cooperative_matrix2 support for mul_mat and flash attention (#10206) 2024-12-05 20:15:05 +01:00
Riccardo Orlando 6fe6247831 llama : add Minerva 7B model support (#10673) 
* Support for Minerva 7B

* Update convert_hf_to_gguf_update.py
 2024-12-05 20:30:59 +02:00
Concedo 836c06d91a minor edit 2024-12-06 00:37:38 +08:00
Concedo 746cb01843 remove test since it wont work on x64 2024-12-06 00:26:58 +08:00
Concedo 65a11451e3 fix missing bundled files 2024-12-06 00:21:08 +08:00
Concedo fe72c8db9f CI for ARM should appear as ARM 2024-12-06 00:12:30 +08:00
Concedo 5cddd0a878 Merge branch 'concedo' into concedo_experimental 2024-12-05 23:58:31 +08:00
Concedo ece96e19bf clean up makefile 2024-12-05 23:58:23 +08:00
Concedo 8d5bb06aeb test aarch64 ci workflow 2024-12-05 23:57:25 +08:00
Georgi Gerganov 0cd182ebcc sync : ggml 2024-12-05 13:27:42 +02:00
PAB a8cbab201d ggml: add GGML_SET Metal kernel + i32 CPU kernel (ggml/1037) 
* implemented cpu kernel

* add i32 test cases in test-backend-ops

* typedef `ggml_metal_kargs_set`

* implemented `kernel_set`

* memcpy
 2024-12-05 13:27:33 +02:00
PAB c2082d93a8 ggml : add GGML_PAD_REFLECT_1D operation (ggml/1034) 
* ggml_pad_reflect_1d defined in header

* implemented on CPU

* called the forward pass

* impl Metal kernel

* added Metal kernel

* added OP_PAD_REFLECT_1D in test-backend-ops.cpp

* add test-pad-reflect-1d test case

* test case support multiple backend
 2024-12-05 13:27:31 +02:00
Concedo d0d1d922de handle and fix temp paths to chat completions adapter 2024-12-05 17:22:35 +08:00
Concedo 5106816eac drafted tokens debug prints 2024-12-05 17:05:20 +08:00
Concedo 2787fca6b4 refactored library selection, fixed ollama params 2024-12-05 16:47:52 +08:00
Daniel Bevenius d405804be8 py : update outdated copy-paste instructions [no ci] (#10667) 
This commit updates the copy-paste instruction in
convert_hf_to_gguf_update.py to reflect that convert_hf_to_gguf.py
will have already been updated with the new get_vocab_base_pre()
function when this script completes.
 2024-12-05 09:47:55 +02:00
aryantandon01 f112d198cd Update deprecation-warning.cpp (#10619) 
Fixed Path Separator Handling for Cross-Platform Support (Windows File Systems)
 2024-12-04 23:19:20 +01:00
Georgi Gerganov 1da7b76569 server : fix speculative decoding with context shift (#10641) 
* server : fix speculative decoding with context shift

ggml-ci

* server : take into account speculative limits

ggml-ci

* server : add tests
 2024-12-04 22:38:20 +02:00
Diego Devesa 59f4db1088 ggml : add predefined list of CPU backend variants to build (#10626) 
* ggml : add predefined list of CPU backend variants to build

* update CPU dockerfiles
 2024-12-04 14:45:40 +01:00
Diego Devesa 2803540814 ggml-cpu : fix HWCAP2_I8MM value (#10646) 2024-12-04 14:40:44 +01:00
ltoniazzi 253b7fde91 Fix HF repo commit to clone lora test models (#10649) 2024-12-04 10:45:48 +01:00
JFLFY2255 8d0cfd554a llama: Support MiniCPM-1B (with & w/o longrope) (#10559) 2024-12-04 11:42:50 +02:00
Jeff Bolz 2759916d86 vulkan: Implement "fast divide" (mul+shift) for unary ops like copy (#10642) 2024-12-04 08:28:59 +01:00
Nicolò Scipione 40c6d79fb5 SYCL : Move to compile time oneMKL interface backend selection for NVIDIA backend (#10584) 
* [SYCL] Move to Compile Time backend selection on oneMKL Interface for NVIDIA backend

Move to compile time selection to backend to avoid latency at run time.
Add it to all mkl gemm calls and only for NVIDIA backend.

Signed-off-by: nscipione <nicolo.scipione@codeplay.com>

* Formatting

* Address PR comments to increase readibility

---------

Signed-off-by: nscipione <nicolo.scipione@codeplay.com>
 2024-12-04 09:29:20 +08:00
Wang Ran (汪然) 98036d5670 fix typo of README.md (#10605) 2024-12-04 02:22:50 +01:00
Frankie Robertson cd2f37b304 Avoid using __fp16 on ARM with old nvcc (#10616) 2024-12-04 01:41:37 +01:00
Benson Wong da6aac91f1 Add docs for creating a static build (#10268) (#10630) 
* Add notes for a static build

* Update docs/build.md

---------

Co-authored-by: Diego Devesa <slarengh@gmail.com>
 2024-12-04 01:40:36 +01:00
piDack 01e6d9bb71 clip : add sycl support (#10574) 
Co-authored-by: piDack <pcdack@hotmail.co>
 2024-12-04 01:26:37 +01:00
Jeff Bolz cc98896db8 vulkan: optimize and reenable split_k (#10637) 
Use vector loads when possible in mul_mat_split_k_reduce. Use split_k
when there aren't enough workgroups to fill the shaders.
 2024-12-03 20:29:54 +01:00
Xuan Son Nguyen 91c36c269b server : (web ui) Various improvements, now use vite as bundler (#10599) 
* hide buttons in dropdown menu

* use npm as deps manager and vite as bundler

* fix build

* fix build (2)

* fix responsive on mobile

* fix more problems on mobile

* sync build

* (test) add CI step for verifying build

* fix ci

* force rebuild .hpp files

* cmake: clean up generated files pre build
 2024-12-03 19:38:44 +01:00
Georgi Gerganov 1cd3df46bd scripts : remove amx sync 
ggml-ci
 2024-12-03 20:04:49 +02:00
Georgi Gerganov c505471857 sync : ggml 2024-12-03 20:04:49 +02:00
mahorozte e9e661bd59 CUDA: remove unnecessary warp reduce in FA (ggml/1032) 
* kqmax_new_j in every thread within warp is same after operate at line 199,this reduce can be omit

* same problem in vec32

---------

Co-authored-by: ZhaoXiaoYu <zhao.xiaoyu@zte.com.cn>
 2024-12-03 20:04:49 +02:00
PAB efb6ae9630 feat: add GGML_UNARY_OP_ARGMAX Metal kernel (ggml/1019) 
* implemented argmax kernel

* tpig -> tgpig

* change to strides

* contiguous assertions

* kernel working and tested

* argmax simd parallel implementation

* added 2 new tests for argmax in test-backend-ops

* cosmit

* added 3 tests cases for perf eval

* add test_argmax in make_test_cases_perf

* Update test-backend-ops.cpp

Co-authored-by: Diego Devesa <slarengh@gmail.com>

---------

Co-authored-by: Diego Devesa <slarengh@gmail.com>
 2024-12-03 20:04:49 +02:00
PAB 667d70d170 metal : add GGML_OP_CONV_TRANSPOSE_1D kernels (ggml/1026) 
* wip

* wip implementation f32

* kernel conv transpose 1d f32 working

* initial commit
 2024-12-03 20:04:49 +02:00
Concedo 52cc908f7f default trim_stop to true, which trims any tokens after a stop sequence and the stop sequence itself. This is potentially a breaking change. 2024-12-03 22:44:10 +08:00
Concedo 7d11d2946c only show warning if more than 1 moved tensor 2024-12-03 22:09:26 +08:00
Xuan Son Nguyen 3b4f2e33e2 llama : add missing LLAMA_API for llama_chat_builtin_templates (#10636) 2024-12-03 12:54:30 +01:00