Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-05 12:31:15 +02:00
Code Issues Packages Projects Releases Wiki Activity
1,647 Commits 3 Branches 253 Tags
9589d52079e0fcef9fa177aaefc4e2430a652c8b
Commit Graph

1647 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Concedo 9589d52079 added help link 2023-07-29 17:33:15 +08:00
Concedo e4b42e5b15 fixed gui bugs 2023-07-29 11:15:57 +08:00
Concedo b40550cf1a change wiki link 2023-07-28 13:01:12 +08:00
Concedo 31486ebc8d updated readme 2023-07-28 11:32:55 +08:00
Concedo 94e0a06daf updated lite, up ver (+1 squashed commits) 
Squashed commits:

[7d6520f] updated lite, up ver
v1.37.1a 		2023-07-26 11:03:17 +08:00
Concedo b184380aae Revert "a better default rms_norm_eps" 
This reverts commit 0c26799e77.
 2023-07-26 10:23:45 +08:00
Concedo f53d2aabb4 Merge branch 'master' into concedo_experimental 2023-07-26 10:19:59 +08:00
Kawrakow eb542d3932 Add LLAMA_DEFAULT_RMS_EPS so we can change the default (#2384) 
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 2023-07-25 18:35:53 +03:00
Concedo 6a054b80b0 Merge branch 'master' into concedo_experimental 
# Conflicts:
#	.github/workflows/build.yml
#	scripts/build-info.sh
 2023-07-25 22:55:55 +08:00
Concedo 0c26799e77 a better default rms_norm_eps 2023-07-25 22:51:01 +08:00
slaren 07aaa0f63f ggml : fix ggml_flash_attn to use op_params (#2387) 
* ggml : fix ggml_flash_attn to use op_params
 2023-07-25 16:20:12 +02:00
ldwang fce48caf9a convert.py : support bpe tokenizer (#2228) 
* support bpe tokenizer in convert

Signed-off-by: ldwang <ftgreat@gmail.com>

* support bpe tokenizer in convert

Signed-off-by: ldwang <ftgreat@gmail.com>

* support bpe tokenizer in convert, fix

Signed-off-by: ldwang <ftgreat@gmail.com>

---------

Signed-off-by: ldwang <ftgreat@gmail.com>
Co-authored-by: ldwang <ftgreat@gmail.com>
 2023-07-25 16:22:09 +03:00
Jiahao Li 875086bdb9 ggml : relax contiguous constraints in activation function (#2371) 2023-07-25 15:58:32 +03:00
slaren da1889834a ggml : improve graph build time via hash table lookup (#2329) 
* improve graph build time

* ggml_tensor : use 1 bit per flag

* use a hash table instead
 2023-07-25 15:32:20 +03:00
Hesen Peng 82552b7f54 build : fix line breaking error in build-info.sh (#2349) 
* fix line breaking

* build number line break removal
 2023-07-25 15:24:09 +03:00
Xiao-Yong Jin 0c06204fb3 main : add --in-prefix-bos to prefix BOS to user inputs; keep EOS (#2304) 
* add `--in-prefix-bos` to prefix BOS to user inputs; keep EOS

The BOS precedes the string specified by `--in-prefix`.
Model generated EOS is now kept in the context.

It provides a way to strictly following the prompt format used in
Llama-2-chat.

The EOS handling also benefits some existing finetunes that uses
EOS to mark the end of turn.

* examples/common: move input_prefix_bos to other bools
 2023-07-25 15:19:11 +03:00
Eve 1fed755b1f ci : add non-AVX scalar build/test (#2356) 
* noavx build and test

* we don't need to remove f16c in windows
 2023-07-25 15:16:13 +03:00
katsu560 be2301bcda k_quants : add AVX support to dot functions with QK_K as 64 (#2339) 
* add AVX to ggml_vec_dot_q2_K_q8_K()

* add AVX to ggml_vec_dot_q3_K_q8_K()

* add AVX to ggml_vec_dot_q4_K_q8_K()

* add AVX to ggml_vec_dot_q5_K_q8_K()

* add AVX to ggml_vec_dot_q6_K_q8_K()

* refactor AVX code in ggml_vec_dot_q6_K_q8_K()
 2023-07-25 15:13:41 +03:00
Shouzheng Liu 1aa18ef994 metal : concurrently dispatch commands (#2358) 
* metal: concurrently dispatch commands

Function `ggml_metal_graph_find_concurrency` will run and write
commands that can be issued concurrently to metal context `concur_list`
array, when `ggml_metal_graph_compute` is called for the first time.

* metal: don't call find_concurrency automatically.

* metal : code style changes

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-07-25 15:00:19 +03:00
Concedo 3e68cdd26a Merge branch 'master' into concedo_experimental 
# Conflicts:
#	Makefile
#	tests/test-grad0.c
 2023-07-25 18:52:48 +08:00
Kawrakow 9a08eaf3c4 Another speed gain for Q4_0 and Q4_1 on Metal (#2375) 
* Another speed gain for Q4_0 and Q4_1 on Metal

* Have N_DST, etc., be template parameters

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 2023-07-25 13:48:29 +03:00
Kawrakow 129d844c87 Fix Q4_K and Q5_K for QK_K = 64 on CUDA (#2359) 
* Fix Q4_K and Q5_K for QK_K = 64

* Very slightly better Q5_K bit fiddling

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 2023-07-25 13:48:04 +03:00
Concedo 66e4b5141e fix horde worker host and client agent 2023-07-25 18:18:41 +08:00
slaren d5512b782b server: add rms_norm_eps parameter (#2380) 2023-07-25 12:36:17 +03:00
Henri Vasserman c798308e3a [Server] Escape HTML in webchat (#2368) 
* escape HTML in webchat
* add amp
 2023-07-25 10:27:34 +03:00
Concedo 48c27a9ce1 hotfix for 70b broadcast issues 2023-07-25 01:32:47 +08:00
Александр Герман 9731682ad6 Update Makefile (#345) 
fix requirements for idiotic source file concatenation (lol)
v1.37.1 		2023-07-25 00:21:32 +08:00
slaren 41c674161f make rms_norm_eps a parameter (#2374) 
* make rms_norm_eps a parameter

* add rms_norm_eps to command line

* fix baby llama, test-grad0

* use scientific notation for eps param in the help

ggml-ci
 2023-07-24 17:57:12 +02:00
Concedo d8d2449bfb better label (+1 squashed commits) 
Squashed commits:

[f573b2c] cuda 3 target arch
v1.37 		2023-07-24 23:07:31 +08:00
Aarni Koskela b3f138d058 Chat UI extras (#2366) 
* makefile: correct deps for server

* server: tighten settings layout a little

* server: expose all currently configured generation params in UI

* server: expose remaining generation params, for the adventurous

* server: embetter mirostat fields
 2023-07-24 17:54:22 +03:00
Concedo 7555dae4cc ditch advanced subparsers 2023-07-24 22:40:36 +08:00
Concedo 8a9b40840b Merge branch 'master' into concedo_experimental 
# Conflicts:
#	tests/test-grad0.c
#	tests/test-opt.c
 2023-07-24 20:51:28 +08:00
Concedo 6d71e100fe buff buffers 2023-07-24 20:33:17 +08:00
Georgi Gerganov 5b2b2dc6ae ggml : sync (unary ops refactor, static-correctness) (#2370) 
* ggml : sync (unary ops, tests)

ggml-ci

* tests : remove unnecessary funcs
 2023-07-24 14:46:21 +03:00
Concedo 825e34baa3 default horde name and better handling for horde (+3 squashed commit) 
Squashed commit:

[fadfa60] better idle handling for horde worker

[a3971e6] updated lite

[2ca2b79] seems to not generate rubbish
 2023-07-24 18:41:41 +08:00
Kawrakow 42f70cb2f6 Fix scalar version of Q5_K when QK_K = 64 (#2362) 
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 2023-07-24 12:55:02 +03:00
Concedo c7136f03d9 added support for tensor_split parameter as an advanced parameter. 2023-07-24 17:16:19 +08:00
Concedo 66328fcd80 Merge branch 'master' into concedo_experimental 
# Conflicts:
#	Makefile
 2023-07-24 15:44:26 +08:00
Concedo 94499dba25 added support for 70b llama 2 2023-07-24 15:20:18 +08:00
Concedo 993ba3b026 Merge branch 'master' into concedo_experimental 
# Conflicts:
#	README.md
 2023-07-24 11:59:00 +08:00
Evan Jones 84e09a7d8b llama : add grammar-based sampling (#1773) 
* llama, main : constrain sampling to grammar

* allow loading grammar from file

* fix whitespace errors

* handle & print parser errors

* add comments to grammar syntax and allow newlines where unambiguous

* add missing include

* support alternates in root rule

* fix bugs with empty token and EOS

* adjust JSON grammar

* remove swp file

* rewrite ternary expressions

Co-authored-by: Henri Vasserman <henv@hot.ee>

* use struct for grammar elements and add Unicode support

* add unicode escapes

* add inverse char ranges

* only sample full tokens (no peeking or truncation)

* llama : minor style changes

blindly applied in online editor - hopefully I didn't break something

* update help text

* add warning message if EOS is disabled

---------

Co-authored-by: Henri Vasserman <henv@hot.ee>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-07-23 23:58:10 -04:00
Concedo 280abaf029 added stop reason in the perf endpoint 2023-07-24 11:55:35 +08:00
Kawrakow 2f9cf974a0 Some more Q4_K and Q5_K speedup on CUDA (#2346) 
* Faster Q5_K on CUDA

* Small Q5_K improvement on older GPUs

* Spped up Q4_K on CUDA

GTX1660: 29.5 ms/t -> 25.6 ms/t
RTX4080: 8.40 ms/t -> 8.25 ms/t

* Spped up Q4_K on CUDA

GTX1660: 36.7 ms/t -> 35.6 ms/t
RTX4080:  9.8 ms/t ->  9.5 ms/t

* Address PR comments

* Add some comments to satisfy PR reviewer

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 2023-07-24 00:19:47 +03:00
IgnacioFDM 4f06592cc6 Add gqa parameter support to the server (#2351) 
* Add gqa parameter support to the server
* Change help from stderr to stdout
 2023-07-23 23:31:17 +03:00
Johannes Gäßler 70d26ac388 Fix __dp4a documentation (#2348) 2023-07-23 17:49:06 +02:00
Concedo 910744e2c0 Merge branch 'master' into concedo_experimental 
# Conflicts:
#	Makefile
#	README.md
#	flake.nix
#	llama.cpp
 2023-07-23 22:37:38 +08:00
Concedo c28ab4e1b7 update lite, try support k80 2023-07-23 21:50:35 +08:00
wzy 57921ca6db common : n_threads == -1 uses std::thread::hardware_concurrency() (#2347) 
* Fix #2345, fix incorrect n_threads

* Update examples/common.cpp

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-07-23 16:33:02 +03:00
slaren 3602ac4255 fix n_tasks (#2342) 
ggml-ci
 2023-07-23 15:19:39 +02:00
slaren 95a6c595e7 ggml: move op parameters from tensors to ggml_tensor::op_params (#2333) 
* ggml: move op parameters from tensors to ggml_tensor::op_params

* alibi: use memcpy for float params

* remove `src[1] = NULL` in ops
 2023-07-23 14:36:02 +02:00