8567c76b53
Update server instructions for web front end ( #2103 )
...
Co-authored-by: Jesse Johnson <thatguy@jessejojojohnson.com >
2023-07-05 18:13:35 +03:00
924dd22fd3
Quantized dot products for CUDA mul mat vec ( #2067 )
2023-07-05 14:19:42 +02:00
051c70dcd5
llama: Don't double count the sampling time ( #2107 )
2023-07-05 18:31:23 +08:00
ea79e549f0
fixed refusing to quantize some models
2023-07-05 17:29:35 +08:00
9e4475f5cf
Fixed OpenCL offloading prints ( #2082 )
2023-07-05 08:58:05 +02:00
7f0e9a775e
embd-input: Fix input embedding example unsigned int seed ( #2105 )
2023-07-05 07:33:33 +08:00
b472f3fca5
readme : add link web chat PR
2023-07-04 22:25:22 +03:00
ed9a54e512
ggml : sync latest (new ops, macros, refactoring) ( #2106 )
...
- add ggml_argmax()
- add ggml_tanh()
- add ggml_elu()
- refactor ggml_conv_1d() and variants
- refactor ggml_conv_2d() and variants
- add helper macros to reduce code duplication in ggml.c
2023-07-04 21:54:11 +03:00
f257fd2550
Add an API example using server.cpp similar to OAI. ( #2009 )
...
* add api_like_OAI.py
* add evaluated token count to server
* add /v1/ endpoints binding
2023-07-04 21:06:12 +03:00
7ee76e45af
Simple webchat for server ( #1998 )
...
* expose simple web interface on root domain
* embed index and add --path for choosing static dir
* allow server to multithread
because web browsers send a lot of garbage requests we want the server
to multithread when serving 404s for favicon's etc. To avoid blowing up
llama we just take a mutex when it's invoked.
* let's try this with the xxd tool instead and see if msvc is happier with that
* enable server in Makefiles
* add /completion.js file to make it easy to use the server from js
* slightly nicer css
* rework state management into session, expose historyTemplate to settings
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2023-07-04 16:05:27 +02:00
acc111caf9
Allow old Make to build server. ( #2098 )
...
Also make server build by default.
Tested with Make 3.82
2023-07-04 15:38:04 +03:00
23c7c6fc91
Update Makefile: clean simple ( #2097 )
2023-07-04 14:15:16 +02:00
69add28324
Merge branch 'master' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
2023-07-04 18:51:42 +08:00
00e35d0bbf
Merge branch 'concedo' into concedo_experimental
2023-07-04 18:46:40 +08:00
f9108ba401
Make koboldcpp.py executable on Linux ( #293 )
2023-07-04 18:46:08 +08:00
fff705d4f6
Merge remote-tracking branch 'ycros/improve-sampler-api-access' into concedo_experimental
2023-07-04 18:42:02 +08:00
c6c0afdf18
refactor to avoid code duplication
2023-07-04 18:35:54 +08:00
784628a2be
Merge remote-tracking branch 'ycros/improve-sampler-api-access' into concedo_experimental
2023-07-04 16:38:32 +08:00
698efad5fb
CI: make the brew update temporarily optional. ( #2092 )
...
until they decide to fix the brew installation in the macos runners.
see the open issues. eg https://github.com/actions/runner-images/pull/7710
2023-07-04 01:50:12 +02:00
14a2cc71f6
[ggml] fix index for ne03 value in ggml_cl_mul_f32 ( #2088 )
2023-07-04 07:50:00 +08:00
1cf14ccef1
fix server crashes ( #2076 )
2023-07-04 00:05:23 +03:00
cc45a7feb8
Fix crash of test-tokenizer-0 under Debug build ( #2064 )
...
* Fix crash of test-tokenizer-0 under Debug build
* Change per comment
2023-07-03 20:43:55 +02:00
ca9a11697c
possibly slower, but cannot use larger batches without modifying ggml library.
2023-07-04 00:35:02 +08:00
bfeb3471d7
fix typos
2023-07-03 21:36:42 +08:00
55dbb915cc
[llama] No need to check file version when loading vocab score ( #2079 )
2023-07-03 19:58:58 +08:00
d7d2e6a0f0
server: add option to output probabilities for completion ( #1962 )
...
* server: add option to output probabilities for completion
* server: fix issue when handling probability output for incomplete tokens for multibyte character generation
* server: fix llama_sample_top_k order
* examples/common.h: put all bool variables in gpt_params together
2023-07-03 00:38:44 +03:00
309534dcd0
implement sampler order, expose sampler order and mirostat in api
2023-07-02 18:15:34 +00:00
3d2907d208
make gptneox and gptj work with extended context too
2023-07-02 18:28:09 +08:00
d6b47e6a5b
Merge branch 'master' into concedo_experimental
2023-07-02 17:26:39 +08:00
e17c8497cf
switched to NTK aware scaling
2023-07-02 17:25:08 +08:00
e19483ca0f
increase scratch for above 4096
2023-07-02 14:55:08 +08:00
46088f7231
ggml : fix build with OpenBLAS ( close #2066 )
2023-07-02 09:46:46 +03:00
b85ea580d3
Merge branch 'master' into concedo_experimental
...
# Conflicts:
# README.md
2023-07-02 14:45:25 +08:00
0bc2cdfc87
Better CUDA synchronization logic ( #2057 )
2023-07-01 21:49:44 +02:00
befb3a3562
Test-based VRAM scratch size + context adjustment ( #2056 )
2023-07-01 21:47:26 +02:00
b213227067
cmake : don't force -mcpu=native on aarch64 ( #2063 )
...
It's currently not possible to cross-compile llama.cpp for aarch64
because CMakeLists.txt forces -mcpu=native for that target.
-mcpu=native doesn't make sense if your build host is not the
target architecture, and clang rejects it for that reason, aborting the
build. This can be easily reproduced using the current Android NDK to build
for aarch64 on an x86_64 host.
If there is not a specific CPU-tuning target for aarch64 then -mcpu
should be omitted completely. I think that makes sense, there is not
enough variance in the aarch64 instruction set to warrant a fixed -mcpu
optimization at this point. And if someone is building natively and wishes
to enable any possible optimizations for the host device, then there is
already the LLAMA_NATIVE option available.
Fixes #495 .
2023-07-01 21:31:44 +03:00
2f8cd979ec
metal : release buffers when freeing metal context ( #2062 )
2023-07-01 21:14:59 +03:00
471aab6e4c
convert : add support of baichuan-7b ( #2055 )
...
Co-authored-by: Judd <foldl@boxvest.com >
2023-07-01 20:00:25 +03:00
ef3b8dc0d9
GPU accel for rwkv is slow, disable it
2023-07-02 00:41:46 +08:00
e1a7042943
try out the new rwkv but it seems worse, may revert
2023-07-02 00:10:56 +08:00
463f2f4c4f
llama : fix return value of llama_load_session_file_internal ( #2022 )
2023-07-01 19:05:09 +03:00
cb44dbc7de
llama : catch llama_load_session_file_internal exceptions ( #2022 )
...
* convert checks in llama_load_session_file to throw and handle them
* make llama_load_session_file_internal static
* address feedbacks to avoid using exceptions
2023-07-01 19:02:58 +03:00
79f634a19d
embd-input : fix returning ptr to temporary
2023-07-01 18:46:00 +03:00
04606a1599
train : fix compile warning
2023-07-01 18:45:44 +03:00
b1ca8f36a9
ggml : disable GGML_TASK_INIT and GGML_TASK_FINALIZE by default ( #1995 )
...
Will not be scheduled unless explicitly enabled.
2023-07-01 18:42:43 +03:00
632bf27b65
more granular context size selections
2023-07-01 11:02:44 +08:00
eda663f15f
update lite and up ver
2023-07-01 00:15:26 +08:00
0cb8a9eab3
Merge remote-tracking branch 'Johannes/cuda-scratch-size-adjust' into concedo_experimental
...
# Conflicts:
# llama.cpp
2023-06-30 23:29:38 +08:00
67cb0b2760
Merge branch 'master' into concedo_experimental
2023-06-30 23:25:40 +08:00
d16926dff4
Merge branch 'concedo' into concedo_experimental
2023-06-30 23:06:21 +08:00
Jesse Jojo Johnson
Johannes Gäßler
Howard Su
Concedo
Nigel Bosch
Georgi Gerganov
jwj7140
Tobias Lütke
Henri Vasserman
ZhouYuChen
Michael Moon
Erik Scholz
Govlzkoy
WangHaoranRobin
Ycros
Daniel Drake
Aaron Miller
Judd
Rand Xie
Qingyou Meng