11 Commits

Author	SHA1	Message	Date
Concedo	6548d3b3fb	Added prints for stopping sequences, made makefile 1% friendlier to arch linux users	2023-04-16 20:43:17 +08:00
Concedo	525184930d	added a kobold API compatible implementation of stopping sequences	2023-04-16 18:37:49 +08:00
Concedo	ad5676810a	merge CLBlast improvements - GPU dequant	2023-04-16 01:17:40 +08:00
Concedo	8ad42a1102	read from inputs	2023-04-14 21:30:26 +08:00
Concedo	adb4df78d6	Added SmartContext mode, a way of prompt context manipulation that avoids frequent context recalculation.	2023-04-14 21:24:16 +08:00
Concedo	5c22f7e4c4	reduce batch sizes and skip all intrinsic flags except AVX when building in compatibility mode.	2023-04-13 11:32:05 +08:00
Concedo	69b85f5b61	fixed a few OOM errors with larger contexts - I cannot figure out why they happen, so I am forced to increase the buffer size.	2023-04-11 00:14:57 +08:00
Concedo	18a154715e	added version label, improved file type checks	2023-04-10 01:03:09 +08:00
Concedo	b91abc3316	increase default blas batch size	2023-04-09 15:27:43 +08:00
Concedo	d8e37bfe75	new gpt2 format supported	2023-04-08 17:35:36 +08:00
Concedo	14273fea7a	integrated gpt2 support	2023-04-04 23:15:47 +08:00