9 Commits

Author	SHA1	Message	Date
Concedo	ad5676810a	merge CLBlast improvements - GPU dequant	2023-04-16 01:17:40 +08:00
Concedo	8ad42a1102	read from inputs	2023-04-14 21:30:26 +08:00
Concedo	adb4df78d6	Added SmartContext mode, a way of prompt context manipulation that avoids frequent context recalculation.	2023-04-14 21:24:16 +08:00
Concedo	5c22f7e4c4	reduce batch sizes and skip all intrinsic flags except AVX when building in compatibility mode.	2023-04-13 11:32:05 +08:00
Concedo	69b85f5b61	fixed a few OOM errors with larger contexts - I cannot figure out why they happen, so I am forced to increase the buffer size.	2023-04-11 00:14:57 +08:00
Concedo	18a154715e	added version label, improved file type checks	2023-04-10 01:03:09 +08:00
Concedo	b91abc3316	increase default blas batch size	2023-04-09 15:27:43 +08:00
Concedo	d8e37bfe75	new gpt2 format supported	2023-04-08 17:35:36 +08:00
Concedo	14273fea7a	integrated gpt2 support	2023-04-04 23:15:47 +08:00