13 Commits

Author	SHA1	Message	Date
Concedo	b3315459c7	pilled the new dequants for clblast, fixed some ooms	2023-04-30 14:15:44 +08:00
Concedo	032a171867	integrated q5 formats	2023-04-28 12:58:39 +08:00
Concedo	0aa3d839fb	free old ctx on retry	2023-04-25 23:42:57 +08:00
Concedo	72b2331ad6	edge cases with mem crash? need verify	2023-04-25 20:42:30 +08:00
Concedo	5eec5d6ed9	Added backwards compatibility to an earlier version of NeoX.	2023-04-25 20:34:18 +08:00
Concedo	59fb174678	fixed compile errors, made mmap automatic when lora is selected, added updated quantizers and quantization handling for gpt neox gpt 2 and gptj	2023-04-24 23:20:06 +08:00
Concedo	432cc91649	still needs to be a bit higher for very small contexts	2023-04-23 15:01:38 +08:00
Concedo	4e1ea2ac61	hopefully fixed the ooms for good	2023-04-23 13:49:50 +08:00
Concedo	d41490c27b	just revert back to the working commit	2023-04-23 00:35:42 +08:00
Concedo	b5d6284190	increase initial buffer too	2023-04-23 00:07:33 +08:00
Concedo	d2f14b2b1f	add an extra buffer to mem allocations	2023-04-23 00:04:32 +08:00
Concedo	c454f8b848	Gpt NeoX / Pythia integration completed	2023-04-22 11:23:25 +08:00
Concedo	68898046c2	accidentally added the binaries onto repo again.	2023-04-22 00:41:19 +08:00