52 Commits

Author	SHA1	Message	Date
Disty0	4e3795a0a5	SDNQ fix packed int8 matmul	2025-06-02 03:31:51 +03:00
Disty0	82f5634d53	SDNQ use torch.bool for uint1	2025-06-02 01:39:51 +03:00
Disty0	e8588c91ea	SDNQ enable matmul support for float8_e5m2	2025-06-02 00:53:10 +03:00
Disty0	8f1a1d7311	SDNQ expand quantized_matmul_dtypes for CPU	2025-06-02 00:28:29 +03:00
Disty0	b146025a5e	SDNQ add int2	2025-06-02 00:17:39 +03:00
Disty0	9669b36010	SDNQ fix older PyTorch with FP8 matmul	2025-06-01 23:29:16 +03:00
Disty0	acefa58834	SDNQ don't force fp32 with fp8 tensorwise matmul	2025-06-01 23:16:00 +03:00
Disty0	839295f79a	Add fp8 fnuz to sdnq options	2025-06-01 23:10:08 +03:00
Disty0	cefe460052	SDNQ skip FP8 matmul for input len < 32	2025-05-31 01:27:59 +03:00
Disty0	109c0d7e49	SDNQ use tensorwise FP8 matmul on CPU	2025-05-30 21:09:53 +03:00
Disty0	959b759721	Cleanup	2025-05-30 16:45:59 +03:00
Disty0	b5d588fa45	SDNQ remove unnecessary bitwise ands	2025-05-30 16:29:59 +03:00
Disty0	db816d7088	Cleanup	2025-05-30 16:02:26 +03:00
Disty0	c85cc6b397	SDNQ enable quant with GPU by default and don't do unnecessary clones	2025-05-30 15:21:29 +03:00
Disty0	4654acde3c	SDNQ re-enable memory fix for diffusers	2025-05-30 14:59:45 +03:00
Disty0	87a801e24d	SDNQ remove memory fix hijack	2025-05-30 13:54:49 +03:00
Disty0	f81cb22c00	SDNQ fix new transformers	2025-05-30 13:32:03 +03:00
Disty0	36febda6e6	SDNQ update supported dtypes	2025-05-30 13:07:23 +03:00
Disty0	29bd2af779	SDNQ add 6-bit support	2025-05-30 12:20:13 +03:00
Disty0	d1491962d9	One bit	2025-05-30 05:41:02 +03:00
Disty0	3c8be0f55f	SDNQ add uint2	2025-05-30 04:47:29 +03:00
Disty0	599224d392	SDNQ reduce 5 reshape ops to 2 with quantized input	2025-05-30 01:31:41 +03:00
Disty0	d8dea9031f	SDNQ do FP8 matmul shape check only once	2025-05-30 01:13:37 +03:00
Disty0	b4e615e760	SDNQ add FP8 row wise scaling workaround for SM89 on Windows	2025-05-30 00:16:54 +03:00
Disty0	54154cf698	Cleanup	2025-05-29 20:22:49 +03:00
Disty0	df8b31fcfc	Don't downcast scale with fp8 matmul	2025-05-29 16:35:40 +03:00
Disty0	2351efb8f7	Remove redundant shape check	2025-05-29 14:58:00 +03:00
Disty0	14893b7617	Don't make the weights contiguous with int8 matmul	2025-05-29 03:43:57 +03:00
Disty0	2cc5a58b0f	Update changelog	2025-05-29 03:26:47 +03:00
Disty0	67e0f4d833	Cleanup	2025-05-29 03:22:40 +03:00
Disty0	3698f8bb84	SDNQ add experimental FP8 matmul	2025-05-29 03:11:59 +03:00
Disty0	dd33c4d583	Fix scale and zero_point not being moved by tensor.to	2025-05-28 17:46:06 +03:00
Disty0	dd0dbc476f	SDNQ fix asym quant formula for dtypes with non zero minimums	2025-05-28 17:25:38 +03:00
Disty0	e06cbea7aa	Cleanup	2025-05-28 15:55:08 +03:00
Disty0	d8e8f47ce5	SDNQ add an option to toggle quantize with GPU	2025-05-28 15:18:39 +03:00
Disty0	569e9099d7	Use torch.amax instead of torch.max	2025-05-28 12:44:07 +03:00
Disty0	0b564e2373	Cleanup	2025-05-28 04:07:45 +03:00
Disty0	1433dfe3de	SDNQ fix high RAM usage with pre mode	2025-05-28 03:16:29 +03:00
Disty0	5d3c1832b2	SDNQ add FP8 quants	2025-05-27 20:29:15 +03:00
Disty0	3618e39cff	SDNQ use device_map = gpu	2025-05-27 19:46:30 +03:00
Disty0	73999ac710	Add soft gc to nncf quant layer	2025-05-27 16:24:04 +03:00
Disty0	e94128a02e	SDNQ add force torch_gc to pre load mode	2025-05-27 16:11:04 +03:00
Disty0	dece497f10	Refactor SDNQ to use weights_dtype and rename decompress_int8_matmul to use_quantized_matmul	2025-05-27 15:49:21 +03:00
Disty0	dec460e665	SDNQ use torch.bitwise ops instead of python	2025-05-27 03:02:36 +03:00
Disty0	4d9c2a8608	Cleanup	2025-05-26 22:41:12 +03:00
Disty0	84ddfb2868	SDNQ fix lora apply	2025-05-26 22:39:20 +03:00
Disty0	687c50dcc8	SDNQ fix Lora	2025-05-26 19:48:45 +03:00
Disty0	02f15b28cc	Cleanup	2025-05-26 15:57:17 +03:00
Disty0	91bb07f650	SDNQ remove unused args and simplify decompressors	2025-05-26 15:51:53 +03:00
Disty0	d2159af10e	cleanup	2025-05-26 04:24:28 +03:00