Public/koboldcpp

mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-08-26 23:20:49 +02:00

Files

T

History

Concedo 22f543d09b Merge commit '32c8486e1f0297393cb22ac0a0d26a6b17ad4d54' into concedo_experimental

# Conflicts:
#	.devops/nix/package.nix
#	CMakeLists.txt
#	Makefile
#	Package.swift
#	README.md
#	build.zig
#	llama.cpp
#	tests/test-backend-ops.cpp

2024-04-07 20:39:17 +08:00

..

CMakeLists.txt

build : link against build info instead of compiling against it (#3879 )

2023-11-02 08:50:16 +02:00

quantize.cpp

Merge commit '32c8486e1f0297393cb22ac0a0d26a6b17ad4d54' into concedo_experimental

2024-04-07 20:39:17 +08:00

README.md

readme : add some recent perplexity and bpw measurements to READMES, link for k-quants (#3340 )

2023-09-27 18:30:36 +03:00

README.md

quantize

TODO

Llama 2 7B

Quantization	Bits per Weight (BPW)
Q2_K	3.35
Q3_K_S	3.50
Q3_K_M	3.91
Q3_K_L	4.27
Q4_K_S	4.58
Q4_K_M	4.84
Q5_K_S	5.52
Q5_K_M	5.68
Q6_K	6.56

Llama 2 13B

Quantization	Bits per Weight (BPW)
Q2_K	3.34
Q3_K_S	3.48
Q3_K_M	3.89
Q3_K_L	4.26
Q4_K_S	4.56
Q4_K_M	4.83
Q5_K_S	5.51
Q5_K_M	5.67
Q6_K	6.56

Llama 2 70B

Quantization	Bits per Weight (BPW)
Q2_K	3.40
Q3_K_S	3.47
Q3_K_M	3.85
Q3_K_L	4.19
Q4_K_S	4.53
Q4_K_M	4.80
Q5_K_S	5.50
Q5_K_M	5.65
Q6_K	6.56