Commit Graph

55 Commits

Author SHA1 Message Date
Vladimir Mandic 7b150ba361 fix bnb loader
Signed-off-by: Vladimir Mandic <mandic00@live.com>
2024-10-29 12:36:03 -04:00
Vladimir Mandic 58220b6497 flux enabled bnb quant on-the-fly
Signed-off-by: Vladimir Mandic <mandic00@live.com>
2024-10-29 12:30:20 -04:00
Vladimir Mandic 6760632f38 major model load refactor
Signed-off-by: Vladimir Mandic <mandic00@live.com>
2024-10-26 13:22:29 -04:00
Vladimir Mandic f191134aa6 support bnb quantization during load
Signed-off-by: Vladimir Mandic <mandic00@live.com>
2024-10-22 15:43:09 -04:00
Vladimir Mandic ea0dfebe2d better handle any quant lib requirements
Signed-off-by: Vladimir Mandic <mandic00@live.com>
2024-10-12 13:36:16 -04:00