* Functionality addition CMakeLists.txt
Refactoring:
1. Simplify more options that are negation of negation.
LLAMA_NO_ACCELERATE -> LLAMA_ACCELERATE
2. Changed to an optional expression instead of forcing to enable AVX2 in MSVC.
3. Make CMAKE_CXX_STANDARD, which is different from Makefile, the same.
4. Use add_compile_options instead of adding options to CMAKE_C_FLAGS.
5. Make utils use target_link_libraries instead of directly referencing code.
Added features:
1. Added some options.
LLAMA_STATIC_LINK,LLAMA_NATIVE,LLAMA_LTO,LLAMA_GPROF,LLAMA_OPENBLAS
* Fix Accelerate link in CMake
* Windows build Fix
* C++11 to C++17
* Reflects C/C++ standard individually
* Change the version to 3.12
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* potential out of bounds read
* fix quantize
* style
* Update convert-pth-to-ggml.py
* mild cleanup
* don't need the space-prefixing here rn since main.cpp already does it
* new file magic + version header field
* readme notice
* missing newlines
Co-authored-by: slaren <2141330+slaren@users.noreply.github.com>
* fix coloring of last `n_batch` of prompt, and refactor line input
* forgot the newline that needs to be sent to the model
* (per #283) try to force flush of color reset in SIGINT handler
* Improved quantize script
I improved the quantize script by adding error handling and allowing to select many models for quantization at once in the command line. I also converted it to Python for generalization as well as extensibility.
* Fixes and improvements based on Matt's observations
Fixed and improved many things in the script based on the reviews made by @mattsta. The parallelization suggestion is still to be revised, but code for it was still added (commented).
* Small fixes to the previous commit
* Corrected to use the original glob pattern
The original Bash script uses a glob pattern to match files that have endings such as ...bin.0, ...bin.1, etc. That has been translated correctly to Python now.
* Added support for Windows and updated README to use this script
New code to set the name of the quantize script binary depending on the platform has been added (quantize.exe if working on Windows) and the README.md file has been updated to use this script instead of the Bash one.
* Fixed a typo and removed shell=True in the subprocess.run call
Fixed a typo regarding the new filenames of the quantized models and removed the shell=True parameter in the subprocess.run call as it was conflicting with the list of parameters.
* Corrected previous commit
* Small tweak: changed the name of the program in argparse
This was making the automatic help message to be suggesting the program's usage as being literally "$ Quantization Script [arguments]". It should now be something like "$ python3 quantize.py [arguments]".
* Use F16 for memory_k and memory_v
* add command line switch to use f16 instead of f32 for memory k+v
---------
Co-authored-by: Ty Everett <ty@tyweb.us>
* Refactor get_n_parts function to simplify code and improve readability
* Use f-strings instead of concatenation
* Refactoring: more concise and readable
* modularize
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Concedo
Henk
nusu-github
Ben Siraphob
Qingyou Meng
Mack Straight
Bernat Vadell
Stephan Walter
cocktailpeanut
Georgi Gerganov
Rickey Bowers Jr
tjohnman
Suaj Carrot
slaren
Erik Scholz
qunash
Ronsor
Pavol Rusnak