Public/koboldcpp
1
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-09-19 17:25:07 +02:00
Code Issues Packages Projects Releases Wiki Activity

Updated readme

Browse Source
This commit is contained in:
Concedo
2023-03-22 16:20:37 +08:00
parent 5f142df76e
commit 4e95e7f87f
2 changed files with 6 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+5 -4
View File
@@ -1,10 +1,8 @@
# llama-for-kobold # llama-for-kobold
A hacky little script from Concedo that exposes llama.cpp function bindings, allowing it to be used via a simulated Kobold API endpoint. A self contained distributable from Concedo that exposes llama.cpp function bindings, allowing it to be used via a simulated Kobold API endpoint.
It's not very usable as there is a fundamental flaw with llama.cpp, which causes generation delay to scale linearly with original prompt length. Nobody knows why or really cares much, so I'm just going to publish whatever I have at this point. ![Preview](preview.png)
If you care, **please contribute to [this discussion](https://github.com/ggerganov/llama.cpp/discussions/229)** which, if resolved, will actually make this viable.
## Considerations ## Considerations
- Don't want to use pybind11 due to dependencies on MSVCC - Don't want to use pybind11 due to dependencies on MSVCC
@@ -23,3 +21,6 @@ If you care, **please contribute to [this discussion](https://github.com/ggergan
- The original GGML library and llama.cpp by ggerganov are licensed under the MIT License - The original GGML library and llama.cpp by ggerganov are licensed under the MIT License
- However, Kobold Lite is licensed under the AGPL v3.0 License - However, Kobold Lite is licensed under the AGPL v3.0 License
- The provided python ctypes bindings in llamacpp.dll are also under the AGPL v3.0 License - The provided python ctypes bindings in llamacpp.dll are also under the AGPL v3.0 License
## Notes
- There is a fundamental flaw with llama.cpp, which causes generation delay to scale linearly with original prompt length. If you care, **please contribute to [this discussion](https://github.com/ggerganov/llama.cpp/discussions/229)** which, if resolved, will actually make this viable.
preview.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 92 KiB