e9dfe2cefd
try using qemu to do the pyinstaller
2024-12-07 00:53:35 +08:00
b411192db4
revert
2024-12-07 00:33:46 +08:00
25b5301e5a
try using qemu to do the pyinstaller
2024-12-07 00:24:35 +08:00
58038cddc3
try using qemu to do the pyinstaller
2024-12-07 00:19:13 +08:00
e9d2332dd8
improved tool calls and whisper
2024-12-06 14:34:31 +08:00
836c06d91a
minor edit
2024-12-06 00:37:38 +08:00
746cb01843
remove test since it wont work on x64
2024-12-06 00:26:58 +08:00
65a11451e3
fix missing bundled files
2024-12-06 00:21:08 +08:00
fe72c8db9f
CI for ARM should appear as ARM
2024-12-06 00:12:30 +08:00
5cddd0a878
Merge branch 'concedo' into concedo_experimental
2024-12-05 23:58:31 +08:00
ece96e19bf
clean up makefile
2024-12-05 23:58:23 +08:00
8d5bb06aeb
test aarch64 ci workflow
2024-12-05 23:57:25 +08:00
d0d1d922de
handle and fix temp paths to chat completions adapter
2024-12-05 17:22:35 +08:00
5106816eac
drafted tokens debug prints
2024-12-05 17:05:20 +08:00
2787fca6b4
refactored library selection, fixed ollama params
2024-12-05 16:47:52 +08:00
52cc908f7f
default trim_stop to true, which trims any tokens after a stop sequence and the stop sequence itself. This is potentially a breaking change.
2024-12-03 22:44:10 +08:00
7d11d2946c
only show warning if more than 1 moved tensor
2024-12-03 22:09:26 +08:00
ed9e229372
docs: update README.md ( #1244 )
...
recomended -> recommended
2024-12-02 17:20:20 +08:00
2ba5949054
updated sdcpp, also set euler as default sampler
2024-12-01 17:00:20 +08:00
e93c2427b4
allow incompatible vocab in debugmode
2024-12-01 14:11:03 +08:00
42228b9746
warning when selecting non gguf models
2024-12-01 13:35:51 +08:00
d5e732f3ab
updated lite
2024-12-01 01:49:09 +08:00
b7cd210cd2
more linting with Ruff (+1 squashed commits)
...
Squashed commits:
[43802cfe2] Applied default Ruff linting
2024-12-01 01:23:13 +08:00
409e393d10
fixed critical bug in image model loader
2024-11-30 23:28:24 +08:00
153da19274
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# README.md
2024-11-30 16:59:25 +08:00
0028e71993
special handling to resolve incomplete utf8 token sequences in qwen
2024-11-30 16:54:01 +08:00
32ac3153e4
default speculative set to 8. added more adapter fields
2024-11-30 16:18:27 +08:00
3e0ba0e604
readme : remove old badge
2024-11-30 10:09:21 +02:00
abadba05be
readme : refresh ( #10587 )
...
* readme : refresh
* readme : move section [no ci]
* readme : clarify [no ci]
* readme : fixes [no ci]
* readme : more fixes [no ci]
* readme : simplify [no ci]
* readme : clarify GGUF
2024-11-30 09:47:07 +02:00
0533e7fb38
vulkan: Dynamic subgroup size support for Q6_K mat_vec ( #10536 )
...
* subgroup 64 version with subgroup add. 15% faster
scalable version
tested for subgroup sizes 16-128
* check for subgroup multiple of 16 and greater than 16
* subgroup sizes are always a power of 2 (https://github.com/KhronosGroup/GLSL/issues/45 )
* force 16 sequential threads per block
* make 16 subgroup size a constant
2024-11-30 08:00:02 +01:00
5353bfa983
updated lite
2024-11-30 12:26:20 +08:00
557bcaf86e
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .clang-tidy
# .github/workflows/build.yml
# Makefile
# Package.swift
# common/CMakeLists.txt
# examples/batched-bench/CMakeLists.txt
# examples/batched/CMakeLists.txt
# examples/convert-llama2c-to-ggml/CMakeLists.txt
# examples/cvector-generator/CMakeLists.txt
# examples/embedding/CMakeLists.txt
# examples/eval-callback/CMakeLists.txt
# examples/export-lora/CMakeLists.txt
# examples/gbnf-validator/CMakeLists.txt
# examples/gguf-split/CMakeLists.txt
# examples/gguf/CMakeLists.txt
# examples/gritlm/CMakeLists.txt
# examples/imatrix/CMakeLists.txt
# examples/infill/CMakeLists.txt
# examples/llama-bench/CMakeLists.txt
# examples/llava/CMakeLists.txt
# examples/lookahead/CMakeLists.txt
# examples/lookup/CMakeLists.txt
# examples/main-cmake-pkg/CMakeLists.txt
# examples/main/CMakeLists.txt
# examples/parallel/CMakeLists.txt
# examples/passkey/CMakeLists.txt
# examples/perplexity/CMakeLists.txt
# examples/quantize-stats/CMakeLists.txt
# examples/quantize/CMakeLists.txt
# examples/retrieval/CMakeLists.txt
# examples/run/CMakeLists.txt
# examples/save-load-state/CMakeLists.txt
# examples/server/CMakeLists.txt
# examples/simple-chat/CMakeLists.txt
# examples/simple/CMakeLists.txt
# examples/speculative-simple/CMakeLists.txt
# examples/speculative/CMakeLists.txt
# examples/tokenize/CMakeLists.txt
# ggml/CMakeLists.txt
# ggml/src/CMakeLists.txt
# ggml/src/ggml-backend.cpp
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
# pocs/vdot/CMakeLists.txt
# src/CMakeLists.txt
# src/unicode.cpp
# tests/test-sampling.cpp
2024-11-30 12:24:51 +08:00
697ca70115
temp checkpoint
2024-11-30 12:13:20 +08:00
ec95241e38
temp checkpoint
2024-11-30 11:59:27 +08:00
0c8939be19
temp checkpoint
2024-11-30 11:57:28 +08:00
e0c59486ee
default to 12 tokens drafted
2024-11-30 11:52:07 +08:00
b21d0fe3ac
customizable speculative size
2024-11-30 11:28:19 +08:00
f75bbb945f
speculative decoding initial impl completed (+6 squashed commit)
...
Squashed commit:
[0a6306ca0] draft wip dont use (will be squashed)
[a758a1c9c] wip dont use (will be squashed)
[e1994d3ce] wip dont use
[f59690d68] wip
[77228147d] wip on spec decoding. dont use yet
[2445bca54] wip adding speculative decoding (+1 squashed commits)
Squashed commits:
[50e341bb7] wip adding speculative decoding
2024-11-30 10:41:10 +08:00
7cc2d2c889
ggml : move AMX to the CPU backend ( #10570 )
...
* ggml : move AMX to the CPU backend
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2024-11-29 21:54:58 +01:00
b782e5c7d4
server : add more test cases ( #10569 )
...
* server : add split model test
* add test speculative
* add invalid cases
2024-11-29 21:48:56 +01:00
3a8e9af402
imatrix : support combine-only ( #10492 )
...
* imatrix-combine-only idea
* ensured that behavior consistent with log
2024-11-29 19:21:37 +02:00
a3a3048e7a
cleanup UI link list ( #10577 )
...
* cleanup UI link list
* sort list alphabetically
* add missing licenses
2024-11-29 17:45:08 +01:00
f0678c5ff4
ggml : fix I8MM Q4_1 scaling factor conversion ( #10562 )
...
ggml-ci
2024-11-29 16:25:39 +02:00
4b3242bbea
ggml-cpu: fix typo in gemv/gemm iq4_nl_4_4 ( #10580 )
2024-11-29 14:49:02 +01:00
0f77aae560
sycl : offload of get_rows set to 0 ( #10432 )
2024-11-29 20:38:45 +08:00
266b8519ee
sycl : Reroute permuted mul_mats through oneMKL ( #10408 )
...
This PR fixes the failing MUL_MAT tests for the sycl backend.
2024-11-29 09:49:43 +00:00
938f608742
CANN: RoPE operator optimization ( #10563 )
...
* [cann] RoPE operator optimization
* [CANN]Code Formatting
---------
Co-authored-by: noemotiovon <noemotiovon@gmail.com >
2024-11-29 14:46:55 +08:00
f095a649ec
vulkan: get the first command buffer submitted sooner ( #10499 )
...
This is an incremental improvement over #9118 to get work to the GPU a bit
sooner. The first part is to start with a smaller number of nodes before
the first submit, and ramp it up to the current 100 nodes/submit. The
second part is to reduce the dryrun overhead for all the nodes that just
need to request descriptor space.
With these changes I get around 1-2% speedup on RTX 4070 combined with my
old Haswell-era CPU.
2024-11-29 07:18:02 +01:00
678d7994f4
llava: return false instead of exit ( #10546 )
2024-11-29 01:09:46 +01:00
dc22344088
ggml : remove redundant copyright notice + update authors
2024-11-28 20:46:40 +02:00
Concedo
Ikko Eltociear Ashimine
Georgi Gerganov
Eve
Diego Devesa
Xuan Son Nguyen
Robert Collins
Shupei Fan
Alberto Cabrera Pérez
Chenguang Li
Jeff Bolz
Ting Lou