ea9bd61e47
Merge commit '64eda5deb9859e87a020e56bab5d2f9ca956f1de' into concedo_experimental
...
# Conflicts:
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/llama-cli-cann.Dockerfile
# .devops/musa.Dockerfile
# .devops/rocm.Dockerfile
# .devops/vulkan.Dockerfile
# .github/workflows/build.yml
# .github/workflows/docker.yml
# README.md
# docs/backend/SYCL.md
# examples/llava/clip.cpp
# examples/server_embd.py
# ggml/src/ggml-cann/acl_tensor.cpp
# ggml/src/ggml-cann/aclnn_ops.cpp
# ggml/src/ggml-cann/aclnn_ops.h
# ggml/src/ggml-cann/ggml-cann.cpp
# src/CMakeLists.txt
# tests/test-chat-template.cpp
2025-04-12 08:31:22 +08:00
381603a775
ci: detach common from the library ( #12827 )
...
* fix: detach common from the library
* fix: building chat test template
2025-04-09 10:11:11 +02:00
65a69e6e1b
clip : do not print ftype ( #12832 )
2025-04-09 10:09:53 +02:00
b32efad2bc
llava: improve clip_ctx destructor to not memleak load_image_size ( #12834 )
2025-04-08 22:01:58 +02:00
a19b5cef16
llama : fix FA when KV cache is not used (i.e. embeddings) ( #12825 )
...
* ggml : FA supports F32 V
* graph : cast KV to F16 when the KV cache is not used
ggml-ci
* server : add test that exercises embeddings with FA enabled
ggml-ci
2025-04-08 19:54:51 +03:00
78a1ba0a4f
server : fix thread.join() on exit ( #12831 )
2025-04-08 18:37:06 +02:00
2dabf759e7
llava: add more helper functions to check projector types in clip context ( #12824 )
...
Signed-off-by: dm4 <sunrisedm4@gmail.com >
2025-04-08 15:49:13 +02:00
ebf924c5d1
Merge branch 'upstream' into concedo_experimental
2025-04-08 21:46:30 +08:00
88660dd59d
merged qwen2.5vl again
2025-04-08 21:32:25 +08:00
822cf2430e
Merge commit 'f1e3eb4249db68d97be352c0adf16eef7ae53795' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# README.md
# docs/backend/SYCL.md
# examples/llava/clip.cpp
# ggml/src/ggml-sycl/CMakeLists.txt
# ggml/src/ggml-vulkan/cmake/host-toolchain.cmake.in
2025-04-08 20:48:53 +08:00
c58e9a2be3
revert q2.5vl before merge (+1 squashed commits)
...
Squashed commits:
[3197ea95] Revert "add tentative support for qwen2.5vl vision from HimariO fork"
This reverts commit 911669087a
2025-04-08 20:38:41 +08:00
8ca6e1c3a4
server : webui : Improve Chat Input with Auto-Sizing Textarea ( #12785 )
...
* Update ChatScreen.tsx
* useAutosizeTextarea.ts
useAutosizeTextarea to encapsulate the logic.
* Implement responsive auto-sizing chat textarea
Replaces the manual textarea resizing with an automatic height adjustment based on content.
- `useChatTextarea` hook to manage textarea state and auto-sizing logic via refs, preserving the optimization
- Textarea now grows vertically up to a maximum height (`lg:max-h-48`) on large screens (lg breakpoint and up).
- Disables auto-sizing and enables manual vertical resizing (`resize-vertical`) on smaller screens for better mobile usability.
- Aligns the "Send" button to the bottom of the textarea (`items-end`) for consistent positioning during resize.
* -update compressed index.html.gz after npm run build
-refactor: replace OptimizedTextareaValue with AutosizeTextareaApi in VSCode context hook
* chore: normalize line endings to LF
refactor: AutosizeTextareaApi -> chatTextareaApi
* refactor: Rename interface to PascalCase
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co >
2025-04-08 11:14:59 +02:00
4ccea213bc
hellaswag: display estimated score confidence interval ( #12797 )
2025-04-07 18:47:08 +03:00
b28ad7ecca
fix attn weight scaling after rebase
2025-04-07 22:07:56 +08:00
223edef897
remove commented-out code blocks
2025-04-07 21:52:37 +08:00
dde96b4774
remove not so often use qwen2vl-cli debug functions
2025-04-07 21:52:37 +08:00
8fcf682b28
ignore transformers Qwen2_5_xxx type check
2025-04-07 21:52:37 +08:00
fdae70a832
cleaning up
2025-04-07 21:52:37 +08:00
c891300c1e
move position id remap out of ggml to avoid int32 cuda operations
2025-04-07 21:52:37 +08:00
e18f6a3238
fix few incorrect tensor memory layout
2025-04-07 21:52:37 +08:00
ecd673f0c5
add debug utils
2025-04-07 21:51:18 +08:00
9c827814e6
handle window attention inputs
2025-04-07 21:51:18 +08:00
9c7cc6de9c
implment vision model architecture, gguf convertor
2025-04-07 21:46:06 +08:00
a3f7de7142
fixed outetts docs
2025-04-07 21:31:43 +08:00
bd3f59f812
cmake : enable curl by default ( #12761 )
...
* cmake : enable curl by default
* no curl if no examples
* fix build
* fix build-linux-cross
* add windows-setup-curl
* fix
* shell
* fix path
* fix windows-latest-cmake*
* run: include_directories
* LLAMA_RUN_EXTRA_LIBS
* sycl: no llama_curl
* no test-arg-parser on windows
* clarification
* try riscv64 / arm64
* windows: include libcurl inside release binary
* add msg
* fix mac / ios / android build
* will this fix xcode?
* try clearing the cache
* add bunch of licenses
* revert clear cache
* fix xcode
* fix xcode (2)
* fix typo
2025-04-07 13:35:19 +02:00
5edbacdd0e
fix tools (+3 squashed commit)
...
Squashed commit:
[95a489ee] fix tools build
[1d3d3451] add accelerate
[2837705c
2025-04-06 21:30:48 +08:00
f1e3eb4249
common : fix includes in arg.cpp and gemma3-cli.cpp ( #12766 )
...
* arg.cpp: add a missing include
* gemma3-cli.cpp: fix cinttypes include
2025-04-05 17:46:00 +02:00
0364178ca2
clip : refactor clip_init, add tests ( #12757 )
...
* refactor clip_init
* fix loading file
* fix style
* test ok
* better test with report
* add missing headers
* clarify
* add KEY_MM_PATCH_MERGE_TYPE
* remove bool has_* pattern
* Apply suggestions from code review
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* Update examples/llava/clip.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* use ggml_soft_max_ext
* refactor logging system
* add minicpm-v-o 2.6 for testing
* use nullptr everywhere
* fix Yi-VL model
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2025-04-05 17:17:40 +02:00
b772394297
server : webui : Upgrade daisyui, tailwindcss. ( #12735 )
...
* Upgrade daisyui, tailwindcss.
* Switch to all themes.
* Revert a change.
* Update formatting.
* Install packages before npm build.
* Revert "Install packages before npm build."
This reverts commit 336c5147e614e60993162794ba9d9d4629a916f8.
* Add index.html.gz
* run build
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co >
2025-04-04 16:09:52 +02:00
23106f94ea
gguf-split : --merge now respects --dry-run option ( #12681 )
...
* gguf-split now respects dry-run option
* removing trailing space
2025-04-04 16:09:12 +02:00
103d60ed2c
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# common/common.cpp
# examples/batched-bench/batched-bench.cpp
# examples/batched/batched.cpp
# examples/export-lora/export-lora.cpp
# examples/gritlm/gritlm.cpp
# examples/parallel/parallel.cpp
# examples/passkey/passkey.cpp
# examples/speculative-simple/speculative-simple.cpp
# examples/speculative/speculative.cpp
# ggml/src/ggml-cann/CMakeLists.txt
# ggml/src/ggml-cann/acl_tensor.cpp
# ggml/src/ggml-cann/acl_tensor.h
# ggml/src/ggml-cann/aclnn_ops.cpp
# ggml/src/ggml-cann/aclnn_ops.h
# ggml/src/ggml-vulkan/CMakeLists.txt
# tests/test-arg-parser.cpp
# tests/test-backend-ops.cpp
2025-04-03 18:57:49 +08:00
a10b36c91a
llama : refactor kv cache guard ( #12695 )
...
* llama : refactor kv cache guard
ggml-ci
* cont : fix comment [no ci]
* llama : fix kv_cache restore logic
ggml-ci
* context : simplify kv cache updates
ggml-ci
* cont : better name [no ci]
* llama : fix llama_decode return code when could not find KV slot
ggml-ci
* context : change log err -> warn [no ci]
* kv-cache : add comment + warning
2025-04-02 14:32:59 +03:00
42eb248f46
common : remove json.hpp from common.cpp ( #12697 )
...
* common : remove json.hpp from common.cpp
* fix comment
2025-04-02 09:58:34 +02:00
267c1399f1
common : refactor downloading system, handle mmproj with -hf option ( #12694 )
...
* (wip) refactor downloading system [no ci]
* fix all examples
* fix mmproj with -hf
* gemma3: update readme
* only handle mmproj in llava example
* fix multi-shard download
* windows: fix problem with std::min and std::max
* fix 2
2025-04-01 23:44:05 +02:00
9e182b3e78
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# README.md
# docs/backend/SYCL.md
# ggml/src/ggml-sycl/CMakeLists.txt
# ggml/src/ggml-vulkan/CMakeLists.txt
# ggml/src/ggml-vulkan/ggml-vulkan.cpp
# scripts/sync-ggml.last
# tests/test-chat-template.cpp
2025-04-01 20:16:07 +08:00
1a85949067
llava : proper description fix ( #12668 )
2025-03-31 11:28:30 +02:00
f52d59d771
llava : fix clip loading GGUFs with missing description ( #12660 )
2025-03-31 11:07:07 +02:00
1ebadc515e
add streaming support for oai tools (+2 squashed commit)
...
Squashed commit:
[4d080b37] qwen2.5vl surgery script
[4bebe7e5] add streaming support for oai tools
2025-03-31 16:49:15 +08:00
52de2e5949
tts : remove printfs ( #12640 )
...
* tts.cpp : llama tokens console output is done using LOG_INF instead of printf(). Therefore the options '--log-disable' and '--log-file' have now uniform impact on all output.
2025-03-31 11:20:30 +03:00
911669087a
add tentative support for qwen2.5vl vision from HimariO fork
2025-03-29 22:52:43 +08:00
396875e1c4
update api docs and lite
2025-03-29 15:39:25 +08:00
5d01670266
server : include speculative decoding stats when timings_per_token is enabled ( #12603 )
...
* Include speculative decoding stats when timings_per_token is true
New fields added to the `timings` object:
- draft_n : number of draft tokens generated
- draft_accepted_n : number of draft tokens accepted
- draft_accept_ratio: ratio of accepted/generated
* Remove redundant draft_accept_ratio var
* add draft acceptance rate to server console output
2025-03-28 10:05:44 +02:00
ef03229ff4
rpc : update README for cache usage ( #12620 )
2025-03-28 09:44:13 +02:00
ab6ab8f809
rpc : send hash when tensor data is above some fixed threshold ( #12496 )
...
* rpc : send hash when tensor data is above some fixed threshold
ref #10095
* rpc : put cache under $HOME/.cache/llama.cpp
* try to fix win32 build
* another try to fix win32 build
* remove llama as dependency
2025-03-28 08:18:04 +02:00
2099a9d5db
server : Support listening on a unix socket ( #12613 )
...
* server : Bump cpp-httplib to include AF_UNIX windows support
Signed-off-by: Piotr Stankiewicz <piotr.stankiewicz@docker.com >
* server : Allow running the server example on a unix socket
Signed-off-by: Piotr Stankiewicz <piotr.stankiewicz@docker.com >
---------
Signed-off-by: Piotr Stankiewicz <piotr.stankiewicz@docker.com >
2025-03-27 23:41:04 +01:00
02082f1519
clip: Fix llama-llava-clip-quantize-cli quantization error under CUDA backend ( #12566 )
...
* [Fix] Compiling clip-quantize-cli and running it in a CUDA environment will cause ggml_fp16_to_fp32 to report an error when trying to access video memory. You need to switch to the CPU backend to run quantize.
After the fix, it will automatically run in the CPU backend and will no longer be bound to CUDA.
* [Fix]Roll back the signature and implementation of clip_model_load, and change the call in clip_model_quantize to clip_init.
2025-03-26 15:06:04 +01:00
ef19c71769
run: de-duplicate fmt and format functions and optimize ( #11596 )
2025-03-25 18:46:11 +01:00
ea358369cc
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# ci/README.md
# ci/run.sh
# docs/backend/CUDA-FEDORA.md
# docs/build.md
# docs/install.md
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-cuda/common.cuh
# tests/test-backend-ops.cpp
2025-03-26 00:18:01 +08:00
77f9c6bbe5
server : Add verbose output to OAI compatible chat endpoint. ( #12246 )
...
Add verbose output to server_task_result_cmpl_final::to_json_oaicompat_chat_stream, making it conform with server_task_result_cmpl_final::to_json_oaicompat_chat, as well as the other to_json methods.
2025-03-23 19:30:26 +01:00
7030ebf401
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# docs/backend/SYCL.md
# ggml/src/CMakeLists.txt
# ggml/src/ggml-cpu/ggml-cpu-aarch64.cpp
# ggml/src/ggml-sycl/CMakeLists.txt
# tests/test-backend-ops.cpp
2025-03-22 00:32:42 +08:00
Concedo
Plamen Minev
Xuan-Son Nguyen
Matt Clayton
Georgi Gerganov
dm4
characharm
stduhpf
HimariO
Sergey Fedorov
Nauful Shaikh
nick huang
Sigbjørn Skjæret
marcoStocchi
Benson Wong
Radoslav Gerganov
Piotr
Ivy233
Eric Curtin
Marius Gerdes