8f1edcbdac
Merge commit 'dc39a5e7a84815a90fa0c515ed8927870cf858c9' into concedo_experimental
...
# Conflicts:
# README.md
# SECURITY.md
# docs/multimodal/MobileVLM.md
# examples/llava/CMakeLists.txt
# examples/llava/README.md
# examples/llava/android/adb_run.sh
# ggml/CMakeLists.txt
# ggml/src/CMakeLists.txt
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/rope.cpp
# ggml/src/ggml-sycl/rope.hpp
2025-04-24 11:49:08 +08:00
dc39a5e7a8
mtmd : support SmolVLM (version 1 and 2) ( #13050 )
...
* mtmd : support SmolVLM (version 1 and 2)
* correct chat template
* fix n_patches
* scale_factor is an int
* add more models to test
2025-04-22 16:24:54 +02:00
3e8b84b8e5
added support for structured output in chat completions
2025-04-22 22:23:36 +08:00
ab47dec3d3
security : add note about RPC and server functionality ( #13061 )
...
* security : add note about RPC functionality
* security : add note about llama-server
2025-04-22 16:16:10 +03:00
7b53389c24
metal : add memory pool for temp allocs ( #12850 )
...
* metal : add memory pool for temp allocs (wip) [no ci]
* cont : free buffers from the heap
* cont : resize heap [no ci]
* cont : refactor heap [no ci]
* cont : heap for each cmd buffer [no ci]
* cont : fix free
* wip
* cont : fix alignment [no ci]
* cont : not working .. [no ci]
* cont : heap allocation now works [no ci]
* cont : use MTLHeapTypePlacement
ggml-ci
* metal : use dynamic MTLHeap allocations
ggml-ci
* metal : add comments
* metal : disable softmax use of mem_pool
ggml-ci
* metal : final touches
2025-04-22 16:15:51 +03:00
243453533e
llava : update documentations ( #13055 )
...
* llava : update documentations
* fix typo
2025-04-22 10:37:00 +02:00
e8b3aeaa28
update some defaults for max length and max ctx
2025-04-22 15:47:01 +08:00
6dbee2f2f8
more robust glslc checks, increase default denoise str
2025-04-22 15:19:47 +08:00
1d735c0b4f
ggml : add SSE 4.2 and x64 base variant for CPUs without AVX ( #12871 )
...
* ggml : add SSE 4.2 variant for CPUs without AVX
* ggml : add x64 base ABI variant
2025-04-21 18:13:51 +02:00
16156f0d86
updated lite
2025-04-21 22:24:33 +08:00
6494dce405
handle estimation for multipart gguf (+1 squashed commits)
...
Squashed commits:
[c7b4af92] handle estimation for multipart gguf
2025-04-21 22:07:22 +08:00
5368ddda7a
SYCL: Add non-contiguous support in ROPE ( #12993 )
...
ggml-ci
2025-04-21 19:13:30 +05:30
84a9bf2fc2
mtmd : merge llava, gemma3 and minicpmv CLI into single llama-mtmd-cli ( #13012 )
...
* mtmd : merge `llava-cli` and `gemma3-cli` into single `mtmd-cli`
* support for minicpmv
* remove cpp files of llava and minicpmv
* update hot topics
* mtmd : add not supported msg for qwen2vl
* Update examples/llava/mtmd.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2025-04-21 15:32:58 +02:00
9cd6a1add2
allow mmproj to be run on cpu
2025-04-21 21:03:10 +08:00
f968079290
randomize image names to prevent caching in noscript
2025-04-21 13:24:40 +08:00
2016f07bd1
convert : experimental support for --mmproj flag ( #13023 )
...
* convert : experimental support for `--mmproj` flag
* fix bad ctrl+f replace
* fix style
* split into subclasses TextModel and VisionModel
* rename Mode --> ModelBase
* small fix
* correct CLIP_VISION arch name (because existing GGUF already use it)
* Apply suggestions from code review
Co-authored-by: compilade <git@compilade.net >
* fix Mistral3Model
* fix typo
Co-authored-by: compilade <git@compilade.net >
---------
Co-authored-by: compilade <git@compilade.net >
2025-04-20 23:29:36 +02:00
687bb5375c
Merge branch 'upstream' into concedo_experimental
2025-04-20 20:57:11 +08:00
2ed6850c0b
added override tensor
2025-04-20 20:56:17 +08:00
6602304814
llava: fix errors in clip.h on certain compilers ( #13030 )
2025-04-20 12:15:41 +02:00
17360a3b32
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# examples/llava/clip.cpp
2025-04-20 17:59:58 +08:00
636b92ec1d
updated lite
2025-04-20 17:52:55 +08:00
66168204be
vulkan: support noncontiguous rms_norm ( #13031 )
2025-04-20 10:50:02 +02:00
4ba9d711ba
metal: add neg operator ( #13029 )
2025-04-20 08:28:40 +03:00
00137157fc
Disable CI cross-compile builds ( #13022 )
2025-04-19 18:05:03 +02:00
75dfad2bb0
fixed noscript (+1 squashed commits)
...
Squashed commits:
[dba28399] fixed noscript
2025-04-19 23:16:08 +08:00
fb28f4f80e
gguf-py : fix upload python package workflow ( #13020 )
2025-04-19 16:26:38 +02:00
12c2efdadd
noscript image gen
2025-04-19 18:56:52 +08:00
37b9f0d29d
clip : refactor, add image_manipulation and llava_uhd classes ( #13011 )
...
* clip : refactor, add `image_manipulation` and `llava_uhd`
* refactor llava-1.6 preprocessing
* simplify logic for llava-1.5
* missing include
2025-04-19 09:15:45 +02:00
95d1aaf4d4
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# examples/rpc/rpc-server.cpp
# ggml/src/ggml-rpc/ggml-rpc.cpp
# ggml/src/ggml-sycl/backend.hpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/element_wise.cpp
# ggml/src/ggml-sycl/element_wise.hpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# requirements/requirements-all.txt
2025-04-19 13:17:13 +08:00
305e533dc6
i already knew zenity would cause issues
2025-04-19 13:04:41 +08:00
78a910be26
noscript chat mode tweaks
2025-04-19 12:40:13 +08:00
6408210082
main : Fix Ctrl+D/newline handling ( #12951 )
...
This restores the behavior from #491 . This does not affect Ctrl+D's ability to
terminate --multiline-input lines (#1040 ).
This also actually implements #587 : "If the user wants the text to end in a
newline, this should be accomplished by explicitly adding a newline by using
\ followed by return, then returning control by pressing return again."
Fixes #12949
2025-04-18 22:02:55 +02:00
aff9d107b0
gguf-py : GGUF Editor GUI - Python + Qt6 ( #12930 )
2025-04-18 20:30:41 +02:00
35370ba945
server : use std::move whenever possible ( #12936 )
...
* server : use std::move whenever possible
* use r-value ref
* Apply suggestions from code review
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* make task creation scoped
* restore std::move
* fix task_id not set correctly
* apply changes from suggestion
Co-authored-by: ggerganov <ggerganov@gmail.com >
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2025-04-18 19:58:12 +02:00
a5b5d21cca
added chat mode to noscript
2025-04-19 00:59:00 +08:00
4b0f63ed62
cleanup
2025-04-18 22:57:10 +08:00
29b57d2175
updated vulkan to make use of cm2
2025-04-18 22:10:57 +08:00
8d66005763
SYCL: Refactor and enable FP16 in binary broadcast OPs ( #12975 )
...
* SYCL: refactor move to a separate file
* Fix binbcast
* Remove duplicates
* fix include formatting
* fix typo
2025-04-18 15:57:56 +02:00
b9154ecff9
mtmd : add methods to access mtmd_image_tokens ( #12906 )
...
* mtmd : add more api around mtmd_image_tokens
* mtmd : ability to calc image hash
* shared_ptr for mtmd_image_tokens
* move hash to user-define ID (fixed)
* fix prompt_modified
* rm redundant data member
2025-04-18 10:04:51 +02:00
2db9ba1464
rpc : add RPC_CMD_HELLO ( #12955 )
...
Add RPC_CMD_HELLO for getting the version of the protocol implemend by
the server. Follow the semantic versioning rules at https://semver.org
Hopefully this bring better user experience when we make breaking
changes at the protocol level and avoid issues like #12465
2025-04-18 10:13:42 +03:00
40adb8af35
update lite, remove aetherroom dead site
2025-04-18 13:23:14 +08:00
5d57d62665
add a timeout for zenity check
2025-04-18 13:07:26 +08:00
bce519cee7
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# ggml/src/ggml-cann/aclnn_ops.cpp
# ggml/src/ggml-cann/aclnn_ops.h
# ggml/src/ggml-cann/common.h
# ggml/src/ggml-cann/ggml-cann.cpp
# tests/test-backend-ops.cpp
2025-04-18 12:44:20 +08:00
1a09d9cf0e
increase to 10 save slots
2025-04-18 11:30:32 +08:00
2f74c354c0
graph : make FA compatible with MLA + add initial Metal kernels ( #12953 )
...
* graph : make mla compatible with FA
* metal : add exp FA kernels for DeepSeek models
ggml-ci
* llama : minor naming updates
ggml-ci
* ggml : disable FA for DS head sizes
* tests : add FA tests for MLA shapes
ggml-ci
2025-04-17 18:16:36 +03:00
64dd4f932f
update readme
2025-04-17 23:06:07 +08:00
207c22ec2d
ggml: Re-enable CUDA graphs in presence of CONT and DUP nodes ( #12970 )
2025-04-17 15:19:42 +02:00
7a395f67a7
CANN: Add support for async operator submission ( #12864 )
...
Submit operators using asynchronous threads to improve performance.
Use the environment variable GGML_CANN_ASYNC_MODE to control whether
asynchronous submission is enabled. It is disabled by default.
Testing shows a 10%–20% performance improvement in scenarios with
small parameter sizes, especially in quantized models.
2025-04-17 20:34:16 +08:00
971f245b3b
llama : recognize IBM Granite 3.3 FIM tokens ( #12988 )
...
The Granite's FIM tokens are very similar to Qwen's; it's just that
they use underscore instead of a dash. So <fim_middle> for example
instead of <fim-middle>.
Opening up tokenizer_config.json in ibm-granite/granite-3.3-8b-base
shows:
```
"<fim_prefix>",
"<fim_middle>",
"<fim_suffix>",
"<fim_pad>",
...
"<reponame>",
```
2025-04-17 11:37:05 +03:00
c67510718e
kv override option (+1 squashed commits)
...
Squashed commits:
[e615fc01] kv override option
2025-04-17 14:22:30 +08:00
Concedo
Xuan-Son Nguyen
Georgi Gerganov
Diego Devesa
Akarshan Biswas
Jeffrey Morgan
Jeff Bolz
bandoti
Sigbjørn Skjæret
Daniel Tang
Chris Thompson
Radoslav Gerganov
Alan Gray
hipudding
Mikko Juola