1daeed5d4d
Merge commit '9963b81f6392da8066958c177db77ad4b4a8f284' into concedo_experimental
...
# Conflicts:
# .github/workflows/server.yml
# SECURITY.md
# docs/backend/SYCL.md
# examples/model-conversion/README.md
# examples/model-conversion/scripts/embedding/compare-embeddings-logits.sh
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# tests/CMakeLists.txt
# tests/test-chat.cpp
# tests/test-json-schema-to-grammar.cpp
2025-12-17 20:30:34 +08:00
51d00f3744
updated lite
2025-12-17 19:10:11 +08:00
1e083d9c8b
integrate autofit for upstream, removed forceversion
2025-12-17 18:42:47 +08:00
9bc724f86c
rearrage some elements in launcher
2025-12-17 17:00:26 +08:00
cacfa37611
wip
2025-12-17 16:04:45 +08:00
c93c4c5505
Merge commit '4a4f7e6550cf5b327ea0fb241ce7417ab46e1ace' into concedo_experimental
...
# Conflicts:
# .github/ISSUE_TEMPLATE/011-bug-results.yml
# CODEOWNERS
# README.md
# ci/run.sh
# docs/development/HOWTO-add-model.md
# grammars/README.md
# src/llama-context.cpp
# src/llama.cpp
# tools/CMakeLists.txt
# tools/completion/README.md
# tools/llama-bench/README.md
2025-12-17 14:30:39 +08:00
050a5b1f52
Merge commit '4aced7a63156555911157d3002f9d3ddef4a1e55' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/rocm.Dockerfile
# .devops/tools.sh
# .devops/vulkan.Dockerfile
# .github/workflows/build.yml
# .github/workflows/release.yml
# .gitignore
# docs/ops.md
# docs/ops/SYCL.csv
# examples/batched/batched.cpp
# examples/eval-callback/eval-callback.cpp
# examples/gen-docs/gen-docs.cpp
# examples/lookahead/lookahead.cpp
# examples/lookup/lookup-create.cpp
# examples/lookup/lookup-stats.cpp
# examples/lookup/lookup.cpp
# examples/model-conversion/scripts/causal/compare-logits.py
# examples/model-conversion/scripts/causal/run-org-model.py
# examples/model-conversion/scripts/utils/check-nmse.py
# examples/parallel/parallel.cpp
# examples/retrieval/retrieval.cpp
# examples/save-load-state/save-load-state.cpp
# examples/speculative-simple/speculative-simple.cpp
# examples/speculative/speculative.cpp
# examples/training/finetune.cpp
# ggml/CMakeLists.txt
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-cpu/repack.cpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/dequantize.hpp
# ggml/src/ggml-sycl/dpct/helper.hpp
# ggml/src/ggml-sycl/element_wise.cpp
# ggml/src/ggml-sycl/element_wise.hpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/mmvq.cpp
# ggml/src/ggml-sycl/pad.cpp
# ggml/src/ggml-sycl/ssm_conv.cpp
# ggml/src/ggml-sycl/vecdotq.hpp
# pyrightconfig.json
# scripts/sync-ggml.last
# tests/test-arg-parser.cpp
# tests/test-backend-ops.cpp
# tools/cvector-generator/cvector-generator.cpp
# tools/imatrix/imatrix.cpp
# tools/mtmd/CMakeLists.txt
# tools/mtmd/clip.cpp
# tools/perplexity/perplexity.cpp
# tools/server/README.md
2025-12-16 23:14:12 +08:00
9963b81f63
model-conversion : add note about verifying previous models ( #18082 )
...
This commit adds a note to the README in the model-conversion
examples, advising developers to verify that previous versions of models
pass logits verification before adding new models from the same family.
2025-12-16 11:17:40 +01:00
db81d5ec4b
model-conversion : use CONVERTED_EMBEDDING_MODEL for embedding_verify_logits ( #18079 )
...
This commit updates the embedding model verification script to use the
CONVERTED_EMBEDDING_MODEL environment variable instead of using the
EMBEDDING_MODEL_PATH (the original embedding model path) as the basis
for the converted model file name.
The motivation for this that currently if the converted embedding model
file name differs from the original embedding model directory/name the
verification script will look for the wrong .bin files that were
generating when running the models.
2025-12-16 11:17:20 +01:00
c05aa69f32
common : add nemotron 3 parsing ( #18077 )
...
* common : expose json-schema functionality to extract type info
* common : fix peg parser negation during needs_more_input
* common : add some defensive measures in constructed peg parser
* common : add nemotron nano 3 support
* common : add nemotron nano 3 tests
* remove debug line
2025-12-16 04:05:23 -06:00
279cef27c2
added note for old Intel hardware pre sycl ( #18017 )
...
* added note for old Intel hardware pre sycl
Older hardware used opencl
* typo
* use consistent terms
2025-12-16 17:45:09 +08:00
5ba95754ee
security : add collaborator guidance ( #18081 )
2025-12-16 11:17:11 +02:00
e88bf41fdc
Merge commit '12280ae9058600b0b14335fe59848e601e10eebc' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# common/CMakeLists.txt
# docs/docker.md
# examples/model-conversion/scripts/causal/compare-logits.py
# ggml/src/ggml-hexagon/htp/rope-ops.c
# tests/test-backend-ops.cpp
# tests/test-barrier.cpp
# tools/server/CMakeLists.txt
# tools/server/README.md
2025-12-16 16:29:01 +08:00
78bbe89956
sd: sync to master-417-43a70e8 ( #1889 )
...
* sd: sync to master-417-43a70e8
* fix sdmain build
* switch to upstream apply_loras()
* refactor u8 path conversions and add it to the gguf reader
2025-12-16 16:16:48 +08:00
2aa45ef9e3
llama: Include algorithm header needed for C++23 ( #18078 )
2025-12-16 09:37:55 +02:00
c560316440
graph : reuse SSM graphs ( #16490 )
...
* graph : reuse hybrid graphs
* graph : reuse recurrent graphs
* graph : fix reuse check for recurrent inputs
* memory : move the recurrent state into the memory context
* Revert "memory : move the recurrent state into the memory context"
This reverts commit 00f115fe810815d4a22a6dee0acc346131e970e1.
* cont : fix build
2025-12-16 09:36:21 +02:00
d6742125c3
ci : separate webui from server ( #18072 )
...
* separate webui from server
* add public to path
2025-12-16 08:17:26 +01:00
3034836d36
webui: Improve copy to clipboard with text attachments ( #17969 )
...
* feat: Create copy/paste user message including "pasted text" attachments
* chore: update webui build output
* chore: update webui static output
* fix: UI issues
* chore: update webui static output
* fix: Decode HTML entities using `DOMParser`
* chore: update webui build output
* chore: update webui static output
2025-12-16 07:38:46 +01:00
a20979d433
webui: Add setting to always show sidebar on Desktop ( #17809 )
...
* feat: Add setting to always show Sidebar on Desktop
* chore: update webui build output
* feat: Add auto-show sidebar setting
* fix: Mobile settings dialog UI
* chore: update webui build output
* feat: UI label update
* chore: update webui build output
* chore: update webui build output
* chore: update webui build output
* refactor: Cleanup
* chore: update webui build output
2025-12-16 07:31:37 +01:00
2995341730
llama : add support for NVIDIA Nemotron 3 Nano ( #18058 )
...
* llama : add support for NVIDIA Nemotron Nano 3
This commit adds support for the NVIDIA Nemotron Nano 3 model, enabling
the conversion and running of this model.
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2025-12-16 07:19:26 +01:00
40d9c394f4
Webui: Disable attachment button and model selector button when prompt textbox is disabled. ( #17925 )
...
* Pass disabled state to the file attachments button and the model
selector button.
* Update index.html.gz
* Fix model info card in non-router mode.
* Update index.html.gz
2025-12-16 07:15:49 +01:00
d6a1e18c65
convert : move rope_parameters to TextModel class ( #18061 )
...
* make sure to search text_config for rope parameters
* move rope_parameters to TextModel class
2025-12-15 22:03:16 +01:00
c45f89d551
ggml-hexagon: mm for mtmd ( #17894 )
...
* feat: add run_mtmd script for hexagon
* fix: fix issue in fp16xfp32 mm
* fix: remove opt_experiment for fp16xfp32 mm
* fix: ggml-hexagon: matmul fp16xfp32 support non-contigious src0
* fix: fix syntax check for run-mtmd.sh for cli
2025-12-15 10:53:56 -08:00
9d52f17ae3
model : add KORMo model ( #18032 )
...
* vocab: add KORMo Tokenizer
* model: add KORMoForCausalLM
* vocab: change pretokenizer to qwen2
* lint: fix unintended line removal
* model: make qwen2 bias tensor optional
* model: use qwen2 architecture for KORMo
2025-12-15 18:51:43 +01:00
4529c660c8
kv-cache: Fix state restore fragmented cache ( #17982 )
...
* kv-cache : fix state restore with fragmented cache (#17527 )
Change find_slot to allow non-contiguous allocation during state restore. Fixes 'failed to find available cells in kv cache' error when restoring state to fragmented cache.
* tests : update logic
* cleanup: tightened state_read_meta sig, added is_contiguous case
* fix: state_read_meta arg reorder loose ends
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2025-12-15 19:28:35 +02:00
0f4f35e7be
Fix unreadable user markdown colors and truncate long texts in deletion dialogs ( #17555 )
...
* webui: limit conversation name length in dialogs
* webui: fix unreadable colors on links and table cell hover in user markdown
* webui: keep table borders visible in user markdown
* webui: updating unified exports
* Update tools/server/webui/src/lib/components/app/chat/ChatAttachments/ChatAttachmentThumbnailFile.svelte
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com >
* chore: update webui build output
* chore: update webui build output
* chore: update webui build output
---------
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com >
2025-12-15 16:34:53 +01:00
165caaf5fb
metal: use shared buffers on eGPU ( #17866 )
...
* metal: use shared buffers on eGPU
With #15906 , I noticed on important regression when using metal backend on eGPU.
This commit restore the previous behavior and add an option to force its activation.
* metal: use shared buffers on eGPU
* metal: use shared buffers on eGPU
2025-12-15 16:14:49 +02:00
96a181a933
mtmd: refactor audio preprocessing ( #17978 )
...
* mtmd: refactor audio preprocessing
* refactor
Co-authored-by: Tarek <tdakhran@users.noreply.github.com >
* wip
* wip (2)
* improve constructor
* fix use_natural_log
* fix padding for short input
* clean up
* remove need_chunking
---------
Co-authored-by: Tarek <tdakhran@users.noreply.github.com >
2025-12-15 14:16:52 +01:00
4a4f7e6550
cli: fixed dead links to tools/main for cli and completion, fixed code owners ( #17993 )
...
Co-authored-by: Andrew Aladjev <andrew.aladjev@gmail.com >
2025-12-15 11:47:04 +01:00
e73d548659
webui: add "delete all conversations" button to import/export tab ( #17444 )
...
* webui: add "delete all conversations" button to import/export tab
- Add 'Delete all conversations' functionality with confirmation dialog
- Add Trash icon and destructive styling for clear visual indication
- Redirects to "?new_chat=true#/" by using conversationsStore.deleteAll()
* chore: update webui build output
2025-12-15 11:29:29 +01:00
b1f3a6e5db
llama: automatically set parameters not set by the user in such a way that maximizes GPU utilization ( #16653 )
...
* llama: automatically fit args to free memory
llama-fit-params tool
* fix CI
* hints for bug reports, ensure no reallocation
* fix segfault with Vulkan
* add llama-fit-params to CI
* fix CI
* fix CI
* fix CI
* minor adjustments
* fix assignment of 1 dense layer
* fix logger not being reset on model load failure
* remove --n-gpu-layer hint on model load failure
* fix llama-fit-params verbosity
* fix edge case
* fix typo [no ci]
2025-12-15 09:24:59 +01:00
4aced7a631
[SYCL] Support gpt-oss by OPs add-id, mul_mat for mxfp4, swiglu_oai ( #17826 )
...
* support gpt-oss GPU by OP add-id, mul_mat for mxfp4, swiglu_oai, fix warning
* fix fault ut case, update ops.md
* rebase, fix format issue
2025-12-15 10:35:15 +08:00
745fa0e78b
model : add glm-asr support ( #17901 )
...
* [model] add glm-asr support
* fix format for ci
* fix convert format for ci
* update glm_asr convert script & use build_ffn for glm_asr clip & use build_stack for padding and review
* check root architecture for convert hf script
* fix conficlt with upstream
* fix convert script for glm asr & format clip-impl
* format
* restore hparams text
* improved conversion
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2025-12-15 03:18:46 +01:00
52392291b2
preset: handle negated arg, reverse the meaning if needed ( #18041 )
2025-12-14 22:08:10 +01:00
5c8a717128
convert : refactor rope scaling handling ( #18013 )
...
* refactor rope scaling handling
* ws--
* missed a couple
* use find_hparam
2025-12-14 16:04:37 +01:00
37f5a1093b
mtmd: enhance image resizing in llava_uhd ( #18014 )
2025-12-14 15:57:52 +01:00
bca0258c2a
bump default gen amount by 128 to 896
2025-12-14 22:17:31 +08:00
9e6649ecf2
vulkan: fix mul_mat_vec_iq1_s formatting ( #18026 )
2025-12-14 14:52:46 +01:00
0759b09c90
graph: add f_attn_temp_offset ( #18025 )
2025-12-14 13:05:59 +01:00
254098a279
common : refactor common_sampler + grammar logic changes ( #17937 )
...
* common : refactor common_sampler + grammar logic changes
* tests : increase max_tokens to get needed response
* batched : fix uninitialized samplers
2025-12-14 10:11:13 +02:00
3238b1400c
vulkan: Fix data race/hang in scalar/cm1 flash attention ( #17887 )
2025-12-14 09:00:00 +01:00
4722671641
vulkan: improve mul_mat_vec_iq1_s speed ( #17874 )
2025-12-14 08:47:49 +01:00
d15d177f43
vulkan: faster q6_k matmul ( #17813 )
...
* q6_k faster mul mat
* 8 values
* fix comment
* switch to two at a time
* start ci for .glsl files
2025-12-14 08:29:37 +01:00
77ad8542bd
model-conversion : cast logits to float32 ( #18009 )
2025-12-14 08:58:13 +02:00
609a2d0268
models : fix YaRN regression + consolidate logic ( #18006 )
...
* models : fix YaRN regression + consolidate logic
* cont : fix the fix
* cont : remove header
* cont : add header
2025-12-14 08:34:56 +02:00
a63cbafbbc
ggml : arm repack fix build
2025-12-14 08:33:51 +02:00
0e59224990
sync : ggml
2025-12-14 08:33:51 +02:00
71fdcf0616
ggml : arm repack fix build (whisper/0)
2025-12-14 08:33:51 +02:00
615655aafe
cmake : set CMAKE_RUNTIME_OUTPUT_DIRECTORY for non standalone build (ggml/1394)
...
Some backend depends on CMAKE_RUNTIME_OUTPUT_DIRECTORY to create temporary file like metal backened.
Missing CMAKE_RUNTIME_OUTPUT_DIRECTORY will cause some cmake error like permission denied (try to copy file to root).
This PR wants to setup a default path for CMAKE_RUNTIME_OUTPUT_DIRECTORY when it does not exist.
2025-12-14 08:33:51 +02:00
c00ff929dc
scripts: add script to compare logprobs of llama.cpp against other frameworks ( #17947 )
...
* scripts: add script to compare logits of llama.cpp against other frameworks
* accept custom prompt file
* fix code style
* clarify endpoint
* fix displaying
* use abs for diff
* fix vllm case
* rm output file
* rename to compare-logprobs
* add "pattern"
2025-12-13 22:33:29 +01:00
Concedo
Daniel Bevenius
Aldehir Rojas
Francisco Herrera
Georgi Gerganov
Wagner Bruna
Chris Peterson
Sigbjørn Skjæret
Aleksander Grygier
Darius Lukas
Shouyu
HelloKS
ssweens
Pascal
Jeremy Demeule
Xuan-Son Nguyen
Andrew Aladjev
Thomas Jarosch
Johannes Gäßler
Neo Zhang Jianyu
piDack
Haowei Wu
Ruben Ortlam
Jeff Bolz
lovedheart
Eve
Congcong Cai