7304640f72
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# .github/workflows/release.yml
# docs/android.md
# docs/backend/hexagon/CMakeUserPresets.json
# examples/llama.android/app/src/main/res/layout/activity_main.xml
# examples/llama.android/app/src/main/res/layout/item_message_assistant.xml
# examples/llama.android/app/src/main/res/layout/item_message_user.xml
# examples/model-conversion/scripts/causal/run-org-model.py
# examples/model-conversion/scripts/utils/common.py
# ggml/CMakeLists.txt
# ggml/src/ggml-hexagon/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# tests/test-arg-parser.cpp
# tools/server/README.md
2025-12-20 09:32:06 +08:00
714ab0682e
Revert "Revert "llama : Async DirectIO model loading on Linux ( #18012 )""
...
This reverts commit a45fc5ee88
2025-12-20 09:25:10 +08:00
f99ef53d2a
llama : Changing off_t to size_t for Windows ( #18204 )
2025-12-19 16:42:46 +02:00
a45fc5ee88
Revert "llama : Async DirectIO model loading on Linux ( #18012 )"
...
This reverts commit 4d4f4cacd1
2025-12-19 19:06:30 +08:00
58eb5573de
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/act-ops.c
# ggml/src/ggml-hexagon/htp/hvx-utils.c
# ggml/src/ggml-hexagon/htp/main.c
# src/llama-model.cpp
# tools/server/README.md
2025-12-19 11:00:43 +08:00
e005fc2587
Merge commit '8dcc3662a292c14f003be2c465895d40c9460511' into concedo_experimental
...
Keep changes from https://github.com/ggml-org/llama.cpp/pull/18096 without https://github.com/ggml-org/llama.cpp/pull/14904
Reason is to maintain compatibility with 2023 w64devkit
# Conflicts:
# .github/ISSUE_TEMPLATE/019-bug-misc.yml
# examples/model-conversion/scripts/causal/run-org-model.py
# examples/speculative/speculative.cpp
# ggml/src/ggml-cpu/arch-fallback.h
# ggml/src/ggml-cpu/repack.cpp
# ggml/src/ggml-cpu/repack.h
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/act-ops.c
# ggml/src/ggml-hexagon/htp/htp-msg.h
# ggml/src/ggml-hexagon/htp/hvx-utils.c
# ggml/src/ggml-hexagon/htp/hvx-utils.h
# ggml/src/ggml-hexagon/htp/main.c
2025-12-19 02:11:55 +08:00
57c1e05643
llama: offload output layer to GPU first ( #18148 )
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Successful in 1m58s
Python check requirements.txt / check-requirements (push) Failing after 10s
Python Type-Check / pyright type-check (push) Failing after 1m25s
2025-12-18 08:12:18 +01:00
4d4f4cacd1
llama : Async DirectIO model loading on Linux ( #18012 )
...
* Uncached model read
* Removing additional --mmap arg
* Removing trailing whitespaces
* Adding fallback when O_DIRECT is not supported
* Remove branching in llama-model-loader.cpp and reduce code duplications in llama-mmap.cpp
* Adding maybe unused keyword for Mac and Windows.
* File seek aligned
* Removing all branches for direct_io in llama-model-loader.cpp
* Always use alignment from llama_file
* use_mmap=true
2025-12-18 08:27:19 +02:00
8dcc3662a2
llama-fit-params: fix memory print ( #18136 )
2025-12-17 21:10:03 +01:00
4301e27319
common : restore grammar-based rejection sampling ( #18137 )
...
* common : restart grammar-based rejection sampling
* sampling : allow null samplers
2025-12-17 19:46:00 +02:00
1f2c9f6b62
gpt4v not working correctly
2025-12-17 21:02:16 +08:00
1daeed5d4d
Merge commit '9963b81f6392da8066958c177db77ad4b4a8f284' into concedo_experimental
...
# Conflicts:
# .github/workflows/server.yml
# SECURITY.md
# docs/backend/SYCL.md
# examples/model-conversion/README.md
# examples/model-conversion/scripts/embedding/compare-embeddings-logits.sh
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# tests/CMakeLists.txt
# tests/test-chat.cpp
# tests/test-json-schema-to-grammar.cpp
2025-12-17 20:30:34 +08:00
982060fadc
model: fix LFM2_MOE missing tensors ( #18132 )
2025-12-17 12:17:11 +01:00
c93c4c5505
Merge commit '4a4f7e6550cf5b327ea0fb241ce7417ab46e1ace' into concedo_experimental
...
# Conflicts:
# .github/ISSUE_TEMPLATE/011-bug-results.yml
# CODEOWNERS
# README.md
# ci/run.sh
# docs/development/HOWTO-add-model.md
# grammars/README.md
# src/llama-context.cpp
# src/llama.cpp
# tools/CMakeLists.txt
# tools/completion/README.md
# tools/llama-bench/README.md
2025-12-17 14:30:39 +08:00
d0794e89d9
llama-fit-params: force disable mlock ( #18103 )
2025-12-17 00:50:12 +01:00
9dcac6cf9f
llama-fit-params: lower ctx size for multi GPU ( #18101 )
2025-12-17 00:49:34 +01:00
0e49a7b8b4
llama-fit-params: fix underflow for dense models ( #18095 )
2025-12-17 00:47:37 +01:00
ef83fb8601
model: fix LFM2 missing tensors ( #18105 )
2025-12-16 19:07:43 +01:00
050a5b1f52
Merge commit '4aced7a63156555911157d3002f9d3ddef4a1e55' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/rocm.Dockerfile
# .devops/tools.sh
# .devops/vulkan.Dockerfile
# .github/workflows/build.yml
# .github/workflows/release.yml
# .gitignore
# docs/ops.md
# docs/ops/SYCL.csv
# examples/batched/batched.cpp
# examples/eval-callback/eval-callback.cpp
# examples/gen-docs/gen-docs.cpp
# examples/lookahead/lookahead.cpp
# examples/lookup/lookup-create.cpp
# examples/lookup/lookup-stats.cpp
# examples/lookup/lookup.cpp
# examples/model-conversion/scripts/causal/compare-logits.py
# examples/model-conversion/scripts/causal/run-org-model.py
# examples/model-conversion/scripts/utils/check-nmse.py
# examples/parallel/parallel.cpp
# examples/retrieval/retrieval.cpp
# examples/save-load-state/save-load-state.cpp
# examples/speculative-simple/speculative-simple.cpp
# examples/speculative/speculative.cpp
# examples/training/finetune.cpp
# ggml/CMakeLists.txt
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-cpu/repack.cpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/dequantize.hpp
# ggml/src/ggml-sycl/dpct/helper.hpp
# ggml/src/ggml-sycl/element_wise.cpp
# ggml/src/ggml-sycl/element_wise.hpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/mmvq.cpp
# ggml/src/ggml-sycl/pad.cpp
# ggml/src/ggml-sycl/ssm_conv.cpp
# ggml/src/ggml-sycl/vecdotq.hpp
# pyrightconfig.json
# scripts/sync-ggml.last
# tests/test-arg-parser.cpp
# tests/test-backend-ops.cpp
# tools/cvector-generator/cvector-generator.cpp
# tools/imatrix/imatrix.cpp
# tools/mtmd/CMakeLists.txt
# tools/mtmd/clip.cpp
# tools/perplexity/perplexity.cpp
# tools/server/README.md
2025-12-16 23:14:12 +08:00
ec98e20021
llama: fix early stop in params_fit if ctx is set ( #18070 )
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Successful in 1m28s
Python check requirements.txt / check-requirements (push) Failing after 7s
Python Type-Check / pyright type-check (push) Failing after 39s
2025-12-16 14:24:00 +01:00
7f2b2f3c77
arch: refactor LLM_TENSOR_NAMES ( #18051 )
...
* arch: refactor LLM_TENSOR_NAMES
* update docs
* typo
* fix LLM_ARCH_NEMOTRON_H_MOE
* show more meaningful error message on missing tensor
* fix and tested LLM_ARCH_NEMOTRON_H_MOE
2025-12-16 13:22:30 +01:00
a5251ca11d
Optimization: Qwen3 next autoregressive pass ( #17996 )
...
* It's Qwen3 Next, the lean mean token generation machine!
* Apply patches from thread
* Remove recurrent version, only keep chunked and autoregressive
* Remove unnecessary conts and asserts
* Remove more extra conts and asserts
* Cleanup masking
2025-12-16 11:59:53 +01:00
3d86c6c2b5
model: support GLM4V vision encoder ( #18042 )
...
* convert ok
* no deepstack
* less new tensors
* cgraph ok
* add mrope for text model
* faster patch merger
* add GGML_ROPE_TYPE_MRNORM
* add support for metal
* move glm4v do dedicated graph
* convert: add norm_embd
* clip: add debugging fn
* working correctly
* fix style
* use bicubic
* fix mrope metal
* improve cpu
* convert to neox ordering on conversion
* revert backend changes
* force stop if using old weight
* support moe variant
* fix conversion
* fix convert (2)
* Update tools/mtmd/clip-graph.h
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* process mrope_section on TextModel base class
* resolve conflict merge
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2025-12-16 11:25:26 +01:00
e88bf41fdc
Merge commit '12280ae9058600b0b14335fe59848e601e10eebc' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# common/CMakeLists.txt
# docs/docker.md
# examples/model-conversion/scripts/causal/compare-logits.py
# ggml/src/ggml-hexagon/htp/rope-ops.c
# tests/test-backend-ops.cpp
# tests/test-barrier.cpp
# tools/server/CMakeLists.txt
# tools/server/README.md
2025-12-16 16:29:01 +08:00
2aa45ef9e3
llama: Include algorithm header needed for C++23 ( #18078 )
2025-12-16 09:37:55 +02:00
c560316440
graph : reuse SSM graphs ( #16490 )
...
* graph : reuse hybrid graphs
* graph : reuse recurrent graphs
* graph : fix reuse check for recurrent inputs
* memory : move the recurrent state into the memory context
* Revert "memory : move the recurrent state into the memory context"
This reverts commit 00f115fe810815d4a22a6dee0acc346131e970e1.
* cont : fix build
2025-12-16 09:36:21 +02:00
2995341730
llama : add support for NVIDIA Nemotron 3 Nano ( #18058 )
...
* llama : add support for NVIDIA Nemotron Nano 3
This commit adds support for the NVIDIA Nemotron Nano 3 model, enabling
the conversion and running of this model.
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2025-12-16 07:19:26 +01:00
9d52f17ae3
model : add KORMo model ( #18032 )
...
* vocab: add KORMo Tokenizer
* model: add KORMoForCausalLM
* vocab: change pretokenizer to qwen2
* lint: fix unintended line removal
* model: make qwen2 bias tensor optional
* model: use qwen2 architecture for KORMo
2025-12-15 18:51:43 +01:00
4529c660c8
kv-cache: Fix state restore fragmented cache ( #17982 )
...
* kv-cache : fix state restore with fragmented cache (#17527 )
Change find_slot to allow non-contiguous allocation during state restore. Fixes 'failed to find available cells in kv cache' error when restoring state to fragmented cache.
* tests : update logic
* cleanup: tightened state_read_meta sig, added is_contiguous case
* fix: state_read_meta arg reorder loose ends
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2025-12-15 19:28:35 +02:00
b1f3a6e5db
llama: automatically set parameters not set by the user in such a way that maximizes GPU utilization ( #16653 )
...
* llama: automatically fit args to free memory
llama-fit-params tool
* fix CI
* hints for bug reports, ensure no reallocation
* fix segfault with Vulkan
* add llama-fit-params to CI
* fix CI
* fix CI
* fix CI
* minor adjustments
* fix assignment of 1 dense layer
* fix logger not being reset on model load failure
* remove --n-gpu-layer hint on model load failure
* fix llama-fit-params verbosity
* fix edge case
* fix typo [no ci]
2025-12-15 09:24:59 +01:00
0759b09c90
graph: add f_attn_temp_offset ( #18025 )
2025-12-14 13:05:59 +01:00
609a2d0268
models : fix YaRN regression + consolidate logic ( #18006 )
...
* models : fix YaRN regression + consolidate logic
* cont : fix the fix
* cont : remove header
* cont : add header
2025-12-14 08:34:56 +02:00
5266379bca
llama_context: synchronize before reallocating output buffer ( #17974 )
2025-12-13 09:19:51 -06:00
7bed317f53
models : fix the attn_factor for mistral3 graphs + improve consistency ( #17945 )
...
* models : fix the attn_factor for mistral3 graphs
* cont : rework attn_factor correction logic
* cont : make deepseek2 consistent
* cont : add TODO
* cont : special-case DSv2
* cont : revert Mistral 3 Large changes
* cont : fix DS2 to use the original attn_factor
* cont : minor comments
2025-12-12 17:12:40 +02:00
34d243bf3c
Merge commit 'b6777218198888ed4b18d839e805f887562a9be4' into concedo_experimental
...
# Conflicts:
# CONTRIBUTING.md
# common/chat.cpp
# docs/ops.md
# docs/ops/CPU.csv
# docs/ops/CUDA.csv
# docs/ops/OpenCL.csv
# ggml/src/ggml-cann/aclnn_ops.cpp
# ggml/src/ggml-cann/common.h
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-sycl/softmax.cpp
# grammars/README.md
# src/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tests/test-grammar-integration.cpp
# tests/test-grammar-parser.cpp
# tests/test-llama-grammar.cpp
# tools/mtmd/CMakeLists.txt
2025-12-11 23:33:19 +08:00
278e45becf
Merge commit '2fa51c19b028180b35d316e9ed06f5f0f7ada2c1' into concedo_experimental
...
# Conflicts:
# .github/actions/windows-setup-cuda/action.yml
# .github/workflows/build-linux-cross.yml
# .github/workflows/release.yml
# README.md
# docs/build-riscv64-spacemit.md
# examples/model-conversion/logits.cpp
# ggml/CMakeLists.txt
# ggml/src/ggml-cpu/CMakeLists.txt
# models/templates/Kimi-K2-Instruct.jinja
# models/templates/Kimi-K2-Thinking.jinja
# tests/test-chat.cpp
# tools/server/README.md
2025-12-11 23:04:48 +08:00
fd0d0cab03
move pipeline parallelism to a --pipelineparallel launch flag
2025-12-11 21:03:41 +08:00
d9f8f60618
batch : fix sequence id ownership ( #17915 )
...
* batch : fix sequence id ownage
* cont : reduce allocations
2025-12-11 14:29:47 +02:00
4dff236a52
ggml : remove GGML_KQ_MASK_PAD constant ( #17910 )
...
* ggml : remove GGML_KQ_MASK_PAD constant
* cont : remove comment
2025-12-10 20:53:16 +02:00
b677721819
model : Qwen3-Next-80B-A3B has 48 layers ( #17898 )
...
* model : Qwen3-Next-80B-A3B has 48 layers
* model : Add 80B-A3B type name
2025-12-10 15:22:40 +01:00
63908b631a
cmake: fix Mach-O current version number ( #17877 )
...
PR #17091 set the VERSION of various libraries to 0.0.abcd, where abcd
is the LLAMA_BUILD_NUMBER. That build number is too large to fit in the
Mach-O 'current version' field's 'micro' part, which only goes up to
255. This just sets the Mach-O current version to 0 to get it building
properly again.
Fixes #17258 .
2025-12-09 13:17:41 +02:00
42b12b5608
model : nit, DeepSeek V1 MoE is 16B and GigaChat is 20B ( #12652 )
...
* nit, DeepSeek V1 MoE is 16B
* base type on n_ff_exp instead
2025-12-09 12:15:06 +01:00
e39502e74b
llama : add token matching support to llama-grammar ( #17816 )
...
* llama : add token support to llama-grammar
* fix inverse token comment
* refactor trigger_patterns to replay tokens instead of the entire string
* add token documentation
* fix test-llama-grammar
* improve test cases for tokens
2025-12-09 00:32:57 -06:00
1d2a1ab73d
model : support Rnj-1 ( #17811 )
...
* add support for rnj1
* refactor gemma3 to support rnj-1
* address review comments
2025-12-09 04:49:03 +01:00
c8554b66e0
graph : use fill instead of scale_bias in grouped expert selection ( #17867 )
...
* use fill instead of scale_bias in grouped expert selection
* do not explicitly use _inplace
2025-12-08 21:29:59 +01:00
e4e9c4329c
Make graph_max_nodes vary by ubatch size ( #17794 )
...
* Make graph_max_nodes vary by ubatch size for models where chunking might explode the graph
* Update src/llama-context.h
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* Add missing const
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2025-12-08 14:32:41 +01:00
4d3726278b
model: add llama 4 scaling for mistral-large (deepseek arch) ( #17744 )
2025-12-07 22:29:54 +01:00
17c0c8d55d
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# README.md
# docs/backend/zDNN.md
# docs/build.md
# docs/ops.md
# ggml/CMakeLists.txt
# ggml/src/CMakeLists.txt
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-rpc/ggml-rpc.cpp
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# src/llama-quant.cpp
# tests/test-backend-ops.cpp
# tools/llama-bench/llama-bench.cpp
# tools/server/README.md
2025-12-07 16:48:38 +08:00
7c5d271d6c
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build.yml
# .github/workflows/release.yml
# .github/workflows/winget.yml
# CMakeLists.txt
# CODEOWNERS
# CONTRIBUTING.md
# cmake/build-info.cmake
# docs/ops.md
# docs/ops/BLAS.csv
# docs/ops/Metal.csv
# examples/CMakeLists.txt
# examples/save-load-state/save-load-state.cpp
# examples/simple-cmake-pkg/README.md
# ggml/CMakeLists.txt
# ggml/src/CMakeLists.txt
# ggml/src/ggml-rpc/ggml-rpc.cpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/embed_wgsl.py
# src/llama-quant.cpp
# tests/test-backend-ops.cpp
# tools/server/CMakeLists.txt
2025-12-07 16:37:32 +08:00
444f00b0ec
llama : remove quantization sanity check ( #17788 )
...
* llama : remove quantization sanity check
This commit removes the quantization sanity check for attention layers.
The motivation for this is that there are model that are hybrid models
that have recurrent layers, experts layers, and attention layers. For
these models the current check fails as the experts layers are not
taking into account. After consideration, it was decided that this check
is not strictly necessary, and can be removed to allow for more flexible
model architectures.
* llama : remove unused pruned_attention_w and is_clip_model vars
2025-12-06 12:26:20 +01:00
Concedo
Julius Tischbein
Johannes Gäßler
Georgi Gerganov
Tarek Dakhran
Xuan-Son Nguyen
Piotr Wilkin (ilintar)
Chris Peterson
Daniel Bevenius
HelloKS
ssweens
Jeff Bolz
Eric Zhang
Rhys-T
Sigbjørn Skjæret
Aldehir Rojas
philip-essential