* rpc: support apple RDMA as an RPC transport
* remove set_tensor micro optimization, rpc socket pinning per CR
* remove transparent reconnect
* trigger apple builds on RPC changes
---------
Co-authored-by: Ryan Churaman <rschu@meta.com>
* add ccache-buckets action
* use ccache-buckets
* only save on master
* install python3-venv for hip
* add jq and python3 for cuda
* only delete caches older than 5 minutes
* ci : apply ccache-clear with older/min/dry-run to all ccache jobs
Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731
* ci : install gh in ccache-clear if missing (container jobs)
The ccache-clear action relies on the gh CLI, which is not present in
container-based jobs. Install it on demand so those jobs can clear caches.
Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731
* ci : install gh via apt repo in ccache-clear
The install.sh script used previously is no longer served (404). Switch to
the official GitHub CLI apt repository, which is still available.
Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731
* ci : pass --repo to gh cache commands in ccache-clear
In container jobs gh cannot auto-detect the repository from git, so
gh cache list/delete fail with 'failed to run git: not a git repository'.
Pass the repository explicitly via --repo using GITHUB_REPOSITORY.
Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731
* ci : drop -new suffix from vulkan ccache key
The -new suffix was only needed to force a fresh cache. With
ccache-clear now evicting stale caches, the original key can be used
again. The old ccache-vulkan-ubuntu-24.04-arm-new entries still match
the ccache-clear key prefix and are cleaned up automatically.
Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731
* ci : fix ccache-clear date parsing on macOS (BSD date)
macOS ships BSD date, which has no -d option. The older cutoff check
was silently disabled there: 'date: illegal option -- d' errors in the
log and the loop was only stopped by the min limit, risking deletion
of caches not older than the cutoff (e.g. saved by a concurrent job).
Parse the ISO-8601 timestamps with GNU date when available and fall
back to BSD date otherwise (TZ=UTC, fractional seconds dropped).
Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731
* ci : extract ccache-clear logic into scripts/ccache-clear.sh
The composite action now consists of a dedicated step that installs the
GitHub CLI when missing (e.g. in container jobs) and a thin step that
calls the new script. The script follows the make-release-checks.sh
conventions (usage/env header, set -euo pipefail, CLI flags) and only
checks that gh is available. The action inputs are unchanged, so the
workflow steps are untouched.
Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731
* ci : remove unused apple ccaches
* Revert "ci : disable ubuntu-rocm (#26969)"
This reverts commit 9558fa44c9.
* ci: set ccache compiler_check=content for ROCm build
The ROCm toolchain is pip-installed fresh on every run, so the clang binary's
mtime changes each time. With ccache's default compiler_check=mtime that
invalidates the whole cache and warm builds only reached ~70% hits. Hash the
compiler contents instead so the cache survives toolchain reinstalls.
* Update ccache size to 1GB
We're waivering with so many architectures built, we need a bigger
ccache limit.
* merge fix
---------
Co-authored-by: Jim Wu <ywu@xilinx.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* ci : add older, min and dry-run options to ccache-clear
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* pi : add note about not wrapping lines in PR descriptions
[no ci]
Assisted-by: pi:llama.cpp/Qwen3.8-27B
The 'Create release' step had no id, so steps.create_release.outputs.id
resolved to an empty string in the 'Upload nightly-tag.txt' step. The
uploadReleaseAsset call then hit /releases//assets and failed with HTTP
404 (Unhandled error: HttpError), e.g. run 32513839499.
Add id: create_release to the step; the action already exposes the id
output.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* ci : run ccache-clear as the last step of release jobs
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* update disabled job too to force rebase
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
As agreed in ggml discussion #1579, the official semver releases now
include a nightly-tag.txt asset containing the tag of the corresponding
nightly release (e.g. b10485). The Web UI assets are published to the
HF bucket under the nightly tag, so this makes them discoverable for
each official release.
- make-release-desc.sh: expose the resolved nightly tag as a
nightly_tag output
- make-release.yml: create nightly-tag.txt from that tag, upload it
as a release asset (skipped on dry-run), mention it in the release
body and in the dry-run summary
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* CI: Use LLVM's OpenMP over MSFT_DEBUG_non_redist on Windows
Currently, we ship the non-redist debug version of microsoft's libomp.
This PR changes this to official LLVM's release, also packaging
the license as needed.
* Remove LLVM SHA from job name to increase legibility
* Add temp validations to CI
* Revert "Add temp validations to CI"
This reverts commit eef97c88b5.
* Build OpenMP in CI
* Make OpenMP fetch self-contained in cmake and cache in CI
* Robustify Licens-packaging
1. Ship OpenMP license, not LLVM's.
2. Invalidate cache also on checksum of the license
* Remove stale reference in docs/build.md
* No longer package base license in release
This was scope-creep
* Add explanatory comment to OpenMP license
* Remove arm64 smoke
Forgot this during conflict resolution during rebase of
c54c0e9cf6
* Remove GGML_OPENMP_FETCH_CACHE_DIR as requested by @CISC
* whitespace changes
* update to ov-2026.3, update device drivers
* ci: skip nemotron-h rollback test on OpenVINO
The OpenVINO backend does not support SSM_SCAN, so the Nemotron-H recurrent state rollback graph is split and cannot preserve the recurrent cache output shape. Keep the test enabled for other backends and retain the qwen35 OpenVINO rollback coverage.
---------
Co-authored-by: ravi9 <ravi.panchumarthy@intel.com>
* ci : create pre-release with change log and nightly link in make-release
After pushing the tag, create a pre-release using
ggml-org/action-create-release. The release description is generated by
scripts/make-release-desc.sh: the change log between the current and
previous version (one line per commit), a link to the corresponding
nightly build when it exists, and a note that semantic versioning is
still work in progress.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* cmake : bump version to 0.1.2
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* ci : find the nightly tag by commit in make-release-desc.sh
The nightly release is guaranteed by the release checks to point at HEAD,
so instead of reconstructing its name (commit count, branch, hash) just
pick the b* tag pointing at HEAD. This also drops the RELEASE_BRANCH env
var from the workflow.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* ci : resolve the release commit from the version tag in make-release-desc.sh
The change log and nightly lookup now use the commit the version tag
points at (HEAD when the tag does not exist), instead of always HEAD.
This makes the script usable locally for older versions, e.g.
./scripts/make-release-desc.sh v0.1.1. The tag is resolved to a SHA
first, since --points-at does not peel annotated tags.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* ci : normalize the version argument in make-release-desc.sh
Accept the version with or without the leading v (0.1.1 == v0.1.1) and
reject anything else, instead of silently treating a bare version as a
non-existent tag name.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* cont : clean-up
Add a "Create and push git tag" step to the release job, right before
the "Create release" step. The tag is created with git tag and pushed
with the deploy key already configured by the Clone step, instead of
relying on the Releases API (action-create-release) to create it as a
side effect.
The tag is lightweight, matching all existing b<number> release tags.
The step is idempotent: if the tag already exists (e.g. on a re-run),
creation and push are skipped.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* ci : parallelize platform builds in build-xcframework.sh
The ios-xcode release job builds 7 platform/simulator configurations
sequentially, each with -j $(nproc). Run them with at most 3 concurrent
builds (the release runner has 3 cores), splitting the cores between the
builds (-j 1 each on the runner), so total CPU pressure is unchanged while
the build phase runs about 2.3x faster.
- convert the 7 build blocks into functions (flags unchanged)
- add a run_builds_parallel pool: 3-slot sliding window, per-build logs,
dumps the failing log and aborts on error (background job failures do
not trigger set -e)
- queue the 2-arch builds first so the slower builds occupy the slots early
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* pi : add guideline for comments
* cont : disable 5/7 builds
* ci : make build-xcframework.sh builds configurable via CLI args
The script now takes an optional list of builds to run
(ios-sim ios-device macos visionos visionos-sim tvos-sim tvos-device);
with no arguments it builds all of them, as before. The per-build
lists for the build pool, framework setup, static library combining
and xcframework creation are now driven by a single build_spec
lookup instead of four hardcoded (partially commented-out) lists.
release.yml builds only macos and ios-device to cut the build time.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* ci : allow make-release to target a specific commit
The make-release workflow now accepts an optional 'commit' input. When
set, that commit is checked out and the release checks verify that it
belongs to the branch selected in the "Run workflow" dialog and is not
older than 3 days from the branch tip. The check is part of
make-release-checks.sh (driven by the RELEASE_BRANCH env), so it follows
the same dry-run semantics as the other checks.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* cont : scan latest 100 relase workflow runs
* cont : do not check manually for release.yml success
* test address on Intel-LNL-U7-258V
* retry
* run address on github
* use native build for cpu
* this should be runnable everywhere multicore
* disable ccache
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* cmake : introduce semantic versioning (wip)
This commit introduces semantic versioning to llama.cpp.
* squash! cmake : introduce semantic versioning (wip)
* cmake : update test-cmake README notes [no ci]
* include libmtmd in output so show its semversioned
* ci : add make-release workflow
* ci : fix build number check in build-cmake-pkg.yml
* examples : remove trailing whitespace
* ci : abort if upstream ggml version does not exist
* ci : extract step contents into scripts
* ci : add GGML_NATIVE=OFF to ubuntu job
* examples : remove CI build information from test-cmake [no ci]
This commit removes the nightly/release information that I added
previously to keep this focused only on using building and installing
llama.cpp with cmake and being able to quickly verify changes or
troubleshoot issues.
* ci : merge scripts into single script
* remove -dev-build_number support
This commit removes the incremental build number (versioning) support
that I added. This was incorrect and we should only use the semver for
the version. Releases will be tag a nightly build and package
maintainers/managers that build from source can use the tag and it is
therefor important that the correct version is reported. So a
nightly-build will report the semver without the build number. The build
number and commit as availble via cmake and test-cmake has been updated
to include an example of using them:
```console
$ ./build.sh
[test-cmake] version: 0.1.0, build: 10360 (08c69e381)
...
```
Refs: https://github.com/ggml-org/llama.cpp/pull/26839#discussion_r3755836969
* docs: add initial release.md documentation
* cmake : clean-up and add LLAMA_BUILD_IS_DEV option
* ci : remove version input from make-release job
* ci : add LLAMA_BUILD_IS_DEV=OFF to build-cmake-pkg.yml
Refs: https://github.com/danbev/llama.cpp/actions/runs/31576801921/job/94050639145
* docs : update release notes with LLAMA_BUILD_IS_DEV info [no ci]
* ci : add TODO to winget workflow [no ci]
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* Switch ROCm from 7.2.1 to 7.14
ROCm 7.14 is the first production release using TheRock build system.
It can be installed using multi-arch deliverables from wheels, debs,
rpms, tarballs or runfiles.
Adjust ROCm targets for Linux and Windows to use this instead.
* ci: switch all other Windows ROCm jobs to ROCm 7.14 wheels
Move the shared windows-setup-rocm composite action from the HIP SDK PRO
Edition installer to the multi-arch ROCm wheels (rocm[libraries,devel]).
The wheel-install logic that previously lived inline in release.yml is now
in the shared action, and both build-cache.yml and release.yml call it.
Also migrate the build-cuda-windows.yml hip job to the same wheel-based
layout (cache path/key, rocm-sdk environment setup, llvm/bin compiler
paths) so it keeps working after the action's contract changed; drop its
now-unused ROCm 7.2.1 rocWMMA download and stale include path.
* ci: Add support for CUDA 13.4 ARM64 builds for Windows
Added an architecture-specific CUDA 13.4 Windows build entry targeting ARM64.
Added a CMake configuration to enable ARM64 CUDA cross-compilation from an x64 Windows environment using the x64-hosted CUDA and MSVC toolchain while linking against the ARM64 CUDA import libraries to produce ggml-cuda.dll.
Validated the self-hosted Windows x64 workflow, including toolkit acquisition, CMake configuration, ARM64 CUDA cross-compilation, and packaging. Runtime validation was performed separately on a native ARM64 RTX Spark system using TinyLlama 1.1B Q4_K_M to verify the generated binaries.
The ARM64 CUDA job builds only the ggml-cuda.dll backend (LLAMA_BUILD_SERVER=OFF). The release consists of two packages: the main ARM64 release package, which combines the existing ARM64 CPU outputs with ggml-cuda.dll, and a separate runtime package containing the required CUDA runtime libraries (cudart64_13.dll, cublas64_13.dll, and cublasLt64_13.dll).
The CUDA 13.4 setup uses NVIDIA Developer Preview component archives instead of the GA component downloads used by the existing CUDA setups and will require updates once CUDA 13.4 reaches GA.
* ci: cleans up to align with x64 CUDA setup
- Moves CUDA-specific CMake options into matrix defines.
- Keeps the CUB 3DOT2 option only for CUDA 12.4.
- Removes runtime argument construction and the unnecessary server option.
- Aligns ARM64 CUDA runtime packaging with the existing robocopy approach.
- Generalizes the ARM64 release label from CUDA 13.4 to CUDA 13.
* ci: Set CUDA job name as version-architecture pair
* mark as preview
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* Update build-sanitize.yml
* make it run on pr
* fix thread
* Update build-sanitize.yml
* Update build-sanitize.yml
* just run thread on github machine
* ci: prepare for amd rocm ci
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
* ci: fix editorconfig-checker
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
* ci: fix device not recognised
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
* ci: rename gpu-amd to gpu-hip
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
* ci: gpu-hip to gpu-rocm
haha
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
* CUDA: allow integrated-GPU host output buffer in debug assert
On integrated GPUs (APUs), the scheduler can legitimately place a graph
node's output on the host-visible buffer, which ggml_cuda_compute_forward
already handles. The debug assert in ggml_cuda_graph_evaluate_and_capture
required every node output to be on the device buffer, so a debug build
aborts on such a node (e.g. attn_residual ADD -> ROCm_Host on RDNA3.5).
The source-tensor assert directly below already permits this via the
integrated + cuda_host exception; apply the same exception to the node's
own output buffer. Debug-only; no effect on release/compute.
Fixes test-recurrent-state-rollback on gfx1151 (Strix Halo).
* ci: enable unified memory for ROCm gfx1151 job
Work around a coherence issue on integrated RDNA3.5 (gfx1151) where GPU
kernels reading mmap-loaded weights can return incorrect output, which
makes test-llama-archs (and real inference) intermittently wrong.
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 uses managed memory, which restores
coherence. Remove once the underlying ROCm/HIP issue is fixed.
* test-llama-archs: skip jamba on HIP backend
jamba produces incorrect output (~0.55 NMSE vs CPU) on the HIP backend on
RDNA3.5 (gfx1151); the SSM kernels need separate investigation. Skip it
for now, matching the existing per-backend carve-outs (WebGPU), so the
ROCm CI can run the test for the remaining architectures.
* ci: use HIP_LAUNCH_BLOCKING for ROCm gfx1151 job
The gfx1151 ROCm CI job produced incorrect inference output (qwen3 perplexity ~88 vs ~9.4) due to an async-execution correctness issue in the HIP path. Serializing kernel launches with HIP_LAUNCH_BLOCKING=1 restores correctness. This replaces the earlier GGML_CUDA_ENABLE_UNIFIED_MEMORY workaround, which did not fix batched inference.
* test-backend-sampler: skip top-k subtests on HIP backend
The ROCm backend does not support the TOP_K/ARGSORT op at vocab scale (no CUB; bitonic argsort is capped at ncols <= 1024), so top-k/top-p backend samplers cannot be offloaded. The penalties, set_sampler, mixed, and top_p subtests assert that offload happened, so they fail on HIP. Skip them until TOP_K is supported on the ROCm backend.
* Update tests/test-backend-sampler.cpp
Co-authored-by: Aaron Teo <taronaeo@gmail.com>
* Update tests/test-backend-sampler.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
---------
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
Co-authored-by: Aaron Teo <aaron.teo1@ibm.com>
Co-authored-by: Jim Wu <ywu@xilinx.com>
Co-authored-by: Aaron Teo <taronaeo@gmail.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* Fix emscripten compilation with openmp
* Separate wasm job to its own workflow
* Add flags necessary for newer emsdk
* Just disable openmp
* Update triggers
* support cuda virtual devices
* disable NCCL path when virtual devices are used
* label virtual devices in description; add GPUx2 server CI jobs
* code refactor
* ci : add HF_TOKEN to self-hosted workflows
Pass the HF_TOKEN_CI repo secret as HF_TOKEN env var in the self-hosted
build and server workflows.
Fix the stale build.yml path reference.
Assisted-by: pi:llama.cpp/Qwen3.6-27B
* cont : add comment
---------
Co-authored-by: ggerganov <ggerganov@users.noreply.github.com>
Mario Limonciello
Ryan C
Sigbjørn Skjæret
Georgi Gerganov
Eve
Oliver Simons
Jeremie Miller
Zijun Yu
shivamkumard-ctrl
Daniel Bevenius
Xuan-Son Nguyen
Aaron Teo
Jim Wu
Niklas Wenzel
Jeff Bolz
Reese Levine
KyleHagy
Anav Prasad