metal : port new kernels into the split sources

Move the kernels added on master after the split (lightning indexer, DSv4 hyper-connections, silu_back, f16 bin ops) into the corresponding kernels/*.metal sources. Copied verbatim, no functional change.
ggml-metal: FWHT kernel for metal backend (#25924 )
2026-08-17 18:23:01 +02:00 · 2026-08-05 16:22:32 +08:00 · 2026-08-05 16:22:32 +08:00 · 2026-08-05 16:22:32 +08:00 · 2026-08-05 16:22:32 +08:00 · 2026-08-05 16:22:32 +08:00
1123 changed files with 27133 additions and 59604 deletions
@@ -57,6 +57,7 @@ COPY --from=web /app/tools/ui/dist tools/ui/dist
 RUN HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
    cmake -S . -B build \
        -DGGML_HIP=ON \
+        -DGGML_HIP_ROCWMMA_FATTN=ON \
        -DAMDGPU_TARGETS="$ROCM_DOCKER_ARCH" \
        -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON \
        -DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_TESTS=OFF \
@@ -4,10 +4,6 @@ inputs:
  cuda_version:
    description: "CUDA toolkit version"
    required: true
-  cuda_arch:
-    description: "CUDA target architecture"
-    required: false
-    default: "x64"

 runs:
  using: "composite"
@@ -131,26 +127,3 @@ runs:
          echo "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.3\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
          echo "CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.3" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
          echo "CUDA_PATH_V13_3=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.3" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
-
-    - name: Install Cuda Toolkit 13.4 for ARM64
-      if: ${{ inputs.cuda_version == '13.4' && inputs.cuda_arch == 'arm64' }}
-      shell: pwsh
-      run: |
-          mkdir -p "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4"
-          choco install unzip -y
-          curl -O "https://packages.nvidia.com/bin-archive/pool/windows-x86_64/5B515474-7E78-11F1-8656-C51E4F4B317F/cccl-windows-x86_64-13.3.4.1.2-archive.zip"
-          curl -O "https://packages.nvidia.com/bin-archive/pool/windows-x86_64/5B515474-7E78-11F1-8656-C51E4F4B317F/cuda_crt-windows-x86_64-13.4.46-archive.zip"
-          curl -O "https://packages.nvidia.com/bin-archive/pool/windows-x86_64/5B515474-7E78-11F1-8656-C51E4F4B317F/cuda_nvcc-windows-x86_64-13.4.46-archive.zip"
-          curl -O "https://packages.nvidia.com/bin-archive/pool/windows-x86_64/5B515474-7E78-11F1-8656-C51E4F4B317F/libnvvm-windows-x86_64-13.4.46-archive.zip"
-          curl -O "https://packages.nvidia.com/bin-archive/pool/windows-arm64/5B515474-7E78-11F1-8656-C51E4F4B317F/cuda_cudart-windows-arm64-13.4.46-archive.zip"
-          curl -O "https://packages.nvidia.com/bin-archive/pool/windows-arm64/5B515474-7E78-11F1-8656-C51E4F4B317F/libcublas-windows-arm64-13.7.0.10-archive.zip"
-          unzip '*.zip' -d "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4"
-          xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cccl-windows-x86_64-13.3.4.1.2-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
-          xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_crt-windows-x86_64-13.4.46-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
-          xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_nvcc-windows-x86_64-13.4.46-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
-          xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\libnvvm-windows-x86_64-13.4.46-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
-          xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\cuda_cudart-windows-arm64-13.4.46-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
-          xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\libcublas-windows-arm64-13.7.0.10-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" /E /I /H /Y
-          echo "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
-          echo "CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
-          echo "CUDA_PATH_V13_4=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.4" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
@@ -8,26 +8,8 @@ inputs:
 runs:
  using: "composite"
  steps:
-    - name: Install ROCm with Wheels
-      shell: pwsh
-      run: |
-        $ErrorActionPreference = "Stop"
-        write-host "Setting up Python virtual environment"
-
-        # Create the venv directly at the cache location to avoid relocation issues
-        New-Item -Path "C:\TheRock\build" -ItemType Directory -Force | Out-Null
-        python -m venv C:\TheRock\build\.venv
-        & C:\TheRock\build\.venv\Scripts\Activate.ps1
-
-        write-host "Upgrading pip"
-        python -m pip install --upgrade pip
-
-        write-host "Installing ROCm wheels for multi-arch support"
-        # Install ROCm wheels for multi-arch support (this may take several minutes)
-        python -m pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "rocm[libraries,devel]==${{ inputs.version }}"
-
-        # Pre-expand the devel tree so it is included in the cache
-        write-host "Initializing ROCm devel tree"
-        rocm-sdk init
-        if ($LASTEXITCODE -ne 0) { throw "rocm-sdk init failed with exit code $LASTEXITCODE" }
-        write-host "Completed ROCm wheel installation to C:\TheRock\build"
+    - name: Setup ROCm
+      uses: ./.github/actions/install-exe
+      with:
+        url: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-${{ inputs.version }}-Win11-For-HIP.exe
+        args: -install
@@ -60,6 +60,7 @@ jobs:
            -DCMAKE_BUILD_RPATH="@loader_path" \
            -DLLAMA_FATAL_WARNINGS=ON \
            -DLLAMA_BUILD_BORINGSSL=ON \
+            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=OFF \
            -DGGML_METAL_SHADER_DEBUG=ON \
            -DGGML_RPC=ON \
@@ -126,6 +127,7 @@ jobs:
        run: |
          sysctl -a
          cmake -B build -G Xcode \
+            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=ON \
            -DLLAMA_OPENSSL=OFF \
            -DLLAMA_BUILD_APP=OFF \
@@ -176,6 +178,7 @@ jobs:
        run: |
          sysctl -a
          cmake -B build -G Xcode \
+            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=ON \
            -DLLAMA_BUILD_COMMON=OFF \
            -DLLAMA_BUILD_APP=OFF \
@@ -209,6 +212,7 @@ jobs:
        run: |
          sysctl -a
          cmake -B build -G Xcode \
+            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=ON \
            -DLLAMA_BUILD_COMMON=OFF \
            -DLLAMA_BUILD_APP=OFF \
@@ -253,6 +257,7 @@ jobs:
        run: |
          sysctl -a
          cmake -B build -G Xcode \
+            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=ON \
            -DLLAMA_OPENSSL=OFF \
            -DLLAMA_BUILD_APP=OFF \
@@ -119,27 +119,27 @@ jobs:
          version_major: ${{ env.OPENVINO_VERSION_MAJOR }}
          version_full: ${{ env.OPENVINO_VERSION_FULL }}

-  # windows-2022-rocm-cache:
-  #   runs-on: windows-2022
+  windows-2022-rocm-cache:
+    runs-on: windows-2022

-  #   env:
-  #     # Make sure this is in sync with release.yml and build-cuda-windows.yml
-  #     ROCM_VERSION: "7.14.0"
+    env:
+      # Make sure this is in sync with build.yml
+      HIPSDK_INSTALLER_VERSION: "26.Q1"

-  #   steps:
-  #     - name: Clone
-  #       id: checkout
-  #       uses: actions/checkout@v6
+    steps:
+      - name: Clone
+        id: checkout
+        uses: actions/checkout@v6

-  #     - name: Setup Cache
-  #       uses: actions/cache@v5
-  #       id: cache-rocm
-  #       with:
-  #         path: C:\TheRock\build
-  #         key: rocm-wheels-${{ env.ROCM_VERSION }}-multi-arch-${{ runner.os }}
+      - name: Setup Cache
+        uses: actions/cache@v5
+        id: cache-rocm
+        with:
+          path: C:\Program Files\AMD\ROCm
+          key: cache-gha-rocm-${{ env.HIPSDK_INSTALLER_VERSION }}-${{ runner.os }}

-  #     - name: Setup ROCm
-  #       if: steps.cache-rocm.outputs.cache-hit != 'true'
-  #       uses: ./.github/actions/windows-setup-rocm
-  #       with:
-  #         version: ${{ env.ROCM_VERSION }}
+      - name: Setup ROCm
+        if: steps.cache-rocm.outputs.cache-hit != 'true'
+        uses: ./.github/actions/windows-setup-rocm
+        with:
+          version: ${{ env.HIPSDK_INSTALLER_VERSION }}
@@ -5,7 +5,7 @@ on:

 jobs:
  linux:
-    runs-on: [self-hosted, Linux]
+    runs-on: [self-hosted, Linux, CPU]
    steps:
      - uses: actions/checkout@v6
        with:
@@ -21,21 +21,15 @@ jobs:
                -DLLAMA_BUILD_TOOLS=OFF \
                -DLLAMA_BUILD_EXAMPLES=OFF \
                -DLLAMA_BUILD_APP=OFF \
-                -DLLAMA_BUILD_IS_DEV=OFF \
                -DCMAKE_BUILD_TYPE=Release
-          cmake --build build --config Release -j $(nproc)
+          cmake --build build --config Release
          cmake --install build --prefix "$PREFIX" --config Release

          export LLAMA_CONFIG="$PREFIX"/lib/cmake/llama/llama-config.cmake
          tclsh <<'EOF'
          set build(commit)  [string trim [exec git rev-parse --short HEAD]]
          set build(number)  [string trim [exec git rev-list  --count HEAD]]
-
-          set cmakelists [read [open "CMakeLists.txt" r]]
-          regexp {set\(LLAMA_VERSION_MAJOR\s+(\d+)\)} $cmakelists -> major
-          regexp {set\(LLAMA_VERSION_MINOR\s+(\d+)\)} $cmakelists -> minor
-          regexp {set\(LLAMA_VERSION_PATCH\s+(\d+)\)} $cmakelists -> patch
-          set build(version) "$major.$minor.$patch"
+          set build(version) "0.0.$build(number)"

          set llamaconfig [read [open "$env(LLAMA_CONFIG)" r]]
          set checks [list "set\\(LLAMA_VERSION     \\s+$build(version)\\)" \
@@ -54,4 +48,4 @@ jobs:

          cd examples/simple-cmake-pkg
          cmake -S . -B build -DCMAKE_PREFIX_PATH="$PREFIX"/lib/cmake
-          cmake --build build -j $(nproc)
+          cmake --build build
@@ -94,10 +94,8 @@ jobs:
        id: cmake_build
        run: |
          cmake -B build \
-            -DGGML_NATIVE=OFF \
            -DLLAMA_FATAL_WARNINGS=ON \
-            -DGGML_RPC=ON \
-            -DGGML_NATIVE=OFF
+            -DGGML_RPC=ON
          time cmake --build build --config Release -j $(nproc)

      - name: Test
@@ -99,6 +99,7 @@ jobs:
        run: |
          cmake -B build -S . \
            -DCMAKE_HIP_COMPILER="$(hipconfig -l)/clang" \
+            -DGGML_HIP_ROCWMMA_FATTN=ON \
            -DGPU_TARGETS="gfx1030" \
            -DGGML_HIP=ON
          cmake --build build --config Release -j $(nproc)
@@ -83,7 +83,7 @@ jobs:

    env:
      # Make sure this is in sync with build-cache.yml
-      ROCM_VERSION: "7.14.0"
+      HIPSDK_INSTALLER_VERSION: "26.Q1"

    strategy:
      matrix:
@@ -97,53 +97,36 @@ jobs:
        id: checkout
        uses: actions/checkout@v6

-      # - name: Cache ROCm Installation
-      #   uses: actions/cache@v5
-      #   id: cache-rocm
-      #   with:
-      #     path: C:\TheRock\build
-      #     key: rocm-wheels-${{ env.ROCM_VERSION }}-multi-arch-${{ runner.os }}
+      - name: Grab rocWMMA package
+        id: grab_rocwmma
+        run: |
+          curl -o rocwmma.deb "https://repo.radeon.com/rocm/apt/7.2.1/pool/main/r/rocwmma-dev/rocwmma-dev_2.2.0.70201-81~24.04_amd64.deb"
+          7z x rocwmma.deb
+          7z x data.tar
+
+      - name: Use ROCm Installation Cache
+        uses: actions/cache@v5
+        id: cache-rocm
+        with:
+          path: C:\Program Files\AMD\ROCm
+          key: cache-gha-rocm-${{ env.HIPSDK_INSTALLER_VERSION }}-${{ runner.os }}

      - name: Setup ROCm
-        # if: steps.cache-rocm.outputs.cache-hit != 'true'
+        if: steps.cache-rocm.outputs.cache-hit != 'true'
        uses: ./.github/actions/windows-setup-rocm
        with:
-          version: ${{ env.ROCM_VERSION }}
-
-      - name: Setup ROCm Environment
-        run: |
-          $ErrorActionPreference = "Stop"
-
-          # Activate venv from cache or fresh install
-          & C:\TheRock\build\.venv\Scripts\Activate.ps1
-
-          # Expand the devel tree (idempotent; no-op if already done during install)
-          rocm-sdk init
-          if ($LASTEXITCODE -ne 0) { throw "rocm-sdk init failed with exit code $LASTEXITCODE" }
-
-          # Get ROCm installation paths using the rocm-sdk CLI tool
-          $rocmPath = (rocm-sdk path --root)
-          if (-not $rocmPath) { throw "rocm-sdk path --root returned empty - devel package may not be installed" }
-          $rocmPath = $rocmPath.Trim()
-          $cmakePath = (rocm-sdk path --cmake).Trim()
-          $binPath = (rocm-sdk path --bin).Trim()
-          write-host "ROCm root: $rocmPath"
-
-          echo "HIP_PATH=$rocmPath" >> $env:GITHUB_ENV
-          echo "CMAKE_PREFIX_PATH=$cmakePath" >> $env:GITHUB_ENV
-          echo "HIP_DEVICE_LIB_PATH=$rocmPath\lib\llvm\amdgcn\bitcode" >> $env:GITHUB_ENV
-          echo "HIP_PLATFORM=amd" >> $env:GITHUB_ENV
-          echo "LLVM_PATH=$rocmPath\lib\llvm" >> $env:GITHUB_ENV
-          echo "$binPath" >> $env:GITHUB_PATH
-
-          # Keep venv in PATH for subsequent steps
-          echo "C:\TheRock\build\.venv\Scripts" >> $env:GITHUB_PATH
+          version: ${{ env.HIPSDK_INSTALLER_VERSION }}

      - name: Verify ROCm
        id: verify
        run: |
-          # Test the ROCm clang shipped in the installed wheel
-          & "${env:HIP_PATH}\lib\llvm\bin\clang.exe" --version
+          # Find and test ROCm installation
+          $clangPath = Get-ChildItem 'C:\Program Files\AMD\ROCm\*\bin\clang.exe' | Select-Object -First 1
+          if (-not $clangPath) {
+            Write-Error "ROCm installation not found"
+            exit 1
+          }
+          & $clangPath.FullName --version

      - name: ccache
        uses: ggml-org/ccache-action@v1.2.21
@@ -151,27 +134,29 @@ jobs:
          # TODO: this build does not match the build in release.yml, so we use a different cache key
          #       ideally, the builds should match, similar to the CUDA build above so that we would be able
          #       to populate the ccache for the release with manual runs of this workflow
-          #key: release-windows-2022-x64-hip-${{ env.ROCM_VERSION }}-${{ matrix.name }}
-          key: cuda-windows-2022-x64-hip-${{ env.ROCM_VERSION }}-${{ matrix.name }}
+          #key: release-windows-2022-x64-hip-${{ env.HIPSDK_INSTALLER_VERSION }}-${{ matrix.name }}
+          key: cuda-windows-2022-x64-hip-${{ env.HIPSDK_INSTALLER_VERSION }}-${{ matrix.name }}

      - name: Build
        id: cmake_build
        run: |
+          $env:HIP_PATH=$(Resolve-Path 'C:\Program Files\AMD\ROCm\*\bin\clang.exe' | split-path | split-path)
+          $env:CMAKE_PREFIX_PATH="${env:HIP_PATH}"
          cmake -G "Unix Makefiles" -B build -S . `
-            -DCMAKE_PREFIX_PATH="${env:HIP_PATH}" `
-            -DCMAKE_C_COMPILER="${env:HIP_PATH}\lib\llvm\bin\clang.exe" `
-            -DCMAKE_CXX_COMPILER="${env:HIP_PATH}\lib\llvm\bin\clang++.exe" `
-            -DCMAKE_HIP_COMPILER="${env:HIP_PATH}\lib\llvm\bin\clang.exe" `
+            -DCMAKE_C_COMPILER="${env:HIP_PATH}\bin\clang.exe" `
+            -DCMAKE_CXX_COMPILER="${env:HIP_PATH}\bin\clang++.exe" `
+            -DCMAKE_CXX_FLAGS="-I$($PWD.Path.Replace('\', '/'))/opt/rocm-7.2.1/include/" `
            -DCMAKE_BUILD_TYPE=Release `
            -DLLAMA_BUILD_BORINGSSL=ON `
-            -DHIP_PATH="${env:HIP_PATH}" `
+            -DROCM_DIR="${env:HIP_PATH}" `
            -DGGML_HIP=ON `
-            -DGPU_TARGETS="gfx1100" `
+            -DGGML_HIP_ROCWMMA_FATTN=ON `
+            -DGPU_TARGETS="gfx1100"  `
            -DGGML_RPC=ON
          cmake --build build -j ${env:NUMBER_OF_PROCESSORS}

      - name: ccache-clear
        uses: ./.github/actions/ccache-clear
        with:
-          #key: release-windows-2022-x64-hip-${{ env.ROCM_VERSION }}-${{ matrix.name }}
-          key: cuda-windows-2022-x64-hip-${{ env.ROCM_VERSION }}-${{ matrix.name }}
+          #key: release-windows-2022-x64-hip-${{ env.HIPSDK_INSTALLER_VERSION }}-${{ matrix.name }}
+          key: cuda-windows-2022-x64-hip-${{ env.HIPSDK_INSTALLER_VERSION }}-${{ matrix.name }}
@@ -15,12 +15,6 @@ on:
      '**/*.cpp'
    ]

-  pull_request:
-    types: [opened, synchronize, reopened]
-    paths: [
-      '.github/workflows/build-sanitize.yml'
-    ]
-
 concurrency:
  group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
  cancel-in-progress: true
@@ -34,35 +28,19 @@ env:

 jobs:
  ctest:
+    runs-on: [self-hosted, X64, CPU, Linux]
+
    continue-on-error: true

    strategy:
      matrix:
-        include:
-        # thread and address doesn't run properly on some self hosted machines, so run it on Github instead
-        - sanitizer: ADDRESS
-          machine: ubuntu-24.04
-        - sanitizer: THREAD
-          machine: ubuntu-24.04
-        - sanitizer: UNDEFINED
-          machine: [self-hosted, X64, Linux]
-
-    runs-on: ${{ matrix.machine }}
+        sanitizer: [ADDRESS, THREAD, UNDEFINED]

    steps:
      - name: Clone
        id: checkout
        uses: actions/checkout@v6

-      # - name: ccache
-      #   uses: ggml-org/ccache-action@v1.2.21
-      #   if: ${{ matrix.sanitizer != 'UNDEFINED' }}
-      #   with:
-      #     key: ctest-${{ matrix.sanitizer }}-ubuntu-24.04
-      #     variant: ccache
-      #     evict-old-files: 1d
-      #     save: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }}
-
      # with UNDEFINED sanitizer, we have to build in Debug to avoid GCC 13 false-positive warnings
      - name: Build (undefined)
        id: cmake_build_undefined
@@ -71,26 +71,6 @@ jobs:
          nvidia-smi
          GG_BUILD_CUDA=1 bash ./ci/run.sh ~/results/llama.cpp ~/mnt/llama.cpp

-  gpu-rocm:
-    runs-on: [self-hosted, Linux, AMD]
-
-    steps:
-      - name: Clone
-        id: checkout
-        uses: actions/checkout@v6
-
-      - name: Test
-        id: ggml-ci
-        # HIP_LAUNCH_BLOCKING=1: workaround for an async-execution correctness
-        # issue on integrated RDNA3.5 (gfx1151) where batched inference returns
-        # incorrect output (perplexity ~88 vs ~9.4). Serializing kernel launches
-        # restores correctness. Remove once the underlying ROCm/HIP issue is fixed.
-        env:
-          HIP_LAUNCH_BLOCKING: "1"
-        run: |
-          rocminfo
-          GG_BUILD_ROCM=1 GG_BUILD_AMDGPU_TARGETS=gfx1151 bash ./ci/run.sh ~/results/llama.cpp ~/mnt/llama.cpp
-
  gpu-vulkan-nvidia-cm:
    runs-on: [self-hosted, Linux, NVIDIA]

@@ -1,51 +0,0 @@
-name: Make Release
-
-on:
-  workflow_dispatch:
-    inputs:
-      dry_run:
-        description: 'Dry run - validate without creating the tag'
-        required: true
-        type: boolean
-        default: true
-
-env:
-  GH_TOKEN: ${{ github.token }}
-
-permissions:
-  contents: write
-
-jobs:
-  make-release:
-    runs-on: ubuntu-latest
-
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v6
-
-      - name: Run release checks
-        id: checks
-        run: bash scripts/make-release-checks.sh ${{ github.event.inputs.dry_run == 'true' && '--dry-run' || '' }}
-        env:
-          GITHUB_REPOSITORY: ${{ github.repository }}
-
-      - name: Create release tag
-        if: ${{ github.event.inputs.dry_run == 'false' }}
-        run: |
-          VERSION="${{ steps.checks.outputs.version }}"
-          git config user.name "github-actions[bot]"
-          git config user.email "github-actions[bot]@users.noreply.github.com"
-          git tag -a "${VERSION}" -m "Release ${VERSION}"
-          git push origin "${VERSION}"
-          echo "Created and pushed tag ${VERSION}"
-
-      - name: Dry run summary
-        if: ${{ github.event.inputs.dry_run == 'true' }}
-        run: |
-          if [[ "${{ steps.checks.outputs.checks_passed }}" == "true" ]]; then
-            echo "Dry run complete - all checks passed."
-            echo "Would have created tag: ${{ steps.checks.outputs.version }}"
-          else
-            echo "::error::Dry run found release check failures. A release tag would not be created."
-            exit 1
-          fi
@@ -1,23 +0,0 @@
-name: Convert PR to draft
-
-on:
-  pull_request_target:
-    types: [labeled]
-
-permissions:
-  pull-requests: write
-  issues: write
-  contents: write # required for "gh pr ready" command, see https://github.com/cli/cli/issues/8910
-
-jobs:
-  convert-to-draft:
-    if: github.event.label.name == 'draft' && github.event.pull_request.draft == false
-    runs-on: ubuntu-slim
-    steps:
-      - name: Convert PR to draft
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          PR_URL: ${{ github.event.pull_request.html_url }}
-        run: |
-          gh pr ready --undo "$PR_URL"
-          gh pr edit "$PR_URL" --remove-label draft
@@ -93,13 +93,13 @@ jobs:
          - build: 'arm64'
            arch: 'arm64'
            os: macos-26
-            defines: "-DGGML_METAL_EMBED_LIBRARY=ON -DCMAKE_OSX_DEPLOYMENT_TARGET=13.3"
+            defines: "-DGGML_METAL_USE_BF16=ON -DGGML_METAL_EMBED_LIBRARY=ON -DCMAKE_OSX_DEPLOYMENT_TARGET=13.3"
          # TODO: this build is disabled to save Github Actions resources (https://github.com/ggml-org/llama.cpp/pull/23780)
          #       in order to enable it again, we have to provision dedicated runners  to run it
          #- build: 'arm64-kleidiai'
          #  arch: 'arm64'
          #  os: macos-14
-          #  defines: "-DGGML_METAL_EMBED_LIBRARY=ON -DCMAKE_OSX_DEPLOYMENT_TARGET=13.3 -DGGML_CPU_KLEIDIAI=ON"
+          #  defines: "-DGGML_METAL_USE_BF16=ON -DGGML_METAL_EMBED_LIBRARY=ON -DCMAKE_OSX_DEPLOYMENT_TARGET=13.3 -DGGML_CPU_KLEIDIAI=ON"
          - build: 'x64'
            arch: 'x64'
            os: macos-15-intel
@@ -748,135 +748,6 @@ jobs:
          path: llama-bin-win-cpu-${{ matrix.arch }}.zip
          name: llama-bin-win-cpu-${{ matrix.arch }}.zip

-  windows-rocm:
-    needs: [check-release]
-    if: ${{ needs.check-release.outputs.should_release == 'true' }}
-
-    runs-on: windows-2022
-
-    strategy:
-      matrix:
-        include:
-          - ROCM_VERSION: "7.14.0"
-            gpu_targets: "gfx1010;gfx1011;gfx1012;gfx1030;gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1153;gfx1200;gfx1201"
-            build: x64
-
-    steps:
-      - name: Clone
-        id: checkout
-        uses: actions/checkout@v6
-        with:
-          fetch-depth: 0
-
-      - name: ccache
-        uses: ggml-org/ccache-action@v1.2.21
-        with:
-          key: windows-rocm-${{ matrix.ROCM_VERSION }}-${{ matrix.build }}
-          evict-old-files: 1d
-
-      # - name: Cache ROCm Installation
-      #   id: cache-rocm
-      #   uses: actions/cache@v5
-      #   with:
-      #     path: C:\TheRock\build
-      #     key: rocm-wheels-${{ matrix.ROCM_VERSION }}-multi-arch-${{ runner.os }}
-
-      - name: Setup ROCm
-        # if: steps.cache-rocm.outputs.cache-hit != 'true'
-        uses: ./.github/actions/windows-setup-rocm
-        with:
-          version: ${{ matrix.ROCM_VERSION }}
-
-      - name: Setup ROCm Environment
-        run: |
-          $ErrorActionPreference = "Stop"
-
-          # Activate venv from cache or fresh install
-          & C:\TheRock\build\.venv\Scripts\Activate.ps1
-
-          # Expand the devel tree (idempotent; no-op if already done during install)
-          rocm-sdk init
-          if ($LASTEXITCODE -ne 0) { throw "rocm-sdk init failed with exit code $LASTEXITCODE" }
-
-          # Get ROCm installation paths using the rocm-sdk CLI tool
-          $rocmPath = (rocm-sdk path --root)
-          if (-not $rocmPath) { throw "rocm-sdk path --root returned empty - devel package may not be installed" }
-          $rocmPath = $rocmPath.Trim()
-          $cmakePath = (rocm-sdk path --cmake).Trim()
-          $binPath = (rocm-sdk path --bin).Trim()
-          write-host "ROCm root: $rocmPath"
-          write-host "CMake path: $cmakePath"
-          write-host "Bin path: $binPath"
-
-          echo "HIP_PATH=$rocmPath" >> $env:GITHUB_ENV
-          echo "CMAKE_PREFIX_PATH=$cmakePath" >> $env:GITHUB_ENV
-          echo "HIP_DEVICE_LIB_PATH=$rocmPath\lib\llvm\amdgcn\bitcode" >> $env:GITHUB_ENV
-          echo "HIP_PLATFORM=amd" >> $env:GITHUB_ENV
-          echo "LLVM_PATH=$rocmPath\lib\llvm" >> $env:GITHUB_ENV
-          echo "$binPath" >> $env:GITHUB_PATH
-
-          # Keep venv in PATH for subsequent steps
-          echo "C:\TheRock\build\.venv\Scripts" >> $env:GITHUB_PATH
-
-      - name: Build
-        run: |
-          mkdir build
-          cd build
-          cmake .. `
-            -G "Unix Makefiles" `
-            -DCMAKE_PREFIX_PATH="${env:HIP_PATH}" `
-            -DCMAKE_BUILD_TYPE=Release `
-            -DGGML_BACKEND_DL=ON `
-            -DGGML_NATIVE=OFF `
-            -DGGML_CPU=ON `
-            -DGGML_CPU_ALL_VARIANTS=ON `
-            -DGGML_HIP=ON `
-            -DCMAKE_C_COMPILER="${env:HIP_PATH}\lib\llvm\bin\clang.exe" `
-            -DCMAKE_CXX_COMPILER="${env:HIP_PATH}\lib\llvm\bin\clang++.exe" `
-            -DCMAKE_C_FLAGS="-Wno-error=incompatible-pointer-types" `
-            -DCMAKE_HIP_COMPILER="${env:HIP_PATH}\lib\llvm\bin\clang.exe" `
-            -DHIP_PATH="${env:HIP_PATH}" `
-            -DGGML_HIP_ROCWMMA_FATTN=ON `
-            -DAMDGPU_TARGETS="${{ matrix.gpu_targets }}"
-          cmake --build . --config Release --parallel ${env:NUMBER_OF_PROCESSORS}
-
-      - name: ccache-clear
-        uses: ./.github/actions/ccache-clear
-        with:
-          key: windows-rocm-${{ matrix.ROCM_VERSION }}-${{ matrix.build }}
-
-      - name: Verify HIP backend was built
-        run: |
-          $hipDll = Get-ChildItem -Path build\bin -Filter "ggml-hip*.dll" -ErrorAction SilentlyContinue
-          if (-not $hipDll) {
-            Write-Host "##[error]ggml-hip*.dll was NOT produced. The HIP backend silently failed to build."
-            Write-Host "Contents of build\bin:"
-            Get-ChildItem build\bin | Format-Table -AutoSize
-            exit 1
-          }
-          Write-Host "HIP backend artifact found:"
-          $hipDll | Format-Table FullName, Length -AutoSize
-
-      - name: Determine tag name
-        id: tag
-        uses: ./.github/actions/get-tag-name
-
-      - name: Get ROCm short version
-        run: |
-          $rocmVersionShort = ('${{ matrix.ROCM_VERSION }}'.Split('.')[0..1] -join '.')
-          echo "ROCM_VERSION_SHORT=$rocmVersionShort" >> $env:GITHUB_ENV
-
-      - name: Pack artifacts
-        run: |
-          cp "LICENSE" "build\bin\"
-          7z a -snl llama-bin-win-rocm-${{ env.ROCM_VERSION_SHORT }}-${{ matrix.build }}.zip .\build\bin\*
-
-      - name: Upload artifacts
-        uses: actions/upload-artifact@v6
-        with:
-          path: llama-bin-win-rocm-${{ env.ROCM_VERSION_SHORT }}-${{ matrix.build }}.zip
-          name: llama-bin-win-rocm-${{ env.ROCM_VERSION_SHORT }}-${{ matrix.build }}.zip
-
  windows:
    needs: [check-release]
    if: ${{ needs.check-release.outputs.should_release == 'true' }}
@@ -977,7 +848,6 @@ jobs:
          name: llama-bin-win-${{ matrix.backend }}-${{ matrix.arch }}.zip

  windows-cuda:
-    name: windows-cuda (${{ matrix.cuda }}, ${{ matrix.arch }})
    needs: [check-release]
    if: ${{ needs.check-release.outputs.should_release == 'true' }}

@@ -988,16 +858,7 @@ jobs:

    strategy:
      matrix:
-        include:
-          - cuda: '12.4'
-            arch: x64
-            defines: '-DGGML_CUDA_CUB_3DOT2=ON'
-          - cuda: '13.3'
-            arch: x64
-            defines: ''
-          - cuda: '13.4'
-            arch: arm64
-            defines: '-DCMAKE_TOOLCHAIN_FILE=cmake/arm64-windows-msvc-cuda.cmake'
+        cuda: ['12.4', '13.3']

    steps:
      - name: Clone
@@ -1015,7 +876,6 @@ jobs:
        uses: ./.github/actions/windows-setup-cuda
        with:
          cuda_version: ${{ matrix.cuda }}
-          cuda_arch: ${{ matrix.arch }}

      - name: Install Ninja
        id: install_ninja
@@ -1025,62 +885,54 @@ jobs:
      - name: ccache
        uses: ggml-org/ccache-action@v1.2.21
        with:
-          key: release-windows-2022-${{ matrix.arch }}-cuda-${{ matrix.cuda }}
+          key: release-windows-2022-x64-cuda-${{ matrix.cuda }}

      - name: Build
        id: cmake_build
        shell: cmd
        # TODO: Remove GGML_CUDA_CUB_3DOT2 flag once CCCL 3.2 is bundled within CTK and that CTK version is used in this project
        run: |
-          call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" ${{ matrix.arch == 'x64' && 'x64' || 'amd64_arm64' }}
+          call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
          cmake -S . -B build -G "Ninja Multi-Config" ^
            -DGGML_BACKEND_DL=ON ^
            -DGGML_NATIVE=OFF ^
            -DGGML_CPU=OFF ^
            -DGGML_CUDA=ON ^
-            -DLLAMA_BUILD_BORINGSSL=ON ${{ matrix.defines }}
+            -DLLAMA_BUILD_BORINGSSL=ON ^
+            -DGGML_CUDA_CUB_3DOT2=ON
          set /A NINJA_JOBS=%NUMBER_OF_PROCESSORS%-1
          cmake --build build --config Release -j %NINJA_JOBS% --target ggml-cuda

      - name: ccache-clear
        uses: ./.github/actions/ccache-clear
        with:
-          key: release-windows-2022-${{ matrix.arch }}-cuda-${{ matrix.cuda }}
+          key: release-windows-2022-x64-cuda-${{ matrix.cuda }}

      - name: Pack artifacts
        id: pack_artifacts
        run: |
-          7z a -snl llama-bin-win-cuda-${{ matrix.cuda }}-${{ matrix.arch }}.zip .\build\bin\Release\ggml-cuda.dll
+          7z a -snl llama-bin-win-cuda-${{ matrix.cuda }}-x64.zip .\build\bin\Release\ggml-cuda.dll

      - name: Upload artifacts
        uses: actions/upload-artifact@v6
        with:
-          path: llama-bin-win-cuda-${{ matrix.cuda }}-${{ matrix.arch }}.zip
-          name: llama-bin-win-cuda-${{ matrix.cuda }}-${{ matrix.arch }}.zip
+          path: llama-bin-win-cuda-${{ matrix.cuda }}-x64.zip
+          name: llama-bin-win-cuda-${{ matrix.cuda }}-x64.zip

-      - name: Copy and pack Cuda runtime (x64)
-        if: ${{ matrix.arch == 'x64' }}
+      - name: Copy and pack Cuda runtime
        run: |
          echo "Cuda install location: ${{ env.CUDA_PATH }}"
          $dst='.\build\bin\cudart\'
          robocopy "${{env.CUDA_PATH}}\bin" $dst cudart64_*.dll cublas64_*.dll cublasLt64_*.dll
          robocopy "${{env.CUDA_PATH}}\lib" $dst cudart64_*.dll cublas64_*.dll cublasLt64_*.dll
          robocopy "${{env.CUDA_PATH}}\bin\x64" $dst cudart64_*.dll cublas64_*.dll cublasLt64_*.dll
-          7z a cudart-llama-bin-win-cuda-${{ matrix.cuda }}-${{ matrix.arch }}.zip $dst\*
-
-      - name: Copy and pack Cuda runtime (ARM64)
-        if: ${{ matrix.arch == 'arm64' }}
-        run: |
-          echo "Cuda install location: ${{ env.CUDA_PATH }}"
-          $dst='.\build\bin\cudart\'
-          robocopy "${{env.CUDA_PATH}}\bin\arm64" $dst cudart64_*.dll cublas64_*.dll cublasLt64_*.dll
-          7z a cudart-llama-bin-win-cuda-${{ matrix.cuda }}-${{ matrix.arch }}.zip $dst\*
+          7z a cudart-llama-bin-win-cuda-${{ matrix.cuda }}-x64.zip $dst\*

      - name: Upload Cuda runtime
        uses: actions/upload-artifact@v6
        with:
-          path: cudart-llama-bin-win-cuda-${{ matrix.cuda }}-${{ matrix.arch }}.zip
-          name: cudart-llama-bin-win-cuda-${{ matrix.cuda }}-${{ matrix.arch }}.zip
+          path: cudart-llama-bin-win-cuda-${{ matrix.cuda }}-x64.zip
+          name: cudart-llama-bin-win-cuda-${{ matrix.cuda }}-x64.zip

  windows-sycl:
    needs: [check-release]
@@ -1285,123 +1137,250 @@ jobs:
          path: llama-${{ steps.tag.outputs.name }}-bin-ubuntu-sycl-${{ matrix.build }}-x64.tar.gz
          name: llama-bin-ubuntu-sycl-${{ matrix.build }}-x64.tar.gz

-#   ubuntu-22-rocm:
-#     needs: [check-release, get-version]
-#     if: ${{ needs.check-release.outputs.should_release == 'true' }}
+  ubuntu-22-rocm:
+    needs: [check-release, get-version]
+    if: ${{ needs.check-release.outputs.should_release == 'true' }}

-#     runs-on: ubuntu-22.04
+    runs-on: ubuntu-22.04

-#     permissions:
-#       actions: write
+    permissions:
+      actions: write

-#     strategy:
-#       matrix:
-#         include:
-#           - ROCM_VERSION: "7.14.0"
-#             gpu_targets: "gfx908;gfx90a;gfx942;gfx950;gfx1010;gfx1011;gfx1012;gfx1030;gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1150;gfx1151;gfx1152;gfx1200;gfx1201"
-#             build: 'x64'
+    strategy:
+      matrix:
+        include:
+          - ROCM_VERSION: "7.2.1"
+            gpu_targets: "gfx908;gfx90a;gfx942;gfx1030;gfx1100;gfx1101;gfx1102;gfx1151;gfx1150;gfx1200;gfx1201"
+            build: 'x64'

-#     steps:
-#       - name: Clone
-#         id: checkout
-#         uses: actions/checkout@v6
-#         with:
-#           fetch-depth: 0
+    steps:
+      - name: Clone
+        id: checkout
+        uses: actions/checkout@v6
+        with:
+          fetch-depth: 0

-#       - name: Setup Node.js
-#         uses: actions/setup-node@v6
-#         with:
-#           node-version: "24"
-#           cache: "npm"
-#           cache-dependency-path: "tools/ui/package-lock.json"
+      - name: Setup Node.js
+        uses: actions/setup-node@v6
+        with:
+          node-version: "24"
+          cache: "npm"
+          cache-dependency-path: "tools/ui/package-lock.json"

-#       - name: Free up disk space
-#         uses: ggml-org/free-disk-space@v1.3.1
-#         with:
-#           tool-cache: true
+      - name: Free up disk space
+        uses: ggml-org/free-disk-space@v1.3.1
+        with:
+          tool-cache: true

-#       # - name: ccache
-#       #   uses: ggml-org/ccache-action@v1.2.21
-#       #   with:
-#       #     key: release-ubuntu-22.04-rocm-${{ matrix.ROCM_VERSION }}
+      - name: ccache
+        uses: ggml-org/ccache-action@v1.2.21
+        with:
+          key: release-ubuntu-22.04-rocm-${{ matrix.ROCM_VERSION }}

-#       - name: Dependencies
-#         id: depends
-#         run: |
-#           sudo apt install -y build-essential git cmake wget
+      - name: Dependencies
+        id: depends
+        run: |
+          sudo apt install -y build-essential git cmake wget

-#       - name: Setup TheRock with Wheels
-#         id: therock_env
-#         run: |
-#           # Create Python virtual environment
-#           python3 -m venv .venv
-#           source .venv/bin/activate
+      - name: Setup Legacy ROCm
+        if: matrix.ROCM_VERSION == '7.2.1'
+        id: legacy_env
+        run: |
+          sudo mkdir --parents --mode=0755 /etc/apt/keyrings
+          wget https://repo.radeon.com/rocm/rocm.gpg.key -O - | \
+            gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null

-#           # Install ROCm wheels for build
-#           # libraries = HIP runtime and CMake configs needed for linking
-#           # devel = compilers, headers, static libs
-#           python -m pip install --upgrade pip
-#           python -m pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "rocm[libraries,devel]==${{ matrix.ROCM_VERSION }}"
+          sudo tee /etc/apt/sources.list.d/rocm.list << EOF
+          deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/${{ matrix.ROCM_VERSION }} jammy main
+          EOF

-#           # Get ROCm installation paths using the rocm-sdk CLI tool
-#           ROCM_PATH=$(rocm-sdk path --root)
-#           CMAKE_PATH=$(rocm-sdk path --cmake)
-#           BIN_PATH=$(rocm-sdk path --bin)
-#           echo "ROCM_PATH=$ROCM_PATH"
-#           echo "CMAKE_PATH=$CMAKE_PATH"
-#           echo "BIN_PATH=$BIN_PATH"
+          sudo tee /etc/apt/preferences.d/rocm-pin-600 << EOF
+          Package: *
+          Pin: release o=repo.radeon.com
+          Pin-Priority: 600
+          EOF

-#           # Set environment variables
-#           echo "ROCM_PATH=$ROCM_PATH" >> $GITHUB_ENV
-#           echo "CMAKE_PREFIX_PATH=$CMAKE_PATH" >> $GITHUB_ENV
-#           echo "HIP_PATH=$ROCM_PATH" >> $GITHUB_ENV
-#           echo "PATH=$BIN_PATH:${PATH}" >> $GITHUB_ENV
-#           echo "LD_LIBRARY_PATH=$ROCM_PATH/lib:${LD_LIBRARY_PATH:-}" >> $GITHUB_ENV
+          sudo apt update
+          sudo apt-get install -y libssl-dev rocm-hip-sdk

-#           # Keep venv activated for subsequent steps
-#           echo "$(pwd)/.venv/bin" >> $GITHUB_PATH
+      - name: Setup TheRock
+        if: matrix.ROCM_VERSION != '7.2.1'
+        id: therock_env
+        run: |
+          wget https://repo.amd.com/rocm/tarball/therock-dist-linux-gfx1151-${{ matrix.ROCM_VERSION }}.tar.gz
+          mkdir install
+          tar -xf *.tar.gz -C install
+          export ROCM_PATH=$(pwd)/install
+          echo ROCM_PATH=$ROCM_PATH >> $GITHUB_ENV
+          echo PATH=$PATH:$ROCM_PATH/bin >> $GITHUB_ENV
+          echo LD_LIBRARY_PATH=$ROCM_PATH/lib:$ROCM_PATH/llvm/lib:$ROCM_PATH/lib/rocprofiler-systems >> $GITHUB_ENV

-#       - name: Build with native CMake HIP support
-#         id: cmake_build
-#         run: |
-#           cmake -B build -S . \
-#             -DCMAKE_HIP_COMPILER="$(hipconfig -l)/clang" \
-#             -DCMAKE_BUILD_TYPE=Release \
-#             -DGGML_BACKEND_DL=ON \
-#             -DGGML_NATIVE=OFF \
-#             -DCMAKE_INSTALL_RPATH='$ORIGIN' \
-#             -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-#             -DGGML_CPU_ALL_VARIANTS=ON \
-#             -DGPU_TARGETS="${{ matrix.gpu_targets }}" \
-#             -DGGML_HIP=ON \
-#             -DHIP_PLATFORM=amd \
-#             -DHF_UI_VERSION=${{ needs.get-version.outputs.ui_version }} \
-#             ${{ env.CMAKE_ARGS }}
-#           cmake --build build --config Release -j $(nproc)
+      - name: Build with native CMake HIP support
+        id: cmake_build
+        run: |
+          cmake -B build -S . \
+            -DCMAKE_HIP_COMPILER="$(hipconfig -l)/clang" \
+            -DCMAKE_BUILD_TYPE=Release \
+            -DGGML_BACKEND_DL=ON \
+            -DGGML_NATIVE=OFF \
+            -DCMAKE_INSTALL_RPATH='$ORIGIN' \
+            -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
+            -DGGML_CPU_ALL_VARIANTS=ON \
+            -DGPU_TARGETS="${{ matrix.gpu_targets }}" \
+            -DGGML_HIP=ON \
+            -DHIP_PLATFORM=amd \
+            -DGGML_HIP_ROCWMMA_FATTN=ON \
+            -DHF_UI_VERSION=${{ needs.get-version.outputs.ui_version }} \
+            ${{ env.CMAKE_ARGS }}
+          cmake --build build --config Release -j $(nproc)

-#       # - name: ccache-clear
-#       #   uses: ./.github/actions/ccache-clear
-#       #   with:
-#       #     key: release-ubuntu-22.04-rocm-${{ matrix.ROCM_VERSION }}
+      - name: ccache-clear
+        uses: ./.github/actions/ccache-clear
+        with:
+          key: release-ubuntu-22.04-rocm-${{ matrix.ROCM_VERSION }}

-#       - name: Determine tag name
-#         id: tag
-#         uses: ./.github/actions/get-tag-name
+      - name: Determine tag name
+        id: tag
+        uses: ./.github/actions/get-tag-name

-#       - name: Get ROCm short version
-#         run: echo "ROCM_VERSION_SHORT=$(echo '${{ matrix.ROCM_VERSION }}' | cut -d '.' -f 1,2)" >> $GITHUB_ENV
+      - name: Get ROCm short version
+        run: echo "ROCM_VERSION_SHORT=$(echo '${{ matrix.ROCM_VERSION }}' | cut -d '.' -f 1,2)" >> $GITHUB_ENV

-#       - name: Pack artifacts
-#         id: pack_artifacts
-#         run: |
-#           cp LICENSE ./build/bin/
-#           tar -czvf llama-${{ steps.tag.outputs.name }}-bin-ubuntu-rocm-${{ env.ROCM_VERSION_SHORT }}-${{ matrix.build }}.tar.gz --transform "s,^\.,llama-${{ steps.tag.outputs.name }}," -C ./build/bin .
+      - name: Pack artifacts
+        id: pack_artifacts
+        run: |
+          cp LICENSE ./build/bin/
+          tar -czvf llama-${{ steps.tag.outputs.name }}-bin-ubuntu-rocm-${{ env.ROCM_VERSION_SHORT }}-${{ matrix.build }}.tar.gz --transform "s,^\.,llama-${{ steps.tag.outputs.name }}," -C ./build/bin .

-#       - name: Upload artifacts
-#         uses: actions/upload-artifact@v6
-#         with:
-#           path: llama-${{ steps.tag.outputs.name }}-bin-ubuntu-rocm-${{ env.ROCM_VERSION_SHORT }}-${{ matrix.build }}.tar.gz
-#           name: llama-bin-ubuntu-rocm-${{ env.ROCM_VERSION_SHORT }}-${{ matrix.build }}.tar.gz
+      - name: Upload artifacts
+        uses: actions/upload-artifact@v6
+        with:
+          path: llama-${{ steps.tag.outputs.name }}-bin-ubuntu-rocm-${{ env.ROCM_VERSION_SHORT }}-${{ matrix.build }}.tar.gz
+          name: llama-bin-ubuntu-rocm-${{ env.ROCM_VERSION_SHORT }}-${{ matrix.build }}.tar.gz
+
+  windows-hip:
+    needs: [check-release, get-version]
+    if: ${{ needs.check-release.outputs.should_release == 'true' }}
+
+    runs-on: windows-2022
+
+    permissions:
+      actions: write
+
+    env:
+      HIPSDK_INSTALLER_VERSION: "26.Q1"
+
+    strategy:
+      matrix:
+        include:
+          - name: "radeon"
+            gpu_targets: "gfx1150;gfx1151;gfx1200;gfx1201;gfx1100;gfx1101;gfx1102;gfx1030;gfx1031;gfx1032"
+
+    steps:
+      - name: Clone
+        id: checkout
+        uses: actions/checkout@v6
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v6
+        with:
+          node-version: "24"
+          cache: "npm"
+          cache-dependency-path: "tools/ui/package-lock.json"
+
+      - name: Grab rocWMMA package
+        id: grab_rocwmma
+        run: |
+          curl -o rocwmma.deb "https://repo.radeon.com/rocm/apt/7.2.1/pool/main/r/rocwmma-dev/rocwmma-dev_2.2.0.70201-81~24.04_amd64.deb"
+          7z x rocwmma.deb
+          7z x data.tar
+
+      - name: Cache ROCm Installation
+        id: cache-rocm
+        uses: actions/cache@v5
+        with:
+          path: C:\Program Files\AMD\ROCm
+          key: cache-gha-rocm-${{ env.HIPSDK_INSTALLER_VERSION }}-${{ runner.os }}
+
+      - name: ccache
+        uses: ggml-org/ccache-action@v1.2.21
+        with:
+          key: release-windows-2022-x64-hip-${{ env.HIPSDK_INSTALLER_VERSION }}-${{ matrix.name }}
+
+      - name: Install ROCm
+        if: steps.cache-rocm.outputs.cache-hit != 'true'
+        id: depends
+        run: |
+          $ErrorActionPreference = "Stop"
+          write-host "Downloading AMD HIP SDK Installer"
+          Invoke-WebRequest -Uri "https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-${{ env.HIPSDK_INSTALLER_VERSION }}-Win11-For-HIP.exe" -OutFile "${env:RUNNER_TEMP}\rocm-install.exe"
+          write-host "Installing AMD HIP SDK"
+          $proc = Start-Process "${env:RUNNER_TEMP}\rocm-install.exe" -ArgumentList '-install' -NoNewWindow -PassThru
+          $completed = $proc.WaitForExit(600000)
+          if (-not $completed) {
+              Write-Error "ROCm installation timed out after 10 minutes. Killing the process"
+              $proc.Kill()
+              exit 1
+          }
+          if ($proc.ExitCode -ne 0) {
+              Write-Error "ROCm installation failed with exit code $($proc.ExitCode)"
+              exit 1
+          }
+          write-host "Completed AMD HIP SDK installation"
+
+      - name: Verify ROCm
+        id: verify
+        run: |
+          # Find and test ROCm installation
+          $clangPath = Get-ChildItem 'C:\Program Files\AMD\ROCm\*\bin\clang.exe' | Select-Object -First 1
+          if (-not $clangPath) {
+            Write-Error "ROCm installation not found"
+            exit 1
+          }
+          & $clangPath.FullName --version
+
+      - name: Build
+        id: cmake_build
+        run: |
+          $env:HIP_PATH=$(Resolve-Path 'C:\Program Files\AMD\ROCm\*\bin\clang.exe' | split-path | split-path)
+          $env:CMAKE_PREFIX_PATH="${env:HIP_PATH}"
+          cmake -G "Unix Makefiles" -B build -S . `
+            -DCMAKE_C_COMPILER="${env:HIP_PATH}\bin\clang.exe" `
+            -DCMAKE_CXX_COMPILER="${env:HIP_PATH}\bin\clang++.exe" `
+            -DCMAKE_CXX_FLAGS="-I$($PWD.Path.Replace('\', '/'))/opt/rocm-7.2.1/include/ -Wno-ignored-attributes -Wno-nested-anon-types" `
+            -DCMAKE_BUILD_TYPE=Release `
+            -DGGML_BACKEND_DL=ON `
+            -DGGML_NATIVE=OFF `
+            -DGGML_CPU=OFF `
+            -DGPU_TARGETS="${{ matrix.gpu_targets }}" `
+            -DGGML_HIP_ROCWMMA_FATTN=ON `
+            -DGGML_HIP=ON `
+            -DHF_UI_VERSION=${{ needs.get-version.outputs.ui_version }} `
+            -DLLAMA_BUILD_BORINGSSL=ON
+          cmake --build build --target ggml-hip -j ${env:NUMBER_OF_PROCESSORS}
+          md "build\bin\rocblas\library\"
+          md "build\bin\hipblaslt\library"
+          cp "${env:HIP_PATH}\bin\libhipblas.dll" "build\bin\"
+          cp "${env:HIP_PATH}\bin\libhipblaslt.dll" "build\bin\"
+          cp "${env:HIP_PATH}\bin\rocblas.dll" "build\bin\"
+          cp "${env:HIP_PATH}\bin\rocblas\library\*" "build\bin\rocblas\library\"
+          cp "${env:HIP_PATH}\bin\hipblaslt\library\*" "build\bin\hipblaslt\library\"
+
+      - name: ccache-clear
+        uses: ./.github/actions/ccache-clear
+        with:
+          key: release-windows-2022-x64-hip-${{ env.HIPSDK_INSTALLER_VERSION }}-${{ matrix.name }}
+
+      - name: Pack artifacts
+        id: pack_artifacts
+        run: |
+          7z a -snl llama-bin-win-hip-${{ matrix.name }}-x64.zip .\build\bin\*
+
+      - name: Upload artifacts
+        uses: actions/upload-artifact@v6
+        with:
+          path: llama-bin-win-hip-${{ matrix.name }}-x64.zip
+          name: llama-bin-win-hip-${{ matrix.name }}-x64.zip

  ios-xcode:
    needs: [check-release, get-version]
@@ -1423,6 +1402,7 @@ jobs:
        run: |
          sysctl -a
          cmake -B build -G Xcode \
+            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=ON \
            -DLLAMA_OPENSSL=OFF \
            -DLLAMA_BUILD_APP=OFF \
@@ -1576,9 +1556,9 @@ jobs:
      - windows-cpu
      - windows-cuda
      #- windows-sycl
-      - windows-rocm
+      - windows-hip
      - windows-openvino
-      #- ubuntu-22-rocm
+      - ubuntu-22-rocm
      - ubuntu-cpu
      - ubuntu-vulkan
      - ubuntu-24-openvino
@@ -1688,7 +1668,7 @@ jobs:
            - [Ubuntu s390x (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-ubuntu-s390x.tar.gz)
            - [Ubuntu x64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-ubuntu-vulkan-x64.tar.gz)
            - [Ubuntu arm64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-ubuntu-vulkan-arm64.tar.gz)
-            - Ubuntu x64 (ROCm 7.14)[DISABLED](https://github.com/ggml-org/llama.cpp/pull/26969)
+            - [Ubuntu x64 (ROCm 7.2)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-ubuntu-rocm-7.2-x64.tar.gz)
            - [Ubuntu x64 (OpenVINO)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-ubuntu-openvino-${{ needs.ubuntu-24-openvino.outputs.openvino_version }}-x64.tar.gz)
            - [Ubuntu x64 (SYCL FP32)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-ubuntu-sycl-fp32-x64.tar.gz)
            - [Ubuntu x64 (SYCL FP16)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-ubuntu-sycl-fp16-x64.tar.gz)
@@ -1702,11 +1682,10 @@ jobs:
            - [Windows arm64 (OpenCL Adreno)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-win-opencl-adreno-arm64.zip)
            - [Windows x64 (CUDA 12)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-win-cuda-12.4-x64.zip) - [CUDA 12.4 DLLs](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/cudart-llama-bin-win-cuda-12.4-x64.zip)
            - [Windows x64 (CUDA 13)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-win-cuda-13.3-x64.zip) - [CUDA 13.3 DLLs](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/cudart-llama-bin-win-cuda-13.3-x64.zip)
-            - [Windows arm64 (CUDA 13) (preview)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-win-cuda-13.4-arm64.zip) - [CUDA 13.4 DLLs](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/cudart-llama-bin-win-cuda-13.4-arm64.zip)
            - [Windows x64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-win-vulkan-x64.zip)
            - [Windows x64 (OpenVINO)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-win-openvino-${{ needs.windows-openvino.outputs.openvino_version }}-x64.zip)
            - [Windows x64 (SYCL)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-win-sycl-x64.zip)
-            - [Windows x64 (ROCm 7.14)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-win-rocm-7.14-x64.zip)
+            - [Windows x64 (HIP)](https://github.com/ggml-org/llama.cpp/releases/download/${{ steps.tag.outputs.name }}/llama-${{ steps.tag.outputs.name }}-bin-win-hip-radeon-x64.zip)

            **openEuler:**
            - [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23705)
@@ -25,12 +25,6 @@ on:
      'tools/server/**.*'
    ]

-  pull_request:
-    types: [opened, synchronize, reopened]
-    paths: [
-      '.github/workflows/server-sanitize.yml'
-    ]
-
 env:
  LLAMA_ARG_LOG_COLORS: 1
  LLAMA_ARG_LOG_PREFIX: 1
@@ -96,27 +90,23 @@ jobs:

      - name: Python setup
        id: setup_python
-        uses: actions/setup-python@v7
-
-      - name: Install Python dependencies
-        run: |
-          python3 -m venv .venv
-          .venv/bin/pip install -r tools/server/tests/requirements.txt
+        uses: actions/setup-python@v6
+        with:
+          python-version: '3.11'
+          pip-install: -r tools/server/tests/requirements.txt

      - name: Tests
        id: server_integration_tests
        if: ${{ (!matrix.disabled_on_pr || !github.event.pull_request) }}
        run: |
-          source .venv/bin/activate
          cd tools/server/tests
          export ${{ matrix.extra_args }}
-          ./tests.sh
+          pytest -v -x -m "not slow"

      - name: Slow tests
        id: server_integration_tests_slow
        if: ${{ (github.event.schedule || github.event.inputs.slow_tests == 'true') && matrix.build_type == 'Release' }}
        run: |
-          source .venv/bin/activate
          cd tools/server/tests
          export ${{ matrix.extra_args }}
-          SLOW_TESTS=1 ./tests.sh
+          SLOW_TESTS=1 pytest -v -x
@@ -72,7 +72,7 @@ jobs:
        run: |
          cd tools/server/tests
          source venv/bin/activate
-          ./tests.sh
+          pytest -v -x -m "not slow"

      - name: Tests (GPUx1, backend-sampling)
        id: server_integration_tests_backend_sampling
@@ -81,7 +81,7 @@ jobs:
          cd tools/server/tests
          source venv/bin/activate
          export LLAMA_ARG_BACKEND_SAMPLING=1
-          ./tests.sh
+          pytest -v -x -m "not slow"

      - name: Tests (GPUx2)
        id: server_integration_tests_gpu2
@@ -90,7 +90,7 @@ jobs:
          cd tools/server/tests
          source venv/bin/activate
          export GGML_METAL_DEVICES=2
-          ./tests.sh
+          pytest -v -x -m "not slow"

      - name: Tests (GPUx2, backend-sampling)
        id: server_integration_tests_gpu2_backend_sampling
@@ -99,7 +99,7 @@ jobs:
          cd tools/server/tests
          source venv/bin/activate
          export GGML_METAL_DEVICES=2 LLAMA_ARG_BACKEND_SAMPLING=1
-          ./tests.sh
+          pytest -v -x -m "not slow"

  server-cuda:
    runs-on: [self-hosted, llama-server, Linux, NVIDIA]
@@ -132,7 +132,7 @@ jobs:
        run: |
          cd tools/server/tests
          source venv/bin/activate
-          ./tests.sh
+          pytest -v -x -m "not slow"

      - name: Tests (GPUx1, backend-sampling)
        id: server_integration_tests_backend_sampling
@@ -141,7 +141,7 @@ jobs:
          cd tools/server/tests
          source venv/bin/activate
          export LLAMA_ARG_BACKEND_SAMPLING=1
-          ./tests.sh
+          pytest -v -x -m "not slow"

      - name: Tests (GPUx2)
        id: server_integration_tests_gpu2
@@ -150,7 +150,7 @@ jobs:
          cd tools/server/tests
          source venv/bin/activate
          export GGML_CUDA_DEVICES=2
-          ./tests.sh
+          pytest -v -x -m "not slow"

      - name: Tests (GPUx2, backend-sampling)
        id: server_integration_tests_gpu2_backend_sampling
@@ -159,7 +159,7 @@ jobs:
          cd tools/server/tests
          source venv/bin/activate
          export GGML_CUDA_DEVICES=2 LLAMA_ARG_BACKEND_SAMPLING=1
-          ./tests.sh
+          pytest -v -x -m "not slow"

  server-kleidiai:
    runs-on: ah-ubuntu_22_04-c8g_8x
@@ -219,4 +219,4 @@ jobs:
        run: |
          cd tools/server/tests
          source venv/bin/activate
-          ./tests.sh
+          pytest -v -x -m "not slow"
@@ -104,21 +104,21 @@ jobs:
        id: server_integration_tests
        run: |
          cd tools/server/tests
-          ./tests.sh
+          pytest -v -x -m "not slow"

      - name: Slow tests
        id: server_integration_tests_slow
        if: ${{ github.event.schedule || github.event.inputs.slow_tests == 'true' }}
        run: |
          cd tools/server/tests
-          SLOW_TESTS=1 ./tests.sh
+          SLOW_TESTS=1 pytest -v -x

      - name: Tests (Backend sampling)
        id: server_integration_tests_backend_sampling
        run: |
          cd tools/server/tests
          export LLAMA_ARG_BACKEND_SAMPLING=1
-          ./tests.sh
+          pytest -v -x -m "not slow"

      - name: Slow tests (Backend sampling)
        id: server_integration_tests_slow_backend_sampling
@@ -126,7 +126,7 @@ jobs:
        run: |
          cd tools/server/tests
          export LLAMA_ARG_BACKEND_SAMPLING=1
-          SLOW_TESTS=1 ./tests.sh
+          SLOW_TESTS=1 pytest -v -x

  windows:
    runs-on: windows-2025
@@ -167,17 +167,15 @@ jobs:

      - name: Tests
        id: server_integration_tests
-        shell: bash
        run: |
          cd tools/server/tests
-          export PYTHONIOENCODING=":replace"
-          ./tests.sh
+          $env:PYTHONIOENCODING = ":replace"
+          pytest -v -x -m "not slow"

      - name: Slow tests
        id: server_integration_tests_slow
        if: ${{ github.event.schedule || github.event.inputs.slow_tests == 'true' }}
-        shell: bash
        run: |
          cd tools/server/tests
-          export SLOW_TESTS="1"
-          ./tests.sh
+          $env:SLOW_TESTS = "1"
+          pytest -v -x
@@ -19,8 +19,6 @@ jobs:
        run: |
          cargo binstall komac@2.16.0 -y

-      # TODO: This should later be updated to publish releases instead of
-      # development release builds.
      - name: Find latest release
        id: find_latest_release
        uses: actions/github-script@v8
@@ -2,26 +2,6 @@ cmake_minimum_required(VERSION 3.14...3.28) # for add_link_options and implicit
 project("llama.cpp" C CXX)
 include(CheckIncludeFileCXX)

-### llama.cpp version
-set(LLAMA_VERSION_MAJOR 0)
-set(LLAMA_VERSION_MINOR 1)
-set(LLAMA_VERSION_PATCH 0)
-set(LLAMA_VERSION_BASE "${LLAMA_VERSION_MAJOR}.${LLAMA_VERSION_MINOR}.${LLAMA_VERSION_PATCH}")
-
-# whether this is a development/nightly build
-# set this to OFF when making a release from a release tag (vX.Y.Z)
-# ref: https://github.com/ggml-org/ggml/discussions/1579
-option(LLAMA_BUILD_IS_DEV "llama: dev build" ON)
-
-if (LLAMA_BUILD_IS_DEV)
-    set(LLAMA_VERSION "${LLAMA_VERSION_BASE}-dev")
-else()
-    # TODO: check that the current commit is tagged correctly according to the version specified above
-    set(LLAMA_VERSION "${LLAMA_VERSION_BASE}")
-endif()
-
-message(STATUS "llama.cpp version: ${LLAMA_VERSION}")
-
 #set(CMAKE_WARN_DEPRECATED YES)
 set(CMAKE_WARN_UNUSED_CLI YES)

@@ -44,6 +24,9 @@ if (CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
    set(LLAMA_STANDALONE ON)

    include(git-vars)
+
+    # configure project version
+    # TODO
 else()
    set(LLAMA_STANDALONE OFF)
 endif()
@@ -156,6 +139,7 @@ endif()
 if (NOT DEFINED LLAMA_BUILD_COMMIT)
    set(LLAMA_BUILD_COMMIT        ${BUILD_COMMIT})
 endif()
+set(LLAMA_INSTALL_VERSION 0.0.${LLAMA_BUILD_NUMBER})

 # override ggml options
 set(GGML_ALL_WARNINGS   ${LLAMA_ALL_WARNINGS})
@@ -291,12 +275,12 @@ configure_package_config_file(
              LLAMA_BIN_INSTALL_DIR )

 write_basic_package_version_file(
-        ${CMAKE_CURRENT_BINARY_DIR}/llama-config-version.cmake
-        VERSION ${LLAMA_VERSION}
+        ${CMAKE_CURRENT_BINARY_DIR}/llama-version.cmake
+    VERSION ${LLAMA_INSTALL_VERSION}
    COMPATIBILITY SameMajorVersion)

 install(FILES ${CMAKE_CURRENT_BINARY_DIR}/llama-config.cmake
-              ${CMAKE_CURRENT_BINARY_DIR}/llama-config-version.cmake
+              ${CMAKE_CURRENT_BINARY_DIR}/llama-version.cmake
        DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/llama)

 configure_file(cmake/llama.pc.in
@@ -12,7 +12,7 @@
 [![Docker](https://github.com/ggml-org/llama.cpp/actions/workflows/docker.yml/badge.svg)](https://github.com/ggml-org/llama.cpp/actions/workflows/docker.yml)
 [![Winget](https://github.com/ggml-org/llama.cpp/actions/workflows/winget.yml/badge.svg)](https://github.com/ggml-org/llama.cpp/actions/workflows/winget.yml)

-[manifesto](https://github.com/ggml-org/llama.cpp/discussions/205) / [ggml](https://github.com/ggml-org/ggml) / [ops](https://github.com/ggml-org/llama.cpp/blob/master/docs/ops.md) / [maintainer PRs](https://github.com/ggml-org/llama.cpp/issues?q=is%3Apr%20is%3Aopen%20draft%3AFalse%20(author%3Argerganov%20OR%20author%3AKitaitiMakoto%20OR%20author%3Adanbev%20OR%20author%3Aaldehir%20OR%20author%3Amax-krasnyansky%20OR%20author%3ACISC%20OR%20author%3Aggerganov%20OR%20author%3Aam17an%20OR%20author%3Abartowski1182%20OR%20author%3Ahipudding%20OR%20author%3AServeurpersoCom%20OR%20author%3Apwilkin%20OR%20author%3Areeselevine%20OR%20author%3Angxson%20OR%20author%3Ajeffbolznv%20OR%20author%3A0cc4m%20OR%20author%3Aangt%20OR%20author%3AIMbackK%20OR%20author%3Aarthw%20OR%20author%3AJohannesGaessler%20OR%20author%3AORippler%20OR%20author%3Aruixiang63%20OR%20author%3Axctan%20OR%20author%3Aallozaur%20OR%20author%3Ayomaytk%20OR%20author%3Aaendk%20OR%20author%3Agaugarg-nv%20OR%20author%3Ataronaeo%20OR%20author%3Aforforever73%20OR%20author%3Alhez%20OR%20author%3Anetrunnereve%20OR%20author%3Afairydreaming)%20sort%3Aupdated-desc) / [compile times](https://github.com/ggml-org/llama.cpp-dev/blob/master/README-compile-times.md) / [lib llama API](https://github.com/ggml-org/llama.cpp/issues/9289) / [llama-server REST API](https://github.com/ggml-org/llama.cpp/issues/9291)
+[manifesto](https://github.com/ggml-org/llama.cpp/discussions/205) / [ggml](https://github.com/ggml-org/ggml) / [ops](https://github.com/ggml-org/llama.cpp/blob/master/docs/ops.md) / [maintainer PRs](https://github.com/ggml-org/llama.cpp/issues?q=is%3Apr%20is%3Aopen%20draft%3AFalse%20(author%3Argerganov%20OR%20author%3AKitaitiMakoto%20OR%20author%3Adanbev%20OR%20author%3Aaldehir%20OR%20author%3Amax-krasnyansky%20OR%20author%3ACISC%20OR%20author%3Aggerganov%20OR%20author%3Aam17an%20OR%20author%3Abartowski1182%20OR%20author%3Ahipudding%20OR%20author%3AServeurpersoCom%20OR%20author%3Apwilkin%20OR%20author%3Areeselevine%20OR%20author%3Angxson%20OR%20author%3Ajeffbolznv%20OR%20author%3A0cc4m%20OR%20author%3Aangt%20OR%20author%3AIMbackK%20OR%20author%3Aarthw%20OR%20author%3AJohannesGaessler%20OR%20author%3AORippler%20OR%20author%3Aruixiang63%20OR%20author%3Axctan%20OR%20author%3Aallozaur%20OR%20author%3Ayomaytk%20OR%20author%3Aaendk%20OR%20author%3Agaugarg-nv%20OR%20author%3Ataronaeo%20OR%20author%3Aforforever73%20OR%20author%3Alhez%20OR%20author%3Anetrunnereve%20OR%20author%3Afairydreaming)%20sort%3Aupdated-desc) / [dev branches](https://github.com/ggml-org/llama.cpp-dev/blob/master/README-features.md) / [compile times](https://github.com/ggml-org/llama.cpp-dev/blob/master/README-compile-times.md) / [lib llama API](https://github.com/ggml-org/llama.cpp/issues/9289) / [llama-server REST API](https://github.com/ggml-org/llama.cpp/issues/9291)

 </div>

@@ -106,7 +106,6 @@ The `llama.cpp` project is build on top of the [ggml](https://github.com/ggml-or
 - [XCFramework](docs/xcframework.md)
 - [Completions](docs/completions.md)
 - [Models](docs/models.md)
- [Release process](docs/release.md)

 ## Contributing

@@ -21,18 +21,11 @@ Please disclose it as a private [security advisory](https://github.com/ggml-org/

 A team of volunteers on a reasonable-effort basis maintains this project. As such, please give us at least 90 days to work on a fix before public exposure.

-### AI-powered code scan
-
-llama.cpp has an AI security scanner that scans the code periodically. The full prompts and tool set can be found in [ggml-org/security-scan-prompt](https://github.com/ggml-org/security-scan-prompt).
-
-We greatly appreciate reports that reflect genuine research effort, and we are happy to spend our time reviewing them. Findings that an autonomous AI agent can surface on its own add little on top of the scans we already run.
-
 ### Requirements

 Before submitting your report, ensure you meet the following requirements:

 - You have read this policy and fully understand it.
- You have searched for existing discussions of the issue. If it has already been reported, your report will likely be rejected as a duplicate.
 - AI is only permitted in an assistive capacity as stated in [AGENTS.md](AGENTS.md). We do not accept reports that are written exclusively by AI.
 - Your report must include a working Proof-of-Concept in the form of a script and/or attached files.

@@ -53,8 +46,6 @@ Only vulnerabilities that fall within these parts of the project are considered

 Note that none of the topics under [Using llama.cpp securely](#using-llamacpp-securely) are considered vulnerabilities in LLaMA C++.

-Denial-of-Service (DoS) bugs are generally not treated as vulnerabilities. We don't reject them outright, but we look at them case-by-case and only accept those that are genuinely worth fixing.
-
 For vulnerabilities that fall within the `vendor` directory, please report them directly to the third-party project.

 ## Using llama.cpp securely
@@ -1,7 +1,5 @@
 #include "build-info.h"

-#include "llama.h"
-
 #include <cstdio>
 #include <cstdlib>
 #include <string>
@@ -79,12 +77,12 @@ static const command cmds[] = {

 #undef UPDATE_HIDDEN

-static int version(int /*argc*/, char ** /*argv*/) {
-    llama_print_build_info(llama_version());
+static int version(int argc, char ** argv) {
+    printf("%s\n", llama_build_info());
    return 0;
 }

-static int licenses(int /*argc*/, char ** /*argv*/) {
+static int licenses(int argc, char ** argv) {
    for (int i = 0; LICENSES[i]; ++i) {
        printf("%s\n", LICENSES[i]);
    }
@@ -17,6 +17,7 @@ LLAMA_BUILD_MTMD=ON
 GGML_METAL=ON
 GGML_METAL_EMBED_LIBRARY=ON
 GGML_BLAS_DEFAULT=ON
+GGML_METAL_USE_BF16=ON
 GGML_OPENMP=OFF

 COMMON_C_FLAGS="-Wno-macro-redefined -Wno-shorten-64-to-32 -Wno-unused-command-line-argument -g"
@@ -43,6 +44,7 @@ COMMON_CMAKE_ARGS=(
    -DGGML_METAL_EMBED_LIBRARY=${GGML_METAL_EMBED_LIBRARY}
    -DGGML_BLAS_DEFAULT=${GGML_BLAS_DEFAULT}
    -DGGML_METAL=${GGML_METAL}
+    -DGGML_METAL_USE_BF16=${GGML_METAL_USE_BF16}
    -DGGML_NATIVE=OFF
    -DGGML_OPENMP=${GGML_OPENMP}
 )
@@ -10,9 +10,6 @@
 # # with CUDA support
 # GG_BUILD_CUDA=1 bash ./ci/run.sh ./tmp/results ./tmp/mnt
 #
-# # with ROCm support
-# GG_BUILD_ROCM=1 GG_BUILD_AMDGPU_TARGETS=gfx1151 bash ./ci/run.sh ./tmp/results ./tmp/mnt
-#
 # # with SYCL support
 # GG_BUILD_SYCL=1 bash ./ci/run.sh ./tmp/results ./tmp/mnt
 #
@@ -49,14 +46,6 @@ mkdir -p "$2"
 OUT=$(realpath "$1")
 MNT=$(realpath "$2")

-# gpu-rocm self-hosted runner can't upload logs to blob; keep each run's logs in
-# their own dir keyed by the GitHub run id so an Actions run URL maps to its logs.
-if [ -n "${GG_BUILD_ROCM}" ] && [ -n "${GITHUB_RUN_ID}" ]; then
-    OUT="$OUT/run-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT:-1}"
-    mkdir -p "$OUT"
-    echo "ci results dir: $OUT"
-fi
-
 rm -f $OUT/*.log
 rm -f $OUT/*.exit
 rm -f $OUT/*.md
@@ -100,7 +89,7 @@ if [ ! -z ${GG_BUILD_CUDA} ]; then
 fi

 if [ ! -z ${GG_BUILD_ROCM} ]; then
-    CMAKE_EXTRA="${CMAKE_EXTRA} -DCMAKE_HIP_COMPILER=$(hipconfig -l)/clang -DGGML_HIP=ON"
+    CMAKE_EXTRA="${CMAKE_EXTRA} -DGGML_HIP=ON"
    if [ -z ${GG_BUILD_AMDGPU_TARGETS} ]; then
        echo "Missing GG_BUILD_AMDGPU_TARGETS, please set it to your GPU architecture (e.g. gfx90a, gfx1100, etc.)"
        exit 1
@@ -651,52 +640,39 @@ function gg_sum_rerank_tiny {

 function gg_check_build_requirements {
    if ! command -v git &> /dev/null; then
-        gg_printf 'git not found, please install\n'
-        exit 1
+        gg_printf 'git not found, please install'
    fi

    if ! command -v git-lfs &> /dev/null; then
-        gg_printf 'git-lfs not found, please install\n'
-        exit 1
-    fi
-
-    if ! git config --get filter.lfs.clean &> /dev/null; then
-        gg_printf 'git-lfs not initialized, please run `git lfs install`\n'
-        exit 1
+        gg_printf 'git-lfs not found, please install'
    fi

    if ! command -v wget &> /dev/null; then
-        gg_printf 'wget not found, please install\n'
-        exit 1
+        gg_printf 'wget not found, please install'
    fi

    if ! command -v python3 &> /dev/null; then
-        gg_printf 'python3 not found, please install\n'
-        exit 1
+        gg_printf 'python3 not found, please install'
    fi

    if ! command -v pip3 &> /dev/null; then
-        gg_printf 'pip3 not found, please install\n'
-        exit 1
+        gg_printf 'pip3 not found, please install'
    fi

    if ! python3 -m ensurepip --help &> /dev/null; then
-        gg_printf 'ensurepip not found, please install python3-venv package\n'
-        exit 1
+        gg_printf 'ensurepip not found, please install python3-venv package'
    fi

    if ! command -v cmake &> /dev/null; then
-        gg_printf 'cmake not found, please install\n'
-        exit 1
+        gg_printf 'cmake not found, please install'
    fi

    if ! command -v ccache &> /dev/null; then
-        gg_printf 'ccache not found, please consider installing for faster builds\n'
+        gg_printf 'ccache not found, please consider installing for faster builds'
    fi

    if ! command -v ctest &> /dev/null; then
-        gg_printf 'ctest not found, please install\n'
-        exit 1
+        gg_printf 'ctest not found, please install'
    fi
 }

@@ -1,26 +0,0 @@
-# Used to cross-compile ggml-cuda for Windows ARM64 on an x64 Windows host.
-set( CMAKE_SYSTEM_NAME Windows )
-set( CMAKE_SYSTEM_PROCESSOR arm64 )
-
-if ( DEFINED CUDAToolkit_ROOT )
-    file( TO_CMAKE_PATH "${CUDAToolkit_ROOT}" CUDA_ROOT )
-elseif ( DEFINED ENV{CUDA_PATH} )
-    file( TO_CMAKE_PATH "$ENV{CUDA_PATH}" CUDA_ROOT )
-else()
-    message( FATAL_ERROR "Set CUDAToolkit_ROOT or CUDA_PATH to a Windows CUDA Toolkit with ARM64 target libraries" )
-endif()
-
-if ( DEFINED ENV{VCToolsInstallDir} )
-    file( TO_CMAKE_PATH "$ENV{VCToolsInstallDir}" MSVC_TOOLS_ROOT )
-    set( CMAKE_CUDA_HOST_COMPILER "${MSVC_TOOLS_ROOT}/bin/Hostx64/arm64/cl.exe" CACHE FILEPATH "" )
-endif()
-
-set( CMAKE_CUDA_COMPILER "${CUDA_ROOT}/bin/nvcc.exe" CACHE FILEPATH "" )
-set( CMAKE_CUDA_FLAGS_INIT "-target-dir=arm64" )
-
-# FindCUDAToolkit selects lib/x64 from the host architecture on Windows.
-set( CUDA_CUDART              "${CUDA_ROOT}/lib/arm64/cudart.lib"   CACHE FILEPATH "" )
-set( CUDA_cudart_LIBRARY      "${CUDA_ROOT}/lib/arm64/cudart.lib"   CACHE FILEPATH "" )
-set( CUDA_cublas_LIBRARY      "${CUDA_ROOT}/lib/arm64/cublas.lib"   CACHE FILEPATH "" )
-set( CUDA_cublasLt_LIBRARY    "${CUDA_ROOT}/lib/arm64/cublasLt.lib" CACHE FILEPATH "" )
-set( CUDA_cuda_driver_LIBRARY "${CUDA_ROOT}/lib/arm64/cuda.lib"     CACHE FILEPATH "" )
@@ -1,4 +1,4 @@
-set(LLAMA_VERSION      @LLAMA_VERSION@)
+set(LLAMA_VERSION      @LLAMA_INSTALL_VERSION@)
 set(LLAMA_BUILD_COMMIT @LLAMA_BUILD_COMMIT@)
 set(LLAMA_BUILD_NUMBER @LLAMA_BUILD_NUMBER@)
 set(LLAMA_SHARED_LIB   @BUILD_SHARED_LIBS@)
@@ -5,6 +5,6 @@ includedir=@CMAKE_INSTALL_FULL_INCLUDEDIR@

 Name: llama
 Description: Port of Facebook's LLaMA model in C/C++
-Version: @LLAMA_VERSION@
+Version: @LLAMA_INSTALL_VERSION@
 Libs: -L${libdir} -lggml -lggml-base -lllama
 Cflags: -I${includedir}
@@ -121,8 +121,8 @@ add_library(${TARGET}
    )

 set_target_properties(${TARGET} PROPERTIES
-    VERSION ${LLAMA_VERSION_BASE}
-    SOVERSION ${LLAMA_VERSION_MAJOR}
+    VERSION ${LLAMA_INSTALL_VERSION}
+    SOVERSION 0
    MACHO_CURRENT_VERSION 0 # keep macOS linker from seeing oversized version number
 )

@@ -35,7 +35,6 @@
 #include <regex>
 #include <set>
 #include <string>
-#include <system_error>
 #include <thread> // for hardware_concurrency
 #include <vector>

@@ -561,15 +560,6 @@ void common_models_handler_apply(common_models_handler & handler, common_params
        }
    }

-    // infer the speculative type from the draft GGUF metadata when none is requested
-    // note: reads only the first split - sharded drafts need an explicit --spec-type
-    if (spec_types_is_default(params) && !params.speculative.draft.mparams.path.empty()) {
-        const auto types_gguf = common_speculative_types_from_gguf(params.speculative.draft.mparams.path);
-        if (!types_gguf.empty()) {
-            params.speculative.types = types_gguf;
-        }
-    }
-
    // when a sidecar type is requested, the draft repo resolves to its sidecar instead of a full model
    const bool spec_sidecar_found = !plan_spec.mtp.local_path.empty() ||
                                    !plan_spec.dflash.local_path.empty() ||
@@ -714,61 +704,12 @@ void common_models_handler_apply(common_models_handler & handler, common_params
 // CLI argument parsing functions
 //

-// apply config files (if present), a later file overrides an earlier one:
-// 1. system-wide: /etc/llama.cpp/config.ini (%PROGRAMDATA%\llama.cpp\config.ini on windows)
-// 2. user-level: ${XDG_CONFIG_HOME:-~/.config}/llama.cpp/config.ini (%APPDATA%\llama.cpp\config.ini on windows)
-static void common_params_apply_system_config(common_params & params, llama_example ex) {
-    std::vector<std::string> paths;
-
-#if defined(_WIN32)
-    const std::string program_data = common_get_env("PROGRAMDATA");
-    if (!program_data.empty()) {
-        paths.push_back(program_data + "\\llama.cpp\\config.ini");
-    }
-#else
-    paths.push_back("/etc/llama.cpp/config.ini");
-#endif
-
-    try {
-        paths.push_back(fs_get_config_directory() + "config.ini");
-    } catch (const std::exception & e) {
-        LOG_DBG("cannot read user-level config file, skipping: %s\n", e.what());
-    }
-
-    std::vector<std::string> found;
-    for (const auto & path : paths) {
-        std::error_code ec;
-        if (std::filesystem::exists(path, ec)) {
-            found.push_back(path);
-        }
-    }
-    if (found.empty()) {
-        return;
-    }
-
-    common_preset_context ctx(ex);
-    ctx.ignore_unknown_keys = true; // the same config file is shared by all programs
-    for (const auto & path : found) {
-        LOG_INF("using config file: %s\n", path.c_str());
-        common_preset global;
-        common_presets presets = ctx.load_from_ini(path, global);
-        global.apply_to_params(params);
-        auto it = presets.find(COMMON_PRESET_DEFAULT_NAME);
-        if (it != presets.end()) {
-            it->second.apply_to_params(params);
-        }
-    }
-}
-
 static bool common_params_parse_ex(int argc, char ** argv, common_params_context & ctx_arg) {
    common_params & params = ctx_arg.params;

    // setup log directly from params.verbosity: see tools/cli/cli.cpp
    common_log_set_verbosity_thold(params.verbosity);

-    // config file applies first, so env variables and CLI arguments override it
-    common_params_apply_system_config(params, ctx_arg.ex);
-
    std::unordered_map<std::string, std::pair<common_arg *, bool>> arg_to_options;
    for (auto & opt : ctx_arg.options) {
        for (const auto & arg : opt.args) {
@@ -1449,7 +1390,8 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
        {"--version"},
        "show version and build info",
        [](common_params &) {
-            llama_print_build_info(llama_version());
+            fprintf(stderr, "version: %d (%s)\n", llama_build_number(), llama_commit());
+            fprintf(stderr, "built with %s for %s\n", llama_compiler(), llama_build_target());
            exit(0);
        }
    ));
@@ -2663,16 +2605,14 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
    ).set_env("LLAMA_ARG_DIO"));
    add_opt(common_arg(
        {"-lm", "--load-mode"}, "MODE",
-        "model loading mode (default: auto)\n"
-        "- auto: mmap, unless a device does not support it\n"
+        "model loading mode (default: mmap)\n"
        "- none: no special loading mode\n"
        "- mmap: memory-map model (if mmap disabled, slower load but may reduce pageouts if not using mlock)\n"
        "- mlock: force system to keep model in RAM rather than swapping or compressing\n"
        "- mmap+mlock: mmap + force system to keep model in RAM rather than swapping or compressing\n"
        "- dio: use DirectIO if available\n",
        [](common_params & params, const std::string & value) {
-            /**/ if (value == "auto")       { params.load_mode = LLAMA_LOAD_MODE_AUTO;       }
-            else if (value == "none")       { params.load_mode = LLAMA_LOAD_MODE_NONE;       }
+            /**/ if (value == "none")       { params.load_mode = LLAMA_LOAD_MODE_NONE;       }
            else if (value == "mmap")       { params.load_mode = LLAMA_LOAD_MODE_MMAP;       }
            else if (value == "mlock")      { params.load_mode = LLAMA_LOAD_MODE_MLOCK;      }
            else if (value == "mmap+mlock") { params.load_mode = LLAMA_LOAD_MODE_MMAP_MLOCK; }
@@ -3368,17 +3308,6 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
            params.server_tools = parse_csv_row(value);
        }
    ).set_examples({LLAMA_EXAMPLE_SERVER}).set_env("LLAMA_ARG_TOOLS"));
-    add_opt(common_arg(
-        {"--tools-runtime"}, "OPTION",
-        "experimental: run tools in a separate runtime environment (default: none, use host environment)\n"
-        "available options:\n"
-        "  'docker:<image>', 'podman:<image>': spin up a new container and reuse it for all invocations, clean up on server exit\n"
-        "  'docker-container:<id>', 'podman-container:<id>': use an existing container by ID, won't stop on server exit\n"
-        "  'ssh:<target>': run tools on a remote POSIX host over SSH, key-based auth and a trusted host key are required\n",
-        [](common_params & params, const std::string & value) {
-            params.server_tools_runtime = value;
-        }
-    ).set_examples({LLAMA_EXAMPLE_SERVER}).set_env("LLAMA_ARG_TOOLS_RUNTIME"));
    add_opt(common_arg(
        {"--mcp-servers-config"}, "PATH",
        "experimental: path to JSON file with MCP server definitions (Cursor-compatible format) - do not enable in untrusted environments (default: none)\n"
@@ -3646,18 +3575,6 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
            }
        }
    ).set_examples({LLAMA_EXAMPLE_SERVER, LLAMA_EXAMPLE_COMPLETION, LLAMA_EXAMPLE_CLI}).set_env("LLAMA_ARG_REASONING"));
-    add_opt(common_arg(
-        {"--reasoning-effort"}, "LEVEL",
-        "reasoning effort level given to the chat template: 'default' to keep the template default,\n"
-        "or a level such as 'minimal', 'low', 'medium', 'high', 'xhigh' or 'max' (default: default)",
-        [](common_params & params, const std::string & value) {
-            if (value == "default") {
-                params.default_template_kwargs.erase("reasoning_effort");
-            } else {
-                params.default_template_kwargs["reasoning_effort"] = json(value).dump();
-            }
-        }
-    ).set_examples({LLAMA_EXAMPLE_SERVER, LLAMA_EXAMPLE_COMPLETION, LLAMA_EXAMPLE_CLI}).set_env("LLAMA_ARG_REASONING_EFFORT"));
    add_opt(common_arg(
        {"--reasoning-budget"}, "N",
        "token budget for thinking: -1 for unrestricted, 0 for immediate end, N>0 for token budget (default: -1)",
@@ -4077,9 +3994,6 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
        {"--spec-draft-n-max"}, "N",
        string_format("number of tokens to draft for speculative decoding (default: %d)", params.speculative.draft.n_max),
        [](common_params & params, int value) {
-            if (value < 0) {
-                throw std::invalid_argument("invalid value");
-            }
            params.speculative.draft.n_max = value;
        }
    ).set_spec().set_examples({LLAMA_EXAMPLE_SPECULATIVE, LLAMA_EXAMPLE_LOOKUP, LLAMA_EXAMPLE_SERVER, LLAMA_EXAMPLE_CLI}).set_env("LLAMA_ARG_SPEC_DRAFT_N_MAX"));
@@ -29,7 +29,7 @@ const char * llama_build_info(void) {
    return s.c_str();
 }

-void llama_print_build_info(const char * llama_version) {
-    fprintf(stderr, "version: %s (build %d, commit %s)\n", llama_version, llama_build_number(), llama_commit());
-    fprintf(stderr, "built with %s for %s\n", llama_compiler(), llama_build_target());
+void llama_print_build_info(void) {
+    fprintf(stderr, "%s: build = %d (%s)\n",      __func__, llama_build_number(), llama_commit());
+    fprintf(stderr, "%s: built with %s for %s\n", __func__, llama_compiler(), llama_build_target());
 }
@@ -8,4 +8,4 @@ const char * llama_compiler(void);
 const char * llama_build_target(void);
 const char * llama_build_info(void);

-void llama_print_build_info(const char *);
+void llama_print_build_info(void);
@@ -193,14 +193,6 @@ static std::vector<std::function<void(const common_chat_template & tmpl, autopar
              LOG_DBG(ANSI_ORANGE "[Patch: Laguna]\n" ANSI_RESET);
          }
      },
-      // Bailing V3
-      [](const common_chat_template & tmpl, autoparser & analysis) -> void {
-          if (tmpl.src.find("Bailing V3 chat template") != std::string::npos) {
-              analysis.tools.arguments.value_suffix = trim_whitespace(analysis.tools.arguments.value_suffix);
-              analysis.tools.arguments.tolerate_intertag_whitespace = true;
-              LOG_DBG(ANSI_ORANGE "[Patch: Bailing V3]\n" ANSI_RESET);
-          }
-      },

    });

@@ -594,7 +594,9 @@ common_peg_parser common_chat_peg_builder::python_style_tool_calls(

                // Full argument: name="value" or name=value
                auto arg_rule = tool_arg(
-                    tool_arg_open(tool_arg_name(arg_name_parser) + literal("=")) +
+                    tool_arg_open(eps()) +
+                    tool_arg_name(arg_name_parser) +
+                    literal("=") +
                    arg_value_parser +
                    tool_arg_close(eps())
                );
@@ -470,80 +470,36 @@ std::vector<common_chat_msg> common_chat_msgs_parse_oaicompat(const json & messa
    return msgs;
 }

-struct messages_inp_normalizer {
-    const jinja::caps & caps;
-
-    messages_inp_normalizer(const jinja::caps & c) : caps(c) {}
-
-    // handle supports_string_content / supports_typed_content
-    // if string=true and array=false, convert array to string
-    // if string=false and array=true, convert string to array
-    // if both are true, do nothing
-    json normalize(const json & messages) {
-        bool only_string = caps.supports_string_content && !caps.supports_typed_content;
-        bool only_typed  = !caps.supports_string_content && caps.supports_typed_content;
-        if ((!only_string && !only_typed) || !messages.is_array()) {
-            return messages;
-        }
-        json normalized = json::array();
-        for (const auto & msg : messages) {
-            json copy = msg;
-            auto it = copy.find("content");
-            if (it != copy.end()) {
-                if (only_typed && it->is_string()) {
-                    *it = json::array({
-                        json{
-                            {"type", "text"},
-                            {"text", it->get<std::string>()},
-                        }
-                    });
-                } else if (only_string && it->is_array()) {
-                    *it = concat_content_parts(*it);
-                }
-            }
-            normalized.push_back(std::move(copy));
-        }
-        return normalized;
-    }
-
-    // join parts with newline, do not add newline before or after media markers
-    static std::string concat_content_parts(const json & parts) {
-        std::string text;
-        bool last_was_media_marker = false;
-        for (const auto & part : parts) {
-            std::string type = part.value("type", "");
-            bool add_new_line = true;
-            if (type == "text") {
-                add_new_line = !last_was_media_marker && !text.empty();
-                last_was_media_marker = false;
-            } else if (type == "media_marker") {
-                add_new_line = false;
-                last_was_media_marker = true;
-            } else {
-                LOG_WRN("Ignoring content part type: %s\n", type.c_str());
-                continue;
-            }
-
-            if (add_new_line) {
-                text += '\n';
-            }
-
-            text += part.value("text", "");
-        }
-        return text;
-    }
-};
-
 static json render_message_to_json(const std::vector<common_chat_msg> & msgs, const jinja::caps & c) {
    if (!c.supports_string_content && !c.supports_typed_content) {
        LOG_WRN("%s: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.\n", __func__);
    }

+    bool only_string_accepted =  c.supports_string_content && !c.supports_typed_content;
+    bool only_typed_accepted  = !c.supports_string_content &&  c.supports_typed_content;
+
    json messages = json::array();
    for (const auto & msg : msgs) {
-        messages.push_back(msg.to_json_oaicompat(/* concat_typed_text= */ false));
+        if (only_string_accepted) {
+            json jmsg = msg.to_json_oaicompat(/* concat_typed_text= */ true);
+            messages.push_back(jmsg);
+        } else if (only_typed_accepted) {
+            json jmsg = msg.to_json_oaicompat(/* concat_typed_text= */ false);
+            if (jmsg.at("content").is_string()) {
+                jmsg["content"] = json::array({
+                    json{
+                        {"type", "text"},
+                        {"text", jmsg.at("content").get<std::string>()},
+                    }
+                });
+            }
+            messages.push_back(jmsg);
+        } else {
+            json jmsg = msg.to_json_oaicompat(/* concat_typed_text= */ false);
+            messages.push_back(jmsg);
+        }
    }
-    return messages_inp_normalizer(c).normalize(messages);
+    return messages;
 }

 // DEPRECATED: only used in tests
@@ -936,11 +892,8 @@ static std::string common_chat_template_direct_apply_impl(
    const std::optional<json> & additional_context = std::nullopt) {
    jinja::context ctx(tmpl.source());

-    // messages_override is already built for this template, do not touch its content parts
    nlohmann::ordered_json inp = nlohmann::ordered_json{
-        {"messages", messages_override.has_value()
-            ? *messages_override
-            : messages_inp_normalizer(tmpl.original_caps()).normalize(inputs.messages)},
+        {"messages", messages_override.has_value() ? *messages_override : inputs.messages},
        {"bos_token", tmpl.bos_token()},
        {"eos_token", tmpl.eos_token()},
        {"enable_thinking", inputs.enable_thinking},
@@ -967,10 +920,6 @@ static std::string common_chat_template_direct_apply_impl(
        bool enabled = inp["preserve_reasoning"].get<bool>();
        jinja::caps_apply_preserve_reasoning(ctx, enabled);
    }
-    if (inp.contains("reasoning_effort") && inp["reasoning_effort"].is_string() && !inp["reasoning_effort"].empty()) {
-        std::string reasoning_effort = inp["reasoning_effort"].get<std::string>();
-        jinja::caps_apply_reasoning_effort(ctx, reasoning_effort);
-    }

    jinja::global_from_json(ctx, inp, inputs.mark_input);

@@ -1004,12 +953,14 @@ static std::string common_chat_template_generation_prompt_impl(
    const std::optional<json> & tools_override = std::nullopt,
    const std::optional<json> & additional_context = std::nullopt) {

+    auto adjusted_messages = messages_override ? *messages_override : inputs.messages;
+
    autoparser::generation_params params = inputs;
    params.add_generation_prompt = false;
    params.continue_final_message = COMMON_CHAT_CONTINUATION_NONE;
-    std::string no_gen_prompt    = common_chat_template_direct_apply_impl(tmpl, params, messages_override, tools_override, additional_context);
+    std::string no_gen_prompt    = common_chat_template_direct_apply_impl(tmpl, params, adjusted_messages, tools_override, additional_context);
    params.add_generation_prompt = true;
-    std::string gen_prompt       = common_chat_template_direct_apply_impl(tmpl, params, messages_override, tools_override, additional_context);
+    std::string gen_prompt       = common_chat_template_direct_apply_impl(tmpl, params, adjusted_messages, tools_override, additional_context);

    size_t prefix_len = 0;
    size_t min_size = std::min(no_gen_prompt.size(), gen_prompt.size());
@@ -1215,16 +1166,6 @@ static common_chat_params common_chat_params_init_qwen3_coder(const common_chat_
        data.prompt += data.generation_prompt;
    }

-    std::vector<std::string> tool_call_starts = { "<tool_call>" };
-
-    // Match complete <function=name> opener for Qwen3-Coder models that occasionally omit the
-    // starting <tool_call>. The model may hallucinate a tool name, but it is preferable over
-    // constraining on <function which may occur in valid content generation, e.g. #include <functional>
-    foreach_function(inputs.tools, [&](const json & tool) {
-        const std::string name = tool.at("function").at("name");
-        tool_call_starts.push_back("<function=" + name + ">");
-    });
-
    auto parser = build_chat_peg_parser([&](common_chat_peg_builder & p) {
        auto generation_prompt = p.literal(GEN_PREFIX);

@@ -1297,7 +1238,7 @@ static common_chat_params common_chat_params_init_qwen3_coder(const common_chat_
            auto tool_calls = p.trigger_rule("tool-call-root", p.repeat(calls, min_calls, 1));

            return generation_prompt +
-                   (reasoning << p.content(p.until_one_of(tool_call_starts)) << tool_calls);
+                   (reasoning << p.content(p.until_one_of({ "<tool_call>", "<function=" })) << tool_calls);
        }

        // Content only parser
@@ -1323,9 +1264,12 @@ static common_chat_params common_chat_params_init_qwen3_coder(const common_chat_
        });

        if (data.grammar_lazy) {
-            for (const auto & start : tool_call_starts) {
-                data.grammar_triggers.push_back({ COMMON_GRAMMAR_TRIGGER_TYPE_WORD, start });
-            }
+            data.grammar_triggers = {
+                { COMMON_GRAMMAR_TRIGGER_TYPE_WORD, "<tool_call>" },
+                // Trigger on "<function" and not "<function=" because the trailing "=" is part of
+                // the token with the function name e.g. "=read"
+                { COMMON_GRAMMAR_TRIGGER_TYPE_WORD, "<function"   },
+            };
        }
    }

@@ -2370,179 +2314,6 @@ static common_chat_params common_chat_params_init_deepseek_v3_2(const common_cha
    return data;
 }

-// Kimi K3 - XTML tagged format, built by open_tag/close_tag macros:
-//   open_tag(t, attrs) = <|open|>t k="v"...<|sep|>   close_tag(t) = <|close|>t<|sep|>
-//   assistant := [think] [response] [tools] close_tag(message) <|end_of_msg|>
-// the generation prompt already opens the think (or response) section, so the
-// section opener is optional here - same as Kimi K2 Thinking
-static common_chat_params common_chat_params_init_kimi_k3(const common_chat_template &          tmpl,
-                                                          const autoparser::generation_params & inputs) {
-    common_chat_params data;
-
-    data.prompt            = common_chat_template_direct_apply_impl(tmpl, inputs);
-    data.generation_prompt = common_chat_template_generation_prompt_impl(tmpl, inputs);
-    data.format            = COMMON_CHAT_FORMAT_PEG_NATIVE;
-    data.supports_thinking = true;
-
-    const std::string SEP         = "<|sep|>";
-    const std::string MSG_START   = "<|open|>message role=\"assistant\"<|sep|>";
-    const std::string THINK_START = "<|open|>think<|sep|>";
-    const std::string THINK_END   = "<|close|>think<|sep|>";
-    const std::string RESP_START  = "<|open|>response<|sep|>";
-    const std::string RESP_END    = "<|close|>response<|sep|>";
-    const std::string TOOLS_START = "<|open|>tools<|sep|>";
-    const std::string TOOLS_END   = "<|close|>tools<|sep|>";
-    const std::string CALL_START  = "<|open|>call tool=\"";
-    const std::string CALL_END    = "<|close|>call<|sep|>";
-    const std::string ARG_START   = "<|open|>argument key=\"";
-    const std::string ARG_END     = "<|close|>argument<|sep|>";
-    const std::string MSG_END     = "<|close|>message<|sep|>";
-    const std::string EOM_TOKEN   = "<|end_of_msg|>";
-
-    // only the markers are special tokens. tag names ("think", "response", ...) are
-    // normal tokens and must not be preserved, or prose with those words is broken
-    data.preserved_tokens = {
-        "<|open|>",
-        "<|close|>",
-        "<|sep|>",
-        "<|end_of_msg|>",
-    };
-
-    data.thinking_start_tag = THINK_START;
-    data.thinking_end_tags  = { THINK_END };
-
-    // per-role message-start delimiters. user/assistant messages only have the role
-    // attribute, so the full opener is used. system and tool messages have more
-    // attributes, so those delimiters stop after the closing quote of the role
-    data.message_delimiters = {
-        { COMMON_CHAT_ROLE_ASSISTANT, "<|open|>message role=\"assistant\"<|sep|>" },
-        { COMMON_CHAT_ROLE_USER,      "<|open|>message role=\"user\"<|sep|>"      },
-        { COMMON_CHAT_ROLE_TOOL,      "<|open|>message role=\"tool\""             },
-        { COMMON_CHAT_ROLE_SYSTEM,    "<|open|>message role=\"system\""           },
-    };
-
-    auto has_tools         = inputs.tools.is_array() && !inputs.tools.empty();
-    auto extract_reasoning = inputs.reasoning_format != COMMON_REASONING_FORMAT_NONE;
-    auto include_grammar   = has_tools && inputs.tool_choice != COMMON_CHAT_TOOL_CHOICE_NONE;
-
-    if (inputs.has_continuation()) {
-        const auto & msg = inputs.continue_msg;
-
-        data.generation_prompt = MSG_START + THINK_START + msg.reasoning_content;
-        if (inputs.continue_final_message == COMMON_CHAT_CONTINUATION_CONTENT) {
-            data.generation_prompt += THINK_END + RESP_START + msg.render_content();
-        }
-
-        data.prompt += data.generation_prompt;
-    }
-
-    auto parser = build_chat_peg_parser([&](common_chat_peg_builder & p) {
-        auto end = p.end();
-
-        auto start = p.optional(p.literal(MSG_START));
-
-        // the think section is always consumed, even with reasoning extraction off:
-        // the generation prompt ends with open_tag('think'), so it is always present.
-        // reasoning stops at its own closer, or at the response opener if the model
-        // skips the closer
-        auto think_body = extract_reasoning ? p.reasoning(p.until_one_of({ THINK_END, RESP_START })) :
-                                              p.content(p.until_one_of({ THINK_END, RESP_START }));
-
-        auto reasoning = p.optional(p.optional(p.literal(THINK_START)) + think_body +
-                                    p.optional(p.literal(THINK_END)));
-
-        // content runs to the response closer, or to the next section if truncated
-        auto response = p.optional(p.literal(RESP_START)) +
-                        p.content(p.until_one_of({ RESP_END, TOOLS_START, MSG_END })) +
-                        p.optional(p.literal(RESP_END));
-
-        // the EOG token after the message closer reaches the parser as text,
-        // so it must be consumed or the parse stays incomplete
-        auto trailer = p.optional(p.literal(MSG_END)) + p.optional(p.literal(EOM_TOKEN));
-
-        if (!has_tools || inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_NONE) {
-            return start + reasoning + response + trailer + end;
-        }
-
-        auto tool_choices = p.choice();
-        foreach_function(inputs.tools, [&](const json & tool) {
-            const auto & function = tool.at("function");
-            std::string  name     = function.at("name");
-            const json   schema   = function.contains("parameters") ? function.at("parameters") : json::object();
-
-            // arguments come one tag per key, with the JSON type in a type="..."
-            // attribute. the type is taken from the tool schema instead, as it tells
-            // us if the value is JSON or a literal string
-            auto args = p.eps();
-            if (schema.contains("properties") && !schema.at("properties").empty()) {
-                auto arg_choices = p.choice();
-                for (const auto & prop : schema.at("properties").items()) {
-                    const std::string & key = prop.key();
-
-                    std::string type = "string";
-                    if (prop.value().is_object() && prop.value().contains("type") &&
-                        prop.value().at("type").is_string()) {
-                        type = prop.value().at("type").get<std::string>();
-                    }
-
-                    auto value = type == "string" ? p.tool_arg_string_value(p.until(ARG_END)) :
-                                                    p.tool_arg_value(p.until(ARG_END));
-
-                    // skip the trailing type="..." attribute: anything up to <|sep|>
-                    arg_choices |= p.rule("kimi-k3-arg-" + name + "-" + key,
-                                          p.tool_arg(p.tool_arg_open(p.literal(ARG_START)) +
-                                                     p.tool_arg_name(p.literal(key)) + p.literal("\"") +
-                                                     p.until(SEP) + p.literal(SEP) + value +
-                                                     p.tool_arg_close(p.literal(ARG_END))));
-                }
-                args = p.zero_or_more(arg_choices);
-            }
-
-            // skip the trailing index="N" attribute the same way
-            auto call = p.tool(p.tool_open(p.literal(CALL_START) + p.tool_name(p.literal(name)) + p.literal("\"") +
-                                           p.until(SEP) + p.literal(SEP)) +
-                               p.tool_args(args) + p.tool_close(p.literal(CALL_END)));
-
-            tool_choices |= p.rule("kimi-k3-tool-" + name, call);
-        });
-
-        // all calls go inside one tools section, then the message is closed. the
-        // message closer is part of the trigger rule, or else the lazy grammar
-        // rejects it once tool calls have started
-        auto tools_section =
-            p.trigger_rule("kimi-k3-tool-call", p.literal(TOOLS_START) + p.one_or_more(tool_choices) +
-                                                    p.literal(TOOLS_END) + p.optional(p.literal(MSG_END)) +
-                                                    p.optional(p.literal(EOM_TOKEN)));
-
-        auto tools = inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_REQUIRED ? tools_section :
-                                                                              p.optional(tools_section);
-
-        return start + reasoning + response + tools + trailer + end;
-    });
-
-    data.parser = parser.save();
-
-    if (include_grammar) {
-        data.grammar_lazy = inputs.tool_choice != COMMON_CHAT_TOOL_CHOICE_REQUIRED;
-        data.grammar      = build_grammar([&](const common_grammar_builder & builder) {
-            foreach_function(inputs.tools, [&](const json & tool) {
-                const auto & function = tool.at("function");
-                if (function.contains("parameters")) {
-                    auto schema = function.at("parameters");
-                    builder.resolve_refs(schema);
-                }
-            });
-            parser.build_grammar(builder, data.grammar_lazy);
-        });
-
-        data.grammar_triggers = {
-            { COMMON_GRAMMAR_TRIGGER_TYPE_WORD, TOOLS_START },
-        };
-    }
-
-    return data;
-}
-
 // Cohere2 MoE (a.k.a. "North Code") parser.
 //
 // The assistant turn is fully marker-wrapped:
@@ -3315,153 +3086,6 @@ static common_chat_params common_chat_params_init_minicpm5(const common_chat_tem
    return data;
 }

-// An assistant turn is rendered as one or more messages, each
-// "<|start|>assistant to=<recipient><|message|>{content}{END}" where END is
-// <|eom|> (more messages follow) or <|eot|> (end of turn):
-//   - chain-of-thought: to=self, terminated by <|eom|>
-//   - final answer:     to=user, terminated by <|eot|>
-// The generation prompt is just "<|start|>assistant"; the model emits its own
-// " to=...<|message|>".
-static common_chat_params common_chat_params_init_muse_glimmer(const common_chat_template &          tmpl,
-                                                               const autoparser::generation_params & inputs) {
-    common_chat_params data;
-
-    data.prompt            = common_chat_template_direct_apply_impl(tmpl, inputs);
-    data.generation_prompt = "<|start|>assistant";
-    data.format            = COMMON_CHAT_FORMAT_PEG_NATIVE;
-    data.supports_thinking = true;
-
-    data.preserved_tokens = {
-        "<|start|>", "<|message|>", "<|eom|>", "<|eot|>",
-        // ATEM tool-call markup emitted on " to=<tool>" turns.
-        "<atem:function_calls>", "<atem:invoke", "<atem:parameter", "</atem:parameter>",
-        "</atem:invoke>", "</atem:function_calls>",
-    };
-
-    data.message_delimiters = {
-        { COMMON_CHAT_ROLE_ASSISTANT, "<|start|>assistant" },
-        { COMMON_CHAT_ROLE_USER,      "<|start|>user"      },
-        { COMMON_CHAT_ROLE_SYSTEM,    "<|start|>system"    },
-        { COMMON_CHAT_ROLE_TOOL,      "<|start|>tool"      },
-    };
-
-    if (inputs.has_continuation()) {
-        const auto & msg = inputs.continue_msg;
-
-        data.generation_prompt = "<|start|>assistant to=self<|message|>" + msg.reasoning_content;
-        if (inputs.continue_final_message == COMMON_CHAT_CONTINUATION_CONTENT) {
-            data.generation_prompt += "<|eom|><|start|>assistant to=user<|message|>" + msg.render_content();
-        }
-
-        data.prompt += data.generation_prompt;
-    }
-
-    auto extract_reasoning = inputs.reasoning_format != COMMON_REASONING_FORMAT_NONE;
-
-    auto has_tools = inputs.tools.is_array() && !inputs.tools.empty();
-    // Constrained grammar whenever tools are offered.
-    auto include_grammar = has_tools && inputs.tool_choice != COMMON_CHAT_TOOL_CHOICE_NONE;
-
-    auto parser = build_chat_peg_parser([&](common_chat_peg_builder & p) {
-        auto start = p.rule("start", p.literal("<|start|>assistant"));
-
-        if (!extract_reasoning && !include_grammar) {
-            return start + p.content(p.rest());
-        }
-
-        if (extract_reasoning) {
-            p.rule("analysis", p.literal(" to=self<|message|>") + p.reasoning(p.until("<|eom|>")) + p.literal("<|eom|>"));
-        } else {
-            p.rule("analysis", p.literal(" to=self<|message|>") + p.content(p.until("<|eom|>")) + p.literal("<|eom|>"));
-        }
-        auto analysis = p.ref("analysis");
-
-        auto recipient  = p.optional(p.literal(" to=user"));
-        auto final_msg  = p.rule("final", recipient + p.literal("<|message|>") +
-                                              p.content(p.until_one_of({ "<|eot|>", "<|eom|>" })));
-
-        if (has_tools && inputs.tool_choice != COMMON_CHAT_TOOL_CHOICE_NONE) {
-            auto string_value = p.ac(
-                p.tool_arg_string_value(p.until("</atem:parameter>")) + p.tool_arg_close(p.literal("</atem:parameter>")),
-                "</atem:parameter>");
-
-            auto tool_choice = p.choice();
-            foreach_function(inputs.tools, [&](const json & tool) {
-                const auto &      function = tool.at("function");
-                const std::string name     = function.at("name");
-                auto              params   = function.contains("parameters") ? function.at("parameters") : json::object();
-
-                auto args = p.eps();
-                if (params.contains("properties") && params.at("properties").is_object() && !params.at("properties").empty()) {
-                    auto schema_info = common_schema_info();
-                    schema_info.resolve_refs(params);
-
-                    auto arg_choice = p.choice();
-                    for (const auto & [prop_name, prop_schema] : params.at("properties").items()) {
-                        auto value_parser = p.eps();
-                        if (schema_info.resolves_to_string(prop_schema)) {
-                            value_parser = string_value;
-                        } else {
-                            value_parser = p.tool_arg_json_value(
-                                    p.schema(p.json(), "tool-" + name + "-arg-" + prop_name + "-schema", prop_schema, false))
-                                + p.tool_arg_close(p.literal("</atem:parameter>"));
-                        }
-
-                        auto arg_rule = p.tool_arg(
-                            p.tool_arg_open(p.literal("<atem:parameter name=\"") + p.tool_arg_name(p.literal(prop_name)) + p.literal("\">")) +
-                            value_parser);
-
-                        arg_choice |= arg_rule;
-                    }
-                    args = p.zero_or_more(arg_choice + p.space());
-                }
-
-                auto tool_parser = p.tool(
-                    p.tool_open(p.literal(" to=") + p.until("<|message|>") +
-                                p.literal("<|message|><atem:function_calls>") + p.space() +
-                                p.literal("<atem:invoke name=\"") + p.tool_name(p.literal(name)) + p.literal("\">") + p.space())
-                    << p.tool_args(args)
-                    << p.tool_close(p.literal("</atem:invoke>") + p.space() + p.literal("</atem:function_calls>")));
-
-                tool_choice |= p.rule("tool-" + name, tool_parser);
-            });
-
-            auto tool_calls = inputs.parallel_tool_calls
-                ? p.trigger_rule("tool-call", tool_choice + p.zero_or_more(p.literal("<|eom|>") + start + tool_choice))
-                : p.trigger_rule("tool-call", tool_choice);
-
-
-            if (inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_REQUIRED) {
-                return p.zero_or_more(start + analysis) + start + tool_calls;
-            }
-            auto trailing_calls = p.optional(p.literal("<|eom|>") + start + tool_calls);
-            return p.zero_or_more(start + analysis) + start + (tool_calls | (final_msg + trailing_calls));
-        }
-
-        return p.zero_or_more(start + analysis) + start + final_msg;
-    });
-
-    data.parser = parser.save();
-
-    if (include_grammar) {
-        data.grammar_lazy = inputs.tool_choice != COMMON_CHAT_TOOL_CHOICE_REQUIRED;
-        data.grammar      = build_grammar([&](const common_grammar_builder & builder) {
-            foreach_function(inputs.tools, [&](const json & tool) {
-                const auto & function = tool.at("function");
-                auto         schema   = function.contains("parameters") ? function.at("parameters") : json::object();
-                builder.resolve_refs(schema);
-            });
-            parser.build_grammar(builder, data.grammar_lazy);
-        });
-        data.grammar_triggers = {
-            { COMMON_GRAMMAR_TRIGGER_TYPE_PATTERN,
-              "<\\|start\\|>assistant( to=(?!self<\\|message\\|>)(?!user<\\|message\\|>)[^<]*?<\\|message\\|>)" },
-        };
-    }
-
-    return data;
-}
-
 static json common_chat_extra_context() {
    json ctx = json::object();
    std::chrono::system_clock::time_point now = std::chrono::system_clock::now();
@@ -3490,12 +3114,6 @@ std::optional<common_chat_params> common_chat_try_specialized_template(
        return common_chat_params_init_gpt_oss(tmpl, params);
    }

-    // Muse Glimmer format using " to=<recipient>" recipients and <|eom|>/<|eot|> message terminators.
-    if (src.find("<atem:function_calls>") != std::string::npos && src.find("<|eom|>") != std::string::npos) {
-        LOG_DBG("Using specialized template: Muse Glimmer\n");
-        return common_chat_params_init_muse_glimmer(tmpl, params);
-    }
-
    // Functionary v3.2 - uses recipient-based format with >>>recipient\n{content}
    // Detection: template has ">>>all" for content and ">>>" prefix for tool calls
    if (src.find(">>>all") != std::string::npos && src.find(">>>${recipient}") != std::string::npos) {
@@ -3511,13 +3129,6 @@ std::optional<common_chat_params> common_chat_try_specialized_template(
        return common_chat_params_init_kimi_k2(tmpl, params);
    }

-    // Kimi K3 - the <|open|>/<|close|>/<|end_of_msg|> markers are unique to it
-    if (src.find("<|open|>") != std::string::npos && src.find("<|close|>") != std::string::npos &&
-        src.find("<|end_of_msg|>") != std::string::npos) {
-        LOG_DBG("Using specialized template: Kimi K3\n");
-        return common_chat_params_init_kimi_k3(tmpl, params);
-    }
-
    // Cohere2 MoE / North Code - marker-wrapped format with <|START_TEXT|> content and
    // <|START_ACTION|> JSON tool calls. <|START_TEXT|> is unique to this template (the older
    // Command-R templates use <|START_RESPONSE|>).
@@ -1019,21 +1019,20 @@ std::string fs_get_cache_directory() {
    std::string cache_directory = "";
    auto ensure_trailing_slash = [](std::string p) {
        // Make sure to add trailing slash
-        if (p.empty() || p.back() != DIRECTORY_SEPARATOR) {
+        if (p.back() != DIRECTORY_SEPARATOR) {
            p += DIRECTORY_SEPARATOR;
        }
        return p;
    };
-    cache_directory = common_get_env("LLAMA_CACHE");
-    if (cache_directory.empty()) {
+    if (getenv("LLAMA_CACHE")) {
+        cache_directory = std::getenv("LLAMA_CACHE");
+    } else {
 #if defined(__linux__) || defined(__FreeBSD__) || defined(_AIX) || \
        defined(__OpenBSD__) || defined(__NetBSD__)
-        const std::string xdg_cache_home = common_get_env("XDG_CACHE_HOME");
-        const std::string home           = common_get_env("HOME");
-        if (!xdg_cache_home.empty()) {
-            cache_directory = xdg_cache_home;
-        } else if (!home.empty()) {
-            cache_directory = home + "/.cache/";
+        if (std::getenv("XDG_CACHE_HOME")) {
+            cache_directory = std::getenv("XDG_CACHE_HOME");
+        } else if (std::getenv("HOME")) {
+            cache_directory = std::getenv("HOME") + std::string("/.cache/");
        } else {
 #if defined(__linux__)
            /* no $HOME is defined, fallback to getpwuid */
@@ -1048,16 +1047,9 @@ std::string fs_get_cache_directory() {
 #endif /* defined(__linux__) */
        }
 #elif defined(__APPLE__)
-        cache_directory = common_get_env("HOME");
-        if (cache_directory.empty()) {
-            throw std::runtime_error("Failed to find $HOME directory");
-        }
-        cache_directory += "/Library/Caches/";
+        cache_directory = std::getenv("HOME") + std::string("/Library/Caches/");
 #elif defined(_WIN32)
-        cache_directory = common_get_env("LOCALAPPDATA");
-        if (cache_directory.empty()) {
-            throw std::runtime_error("Failed to find %LOCALAPPDATA% directory");
-        }
+        cache_directory = std::getenv("LOCALAPPDATA");
 #elif defined(__EMSCRIPTEN__)
        GGML_ABORT("not implemented on this platform");
 #else
@@ -1069,51 +1061,6 @@ std::string fs_get_cache_directory() {
    return ensure_trailing_slash(cache_directory);
 }

-std::string fs_get_config_directory() {
-    std::string config_directory = "";
-    auto ensure_trailing_slash = [](std::string p) {
-        if (p.empty() || p.back() != DIRECTORY_SEPARATOR) {
-            p += DIRECTORY_SEPARATOR;
-        }
-        return p;
-    };
-#if defined(__linux__) || defined(__FreeBSD__) || defined(_AIX) || \
-        defined(__OpenBSD__) || defined(__NetBSD__) || defined(__APPLE__)
-    const std::string xdg_config_home = common_get_env("XDG_CONFIG_HOME");
-    const std::string home            = common_get_env("HOME");
-    if (!xdg_config_home.empty()) {
-        config_directory = xdg_config_home;
-    } else if (!home.empty()) {
-        config_directory = home + "/.config/";
-    } else {
-#if defined(__linux__)
-        /* no $HOME is defined, fallback to getpwuid */
-        struct passwd *pw = getpwuid(getuid());
-        if ((!pw) || (!pw->pw_dir)) {
-            throw std::runtime_error("Failed to find $HOME directory");
-        }
-
-        config_directory = std::string(pw->pw_dir) + std::string("/.config/");
-#else
-        throw std::runtime_error("Failed to find $HOME directory");
-#endif
-    }
-#elif defined(_WIN32)
-    config_directory = common_get_env("APPDATA");
-    if (config_directory.empty()) {
-        throw std::runtime_error("Failed to find %APPDATA% directory");
-    }
-#elif defined(__EMSCRIPTEN__)
-    // caller decides what to do when there is no config directory
-    throw std::runtime_error("not implemented on this platform");
-#else
-#  error Unknown architecture
-#endif
-    config_directory = ensure_trailing_slash(config_directory);
-    config_directory += "llama.cpp";
-    return ensure_trailing_slash(config_directory);
-}
-
 std::string fs_get_cache_file(const std::string & filename) {
    GGML_ASSERT(filename.find(DIRECTORY_SEPARATOR) == std::string::npos);
    std::string cache_directory = fs_get_cache_directory();
@@ -1275,8 +1222,6 @@ struct common_init_result::impl {

    // note: the order in which model, context, etc. are declared matters because their destructors will be called bottom-to-top

-    common_threadpools threadpools;
-
    llama_model_ptr   model;
    llama_context_ptr context;

@@ -1378,10 +1323,6 @@ common_init_result::common_init_result(common_params & params, bool model_only)
    }

    pimpl->context.reset(lctx);
-
-    set_process_priority(params.cpuparams.priority);
-
-    pimpl->threadpools.init(lctx, params);
 }

 llama_model * common_init_result::model() {
@@ -1698,7 +1639,6 @@ struct llama_context_params common_context_params_to_llama(const common_params &
    cparams.n_seq_max         = params.n_parallel;
    cparams.n_rs_seq          = params.speculative.need_n_rs_seq();
    cparams.n_outputs_max     = std::max(params.n_outputs_max, 0);
-    cparams.n_outputs_max_per_seq = std::max(params.n_outputs_max_per_seq, 0);
    cparams.n_batch           = params.n_batch;
    cparams.n_ubatch          = params.n_ubatch;
    cparams.n_threads         = params.cpuparams.n_threads;
@@ -1730,10 +1670,6 @@ struct llama_context_params common_context_params_to_llama(const common_params &
    return cparams;
 }

-//
-// Threadpool utils
-//
-
 struct ggml_threadpool_params ggml_threadpool_params_from_cpu_params(const common_cpu_params & params) {
    struct ggml_threadpool_params tpp;

@@ -1750,56 +1686,6 @@ struct ggml_threadpool_params ggml_threadpool_params_from_cpu_params(const commo
    return tpp;
 }

-common_threadpools::~common_threadpools() {
-    if (!free_fn) {
-        return;
-    }
-    free_fn(threadpool);
-    free_fn(threadpool_batch);
-}
-
-void common_threadpools::init(llama_context * ctx, const common_params & params) {
-    GGML_ASSERT(!threadpool);
-    GGML_ASSERT(!threadpool_batch);
-
-    COM_INF("llama threadpool init, n_threads = %d\n", (int) params.cpuparams.n_threads);
-
-    auto * cpu_dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_CPU);
-    if (!cpu_dev) {
-        COM_WRN("%s", "no CPU backend found\n");
-        return;
-    }
-    auto * reg = ggml_backend_dev_backend_reg(cpu_dev);
-    auto * ggml_threadpool_new_fn = (decltype(ggml_threadpool_new) *) ggml_backend_reg_get_proc_address(reg, "ggml_threadpool_new");
-    free_fn = (decltype(ggml_threadpool_free) *) ggml_backend_reg_get_proc_address(reg, "ggml_threadpool_free");
-
-    struct ggml_threadpool_params tpp_batch =
-            ggml_threadpool_params_from_cpu_params(params.cpuparams_batch);
-    struct ggml_threadpool_params tpp =
-            ggml_threadpool_params_from_cpu_params(params.cpuparams);
-
-    if (!ggml_threadpool_params_match(&tpp, &tpp_batch)) {
-        threadpool_batch = ggml_threadpool_new_fn(&tpp_batch);
-        if (!threadpool_batch) {
-            COM_WRN("batch threadpool create failed : n_threads %d\n", tpp_batch.n_threads);
-            return;
-        }
-
-        // start the non-batch threadpool in the paused state
-        tpp.paused = true;
-    }
-
-    threadpool = ggml_threadpool_new_fn(&tpp);
-    if (!threadpool) {
-        COM_WRN("threadpool create failed : n_threads %d\n", tpp.n_threads);
-        free_fn(threadpool_batch);
-        threadpool_batch = nullptr;
-        return;
-    }
-
-    llama_attach_threadpool(ctx, threadpool, threadpool_batch);
-}
-
 //
 // Batch utils
 //
@@ -447,7 +447,6 @@ struct common_params {
    int32_t n_parallel            =     1; // number of parallel sequences to decode
    int32_t n_sequences           =     1; // number of sequences to decode
    int32_t n_outputs_max         =     0; // max outputs in a batch (0 = n_batch)
-    int32_t n_outputs_max_per_seq =     1; // max outputs per sequence
    int32_t grp_attn_n            =     1; // group-attention factor
    int32_t grp_attn_w            =   512; // group-attention width
    int32_t n_print               =    -1; // print token count every n tokens (-1 = disabled)
@@ -473,7 +472,7 @@ struct common_params {
    std::vector<size_t> fit_params_target = std::vector<size_t>(llama_max_devices(), 1024 * 1024*1024);

    enum llama_split_mode split_mode = LLAMA_SPLIT_MODE_LAYER; // how to split the model across GPUs
-    enum llama_load_mode  load_mode  = LLAMA_LOAD_MODE_AUTO; // how to load the model
+    enum llama_load_mode  load_mode  = LLAMA_LOAD_MODE_MMAP; // how to load the model

    common_cpu_params cpuparams;
    common_cpu_params cpuparams_batch;
@@ -656,7 +655,6 @@ struct common_params {

    // enable built-in tools
    std::vector<std::string> server_tools;
-    std::string server_tools_runtime;

    // MCP server configs (Cursor-compatible JSON)
    std::string mcp_servers_config;   // path to JSON file with MCP server definitions
@@ -881,7 +879,6 @@ bool fs_is_directory(const std::string & path);

 std::string fs_get_cache_directory();
 std::string fs_get_cache_file(const std::string & filename);
-std::string fs_get_config_directory();

 struct common_file_info {
    std::string path;
@@ -929,8 +926,9 @@ using common_init_result_ptr = std::unique_ptr<common_init_result>;

 common_init_result_ptr common_init_from_params(common_params & params, bool model_only = false);

-struct llama_model_params   common_model_params_to_llama  (      common_params & params);
-struct llama_context_params common_context_params_to_llama(const common_params & params);
+struct llama_model_params     common_model_params_to_llama  (      common_params & params);
+struct llama_context_params   common_context_params_to_llama(const common_params & params);
+struct ggml_threadpool_params ggml_threadpool_params_from_cpu_params(const common_cpu_params & params);

 // clear LoRA adapters from context, then apply new list of adapters
 void common_set_adapter_lora(struct llama_context * ctx, std::vector<common_adapter_lora_info> & lora);
@@ -941,28 +939,6 @@ std::string common_get_model_endpoint();
 // for testing purposes
 char * common_get_model_or_exit(int, char*[]);

-//
-// Threadpool utils
-//
-
-struct ggml_threadpool_params ggml_threadpool_params_from_cpu_params(const common_cpu_params & params);
-
-struct common_threadpools {
-    common_threadpools() = default;
-    ~common_threadpools();
-
-    common_threadpools(const common_threadpools &) = delete;
-    common_threadpools & operator=(const common_threadpools &) = delete;
-
-    void init(llama_context * ctx, const common_params & params);
-
-private:
-    ggml_threadpool * threadpool       = nullptr;
-    ggml_threadpool * threadpool_batch = nullptr;
-
-    decltype(ggml_threadpool_free) * free_fn = nullptr;
-};
-
 //
 // Context utils
 //
@@ -136,10 +136,7 @@ static std::vector<llama_device_memory_data> common_get_device_memory_data_impl(
        devs.push_back(llama_model_get_device(model, i));
    }

-    hp_ngl         = llama_model_n_layer(model);
-    if (mparams->load_mtp) {
-        hp_ngl    += llama_model_n_layer_nextn(model);
-    }
+    hp_ngl         = llama_model_n_layer(model) + llama_model_n_layer_nextn(model);
    hp_n_ctx_train = llama_model_n_ctx_train(model);
    hp_n_expert    = llama_model_n_expert(model);

@@ -102,8 +102,7 @@ bool common_imatrix_load(const std::string & fname, common_imatrix & imatrix) {
    const int64_t chunk_count_key = gguf_find_key(ctx_gguf, LLM_KV_IMATRIX_CHUNK_COUNT);
    const int64_t chunk_size_key  = gguf_find_key(ctx_gguf, LLM_KV_IMATRIX_CHUNK_SIZE);

-    if (datasets_key != -1 && gguf_get_kv_type(ctx_gguf, datasets_key) == GGUF_TYPE_ARRAY &&
-        gguf_get_arr_type(ctx_gguf, datasets_key) == GGUF_TYPE_STRING) {
+    if (datasets_key != -1 && gguf_get_arr_type(ctx_gguf, datasets_key) == GGUF_TYPE_STRING) {
        const int64_t n = gguf_get_arr_n(ctx_gguf, datasets_key);
        imatrix.datasets.reserve(imatrix.datasets.size() + n);
        for (int64_t i = 0; i < n; ++i) {
@@ -144,13 +143,6 @@ bool common_imatrix_load(const std::string & fname, common_imatrix & imatrix) {
            return false;
        }

-        if (in_sum2->type != GGML_TYPE_F32 || counts->type != GGML_TYPE_F32) {
-            LOG_ERR("%s: sums and counts for %s must be F32\n", __func__, name.c_str());
-            gguf_free(ctx_gguf);
-            ggml_free(ctx);
-            return false;
-        }
-
        auto & e = imatrix.entries[name];

        const int64_t nval    = ggml_nelements(in_sum2);
@@ -17,19 +17,13 @@ namespace jinja {

 using caps_json_fn = std::function<json()>;
 using caps_ctx_fn = std::function<void(context &)>;
-using caps_analyze_fn = std::function<void(context &, bool, value &, value &, const std::string &)>;
+using caps_analyze_fn = std::function<void(bool, value &, value &, const std::string &)>;

 void caps_apply_preserve_reasoning(jinja::context & ctx, bool enabled) {
    ctx.set_val("preserve_thinking",         mk_val<value_bool>(enabled));
    ctx.set_val("clear_thinking",            mk_val<value_bool>(!enabled));
    ctx.set_val("truncate_history_thinking", mk_val<value_bool>(!enabled));
-    ctx.set_val("drop_thinking",             mk_val<value_bool>(!enabled));
-}
-
-void caps_apply_reasoning_effort(jinja::context & ctx, const std::string & effort) {
-    value var = mk_val<value_string>(effort); // bind to the same value for stats
-    ctx.set_val("reasoning_effort",   var);
-    ctx.set_val("reasoning_strength", var);
+    ctx.set_val("drop_thinking",            mk_val<value_bool>(!enabled));
 }

 static void caps_try_execute(jinja::program & prog,
@@ -68,7 +62,7 @@ static void caps_try_execute(jinja::program & prog,
        // ignore exceptions during capability analysis
    }

-    analyze_fn(ctx, success, messages, tools, result);
+    analyze_fn(success, messages, tools, result);
 }

 // for debugging only
@@ -93,7 +87,6 @@ std::map<std::string, bool> caps::to_map() const {
        {"supports_parallel_tool_calls", supports_parallel_tool_calls},
        {"supports_system_role", supports_system_role},
        {"supports_preserve_reasoning", supports_preserve_reasoning},
-        {"supports_reasoning_effort", supports_reasoning_effort},
        {"supports_object_arguments", supports_object_arguments},
    };
 }
@@ -117,8 +110,6 @@ caps caps_get(jinja::program & prog) {

    JJ_DEBUG("%s\n", ">>> Running capability check: typed content");

-    static const std::string content_marker = "STRING_MARKER";
-
    // case: typed content support
    caps_try_execute(
        prog,
@@ -127,26 +118,22 @@ caps caps_get(jinja::program & prog) {
            return json::array({
                {
                    {"role", "user"},
-                    {"content", content_marker}
+                    {"content", "content"}
                }
            });
        },
        nullptr, // ctx_fn
        nullptr, // tools_fn
-        [&](context &, bool success, value & messages, value &, const std::string & rendered) {
+        [&](bool success, value & messages, value &, const std::string &) {
            auto & content = messages->at(0)->at("content");
            caps_print_stats(content, "messages[0].content");
-            bool used_as_array = has_op(content, "selectattr") || has_op(content, "array_access");
-            if (used_as_array) {
+            if (has_op(content, "selectattr") || has_op(content, "array_access")) {
                // accessed as an array
                result.supports_typed_content = true;
            }
            if (!success) {
                // failed to execute with content as string
                result.supports_string_content = false;
-            } else if (used_as_array && rendered.find(content_marker) == std::string::npos) {
-                // edge case: string may be accessed for checking, but does not appear in the output
-                result.supports_string_content = false;
            }
        }
    );
@@ -171,7 +158,7 @@ caps caps_get(jinja::program & prog) {
        },
        nullptr, // ctx_fn
        nullptr, // tools_fn
-        [&](context &, bool, value & messages, value &, const std::string &) {
+        [&](bool, value & messages, value &, const std::string &) {
            auto & content = messages->at(0)->at("content");
            caps_print_stats(content, "messages[0].content");
            if (!content->stats.used) {
@@ -247,7 +234,7 @@ caps caps_get(jinja::program & prog) {
                },
            });
        },
-        [&](context &, bool success, value & messages, value & tools, const std::string &) {
+        [&](bool success, value & messages, value & tools, const std::string &) {
            if (!success) {
                return; // Nothing can be inferred
            }
@@ -340,7 +327,7 @@ caps caps_get(jinja::program & prog) {
                    },
                });
            },
-            [&](context &, bool success, value & messages, value & tools, const std::string &) {
+            [&](bool success, value & messages, value & tools, const std::string &) {
                if (!success) {
                    result.supports_tool_calls = false;
                    result.supports_tools = false;
@@ -442,7 +429,7 @@ caps caps_get(jinja::program & prog) {
                },
            });
        },
-        [&](context &, bool success, value & messages, value &, const std::string &) {
+        [&](bool success, value & messages, value &, const std::string &) {
            if (!success) {
                result.supports_parallel_tool_calls = false;
                return;
@@ -499,7 +486,7 @@ caps caps_get(jinja::program & prog) {
            caps_apply_preserve_reasoning(ctx, true);
        },
        nullptr, // tools_fn
-        [&](context &, bool, value &, value &, const std::string & output) {
+        [&](bool, value &, value &, const std::string & output) {
            // note: we cannot use stats here because the reasoning_content may be used for "if" condition test, but not actually outputted in the final result
            if (output.find(reasoning_placeholder) != std::string::npos) {
                result.supports_preserve_reasoning = true;
@@ -507,32 +494,6 @@ caps caps_get(jinja::program & prog) {
        }
    );

-    JJ_DEBUG("%s\n", ">>> Running capability check: reasoning effort");
-
-    // case: reasoning effort level
-    caps_try_execute(
-        prog,
-        [&]() {
-            // messages
-            return json::array({
-                {
-                    {"role", "user"},
-                    {"content", "User message"}
-                },
-            });
-        },
-        [&](context & ctx) {
-            ctx.set_val("enable_thinking", mk_val<value_bool>(true));
-            caps_apply_reasoning_effort(ctx, "low");
-        },
-        nullptr, // tools_fn
-        [&](context & ctx, bool, value &, value &, const std::string &) {
-            value effort = ctx.get_val("reasoning_effort");
-            caps_print_stats(effort, "reasoning_effort");
-            result.supports_reasoning_effort = effort->stats.used;
-        }
-    );
-
    JJ_DEBUG("%s\n", result.to_string().c_str());

    return result;
@@ -16,9 +16,6 @@ struct caps {
    // supports preserve reasoning trace in the full history, not just the last assistant message
    bool supports_preserve_reasoning = false;

-    // supports reasoning effort levels
-    bool supports_reasoning_effort = false;
-
    // one of the 2 content capabilities must be true
    bool supports_string_content = true;
    bool supports_typed_content = false;
@@ -35,6 +32,5 @@ struct caps {
 caps caps_get(jinja::program & prog);

 void caps_apply_preserve_reasoning(jinja::context & ctx, bool enabled);
-void caps_apply_reasoning_effort(jinja::context & ctx, const std::string & effort);

 } // namespace jinja
@@ -263,7 +263,7 @@ value binary_expression::execute_impl(context & ctx) {
            return res;
        }
        for (int64_t i = 0; i < repeat; ++i) {
-            res->val_str.append(str);
+            res->val_str = res->val_str.append(str);
        }
        return res;
    }
@@ -763,22 +763,14 @@ struct runtime {
        gather_string_parts_recursive(val, parts);
        // join consecutive parts with the same type
        auto & p = parts->val_str.parts;
-        if (p.empty()) {
-            return parts;
-        }
-        size_t w = 0;
-        for (size_t r = 1; r < p.size(); r++) {
-            if (p[w].is_input == p[r].is_input) {
-                p[w].val += p[r].val;
+        for (size_t i = 1; i < p.size(); ) {
+            if (p[i].is_input == p[i - 1].is_input) {
+                p[i - 1].val += p[i].val;
+                p.erase(p.begin() + i);
            } else {
-                w++;
-                if (w != r) {
-                    // the guard is needed, self-move leaves the string in an unspecified state
-                    p[w] = std::move(p[r]);
-                }
+                i++;
            }
        }
-        p.resize(w + 1);
        return parts;
    }

@@ -103,7 +103,7 @@ void string::mark_input_based_on(const string & other) {
    }
 }

-string & string::append(const string & other) {
+string string::append(const string & other) {
    for (const auto & part : other.parts) {
        parts.push_back(part);
    }
@@ -47,7 +47,7 @@ struct string {
    // mark this string as input if other has ALL parts as input
    void mark_input_based_on(const string & other);

-    string & append(const string & other);
+    string append(const string & other);

    // in-place transformations

@@ -116,8 +116,6 @@ static llama_sampler_i llama_sampler_llg_i = {
    /* .backend_accept    = */ NULL,
    /* .backend_apply     = */ NULL,
    /* .backend_set_input = */ NULL,
-    /* .backend_reset     = */ NULL,
-    /* .copy_state        = */ NULL,
 };

 static size_t llama_sampler_llg_tokenize_fn(const void * user_data, const uint8_t * bytes, size_t bytes_len,
@@ -570,34 +570,23 @@ struct parser_executor {
    }

    static common_peg_parse_result handle_escape_sequence(common_peg_parse_context & ctx, size_t start, size_t & pos, const char delimiter) {
-        auto save = pos;
-
        ++pos; // consume '\'
        if (pos >= ctx.input.size()) {
            if (!ctx.is_lenient()) {
                return common_peg_parse_result(COMMON_PEG_PARSE_RESULT_FAIL, start);
            }
-            pos = save; // suppress unmatched '\'
            return common_peg_parse_result(COMMON_PEG_PARSE_RESULT_NEED_MORE_INPUT, start, pos);
        }

        char c = ctx.input[pos];
-
        if (c == delimiter || c == '\\' || c == '/' || c == 'b' || c == 'f' || c == 'n' || c == 'r' || c == 't') {
            ++pos;
            return common_peg_parse_result(COMMON_PEG_PARSE_RESULT_SUCCESS, start, pos);
+        } else if (c == 'u') {
+            return handle_unicode_escape(ctx, start, pos);
+        } else {
+            return common_peg_parse_result(COMMON_PEG_PARSE_RESULT_FAIL, start);
        }
-
-        if (c == 'u') {
-            auto result = handle_unicode_escape(ctx, start, pos);
-            if (result.need_more_input()) {
-                pos = save; // suppress incomplete sequence
-                return common_peg_parse_result(COMMON_PEG_PARSE_RESULT_NEED_MORE_INPUT, start, pos);
-            }
-            return result;
-        }
-
-        return common_peg_parse_result(COMMON_PEG_PARSE_RESULT_FAIL, start);
    }

    static common_peg_parse_result handle_unicode_escape(common_peg_parse_context & ctx, size_t start, size_t & pos) {
@@ -322,8 +322,6 @@ common_presets common_preset_context::load_from_ini(const std::string & path, co
                    preset.options[opt] = value;
                }
                LOG_DBG("accepted option: %s = %s\n", key.c_str(), preset.options[opt].c_str());
-            } else if (ignore_unknown_keys) {
-                LOG_WRN("ignoring option '%s' from %s: not supported by this program\n", key.c_str(), path.c_str());
            } else {
                throw std::runtime_error(string_format(
                    "option '%s' not recognized in preset '%s'",
@@ -365,25 +363,8 @@ struct local_model {
    std::string name;
    std::string path;
    std::string path_mmproj;
-    std::string path_draft;
 };

-// TODO @ngxson: handle "eagle3-" when it's supported by common_speculative_types_from_gguf()
-static const char * draft_prefixes[] = { "mtp-", "dspark-", "dflash-" };
-
-static bool is_mmproj_file(const std::string & fname) {
-    return fname.find("mmproj") != std::string::npos;
-}
-
-static bool is_draft_file(const std::string & fname) {
-    for (const auto & prefix : draft_prefixes) {
-        if (fname.rfind(prefix, 0) == 0) {
-            return true;
-        }
-    }
-    return false;
-}
-
 common_presets common_preset_context::load_from_models_dir(const std::string & models_dir) const {
    if (!std::filesystem::exists(models_dir) || !std::filesystem::is_directory(models_dir)) {
        throw std::runtime_error(string_format("error: '%s' does not exist or is not a directory\n", models_dir.c_str()));
@@ -395,15 +376,10 @@ common_presets common_preset_context::load_from_models_dir(const std::string & m
        common_file_info model_file;
        common_file_info first_shard_file;
        common_file_info mmproj_file;
-        common_file_info draft_file;
        for (const auto & file : files) {
            if (string_ends_with(file.name, ".gguf")) {
-                if (is_mmproj_file(file.name)) {
+                if (file.name.find("mmproj") != std::string::npos) {
                    mmproj_file = file;
-                } else if (is_draft_file(file.name)) {
-                    if (draft_file.path.empty()) {
-                        draft_file = file; // first sidecar found wins
-                    }
                } else if (file.name.find("-00001-of-") != std::string::npos) {
                    first_shard_file = file;
                } else {
@@ -415,8 +391,7 @@ common_presets common_preset_context::load_from_models_dir(const std::string & m
        local_model model{
            /* name        */ name,
            /* path        */ first_shard_file.path.empty() ? model_file.path : first_shard_file.path,
-            /* path_mmproj */ mmproj_file.path, // can be empty
-            /* path_draft  */ draft_file.path   // can be empty
+            /* path_mmproj */ mmproj_file.path // can be empty
        };
        if (!model.path.empty()) {
            models.push_back(model);
@@ -428,17 +403,13 @@ common_presets common_preset_context::load_from_models_dir(const std::string & m
        if (file.is_dir) {
            scan_subdir(file.path, file.name);
        } else if (string_ends_with(file.name, ".gguf")) {
-            if (is_mmproj_file(file.name) || is_draft_file(file.name)) {
-                continue; // companion file, cannot be loaded as a model on its own
-            }
            // single file model
            std::string name = file.name;
            string_replace_all(name, ".gguf", "");
            local_model model{
                /* name        */ name,
                /* path        */ file.path,
-                /* path_mmproj */ "",
-                /* path_draft  */ ""
+                /* path_mmproj */ ""
            };
            models.push_back(model);
        }
@@ -453,9 +424,6 @@ common_presets common_preset_context::load_from_models_dir(const std::string & m
        if (!model.path_mmproj.empty()) {
            preset.set_option(*this, "LLAMA_ARG_MMPROJ", model.path_mmproj);
        }
-        if (!model.path_draft.empty()) {
-            preset.set_option(*this, "LLAMA_ARG_SPEC_DRAFT_MODEL", model.path_draft);
-        }
        out[preset.name] = preset;
    }

@@ -59,10 +59,6 @@ struct common_preset_context {
    bool filter_allowed_keys = false;
    std::set<std::string> allowed_keys;

-    // if true, options unknown to the current example are skipped instead of being an error
-    // used for config files shared by all binaries, where each binary only knows a subset of options
-    bool ignore_unknown_keys = false;
-
    // if only_remote_allowed is true, only accept whitelisted keys
    common_preset_context(llama_example ex);

@@ -217,8 +217,6 @@ static struct llama_sampler_i common_reasoning_budget_i = {
    /* .backend_accept    = */ nullptr,
    /* .backend_apply     = */ nullptr,
    /* .backend_set_input = */ nullptr,
-    /* .backend_reset     = */ nullptr,
-    /* .copy_state        = */ nullptr,
 };

 static struct llama_sampler * common_reasoning_budget_clone(const struct llama_sampler * smpl) {
@@ -518,26 +518,6 @@ struct common_sampler * common_sampler_clone(common_sampler * gsmpl) {
    };
 }

-void common_sampler_copy(const common_sampler * src, common_sampler * dst) {
-    if (!src || !dst || src == dst) {
-        return;
-    }
-
-    GGML_ASSERT((src->grmr == nullptr) == (dst->grmr == nullptr));
-    GGML_ASSERT((src->rbudget == nullptr) == (dst->rbudget == nullptr));
-
-    llama_sampler_copy(src->grmr,    dst->grmr);
-    llama_sampler_copy(src->rbudget, dst->rbudget);
-    llama_sampler_copy(src->chain,   dst->chain);
-
-    dst->params     = src->params;
-    dst->prev       = src->prev;
-    dst->cur        = src->cur;
-    dst->cur_p      = src->cur_p;
-    dst->cur_p.data = src->cur_p.data ? dst->cur.data() : nullptr; // re-point to dst's buffer
-    dst->t_total_us = src->t_total_us;
-}
-
 void common_perf_print(const struct llama_context * ctx, const struct common_sampler * gsmpl) {
    // TODO: measure grammar performance

@@ -47,7 +47,6 @@ void common_sampler_free(struct common_sampler * gsmpl);
 void                    common_sampler_accept(struct common_sampler * gsmpl, llama_token token, bool is_generated);
 void                    common_sampler_reset (struct common_sampler * gsmpl);
 struct common_sampler * common_sampler_clone (struct common_sampler * gsmpl);
-void                    common_sampler_copy  (const struct common_sampler * src, struct common_sampler * dst);

 // arguments can be nullptr to skip printing
 void common_perf_print(const struct llama_context * ctx, const struct common_sampler * gsmpl);
@@ -2,7 +2,6 @@

 #include "common.h"
 #include "ggml.h"
-#include "ggml-cpp.h"
 #include "llama.h"
 #include "log.h"
 #include "ngram-cache.h"
@@ -172,6 +171,12 @@ struct common_speculative_impl {
    // (optional) serialize/restore per-seq internal state (e.g. eagle3's deferred boundary).
    virtual bool get_state(llama_seq_id /*seq_id*/, std::vector<uint8_t> & /*data*/) const { return false; }
    virtual void set_state(llama_seq_id /*seq_id*/, const std::vector<uint8_t> & /*data*/) {}
+
+    // true if this implementation requires the target context to extract post-norm embeddings
+    virtual bool need_embd() const = 0;
+
+    // true if this implementation requires the target context to extract pre-norm embeddings
+    virtual bool need_embd_nextn() const { return false; }
 };

 struct common_speculative_impl_draft_simple : public common_speculative_impl {
@@ -188,10 +193,6 @@ struct common_speculative_impl_draft_simple : public common_speculative_impl {
        auto * ctx_dft = this->params.ctx_dft;
        auto * ctx_tgt = this->params.ctx_tgt;

-        if (!ctx_dft) {
-            throw std::runtime_error("draft-simple requires a draft context");
-        }
-
        SPC_TRC("%s", "adding speculative implementation 'draft-simple'\n");
        SPC_TRC("- n_max=%d, n_min=%d, p_min=%f\n", this->params.n_max, this->params.n_min, this->params.p_min);
        SPC_TRC("- gpu_layers=%d, cache_k=%s, cache_v=%s, ctx_tgt=%s, ctx_dft=%s, devices=[%s]\n",
@@ -384,6 +385,10 @@ struct common_speculative_impl_draft_simple : public common_speculative_impl {
    void accept(llama_seq_id /*seq_id*/, uint16_t /*n_accepted*/, bool /*is_other*/) override {
        // noop
    }
+
+    bool need_embd() const override {
+        return false;
+    }
 };


@@ -902,6 +907,10 @@ struct common_speculative_impl_draft_eagle3 : public common_speculative_impl {
        pending_g_last[seq_id].resize(n_embd_dec);
        std::memcpy(pending_g_last[seq_id].data(), data.data() + sizeof(llama_pos), (size_t) n_embd_dec * sizeof(float));
    }
+
+    bool need_embd() const override {
+        return false;
+    }
 };

 // DFlash: block-diffusion drafting with a draft-side KV cache injection
@@ -913,9 +922,6 @@ struct common_speculative_impl_draft_dflash : public common_speculative_impl {

    std::vector<common_sampler_ptr> smpls;

-    // backend sampler chain per seq, attached to ctx_dft
-    std::vector<llama_sampler *> backend_chains;
-
    int32_t n_embd_dec = 0;  // draft hidden size
    int32_t n_embd_enc = 0;  // target_layer_ids_n * target_hidden_size
    int32_t n_embd_tgt = 0;  // target model hidden size
@@ -989,22 +995,6 @@ struct common_speculative_impl_draft_dflash : public common_speculative_impl {
            s.reset(common_sampler_init(model_dft, sparams));
        }

-        // offload draft sampling to the backend
-        backend_chains.assign(n_seq, nullptr);
-        if (this->params.backend_sampling) {
-            for (llama_seq_id seq_id = 0; seq_id < (llama_seq_id) n_seq; ++seq_id) {
-                llama_sampler * chain = llama_sampler_chain_init(llama_sampler_chain_default_params());
-                llama_sampler_chain_add(chain, llama_sampler_init_top_k(10));
-
-                if (!llama_set_sampler(ctx_dft, seq_id, chain)) {
-                    SPC_WRN("backend offload failed for seq_id=%d; using CPU sampler\n", (int) seq_id);
-                    llama_sampler_free(chain);
-                    chain = nullptr;
-                }
-                backend_chains[seq_id] = chain;
-            }
-        }
-
        // turn on extraction of the target layers' input embeddings
        for (uint32_t k = 0; k < target_layer_ids_n; ++k) {
            llama_set_embeddings_layer_inp(ctx_tgt, (uint32_t) target_layer_ids[k], true);
@@ -1015,18 +1005,6 @@ struct common_speculative_impl_draft_dflash : public common_speculative_impl {
    }

    ~common_speculative_impl_draft_dflash() override {
-        auto * ctx_dft = this->params.ctx_dft;
-        for (llama_seq_id seq_id = 0; seq_id < (llama_seq_id) backend_chains.size(); ++seq_id) {
-            if (backend_chains[seq_id] == nullptr) {
-                continue;
-            }
-            if (ctx_dft) {
-                llama_set_sampler(ctx_dft, seq_id, nullptr);
-            }
-            llama_sampler_free(backend_chains[seq_id]);
-        }
-        backend_chains.clear();
-
        llama_batch_free(batch);
        llama_batch_free(batch_inject);
    }
@@ -1054,14 +1032,7 @@ struct common_speculative_impl_draft_dflash : public common_speculative_impl {
            return true;
        }

-        // Target prefill may contain token IDs or multimodal embeddings. Both
-        // produce the target-layer features used to seed the draft KV cache, so
-        // skipping the embedding batches leaves a hole in the draft's cache and
-        // the next injection fails to initialize.
-        // TODO: revisit after https://github.com/ggml-org/llama.cpp/pull/24669 is merged
-        const bool has_tokens     = batch_in.token != nullptr;
-        const bool has_embeddings = batch_in.embd  != nullptr;
-        if (has_tokens == has_embeddings) {
+        if (batch_in.token == nullptr || batch_in.embd != nullptr) {
            return true;
        }

@@ -1269,6 +1240,10 @@ struct common_speculative_impl_draft_dflash : public common_speculative_impl {
    void accept(llama_seq_id /*seq_id*/, uint16_t /*n_accepted*/, bool /*is_other*/) override {
        // noop
    }
+
+    bool need_embd() const override {
+        return false;
+    }
 };

 struct common_speculative_impl_draft_mtp : public common_speculative_impl {
@@ -1707,6 +1682,14 @@ struct common_speculative_impl_draft_mtp : public common_speculative_impl {
        const size_t row_bytes = (size_t) n_embd * sizeof(float);
        std::memcpy(pending_h[seq_id].data(), verify_h[seq_id].data() + (size_t) i_h * n_embd, row_bytes);
    }
+
+    bool need_embd() const override {
+        return false;
+    }
+
+    bool need_embd_nextn() const override {
+        return true;
+    }
 };

 // state of self-speculation (simple implementation, not ngram-map)
@@ -1753,6 +1736,10 @@ struct common_speculative_impl_ngram_simple : public common_speculative_impl {
    void accept(llama_seq_id /*seq_id*/, uint16_t /*n_accepted*/, bool /*is_other*/) override {
        // noop
    }
+
+    bool need_embd() const override {
+        return false;
+    }
 };

 struct common_speculative_impl_ngram_map_k : public common_speculative_impl {
@@ -1807,6 +1794,10 @@ struct common_speculative_impl_ngram_map_k : public common_speculative_impl {

        common_ngram_map_accept(config[seq_id], n_accepted);
    }
+
+    bool need_embd() const override {
+        return false;
+    }
 };

 struct common_speculative_impl_ngram_mod : public common_speculative_impl {
@@ -1982,6 +1973,10 @@ struct common_speculative_impl_ngram_mod : public common_speculative_impl {
            }
        }
    }
+
+    bool need_embd() const override {
+        return false;
+    }
 };

 struct common_speculative_impl_ngram_cache : public common_speculative_impl {
@@ -2121,6 +2116,10 @@ struct common_speculative_impl_ngram_cache : public common_speculative_impl {
    void accept(llama_seq_id /*seq_id*/, uint16_t /*n_accepted*/, bool /*is_other*/) override {
        // noop
    }
+
+    bool need_embd() const override {
+        return false;
+    }
 };

 struct common_speculative {
@@ -2228,43 +2227,6 @@ common_speculative_type common_speculative_type_from_name(const std::string & na
    return it->second;
 }

-std::vector<common_speculative_type> common_speculative_types_from_gguf(const std::string & path) {
-    struct gguf_init_params gguf_params = {
-        /* .no_alloc = */ true,
-        /* .ctx      = */ nullptr,
-    };
-
-    gguf_context_ptr gguf_ctx(gguf_init_from_file(path.c_str(), gguf_params));
-    if (!gguf_ctx) {
-        return {};
-    }
-
-    const int64_t arch_id = gguf_find_key(gguf_ctx.get(), "general.architecture");
-    if (arch_id < 0 || gguf_get_kv_type(gguf_ctx.get(), arch_id) != GGUF_TYPE_STRING) {
-        return {};
-    }
-
-    const std::string arch = gguf_get_val_str(gguf_ctx.get(), arch_id);
-    if (arch != "dflash") {
-        const uint32_t block_count = gguf_get_val_u32(gguf_ctx.get(), gguf_find_key(gguf_ctx.get(), (arch + ".block_count").c_str()));
-
-        if (gguf_find_tensor(gguf_ctx.get(), ("blk." + std::to_string(block_count - 1) + ".nextn.eh_proj.weight").c_str()) >= 0) {
-            return { COMMON_SPECULATIVE_TYPE_DRAFT_MTP };
-        }
-
-        return {};
-    }
-
-    // the Markov head distinguishes draft-dspark from draft-dflash
-    const auto type = gguf_find_tensor(gguf_ctx.get(), "markov_w1.weight") >= 0
-                    ? COMMON_SPECULATIVE_TYPE_DRAFT_DSPARK
-                    : COMMON_SPECULATIVE_TYPE_DRAFT_DFLASH;
-
-    SPC_INF("auto-detected speculative type '%s' from the draft model metadata\n", common_speculative_type_to_str(type).c_str());
-
-    return { type };
-}
-
 static uint32_t common_get_enabled_speculative_configs(const std::vector<common_speculative_type> & configs) {
    uint32_t result = 0;
    for (size_t i = 0; i < configs.size(); i++) {
@@ -2330,24 +2292,6 @@ common_params common_base_params_to_speculative(const common_params & params) {
    result.cache_type_k  = params_spec.cache_type_k;
    result.cache_type_v  = params_spec.cache_type_v;
    result.n_outputs_max = params.n_parallel;
-    result.n_outputs_max_per_seq = 1;
-
-    // dflash/dspark decode the whole noise block in a single pass and sample every block position on the backend
-    // TODO: refactor such properties to be announced by the speculative types
-    //       something like `struct common_speculative_type_props common_speculative_type_get_props(...);`
-    const bool has_block_draft = std::any_of(
-        params.speculative.types.begin(), params.speculative.types.end(),
-        [](common_speculative_type t) {
-            return t == COMMON_SPECULATIVE_TYPE_DRAFT_DFLASH || t == COMMON_SPECULATIVE_TYPE_DRAFT_DSPARK;
-        });
-    if (has_block_draft) {
-        // per-seq output positions: DFlash decodes anchor + n_max masks (n_max + 1); DSpark n_max -> +1 covers both
-        const int32_t per_seq = std::max(1, params_spec.n_max + 1);
-        result.n_outputs_max = params.n_parallel * per_seq;
-        if (params_spec.backend_sampling) {
-            result.n_outputs_max_per_seq = per_seq;
-        }
-    }

    return result;
 }
@@ -2370,6 +2314,7 @@ common_speculative_init_result::common_speculative_init_result(
    const bool spec_mtp = std::find(params.speculative.types.begin(),
                                    params.speculative.types.end(),
                                    COMMON_SPECULATIVE_TYPE_DRAFT_MTP) != params.speculative.types.end();
+    GGML_ASSERT(has_draft || spec_mtp);

    auto mparams = common_model_params_to_llama(params);
    auto cparams = common_context_params_to_llama(params);
@@ -2432,17 +2377,6 @@ common_speculative_init_result_ptr common_speculative_init_from_params(common_pa
    return std::make_unique<common_speculative_init_result>(params, model_tgt, ctx_tgt);
 }

-common_speculative_output_limits common_speculative_get_output_limits(
-        int32_t n_batch, int32_t n_parallel, int32_t n_draft) {
-    const int64_t per_seq = 1 + (int64_t) std::max(0, n_draft);
-    const int64_t total   = (int64_t) n_parallel * per_seq;
-
-    return {
-        /* .total   = */ (int32_t) std::min<int64_t>(n_batch, total),
-        /* .per_seq = */ (int32_t) std::min<int64_t>(n_batch, per_seq),
-    };
-}
-
 // initialization of the speculative decoding system
 //
 common_speculative * common_speculative_init(common_params_speculative & params, uint32_t n_seq) {
@@ -2607,6 +2541,34 @@ bool common_speculative_process(common_speculative * spec, const llama_batch & b
    return result;
 }

+bool common_speculative_need_embd(common_speculative * spec) {
+    if (spec == nullptr) {
+        return false;
+    }
+
+    for (auto & impl : spec->impls) {
+        if (impl->need_embd()) {
+            return true;
+        }
+    }
+
+    return false;
+}
+
+bool common_speculative_need_embd_nextn(common_speculative * spec) {
+    if (spec == nullptr) {
+        return false;
+    }
+
+    for (auto & impl : spec->impls) {
+        if (impl->need_embd_nextn()) {
+            return true;
+        }
+    }
+
+    return false;
+}
+
 void common_speculative_draft(common_speculative * spec) {
    if (spec == nullptr) {
        return;
@@ -2691,10 +2653,7 @@ void common_speculative_draft(common_speculative * spec) {
 void common_speculative_accept(common_speculative * spec, llama_seq_id seq_id, uint16_t n_accepted) {
    common_speculative_impl * impl = spec->impl_last[seq_id];

-    if (impl == nullptr) {
-        GGML_ASSERT(n_accepted == 0);
-        return;
-    }
+    GGML_ASSERT(impl);

    {
        common_time_meas tm(impl->t_accept_us, !impl->gen_perf);
@@ -14,9 +14,6 @@ const char * common_speculative_all_types_str();
 // parse user provided types
 std::vector<enum common_speculative_type> common_speculative_types_from_names(const std::vector<std::string> & names);

-// infer the spec types from the GGUF metadata of a draft model; empty if unknown
-std::vector<enum common_speculative_type> common_speculative_types_from_gguf(const std::string & path);
-
 // convert string to type
 enum common_speculative_type common_speculative_type_from_name(const std::string & name);

@@ -28,15 +25,6 @@ int32_t common_speculative_n_max(const common_params_speculative * spec);

 common_params common_base_params_to_speculative(const common_params & params);

-struct common_speculative_output_limits {
-    int32_t total;
-    int32_t per_seq;
-};
-
-// return the output limits needed for speculative decoding
-common_speculative_output_limits common_speculative_get_output_limits(
-        int32_t n_batch, int32_t n_parallel, int32_t n_draft);
-
 common_speculative * common_speculative_init(common_params_speculative & params, uint32_t n_seq);

 void common_speculative_free(common_speculative * spec);
@@ -70,6 +58,12 @@ void common_speculative_begin(common_speculative * spec, llama_seq_id seq_id, co
 // process the batch and update the internal state of the speculative context
 bool common_speculative_process(common_speculative * spec, const llama_batch & batch);

+// true if any implementation requires target post-norm embeddings to be extracted
+bool common_speculative_need_embd(common_speculative * spec);
+
+// true if any implementation requires target nextn embeddings to be extracted
+bool common_speculative_need_embd_nextn(common_speculative * spec);
+
 // generate drafts for the sequences specified with `common_speculative_get_draft_params`
 void common_speculative_draft(common_speculative * spec);

@@ -27,7 +27,6 @@ TEXT_MODEL_MAP: dict[str, str] = {
    "BaichuanForCausalLM": "baichuan",
    "BailingMoeForCausalLM": "bailingmoe",
    "BailingMoeV2ForCausalLM": "bailingmoe",
-    "BailingMoeV3ForCausalLM": "bailingmoe3",
    "BambaForCausalLM": "granite",
    "BertForMaskedLM": "bert",
    "BertForSequenceClassification": "bert",
@@ -71,7 +70,6 @@ TEXT_MODEL_MAP: dict[str, str] = {
    "Exaone4ForCausalLM": "exaone",
    "ExaoneForCausalLM": "exaone",
    "ExaoneMoEForCausalLM": "exaone",
-    "ExaoneMoeForCausalLM": "exaone",
    "FalconForCausalLM": "falcon",
    "FalconH1ForCausalLM": "falcon_h1",
    "FalconMambaForCausalLM": "mamba",
@@ -104,7 +102,6 @@ TEXT_MODEL_MAP: dict[str, str] = {
    "GraniteMoeForCausalLM": "granite",
    "GraniteMoeHybridForCausalLM": "granite",
    "GraniteMoeSharedForCausalLM": "granite",
-    "GraniteSwitchForCausalLM": "granite",
    "GraniteSpeechForConditionalGeneration": "granite",
    "GraniteSpeechPlusForConditionalGeneration": "granite",
    "Grok1ForCausalLM": "grok",
@@ -126,7 +123,6 @@ TEXT_MODEL_MAP: dict[str, str] = {
    "JinaEmbeddingsV5Model": "bert",
    "KORMoForCausalLM": "qwen",
    "KimiK25ForConditionalGeneration": "deepseek",
-    "KimiK3ForConditionalGeneration": "kimi_k3",
    "KimiLinearForCausalLM": "kimi_linear",
    "KimiLinearModel": "kimi_linear",
    "KimiVLForConditionalGeneration": "deepseek",
@@ -163,8 +159,6 @@ TEXT_MODEL_MAP: dict[str, str] = {
    "MiniCPM3ForCausalLM": "minicpm",
    "MiniCPMForCausalLM": "minicpm",
    "MiniCPMV4_6ForConditionalGeneration": "minicpm",
-    "MiniMaxText01ForCausalLM": "minimax",
-    "MiniMaxM1ForCausalLM": "minimax",
    "MiniMaxM2ForCausalLM": "minimax",
    "MiniMaxM3SparseForCausalLM": "minimax",
    "MiniMaxM3SparseForConditionalGeneration": "minimax",
@@ -187,8 +181,6 @@ TEXT_MODEL_MAP: dict[str, str] = {
    "Olmo3ForCausalLM": "olmo",
    "OlmoForCausalLM": "olmo",
    "OlmoeForCausalLM": "olmo",
-    "MuseGlimmerAssistantModel": "muse_glimmer",
-    "MuseGlimmerForConditionalGeneration": "muse_glimmer",
    "OpenELMForCausalLM": "openelm",
    "OrionForCausalLM": "orion",
    "PLMForCausalLM": "plm",
@@ -218,7 +210,6 @@ TEXT_MODEL_MAP: dict[str, str] = {
    "Qwen3MoeForCausalLM": "qwen",
    "Qwen3NextForCausalLM": "qwen",
    "Qwen3OmniMoeForConditionalGeneration": "qwen3vl",
-    "PocketTTSModel": "pockettts",
    "Qwen3TTSForConditionalGeneration": "qwen3tts",
    "Qwen3VLForConditionalGeneration": "qwen3vl",
    "Qwen3VLMoeForConditionalGeneration": "qwen3vl",
@@ -305,7 +296,6 @@ MMPROJ_MODEL_MAP: dict[str, str] = {
    "MiniCPMV4_6ForConditionalGeneration": "minicpm",
    "Mistral3ForConditionalGeneration": "llava",
    "NemotronH_Nano_VL_V2": "nemotron",
-    "MuseGlimmerForConditionalGeneration": "muse_glimmer",
    "PaddleOCRVisionModel": "ernie",
    "Phi4ForCausalLMV": "phi",
    "Qwen2AudioForConditionalGeneration": "ultravox",
@@ -315,7 +305,6 @@ MMPROJ_MODEL_MAP: dict[str, str] = {
    "Qwen2_5_VLForConditionalGeneration": "qwenvl",
    "Qwen3ASRForConditionalGeneration": "qwen3vl",
    "Qwen3OmniMoeForConditionalGeneration": "qwen3vl",
-    "PocketTTSModel": "pockettts",
    "Qwen3TTSForConditionalGeneration": "qwen3tts",
    "Qwen3VLForConditionalGeneration": "qwen3vl",
    "Qwen3VLMoeForConditionalGeneration": "qwen3vl",
@@ -13,7 +13,6 @@ from .llama import LlamaModel


@ModelBase.register("AfmoeForCausalLM")
-@ModelBase.example("arcee-ai/Trinity-Large-Thinking")
 class AfmoeModel(LlamaModel):
    model_arch = gguf.MODEL_ARCH.AFMOE

@@ -16,7 +16,6 @@ from .llama import LlamaModel


@ModelBase.register("ArcticForCausalLM")
-@ModelBase.example("Snowflake/snowflake-arctic-instruct")
 class ArcticModel(TextModel):
    model_arch = gguf.MODEL_ARCH.ARCTIC

@@ -9,7 +9,6 @@ from .base import ModelBase, TextModel, gguf, logger


@ModelBase.register("BaichuanForCausalLM", "BaiChuanForCausalLM")
-@ModelBase.example("baichuan-inc/Baichuan2-7B-Chat", "baichuan-inc/Baichuan-7B")
 class BaichuanModel(TextModel):
    model_arch = gguf.MODEL_ARCH.BAICHUAN

@@ -11,7 +11,6 @@ from .base import ModelBase, TextModel, gguf


@ModelBase.register("BailingMoeForCausalLM")
-@ModelBase.example("inclusionAI/Ling-lite")
 class BailingMoeModel(TextModel):
    model_arch = gguf.MODEL_ARCH.BAILINGMOE

@@ -109,7 +108,6 @@ class BailingMoeModel(TextModel):


@ModelBase.register("BailingMoeV2ForCausalLM")
-@ModelBase.example("inclusionAI/Ling-mini-2.0")
 class BailingMoeV2Model(TextModel):
    model_arch = gguf.MODEL_ARCH.BAILINGMOE2

@@ -191,7 +189,6 @@ class BailingMoeV2Model(TextModel):


@ModelBase.register("SarvamMoEForCausalLM", "modeling_sarvam_moe.SarvamMoEForCausalLM")
-@ModelBase.example("sarvamai/sarvam-30b")
 class SarvamMoEModel(BailingMoeV2Model):
    model_arch = gguf.MODEL_ARCH.BAILINGMOE2
    # Sarvam-MoE shares the BailingMoeV2 architecture; only differences:
@@ -1,193 +0,0 @@
-from __future__ import annotations
-
-import re
-
-from typing import Callable, Iterable, TYPE_CHECKING
-
-import torch
-
-if TYPE_CHECKING:
-    from torch import Tensor
-
-from .base import ModelBase, TextModel, gguf
-
-
-@ModelBase.register("BailingMoeV3ForCausalLM")
-@ModelBase.example("inclusionAI/Ling-3.0-tiny", "inclusionAI/Ling-3.0-flash")
-class BailingMoeV3Model(TextModel):
-    model_arch = gguf.MODEL_ARCH.BAILINGMOE3
-    supports_mtp_export = True
-
-    _experts: list[dict[str, Tensor]] | None = None
-    _main_layers: int | None = None
-
-    def __init__(self, *args, **kwargs):
-        super().__init__(*args, **kwargs)
-        nextn_layers = self.hparams.get("num_nextn_predict_layers", 0) or 0
-        if self.no_mtp:
-            nextn_layers = 0
-        self.block_count = self.hparams["num_hidden_layers"] + nextn_layers
-        self.tensor_map = gguf.get_tensor_name_map(self.model_arch, self.block_count)
-
-    def index_tensors(self, remote_hf_model_id: str | None = None):
-        type(self)._main_layers = self.hparams["num_hidden_layers"]
-        return super().index_tensors(remote_hf_model_id=remote_hf_model_id)
-
-    def set_vocab(self):
-        self._set_vocab_gpt2()
-
-    def is_full_attention(self, bid: int) -> bool:
-        n_layer = self.hparams["num_hidden_layers"]
-        layer_group_size = self.hparams["layer_group_size"]
-        return bid >= n_layer or (bid + 1) % layer_group_size == 0 or bid >= n_layer // layer_group_size * layer_group_size
-
-    def set_gguf_parameters(self):
-        if not self.hparams.get("no_kda_lora", False):
-            raise ValueError("BailingMoeV3 KDA LoRA projections are not supported")
-        if not self.hparams.get("kda_safe_gate", False):
-            raise ValueError("BailingMoeV3 non-safe KDA gates are not supported")
-        if self.hparams.get("gated_attention_proj_granularity_type") != "head_wise":
-            raise ValueError("BailingMoeV3 requires head-wise attention gates")
-
-        self.hparams["num_key_value_heads"] = 1
-        super().set_gguf_parameters()
-
-        n_head_kv = [1 if self.is_full_attention(il) else 0 for il in range(self.block_count)]
-        self.gguf_writer.add_head_count_kv(n_head_kv)
-
-        self.gguf_writer.add_vocab_size(self.hparams["vocab_size"])
-        self.gguf_writer.add_ssm_conv_kernel(self.hparams["short_conv_kernel_size"])
-        self.gguf_writer.add_kda_head_dim(self.hparams["head_dim"])
-        self.gguf_writer.add_kda_safe_gate(self.hparams["kda_safe_gate"])
-        self.gguf_writer.add_kda_gate_lower_bound(self.hparams["kda_lower_bound"])
-
-        kv_lora_rank = self.hparams["kv_lora_rank"]
-        qk_nope_head_dim = self.hparams["qk_nope_head_dim"]
-        qk_rope_head_dim = self.hparams["qk_rope_head_dim"]
-        if (q_lora_rank := self.hparams.get("q_lora_rank")) is not None:
-            self.gguf_writer.add_q_lora_rank(q_lora_rank)
-        self.gguf_writer.add_kv_lora_rank(kv_lora_rank)
-        self.gguf_writer.add_rope_dimension_count(qk_rope_head_dim)
-        self.gguf_writer.add_key_length(kv_lora_rank + qk_rope_head_dim)
-        self.gguf_writer.add_key_length_mla(qk_nope_head_dim + qk_rope_head_dim)
-        self.gguf_writer.add_value_length_mla(self.hparams["v_head_dim"])
-
-        self.gguf_writer.add_expert_feed_forward_length(self.hparams["moe_intermediate_size"])
-        self.gguf_writer.add_expert_shared_feed_forward_length(self.hparams["moe_shared_expert_intermediate_size"])
-        self.gguf_writer.add_expert_shared_count(self.hparams["num_shared_experts"])
-        self.gguf_writer.add_leading_dense_block_count(self.hparams["first_k_dense_replace"])
-        self.gguf_writer.add_expert_weights_scale(self.hparams["routed_scaling_factor"])
-        self.gguf_writer.add_expert_weights_norm(self.hparams["norm_topk_prob"])
-
-        def clamp_limits(key: str) -> list[float] | None:
-            values = self.hparams.get(key)
-            if values is None:
-                return None
-            values = [0.0 if value is None else float(value) for value in values[:self.block_count]]
-            return values + [0.0] * (self.block_count - len(values))
-
-        if (values := clamp_limits("expert_swiglu_limit_list")) is not None:
-            self.gguf_writer.add_swiglu_clamp_exp(values)
-        if (values := clamp_limits("share_expert_swiglu_limit_list")) is not None:
-            self.gguf_writer.add_swiglu_clamp_shexp(values)
-
-        if not self.no_mtp and (nextn_layers := self.hparams.get("num_nextn_predict_layers", 0)):
-            self.gguf_writer.add_nextn_predict_layers(nextn_layers)
-
-    def prepare_metadata(self, vocab_only: bool):
-        from_dir = self.fname_out.is_dir()
-        super().prepare_metadata(vocab_only=vocab_only)
-
-        if not self.mtp_only or not from_dir:
-            return
-
-        output_type: str = self.ftype.name.partition("_")[2]
-        fname_default: str = gguf.naming_convention(
-            self.metadata.name, self.metadata.basename, self.metadata.finetune,
-            self.metadata.version, size_label=None, output_type=output_type, model_type=None)
-        self.fname_out = self.fname_out.parent / f"mtp-{fname_default}.gguf"
-
-    @classmethod
-    def filter_tensors(cls, item: tuple[str, Callable[[], Tensor]]) -> tuple[str, Callable[[], Tensor]] | None:
-        name, gen = item
-        if name.endswith(".expert_bias"):
-            name += ".bias"
-
-        if cls._main_layers is None:
-            return super().filter_tensors((name, gen))
-
-        m = re.match(r"model\.layers\.(\d+)\.", name)
-        is_mtp = m is not None and int(m.group(1)) >= cls._main_layers
-
-        if is_mtp and cls.no_mtp:
-            return None
-        if cls.mtp_only and not is_mtp and name not in (
-            "model.word_embeddings.weight", "model.norm.weight", "lm_head.weight",
-        ):
-            return None
-
-        return super().filter_tensors((name, gen))
-
-    def modify_tensors(self, data_torch: Tensor, name: str, bid: int | None) -> Iterable[tuple[str, Tensor]]:
-        if name.endswith((".q_conv1d.weight", ".k_conv1d.weight", ".v_conv1d.weight")) and data_torch.ndim in (2, 3):
-            d_inner = data_torch.shape[0]
-            d_conv = data_torch.shape[-1]
-            data_torch = data_torch.reshape(1, d_inner, 1, d_conv)
-
-        if name.endswith(".A_log"):
-            data_torch = torch.exp(data_torch).reshape(-1, 1)
-
-        if name.endswith(".dt_bias"):
-            name = name.rpartition(".dt_bias")[0] + ".dt_proj.bias"
-
-        if name.endswith(".attention.f_proj.weight"):
-            assert bid is not None
-            if self.is_full_attention(bid):
-                raise ValueError(f"unexpected f_proj on full-attention layer {bid}")
-            name = self.format_tensor_name(gguf.MODEL_TENSOR.SSM_F_A, bid)
-
-        if name.endswith(".attention.g_proj.weight"):
-            assert bid is not None
-            tensor = gguf.MODEL_TENSOR.ATTN_GATE if self.is_full_attention(bid) else gguf.MODEL_TENSOR.SSM_G_A
-            name = self.format_tensor_name(tensor, bid)
-
-        if ".mlp.experts." in name:
-            n_experts = self.hparams["num_experts"]
-            assert bid is not None
-
-            if self._experts is None:
-                self._experts = [{} for _ in range(self.block_count)]
-
-            self._experts[bid][name] = data_torch
-            if len(self._experts[bid]) >= n_experts * 3:
-                for weight_name in ("down_proj", "gate_proj", "up_proj"):
-                    tensors = []
-                    for expert_id in range(n_experts):
-                        expert_name = f"model.layers.{bid}.mlp.experts.{expert_id}.{weight_name}.weight"
-                        tensors.append(self._experts[bid].pop(expert_name))
-                    merged_name = f"model.layers.{bid}.mlp.experts.{weight_name}.weight"
-                    yield from super().modify_tensors(torch.stack(tensors, dim=0), merged_name, bid)
-            return
-
-        if name.endswith(".attention.kv_b_proj.weight"):
-            assert bid is not None
-            n_head = self.hparams["num_attention_heads"]
-            v_head_dim = self.hparams["v_head_dim"]
-            qk_nope_head_dim = self.hparams["qk_nope_head_dim"]
-            assert data_torch.shape[0] == n_head * (v_head_dim + qk_nope_head_dim)
-            kv_b = data_torch.view(n_head, v_head_dim + qk_nope_head_dim, data_torch.shape[-1])
-            k_b, v_b = torch.split(kv_b, [qk_nope_head_dim, v_head_dim], dim=1)
-            name_k = self.format_tensor_name(gguf.MODEL_TENSOR.ATTN_K_B, bid)
-            name_v = self.format_tensor_name(gguf.MODEL_TENSOR.ATTN_V_B, bid)
-            yield from super().modify_tensors(k_b.transpose(1, 2), name_k, bid)
-            yield from super().modify_tensors(v_b, name_v, bid)
-            return
-
-        yield from super().modify_tensors(data_torch, name, bid)
-
-    def prepare_tensors(self):
-        super().prepare_tensors()
-        if self._experts is not None:
-            experts = [name for layer in self._experts for name in layer]
-            if experts:
-                raise ValueError(f"Unprocessed experts: {experts}")
@@ -58,11 +58,6 @@ logger = logging.getLogger("hf-to-gguf")
 AnyModel = TypeVar("AnyModel", bound="type[ModelBase]")


-# for checkpoints that ship no config.json, we will try to provide a synthetic one
-HparamsMatcher = Callable[[Path], bool]
-HparamsLoader = Callable[[Path], dict[str, Any]]
-
-
 class SentencePieceTokenTypes(IntEnum):
    NORMAL = 1
    UNKNOWN = 2
@@ -82,7 +77,6 @@ class ModelBase:
        ModelType.TEXT: {},
        ModelType.MMPROJ: {},
    }
-    _hparams_loaders: list[tuple[HparamsMatcher, HparamsLoader]] = []

    dir_model: Path
    ftype: gguf.LlamaFileType
@@ -658,43 +652,6 @@ class ModelBase:
    def generate_extra_tensors(self) -> Iterable[tuple[str, Tensor]]:
        return ()

-    @staticmethod
-    def repack_mxfp4_blocks(packed: Tensor, scale: Tensor) -> np.ndarray:
-        """
-        Repack 4-bit MX weights into ggml `block_mxfp4`. Lossless - only moves bits.
-
-        Source (compressed-tensors "mxfp4-pack-quantized", also used by DeepSeek-V4):
-          packed  uint8 [rows, cols/2]  element 2i in the low nibble, 2i+1 in the high one
-          scale   uint8 [rows, cols/32] one E8M0 biased exponent per 32-element group
-
-        Destination, per group: one scale byte then 16 code bytes, where byte j holds
-        element j in the low nibble and element j+16 in the high one.
-
-        The 4-bit codes need no remapping: both sides index into ggml's kvalues_mxfp4
-        order. ggml doubles the kvalues and halves the scale, so the value is the same.
-        """
-        p = packed.contiguous().view(torch.uint8)
-        s = scale.contiguous().view(torch.uint8)
-
-        rows, packed_cols = p.shape
-        cols = packed_cols * 2
-        if cols % 32 != 0:
-            raise ValueError(f"MXFP4 source row has {cols} values, expected a multiple of 32")
-
-        n_blocks = cols // 32
-        if tuple(s.shape) != (rows, n_blocks):
-            raise ValueError(f"MXFP4 scale shape {tuple(s.shape)} does not match {(rows, n_blocks)}")
-
-        src = p.reshape(rows, n_blocks, 16)
-        lo = src & 0x0F           # elements 0, 2, 4, ...
-        hi = (src >> 4) & 0x0F    # elements 1, 3, 5, ...
-
-        vals = torch.stack((lo, hi), dim=-1).reshape(rows, n_blocks, 32)
-        qs = vals[:, :, :16] | (vals[:, :, 16:] << 4)
-
-        raw = torch.cat((s.unsqueeze(-1), qs.to(torch.uint8)), dim=-1)
-        return raw.reshape(rows, n_blocks * 17).cpu().numpy()
-
    @staticmethod
    def _nvfp4_pack(weight: Tensor, scale: Tensor) -> tuple[np.ndarray, list[int]]:
        """Repack NVFP4 ModelOpt tensors into ggml super-block layout.
@@ -866,7 +823,7 @@ class ModelBase:
            elif any(str(v.get("quant_algo")).endswith("NVFP4") for v in quant_layers.values() if isinstance(v, dict)):
                quant_algo = "NVFP4"

-        self._is_nvfp4 = quant_algo in ("NVFP4", "W4A16_NVFP4")
+        self._is_nvfp4 = quant_algo == "NVFP4"
        self._is_mxfp4 = quant_method == "mxfp4"

        # NVFP4 weights are repacked and written directly to gguf_writer.
@@ -1083,24 +1040,6 @@ class ModelBase:

        return part_names

-    @staticmethod
-    def load_hparams_guess(dir_model: Path) -> dict[str, Any] | None:
-        # some models ship no config.json, will try to guess them
-        from conversion import load_all_models
-        load_all_models()
-
-        for matcher, loader in ModelBase._hparams_loaders:
-            if matcher(dir_model):
-                return loader(dir_model)
-        return None
-
-    @classmethod
-    def register_hparams_loader(cls, matcher: HparamsMatcher) -> Callable[[HparamsLoader], HparamsLoader]:
-        def inner(loader: HparamsLoader) -> HparamsLoader:
-            cls._hparams_loaders.append((matcher, loader))
-            return loader
-        return inner
-
    @staticmethod
    def load_hparams(dir_model: Path, is_mistral_format: bool):
        if is_mistral_format:
@@ -1114,10 +1053,6 @@ class ModelBase:
            config = AutoConfig.from_pretrained(dir_model, trust_remote_code=False).to_dict()
        except Exception as e:
            logger.warning(f"Failed to load model config from {dir_model}: {e}")
-            if not (dir_model / "config.json").is_file():
-                config = ModelBase.load_hparams_guess(dir_model)
-                if config is not None:
-                    return config
            logger.warning("Trying to load config.json instead")
            with open(dir_model / "config.json", "r", encoding="utf-8") as f:
                config = json.load(f)
@@ -1149,14 +1084,6 @@ class ModelBase:
            return modelcls
        return func

-    @classmethod
-    def example(cls, *hf_repos: str) -> Callable[[AnyModel], AnyModel]:
-        del hf_repos  # unused
-
-        def func(modelcls: AnyModel) -> AnyModel:
-            return modelcls
-        return func
-
    @classmethod
    def print_registered_models(cls):
        for model_type, model_classes in cls._model_classes.items():
@@ -2706,10 +2633,7 @@ def get_model_architecture(hparams: dict[str, Any], model_type: ModelType) -> st
    # Step3-VL keeps text config under text_config but uses a custom top-level architecture.
    # For text conversion we route to a dedicated text-only class.
    # TODO: refactor this later to avoid adding exception here
-    # Kimi-K3's text_config reports "KimiLinearForCausalLM", which is the older
-    # Kimi-Linear-48B architecture and cannot load K3 (no attention residuals,
-    # latent MoE, situ, ...). Route on the top-level architecture instead.
-    if model_type == ModelType.TEXT and arch in ("StepVLForConditionalGeneration", "Sarashina2VisionForCausalLM", "Exaone4_5_ForConditionalGeneration", "Step3p7ForConditionalGeneration", "KimiK3ForConditionalGeneration"):
+    if model_type == ModelType.TEXT and arch in ("StepVLForConditionalGeneration", "Sarashina2VisionForCausalLM", "Exaone4_5_ForConditionalGeneration", "Step3p7ForConditionalGeneration"):
        return arch

    # if "architectures" is found in the sub-config, use that instead
@@ -15,7 +15,6 @@ from .base import ModelBase, SentencePieceTokenTypes, TextModel, gguf, logger


@ModelBase.register("BertModel", "BertForMaskedLM", "CamembertModel", "BertForSequenceClassification")
-@ModelBase.example("BAAI/bge-small-en-v1.5", "dangvantuan/sentence-camembert-base")
 class BertModel(TextModel):
    model_arch = gguf.MODEL_ARCH.BERT

@@ -241,7 +240,6 @@ class BertModel(TextModel):


@ModelBase.register("DistilBertModel", "DistilBertForMaskedLM", "DistilBertForSequenceClassification")
-@ModelBase.example("distilbert/distilbert-base-uncased")
 class DistilBertModel(BertModel):
    model_arch = gguf.MODEL_ARCH.BERT

@@ -265,7 +263,6 @@ class DistilBertModel(BertModel):


@ModelBase.register("RobertaModel", "RobertaForSequenceClassification")
-@ModelBase.example("sentence-transformers/stsb-roberta-base")
 class RobertaModel(BertModel):
    model_arch = gguf.MODEL_ARCH.BERT

@@ -315,7 +312,6 @@ class RobertaModel(BertModel):


@ModelBase.register("NomicBertModel")
-@ModelBase.example("nomic-ai/nomic-embed-text-v1.5")
 class NomicBertModel(BertModel):
    model_arch = gguf.MODEL_ARCH.BERT

@@ -404,7 +400,6 @@ class NomicBertModel(BertModel):


@ModelBase.register("NeoBERT", "NeoBERTLMHead", "NeoBERTForSequenceClassification")
-@ModelBase.example("chandar-lab/NeoBERT")
 class NeoBert(BertModel):
    model_arch = gguf.MODEL_ARCH.NEO_BERT

@@ -436,7 +431,6 @@ class NeoBert(BertModel):


@ModelBase.register("EuroBertModel", "JinaEmbeddingsV5Model")
-@ModelBase.example("hf-tiny-v2/tiny-random-EuroBertModel", "jinaai/jina-embeddings-v5-text-nano")
 class EuroBertModel(TextModel):
    model_arch = gguf.MODEL_ARCH.EUROBERT

@@ -465,7 +459,6 @@ class EuroBertModel(TextModel):


@ModelBase.register("XLMRobertaModel", "XLMRobertaForSequenceClassification")
-@ModelBase.example("BAAI/bge-m3")
 class XLMRobertaModel(BertModel):
    model_arch = gguf.MODEL_ARCH.BERT
    _lora_files = {}
@@ -568,7 +561,6 @@ class XLMRobertaModel(BertModel):


@ModelBase.register("JinaBertModel", "JinaBertForMaskedLM")
-@ModelBase.example("jinaai/jina-embeddings-v2-base-en")
 class JinaBertV2Model(BertModel):
    model_arch = gguf.MODEL_ARCH.JINA_BERT_V2

@@ -596,7 +588,6 @@ class JinaBertV2Model(BertModel):


@ModelBase.register("ModernBertModel", "ModernBertForMaskedLM", "ModernBertForSequenceClassification")
-@ModelBase.example("answerdotai/ModernBERT-base")
 class ModernBertModel(BertModel):
    model_arch = gguf.MODEL_ARCH.MODERN_BERT

@@ -9,7 +9,6 @@ from .base import ModelBase, TextModel, gguf


@ModelBase.register("BitnetForCausalLM", "BitNetForCausalLM")
-@ModelBase.example("microsoft/bitnet-b1.58-2B-4T")
 class BitnetModel(TextModel):
    model_arch = gguf.MODEL_ARCH.BITNET

@@ -13,7 +13,6 @@ from .base import ModelBase, TextModel, gguf, logger


@ModelBase.register("BloomForCausalLM", "BloomModel")
-@ModelBase.example("bigscience/bloom-560m")
 class BloomModel(TextModel):
    model_arch = gguf.MODEL_ARCH.BLOOM

@@ -12,8 +12,6 @@ from .llama import LlamaModel

@ModelBase.register("ChameleonForConditionalGeneration")
@ModelBase.register("ChameleonForCausalLM")  # obsolete
-# [TAG_HF_EXAMPLE_GATED] facebook/chameleon-7b is gated
-# [TAG_HF_EXAMPLE_MISSING]
 class ChameleonModel(TextModel):
    model_arch = gguf.MODEL_ARCH.CHAMELEON

@@ -9,7 +9,6 @@ from .base import ModelBase, SentencePieceTokenTypes, TextModel, gguf


@ModelBase.register("GlmForCausalLM", "ChatGLMModel", "ChatGLMForConditionalGeneration")
-@ModelBase.example("THUDM/chatglm3-6b", "zai-org/glm-4-9b-chat-hf")
 class ChatGLMModel(TextModel):
    model_arch = gguf.MODEL_ARCH.CHATGLM

@@ -4,7 +4,6 @@ from .base import ModelBase, TextModel, gguf


@ModelBase.register("CodeShellForCausalLM")
-@ModelBase.example("WisdomShell/CodeShell-7B")
 class CodeShellModel(TextModel):
    model_arch = gguf.MODEL_ARCH.CODESHELL

@@ -11,7 +11,6 @@ from .llama import LlamaModel


@ModelBase.register("CogVLMForCausalLM")
-@ModelBase.example("THUDM/cogvlm2-llama3-chat-19B", "THUDM/cogvlm-chat-hf")
 class CogVLMVisionModel(MmprojModel):

    def set_gguf_parameters(self):
@@ -30,6 +29,5 @@ class CogVLMVisionModel(MmprojModel):


@ModelBase.register("CogVLMForCausalLM")
-@ModelBase.example("THUDM/cogvlm2-llama3-chat-19B", "THUDM/cogvlm-chat-hf")
 class CogVLMModel(LlamaModel):
    model_arch = gguf.MODEL_ARCH.COGVLM
@@ -12,8 +12,6 @@ from .base import ModelBase, TextModel, gguf, logger


@ModelBase.register("CohereForCausalLM")
-# [TAG_HF_EXAMPLE_GATED] CohereLabs/c4ai-command-r-v01 is gated
-# [TAG_HF_EXAMPLE_MISSING]
 class CommandR2Model(TextModel):
    model_arch = gguf.MODEL_ARCH.COMMAND_R

@@ -32,8 +30,6 @@ class CommandR2Model(TextModel):


@ModelBase.register("Cohere2ForCausalLM")
-# [TAG_HF_EXAMPLE_GATED] CohereLabs/c4ai-command-r7b-12-2024 is gated
-@ModelBase.example("hf-tiny-v2/tiny-random-Cohere2ForCausalLM")
 class Cohere2Model(TextModel):
    model_arch = gguf.MODEL_ARCH.COHERE2

@@ -63,7 +59,6 @@ class Cohere2Model(TextModel):


@ModelBase.register("Cohere2MoeForCausalLM")
-@ModelBase.example("CohereLabs/North-Mini-Code-1.0")
 class Cohere2MoeModel(TextModel):
    model_arch = gguf.MODEL_ARCH.COHERE2MOE
    _n_main_layers: int | None = None
@@ -9,7 +9,6 @@ from .base import ModelBase, TextModel, gguf, logger


@ModelBase.register("DbrxForCausalLM")
-@ModelBase.example("alpindale/dbrx-instruct")
 class DbrxModel(TextModel):
    model_arch = gguf.MODEL_ARCH.DBRX

@@ -13,7 +13,6 @@ from .base import ModelBase, TextModel, gguf


@ModelBase.register("DeciLMForCausalLM")
-@ModelBase.example("nvidia/Llama-3_1-Nemotron-51B-Instruct", "Deci/DeciLM-7B")
 class DeciModel(TextModel):
    model_arch = gguf.MODEL_ARCH.DECI

@@ -17,12 +17,8 @@ from .base import LazyTorchTensor, MmprojModel, ModelBase, TextModel, gguf, logg
 from .qwen import QwenModel


-@ModelBase.register("DeepseekOCRForCausalLM")
-@ModelBase.example("deepseek-ai/DeepSeek-OCR")
+@ModelBase.register("DeepseekOCRForCausalLM", "UnlimitedOCRForCausalLM")
 class DeepseekOCRVisionModel(MmprojModel):
-    # HF dynamic_preprocess() max_num, which differs per model
-    preproc_max_tiles = 9
-
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.clip_projector_type = gguf.VisionProjectorType.DEEPSEEKOCR
@@ -47,9 +43,6 @@ class DeepseekOCRVisionModel(MmprojModel):
        # @bluebread: there's no window_size in config but just add it here anyway
        self.gguf_writer.add_vision_window_size(self.hparams.get("window_size", 14))

-        self.gguf_writer.add_vision_preproc_min_tiles(2)
-        self.gguf_writer.add_vision_preproc_max_tiles(self.preproc_max_tiles)
-
        # SAM configuration
        sam_hparams = hparams['sam']
        self.gguf_writer.add_vision_sam_layers_count(sam_hparams['layers'])
@@ -100,17 +93,8 @@ class DeepseekOCRVisionModel(MmprojModel):
        return super().filter_tensors((name, gen))


-@ModelBase.register("UnlimitedOCRForCausalLM")
-@ModelBase.example("baidu/Unlimited-OCR")
-class UnlimitedOCRVisionModel(DeepseekOCRVisionModel):
-    preproc_max_tiles = 32
-
-
@ModelBase.register("DeepseekOCR2ForCausalLM")
-@ModelBase.example("deepseek-ai/DeepSeek-OCR-2")
 class DeepseekOCR2VisionModel(DeepseekOCRVisionModel):
-    preproc_max_tiles = 6
-
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.clip_projector_type = gguf.VisionProjectorType.DEEPSEEKOCR2
@@ -137,7 +121,6 @@ class DeepseekOCR2VisionModel(DeepseekOCRVisionModel):


@ModelBase.register("DeepseekForCausalLM")
-@ModelBase.example("deepseek-ai/deepseek-moe-16b-chat")
 class DeepseekModel(TextModel):
    model_arch = gguf.MODEL_ARCH.DEEPSEEK

@@ -232,7 +215,6 @@ class DeepseekModel(TextModel):
    "YoutuForCausalLM",
    "YoutuVLForConditionalGeneration",
 )
-@ModelBase.example("deepseek-ai/DeepSeek-V2-Lite", "deepseek-ai/DeepSeek-V3")
 class DeepseekV2Model(TextModel):
    model_arch = gguf.MODEL_ARCH.DEEPSEEK2

@@ -462,7 +444,6 @@ class DeepseekV2Model(TextModel):


@ModelBase.register("DeepseekV32ForCausalLM")
-@ModelBase.example("deepseek-ai/DeepSeek-V3.2-Exp")
 class DeepseekV32Model(DeepseekV2Model):
    model_arch = gguf.MODEL_ARCH.DEEPSEEK32
    skip_mtp = False
@@ -523,7 +504,6 @@ class DeepseekV32Model(DeepseekV2Model):


@ModelBase.register("DeepseekV4ForCausalLM")
-@ModelBase.example("deepseek-ai/DeepSeek-V4-Flash-Base")
 class DeepseekV4Model(TextModel):
    model_arch = gguf.MODEL_ARCH.DEEPSEEK4
    supports_mtp_export = True
@@ -540,13 +520,6 @@ class DeepseekV4Model(TextModel):
        for key, value in raw_hparams.items():
            self.hparams.setdefault(key, value)

-        # workaround for special rope_parameters (main/compress) in transformers 5.x
-        if self.rope_parameters.get("full_attention", self.rope_parameters).get("rope_type") is None:
-            if (rope_scaling := raw_hparams.get("rope_scaling")) is not None:
-                if "rope_type" not in rope_scaling and (rope_type := rope_scaling.get("type")) is not None:
-                    rope_scaling["rope_type"] = rope_type
-                self.rope_parameters.update(**rope_scaling)
-
        self.block_count = self.hparams["num_hidden_layers"]
        if self.mtp_only:
            self.block_count += self.hparams.get("num_nextn_predict_layers", 0)
@@ -716,6 +689,31 @@ class DeepseekV4Model(TextModel):
        for name in tensors_to_remove:
            del self.model_tensors[name]

+    @staticmethod
+    def _pack_mxfp4_blocks(weight: Tensor, scale: Tensor) -> np.ndarray:
+        packed = weight.contiguous().view(torch.uint8)
+        scale_u8 = scale.contiguous().view(torch.uint8)
+
+        out_features, packed_cols = packed.shape
+        logical_cols = packed_cols * 2
+        if logical_cols % 32 != 0:
+            raise ValueError(f"MXFP4 source row has {logical_cols} values, expected a multiple of 32")
+
+        n_blocks = logical_cols // 32
+        if tuple(scale_u8.shape) != (out_features, n_blocks):
+            raise ValueError(f"MXFP4 scale shape {tuple(scale_u8.shape)} does not match {(out_features, n_blocks)}")
+
+        src = packed.reshape(out_features, n_blocks, 16)
+        low = src & 0x0F
+        high = (src >> 4) & 0x0F
+
+        # The safetensors bytes store adjacent values as low/high nibbles.
+        # ggml MXFP4 blocks store values 0..15 in low nibbles and 16..31 in high nibbles.
+        vals = torch.stack((low, high), dim=-1).reshape(out_features, n_blocks, 32)
+        qs = vals[:, :, :16] | (vals[:, :, 16:] << 4)
+        raw = torch.cat((scale_u8.unsqueeze(-1), qs.to(torch.uint8)), dim=-1)
+        return raw.reshape(out_features, n_blocks * 17).cpu().numpy()
+
    def _write_mxfp4_expert_tensor(self, bid: int, proj: str, tensor_key: gguf.MODEL_TENSOR) -> list[str]:
        n_experts = self.hparams["n_routed_experts"]
        data: np.ndarray | None = None
@@ -729,7 +727,7 @@ class DeepseekV4Model(TextModel):

            weight = LazyTorchTensor.to_eager(self.model_tensors[weight_name]())
            scale = LazyTorchTensor.to_eager(self.model_tensors[scale_name]())
-            packed = self.repack_mxfp4_blocks(weight, scale)
+            packed = self._pack_mxfp4_blocks(weight, scale)
            if data is None:
                data = np.empty((n_experts, *packed.shape), dtype=packed.dtype)
            data[eid] = packed
@@ -918,7 +916,6 @@ class DeepseekV4Model(TextModel):


@ModelBase.register("DeepseekV4DSparkModel")
-@ModelBase.example("deepseek-ai/DeepSeek-V4-Flash-DSpark")
 class DeepseekV4DSparkModel(DeepseekV4Model):
    model_arch = gguf.MODEL_ARCH.DFLASH

@@ -11,7 +11,6 @@ from .qwen import Qwen2MoeModel


@ModelBase.register("Dots1ForCausalLM")
-@ModelBase.example("rednote-hilab/dots.llm1.inst")
 class Dots1Model(Qwen2MoeModel):
    model_arch = gguf.MODEL_ARCH.DOTS1

@@ -9,7 +9,6 @@ from .base import MmprojModel, ModelBase, gguf


@ModelBase.register("DotsOCRForCausalLM")
-@ModelBase.example("rednote-hilab/dots.ocr")
 class DotsOCRVisionModel(MmprojModel):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
@@ -9,7 +9,6 @@ from .base import ModelBase, TextModel, gguf


@ModelBase.register("DreamModel")
-@ModelBase.example("Dream-org/Dream-v0-Instruct-7B")
 class DreamModel(TextModel):
    model_arch = gguf.MODEL_ARCH.DREAM

@@ -15,7 +15,6 @@ from .base import MmprojModel, ModelBase, TextModel, gguf


@ModelBase.register("Ernie4_5_ForCausalLM", "Ernie4_5ForCausalLM")
-@ModelBase.example("baidu/ERNIE-4.5-0.3B-PT")
 class Ernie4_5Model(TextModel):
    model_arch = gguf.MODEL_ARCH.ERNIE4_5

@@ -74,7 +73,6 @@ class Ernie4_5Model(TextModel):


@ModelBase.register("Ernie4_5_MoeForCausalLM")
-@ModelBase.example("baidu/ERNIE-4.5-21B-A3B-PT")
 class Ernie4_5MoeModel(Ernie4_5Model):
    model_arch = gguf.MODEL_ARCH.ERNIE4_5_MOE
    _experts: list[dict[str, Tensor]] | None = None
@@ -158,13 +156,11 @@ class Ernie4_5MoeModel(Ernie4_5Model):


@ModelBase.register("PaddleOCRVLForConditionalGeneration")
-@ModelBase.example("PaddlePaddle/PaddleOCR-VL")
 class PaddleOCRModel(Ernie4_5Model):
    model_arch = gguf.MODEL_ARCH.PADDLEOCR


@ModelBase.register("PaddleOCRVisionModel")
-@ModelBase.example("PaddlePaddle/PaddleOCR-VL")
 class PaddleOCRVisionModel(MmprojModel):
    # PaddleOCR-VL uses a modified version of Siglip
    min_pixels: int = 0
@@ -15,7 +15,6 @@ from .qwenvl import Qwen2VLVisionModel


@ModelBase.register("ExaoneForCausalLM")
-@ModelBase.example("LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct")
 class ExaoneModel(TextModel):
    model_arch = gguf.MODEL_ARCH.EXAONE

@@ -61,7 +60,6 @@ class ExaoneModel(TextModel):


@ModelBase.register("Exaone4ForCausalLM")
-@ModelBase.example("LGAI-EXAONE/EXAONE-4.0-32B")
 class Exaone4Model(TextModel):
    model_arch = gguf.MODEL_ARCH.EXAONE4

@@ -125,10 +123,7 @@ class Exaone4Model(TextModel):
                yield (self.format_tensor_name(gguf.MODEL_TENSOR.ROPE_FREQS), torch.tensor(rope_factors, dtype=torch.float32))


-# note: transformers >= 5.1 renamed the class to "ExaoneMoeForCausalLM" (lowercase 'e'),
-#       so accept both spellings - LG AI have updated the configs of already-released models
-@ModelBase.register("ExaoneMoEForCausalLM", "ExaoneMoeForCausalLM")
-@ModelBase.example("LGAI-EXAONE/K-EXAONE-236B-A23B")
+@ModelBase.register("ExaoneMoEForCausalLM")
 class ExaoneMoEModel(Exaone4Model):
    model_arch = gguf.MODEL_ARCH.EXAONE_MOE

@@ -217,7 +212,6 @@ class ExaoneMoEModel(Exaone4Model):


@ModelBase.register("Exaone4_5_ForConditionalGeneration")
-@ModelBase.example("LGAI-EXAONE/EXAONE-4.5-33B")
 class Exaone4_5_TextModel(Exaone4Model):
    """Text tower of EXAONE 4.5; Tensors match EXAONE4"""

@@ -271,7 +265,6 @@ class Exaone4_5_TextModel(Exaone4Model):


@ModelBase.register("Exaone4_5_ForConditionalGeneration")
-@ModelBase.example("LGAI-EXAONE/EXAONE-4.5-33B")
 class Exaone4_5VisionModel(Qwen2VLVisionModel):
    """Vision tower for EXAONE 4.5; Qwen2-VL-style ViT (GQA) + patch merger"""

@@ -11,7 +11,6 @@ from .base import ModelBase, TextModel, gguf


@ModelBase.register("FalconForCausalLM", "RWForCausalLM")
-@ModelBase.example("tiiuae/falcon-7b")
 class FalconModel(TextModel):
    model_arch = gguf.MODEL_ARCH.FALCON

@@ -12,7 +12,6 @@ from .mamba import Mamba2Model


@ModelBase.register("FalconH1ForCausalLM")
-@ModelBase.example("tiiuae/Falcon-H1-0.5B-Base")
 class FalconH1Model(Mamba2Model):
    model_arch = gguf.MODEL_ARCH.FALCON_H1

@@ -14,8 +14,6 @@ from .base import MmprojModel, ModelBase, TextModel, gguf, logger


@ModelBase.register("GemmaForCausalLM")
-# [TAG_HF_EXAMPLE_GATED] google/gemma-2b is gated
-@ModelBase.example("trl-internal-testing/tiny-GemmaForCausalLM")
 class GemmaModel(TextModel):
    model_arch = gguf.MODEL_ARCH.GEMMA

@@ -70,8 +68,6 @@ class GemmaModel(TextModel):


@ModelBase.register("Gemma2ForCausalLM")
-# [TAG_HF_EXAMPLE_GATED] google/gemma-2-9b-it is gated
-@ModelBase.example("trl-internal-testing/tiny-Gemma2ForCausalLM")
 class Gemma2Model(TextModel):
    model_arch = gguf.MODEL_ARCH.GEMMA2

@@ -122,8 +118,6 @@ class Gemma2Model(TextModel):


@ModelBase.register("Gemma3ForCausalLM", "Gemma3ForConditionalGeneration")
-# [TAG_HF_EXAMPLE_GATED] google/gemma-3-4b-it is gated
-@ModelBase.example("trl-internal-testing/tiny-Gemma3ForConditionalGeneration", "hf-tiny-v2/tiny-random-Gemma3ForCausalLM")
 class Gemma3Model(TextModel):
    model_arch = gguf.MODEL_ARCH.GEMMA3

@@ -180,8 +174,6 @@ class Gemma3Model(TextModel):


@ModelBase.register("Gemma3TextModel")
-# [TAG_HF_EXAMPLE_GATED] google/embeddinggemma-300m is gated
-@ModelBase.example("hf-tiny-v2/tiny-random-Gemma3TextModel")
 class EmbeddingGemma(Gemma3Model):
    model_arch = gguf.MODEL_ARCH.GEMMA_EMBEDDING
    module_paths = []
@@ -256,8 +248,6 @@ class EmbeddingGemma(Gemma3Model):


@ModelBase.register("Gemma3ForConditionalGeneration")
-# [TAG_HF_EXAMPLE_GATED] google/gemma-3-4b-it is gated
-@ModelBase.example("trl-internal-testing/tiny-Gemma3ForConditionalGeneration")
 class Gemma3VisionModel(MmprojModel):
    def set_gguf_parameters(self):
        super().set_gguf_parameters()
@@ -362,8 +352,6 @@ class ConformerAudioModel(MmprojModel):


@ModelBase.register("Gemma3nForConditionalGeneration")
-# [TAG_HF_EXAMPLE_GATED] google/gemma-3n-E2B-it is gated
-@ModelBase.example("hf-tiny-v2/tiny-random-Gemma3nForConditionalGeneration")
 class Gemma3nVisionAudioModel(ConformerAudioModel):
    has_audio_encoder = True
    has_vision_encoder = True
@@ -483,8 +471,6 @@ class Gemma3nVisionAudioModel(ConformerAudioModel):


@ModelBase.register("Gemma3nForCausalLM", "Gemma3nForConditionalGeneration")
-# [TAG_HF_EXAMPLE_GATED] google/gemma-3n-E2B-it is gated
-@ModelBase.example("hf-tiny-v2/tiny-random-Gemma3nForConditionalGeneration")
 class Gemma3NModel(Gemma3Model):
    model_arch = gguf.MODEL_ARCH.GEMMA3N

@@ -629,7 +615,6 @@ class Gemma3NModel(Gemma3Model):


@ModelBase.register("Gemma4ForConditionalGeneration", "Gemma4ForCausalLM")
-@ModelBase.example("google/gemma-4-31B-it", "google/gemma-4-26B-A4B-it", "google/gemma-4-E2B-it")
 class Gemma4Model(Gemma3Model):
    model_arch = gguf.MODEL_ARCH.GEMMA4

@@ -680,18 +665,7 @@ class Gemma4Model(Gemma3Model):
        swa_layers = [t == "sliding_attention" for t in self.hparams["layer_types"]]
        self.gguf_writer.add_sliding_window_pattern(swa_layers)

-        per_layer_config = self.hparams.get("per_layer_config")
-        layer_types = self.hparams.get("layer_types", [])
-        if (head_dim_full := self.hparams.get("global_head_dim")) is None and per_layer_config is not None:
-            for layer_idx, layer_config in per_layer_config.items():
-                layer_idx = int(layer_idx)
-                if layer_idx < len(layer_types):
-                    if layer_types[layer_idx] == "full_attention" and "head_dim" in layer_config:
-                        head_dim_full = layer_config["head_dim"]
-                        break
-
-        assert head_dim_full is not None
-
+        head_dim_full = self.hparams["global_head_dim"]
        head_dim_swa = self.hparams["head_dim"]
        # correct the head dim for global/swa layers
        self.gguf_writer.add_key_length(head_dim_full)
@@ -711,14 +685,8 @@ class Gemma4Model(Gemma3Model):
            n_ff_arr = [n_ff if il < first_kv_shared_layer_idx else n_ff * 2 for il in range(self.block_count)]
            self.gguf_writer.add_feed_forward_length(n_ff_arr)

-        if (num_key_value_heads_full := self.hparams.get("num_global_key_value_heads")) is None and per_layer_config is not None:
-            for layer_idx, layer_config in per_layer_config.items():
-                layer_idx = int(layer_idx)
-                if layer_idx < len(layer_types):
-                    if layer_types[layer_idx] == "full_attention" and "num_key_value_heads" in layer_config:
-                        num_key_value_heads_full = layer_config["num_key_value_heads"]
-                        break
-
+        # handle num_global_key_value_heads
+        num_key_value_heads_full = self.hparams.get("num_global_key_value_heads")
        num_key_value_heads_swa = self.hparams.get("num_key_value_heads")
        if num_key_value_heads_full is not None and num_key_value_heads_swa is not None:
            value_arr = [num_key_value_heads_swa if is_swa else num_key_value_heads_full for is_swa in swa_layers]
@@ -740,19 +708,7 @@ class Gemma4Model(Gemma3Model):
        # IMPORTANT: this ROPE_FREQS tensor is ONLY used by the full_attention layers
        rope_params_full = self.hparams["rope_parameters"]["full_attention"]
        assert rope_params_full["rope_type"] == "proportional"
-
-        per_layer_config = self.hparams.get("per_layer_config")
-        if (head_dim_full := self.hparams.get("global_head_dim")) is None and per_layer_config is not None:
-            layer_types = self.hparams.get("layer_types", [])
-            for layer_idx, layer_config in per_layer_config.items():
-                layer_idx = int(layer_idx)
-                if layer_idx < len(layer_types):
-                    if layer_types[layer_idx] == "full_attention" and "head_dim" in layer_config:
-                        head_dim_full = layer_config["head_dim"]
-                        break
-
-        assert head_dim_full is not None
-
+        head_dim_full = (self.hparams["global_head_dim"])
        partial_rotary_factor_full = rope_params_full["partial_rotary_factor"]
        n_rot_full = int(head_dim_full * partial_rotary_factor_full / 2)
        n_unrot_full = int(head_dim_full / 2) - n_rot_full
@@ -810,7 +766,6 @@ class Gemma4Model(Gemma3Model):


@ModelBase.register("Gemma4UnifiedForConditionalGeneration")
-@ModelBase.example("hf-tiny-v2/tiny-random-Gemma4UnifiedForConditionalGeneration")
 class Gemma4UnifiedModel(Gemma4Model):
    model_arch = gguf.MODEL_ARCH.GEMMA4

@@ -831,7 +786,6 @@ class Gemma4UnifiedModel(Gemma4Model):


@ModelBase.register("Gemma4AssistantForCausalLM", "Gemma4UnifiedAssistantForCausalLM")
-@ModelBase.example("google/gemma-4-31B-it-assistant", "google/gemma-4-26B-A4B-it-assistant", "google/gemma-4-E2B-it-assistant")
 class Gemma4AssistantModel(Gemma4Model):
    model_arch = gguf.MODEL_ARCH.GEMMA4_ASSISTANT

@@ -852,7 +806,6 @@ class Gemma4AssistantModel(Gemma4Model):


@ModelBase.register("Gemma4ForConditionalGeneration")
-@ModelBase.example("google/gemma-4-31B-it", "google/gemma-4-26B-A4B-it", "google/gemma-4-E2B-it")
 class Gemma4VisionAudioModel(MmprojModel):
    has_audio_encoder = True
    has_vision_encoder = True
@@ -931,7 +884,6 @@ class Gemma4VisionAudioModel(MmprojModel):


@ModelBase.register("Gemma4UnifiedForConditionalGeneration")
-@ModelBase.example("hf-tiny-v2/tiny-random-Gemma4UnifiedForConditionalGeneration")
 class Gemma4UnifiedVisionAudioModel(Gemma4VisionAudioModel):
    has_audio_encoder = True
    has_vision_encoder = True
@@ -15,7 +15,6 @@ from .deepseek import DeepseekV2Model


@ModelBase.register("Glm4ForCausalLM", "Glm4vForConditionalGeneration")
-@ModelBase.example("zai-org/GLM-4-9B-0414")
 class Glm4Model(TextModel):
    model_arch = gguf.MODEL_ARCH.GLM4
    use_mrope = False
@@ -87,7 +86,6 @@ class Glm4Model(TextModel):


@ModelBase.register("GlmOcrForConditionalGeneration")
-@ModelBase.example("zai-org/GLM-OCR")
 class GlmOCRModel(Glm4Model):
    model_arch = gguf.MODEL_ARCH.GLM4
    use_mrope = False
@@ -109,7 +107,6 @@ class GlmOCRModel(Glm4Model):


@ModelBase.register("Glm4MoeForCausalLM", "Glm4vMoeForConditionalGeneration")
-@ModelBase.example("zai-org/GLM-4.5-Air")
 class Glm4MoeModel(TextModel):
    model_arch = gguf.MODEL_ARCH.GLM4_MOE

@@ -207,7 +204,6 @@ class Glm4MoeModel(TextModel):


@ModelBase.register("Glm4MoeLiteForCausalLM")
-@ModelBase.example("zai-org/GLM-4.7-Flash")
 class Glm4MoeLiteModel(DeepseekV2Model):
    model_arch = gguf.MODEL_ARCH.DEEPSEEK2
    skip_mtp = False
@@ -276,7 +272,6 @@ class Glm4MoeLiteModel(DeepseekV2Model):


@ModelBase.register("GlmMoeDsaForCausalLM")
-@ModelBase.example("zai-org/GLM-5.2")
 class GlmMoeDsaModel(DeepseekV2Model):
    model_arch = gguf.MODEL_ARCH.GLM_DSA
    skip_mtp = False
@@ -345,7 +340,6 @@ class GlmMoeDsaModel(DeepseekV2Model):


@ModelBase.register("SolarOpenForCausalLM")
-@ModelBase.example("upstage/Solar-Open-100B")
 class SolarOpenModel(Glm4MoeModel):
    model_arch = gguf.MODEL_ARCH.GLM4_MOE

@@ -11,7 +11,6 @@ from .base import ModelBase, TextModel, gguf, logger


@ModelBase.register("GPT2LMHeadModel")
-@ModelBase.example("openai-community/gpt2")
 class GPT2Model(TextModel):
    model_arch = gguf.MODEL_ARCH.GPT2

@@ -39,7 +38,6 @@ class GPT2Model(TextModel):


@ModelBase.register("RuGPT3XLForCausalLM")
-@ModelBase.example("evilfreelancer/ruGPT3XL")
 class RuGPT3XLModel(TextModel):
    model_arch = gguf.MODEL_ARCH.GPT2

@@ -11,7 +11,6 @@ from .base import ModelBase, TextModel, gguf, logger


@ModelBase.register("GptOssForCausalLM")
-@ModelBase.example("openai/gpt-oss-20b")
 class GptOssModel(TextModel):
    model_arch = gguf.MODEL_ARCH.GPT_OSS

@@ -13,7 +13,6 @@ from .base import ModelBase, TextModel, gguf, logger


@ModelBase.register("GPTNeoXForCausalLM")
-@ModelBase.example("EleutherAI/pythia-70m")
 class GPTNeoXModel(TextModel):
    model_arch = gguf.MODEL_ARCH.GPTNEOX

@@ -15,7 +15,6 @@ from .mamba import Mamba2Model


@ModelBase.register("GraniteForCausalLM")
-@ModelBase.example("ibm-granite/granite-3.3-2b-instruct")
 class GraniteModel(LlamaModel):
    """Conversion for IBM's GraniteForCausalLM"""
    model_arch = gguf.MODEL_ARCH.GRANITE
@@ -75,7 +74,6 @@ class GraniteModel(LlamaModel):


@ModelBase.register("GraniteMoeForCausalLM", "GraniteMoeSharedForCausalLM")
-@ModelBase.example("ibm-granite/granite-3.1-3b-a800m-instruct")
 class GraniteMoeModel(GraniteModel):
    """Conversion for IBM's GraniteMoeForCausalLM"""
    model_arch = gguf.MODEL_ARCH.GRANITE_MOE
@@ -125,169 +123,7 @@ class GraniteMoeModel(GraniteModel):
        yield from super().modify_tensors(data_torch, name, bid)


-@ModelBase.register("GraniteSwitchForCausalLM")
-@ModelBase.example("ibm-granite/granite-switch-4.1-3b-preview")
-class GraniteSwitchModel(GraniteMoeModel):
-    """Dense, all-attention Granite with N per-token embedded LoRA adapters, stacked
-    over the adapter dim with a zero adapter at slot 0 (N = num_adapters + 1)."""
-    model_arch = gguf.MODEL_ARCH.GRANITE_SWITCH
-
-    # permute q/k per-slice below (NORM-rope layout), not via the parent's auto-permute
-    undo_permute = False
-
-    def __init__(self, *args, **kwargs):
-        super().__init__(*args, **kwargs)
-        # the weightless switch reserves one cache slot: one fewer block than num_hidden_layers
-        self.block_count = self.block_count - 1
-        self.tensor_map = gguf.get_tensor_name_map(self.model_arch, self.block_count)
-
-        self._n_adapters = int(self.hparams["num_adapters"])
-        self._max_lora_rank = int(self.hparams["max_lora_rank"])
-        self._n_slots = self._n_adapters + 1  # +1 for the zero slot at index 0
-
-        n_head = int(self.hparams["num_attention_heads"])
-        n_kv_head = int(self.hparams["num_key_value_heads"])
-        head_dim = (
-            self.hparams.get("projection_head_dim")
-            or self.hparams.get("head_dim")
-            or (self.hparams["hidden_size"] // n_head)
-        )
-        self._n_head = n_head
-        self._n_kv_head = n_kv_head
-        self._head_dim = int(head_dim)
-        self._q_size = n_head * self._head_dim
-        self._kv_size = n_kv_head * self._head_dim
-
-    def set_gguf_parameters(self):
-        super().set_gguf_parameters()
-
-        # dense: pin expert_used_count to 0 (config carries a leftover num_experts_per_tok)
-        if not self.hparams.get("num_local_experts"):
-            self.gguf_writer.add_expert_used_count(0)
-
-        self.gguf_writer.add_adapter_count(self._n_adapters)
-        self.gguf_writer.add_adapter_lora_rank(self._max_lora_rank)
-        self.gguf_writer.add_adapter_token_ids_activate(self.hparams["adapter_token_ids"])
-        self.gguf_writer.add_adapter_token_ids_substitute(self.hparams["adapter_substitute_token_ids"])
-        router_gain = float(self.hparams.get("control_token_gain", 15.0))
-        self.gguf_writer.add_adapter_router_gain(router_gain)
-        logger.info("gguf: (graniteswitch) num_adapters=%s max_lora_rank=%s n_slots=%s router_gain=%s", self._n_adapters, self._max_lora_rank, self._n_slots, router_gain)
-
-    def _lora_a(self, data: Tensor) -> Tensor:
-        # on-disk A: [n_adapters, 1, max_rank, in] -> [n_adapters+1, max_rank, in]
-        a = data.squeeze(1)
-        zero = torch.zeros_like(a[:1])
-        return torch.cat([zero, a], dim=0).contiguous()
-
-    def _lora_b(self, data: Tensor, permute_n_head: int | None = None) -> Tensor:
-        # on-disk B: [n_adapters, 1, out, max_rank] -> [n_adapters+1, out, max_rank]
-        b = data.squeeze(1)
-        if permute_n_head is not None:
-            # permute each adapter's B output rows to match the permuted q/k base
-            b = torch.stack([self.permute(b[i], permute_n_head, permute_n_head) for i in range(b.shape[0])], dim=0)
-        zero = torch.zeros_like(b[:1])
-        return torch.cat([zero, b], dim=0).contiguous()
-
-    def modify_tensors(self, data_torch: Tensor, name: str, bid: int | None) -> Iterable[tuple[str, Tensor]]:
-        T = gguf.MODEL_TENSOR
-
-        # skip the weightless switch + control-token buffers (rebuilt at load time)
-        bare = name.split(".")[-1]
-        if (
-            name.startswith("model.switch.") or name.startswith("switch.")
-            or bare in ("adapter_token_ids", "control_to_substitute_lut")
-        ):
-            return
-
-        if "self_attn.qkv_proj" in name:
-            if name.endswith("base_layer.weight"):
-                # fused [q|k|v] rows: permute q/k row-blocks for ggml's NORM-rope layout
-                q, k, v = data_torch.split([self._q_size, self._kv_size, self._kv_size], dim=0)
-                q = self.permute(q, self._n_head, self._n_head)
-                k = self.permute(k, self._n_kv_head, self._n_kv_head)
-                fused = torch.cat([q, k, v], dim=0)
-                yield (self.format_tensor_name(T.ATTN_QKV, bid), fused)
-                return
-            if "lora_A_slices." in name:
-                slot = int(name.rsplit(".", 1)[1])
-                key = {0: T.ATTN_Q, 1: T.ATTN_K, 2: T.ATTN_V}[slot]
-                yield (self.format_tensor_name(key, bid, suffix=".lora_a"), self._lora_a(data_torch))
-                return
-            if "lora_B_slices." in name:
-                slot = int(name.rsplit(".", 1)[1])
-                key, ph = {
-                    0: (T.ATTN_Q, self._n_head),
-                    1: (T.ATTN_K, self._n_kv_head),
-                    2: (T.ATTN_V, None),
-                }[slot]
-                yield (self.format_tensor_name(key, bid, suffix=".lora_b"), self._lora_b(data_torch, ph))
-                return
-            raise ValueError(f"Unexpected qkv_proj tensor: {name}")
-
-        if "self_attn.o_proj" in name:
-            if name.endswith("base_layer.weight"):
-                yield (self.format_tensor_name(T.ATTN_OUT, bid), data_torch)
-                return
-            if name.endswith("lora_A"):
-                yield (self.format_tensor_name(T.ATTN_OUT, bid, suffix=".lora_a"), self._lora_a(data_torch))
-                return
-            if name.endswith("lora_B"):
-                yield (self.format_tensor_name(T.ATTN_OUT, bid, suffix=".lora_b"), self._lora_b(data_torch))
-                return
-            raise ValueError(f"Unexpected o_proj tensor: {name}")
-
-        if "shared_mlp.input_linear" in name:
-            ffn = self.hparams["shared_intermediate_size"]
-            if name.endswith("base_layer.weight"):
-                gate, up = data_torch.split([ffn, ffn], dim=0)
-                yield (self.format_tensor_name(T.FFN_GATE, bid), gate)
-                yield (self.format_tensor_name(T.FFN_UP, bid), up)
-                return
-            if "lora_A_slices." in name:
-                slot = int(name.rsplit(".", 1)[1])
-                key = {0: T.FFN_GATE, 1: T.FFN_UP}[slot]
-                yield (self.format_tensor_name(key, bid, suffix=".lora_a"), self._lora_a(data_torch))
-                return
-            if "lora_B_slices." in name:
-                slot = int(name.rsplit(".", 1)[1])
-                key = {0: T.FFN_GATE, 1: T.FFN_UP}[slot]
-                yield (self.format_tensor_name(key, bid, suffix=".lora_b"), self._lora_b(data_torch))
-                return
-            raise ValueError(f"Unexpected shared_mlp.input_linear tensor: {name}")
-
-        if "shared_mlp.output_linear" in name:
-            if name.endswith("base_layer.weight"):
-                yield (self.format_tensor_name(T.FFN_DOWN, bid), data_torch)
-                return
-            if name.endswith("lora_A"):
-                yield (self.format_tensor_name(T.FFN_DOWN, bid, suffix=".lora_a"), self._lora_a(data_torch))
-                return
-            if name.endswith("lora_B"):
-                yield (self.format_tensor_name(T.FFN_DOWN, bid, suffix=".lora_b"), self._lora_b(data_torch))
-                return
-            raise ValueError(f"Unexpected shared_mlp.output_linear tensor: {name}")
-
-        if bid is not None and ".layers." in name and (
-            "input_layernorm" in name or "post_attention_layernorm" in name
-        ):
-            key = T.ATTN_NORM if "input_layernorm" in name else T.FFN_NORM
-            yield (self.format_tensor_name(key, bid), data_torch)
-            return
-
-        if name in ("model.embed_tokens.weight", "embed_tokens.weight"):
-            yield (self.format_tensor_name(T.TOKEN_EMBD), data_torch)
-            return
-        if name in ("model.norm.weight", "norm.weight"):
-            yield (self.format_tensor_name(T.OUTPUT_NORM), data_torch)
-            return
-        if name == "lm_head.weight":
-            return  # tied to token_embd
-
-        raise ValueError(f"graniteswitch: unhandled tensor {name!r} (bid={bid})")
-
-
@ModelBase.register("GraniteMoeHybridForCausalLM", "BambaForCausalLM")
-@ModelBase.example("ibm-granite/granite-4.0-h-tiny", "ibm-ai-platform/Bamba-9B-v2")
 class GraniteHybridModel(Mamba2Model, GraniteMoeModel):
    """GraniteHybrid is a hybrid SSM + Attention model that uses Mamba2 SSM
    layers and optionally uses MoE w/ a shared expert"""
@@ -430,7 +266,6 @@ class GraniteHybridModel(Mamba2Model, GraniteMoeModel):


@ModelBase.register("GraniteSpeechForConditionalGeneration")
-@ModelBase.example("ibm-granite/granite-speech-3.3-2b", "ibm-granite/granite-4.0-1b-speech")
 class GraniteSpeechMmprojModel(MmprojModel):
    has_vision_encoder = False
    has_audio_encoder = True
@@ -514,7 +349,6 @@ class GraniteSpeechMmprojModel(MmprojModel):


@ModelBase.register("GraniteSpeechPlusForConditionalGeneration")
-@ModelBase.example("ibm-granite/granite-speech-4.1-2b-plus")
 class GraniteSpeechPlusMmprojModel(GraniteSpeechMmprojModel):
    """Conversion for GraniteSpeechPlus - extends GraniteSpeech with feature layer concatenation"""
    has_vision_encoder = False
@@ -543,7 +377,6 @@ class GraniteSpeechPlusMmprojModel(GraniteSpeechMmprojModel):


@ModelBase.register("Granite4VisionForConditionalGeneration")
-@ModelBase.example("ibm-granite/granite-4.0-3b-vision")
 class Granite4VisionMmprojModel(MmprojModel):
    has_vision_encoder = True
    has_audio_encoder = False
@@ -13,7 +13,6 @@ from .base import ModelBase, TextModel, gguf, logger


@ModelBase.register("GrokForCausalLM", "Grok1ForCausalLM")
-@ModelBase.example("keyfan/grok-1-hf")
 class GrokModel(TextModel):
    model_arch = gguf.MODEL_ARCH.GROK

@@ -11,7 +11,6 @@ from .base import ModelBase, TextModel, gguf, logger


@ModelBase.register("GroveMoeForCausalLM", "modeling_grove_moe.GroveMoeForCausalLM")
-@ModelBase.example("inclusionAI/GroveMoE-Inst")
 class GroveMoeModel(TextModel):
    model_arch = gguf.MODEL_ARCH.GROVEMOE

@@ -17,7 +17,6 @@ from .qwen import QwenModel


@ModelBase.register("HunYuanMoEV1ForCausalLM")
-@ModelBase.example("tencent/Hunyuan-A13B-Instruct")
 class HunYuanMoEModel(TextModel):
    model_arch = gguf.MODEL_ARCH.HUNYUAN_MOE

@@ -155,7 +154,6 @@ class HunYuanMoEModel(TextModel):


@ModelBase.register("HunYuanDenseV1ForCausalLM")
-@ModelBase.example("tencent/Hunyuan-4B-Instruct")
 class HunYuanModel(TextModel):
    model_arch = gguf.MODEL_ARCH.HUNYUAN_DENSE

@@ -292,7 +290,6 @@ class HunYuanModel(TextModel):


@ModelBase.register("HunYuanVLForConditionalGeneration")
-@ModelBase.example("tencent/HunyuanOCR")
 class HunyuanVLVisionModel(MmprojModel):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
@@ -336,7 +333,6 @@ class HunyuanVLVisionModel(MmprojModel):


@ModelBase.register("HunYuanVLForConditionalGeneration")
-@ModelBase.example("tencent/HunyuanOCR")
 class HunyuanVLTextModel(HunYuanModel):
    model_arch = gguf.MODEL_ARCH.HUNYUAN_VL

@@ -369,7 +365,6 @@ class HunyuanVLTextModel(HunYuanModel):


@ModelBase.register("HYV3ForCausalLM")
-@ModelBase.example("tencent/Hy3")
 class HYV3Model(TextModel):
    model_arch = gguf.MODEL_ARCH.HY_V3
    supports_mtp_export = True
@@ -14,7 +14,6 @@ from .llama import LlamaModel


@ModelBase.register("InternLM2ForCausalLM")
-@ModelBase.example("internlm/internlm2-chat-7b")
 class InternLM2Model(TextModel):
    model_arch = gguf.MODEL_ARCH.INTERNLM2

@@ -171,7 +170,6 @@ class InternLM2Model(TextModel):


@ModelBase.register("InternLM3ForCausalLM")
-@ModelBase.example("internlm/internlm3-8b-instruct")
 class InternLM3Model(TextModel):
    model_arch = gguf.MODEL_ARCH.LLAMA

@@ -9,7 +9,6 @@ from .base import MmprojModel, ModelBase, gguf


@ModelBase.register("InternVisionModel")
-@ModelBase.example("OpenGVLab/InternVL3-2B", "OpenGVLab/InternVL2_5-1B")
 class InternVisionModel(MmprojModel):

    min_dynamic_tiles: int = 0
@@ -11,8 +11,6 @@ from .base import ModelBase, TextModel, gguf


@ModelBase.register("Jais2ForCausalLM")
-# [TAG_HF_EXAMPLE_GATED] inceptionai/Jais-2-8B-Chat is gated
-# [TAG_HF_EXAMPLE_MISSING]
 class Jais2Model(TextModel):
    model_arch = gguf.MODEL_ARCH.JAIS2

@@ -24,7 +22,6 @@ class Jais2Model(TextModel):


@ModelBase.register("JAISLMHeadModel")
-@ModelBase.example("inceptionai/jais-family-590m")
 class JaisModel(TextModel):
    model_arch = gguf.MODEL_ARCH.JAIS

@@ -11,7 +11,6 @@ from .base import ModelBase, TextModel, gguf, logger


@ModelBase.register("JambaForCausalLM")
-@ModelBase.example("ai21labs/Jamba-v0.1")
 class JambaModel(TextModel):
    model_arch = gguf.MODEL_ARCH.JAMBA

@@ -11,7 +11,6 @@ from .llama import LlamaModel


@ModelBase.register("JanusForConditionalGeneration")
-@ModelBase.example("deepseek-community/Janus-Pro-1B")
 class JanusProModel(LlamaModel):
    model_arch = gguf.MODEL_ARCH.LLAMA  # reuse Llama arch

@@ -35,7 +34,6 @@ class JanusProModel(LlamaModel):


@ModelBase.register("JanusForConditionalGeneration")
-@ModelBase.example("deepseek-community/Janus-Pro-1B")
 class JanusProVisionModel(MmprojModel):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
@@ -1,376 +0,0 @@
-from __future__ import annotations
-
-import re
-from pathlib import Path
-from typing import Callable, Iterable, Iterator, TYPE_CHECKING
-
-import numpy as np
-import torch
-
-if TYPE_CHECKING:
-    from torch import Tensor
-
-from .base import LazyTorchTensor, ModelBase, TextModel, gguf, logger
-
-from .kimi_linear import KimiLinearModel
-
-
-@ModelBase.register("KimiK3ForConditionalGeneration")
-@ModelBase.example("moonshotai/Kimi-K3")
-class KimiK3Model(TextModel):
-    """
-    Kimi-K3 text model (KimiLinearForCausalLM under a `language_model.` prefix).
-
-    Shares the hybrid MLA + KDA skeleton with kimi-linear, but that converter
-    cannot load it: K3 adds cross-layer attention residuals, a latent MoE, the
-    situ activation, an MLA output gate and a full-rank KDA gate.
-
-    The vision tower and mm_projector are skipped - text only for now.
-    """
-
-    model_arch = gguf.MODEL_ARCH.KIMI_K3
-
-    _experts: list[dict[str, Tensor]] | None = None
-
-    # `<x>_res_norm.weight` and `<x>_res_proj.weight` are only used as their
-    # elementwise product, so they are fused into one [n_embd] vector here.
-    # they arrive apart, so buffer the first one and tag it with its kind.
-    _res_parts: dict[str, tuple[str, Tensor]]
-
-    # HF suffix -> (gguf tensor, per-layer?)
-    _RES_FUSIONS = {
-        "self_attention_res": (gguf.MODEL_TENSOR.ATTN_RES_SCORE, True),
-        "mlp_res":            (gguf.MODEL_TENSOR.FFN_RES_SCORE,   True),
-        "output_attn_res":    (gguf.MODEL_TENSOR.OUTPUT_RES_SCORE, False),
-    }
-
-    # compressed-tensors MXFP4. the `language_model.` prefix is still there, as
-    # self.model_tensors is keyed by the raw checkpoint names
-    _MXFP4_FORMAT = "mxfp4-pack-quantized"
-    _MXFP4_EXPERT_RE = re.compile(
-        r"^(?:language_model\.)?model\.layers\.(\d+)"
-        r"\.block_sparse_moe\.experts\.(\d+)\.(w[123])\.weight_packed$"
-    )
-    _MXFP4_PROJ = {
-        "w1": gguf.MODEL_TENSOR.FFN_GATE_EXP,
-        "w2": gguf.MODEL_TENSOR.FFN_DOWN_EXP,
-        "w3": gguf.MODEL_TENSOR.FFN_UP_EXP,
-    }
-
-    def __init__(self, *args, **kwargs):
-        super().__init__(*args, **kwargs)
-        self._res_parts = {}
-
-    def set_vocab(self):
-        # K3 has the same TikToken vocab as K2, so kimi-linear's vocab handling works.
-        # borrowed, not inherited: the method only touches TextModel members, and K3
-        # shares none of kimi-linear's tensor layout.
-        KimiLinearModel.set_vocab(self)  # ty: ignore[invalid-argument-type]
-
-        # ...but that forces eos to the tokenizer's eos_id, which is [EOS], the
-        # document terminator. K3's config says <|end_of_msg|>, the turn terminator;
-        # with [EOS] the generation never stops at the end of a turn.
-        if (eos := self.hparams.get("eos_token_id")) is not None:
-            logger.info(f"restoring configured eos_token_id {eos} (kimi-linear forces the tokenizer's)")
-            self.gguf_writer.add_eos_token_id(eos)
-
-        # K3 renders chats in python (encoding_k3.py) and ships no jinja template,
-        # so add the bundled one when the model has none
-        if gguf.SpecialVocab(self.dir_model, load_merges=False).chat_template is None:
-            template_path = Path(__file__).parent.parent / "models" / "templates" / "Kimi-K3.jinja"
-            logger.info(f"gguf: model has no chat template, using {template_path.name}")
-            self.gguf_writer.add_chat_template(template_path.read_text(encoding="utf-8"))
-
-    #
-    # compressed-tensors MXFP4 -> ggml MXFP4
-    #
-
-    def _is_mxfp4_packed(self) -> bool:
-        quant_config = self.hparams.get("quantization_config") or {}
-        return (quant_config.get("quant_method") == "compressed-tensors"
-                and quant_config.get("format") == self._MXFP4_FORMAT)
-
-    def dequant_model(self):
-        if not self._is_mxfp4_packed():
-            return super().dequant_model()
-
-        # skipping base.py's dequant is only safe if the experts are the only
-        # quantized tensors, so check it
-        stray = [n for n in self.model_tensors
-                 if n.endswith(".weight_packed") and not self._MXFP4_EXPERT_RE.match(n)]
-        if stray:
-            raise NotImplementedError(
-                f"{len(stray)} MXFP4 tensor(s) outside the routed experts, e.g. {stray[0]!r}; "
-                "only the routed experts have a repack path"
-            )
-
-    def _mxfp4_expert_tensor(self, loaders: list[tuple[Callable[[], Tensor], Callable[[], Tensor]]]):
-        """
-        One stacked [n_expert, rows, cols] MXFP4 tensor, built lazily.
-
-        gguf_writer holds every added tensor until the final write, so building
-        this eagerly (like the DeepSeek-V4 path does) keeps all ~1.38 TB of
-        experts in memory. lazy means only the tensor being written is resident.
-        """
-        # meta shapes, so this does not read any weights
-        rows, packed_cols = loaders[0][0]().shape
-        n_blocks = (packed_cols * 2) // 32
-        byte_shape = (len(loaders), rows, n_blocks * 17)
-
-        def load(fns: list[tuple[Callable[[], Tensor], Callable[[], Tensor]]]) -> np.ndarray:
-            out = np.empty(byte_shape, dtype=np.uint8)
-            for eid, (packed_fn, scale_fn) in enumerate(fns):
-                out[eid] = self.repack_mxfp4_blocks(
-                    LazyTorchTensor.to_eager(packed_fn()),
-                    LazyTorchTensor.to_eager(scale_fn()),
-                )
-            return out
-
-        # loaders goes through args, not the closure, so that `func` matches
-        # LazyBase's single-argument shape
-        return gguf.LazyNumpyTensor(
-            meta=gguf.LazyNumpyTensor.meta_with_dtype_and_shape(np.uint8, byte_shape),
-            args=(loaders,),
-            func=load,
-        )
-
-    def _write_mxfp4_experts(self) -> None:
-        n_experts = self.hparams["num_experts"]
-
-        # (bid, wid) -> {expert id: (packed name, scale name)}
-        groups: dict[tuple[int, str], dict[int, tuple[str, str]]] = {}
-        for name in self.model_tensors:
-            m = self._MXFP4_EXPERT_RE.match(name)
-            if m is None:
-                continue
-            bid, eid, wid = int(m.group(1)), int(m.group(2)), m.group(3)
-            scale_name = name.removesuffix("_packed") + "_scale"
-            if scale_name not in self.model_tensors:
-                raise KeyError(f"missing {scale_name} for {name}")
-            groups.setdefault((bid, wid), {})[eid] = (name, scale_name)
-
-        consumed: list[str] = []
-        for (bid, wid), experts in sorted(groups.items()):
-            missing = [e for e in range(n_experts) if e not in experts]
-            if missing:
-                raise KeyError(
-                    f"layer {bid} {wid}: {len(missing)} of {n_experts} experts missing, "
-                    f"first is {missing[0]}"
-                )
-            if len(experts) != n_experts:
-                raise KeyError(f"layer {bid} {wid}: {len(experts)} experts, expected {n_experts}")
-
-            loaders = []
-            for eid in range(n_experts):
-                packed_name, scale_name = experts[eid]
-                loaders.append((self.model_tensors[packed_name], self.model_tensors[scale_name]))
-                consumed += [packed_name, scale_name]
-
-            data = self._mxfp4_expert_tensor(loaders)
-            new_name = self.format_tensor_name(self._MXFP4_PROJ[wid], bid)
-            shape = gguf.quant_shape_from_byte_shape(data.shape, gguf.GGMLQuantizationType.MXFP4)
-            logger.info(
-                f"{new_name}: repacked {n_experts} experts to MXFP4, "
-                f"shape = {{{', '.join(str(n) for n in reversed(shape))}}}"
-            )
-            self.gguf_writer.add_tensor(new_name, data, raw_dtype=gguf.GGMLQuantizationType.MXFP4)
-
-        for name in consumed:
-            del self.model_tensors[name]
-
-    def generate_extra_tensors(self) -> Iterable[tuple[str, Tensor]]:
-        # not a generator on purpose: base.py chains this with get_tensors(), so the
-        # tensors used here must be removed from model_tensors before that starts
-        if self._is_mxfp4_packed():
-            self._write_mxfp4_experts()
-        return ()
-
-    def get_tensors(self) -> Iterator[tuple[str, Tensor]]:
-        for name, data in super().get_tensors():
-            if name.startswith(("vision_tower.", "mm_projector.")):
-                continue  # text only
-            if name.startswith("language_model."):
-                name = name[len("language_model."):]
-            yield name, data
-
-    def set_gguf_parameters(self):
-        # MLA is served as MQA with a single large head, then decompressed
-        self.hparams["num_key_value_heads"] = 1
-
-        super().set_gguf_parameters()
-        self.gguf_writer.add_vocab_size(self.hparams["vocab_size"])
-
-        linear_attn_config = self.hparams["linear_attn_config"]
-
-        # n_head_kv == 0 marks a KDA (recurrent) layer. the layer lists are 1-indexed,
-        # as KimiLinearConfig.is_kda_layer uses (layer_idx + 1)
-        full_attn_layers = linear_attn_config["full_attn_layers"]
-        n_kv_heads = [
-            self.hparams["num_key_value_heads"] if (il + 1) in full_attn_layers else 0
-            for il in range(self.hparams["num_hidden_layers"])
-        ]
-        assert len(n_kv_heads) == self.hparams["num_hidden_layers"]
-        self.gguf_writer.add_head_count_kv(n_kv_heads)
-
-        # --- KDA ---
-        self.gguf_writer.add_ssm_conv_kernel(linear_attn_config["short_conv_kernel_size"])
-        self.gguf_writer.add_kda_head_dim(linear_attn_config["head_dim"])
-        if (lb := linear_attn_config.get("gate_lower_bound")) is not None:
-            self.gguf_writer.add_kda_gate_lower_bound(lb)
-
-        # --- MLA ---
-        if (q_lora_rank := self.hparams.get("q_lora_rank")) is not None:
-            self.gguf_writer.add_q_lora_rank(q_lora_rank)
-        kv_lora_rank = self.hparams["kv_lora_rank"]
-        self.gguf_writer.add_kv_lora_rank(kv_lora_rank)
-
-        qk_nope_head_dim = self.hparams["qk_nope_head_dim"]
-        qk_rope_head_dim = self.hparams["qk_rope_head_dim"]
-        v_head_dim = self.hparams["v_head_dim"]
-        # K3 is nope-only; qk_rope_head_dim still sizes the un-absorbed part of K
-        assert self.hparams.get("mla_use_nope"), "K3 MLA is expected to be nope-only"
-        self.gguf_writer.add_rope_dimension_count(qk_rope_head_dim)
-        # MLA is served as MQA, so the cache holds the compressed latent
-        self.gguf_writer.add_key_length(kv_lora_rank + qk_rope_head_dim)
-        self.gguf_writer.add_value_length(kv_lora_rank)
-        self.gguf_writer.add_key_length_mla(qk_nope_head_dim + qk_rope_head_dim)
-        self.gguf_writer.add_value_length_mla(v_head_dim)
-
-        # --- MoE ---
-        self.gguf_writer.add_expert_feed_forward_length(self.hparams["moe_intermediate_size"])
-        self.gguf_writer.add_expert_shared_count(self.hparams["num_shared_experts"])
-        self.gguf_writer.add_leading_dense_block_count(self.hparams["first_k_dense_replace"])
-        self.gguf_writer.add_expert_weights_scale(self.hparams["routed_scaling_factor"])
-        self.gguf_writer.add_expert_weights_norm(self.hparams["moe_renormalize"])
-        assert self.hparams["moe_router_activation_func"] == "sigmoid"
-        self.gguf_writer.add_expert_gating_func(gguf.ExpertGatingFuncType.SIGMOID)
-        # latent MoE: routed experts live in a down-projected space
-        if (latent := self.hparams.get("routed_expert_hidden_size")) is not None:
-            self.gguf_writer.add_expert_latent_length(latent)
-
-        # --- situ activation ---
-        assert self.hparams["hidden_act"] == "situ", \
-            f"unexpected hidden_act {self.hparams['hidden_act']!r}"
-        self.gguf_writer.add_activation_situ_beta(self.hparams["activation_situ_beta"])
-        self.gguf_writer.add_activation_situ_linear_beta(self.hparams["activation_situ_linear_beta"])
-
-        # --- cross-layer attention residuals ---
-        self.gguf_writer.add_attn_res_block_size(self.hparams["attn_res_block_size"])
-
-    def prepare_tensors(self):
-        super().prepare_tensors()
-        if self._experts is not None:
-            leftover = [k for d in self._experts for k in d.keys()]
-            if leftover:
-                raise ValueError(f"Unprocessed experts: {leftover}")
-        if self._res_parts:
-            raise ValueError(f"Unpaired attention-residual tensors: {sorted(self._res_parts)}")
-        if self._is_mxfp4_packed():
-            # label the file for what it is; prepare_metadata runs after this
-            self._is_mxfp4 = True
-            self.ftype = gguf.LlamaFileType.MOSTLY_MXFP4_MOE
-
-    def _try_fuse_res(self, data_torch: Tensor, name: str, bid: int | None):
-        """
-        Pair <x>_res_norm.weight with <x>_res_proj.weight and emit their product.
-
-        Returns None if this is not a res tensor, [] if buffered until its pair.
-        """
-        for prefix, (tensor_id, per_layer) in self._RES_FUSIONS.items():
-            for kind in ("norm", "proj"):
-                if not name.endswith(f"{prefix}_{kind}.weight"):
-                    continue
-                key = f"{prefix}.{bid}"
-                other = self._res_parts.pop(key, None)
-                if other is None:
-                    self._res_parts[key] = (kind, data_torch)
-                    return []
-                other_kind, other_data = other
-                assert other_kind != kind, f"duplicate {kind} for {key}"
-                norm = data_torch if kind == "norm" else other_data
-                proj = data_torch if kind == "proj" else other_data
-                fused = norm.float().flatten() * proj.float().flatten()
-                # ".weight" suffix matches the convention map_tensor_name applies
-                new_name = (self.format_tensor_name(tensor_id, bid) if per_layer
-                            else gguf.TENSOR_NAMES[tensor_id] + ".weight")
-                logger.info(f"fused {prefix}_norm * {prefix}_proj -> {new_name}")
-                return [(new_name, fused)]
-        return None
-
-    def modify_tensors(self, data_torch: Tensor, name: str, bid: int | None) -> Iterable[tuple[str, Tensor]]:
-        # --- cross-layer attention residuals: fuse norm * proj ---
-        fused = self._try_fuse_res(data_torch, name, bid)
-        if fused is not None:
-            yield from fused
-            return
-
-        # --- KDA conv1d: HF [d_inner, 1, d_conv] -> ggml ne [d_conv, 1, d_inner, 1] ---
-        # GGUF reverses the numpy shape on write, so target numpy (1, d_inner, 1, d_conv).
-        # conv_step varies fastest in both layouts, so this is a pure reshape.
-        if name.endswith((".q_conv1d.weight", ".k_conv1d.weight", ".v_conv1d.weight")):
-            if data_torch.ndim == 3:      # [d_inner, 1, d_conv]
-                d_inner, _, d_conv = data_torch.shape
-            elif data_torch.ndim == 2:    # [d_inner, d_conv]
-                d_inner, d_conv = data_torch.shape
-            else:
-                raise ValueError(f"unexpected conv1d rank {data_torch.ndim} for {name}")
-            data_torch = data_torch.reshape(1, d_inner, 1, d_conv)
-
-        # -exp(A_log) is folded here so the graph does not have to
-        if name.endswith(".A_log"):
-            n_head = self.hparams["num_attention_heads"]
-            data_torch = -torch.exp(data_torch.float()[:n_head])
-
-        # dt_bias -> the name SSM_DT's mapping expects
-        if name.endswith(".dt_bias"):
-            name = name.rpartition(".dt_bias")[0] + ".dt_proj.bias"
-
-        # --- g_proj is two different tensors sharing one HF name ---
-        # KDA layers: full-rank gate, [d_inner, n_embd]  (replaces g_a/g_b)
-        # MLA layers: output gate,    [n_head*v_head_dim, n_embd]
-        # Name-based mapping cannot tell them apart, so resolve by layer type.
-        if name.endswith(".self_attn.g_proj.weight"):
-            assert bid is not None
-            is_kda = (bid + 1) not in self.hparams["linear_attn_config"]["full_attn_layers"]
-            tensor_id = gguf.MODEL_TENSOR.SSM_G if is_kda else gguf.MODEL_TENSOR.ATTN_GATE
-            yield self.format_tensor_name(tensor_id, bid), data_torch
-            return
-
-        # --- routed experts: stack per-expert 2D weights into one 3D tensor ---
-        if ".block_sparse_moe.experts." in name:
-            n_experts = self.hparams["num_experts"]
-            assert bid is not None
-
-            if self._experts is None:
-                self._experts = [{} for _ in range(self.block_count)]
-            self._experts[bid][name] = data_torch
-
-            if len(self._experts[bid]) < n_experts * 3:
-                return
-
-            # w1: gate, w2: down, w3: up
-            for wid, tensor_id in (("w1", gguf.MODEL_TENSOR.FFN_GATE_EXP),
-                                   ("w2", gguf.MODEL_TENSOR.FFN_DOWN_EXP),
-                                   ("w3", gguf.MODEL_TENSOR.FFN_UP_EXP)):
-                datas = []
-                for xid in range(n_experts):
-                    ename = f"model.layers.{bid}.block_sparse_moe.experts.{xid}.{wid}.weight"
-                    datas.append(self._experts[bid].pop(ename))
-                stacked = torch.stack(datas, dim=0)
-                yield from super().modify_tensors(stacked, self.format_tensor_name(tensor_id, bid), bid)
-            return
-
-        # --- MLA absorption: split kv_b into k_b (transposed) and v_b ---
-        if name.endswith("kv_b_proj.weight"):
-            n_head_kv = self.hparams["num_key_value_heads"]
-            v_head_dim = self.hparams["v_head_dim"]
-            qk_nope_head_dim = self.hparams["qk_nope_head_dim"]
-            assert data_torch.shape[0] == n_head_kv * (v_head_dim + qk_nope_head_dim)
-            kv_b = data_torch.view(n_head_kv, v_head_dim + qk_nope_head_dim, data_torch.shape[-1])
-            k_b, v_b = torch.split(kv_b, [qk_nope_head_dim, v_head_dim], dim=1)
-            k_b = k_b.transpose(1, 2)
-            yield from super().modify_tensors(k_b, name.replace("kv_b_proj", "k_b_proj"), bid)
-            yield from super().modify_tensors(v_b, name.replace("kv_b_proj", "v_b_proj"), bid)
-            return
-
-        yield from super().modify_tensors(data_torch, name, bid)
@@ -13,7 +13,6 @@ from .qwen import QwenModel


@ModelBase.register("KimiLinearModel", "KimiLinearForCausalLM")
-@ModelBase.example("moonshotai/Kimi-Linear-48B-A3B-Instruct")
 class KimiLinearModel(TextModel):
    """Kimi-Linear model with hybrid MLA+KDA architecture"""
    model_arch = gguf.MODEL_ARCH.KIMI_LINEAR
@@ -11,7 +11,6 @@ from .base import MmprojModel, ModelBase, gguf


@ModelBase.register("KimiVLForConditionalGeneration")
-@ModelBase.example("moonshotai/Kimi-VL-A3B-Instruct")
 class KimiVLModel(MmprojModel):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
@@ -53,7 +52,6 @@ class KimiVLModel(MmprojModel):


@ModelBase.register("KimiK25ForConditionalGeneration")
-@ModelBase.example("moonshotai/Kimi-K2.5")
 class KimiK25Model(MmprojModel):
    """Kimi-K2.5 with MoonViT3d vision encoder"""

@@ -157,7 +155,6 @@ class KimiK25Model(MmprojModel):


@ModelBase.register("Glm5vForConditionalGeneration")
-# [TAG_HF_EXAMPLE_MISSING]
 class Glm5vModel(KimiK25Model):
    """GLM-5.2-Vision MoonViT3d encoder and projector

@@ -13,7 +13,6 @@ from .base import ModelBase, TextModel, gguf, logger


@ModelBase.register("LagunaForCausalLM")
-@ModelBase.example("poolside/Laguna-XS.2", "poolside/Laguna-S-2.1")
 class LagunaModel(TextModel):
    model_arch = gguf.MODEL_ARCH.LAGUNA
    _experts: list[dict] | None = None
@@ -13,7 +13,6 @@ from .gemma import ConformerAudioModel


@ModelBase.register("Lfm2ForCausalLM", "LFM2ForCausalLM")
-@ModelBase.example("LiquidAI/LFM2-1.2B", "LiquidAI/LFM2.5-350M")
 class LFM2Model(TextModel):
    model_arch = gguf.MODEL_ARCH.LFM2

@@ -66,7 +65,6 @@ class LFM2Model(TextModel):


@ModelBase.register("Lfm2Model", "Lfm2BidirectionalModel")
-@ModelBase.example("LiquidAI/LFM2.5-ColBERT-350M", "LiquidAI/LFM2.5-Embedding-350M")
 class LFM2ColBertModel(LFM2Model):
    model_arch = gguf.MODEL_ARCH.LFM2
    dense_tensor_name = "dense_2"
@@ -95,7 +93,6 @@ class LFM2ColBertModel(LFM2Model):


@ModelBase.register("Lfm2MoeForCausalLM")
-@ModelBase.example("LiquidAI/LFM2-8B-A1B")
 class LFM2MoeModel(TextModel):
    model_arch = gguf.MODEL_ARCH.LFM2MOE

@@ -169,7 +166,6 @@ class LFM2MoeModel(TextModel):


@ModelBase.register("Lfm2VlForConditionalGeneration")
-@ModelBase.example("LiquidAI/LFM2-VL-450M")
 class LFM2VLModel(MmprojModel):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
@@ -204,7 +200,6 @@ class LFM2VLModel(MmprojModel):


@ModelBase.register("Lfm2AudioForConditionalGeneration")
-@ModelBase.example("LiquidAI/LFM2.5-Audio-1.5B", "LiquidAI/LFM2-Audio-1.5B")
 class LFM2AudioModel(ConformerAudioModel):
    has_vision_encoder = False
    has_audio_encoder = True
@@ -243,7 +238,6 @@ class LFM2AudioModel(ConformerAudioModel):


@ModelBase.register("Lfm25AudioTokenizer")
-@ModelBase.example("LiquidAI/LFM2.5-Audio-1.5B")
 class LFM25AudioTokenizer(LFM2Model):
    model_arch = gguf.MODEL_ARCH.LFM2

@@ -11,7 +11,6 @@ from .llava import LlavaVisionModel


@ModelBase.register("LightOnOCRForConditionalGeneration")
-@ModelBase.example("lightonai/LightOnOCR-1B-1025")
 class LightOnOCRVisionModel(LlavaVisionModel):
    is_mistral_format = False
    use_break_tok = False
@@ -11,7 +11,6 @@ from .base import ModelBase, TextModel, gguf


@ModelBase.register("LLaDAModelLM")
-@ModelBase.example("GSAI-ML/LLaDA-8B-Instruct")
 class LLaDAModel(TextModel):
    model_arch = gguf.MODEL_ARCH.LLADA
    undo_permute = True
@@ -115,7 +114,6 @@ class LLaDAModel(TextModel):


@ModelBase.register("LLaDAMoEModel", "LLaDAMoEModelLM")
-@ModelBase.example("inclusionAI/LLaDA-MoE-7B-A1B-Instruct")
 class LLaDAMoEModel(TextModel):
    model_arch = gguf.MODEL_ARCH.LLADA_MOE

@@ -28,8 +28,6 @@ from .base import ModelBase, TextModel, gguf, logger
    "Eagle3DraftModel",
    "IQuestCoderForCausalLM",
    "LlamaModel")
-# [TAG_HF_EXAMPLE_GATED] meta-llama/Llama-3.2-1B-Instruct is gated
-@ModelBase.example("unsloth/Llama-3.2-1B-Instruct", "mistralai/Mistral-7B-Instruct-v0.3", "mistralai/Mixtral-8x7B-Instruct-v0.1")
 class LlamaModel(TextModel):
    model_arch = gguf.MODEL_ARCH.LLAMA
    undo_permute = True
@@ -361,7 +359,6 @@ class LlamaModel(TextModel):


@ModelBase.register("ArceeForCausalLM")
-@ModelBase.example("arcee-ai/AFM-4.5B")
 class ArceeModel(LlamaModel):
    model_arch = gguf.MODEL_ARCH.ARCEE

@@ -374,8 +371,6 @@ class ArceeModel(LlamaModel):
    "Llama4ForConditionalGeneration",
    "Llama4ForCausalLM",
 )
-# [TAG_HF_EXAMPLE_GATED] meta-llama/Llama-4-Scout-17B-16E-Instruct is gated
-@ModelBase.example("unsloth/Llama-4-Scout-17B-16E-Instruct")
 class Llama4Model(LlamaModel):
    model_arch = gguf.MODEL_ARCH.LLAMA4
    undo_permute = False
@@ -417,19 +412,16 @@ class Llama4Model(LlamaModel):


@ModelBase.register("LlamaBidirectionalModel")
-@ModelBase.example("nvidia/llama-embed-nemotron-8b")
 class LlamaEmbedNemotronModel(LlamaModel):
    model_arch = gguf.MODEL_ARCH.LLAMA_EMBED


@ModelBase.register("SmolLM3ForCausalLM")
-@ModelBase.example("HuggingFaceTB/SmolLM3-3B")
 class SmolLM3Model(LlamaModel):
    model_arch = gguf.MODEL_ARCH.SMOLLM3


@ModelBase.register("ApertusForCausalLM")
-@ModelBase.example("swiss-ai/Apertus-8B-Instruct-2509")
 class ApertusModel(LlamaModel):
    model_arch = gguf.MODEL_ARCH.APERTUS
    undo_permute = False
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
forforever73	5522498343	metal : port new kernels into the split sources Move the kernels added on master after the split (lightning indexer, DSv4 hyper-connections, silu_back, f16 bin ops) into the corresponding kernels/*.metal sources. Copied verbatim, no functional change.	2026-08-05 16:22:32 +08:00
Georgi Gerganov	db0a8ce844	ggml-metal: FWHT kernel for metal backend (#25924 )	2026-08-05 16:22:32 +08:00
Georgi Gerganov	b2d79a7e7f	metal: fuse snake activation (mul, sin, sqr, mul, add) (#25459 )	2026-08-05 16:22:32 +08:00
Georgi Gerganov	0b6cbb572c	metal : add Q2_0 support (#25419 )	2026-08-05 16:22:32 +08:00
Georgi Gerganov	7649856ea2	metal : add CONV_2D_DW (depthwise convolution) support (#21565 )	2026-08-05 16:22:32 +08:00
Georgi Gerganov	f348d51391	metal : add set_rows with src0 f16 (#25434 )	2026-08-05 16:22:32 +08:00
Georgi Gerganov	d2112aa30e	metal: add col2im_1d op (f32/f16/bf16) (#25176 )	2026-08-05 16:22:32 +08:00
YiChen Lv	ff8169ce01	metal : per-op source split + parallel compile (#24021 ) * preliminary extract common header * op source split * split metallib into 8 libs && load in parallel * derive kernel->library routing from functionNames * x-macro lib list + underscore filenames, dedup QK_NL, MRC fixes * op source split 8 to 20 * improve robustness of source fallback * clean up * change bool -> atomic_bool * only prepend headers that source actually includes * no semaphore, use GCD global queue * dedup library compile path, fix NSError lifetime, rename gla * relocate upstream concat/rope_back/repeat kernel changes into split files * move ggml-common.h from common.h into dequantize.h to shrink binary size --------- Co-authored-by: lvyichen <lvyichen@stepfun.com>	2026-08-05 16:22:32 +08:00