From 087f94d82e597a966c78d2c0afd2b34df561ea34 Mon Sep 17 00:00:00 2001 From: Pascal Date: Mon, 17 Aug 2026 21:32:15 +0200 Subject: [PATCH 01/48] doc: document MCP stdio servers and CORS defaults in the server README [no release] [no ci] (#26847) * doc: document MCP stdio servers and CORS defaults in the server README The MCP arguments were listed but nothing explained what an MCP server is or how to declare one. Cover the stdio transport, the config keys, the tool naming, and add a POSIX shell echo server as a minimal example. Also document the CORS behavior: the default reflected origin, the switch to localhost once tools are enabled, and the recommended setting per deployment. * doc: drop the inline MCP shell example from the server README The example parsed JSON-RPC by hand and sat in a page people copy paste from, into servers spawned with the privileges of llama-server. Point to the specification instead. Link the pull request that introduced the feature, and keep a short mcp.json snippet so the table of configuration keys has a declaration to refer to. --- tools/server/README.md | 52 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/tools/server/README.md b/tools/server/README.md index 017981fd1..f1e1facee 100644 --- a/tools/server/README.md +++ b/tools/server/README.md @@ -343,6 +343,58 @@ The server includes a set of built-in tools that enable the LLM to access the lo To use this feature, start the server with `--tools all`. You can also enable only specific tools by passing a comma-separated list: `--tools name1,name2,...`. Run `--help` for the full list of available tool names. +### MCP servers + +Besides the built-in tools, the server can expose tools coming from MCP servers, added in [#26062](https://github.com/ggml-org/llama.cpp/pull/26062). Only the stdio transport is supported: such a server is a child process reading JSON-RPC messages on its stdin and writing replies on its stdout, so nothing has to be started or maintained outside `llama-server`. + +Servers are declared in a Cursor-compatible JSON file: + +```json +{ + "mcpServers": { + "example": { "command": "/path/to/server", "args": [] } + } +} +``` + +```sh +llama-server -m model.gguf --mcp-servers-config mcp.json +``` + +The same JSON can be passed inline with `--mcp-servers-json`. Each entry under `mcpServers` accepts: + +| Key | Explanation | +| --- | ----------- | +| `command` | executable to spawn, required, entries without it are skipped | +| `args` | array of arguments | +| `env` | object merged over the parent environment | +| `cwd` | working directory of the child process | +| `timeout_ms` | per-tool-call timeout (default: 30000) | + +Every server is spawned once at startup to list its tools, then stopped, and respawned on demand when one of its tools is called. Tools are exposed as `_` alongside the built-in ones: they show up in the Web UI and in `GET /tools`, and the model calls them like any other tool. A name colliding with an already registered tool is skipped. This is independent of `--tools`, MCP servers can be the only tools available. + +The child process runs with the same privileges as the server, so only declare commands you trust. As with `--tools`, `--cors-origins` then defaults to `localhost`. + +Note: `--ui-mcp-proxy` is unrelated, it only lets the Web UI reach remote MCP servers from the browser. + +Any server written against the [MCP specification](https://modelcontextprotocol.io) works as is, whether it uses an official SDK or not: the transport is one JSON-RPC message per line on stdio, so a script wrapping an existing program is a valid server too. + +### CORS + +By default the server reflects any `Origin` header back with credentials allowed. This matches the old, always-on `*` behavior and is fine as long as the server only exposes stateless, read-only endpoints. + +Enabling `--tools` or `--agent` exposes file read/write over the API, so in that case `--cors-origins` defaults to `localhost` instead: only pages served from localhost can reach the server. Pass `--cors-origins` explicitly to override either default. + +Recommended `--cors-origins` setting, depending on where the server runs: + +| Deployment | Recommendation | +| ---------- | --------------- | +| Public | set an API key, put the server behind a reverse proxy, `--cors-origins` optional | +| Local network | set `--cors-origins` to your frontend's origin | +| Same machine | `--cors-origins localhost` (default once `--agent` is set) | + +Related flags: `--cors-origins`, `--cors-methods`, `--cors-headers`, `--cors-credentials` / `--no-cors-credentials`. Background and rationale: [#25655](https://github.com/ggml-org/llama.cpp/pull/25655). + ## Build `llama-server` is built alongside everything else from the root of the project From 058df671b214a7447f33b3e962501175b3a752f4 Mon Sep 17 00:00:00 2001 From: Eve <139727413+netrunnereve@users.noreply.github.com> Date: Mon, 17 Aug 2026 19:59:40 +0000 Subject: [PATCH 02/48] ci: more optimizations (#26983) * replace rpc job with cpu * remove vulkan cache * move windows to build vulkan --- .github/actions/linux-setup-vulkan/action.yml | 20 ------ .github/workflows/build-cache.yml | 27 -------- .github/workflows/build-cpu.yml | 14 +--- .github/workflows/build-rpc.yml | 66 ------------------ .github/workflows/build-vulkan.yml | 67 ++++++++++++++++--- 5 files changed, 58 insertions(+), 136 deletions(-) delete mode 100644 .github/actions/linux-setup-vulkan/action.yml delete mode 100644 .github/workflows/build-rpc.yml diff --git a/.github/actions/linux-setup-vulkan/action.yml b/.github/actions/linux-setup-vulkan/action.yml deleted file mode 100644 index 4d29837fe..000000000 --- a/.github/actions/linux-setup-vulkan/action.yml +++ /dev/null @@ -1,20 +0,0 @@ -name: "Linux - Setup Vulkan SDK" -description: "Setup Vulkan SDK for Linux" -inputs: - path: - description: "Installation path" - required: true - version: - description: "Vulkan SDK version" - required: true - -runs: - using: "composite" - steps: - - name: Setup Vulkan SDK - id: setup - uses: ./.github/actions/unarchive-tar - with: - url: https://sdk.lunarg.com/sdk/download/${{ inputs.version }}/linux/vulkan_sdk.tar.xz - path: ${{ inputs.path }} - strip: 1 diff --git a/.github/workflows/build-cache.yml b/.github/workflows/build-cache.yml index 2a1031728..604f84241 100644 --- a/.github/workflows/build-cache.yml +++ b/.github/workflows/build-cache.yml @@ -10,33 +10,6 @@ concurrency: cancel-in-progress: true jobs: - ubuntu-24-vulkan-cache: - runs-on: ubuntu-24.04 - - steps: - - name: Clone - id: checkout - uses: actions/checkout@v6 - - - name: Get latest Vulkan SDK version - id: vulkan_sdk_version - run: | - echo "VULKAN_SDK_VERSION=$(curl https://vulkan.lunarg.com/sdk/latest/linux.txt)" >> "$GITHUB_ENV" - - - name: Setup Cache - uses: actions/cache@v5 - id: cache-sdk - with: - path: ./vulkan_sdk - key: cache-gha-vulkan-sdk-${{ env.VULKAN_SDK_VERSION }}-${{ runner.os }} - - - name: Setup Vulkan SDK - if: steps.cache-sdk.outputs.cache-hit != 'true' - uses: ./.github/actions/linux-setup-vulkan - with: - path: ./vulkan_sdk - version: ${{ env.VULKAN_SDK_VERSION }} - #ubuntu-24-spacemit-cache: # runs-on: ubuntu-24.04 diff --git a/.github/workflows/build-cpu.yml b/.github/workflows/build-cpu.yml index df70f4d10..2016a57f8 100644 --- a/.github/workflows/build-cpu.yml +++ b/.github/workflows/build-cpu.yml @@ -21,6 +21,7 @@ on: paths: [ '.github/workflows/build-cpu.yml', '.github/workflows/build-cmake-pkg.yml', + 'ggml/src/ggml-rpc/**', '**/CMakeLists.txt', '**/.cmake', '**/*.h', @@ -123,7 +124,6 @@ jobs: env: OPENBLAS_VERSION: 0.3.23 SDE_VERSION: 9.33.0-2024-01-07 - VULKAN_VERSION: 1.4.357.0 strategy: matrix: @@ -134,9 +134,6 @@ jobs: - build: 'x64-openblas' arch: 'x64' defines: '-G "Ninja Multi-Config" -D CMAKE_TOOLCHAIN_FILE=cmake/x64-windows-llvm.cmake -DGGML_NATIVE=OFF -DLLAMA_BUILD_SERVER=ON -DGGML_RPC=ON -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON -DGGML_OPENMP=OFF -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS -DBLAS_INCLUDE_DIRS="$env:RUNNER_TEMP/openblas/include" -DBLAS_LIBRARIES="$env:RUNNER_TEMP/openblas/lib/openblas.lib"' - - build: 'x64-vulkan' - arch: 'x64' - defines: '-G "Ninja Multi-Config" -D CMAKE_TOOLCHAIN_FILE=cmake/x64-windows-llvm.cmake -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=OFF -DLLAMA_BUILD_SERVER=ON -DGGML_RPC=ON -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON -DGGML_VULKAN=ON' - build: 'arm64' arch: 'arm64' defines: '-G "Ninja Multi-Config" -D CMAKE_TOOLCHAIN_FILE=cmake/arm64-windows-llvm.cmake -DGGML_NATIVE=OFF -DLLAMA_BUILD_SERVER=ON' @@ -167,15 +164,6 @@ jobs: $lib = $(join-path $msvc 'bin\Hostx64\x64\lib.exe') & $lib /machine:x64 "/def:${env:RUNNER_TEMP}/openblas/lib/libopenblas.def" "/out:${env:RUNNER_TEMP}/openblas/lib/openblas.lib" /name:openblas.dll - - name: Install Vulkan SDK - id: get_vulkan - if: ${{ matrix.build == 'x64-vulkan' }} - run: | - curl.exe -o $env:RUNNER_TEMP/VulkanSDK-Installer.exe -L "https://sdk.lunarg.com/sdk/download/${env:VULKAN_VERSION}/windows/vulkansdk-windows-X64-${env:VULKAN_VERSION}.exe" - & "$env:RUNNER_TEMP\VulkanSDK-Installer.exe" --accept-licenses --default-answer --confirm-command install - Add-Content $env:GITHUB_ENV "VULKAN_SDK=C:\VulkanSDK\${env:VULKAN_VERSION}" - Add-Content $env:GITHUB_PATH "C:\VulkanSDK\${env:VULKAN_VERSION}\bin" - - name: Install Ninja id: install_ninja run: | diff --git a/.github/workflows/build-rpc.yml b/.github/workflows/build-rpc.yml deleted file mode 100644 index d04dc375b..000000000 --- a/.github/workflows/build-rpc.yml +++ /dev/null @@ -1,66 +0,0 @@ -name: CI (rpc) - -on: - workflow_dispatch: # allows manual triggering - push: - branches: - - master - paths: [ - '.github/workflows/build-rpc.yml', - '**/CMakeLists.txt', - '**/.cmake', - '**/*.h', - '**/*.hpp', - '**/*.c', - '**/*.cpp' - ] - - pull_request: - types: [opened, synchronize, reopened] - paths: [ - '.github/workflows/build-rpc.yml', - 'ggml/src/ggml-rpc/**' - ] - -concurrency: - group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }} - cancel-in-progress: true - -env: - GGML_NLOOP: 3 - GGML_N_THREADS: 1 - LLAMA_ARG_LOG_COLORS: 1 - LLAMA_ARG_LOG_PREFIX: 1 - LLAMA_ARG_LOG_TIMESTAMPS: 1 - -jobs: - ubuntu-24-rpc: - runs-on: ${{ 'ubuntu-24.04-arm' || 'ubuntu-24.04' }} - - continue-on-error: true - - steps: - - name: Clone - id: checkout - uses: actions/checkout@v6 - - - name: Dependencies - id: depends - run: | - sudo apt-get update - sudo apt-get install build-essential libssl-dev ninja-build - - - name: Build - id: cmake_build - run: | - cmake -B build \ - -G "Ninja" \ - -DCMAKE_BUILD_TYPE=Release \ - -DGGML_RPC=ON - time cmake --build build --config Release -j $(nproc) - - - name: Test - id: cmake_test - run: | - cd build - ctest -L main --verbose diff --git a/.github/workflows/build-vulkan.yml b/.github/workflows/build-vulkan.yml index 01113803f..15ff4b0af 100644 --- a/.github/workflows/build-vulkan.yml +++ b/.github/workflows/build-vulkan.yml @@ -93,19 +93,13 @@ jobs: run: | echo "VULKAN_SDK_VERSION=$(curl https://vulkan.lunarg.com/sdk/latest/linux.txt)" >> "$GITHUB_ENV" - - name: Use Vulkan SDK Cache - uses: actions/cache@v5 - id: cache-sdk - with: - path: ./vulkan_sdk - key: cache-gha-vulkan-sdk-${{ env.VULKAN_SDK_VERSION }}-${{ runner.os }} - - name: Setup Vulkan SDK - if: steps.cache-sdk.outputs.cache-hit != 'true' - uses: ./.github/actions/linux-setup-vulkan + id: setup + uses: ./.github/actions/unarchive-tar with: + url: https://sdk.lunarg.com/sdk/download/${{ env.VULKAN_SDK_VERSION }}/linux/vulkan_sdk.tar.xz path: ./vulkan_sdk - version: ${{ env.VULKAN_SDK_VERSION }} + strip: 1 - name: ccache uses: ggml-org/ccache-action@v1.2.21 @@ -133,3 +127,56 @@ jobs: # This is using llvmpipe and runs slower than other backends # test-backend-ops is too slow on llvmpipe, skip it ctest -L main -E test-backend-ops --verbose --timeout 900 + + windows: + runs-on: windows-2025 + + env: + VULKAN_VERSION: 1.4.357.0 + + steps: + - name: Clone + id: checkout + uses: actions/checkout@v6 + + - name: ccache + uses: ggml-org/ccache-action@v1.2.21 + with: + key: cpu-windows-2025-x64-vulkan + variant: ccache + evict-old-files: 1d + save: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} + + - name: Install Vulkan SDK + id: get_vulkan + run: | + curl.exe -o $env:RUNNER_TEMP/VulkanSDK-Installer.exe -L "https://sdk.lunarg.com/sdk/download/${env:VULKAN_VERSION}/windows/vulkansdk-windows-X64-${env:VULKAN_VERSION}.exe" + & "$env:RUNNER_TEMP\VulkanSDK-Installer.exe" --accept-licenses --default-answer --confirm-command install + Add-Content $env:GITHUB_ENV "VULKAN_SDK=C:\VulkanSDK\${env:VULKAN_VERSION}" + Add-Content $env:GITHUB_PATH "C:\VulkanSDK\${env:VULKAN_VERSION}\bin" + + - name: Install Ninja + id: install_ninja + run: | + choco install ninja + + - name: Build + id: cmake_build + run: | + cmake -S . -B build -G "Ninja Multi-Config" ` + -D CMAKE_TOOLCHAIN_FILE=cmake/x64-windows-llvm.cmake ` + -DCMAKE_BUILD_TYPE=Release ` + -DGGML_NATIVE=OFF ` + -DLLAMA_BUILD_SERVER=ON ` + -DGGML_RPC=ON ` + -DGGML_BACKEND_DL=ON ` + -DGGML_CPU_ALL_VARIANTS=ON ` + -DGGML_VULKAN=ON ` + -DLLAMA_BUILD_BORINGSSL=ON + cmake --build build --config Release -j ${env:NUMBER_OF_PROCESSORS} + + - name: Test + id: cmake_test + run: | + cd build + ctest -L main -C Release --verbose --timeout 900 From 0021a77de0a8966059dc94548fb3b96654e0bb12 Mon Sep 17 00:00:00 2001 From: Aleksander Grygier Date: Mon, 17 Aug 2026 22:23:22 +0200 Subject: [PATCH 03/48] ui: Refactor Built-In Tools naming (Server/Browser) (#27271) * server: rename built-in tools to server tools * ui: rename built-in tools to server/browser tools --- tools/server/README-dev.md | 2 +- tools/server/README.md | 12 +- tools/server/server-tools.cpp | 2 +- tools/server/server-tools.h | 2 +- tools/server/server.cpp | 2 +- .../ChatFormActionAddToolsSubmenu.svelte | 2 +- .../ChatFormCurrentWorkingDirectory.svelte | 4 +- .../ChatFormPickerMention.svelte | 2 +- .../ChatMessageToolCallBlock.svelte | 20 +-- .../ChatMessageToolCallBlockDefault.svelte | 4 +- ...atMessageToolCallBlockRunJavascript.svelte | 4 +- .../ChatMessageToolCall/ToolCallBlock.svelte | 6 +- .../ChatMessageToolCall/parsers/edit-file.ts | 2 +- .../parsers/exec-shell-command.ts | 2 +- .../parsers/file-glob-search.ts | 2 +- .../parsers/grep-search.ts | 2 +- .../ChatMessageToolCall/parsers/read-file.ts | 2 +- .../parsers/run-javascript.ts | 2 +- .../ChatMessageToolCall/parsers/write-file.ts | 2 +- ...tMessageActionCardPermissionRequest.svelte | 4 +- tools/ui/src/lib/components/app/chat/index.ts | 4 +- .../SettingsChat/SettingsChatToolsTab.svelte | 12 +- .../src/lib/components/app/settings/index.ts | 2 +- tools/ui/src/lib/constants/browser-info.ts | 4 +- .../lib/constants/built-in-tools.constants.ts | 52 -------- tools/ui/src/lib/constants/get-datetime.ts | 2 +- tools/ui/src/lib/constants/index.ts | 2 +- tools/ui/src/lib/constants/read-media.ts | 2 +- .../ui/src/lib/constants/sandbox.constants.ts | 2 +- .../ui/src/lib/constants/tool-ui.constants.ts | 60 +++++++++ tools/ui/src/lib/constants/ui.constants.ts | 8 +- tools/ui/src/lib/enums/tools.enums.ts | 42 +++--- .../src/lib/hooks/use-tools-panel.svelte.ts | 8 +- tools/ui/src/lib/services/index.ts | 6 +- .../ui/src/lib/services/read-media.service.ts | 8 +- tools/ui/src/lib/services/sandbox.service.ts | 4 +- tools/ui/src/lib/services/tools.service.ts | 14 +- tools/ui/src/lib/stores/agentic.svelte.ts | 24 ++-- tools/ui/src/lib/stores/tools.svelte.ts | 74 +++++------ tools/ui/src/lib/types/index.ts | 4 +- tools/ui/src/lib/types/mcp.d.ts | 4 +- tools/ui/src/lib/types/tools.d.ts | 8 +- tools/ui/src/lib/utils/built-in-tools.ts | 13 -- tools/ui/src/lib/utils/get-datetime.ts | 2 +- tools/ui/src/lib/utils/glob-search.ts | 2 +- tools/ui/src/lib/utils/index.ts | 8 +- tools/ui/src/lib/utils/tool-ui.ts | 13 ++ ...at-form-mention-picker-gate.svelte.test.ts | 16 +-- tools/ui/tests/unit/tool-calls.test.ts | 121 ++++++++++-------- 49 files changed, 315 insertions(+), 286 deletions(-) delete mode 100644 tools/ui/src/lib/constants/built-in-tools.constants.ts create mode 100644 tools/ui/src/lib/constants/tool-ui.constants.ts delete mode 100644 tools/ui/src/lib/utils/built-in-tools.ts create mode 100644 tools/ui/src/lib/utils/tool-ui.ts diff --git a/tools/server/README-dev.md b/tools/server/README-dev.md index 613017acf..94fbbde80 100644 --- a/tools/server/README-dev.md +++ b/tools/server/README-dev.md @@ -189,7 +189,7 @@ This endpoint is intended to be used internally by the Web UI and subject to cha Get a list of tools, each tool has these fields: - `tool` (string): the ID name of the tool, to be used in POST call. Example: `read_file` - `display_name` (string): the name to be displayed on UI. Example: `Read file` -- `type` (string): `"builtin"` for a built-in tool, or `"mcp"` for a tool exposed by an MCP server +- `type` (string): `"server"` for a server tool, or `"mcp"` for a tool exposed by an MCP server - `permissions` (object): a mapping string --> boolean that indicates the permission required by this tool. This is useful for the UI to ask the user before calling the tool. For now, the only permission supported is `"write"` - `definition` (object): the OAI-compat definition of this tool diff --git a/tools/server/README.md b/tools/server/README.md index f1e1facee..67e52b1db 100644 --- a/tools/server/README.md +++ b/tools/server/README.md @@ -196,11 +196,11 @@ For the full list of features, please refer to [server's changelog](https://gith | `--ui-config, --webui-config JSON` | JSON that provides default UI settings (overrides UI defaults)
(env: LLAMA_ARG_UI_CONFIG) | | `--ui-config-file, --webui-config-file PATH` | JSON file that provides default UI settings (overrides UI defaults)
(env: LLAMA_ARG_UI_CONFIG_FILE) | | `--ui-mcp-proxy, --webui-mcp-proxy, --no-ui-mcp-proxy, --no-webui-mcp-proxy` | experimental: whether to enable MCP CORS proxy - do not enable in untrusted environments (default: disabled)
(env: LLAMA_ARG_UI_MCP_PROXY) | -| `--tools TOOL1,TOOL2,...` | experimental: whether to enable built-in tools for AI agents - do not enable in untrusted environments (default: no tools)
specify "all" to enable all tools
available tools: read_file, file_glob_search, grep_search, exec_shell_command, write_file, edit_file, get_info
note: for security reasons, this will limit --cors-origins to localhost by default
(env: LLAMA_ARG_TOOLS) | +| `--tools TOOL1,TOOL2,...` | experimental: whether to enable server tools for AI agents - do not enable in untrusted environments (default: no tools)
specify "all" to enable all tools
available tools: read_file, file_glob_search, grep_search, exec_shell_command, write_file, edit_file, get_info
note: for security reasons, this will limit --cors-origins to localhost by default
(env: LLAMA_ARG_TOOLS) | | `--tools-runtime OPTION` | experimental: run tools in a separate runtime environment (default: none, use host environment)
available options:
'docker:', 'podman:': spin up a new container and reuse it for all invocations, clean up on server exit
'docker-container:', 'podman-container:': use an existing container by ID, won't stop on server exit
'ssh:': run tools on a remote POSIX host over SSH, key-based auth and a trusted host key are required

(env: LLAMA_ARG_TOOLS_RUNTIME) | | `--mcp-servers-config PATH` | experimental: path to JSON file with MCP server definitions (Cursor-compatible format) - do not enable in untrusted environments (default: none)
note: for security reasons, this will limit --cors-origins to localhost by default
(env: LLAMA_ARG_MCP_SERVERS_CONFIG) | | `--mcp-servers-json JSON` | experimental: inline JSON with MCP server definitions (Cursor-compatible format) - do not enable in untrusted environments (default: none)
note: for security reasons, this will limit --cors-origins to localhost by default
(env: LLAMA_ARG_MCP_SERVERS_JSON) | -| `-ag, --agent, -no-ag, --no-agent` | whether to enable CORS proxy and all built-in tools - do not enable in untrusted environments (default: disabled)
note: for security reasons, this will limit --cors-origins to localhost by default
(env: LLAMA_ARG_AGENT) | +| `-ag, --agent, -no-ag, --no-agent` | whether to enable CORS proxy and all server tools - do not enable in untrusted environments (default: disabled)
note: for security reasons, this will limit --cors-origins to localhost by default
(env: LLAMA_ARG_AGENT) | | `--ui, --webui, --no-ui, --no-webui` | whether to enable the Web UI (default: enabled)
(env: LLAMA_ARG_UI) | | `--embedding, --embeddings` | restrict to only support embedding use case; use only with dedicated embedding models (default: disabled)
(env: LLAMA_ARG_EMBEDDINGS) | | `--rerank, --reranking` | enable reranking endpoint on server (default: disabled)
(env: LLAMA_ARG_RERANKING) | @@ -337,9 +337,9 @@ It is currently available in the following endpoints: For more details, please refer to [multimodal documentation](../../docs/multimodal.md) -### Built-in tools support +### Server tools support -The server includes a set of built-in tools that enable the LLM to access the local file system directly from the Web UI. +The server includes a set of server tools that enable the LLM to access the local file system directly from the Web UI. To use this feature, start the server with `--tools all`. You can also enable only specific tools by passing a comma-separated list: `--tools name1,name2,...`. Run `--help` for the full list of available tool names. @@ -1631,9 +1631,9 @@ curl http://localhost:8080/v1/messages/count_tokens \ {"input_tokens": 10} ``` -## Server built-in tools +## Server tools -The server exposes a REST API under `/tools` that allows the Web UI to call built-in tools. This endpoint is intended to be used internally by the Web UI and subject to change or to be removed in the future. +The server exposes a REST API under `/tools` that allows the Web UI to call server tools. This endpoint is intended to be used internally by the Web UI and subject to change or to be removed in the future. **Please do NOT use this endpoint in a downstream application** diff --git a/tools/server/server-tools.cpp b/tools/server/server-tools.cpp index 5e5e60efd..b5c5c078a 100644 --- a/tools/server/server-tools.cpp +++ b/tools/server/server-tools.cpp @@ -2035,7 +2035,7 @@ void server_tools::setup(const std::vector & enabled_tools, } } - // append MCP tools, skipping any that collide with a built-in or another MCP tool of the same "_" name + // append MCP tools, skipping any that collide with a server tool or another MCP tool of the same "_" name if (!mcp_mgr.empty()) { std::unordered_set seen_names; for (auto & t : tools) { diff --git a/tools/server/server-tools.h b/tools/server/server-tools.h index c4509ca80..e7332f2e5 100644 --- a/tools/server/server-tools.h +++ b/tools/server/server-tools.h @@ -18,7 +18,7 @@ struct server_tool { virtual ~server_tool() = default; virtual json get_definition() const = 0; - virtual std::string type() const { return "builtin"; } + virtual std::string type() const { return "server"; } struct stream { server_response & qr; diff --git a/tools/server/server.cpp b/tools/server/server.cpp index 6d1aa4351..77722b9a6 100644 --- a/tools/server/server.cpp +++ b/tools/server/server.cpp @@ -346,7 +346,7 @@ int llama_server(common_params & params, int argc, char ** argv) { ctx_http.get ("/tools", ex_wrapper(tools.handle_get)); ctx_http.post("/tools", ex_wrapper(tools.handle_post)); if (!params.server_tools.empty()) { - warn_names.push_back("built-in tools (experimental)"); + warn_names.push_back("server tools (experimental)"); } if (!params.server_tools_runtime.empty()) { warn_names.push_back("tools runtime (experimental)"); diff --git a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionAddToolsSubmenu.svelte b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionAddToolsSubmenu.svelte index 1204390fd..58ae10673 100644 --- a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionAddToolsSubmenu.svelte +++ b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionAddToolsSubmenu.svelte @@ -35,7 +35,7 @@ Run llama-server with {CLI_FLAGS.TOOLS} flag to enable - Built-in Tools. + Server Tools. diff --git a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormCurrentWorkingDirectory/ChatFormCurrentWorkingDirectory.svelte b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormCurrentWorkingDirectory/ChatFormCurrentWorkingDirectory.svelte index 856b05cb0..99b7763e3 100644 --- a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormCurrentWorkingDirectory/ChatFormCurrentWorkingDirectory.svelte +++ b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormCurrentWorkingDirectory/ChatFormCurrentWorkingDirectory.svelte @@ -62,7 +62,7 @@ // it, the picker still opens for manual entry but explains why search is // unavailable instead of firing searches that would only fail. Browse is // hidden too: it resolves the picked folder name through the same tool. - const fileSearchKey = $derived(toolsStore.getPermissionKey(BuiltInTool.FILE_GLOB_SEARCH)); + const fileSearchKey = $derived(toolsStore.getPermissionKey(BuiltInTool.SERVER_FILE_GLOB_SEARCH)); const fileSearchEnabled = $derived( fileSearchKey !== null && toolsStore.isToolEnabled(fileSearchKey) ); @@ -212,7 +212,7 @@ // so the caller fails visibly instead of committing a bare leaf name. async function resolveNativeName(name: string): Promise { try { - const res = await ToolsService.executeToolRaw(BuiltInTool.FILE_GLOB_SEARCH, { + const res = await ToolsService.executeToolRaw(BuiltInTool.SERVER_FILE_GLOB_SEARCH, { include: buildCaseInsensitiveGlob(name), limit: SEARCH.NATIVE_LIMIT, max_depth: SEARCH.NATIVE_MAX_DEPTH, diff --git a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormPickers/ChatFormPickerMention.svelte b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormPickers/ChatFormPickerMention.svelte index 1c7c8f7d4..fbe6ce101 100644 --- a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormPickers/ChatFormPickerMention.svelte +++ b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormPickers/ChatFormPickerMention.svelte @@ -51,7 +51,7 @@ // When the server does not expose file_glob_search (started without // --tools) or the user disabled it, the picker still opens but explains // why instead of firing searches that would only fail. - const fileSearchKey = $derived(toolsStore.getPermissionKey(BuiltInTool.FILE_GLOB_SEARCH)); + const fileSearchKey = $derived(toolsStore.getPermissionKey(BuiltInTool.SERVER_FILE_GLOB_SEARCH)); const fileSearchEnabled = $derived( fileSearchKey !== null && toolsStore.isToolEnabled(fileSearchKey) ); diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlock.svelte b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlock.svelte index a6fa2e250..3f7593315 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlock.svelte +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlock.svelte @@ -35,19 +35,19 @@ {#if isSearchCall} -{:else if section.toolName === BuiltInTool.GET_DATETIME} +{:else if section.toolName === BuiltInTool.BROWSER_GET_DATETIME} -{:else if section.toolName === BuiltInTool.GET_INFO} +{:else if section.toolName === BuiltInTool.SERVER_GET_INFO} -{:else if section.toolName === BuiltInTool.READ_FILE} +{:else if section.toolName === BuiltInTool.SERVER_READ_FILE} -{:else if section.toolName === BuiltInTool.READ_MEDIA} +{:else if section.toolName === BuiltInTool.BROWSER_READ_MEDIA} -{:else if section.toolName === BuiltInTool.EDIT_FILE} +{:else if section.toolName === BuiltInTool.SERVER_EDIT_FILE} -{:else if section.toolName === BuiltInTool.WRITE_FILE} +{:else if section.toolName === BuiltInTool.SERVER_WRITE_FILE} -{:else if section.toolName === BuiltInTool.EXEC_SHELL_COMMAND} +{:else if section.toolName === BuiltInTool.SERVER_EXEC_SHELL_COMMAND} -{:else if section.toolName === BuiltInTool.FILE_GLOB_SEARCH} +{:else if section.toolName === BuiltInTool.SERVER_FILE_GLOB_SEARCH} -{:else if section.toolName === BuiltInTool.GREP_SEARCH} +{:else if section.toolName === BuiltInTool.SERVER_GREP_SEARCH} -{:else if section.toolName === BuiltInTool.RUN_JAVASCRIPT} +{:else if section.toolName === BuiltInTool.BROWSER_RUN_JAVASCRIPT} {:else} diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockDefault.svelte b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockDefault.svelte index c45ec5584..3ca64a782 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockDefault.svelte +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockDefault.svelte @@ -12,7 +12,7 @@ import { classifyToolResult, formatJsonPretty, - getBuiltinToolUi, + getToolUi, parseToolResultWithMedia } from '$lib/utils'; import { createBase64DataUrl } from '$lib/utils/data-url'; @@ -27,7 +27,7 @@ let { attachments, isStreaming, onToggle, open, section }: Props = $props(); - const title = $derived(getBuiltinToolUi(section.toolName)?.label ?? section.toolName ?? ''); + const title = $derived(getToolUi(section.toolName)?.label ?? section.toolName ?? ''); const outputKind = $derived(classifyToolResult(section.toolResult)); const parsedLines: ToolResultLine[] = $derived( section.toolResult ? parseToolResultWithMedia(section.toolResult, attachments) : [] diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockRunJavascript.svelte b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockRunJavascript.svelte index a566f9be4..5c60457db 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockRunJavascript.svelte +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockRunJavascript.svelte @@ -6,7 +6,7 @@ import { MAX_HEIGHT_CODE_BLOCK } from '$lib/constants'; import { FileTypeText } from '$lib/enums'; import type { AgenticSection } from '$lib/types'; - import { getBuiltinToolUi } from '$lib/utils'; + import { getToolUi } from '$lib/utils'; interface Props { section: AgenticSection; @@ -18,7 +18,7 @@ let { isStreaming, onToggle, open, section }: Props = $props(); const runJsMeta = $derived(parseRunJavascriptMeta(section)); - const title = $derived(getBuiltinToolUi(section.toolName)?.label ?? section.toolName ?? ''); + const title = $derived(getToolUi(section.toolName)?.label ?? section.toolName ?? ''); diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ToolCallBlock.svelte b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ToolCallBlock.svelte index 4b8524fe6..08da39a2a 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ToolCallBlock.svelte +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ToolCallBlock.svelte @@ -14,8 +14,8 @@ import { ICON_CLASS_DEFAULT, ICON_CLASS_SPIN } from '$lib/constants'; import { AgenticSectionType } from '$lib/enums'; import { mcpStore } from '$lib/stores'; - import type { AgenticSection, BuiltinToolUiEntry } from '$lib/types'; - import { getBuiltinToolUi } from '$lib/utils'; + import type { AgenticSection, ToolUiEntry } from '$lib/types'; + import { getToolUi } from '$lib/utils'; import type { Component, Snippet } from 'svelte'; type ToolCallBlockMetaWithError = TMeta & { errorMessage?: string }; @@ -82,7 +82,7 @@ const showSpinner = $derived(isPending || (isStreamingCall && isStreaming) || extraLiveStreaming); const isCodeStreaming = $derived(isStreaming && (isPending || isStreamingCall)); - const toolUi: BuiltinToolUiEntry | null = $derived(getBuiltinToolUi(section.toolName)); + const toolUi: ToolUiEntry | null = $derived(getToolUi(section.toolName)); const toolIcon: Component = $derived( spinIconWhenActive && showSpinner ? Loader2 : (toolUi?.icon ?? Wrench) ); diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/edit-file.ts b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/edit-file.ts index dadb33a49..9ed6f92bc 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/edit-file.ts +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/edit-file.ts @@ -24,7 +24,7 @@ export type EditFileMeta = { }; export function parseEditFileMeta(section: AgenticSection): EditFileMeta | null { - const args = parseToolArgs(BuiltInTool.EDIT_FILE, section, { partial: true }); + const args = parseToolArgs(BuiltInTool.SERVER_EDIT_FILE, section, { partial: true }); if (!args) return null; diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/exec-shell-command.ts b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/exec-shell-command.ts index 496fcdde6..7cf767535 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/exec-shell-command.ts +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/exec-shell-command.ts @@ -14,7 +14,7 @@ export type ExecShellCommandMeta = { }; export function parseExecShellCommandMeta(section: AgenticSection): ExecShellCommandMeta | null { - const args = parseToolArgs(BuiltInTool.EXEC_SHELL_COMMAND, section); + const args = parseToolArgs(BuiltInTool.SERVER_EXEC_SHELL_COMMAND, section); if (!args) return null; diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/file-glob-search.ts b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/file-glob-search.ts index 0acd53d77..237afa599 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/file-glob-search.ts +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/file-glob-search.ts @@ -19,7 +19,7 @@ export type FileGlobSearchMeta = { }; export function parseFileGlobSearchMeta(section: AgenticSection): FileGlobSearchMeta | null { - const args = parseToolArgs(BuiltInTool.FILE_GLOB_SEARCH, section); + const args = parseToolArgs(BuiltInTool.SERVER_FILE_GLOB_SEARCH, section); if (!args) return null; diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/grep-search.ts b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/grep-search.ts index c25d76b1a..90889ff27 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/grep-search.ts +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/grep-search.ts @@ -28,7 +28,7 @@ export type GrepSearchMeta = { }; export function parseGrepSearchMeta(section: AgenticSection): GrepSearchMeta | null { - const args = parseToolArgs(BuiltInTool.GREP_SEARCH, section); + const args = parseToolArgs(BuiltInTool.SERVER_GREP_SEARCH, section); if (!args) return null; diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/read-file.ts b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/read-file.ts index 9ee748ed7..af0f3d925 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/read-file.ts +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/read-file.ts @@ -16,7 +16,7 @@ export type ReadFileMeta = { }; export function parseReadFileMeta(section: AgenticSection): ReadFileMeta | null { - const args = parseToolArgs(BuiltInTool.READ_FILE, section, { partial: true }); + const args = parseToolArgs(BuiltInTool.SERVER_READ_FILE, section, { partial: true }); if (!args) return null; diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/run-javascript.ts b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/run-javascript.ts index a524478ab..440a1f5d6 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/run-javascript.ts +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/run-javascript.ts @@ -16,7 +16,7 @@ export type RunJavascriptMeta = { }; export function parseRunJavascriptMeta(section: AgenticSection): RunJavascriptMeta | null { - const args = parseToolArgs(BuiltInTool.RUN_JAVASCRIPT, section); + const args = parseToolArgs(BuiltInTool.BROWSER_RUN_JAVASCRIPT, section); if (!args) return null; diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/write-file.ts b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/write-file.ts index 53ba38e12..5b9bf9f88 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/write-file.ts +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/write-file.ts @@ -20,7 +20,7 @@ export type WriteFileMeta = { }; export function parseWriteFileMeta(section: AgenticSection): WriteFileMeta | null { - const args = parseToolArgs(BuiltInTool.WRITE_FILE, section, { partial: true }); + const args = parseToolArgs(BuiltInTool.SERVER_WRITE_FILE, section, { partial: true }); if (!args) return null; diff --git a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessageActions/ChatMessageActionCard/ChatMessageActionCardPermissionRequest.svelte b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessageActions/ChatMessageActionCard/ChatMessageActionCardPermissionRequest.svelte index a0cee94f4..e8af94464 100644 --- a/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessageActions/ChatMessageActionCard/ChatMessageActionCardPermissionRequest.svelte +++ b/tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessageActions/ChatMessageActionCard/ChatMessageActionCardPermissionRequest.svelte @@ -61,8 +61,8 @@ {:else} {@const source = toolsStore.getToolSource(toolName)} {@const providerName = - source === ToolSource.BUILTIN - ? TOOL_SERVER_LABELS[ToolSource.BUILTIN] + source === ToolSource.SERVER + ? TOOL_SERVER_LABELS[ToolSource.SERVER] : source === ToolSource.CUSTOM ? TOOL_SERVER_LABELS[ToolSource.CUSTOM] : 'MCP Tools'} diff --git a/tools/ui/src/lib/components/app/chat/index.ts b/tools/ui/src/lib/components/app/chat/index.ts index 2de8a6ace..34571d53b 100644 --- a/tools/ui/src/lib/components/app/chat/index.ts +++ b/tools/ui/src/lib/components/app/chat/index.ts @@ -278,7 +278,7 @@ export { default as ChatFormInput } from './ChatForm/ChatFormInput/ChatFormInput /** * Working directory selector for agent mode. Renders a chip below the chat * form; clicking it opens a popover with a directory picker backed by the - * server's `file_glob_search` built-in tool (POST /tools). The picked + * server's `file_glob_search` server tool (POST /tools). The picked * directory is exposed via `bind:directory`; changing it records a * synthetic "Set working directory to ..." user message into chat history * and is enforced on tool calls via the `x-tool-cwd` request header. @@ -380,7 +380,7 @@ export { default as ChatFormPickerListItemSkeleton } from './ChatForm/ChatFormPi /** * `@`-triggered file/folder mention picker. Resolves `@` in the chat - * input to a filesystem match via the server's `file_glob_search` built-in + * input to a filesystem match via the server's `file_glob_search` server tool * tool, scoped to the conversation cwd (or server home when unset). * Selection splices a `[name](file:///)` link into the input. */ diff --git a/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChatToolsTab.svelte b/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChatToolsTab.svelte index 53cd76912..08046b084 100644 --- a/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChatToolsTab.svelte +++ b/tools/ui/src/lib/components/app/settings/SettingsChat/SettingsChatToolsTab.svelte @@ -6,7 +6,7 @@ import { ICON_CLASS_DEFAULT } from '$lib/constants'; import { ToolSource } from '$lib/enums/tools.enums'; import { mcpStore, permissionsStore, toolsStore } from '$lib/stores'; - import { getBuiltinToolUi } from '$lib/utils'; + import { getToolUi } from '$lib/utils'; import { SvelteSet } from 'svelte/reactivity'; let expandedGroups = new SvelteSet(); @@ -69,12 +69,12 @@ {#each group.tools as entry (entry.key)} {@const toolName = entry.definition.function.name} - {@const builtinUi = - entry.source === ToolSource.BUILTIN || entry.source === ToolSource.FRONTEND - ? getBuiltinToolUi(toolName) + {@const toolUi = + entry.source === ToolSource.SERVER || entry.source === ToolSource.BROWSER + ? getToolUi(toolName) : null} - {@const displayLabel = builtinUi?.label ?? toolName} - {@const IconComponent = builtinUi?.icon ?? null} + {@const displayLabel = toolUi?.label ?? toolName} + {@const IconComponent = toolUi?.icon ?? null} {@const isEnabled = toolsStore.isToolEnabled(entry.key)} {@const permissionKey = entry.key} {@const isAlwaysAllowed = permissionsStore.hasTool(permissionKey)} diff --git a/tools/ui/src/lib/components/app/settings/index.ts b/tools/ui/src/lib/components/app/settings/index.ts index 63f9651df..9318fd4d3 100644 --- a/tools/ui/src/lib/components/app/settings/index.ts +++ b/tools/ui/src/lib/components/app/settings/index.ts @@ -69,7 +69,7 @@ export { default as SettingsChatFields } from './SettingsChat/SettingsChatFields /** * **SettingsChatToolsTab** - Tools configuration tab for chat settings * - * Displays available tools grouped by source (built-in, MCP, custom) with + * Displays available tools grouped by source (server, browser, MCP, custom) with * toggles to enable/disable individual tools and tool groups. Shows MCP * server favicons and permission management controls. */ diff --git a/tools/ui/src/lib/constants/browser-info.ts b/tools/ui/src/lib/constants/browser-info.ts index 86d999643..e99c324aa 100644 --- a/tools/ui/src/lib/constants/browser-info.ts +++ b/tools/ui/src/lib/constants/browser-info.ts @@ -2,7 +2,9 @@ import { CLI_FLAGS } from './cli-flags.constants'; import { BuiltInTool, JsonSchemaType, ToolCallType } from '$lib/enums'; import type { OpenAIToolDefinition } from '$lib/types'; -export const BROWSER_INFO_TOOL_NAME = BuiltInTool.GET_INFO; +// get_info is served by the server, but the browser falls back to this +// implementation when the server does not provide it - same wire name. +export const BROWSER_INFO_TOOL_NAME = BuiltInTool.SERVER_GET_INFO; /** UA token to OS name, first match wins - Android and iOS UAs also carry the Linux / Mac OS X tokens */ export const BROWSER_INFO_OS_UA_PATTERNS: readonly [RegExp, string][] = [ diff --git a/tools/ui/src/lib/constants/built-in-tools.constants.ts b/tools/ui/src/lib/constants/built-in-tools.constants.ts deleted file mode 100644 index 61e90e105..000000000 --- a/tools/ui/src/lib/constants/built-in-tools.constants.ts +++ /dev/null @@ -1,52 +0,0 @@ -// Registry of built-in and frontend (browser) tools whose renderer -// shows a recognizable icon and friendly label inline in the chat UI. -// -// To add a new built-in tool, add an entry to BUILTIN_TOOL_UI. To give a -// tool a custom title or body renderer, add a dedicated component under -// ChatMessageToolCall/ and route it in ChatMessageToolCallBlock.svelte -// (see ChatMessageToolCallBlockGetDatetime and -// ChatMessageToolCallBlockSearchResults for prior art). - -import { - Braces, - Clock, - Eye, - FilePen, - FilePlus, - FileSearch, - FileText, - Info, - SearchCode, - Terminal -} from '@lucide/svelte'; -import { BuiltInTool, ToolSource } from '$lib/enums'; -import type { BuiltinToolUiEntry } from '$lib/types'; - -export const BUILTIN_TOOL_UI: Readonly> = { - [BuiltInTool.EDIT_FILE]: { icon: FilePen, label: 'Edit file', source: ToolSource.BUILTIN }, - [BuiltInTool.EXEC_SHELL_COMMAND]: { - icon: Terminal, - label: 'Run command', - source: ToolSource.BUILTIN - }, - [BuiltInTool.FILE_GLOB_SEARCH]: { - icon: FileSearch, - label: 'Search files', - source: ToolSource.BUILTIN - }, - [BuiltInTool.GET_DATETIME]: { icon: Clock, label: 'Current time', source: ToolSource.FRONTEND }, - [BuiltInTool.GET_INFO]: { icon: Info, label: 'Runtime info', source: ToolSource.BUILTIN }, - [BuiltInTool.GREP_SEARCH]: { - icon: SearchCode, - label: 'Search in files', - source: ToolSource.BUILTIN - }, - [BuiltInTool.READ_FILE]: { icon: FileText, label: 'Read file', source: ToolSource.BUILTIN }, - [BuiltInTool.READ_MEDIA]: { icon: Eye, label: 'Read media', source: ToolSource.FRONTEND }, - [BuiltInTool.RUN_JAVASCRIPT]: { - icon: Braces, - label: 'Run JavaScript', - source: ToolSource.FRONTEND - }, - [BuiltInTool.WRITE_FILE]: { icon: FilePlus, label: 'Write file', source: ToolSource.BUILTIN } -} as const; diff --git a/tools/ui/src/lib/constants/get-datetime.ts b/tools/ui/src/lib/constants/get-datetime.ts index c8726d9b5..19418dcef 100644 --- a/tools/ui/src/lib/constants/get-datetime.ts +++ b/tools/ui/src/lib/constants/get-datetime.ts @@ -1,7 +1,7 @@ import { BuiltInTool, JsonSchemaType, ToolCallType } from '$lib/enums'; import type { OpenAIToolDefinition } from '$lib/types'; -export const GET_DATETIME_TOOL_NAME = BuiltInTool.GET_DATETIME; +export const GET_DATETIME_TOOL_NAME = BuiltInTool.BROWSER_GET_DATETIME; export function buildGetDatetimeToolDefinition(): OpenAIToolDefinition { return { diff --git a/tools/ui/src/lib/constants/index.ts b/tools/ui/src/lib/constants/index.ts index 289239113..8ab921d75 100644 --- a/tools/ui/src/lib/constants/index.ts +++ b/tools/ui/src/lib/constants/index.ts @@ -15,7 +15,7 @@ export * from './context-gauge-popup.constants'; export * from './conversation-import.constants'; export * from './binary-detection.constants'; export * from './content-detection.constants'; -export * from './built-in-tools.constants'; +export * from './tool-ui.constants'; export * from './cache.constants'; export * from './chat-form.constants'; export * from './cli-flags.constants'; diff --git a/tools/ui/src/lib/constants/read-media.ts b/tools/ui/src/lib/constants/read-media.ts index 525c5e902..f9ac2282c 100644 --- a/tools/ui/src/lib/constants/read-media.ts +++ b/tools/ui/src/lib/constants/read-media.ts @@ -7,7 +7,7 @@ import { } from '$lib/enums'; import type { OpenAIToolDefinition } from '$lib/types'; -export const READ_MEDIA_TOOL_NAME = BuiltInTool.READ_MEDIA; +export const READ_MEDIA_TOOL_NAME = BuiltInTool.BROWSER_READ_MEDIA; // header lines of the tool result, parsed back by the read_media renderer export const PREFIX_FILE = 'File: '; diff --git a/tools/ui/src/lib/constants/sandbox.constants.ts b/tools/ui/src/lib/constants/sandbox.constants.ts index 9846e471a..68462a23d 100644 --- a/tools/ui/src/lib/constants/sandbox.constants.ts +++ b/tools/ui/src/lib/constants/sandbox.constants.ts @@ -1,6 +1,6 @@ import { BuiltInTool } from '$lib/enums'; -export const SANDBOX_TOOL_NAME = BuiltInTool.RUN_JAVASCRIPT; +export const SANDBOX_TOOL_NAME = BuiltInTool.BROWSER_RUN_JAVASCRIPT; export const SANDBOX_TIMEOUT_MS_DEFAULT = 10000; diff --git a/tools/ui/src/lib/constants/tool-ui.constants.ts b/tools/ui/src/lib/constants/tool-ui.constants.ts new file mode 100644 index 000000000..b5c09a653 --- /dev/null +++ b/tools/ui/src/lib/constants/tool-ui.constants.ts @@ -0,0 +1,60 @@ +// Registry of server and browser tools whose renderer +// shows a recognizable icon and friendly label inline in the chat UI. +// +// To add a new tool, add an entry to TOOL_UI. To give a +// tool a custom title or body renderer, add a dedicated component under +// ChatMessageToolCall/ and route it in ChatMessageToolCallBlock.svelte +// (see ChatMessageToolCallBlockGetDatetime and +// ChatMessageToolCallBlockSearchResults for prior art). + +import { + Braces, + Clock, + Eye, + FilePen, + FilePlus, + FileSearch, + FileText, + Info, + SearchCode, + Terminal +} from '@lucide/svelte'; +import { BuiltInTool, ToolSource } from '$lib/enums'; +import type { ToolUiEntry } from '$lib/types'; + +export const TOOL_UI: Readonly> = { + [BuiltInTool.BROWSER_GET_DATETIME]: { + icon: Clock, + label: 'Current time', + source: ToolSource.BROWSER + }, + [BuiltInTool.BROWSER_READ_MEDIA]: { icon: Eye, label: 'Read media', source: ToolSource.BROWSER }, + [BuiltInTool.BROWSER_RUN_JAVASCRIPT]: { + icon: Braces, + label: 'Run JavaScript', + source: ToolSource.BROWSER + }, + [BuiltInTool.SERVER_EDIT_FILE]: { icon: FilePen, label: 'Edit file', source: ToolSource.SERVER }, + [BuiltInTool.SERVER_EXEC_SHELL_COMMAND]: { + icon: Terminal, + label: 'Run command', + source: ToolSource.SERVER + }, + [BuiltInTool.SERVER_FILE_GLOB_SEARCH]: { + icon: FileSearch, + label: 'Search files', + source: ToolSource.SERVER + }, + [BuiltInTool.SERVER_GET_INFO]: { icon: Info, label: 'Runtime info', source: ToolSource.SERVER }, + [BuiltInTool.SERVER_GREP_SEARCH]: { + icon: SearchCode, + label: 'Search in files', + source: ToolSource.SERVER + }, + [BuiltInTool.SERVER_READ_FILE]: { icon: FileText, label: 'Read file', source: ToolSource.SERVER }, + [BuiltInTool.SERVER_WRITE_FILE]: { + icon: FilePlus, + label: 'Write file', + source: ToolSource.SERVER + } +} as const; diff --git a/tools/ui/src/lib/constants/ui.constants.ts b/tools/ui/src/lib/constants/ui.constants.ts index e4889649f..feae08742 100644 --- a/tools/ui/src/lib/constants/ui.constants.ts +++ b/tools/ui/src/lib/constants/ui.constants.ts @@ -18,15 +18,15 @@ export const UI_DATA_ATTRS = { } as const; export const TOOL_GROUP_LABELS = { - [ToolSource.BUILTIN]: 'Built-in', + [ToolSource.BROWSER]: 'Browser', [ToolSource.CUSTOM]: 'JSON Schema', - [ToolSource.FRONTEND]: 'Browser' + [ToolSource.SERVER]: 'Server' } as const; export const TOOL_SERVER_LABELS = { - [ToolSource.BUILTIN]: 'Built-in Tools', + [ToolSource.BROWSER]: 'Browser Tools', [ToolSource.CUSTOM]: 'Custom Tools', - [ToolSource.FRONTEND]: 'Browser Tools' + [ToolSource.SERVER]: 'Server Tools' } as const; export const TOOLTIP_DELAY_DURATION = 500; diff --git a/tools/ui/src/lib/enums/tools.enums.ts b/tools/ui/src/lib/enums/tools.enums.ts index 31c992fef..0c47be2cb 100644 --- a/tools/ui/src/lib/enums/tools.enums.ts +++ b/tools/ui/src/lib/enums/tools.enums.ts @@ -1,8 +1,8 @@ export enum ToolSource { - BUILTIN = 'builtin', - MCP = 'mcp', + BROWSER = 'browser', CUSTOM = 'custom', - FRONTEND = 'frontend' + MCP = 'mcp', + SERVER = 'server' } export enum ToolPermissionDecision { @@ -28,22 +28,28 @@ export enum GlobSearchType { } /** - * Wire-format identifiers for built-in and frontend tools. The string + * Wire-format identifiers for server and browser tools. The string * value matches what the model emits in tool call names, so comparing - * against `BuiltInTool.READ_FILE` is equivalent to comparing against the - * raw `'read_file'` literal - the enum just keeps the two in lock-step - * and gives TypeScript a single source of truth for autocomplete / rename - * support. + * against `BuiltInTool.SERVER_READ_FILE` is equivalent to comparing + * against the raw `'read_file'` literal - the enum just keeps the two in + * lock-step and gives TypeScript a single source of truth for autocomplete + * / rename support. + * + * The `SERVER_` / `BROWSER_` prefixes mirror the tool's primary source + * (llama-server vs llama-ui). `get_info` is the exception: it is served by + * the server, but llama-ui falls back to a browser implementation when the + * server does not provide it, so it can surface under both categories in + * the UI while keeping a single wire name. */ export enum BuiltInTool { - READ_FILE = 'read_file', - READ_MEDIA = 'read_media', - EDIT_FILE = 'edit_file', - WRITE_FILE = 'write_file', - GET_DATETIME = 'get_datetime', - GET_INFO = 'get_info', - FILE_GLOB_SEARCH = 'file_glob_search', - GREP_SEARCH = 'grep_search', - EXEC_SHELL_COMMAND = 'exec_shell_command', - RUN_JAVASCRIPT = 'run_javascript' + BROWSER_GET_DATETIME = 'get_datetime', + BROWSER_READ_MEDIA = 'read_media', + BROWSER_RUN_JAVASCRIPT = 'run_javascript', + SERVER_EDIT_FILE = 'edit_file', + SERVER_EXEC_SHELL_COMMAND = 'exec_shell_command', + SERVER_FILE_GLOB_SEARCH = 'file_glob_search', + SERVER_GET_INFO = 'get_info', + SERVER_GREP_SEARCH = 'grep_search', + SERVER_READ_FILE = 'read_file', + SERVER_WRITE_FILE = 'write_file' } diff --git a/tools/ui/src/lib/hooks/use-tools-panel.svelte.ts b/tools/ui/src/lib/hooks/use-tools-panel.svelte.ts index e60c95429..80b3b85a9 100644 --- a/tools/ui/src/lib/hooks/use-tools-panel.svelte.ts +++ b/tools/ui/src/lib/hooks/use-tools-panel.svelte.ts @@ -46,13 +46,13 @@ export function useToolsPanel(): UseToolsPanelReturn { // Tools endpoint is unreachable (404) — server started without --tools if (toolsStore.isToolsEndpointUnreachable) { - return `To enable Built-In Tools you need to run llama-server with ${CLI_FLAGS.TOOLS} all or ${CLI_FLAGS.TOOLS} flag. To see MCP Tools you need to add / enable MCP Server(s).`; + return `To enable Server Tools you need to run llama-server with ${CLI_FLAGS.TOOLS} all or ${CLI_FLAGS.TOOLS} flag. To see MCP Tools you need to add / enable MCP Server(s).`; } // Other errors — return null so UI shows "Failed to load tools" if (toolsStore.error) return null; - return `To enable Built-In Tools you need to run llama-server with ${CLI_FLAGS.TOOLS} all or ${CLI_FLAGS.TOOLS} flag. To see MCP Tools you need to add / enable MCP Server(s).`; + return `To enable Server Tools you need to run llama-server with ${CLI_FLAGS.TOOLS} all or ${CLI_FLAGS.TOOLS} flag. To see MCP Tools you need to add / enable MCP Server(s).`; }); function isGroupChecked(group: ToolGroup): boolean { @@ -95,8 +95,8 @@ export function useToolsPanel(): UseToolsPanelReturn { } function handleOpen(): void { - if (toolsStore.builtinTools.length === 0 && !toolsStore.loading) { - toolsStore.fetchBuiltinTools(); + if (toolsStore.serverTools.length === 0 && !toolsStore.loading) { + toolsStore.fetchServerTools(); } mcpStore.runHealthChecksForServers(mcpStore.getServers().filter((s) => s.enabled)); diff --git a/tools/ui/src/lib/services/index.ts b/tools/ui/src/lib/services/index.ts index 328d3b482..220edc51e 100644 --- a/tools/ui/src/lib/services/index.ts +++ b/tools/ui/src/lib/services/index.ts @@ -262,9 +262,9 @@ export { ParameterSyncService } from './parameter-sync.service'; export { MCPService } from './mcp.service'; /** - * **SandboxService** - Frontend JavaScript execution in a browser sandbox + * **SandboxService** - Browser JavaScript execution in a browser sandbox * - * Stateless executor for the run_javascript frontend tool. Model generated + * Stateless executor for the run_javascript browser tool. Model generated * code runs in a Web Worker spawned inside a sandboxed iframe with an opaque * origin: no access to the app origin, its storage or its API, and outgoing * requests carry a null origin. The code never touches a main thread, so the @@ -274,7 +274,7 @@ export { MCPService } from './mcp.service'; * **Architecture & Relationships:** * - **SandboxService** (this class): Stateless sandbox execution * - **toolsStore**: Exposes the tool definition when the sandbox is enabled - * - **agenticStore**: Dispatches ToolSource.FRONTEND calls here + * - **agenticStore**: Dispatches ToolSource.BROWSER calls here * * @see buildSandboxToolDefinition in utils/sandbox-tool - tool schema sent to the LLM * @see agenticStore in stores/agentic.svelte.ts - tool dispatch diff --git a/tools/ui/src/lib/services/read-media.service.ts b/tools/ui/src/lib/services/read-media.service.ts index fd66350d0..8de9bbbea 100644 --- a/tools/ui/src/lib/services/read-media.service.ts +++ b/tools/ui/src/lib/services/read-media.service.ts @@ -28,15 +28,15 @@ function fileExtension(path: string): string { } /** - * **ReadMediaService** - frontend executor for the `read_media` tool + * **ReadMediaService** - browser executor for the `read_media` tool * * The tool is synthetic: no such tool exists on the server. It reads the file - * through the built-in `read_file` tool with the `base64` response type, then + * through the server `read_file` tool with the `base64` response type, then * turns the bytes into a data URI line. The agentic store lifts that line into * an image or audio attachment on the tool result message, which is what makes * the model perceive the file instead of reading a wall of base64. * - * Living in the frontend is what lets it exist only for models that can + * Living in the browser is what lets it exist only for models that can * actually use the result - the server has no idea which model is selected. * * @see buildReadMediaToolDefinition in constants/read-media.ts - tool schema sent to the LLM @@ -82,7 +82,7 @@ export class ReadMediaService { } const raw = await ToolsService.executeToolRaw( - BuiltInTool.READ_FILE, + BuiltInTool.SERVER_READ_FILE, { path }, signal, cwd, diff --git a/tools/ui/src/lib/services/sandbox.service.ts b/tools/ui/src/lib/services/sandbox.service.ts index c0b7e9c60..27da9d263 100644 --- a/tools/ui/src/lib/services/sandbox.service.ts +++ b/tools/ui/src/lib/services/sandbox.service.ts @@ -68,7 +68,7 @@ function formatReply(reply: SandboxReply): ToolExecutionResult { export class SandboxService { /** - * Execute a frontend sandbox tool call and return its output. + * Execute a browser sandbox tool call and return its output. * One disposable iframe per execution, removed on completion, * timeout or abort. Removing the iframe terminates the worker * at the browser level, so runaway code cannot outlive it. @@ -79,7 +79,7 @@ export class SandboxService { signal?: AbortSignal ): Promise { if (toolName !== SANDBOX_TOOL_NAME) { - return { content: `Unknown frontend tool: ${toolName}`, isError: true }; + return { content: `Unknown browser tool: ${toolName}`, isError: true }; } const code = typeof params.code === 'string' ? params.code : ''; diff --git a/tools/ui/src/lib/services/tools.service.ts b/tools/ui/src/lib/services/tools.service.ts index f100b74f9..2b3a2c0dc 100644 --- a/tools/ui/src/lib/services/tools.service.ts +++ b/tools/ui/src/lib/services/tools.service.ts @@ -1,23 +1,23 @@ import { base } from '$app/paths'; import { API_TOOLS, HEADERS } from '$lib/constants'; import { ToolResponseField } from '$lib/enums'; -import type { ServerBuiltinToolInfo, ToolExecutionResult } from '$lib/types'; +import type { ServerToolInfo, ToolExecutionResult } from '$lib/types'; import { apiFetch } from '$lib/utils'; import { getJsonHeaders } from '$lib/utils/api-headers'; import { parseSseJsonStream, type SseJsonEvent } from '$lib/utils/sse'; export class ToolsService { /** - * Fetch the list of built-in tools from the server. + * Fetch the list of server tools from the server. * * @returns Array of tool definitions in OpenAI-compatible format */ - static async list(): Promise { - return apiFetch(API_TOOLS.LIST); + static async list(): Promise { + return apiFetch(API_TOOLS.LIST); } /** - * Execute a built-in tool on the server. + * Execute a server tool on the server. * * @param cwd - Working directory for the tool call, sent as the * x-tool-cwd request header. The server resolves relative paths @@ -48,7 +48,7 @@ export class ToolsService { } /** - * Execute a built-in tool and return the raw JSON response. Unlike + * Execute a server tool and return the raw JSON response. Unlike * executeTool, this preserves structured fields (e.g. file_glob_search's * `entries` and `base`) that the flattened ToolExecutionResult drops. * @@ -77,7 +77,7 @@ export class ToolsService { } /** - * Stream a built-in tool's output chunks from the server. The server + * Stream a server tool's output chunks from the server. The server * `POST /tools` endpoint with `{stream: true}` emits `data: {"chunk": "..."}` * events followed by a terminal `data: {"done": true}` (optionally with * `error`). Yields the chunk string for each partial event. diff --git a/tools/ui/src/lib/stores/agentic.svelte.ts b/tools/ui/src/lib/stores/agentic.svelte.ts index 06c7661fe..d2a2ea887 100644 --- a/tools/ui/src/lib/stores/agentic.svelte.ts +++ b/tools/ui/src/lib/stores/agentic.svelte.ts @@ -326,8 +326,8 @@ class AgenticStore { const maxTurns = Number(settings.agenticMaxTurns) || DEFAULT_AGENTIC_CONFIG.maxTurns; const hasTools = mcpStore.hasEnabledServers(perChatOverrides) || - toolsStore.builtinTools.length > 0 || - toolsStore.frontendTools.length > 0 || + toolsStore.serverTools.length > 0 || + toolsStore.browserTools.length > 0 || toolsStore.customTools.length > 0; return { @@ -455,9 +455,9 @@ class AgenticStore { this._continueResolvers.delete(conversationId); this._steeringMessages.delete(conversationId); - // Ensure built-in tools are fetched before checking if agentic is enabled - if (toolsStore.builtinTools.length === 0 && !toolsStore.loading) { - await toolsStore.fetchBuiltinTools(); + // Ensure server tools are fetched before checking if agentic is enabled + if (toolsStore.serverTools.length === 0 && !toolsStore.loading) { + await toolsStore.fetchServerTools(); } const agenticConfig = this.getConfig(settingsStore.config, perChatOverrides); @@ -906,8 +906,8 @@ class AgenticStore { } else { try { if ( - toolSource === ToolSource.BUILTIN && - toolName === BuiltInTool.EXEC_SHELL_COMMAND && + toolSource === ToolSource.SERVER && + toolName === BuiltInTool.SERVER_EXEC_SHELL_COMMAND && createToolResultMessage && updateToolResultMessage ) { @@ -938,7 +938,7 @@ class AgenticStore { } } result = accumulated; - } else if (toolSource === ToolSource.BUILTIN) { + } else if (toolSource === ToolSource.SERVER) { const args = this.parseToolArguments(toolCall.function.arguments); const cwd = conversationsStore.activeConversation?.cwd; const executionResult = await ToolsService.executeTool(toolName, args, signal, cwd); @@ -946,16 +946,16 @@ class AgenticStore { result = executionResult.content; if (executionResult.isError) toolSuccess = false; - } else if (toolSource === ToolSource.FRONTEND) { + } else if (toolSource === ToolSource.BROWSER) { const args = this.parseToolArguments(toolCall.function.arguments); let executionResult: ToolExecutionResult; - if (toolName === BuiltInTool.GET_DATETIME) { + if (toolName === BuiltInTool.BROWSER_GET_DATETIME) { executionResult = executeGetDatetimeTool(); - } else if (toolName === BuiltInTool.GET_INFO) { + } else if (toolName === BuiltInTool.SERVER_GET_INFO) { executionResult = executeBrowserInfoTool(); - } else if (toolName === BuiltInTool.READ_MEDIA) { + } else if (toolName === BuiltInTool.BROWSER_READ_MEDIA) { executionResult = await ReadMediaService.executeTool( args, { diff --git a/tools/ui/src/lib/stores/tools.svelte.ts b/tools/ui/src/lib/stores/tools.svelte.ts index 4cfb9f370..6699da1ec 100644 --- a/tools/ui/src/lib/stores/tools.svelte.ts +++ b/tools/ui/src/lib/stores/tools.svelte.ts @@ -28,11 +28,11 @@ import { SvelteMap, SvelteSet } from 'svelte/reactivity'; /** Stable selection identity for a tool, shared by the disabled set and the permission store */ class ToolsStore { - private _builtinTools = $state([]); + private _serverTools = $state([]); private _loading = $state(false); private _error = $state(null); private _disabledTools = $state(new SvelteSet()); - // builtin tools that resolve their paths against the working directory, + // server tools that resolve their paths against the working directory, // as declared by the server in its `/tools` listing private _cwdAwareTools = $state(new SvelteSet()); private _toolsEndpointUnreachable = $state(false); @@ -58,7 +58,7 @@ class ToolsStore { console.error('[ToolsStore] Failed to load disabled tools from localStorage:', err); } - this.fetchBuiltinTools(); + this.fetchServerTools(); } private persistDisabledTools(): void { @@ -78,10 +78,10 @@ class ToolsStore { return serverId ? `mcp-${serverId}:${name}` : `mcp:${name}`; case ToolSource.CUSTOM: return `custom:${name}`; - case ToolSource.FRONTEND: - return `frontend:${name}`; + case ToolSource.BROWSER: + return `browser:${name}`; default: - return `builtin:${name}`; + return `server:${name}`; } } @@ -164,8 +164,8 @@ class ToolsStore { }; } - get builtinTools(): OpenAIToolDefinition[] { - return this._builtinTools; + get serverTools(): OpenAIToolDefinition[] { + return this._serverTools; } get serverHome(): string | null { @@ -176,7 +176,7 @@ class ToolsStore { return this.mcpEntries().map((e) => e.definition); } - get frontendTools(): OpenAIToolDefinition[] { + get browserTools(): OpenAIToolDefinition[] { const tools: OpenAIToolDefinition[] = [buildGetDatetimeToolDefinition()]; if (settingsStore.config.jsSandboxEnabled) { @@ -188,25 +188,25 @@ class ToolsStore { if (readMedia) tools.push(readMedia); // provide browser's get_info tool if server doesn't provide one - if (!this.hasBuiltinTool(BuiltInTool.GET_INFO)) { + if (!this.hasServerTool(BuiltInTool.SERVER_GET_INFO)) { tools.push(buildBrowserInfoToolDefinition()); } return tools; } - private hasBuiltinTool(name: BuiltInTool): boolean { - return this._builtinTools.some((def) => def.function.name === name); + private hasServerTool(name: BuiltInTool): boolean { + return this._serverTools.some((def) => def.function.name === name); } /** - * `read_media` runs in the frontend on top of the server's `read_file`, so it + * `read_media` runs in the browser on top of the server's `read_file`, so it * exists only when that tool is served and the active model can perceive the * bytes. The server cannot make this call - it does not know which model the * conversation uses. */ private readMediaTool(): OpenAIToolDefinition | null { - if (!this.hasBuiltinTool(BuiltInTool.READ_FILE)) return null; + if (!this.hasServerTool(BuiltInTool.SERVER_READ_FILE)) return null; const model = modelsStore.selectedModelName ?? modelsStore.models[0]?.model ?? ''; @@ -304,23 +304,23 @@ class ToolsStore { entries.push(entry); }; - for (const def of this._builtinTools) { + for (const def of this._serverTools) { const name = def.function.name; push({ definition: def, - key: this.toolKey(ToolSource.BUILTIN, name), - source: ToolSource.BUILTIN + key: this.toolKey(ToolSource.SERVER, name), + source: ToolSource.SERVER }); } - for (const def of this.frontendTools) { + for (const def of this.browserTools) { const name = def.function.name; push({ definition: def, - key: this.toolKey(ToolSource.FRONTEND, name), - source: ToolSource.FRONTEND + key: this.toolKey(ToolSource.BROWSER, name), + source: ToolSource.BROWSER }); } @@ -384,17 +384,17 @@ class ToolsStore { return entry.serverName ?? ''; case ToolSource.CUSTOM: return TOOL_GROUP_LABELS[ToolSource.CUSTOM]; - case ToolSource.FRONTEND: - return TOOL_GROUP_LABELS[ToolSource.FRONTEND]; + case ToolSource.BROWSER: + return TOOL_GROUP_LABELS[ToolSource.BROWSER]; default: - return TOOL_GROUP_LABELS[ToolSource.BUILTIN]; + return TOOL_GROUP_LABELS[ToolSource.SERVER]; } } /** * Enabled tool definitions for sending to the LLM. * MCP tool schemas are normalized here so the wire payload is consistent - * across all four sources (built-in, frontend/sandbox, MCP, custom JSON). + * across all four sources (server, browser/sandbox, MCP, custom JSON). * The API identifies tools by name, so a name is sent at most once. */ getEnabledToolsForLLM(): OpenAIToolDefinition[] { @@ -417,8 +417,8 @@ class ToolsStore { result.push(def); }; - for (const def of this._builtinTools) take(def); - for (const def of this.frontendTools) take(def); + for (const def of this._serverTools) take(def); + for (const def of this.browserTools) take(def); // mcpEntries() over mcpStore directly so wire shape stays normalized and aligned with the tools UI. for (const entry of this.mcpEntries()) take(entry.definition); for (const def of this.customTools) take(def); @@ -542,11 +542,11 @@ class ToolsStore { if (entry.serverName) return mcpStore.getServerDisplayName(entry.serverName); - if (entry.source === ToolSource.BUILTIN) return TOOL_SERVER_LABELS[ToolSource.BUILTIN]; + if (entry.source === ToolSource.SERVER) return TOOL_SERVER_LABELS[ToolSource.SERVER]; if (entry.source === ToolSource.CUSTOM) return TOOL_SERVER_LABELS[ToolSource.CUSTOM]; - if (entry.source === ToolSource.FRONTEND) return TOOL_SERVER_LABELS[ToolSource.FRONTEND]; + if (entry.source === ToolSource.BROWSER) return TOOL_SERVER_LABELS[ToolSource.BROWSER]; return ''; } @@ -556,27 +556,27 @@ class ToolsStore { return this.findEntryByName(toolName)?.key ?? null; } - /** Check if there are any enabled tools available (builtin, MCP, or custom) */ + /** Check if there are any enabled tools available (server, MCP, or custom) */ get hasEnabledTools(): boolean { return this.getEnabledToolsForLLM().length > 0; } /** - * Check if a working directory is worth setting: at least one builtin tool + * Check if a working directory is worth setting: at least one server tool * that reads it is both served and left enabled by the user. */ get hasEnabledCwdTools(): boolean { - return this._builtinTools.some((def) => { + return this._serverTools.some((def) => { const name = def.function.name; return ( this._cwdAwareTools.has(name) && - !this._disabledTools.has(this.toolKey(ToolSource.BUILTIN, name)) + !this._disabledTools.has(this.toolKey(ToolSource.SERVER, name)) ); }); } - async fetchBuiltinTools(): Promise { + async fetchServerTools(): Promise { if (this._loading) return; this._loading = true; @@ -586,7 +586,7 @@ class ToolsStore { try { const toolInfos = await ToolsService.list(); - this._builtinTools = toolInfos.map((info) => info.definition); + this._serverTools = toolInfos.map((info) => info.definition); this._cwdAwareTools = new SvelteSet( toolInfos.filter((info) => info.uses_cwd).map((info) => info.tool) ); @@ -599,9 +599,9 @@ class ToolsStore { // TODO: check status code instead of relying on message if (errorMessage.includes('this feature is disabled')) { this._toolsEndpointUnreachable = true; - console.info('[ToolsStore] Built-in tools are disabled on the server'); + console.info('[ToolsStore] Server tools are disabled on the server'); } else { - console.error('[ToolsStore] Failed to fetch built-in tools:', err); + console.error('[ToolsStore] Failed to fetch server tools:', err); } } finally { this._loading = false; @@ -618,7 +618,7 @@ class ToolsStore { if (this._serverHome !== undefined) return this._serverHome; try { - const res = await ToolsService.executeToolRaw(BuiltInTool.FILE_GLOB_SEARCH, { + const res = await ToolsService.executeToolRaw(BuiltInTool.SERVER_FILE_GLOB_SEARCH, { limit: 1, max_depth: 1, path: HOME_TILDE, diff --git a/tools/ui/src/lib/types/index.ts b/tools/ui/src/lib/types/index.ts index 562b63e98..62947cd49 100644 --- a/tools/ui/src/lib/types/index.ts +++ b/tools/ui/src/lib/types/index.ts @@ -147,7 +147,7 @@ export type { ServerStatus, ToolCallParams, ToolExecutionResult, - ServerBuiltinToolInfo, + ServerToolInfo, Tool, Prompt, GetPromptResult, @@ -208,7 +208,7 @@ export type { export type { DesktopIconStripItem } from './navigation'; // Tools types -export type { ToolEntry, ToolGroup, BuiltinToolUiEntry } from './tools'; +export type { ToolEntry, ToolGroup, ToolUiEntry } from './tools'; // Reasoning export type { ReasoningEffortLevel } from './reasoning'; diff --git a/tools/ui/src/lib/types/mcp.d.ts b/tools/ui/src/lib/types/mcp.d.ts index d01bbac00..b9e19c739 100644 --- a/tools/ui/src/lib/types/mcp.d.ts +++ b/tools/ui/src/lib/types/mcp.d.ts @@ -285,10 +285,10 @@ export interface ToolExecutionResult { isError: boolean; } -export interface ServerBuiltinToolInfo { +export interface ServerToolInfo { display_name: string; tool: string; - type: ToolSource.BUILTIN; + type: ToolSource.SERVER; permissions: { write: boolean; }; diff --git a/tools/ui/src/lib/types/tools.d.ts b/tools/ui/src/lib/types/tools.d.ts index 39e6fc579..edcec65c7 100644 --- a/tools/ui/src/lib/types/tools.d.ts +++ b/tools/ui/src/lib/types/tools.d.ts @@ -3,12 +3,12 @@ import type { ToolSource } from '$lib/enums'; import type { Component } from 'svelte'; /** - * UI metadata for a built-in or frontend tool, keyed by its `BuiltInTool` id. + * UI metadata for a server or browser tool, keyed by its `BuiltInTool` id. */ -export interface BuiltinToolUiEntry { +export interface ToolUiEntry { icon: Component; label: string; - source: ToolSource.BUILTIN | ToolSource.FRONTEND; + source: ToolSource.SERVER | ToolSource.BROWSER; } export interface ToolEntry { @@ -17,7 +17,7 @@ export interface ToolEntry { serverName?: string; /** For MCP tools, the server ID (used for permission keys) */ serverId?: string; - /** Stable selection identity: builtin:name, mcp-:name, mcp:name, custom:name */ + /** Stable selection identity: server:name, mcp-:name, mcp:name, custom:name */ key: string; definition: OpenAIToolDefinition; } diff --git a/tools/ui/src/lib/utils/built-in-tools.ts b/tools/ui/src/lib/utils/built-in-tools.ts deleted file mode 100644 index 73b554790..000000000 --- a/tools/ui/src/lib/utils/built-in-tools.ts +++ /dev/null @@ -1,13 +0,0 @@ -import { BUILTIN_TOOL_UI } from '$lib/constants'; -import type { BuiltinToolUiEntry } from '$lib/types'; - -/** - * Resolve the UI metadata (label + icon) for a built-in tool by its name. - * Falls back to null for unknown or non-built-in tools so callers can render - * a generic chrome instead. - */ -export function getBuiltinToolUi(toolName: string | undefined): BuiltinToolUiEntry | null { - if (!toolName) return null; - - return (BUILTIN_TOOL_UI as Record)[toolName] ?? null; -} diff --git a/tools/ui/src/lib/utils/get-datetime.ts b/tools/ui/src/lib/utils/get-datetime.ts index cd17bb1e8..d971d728e 100644 --- a/tools/ui/src/lib/utils/get-datetime.ts +++ b/tools/ui/src/lib/utils/get-datetime.ts @@ -1,5 +1,5 @@ /** - * Frontend executor for the `get_datetime` tool. It runs in the browser, so it + * Browser executor for the `get_datetime` tool. It runs in the browser, so it * reports the user's own clock and time zone instead of the server's UTC time - * a chat about "tomorrow" means the user's tomorrow, not the host's. * diff --git a/tools/ui/src/lib/utils/glob-search.ts b/tools/ui/src/lib/utils/glob-search.ts index 7580f3286..9b35c4fe8 100644 --- a/tools/ui/src/lib/utils/glob-search.ts +++ b/tools/ui/src/lib/utils/glob-search.ts @@ -42,7 +42,7 @@ export async function runGlobSearch( } const res = await ToolsService.executeToolRaw( - BuiltInTool.FILE_GLOB_SEARCH, + BuiltInTool.SERVER_FILE_GLOB_SEARCH, { include: args.include, limit, max_depth: args.maxDepth, path: args.path, type }, signal ); diff --git a/tools/ui/src/lib/utils/index.ts b/tools/ui/src/lib/utils/index.ts index dd997c206..eba1d815b 100644 --- a/tools/ui/src/lib/utils/index.ts +++ b/tools/ui/src/lib/utils/index.ts @@ -127,7 +127,7 @@ export { sanitizeKeyValuePairKey, sanitizeKeyValuePairValue } from './sanitize'; // Image error fallback utilities export { getImageErrorFallbackHtml } from './image-error-fallback'; -// SSE-with-JSON stream iterator (used by built-in tool streaming, decoupled +// SSE-with-JSON stream iterator (used by server tool streaming, decoupled // from chat.service.ts which embeds its own SSE parser for resume support) export { parseSseJsonStream } from './sse'; @@ -310,7 +310,7 @@ export { withAbortSignal } from './abort'; -// Tool-call meta utilities. Parsers for each built-in tool live next to +// Tool-call meta utilities. Parsers for each server tool live next to // their renderer family under // `src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/parsers/`. // This module only carries the helpers that genuinely cross tool @@ -321,7 +321,7 @@ export { tryParseToolResultObject } from './tool-call-meta'; // Per-tool UI metadata (label + icon) used by the tool-call chrome. // Re-exported through $lib/utils so renderer components can read the // label without depending on $lib/constants directly. -export { getBuiltinToolUi } from './built-in-tools'; +export { getToolUi } from './tool-ui'; // Chat command picker @@ -331,7 +331,7 @@ export { getChatCommands } from './chat-commands'; // SANDBOX_TOOL_DEFINITION is deprecated; kept for backward compatibility. export { buildSandboxToolDefinition, SANDBOX_TOOL_DEFINITION } from './sandbox-tool'; -// Frontend `get_datetime` executor (the browser clock, not the server's) +// Browser `get_datetime` executor (the browser clock, not the server's) export { executeGetDatetimeTool } from './get-datetime'; // Browser fallback for the server's get_info tool diff --git a/tools/ui/src/lib/utils/tool-ui.ts b/tools/ui/src/lib/utils/tool-ui.ts new file mode 100644 index 000000000..56130fc95 --- /dev/null +++ b/tools/ui/src/lib/utils/tool-ui.ts @@ -0,0 +1,13 @@ +import { TOOL_UI } from '$lib/constants'; +import type { ToolUiEntry } from '$lib/types'; + +/** + * Resolve the UI metadata (label + icon) for a server or browser tool by its + * name. Falls back to null for unknown tools so callers can render a generic + * chrome instead. + */ +export function getToolUi(toolName: string | undefined): ToolUiEntry | null { + if (!toolName) return null; + + return (TOOL_UI as Record)[toolName] ?? null; +} diff --git a/tools/ui/tests/client/chat-form-mention-picker-gate.svelte.test.ts b/tools/ui/tests/client/chat-form-mention-picker-gate.svelte.test.ts index fc06e5d2c..cbf3747cc 100644 --- a/tools/ui/tests/client/chat-form-mention-picker-gate.svelte.test.ts +++ b/tools/ui/tests/client/chat-form-mention-picker-gate.svelte.test.ts @@ -13,15 +13,15 @@ import { afterEach, describe, expect, it } from 'vitest'; import { render } from 'vitest-browser-svelte'; const FILE_SEARCH_DEF: OpenAIToolDefinition = { - function: { description: '', name: BuiltInTool.FILE_GLOB_SEARCH, parameters: {} }, + function: { description: '', name: BuiltInTool.SERVER_FILE_GLOB_SEARCH, parameters: {} }, type: 'function' }; -const FILE_SEARCH_KEY = `builtin:${BuiltInTool.FILE_GLOB_SEARCH}`; +const FILE_SEARCH_KEY = `server:${BuiltInTool.SERVER_FILE_GLOB_SEARCH}`; -// The store keeps its builtin tool list private; tests inject it through +// The store keeps its server tool list private; tests inject it through // the reactive field so the derived gates recompute. -function setBuiltinTools(defs: OpenAIToolDefinition[]) { - (toolsStore as unknown as { _builtinTools: OpenAIToolDefinition[] })._builtinTools = defs; +function setServerTools(defs: OpenAIToolDefinition[]) { + (toolsStore as unknown as { _serverTools: OpenAIToolDefinition[] })._serverTools = defs; } function renderPicker() { @@ -34,14 +34,14 @@ function renderPicker() { } afterEach(() => { - setBuiltinTools([]); + setServerTools([]); toolsStore.setToolEnabled(FILE_SEARCH_KEY, true); localStorage.removeItem(DISABLED_TOOL_KEYS_LOCALSTORAGE_KEY); }); describe('ChatFormPickerMention file_glob_search gate', () => { it('explains that file search is unavailable when the server has no tools', async () => { - setBuiltinTools([]); + setServerTools([]); renderPicker(); await tick(); @@ -51,7 +51,7 @@ describe('ChatFormPickerMention file_glob_search gate', () => { }); it('explains that file search must be enabled when the user disabled it', async () => { - setBuiltinTools([FILE_SEARCH_DEF]); + setServerTools([FILE_SEARCH_DEF]); toolsStore.setToolEnabled(FILE_SEARCH_KEY, false); renderPicker(); await tick(); diff --git a/tools/ui/tests/unit/tool-calls.test.ts b/tools/ui/tests/unit/tool-calls.test.ts index b0ec76be4..f84a2405e 100644 --- a/tools/ui/tests/unit/tool-calls.test.ts +++ b/tools/ui/tests/unit/tool-calls.test.ts @@ -16,7 +16,7 @@ import { describe, expect, it } from 'vitest'; function makeSection( overrides: Partial = {}, - toolName = BuiltInTool.READ_FILE + toolName = BuiltInTool.SERVER_READ_FILE ): AgenticSection { return { content: '', @@ -115,15 +115,18 @@ describe('formatCwdMessage / parseCwdMessage', () => { describe('parseToolArgs (shared)', () => { it('returns null when the section has no toolArgs', () => { - const result = parseToolArgs(BuiltInTool.READ_FILE, makeSection({ toolArgs: undefined })); + const result = parseToolArgs( + BuiltInTool.SERVER_READ_FILE, + makeSection({ toolArgs: undefined }) + ); expect(result).toBeNull(); }); it('returns null when the tool name does not match', () => { const result = parseToolArgs( - BuiltInTool.READ_FILE, - makeSection({ toolArgs: '{"path":"/x"}' }, BuiltInTool.WRITE_FILE) + BuiltInTool.SERVER_READ_FILE, + makeSection({ toolArgs: '{"path":"/x"}' }, BuiltInTool.SERVER_WRITE_FILE) ); expect(result).toBeNull(); @@ -131,7 +134,7 @@ describe('parseToolArgs (shared)', () => { it('returns null when args are not valid final JSON (partial: false)', () => { const result = parseToolArgs( - BuiltInTool.READ_FILE, + BuiltInTool.SERVER_READ_FILE, makeSection({ toolArgs: '{"path": "/foo.tx' }) ); @@ -140,7 +143,7 @@ describe('parseToolArgs (shared)', () => { it('returns parsed args when valid final JSON', () => { const result = parseToolArgs( - BuiltInTool.READ_FILE, + BuiltInTool.SERVER_READ_FILE, makeSection({ toolArgs: '{"path":"/foo.txt"}' }) ); @@ -149,7 +152,7 @@ describe('parseToolArgs (shared)', () => { it('accepts partial JSON when partial: true', () => { const result = parseToolArgs( - BuiltInTool.READ_FILE, + BuiltInTool.SERVER_READ_FILE, makeSection({ toolArgs: '{"path": "/foo.tx' }), { partial: true } ); @@ -162,7 +165,10 @@ describe('parseWriteFileMeta', () => { it('returns null for sections with a different tool name', () => { expect( parseWriteFileMeta( - makeSection({ toolArgs: '{"path":"/x","content":"y"}', toolName: BuiltInTool.READ_FILE }) + makeSection({ + toolArgs: '{"path":"/x","content":"y"}', + toolName: BuiltInTool.SERVER_READ_FILE + }) ) ).toBeNull(); }); @@ -170,14 +176,14 @@ describe('parseWriteFileMeta', () => { it('returns null when args have no path-like field', () => { expect( parseWriteFileMeta( - makeSection({ toolArgs: '{"content":"x"}', toolName: BuiltInTool.WRITE_FILE }) + makeSection({ toolArgs: '{"content":"x"}', toolName: BuiltInTool.SERVER_WRITE_FILE }) ) ).toBeNull(); }); it('accepts partial args (renders incrementally as content streams in)', () => { const meta = parseWriteFileMeta( - makeSection({ toolArgs: '{"path":"/foo.t', toolName: BuiltInTool.WRITE_FILE }) + makeSection({ toolArgs: '{"path":"/foo.t', toolName: BuiltInTool.SERVER_WRITE_FILE }) ); expect(meta?.filePath).toBe('/foo.t'); @@ -188,10 +194,10 @@ describe('parseWriteFileMeta', () => { makeSection( { toolArgs: '{"path":"/foo.ts","content":"x"}', - toolName: BuiltInTool.WRITE_FILE, + toolName: BuiltInTool.SERVER_WRITE_FILE, toolResult: '{"result":"wrote","bytes":42}' }, - BuiltInTool.WRITE_FILE + BuiltInTool.SERVER_WRITE_FILE ) ); @@ -208,7 +214,7 @@ describe('parseWriteFileMeta', () => { const meta = parseWriteFileMeta( makeSection({ toolArgs: '{"path":"/foo","content":"x"}', - toolName: BuiltInTool.WRITE_FILE, + toolName: BuiltInTool.SERVER_WRITE_FILE, toolResult: '{"error":"permission denied"}' }) ); @@ -223,10 +229,10 @@ describe('parseEditFileMeta', () => { { toolArgs: '{"path":"/foo.ts","edits":[{"old_text":"a","new_text":"b"},{"old_text":"c","new_text":"d"}]}', - toolName: BuiltInTool.EDIT_FILE, + toolName: BuiltInTool.SERVER_EDIT_FILE, toolResult: '{"result":"ok","edits_applied":2}' }, - BuiltInTool.EDIT_FILE + BuiltInTool.SERVER_EDIT_FILE ); const meta = parseEditFileMeta(section); @@ -242,9 +248,9 @@ describe('parseEditFileMeta', () => { const section = makeSection( { toolArgs: '{"path":"/foo","edits":[{"old_text":""},{"old_text":"a","new_text":""}]}', - toolName: BuiltInTool.EDIT_FILE + toolName: BuiltInTool.SERVER_EDIT_FILE }, - BuiltInTool.EDIT_FILE + BuiltInTool.SERVER_EDIT_FILE ); const meta = parseEditFileMeta(section); @@ -257,10 +263,10 @@ describe('parseEditFileMeta', () => { const section = makeSection( { toolArgs: '{"path":"/foo"}', - toolName: BuiltInTool.EDIT_FILE, + toolName: BuiltInTool.SERVER_EDIT_FILE, toolResult: '{"error":"bad path","result":"ok"}' }, - BuiltInTool.EDIT_FILE + BuiltInTool.SERVER_EDIT_FILE ); const meta = parseEditFileMeta(section); @@ -272,7 +278,7 @@ describe('parseEditFileMeta', () => { describe('parseReadFileMeta', () => { it('parses file name alone (no range)', () => { const meta = parseReadFileMeta( - makeSection({ toolArgs: '{"path":"/foo.txt"}' }, BuiltInTool.READ_FILE) + makeSection({ toolArgs: '{"path":"/foo.txt"}' }, BuiltInTool.SERVER_READ_FILE) ); expect(meta?.fileName).toBe('foo.txt'); @@ -283,7 +289,7 @@ describe('parseReadFileMeta', () => { const meta = parseReadFileMeta( makeSection( { toolArgs: '{"path":"/foo.ts","start_line":10,"end_line":20}' }, - BuiltInTool.READ_FILE + BuiltInTool.SERVER_READ_FILE ) ); @@ -294,7 +300,7 @@ describe('parseReadFileMeta', () => { const meta = parseReadFileMeta( makeSection( { toolArgs: '{"path":"/foo.ts","start_line":10,"line_count":5}' }, - BuiltInTool.READ_FILE + BuiltInTool.SERVER_READ_FILE ) ); @@ -302,7 +308,9 @@ describe('parseReadFileMeta', () => { }); it('returns null when args cannot be parsed', () => { - expect(parseReadFileMeta(makeSection({ toolArgs: '{bad' }, BuiltInTool.READ_FILE))).toBeNull(); + expect( + parseReadFileMeta(makeSection({ toolArgs: '{bad' }, BuiltInTool.SERVER_READ_FILE)) + ).toBeNull(); }); }); @@ -310,12 +318,12 @@ describe('parseGrepSearchMeta', () => { it('returns null when path or pattern is missing', () => { expect( parseGrepSearchMeta( - makeSection({ toolArgs: '{"pattern":"foo"}', toolName: BuiltInTool.GREP_SEARCH }) + makeSection({ toolArgs: '{"pattern":"foo"}', toolName: BuiltInTool.SERVER_GREP_SEARCH }) ) ).toBeNull(); expect( parseGrepSearchMeta( - makeSection({ toolArgs: '{"path":"/x"}', toolName: BuiltInTool.GREP_SEARCH }) + makeSection({ toolArgs: '{"path":"/x"}', toolName: BuiltInTool.SERVER_GREP_SEARCH }) ) ).toBeNull(); }); @@ -325,10 +333,10 @@ describe('parseGrepSearchMeta', () => { makeSection( { toolArgs: '{"path":"/x","pattern":"foo"}', - toolName: BuiltInTool.GREP_SEARCH, + toolName: BuiltInTool.SERVER_GREP_SEARCH, toolResult: JSON.stringify({ plain_text_response: 'a.ts:hello\nb.ts:world' }) }, - BuiltInTool.GREP_SEARCH + BuiltInTool.SERVER_GREP_SEARCH ) ); @@ -341,10 +349,10 @@ describe('parseGrepSearchMeta', () => { makeSection( { toolArgs: '{"path":"/x","pattern":"foo"}', - toolName: BuiltInTool.GREP_SEARCH, + toolName: BuiltInTool.SERVER_GREP_SEARCH, toolResult: 'a.ts:hello\nb.ts:world' }, - BuiltInTool.GREP_SEARCH + BuiltInTool.SERVER_GREP_SEARCH ) ); @@ -356,10 +364,10 @@ describe('parseGrepSearchMeta', () => { makeSection( { toolArgs: '{"path":"/x","pattern":"foo","return_line_numbers":true}', - toolName: BuiltInTool.GREP_SEARCH, + toolName: BuiltInTool.SERVER_GREP_SEARCH, toolResult: 'a.ts:12:hello' }, - BuiltInTool.GREP_SEARCH + BuiltInTool.SERVER_GREP_SEARCH ) ); @@ -374,10 +382,10 @@ describe('parseFileGlobSearchMeta', () => { makeSection( { toolArgs: '{"path":"/x"}', - toolName: BuiltInTool.FILE_GLOB_SEARCH, + toolName: BuiltInTool.SERVER_FILE_GLOB_SEARCH, toolResult: 'a.ts\nb.ts' }, - BuiltInTool.FILE_GLOB_SEARCH + BuiltInTool.SERVER_FILE_GLOB_SEARCH ) ); @@ -389,10 +397,10 @@ describe('parseFileGlobSearchMeta', () => { makeSection( { toolArgs: '{"path":"/x"}', - toolName: BuiltInTool.FILE_GLOB_SEARCH, + toolName: BuiltInTool.SERVER_FILE_GLOB_SEARCH, toolResult: JSON.stringify({ plain_text_response: 'a.ts\nb.ts' }) }, - BuiltInTool.FILE_GLOB_SEARCH + BuiltInTool.SERVER_FILE_GLOB_SEARCH ) ); @@ -404,10 +412,10 @@ describe('parseFileGlobSearchMeta', () => { makeSection( { toolArgs: '{"path":"/x"}', - toolName: BuiltInTool.FILE_GLOB_SEARCH, + toolName: BuiltInTool.SERVER_FILE_GLOB_SEARCH, toolResult: JSON.stringify({ error: 'permission denied' }) }, - BuiltInTool.FILE_GLOB_SEARCH + BuiltInTool.SERVER_FILE_GLOB_SEARCH ) ); @@ -418,15 +426,20 @@ describe('parseFileGlobSearchMeta', () => { describe('parseRunJavascriptMeta', () => { it('returns null when code is missing', () => { expect( - parseRunJavascriptMeta(makeSection({ toolArgs: '{}', toolName: BuiltInTool.RUN_JAVASCRIPT })) + parseRunJavascriptMeta( + makeSection({ toolArgs: '{}', toolName: BuiltInTool.BROWSER_RUN_JAVASCRIPT }) + ) ).toBeNull(); }); it('reads code and timeout', () => { const meta = parseRunJavascriptMeta( makeSection( - { toolArgs: '{"code":"Math.PI","timeout_ms":5000}', toolName: BuiltInTool.RUN_JAVASCRIPT }, - BuiltInTool.RUN_JAVASCRIPT + { + toolArgs: '{"code":"Math.PI","timeout_ms":5000}', + toolName: BuiltInTool.BROWSER_RUN_JAVASCRIPT + }, + BuiltInTool.BROWSER_RUN_JAVASCRIPT ) ); @@ -439,10 +452,10 @@ describe('parseRunJavascriptMeta', () => { makeSection( { toolArgs: '{"code":"throw new Error()"}', - toolName: BuiltInTool.RUN_JAVASCRIPT, + toolName: BuiltInTool.BROWSER_RUN_JAVASCRIPT, toolResult: JSON.stringify({ error: 'undefined is not a function' }) }, - BuiltInTool.RUN_JAVASCRIPT + BuiltInTool.BROWSER_RUN_JAVASCRIPT ) ); @@ -457,10 +470,10 @@ describe('parseRunJavascriptMeta', () => { makeSection( { toolArgs: '{"code":"[1,2,3]"}', - toolName: BuiltInTool.RUN_JAVASCRIPT, + toolName: BuiltInTool.BROWSER_RUN_JAVASCRIPT, toolResult: '[1,2,3]' }, - BuiltInTool.RUN_JAVASCRIPT + BuiltInTool.BROWSER_RUN_JAVASCRIPT ) ); @@ -472,10 +485,10 @@ describe('parseRunJavascriptMeta', () => { makeSection( { toolArgs: '{"code":"foo"}', - toolName: BuiltInTool.RUN_JAVASCRIPT, + toolName: BuiltInTool.BROWSER_RUN_JAVASCRIPT, toolResult: 'Error: undefined is not a function\n at :1:1' }, - BuiltInTool.RUN_JAVASCRIPT + BuiltInTool.BROWSER_RUN_JAVASCRIPT ) ); @@ -487,8 +500,8 @@ describe('parseExecShellCommandMeta', () => { it('reads command from the args', () => { const meta = parseExecShellCommandMeta( makeSection( - { toolArgs: '{"command":"ls -la"}', toolName: BuiltInTool.EXEC_SHELL_COMMAND }, - BuiltInTool.EXEC_SHELL_COMMAND + { toolArgs: '{"command":"ls -la"}', toolName: BuiltInTool.SERVER_EXEC_SHELL_COMMAND }, + BuiltInTool.SERVER_EXEC_SHELL_COMMAND ) ); @@ -499,16 +512,16 @@ describe('parseExecShellCommandMeta', () => { expect( parseExecShellCommandMeta( makeSection( - { toolArgs: '{"cmd":"ls"}', toolName: BuiltInTool.EXEC_SHELL_COMMAND }, - BuiltInTool.EXEC_SHELL_COMMAND + { toolArgs: '{"cmd":"ls"}', toolName: BuiltInTool.SERVER_EXEC_SHELL_COMMAND }, + BuiltInTool.SERVER_EXEC_SHELL_COMMAND ) )?.command ).toBe('ls'); expect( parseExecShellCommandMeta( makeSection( - { toolArgs: '{"shell_command":"ls"}', toolName: BuiltInTool.EXEC_SHELL_COMMAND }, - BuiltInTool.EXEC_SHELL_COMMAND + { toolArgs: '{"shell_command":"ls"}', toolName: BuiltInTool.SERVER_EXEC_SHELL_COMMAND }, + BuiltInTool.SERVER_EXEC_SHELL_COMMAND ) )?.command ).toBe('ls'); @@ -518,8 +531,8 @@ describe('parseExecShellCommandMeta', () => { expect( parseExecShellCommandMeta( makeSection( - { toolArgs: '{"cwd":"/x"}', toolName: BuiltInTool.EXEC_SHELL_COMMAND }, - BuiltInTool.EXEC_SHELL_COMMAND + { toolArgs: '{"cwd":"/x"}', toolName: BuiltInTool.SERVER_EXEC_SHELL_COMMAND }, + BuiltInTool.SERVER_EXEC_SHELL_COMMAND ) ) ).toBeNull(); From 01818e4956858fe225a6875aae88b4cbee0b319e Mon Sep 17 00:00:00 2001 From: Aleksander Grygier Date: Mon, 17 Aug 2026 23:52:00 +0200 Subject: [PATCH 04/48] ui: enforce alphabetical enum member ordering (#27272) --- tools/ui/eslint.config.js | 3 + tools/ui/src/lib/enums/agentic.enums.ts | 12 +- tools/ui/src/lib/enums/attachment.enums.ts | 26 +- .../ui/src/lib/enums/boolean-string.enums.ts | 4 +- tools/ui/src/lib/enums/chat.enums.ts | 48 ++-- .../lib/enums/conversation-import.enums.ts | 4 +- tools/ui/src/lib/enums/files.enums.ts | 264 +++++++++--------- tools/ui/src/lib/enums/keyboard.enums.ts | 10 +- tools/ui/src/lib/enums/mcp.enums.ts | 34 +-- tools/ui/src/lib/enums/model.enums.ts | 6 +- .../src/lib/enums/reasoning-effort.enums.ts | 8 +- tools/ui/src/lib/enums/server.enums.ts | 16 +- tools/ui/src/lib/enums/settings.enums.ts | 14 +- tools/ui/src/lib/enums/splash.enums.ts | 4 +- tools/ui/src/lib/enums/tools.enums.ts | 12 +- tools/ui/src/lib/enums/ui.enums.ts | 18 +- tools/ui/src/lib/utils/search-results.ts | 6 +- 17 files changed, 246 insertions(+), 243 deletions(-) diff --git a/tools/ui/eslint.config.js b/tools/ui/eslint.config.js index c65484048..b8bdb216e 100644 --- a/tools/ui/eslint.config.js +++ b/tools/ui/eslint.config.js @@ -61,6 +61,9 @@ export default ts.config( { blankLine: 'always', next: ['return', 'throw', 'break', 'continue'], prev: '*' } ], + // Alphabetical order for enum members + 'perfectionist/sort-enums': ['error', { type: 'natural' }], + 'perfectionist/sort-objects': ['error', { type: 'natural' }], // Alphabetical order for variable declarations and object keys diff --git a/tools/ui/src/lib/enums/agentic.enums.ts b/tools/ui/src/lib/enums/agentic.enums.ts index 59e996e93..6dc46b085 100644 --- a/tools/ui/src/lib/enums/agentic.enums.ts +++ b/tools/ui/src/lib/enums/agentic.enums.ts @@ -9,12 +9,12 @@ export enum ToolCallType { * Types of sections in agentic content display. */ export enum AgenticSectionType { + REASONING = 'reasoning', + REASONING_PENDING = 'reasoning_pending', TEXT = 'text', TOOL_CALL = 'tool_call', TOOL_CALL_PENDING = 'tool_call_pending', - TOOL_CALL_STREAMING = 'tool_call_streaming', - REASONING = 'reasoning', - REASONING_PENDING = 'reasoning_pending' + TOOL_CALL_STREAMING = 'tool_call_streaming' } /** @@ -22,8 +22,8 @@ export enum AgenticSectionType { */ export enum ContinueIntentKind { APPEND_TEXT = 'append_text', - RERUN_TURN = 'rerun_turn', - NEXT_TURN = 'next_turn' + NEXT_TURN = 'next_turn', + RERUN_TURN = 'rerun_turn' } /** @@ -39,7 +39,7 @@ export enum ToolResultKind { * Line classification for the unified-diff renderer of `edit_file` results. */ export enum DiffLineKind { - CONTEXT = 'context', ADD = 'add', + CONTEXT = 'context', REMOVE = 'remove' } diff --git a/tools/ui/src/lib/enums/attachment.enums.ts b/tools/ui/src/lib/enums/attachment.enums.ts index c6fc2b82a..70ed36d89 100644 --- a/tools/ui/src/lib/enums/attachment.enums.ts +++ b/tools/ui/src/lib/enums/attachment.enums.ts @@ -4,12 +4,12 @@ export enum AttachmentType { AUDIO = 'AUDIO', IMAGE = 'IMAGE', - VIDEO = 'VIDEO', + LEGACY_CONTEXT = 'context', // Legacy attachment type for backward compatibility MCP_PROMPT = 'MCP_PROMPT', MCP_RESOURCE = 'MCP_RESOURCE', PDF = 'PDF', TEXT = 'TEXT', - LEGACY_CONTEXT = 'context' // Legacy attachment type for backward compatibility + VIDEO = 'VIDEO' } /** @@ -17,14 +17,14 @@ export enum AttachmentType { * Used to select which file upload or attachment action is triggered. */ export enum AttachmentMenuItemId { - IMAGES = 'images', AUDIO = 'audio', - VIDEO = 'video', - TEXT = 'text', + IMAGES = 'images', + MCP_PROMPT = 'mcp-prompt', + MCP_RESOURCES = 'mcp-resources', PDF = 'pdf', SYSTEM_MESSAGE = 'system-message', - MCP_PROMPT = 'mcp-prompt', - MCP_RESOURCES = 'mcp-resources' + TEXT = 'text', + VIDEO = 'video' } /** @@ -32,9 +32,9 @@ export enum AttachmentMenuItemId { */ export enum AttachmentItemEnabledWhen { ALWAYS = 'always', - HAS_VISION_MODALITY = 'hasVisionModality', HAS_AUDIO_MODALITY = 'hasAudioModality', - HAS_VIDEO_MODALITY = 'hasVideoModality' + HAS_VIDEO_MODALITY = 'hasVideoModality', + HAS_VISION_MODALITY = 'hasVisionModality' } /** @@ -42,9 +42,9 @@ export enum AttachmentItemEnabledWhen { */ export enum AttachmentAction { FILE_UPLOAD = 'onFileUpload', - SYSTEM_PROMPT_CLICK = 'onSystemPromptClick', MCP_PROMPT_CLICK = 'onMcpPromptClick', - MCP_RESOURCES_CLICK = 'onMcpResourcesClick' + MCP_RESOURCES_CLICK = 'onMcpResourcesClick', + SYSTEM_PROMPT_CLICK = 'onSystemPromptClick' } /** @@ -52,9 +52,9 @@ export enum AttachmentAction { */ export enum AttachmentLabel { FILE = 'File', - PDF_FILE = 'PDF File', MCP_PROMPT = 'MCP Prompt', - MCP_RESOURCE = 'MCP Resource' + MCP_RESOURCE = 'MCP Resource', + PDF_FILE = 'PDF File' } /** diff --git a/tools/ui/src/lib/enums/boolean-string.enums.ts b/tools/ui/src/lib/enums/boolean-string.enums.ts index d6b7ac889..80a4f72bc 100644 --- a/tools/ui/src/lib/enums/boolean-string.enums.ts +++ b/tools/ui/src/lib/enums/boolean-string.enums.ts @@ -1,5 +1,5 @@ /** String representation of a boolean used in data attributes and persisted values. */ export enum BooleanString { - TRUE = 'true', - FALSE = 'false' + FALSE = 'false', + TRUE = 'true' } diff --git a/tools/ui/src/lib/enums/chat.enums.ts b/tools/ui/src/lib/enums/chat.enums.ts index 152b42c12..6dcede2a5 100644 --- a/tools/ui/src/lib/enums/chat.enums.ts +++ b/tools/ui/src/lib/enums/chat.enums.ts @@ -1,41 +1,41 @@ export enum ChatMessageStatsView { GENERATION = 'generation', READING = 'reading', - TOOLS = 'tools', - SUMMARY = 'summary' + SUMMARY = 'summary', + TOOLS = 'tools' } export enum ChatMessageStatisticsMode { - SWITCHABLE = 'switchable', + GENERATION = 'generation', READING = 'reading', - GENERATION = 'generation' + SWITCHABLE = 'switchable' } /** * Connection state of a streamed completion, drives the resume status indicator. */ export enum StreamConnectionState { - STREAMING = 'streaming', + LOST = 'lost', RESUMING = 'resuming', - LOST = 'lost' + STREAMING = 'streaming' } /** * Reasoning format options for API requests. */ export enum ReasoningFormat { - NONE = 'none', - AUTO = 'auto' + AUTO = 'auto', + NONE = 'none' } /** * Message roles for chat messages. */ export enum MessageRole { - USER = 'user', ASSISTANT = 'assistant', SYSTEM = 'system', - TOOL = 'tool' + TOOL = 'tool', + USER = 'user' } /** @@ -43,27 +43,27 @@ export enum MessageRole { */ export enum MessageType { ROOT = 'root', + SYSTEM = 'system', TEXT = 'text', - THINK = 'think', - SYSTEM = 'system' + THINK = 'think' } /** * Content part types for API chat message content. */ export enum ContentPartType { - TEXT = 'text', IMAGE_URL = 'image_url', INPUT_AUDIO = 'input_audio', - INPUT_VIDEO = 'input_video' + INPUT_VIDEO = 'input_video', + TEXT = 'text' } /** * Error dialog types for displaying server/timeout errors. */ export enum ErrorDialogType { - TIMEOUT = 'timeout', - SERVER = 'server' + SERVER = 'server', + TIMEOUT = 'timeout' } export enum ConversationSelectionMode { @@ -75,27 +75,27 @@ export enum ConversationSelectionMode { * PDF view mode options for previewing PDF attachments. */ export enum PdfViewMode { - TEXT = 'text', - PAGES = 'pages' + PAGES = 'pages', + TEXT = 'text' } export enum ChatFormCommandAction { - PROMPT = 'prompt', CWD = 'cwd', - MODEL = 'model' + MODEL = 'model', + PROMPT = 'prompt' } export enum FileMentionEntryType { - FILE = 'file', - DIRECTORY = 'directory' + DIRECTORY = 'directory', + FILE = 'file' } /** * Kinds of tokens the chat-form-input-rich produces. */ export enum ChatFormInputRichTokenKind { - TEXT = 'text', BADGE = 'badge', + CODE_BLOCK = 'code_block', CODE_INLINE = 'code_inline', - CODE_BLOCK = 'code_block' + TEXT = 'text' } diff --git a/tools/ui/src/lib/enums/conversation-import.enums.ts b/tools/ui/src/lib/enums/conversation-import.enums.ts index eef47c5cc..c2cf99deb 100644 --- a/tools/ui/src/lib/enums/conversation-import.enums.ts +++ b/tools/ui/src/lib/enums/conversation-import.enums.ts @@ -4,6 +4,6 @@ * message record belongs to it. */ export enum SessionRecordType { - SESSION = 'session', - MESSAGE = 'message' + MESSAGE = 'message', + SESSION = 'session' } diff --git a/tools/ui/src/lib/enums/files.enums.ts b/tools/ui/src/lib/enums/files.enums.ts index 5785428cf..0185da478 100644 --- a/tools/ui/src/lib/enums/files.enums.ts +++ b/tools/ui/src/lib/enums/files.enums.ts @@ -5,11 +5,11 @@ // File type category enum export enum FileTypeCategory { - IMAGE = 'image', AUDIO = 'audio', - VIDEO = 'video', + IMAGE = 'image', PDF = 'pdf', - TEXT = 'text' + TEXT = 'text', + VIDEO = 'video' } /** @@ -21,13 +21,13 @@ export enum SpecialFileType { // Specific file type enums for each category export enum FileTypeImage { + GIF = 'gif', + HEIC = 'heic', + HEIF = 'heif', JPEG = 'jpeg', PNG = 'png', - GIF = 'gif', - WEBP = 'webp', SVG = 'svg', - HEIC = 'heic', - HEIF = 'heif' + WEBP = 'webp' } export enum FileTypeAudio { @@ -46,55 +46,55 @@ export enum FileTypePdf { } export enum FileTypeText { - PLAIN_TEXT = 'plainText', - MARKDOWN = 'md', ASCIIDOC = 'asciidoc', - JAVASCRIPT = 'js', - TYPESCRIPT = 'ts', - JSX = 'jsx', - TSX = 'tsx', - CSS = 'css', - HTML = 'html', - JSON = 'json', - XML = 'xml', - YAML = 'yaml', - CSV = 'csv', - LOG = 'log', - PYTHON = 'python', - JAVA = 'java', + BIBTEX = 'bibtex', CPP = 'cpp', - PHP = 'php', - RUBY = 'ruby', + CSHARP = 'csharp', + CSS = 'css', + CSV = 'csv', + CUDA = 'cuda', + DART = 'dart', GO = 'go', + HASKELL = 'haskell', + HTML = 'html', + JAVA = 'java', + JAVASCRIPT = 'js', + JSON = 'json', + JSX = 'jsx', + KOTLIN = 'kotlin', + LATEX = 'latex', + LOG = 'log', + MARKDOWN = 'md', + PHP = 'php', + PLAIN_TEXT = 'plainText', + PROPERTIES = 'properties', + PYTHON = 'python', + R = 'r', + RUBY = 'ruby', RUST = 'rust', + SCALA = 'scala', SHELL = 'shell', SQL = 'sql', - R = 'r', - SCALA = 'scala', - KOTLIN = 'kotlin', - SWIFT = 'swift', - DART = 'dart', - VUE = 'vue', SVELTE = 'svelte', - LATEX = 'latex', - BIBTEX = 'bibtex', - CUDA = 'cuda', + SWIFT = 'swift', + TSX = 'tsx', + TYPESCRIPT = 'ts', + VUE = 'vue', VULKAN = 'vulkan', - HASKELL = 'haskell', - CSHARP = 'csharp', - PROPERTIES = 'properties' + XML = 'xml', + YAML = 'yaml' } // File extension enums export enum FileExtensionImage { - JPG = '.jpg', - JPEG = '.jpeg', - PNG = '.png', GIF = '.gif', - WEBP = '.webp', - SVG = '.svg', HEIC = '.heic', - HEIF = '.heif' + HEIF = '.heif', + JPEG = '.jpeg', + JPG = '.jpg', + PNG = '.png', + SVG = '.svg', + WEBP = '.webp' } export enum FileExtensionAudio { @@ -112,64 +112,64 @@ export enum FileExtensionPdf { } export enum FileExtensionText { - TXT = '.txt', - MD = '.md', ADOC = '.adoc', - JS = '.js', - TS = '.ts', - JSX = '.jsx', - TSX = '.tsx', + BAT = '.bat', + BIB = '.bib', + C = '.c', + COMP = '.comp', + CPP = '.cpp', + CS = '.cs', CSS = '.css', - HTML = '.html', + CSV = '.csv', + CU = '.cu', + CUH = '.cuh', + DART = '.dart', + GO = '.go', + H = '.h', + HPP = '.hpp', + HS = '.hs', HTM = '.htm', + HTML = '.html', + JAVA = '.java', + JS = '.js', JSON = '.json', JSONL = '.jsonl', - ZIP = '.zip', + JSX = '.jsx', + KT = '.kt', + LOG = '.log', + MD = '.md', + PHP = '.php', + PROPERTIES = '.properties', + PY = '.py', + R = '.r', + RB = '.rb', + RS = '.rs', + SCALA = '.scala', + SH = '.sh', + SQL = '.sql', + SVELTE = '.svelte', + SWIFT = '.swift', + TEX = '.tex', + TS = '.ts', + TSX = '.tsx', + TXT = '.txt', + VUE = '.vue', XML = '.xml', YAML = '.yaml', YML = '.yml', - CSV = '.csv', - LOG = '.log', - PY = '.py', - JAVA = '.java', - CPP = '.cpp', - C = '.c', - H = '.h', - PHP = '.php', - RB = '.rb', - GO = '.go', - RS = '.rs', - SH = '.sh', - BAT = '.bat', - SQL = '.sql', - R = '.r', - SCALA = '.scala', - KT = '.kt', - SWIFT = '.swift', - DART = '.dart', - VUE = '.vue', - SVELTE = '.svelte', - TEX = '.tex', - BIB = '.bib', - CU = '.cu', - CUH = '.cuh', - COMP = '.comp', - HPP = '.hpp', - HS = '.hs', - PROPERTIES = '.properties', - CS = '.cs' + ZIP = '.zip' } // MIME type prefixes and includes for content detection export enum MimeTypePrefix { - IMAGE = 'image/', AUDIO = 'audio/', + IMAGE = 'image/', TEXT = 'text' } export enum MimeTypeIncludes { - JSON = 'json', JAVASCRIPT = 'javascript', + JSON = 'json', TYPESCRIPT = 'typescript' } @@ -182,23 +182,23 @@ export enum UriPattern { // MIME type enums export enum MimeTypeApplication { JSON = 'application/json', - PDF = 'application/pdf', OCTET_STREAM = 'application/octet-stream', + PDF = 'application/pdf', ZIP = 'application/zip' } export enum MimeTypeAudio { - MP3_MPEG = 'audio/mpeg', MP3 = 'audio/mp3', + MP3_MPEG = 'audio/mpeg', MP4 = 'audio/mp4', + VND_WAVE = 'audio/vnd.wave', WAV = 'audio/wav', WAVE = 'audio/wave', - X_WAV = 'audio/x-wav', - X_WAVE = 'audio/x-wave', - VND_WAVE = 'audio/vnd.wave', - X_PN_WAV = 'audio/x-pn-wav', WEBM = 'audio/webm', - WEBM_OPUS = 'audio/webm;codecs=opus' + WEBM_OPUS = 'audio/webm;codecs=opus', + X_PN_WAV = 'audio/x-pn-wav', + X_WAV = 'audio/x-wav', + X_WAVE = 'audio/x-wave' } export enum MimeTypeVideo { @@ -207,62 +207,62 @@ export enum MimeTypeVideo { } export enum MimeTypeImage { + GIF = 'image/gif', + HEIC = 'image/heic', + HEIF = 'image/heif', + ICO = 'image/x-icon', + ICO_MICROSOFT = 'image/vnd.microsoft.icon', JPEG = 'image/jpeg', JPG = 'image/jpg', PNG = 'image/png', - GIF = 'image/gif', - WEBP = 'image/webp', SVG = 'image/svg+xml', - ICO = 'image/x-icon', - ICO_MICROSOFT = 'image/vnd.microsoft.icon', - HEIC = 'image/heic', - HEIF = 'image/heif' + WEBP = 'image/webp' } export enum MimeTypeText { - PLAIN = 'text/plain', - MARKDOWN = 'text/markdown', ASCIIDOC = 'text/asciidoc', - JAVASCRIPT = 'text/javascript', - JAVASCRIPT_APP = 'application/javascript', - TYPESCRIPT = 'text/typescript', - JSX = 'text/jsx', - TSX = 'text/tsx', - CSS = 'text/css', - HTML = 'text/html', - JSON = 'application/json', - JSONL = 'application/jsonl', - XML_TEXT = 'text/xml', - XML_APP = 'application/xml', - YAML_TEXT = 'text/yaml', - YAML_APP = 'application/yaml', - CSV = 'text/csv', - PYTHON = 'text/x-python', - JAVA = 'text/x-java-source', + BAT = 'application/x-bat', + BIBTEX = 'text/x-bibtex', + C_HDR = 'text/x-chdr', + C_SRC = 'text/x-csrc', CPP_HDR = 'text/x-c++hdr', CPP_SRC = 'text/x-c++src', CSHARP = 'text/x-csharp', - HASKELL = 'text/x-haskell', - C_SRC = 'text/x-csrc', - C_HDR = 'text/x-chdr', - PHP = 'text/x-php', - RUBY = 'text/x-ruby', - GO = 'text/x-go', - RUST = 'text/x-rust', - SHELL = 'text/x-shellscript', - BAT = 'application/x-bat', - SQL = 'text/x-sql', - R = 'text/x-r', - SCALA = 'text/x-scala', - KOTLIN = 'text/x-kotlin', - SWIFT = 'text/x-swift', + CSS = 'text/css', + CSV = 'text/csv', + CUDA = 'text/x-cuda', DART = 'text/x-dart', - VUE = 'text/x-vue', + GO = 'text/x-go', + HASKELL = 'text/x-haskell', + HTML = 'text/html', + JAVA = 'text/x-java-source', + JAVASCRIPT = 'text/javascript', + JAVASCRIPT_APP = 'application/javascript', + JSON = 'application/json', + JSONL = 'application/jsonl', + JSX = 'text/jsx', + KOTLIN = 'text/x-kotlin', + LATEX = 'application/x-latex', + MARKDOWN = 'text/markdown', + PHP = 'text/x-php', + PLAIN = 'text/plain', + PROPERTIES = 'text/properties', + PYTHON = 'text/x-python', + R = 'text/x-r', + RUBY = 'text/x-ruby', + RUST = 'text/x-rust', + SCALA = 'text/x-scala', + SHELL = 'text/x-shellscript', + SQL = 'text/x-sql', SVELTE = 'text/x-svelte', + SWIFT = 'text/x-swift', TEX = 'text/x-tex', TEX_APP = 'application/x-tex', - LATEX = 'application/x-latex', - BIBTEX = 'text/x-bibtex', - CUDA = 'text/x-cuda', - PROPERTIES = 'text/properties' + TSX = 'text/tsx', + TYPESCRIPT = 'text/typescript', + VUE = 'text/x-vue', + XML_APP = 'application/xml', + XML_TEXT = 'text/xml', + YAML_APP = 'application/yaml', + YAML_TEXT = 'text/yaml' } diff --git a/tools/ui/src/lib/enums/keyboard.enums.ts b/tools/ui/src/lib/enums/keyboard.enums.ts index 735d3e4b4..fb47fcf3e 100644 --- a/tools/ui/src/lib/enums/keyboard.enums.ts +++ b/tools/ui/src/lib/enums/keyboard.enums.ts @@ -2,19 +2,19 @@ * Keyboard key names for event handling */ export enum KeyboardKey { - ENTER = 'Enter', - ESCAPE = 'Escape', - ARROW_UP = 'ArrowUp', ARROW_DOWN = 'ArrowDown', ARROW_LEFT = 'ArrowLeft', ARROW_RIGHT = 'ArrowRight', - TAB = 'Tab', + ARROW_UP = 'ArrowUp', B_LOWER = 'b', D_LOWER = 'd', D_UPPER = 'D', E_UPPER = 'E', + ENTER = 'Enter', + ESCAPE = 'Escape', K_LOWER = 'k', O_LOWER = 'o', O_UPPER = 'O', - SPACE = ' ' + SPACE = ' ', + TAB = 'Tab' } diff --git a/tools/ui/src/lib/enums/mcp.enums.ts b/tools/ui/src/lib/enums/mcp.enums.ts index 3d9a2070d..fc358202b 100644 --- a/tools/ui/src/lib/enums/mcp.enums.ts +++ b/tools/ui/src/lib/enums/mcp.enums.ts @@ -2,61 +2,61 @@ * Connection lifecycle phases for MCP protocol */ export enum MCPConnectionPhase { - IDLE = 'idle', - TRANSPORT_CREATING = 'transport_creating', - TRANSPORT_READY = 'transport_ready', - INITIALIZING = 'initializing', CAPABILITIES_EXCHANGED = 'capabilities_exchanged', - LISTING_TOOLS = 'listing_tools', CONNECTED = 'connected', + DISCONNECTED = 'disconnected', ERROR = 'error', - DISCONNECTED = 'disconnected' + IDLE = 'idle', + INITIALIZING = 'initializing', + LISTING_TOOLS = 'listing_tools', + TRANSPORT_CREATING = 'transport_creating', + TRANSPORT_READY = 'transport_ready' } /** * Log level for connection events */ export enum MCPLogLevel { + ERROR = 'error', INFO = 'info', - WARN = 'warn', - ERROR = 'error' + WARN = 'warn' } /** * Transport types for MCP connections */ export enum MCPTransportType { - WEBSOCKET = 'websocket', + SSE = 'sse', STREAMABLE_HTTP = 'streamable_http', - SSE = 'sse' + WEBSOCKET = 'websocket' } /** * Health check status for MCP servers */ export enum HealthCheckStatus { - IDLE = 'idle', CONNECTING = 'connecting', - SUCCESS = 'success', - ERROR = 'error' + ERROR = 'error', + IDLE = 'idle', + SUCCESS = 'success' } /** * Content types for MCP tool results */ export enum MCPContentType { - TEXT = 'text', IMAGE = 'image', - RESOURCE = 'resource' + RESOURCE = 'resource', + TEXT = 'text' } /** * JSON Schema types used in MCP tool definitions */ export enum JsonSchemaType { + NUMBER = 'number', OBJECT = 'object', - STRING = 'string', - NUMBER = 'number' + STRING = 'string' } /** diff --git a/tools/ui/src/lib/enums/model.enums.ts b/tools/ui/src/lib/enums/model.enums.ts index 7aa469947..df85c9d89 100644 --- a/tools/ui/src/lib/enums/model.enums.ts +++ b/tools/ui/src/lib/enums/model.enums.ts @@ -1,6 +1,6 @@ export enum ModelModality { - TEXT = 'TEXT', AUDIO = 'AUDIO', - VISION = 'VISION', - VIDEO = 'VIDEO' + TEXT = 'TEXT', + VIDEO = 'VIDEO', + VISION = 'VISION' } diff --git a/tools/ui/src/lib/enums/reasoning-effort.enums.ts b/tools/ui/src/lib/enums/reasoning-effort.enums.ts index 6bf86ed4e..7f00ed593 100644 --- a/tools/ui/src/lib/enums/reasoning-effort.enums.ts +++ b/tools/ui/src/lib/enums/reasoning-effort.enums.ts @@ -4,9 +4,9 @@ */ export enum ReasoningEffort { DEFAULT = 'default', - OFF = 'off', - LOW = 'low', - MEDIUM = 'medium', HIGH = 'high', - MAX = 'max' + LOW = 'low', + MAX = 'max', + MEDIUM = 'medium', + OFF = 'off' } diff --git a/tools/ui/src/lib/enums/server.enums.ts b/tools/ui/src/lib/enums/server.enums.ts index 446af84be..b7e80433c 100644 --- a/tools/ui/src/lib/enums/server.enums.ts +++ b/tools/ui/src/lib/enums/server.enums.ts @@ -13,11 +13,11 @@ export enum ServerRole { * Used as the `value` field in the status object from /models endpoint */ export enum ServerModelStatus { - UNLOADED = 'unloaded', - LOADING = 'loading', + FAILED = 'failed', LOADED = 'loaded', + LOADING = 'loading', SLEEPING = 'sleeping', - FAILED = 'failed' + UNLOADED = 'unloaded' } /** @@ -26,10 +26,10 @@ export enum ServerModelStatus { * tools/server/server-models.cpp from the C++ server. */ export enum ServerModelsSseEventType { - STATUS_CHANGE = 'status_change', - MODEL_STATUS = 'model_status', - STATUS_UPDATE = 'status_update', - MODELS_RELOAD = 'models_reload', + DOWNLOAD_PROGRESS = 'download_progress', MODEL_REMOVE = 'model_remove', - DOWNLOAD_PROGRESS = 'download_progress' + MODEL_STATUS = 'model_status', + MODELS_RELOAD = 'models_reload', + STATUS_CHANGE = 'status_change', + STATUS_UPDATE = 'status_update' } diff --git a/tools/ui/src/lib/enums/settings.enums.ts b/tools/ui/src/lib/enums/settings.enums.ts index 6e0ebbd80..9911670b3 100644 --- a/tools/ui/src/lib/enums/settings.enums.ts +++ b/tools/ui/src/lib/enums/settings.enums.ts @@ -2,26 +2,26 @@ * Parameter source - indicates whether a parameter uses default or custom value */ export enum ParameterSource { - DEFAULT = 'default', - CUSTOM = 'custom' + CUSTOM = 'custom', + DEFAULT = 'default' } /** * Syncable parameter type - data types for parameters that can be synced with server */ export enum SyncableParameterType { + BOOLEAN = 'boolean', NUMBER = 'number', - STRING = 'string', - BOOLEAN = 'boolean' + STRING = 'string' } /** * Settings field type - defines the input type for settings fields */ export enum SettingsFieldType { - INPUT = 'input', - TEXTAREA = 'textarea', CHECKBOX = 'checkbox', + INPUT = 'input', + RADIO = 'radio', SELECT = 'select', - RADIO = 'radio' + TEXTAREA = 'textarea' } diff --git a/tools/ui/src/lib/enums/splash.enums.ts b/tools/ui/src/lib/enums/splash.enums.ts index 7efa89299..2967dfcea 100644 --- a/tools/ui/src/lib/enums/splash.enums.ts +++ b/tools/ui/src/lib/enums/splash.enums.ts @@ -2,6 +2,6 @@ * Splash screen orientation for iOS apple-touch-startup-image */ export enum SplashOrientation { - PORTRAIT = 'portrait', - LANDSCAPE = 'landscape' + LANDSCAPE = 'landscape', + PORTRAIT = 'portrait' } diff --git a/tools/ui/src/lib/enums/tools.enums.ts b/tools/ui/src/lib/enums/tools.enums.ts index 0c47be2cb..db55837a8 100644 --- a/tools/ui/src/lib/enums/tools.enums.ts +++ b/tools/ui/src/lib/enums/tools.enums.ts @@ -8,13 +8,13 @@ export enum ToolSource { export enum ToolPermissionDecision { ALWAYS = 'always', ALWAYS_SERVER = 'always_server', - ONCE = 'once', - DENY = 'deny' + DENY = 'deny', + ONCE = 'once' } export enum ToolResponseField { - PLAIN_TEXT = 'plain_text_response', - ERROR = 'error' + ERROR = 'error', + PLAIN_TEXT = 'plain_text_response' } /** @@ -22,9 +22,9 @@ export enum ToolResponseField { * Mirrors the server-side validation in server-tools.cpp. */ export enum GlobSearchType { - FILE = 'file', + ALL = 'all', DIR = 'dir', - ALL = 'all' + FILE = 'file' } /** diff --git a/tools/ui/src/lib/enums/ui.enums.ts b/tools/ui/src/lib/enums/ui.enums.ts index 5ed4c1edb..0de34ccfa 100644 --- a/tools/ui/src/lib/enums/ui.enums.ts +++ b/tools/ui/src/lib/enums/ui.enums.ts @@ -1,22 +1,22 @@ export enum ColorMode { - LIGHT = 'light', DARK = 'dark', + LIGHT = 'light', SYSTEM = 'system' } export enum TooltipSide { - TOP = 'top', - RIGHT = 'right', BOTTOM = 'bottom', - LEFT = 'left' + LEFT = 'left', + RIGHT = 'right', + TOP = 'top' } /** * MCP prompt display variant */ export enum McpPromptVariant { - MESSAGE = 'message', - ATTACHMENT = 'attachment' + ATTACHMENT = 'attachment', + MESSAGE = 'message' } /** @@ -39,8 +39,8 @@ export enum HtmlInputType { * Alert level that drives the context gauge dial color. */ export enum ColorLevel { - OK = 'ok', - WARNING = 'warning', CRITICAL = 'critical', - NEUTRAL = 'neutral' + NEUTRAL = 'neutral', + OK = 'ok', + WARNING = 'warning' } diff --git a/tools/ui/src/lib/utils/search-results.ts b/tools/ui/src/lib/utils/search-results.ts index db825c3eb..facf7766d 100644 --- a/tools/ui/src/lib/utils/search-results.ts +++ b/tools/ui/src/lib/utils/search-results.ts @@ -50,10 +50,10 @@ const FAVICON_PATH = '/favicon.ico'; // (and that callers read off `SearchResult`), so `FieldKey.TITLE` is a // drop-in for the literal `'title'`. enum FieldKey { - TITLE = 'title', - URL = 'url', + AUTHOR = 'author', PUBLISHED = 'published', - AUTHOR = 'author' + TITLE = 'title', + URL = 'url' } const FIELD_PREFIXES: ReadonlyArray<{ key: FieldKey; prefix: string }> = [ { key: FieldKey.TITLE, prefix: 'Title:' }, From 25ae3a9b331fffea50ff8d07a5cad34c33f1276f Mon Sep 17 00:00:00 2001 From: ynankani Date: Tue, 18 Aug 2026 04:15:53 +0000 Subject: [PATCH 05/48] CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark (#26843) * CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark Signed-off-by: ynankani * skip moe experts and allow others based on k geometry (allow only small idle tail) Signed-off-by: ynankani * rename MMVQ DGX Spark params to GB10 and fix MSVC constexpr lambda capture Signed-off-by: ynankani --------- Signed-off-by: ynankani --- ggml/src/ggml-cuda/mmvq.cu | 122 +++++++++++++++++++++++++++---------- 1 file changed, 91 insertions(+), 31 deletions(-) diff --git a/ggml/src/ggml-cuda/mmvq.cu b/ggml/src/ggml-cuda/mmvq.cu index 0589e65bd..c99923804 100644 --- a/ggml/src/ggml-cuda/mmvq.cu +++ b/ggml/src/ggml-cuda/mmvq.cu @@ -4,6 +4,7 @@ #include "vecdotq.cuh" #include +#include typedef float (*vec_dot_q_cuda_t)(const void * __restrict__ vbq, const block_q8_1 * __restrict__ bq8_1, const int & kbx, const int & iqs); @@ -69,7 +70,8 @@ enum mmvq_parameter_table_id { MMVQ_PARAMETERS_GCN, MMVQ_PARAMETERS_RDNA2, MMVQ_PARAMETERS_RDNA3_0, - MMVQ_PARAMETERS_RDNA4 + MMVQ_PARAMETERS_RDNA4, + MMVQ_PARAMETERS_GB10 }; static constexpr __device__ mmvq_parameter_table_id get_device_table_id() { @@ -83,6 +85,8 @@ static constexpr __device__ mmvq_parameter_table_id get_device_table_id() { return MMVQ_PARAMETERS_GCN; #elif defined(__CUDA_ARCH__) && __CUDA_ARCH__ >= GGML_CUDA_CC_TURING && __CUDA_ARCH__ < GGML_CUDA_CC_AMPERE return MMVQ_PARAMETERS_TURING; +#elif defined(__CUDA_ARCH__) && __CUDA_ARCH__ == GGML_CUDA_CC_DGX_SPARK + return MMVQ_PARAMETERS_GB10; #else return MMVQ_PARAMETERS_GENERIC; #endif @@ -104,6 +108,9 @@ static __host__ mmvq_parameter_table_id get_device_table_id(int cc) { if (GGML_CUDA_CC_IS_NVIDIA(cc) && ggml_cuda_highest_compiled_arch(cc) >= GGML_CUDA_CC_TURING && ggml_cuda_highest_compiled_arch(cc) < GGML_CUDA_CC_AMPERE) { return MMVQ_PARAMETERS_TURING; } + if (GGML_CUDA_CC_IS_NVIDIA(cc) && ggml_cuda_highest_compiled_arch(cc) == GGML_CUDA_CC_DGX_SPARK) { + return MMVQ_PARAMETERS_GB10; + } return MMVQ_PARAMETERS_GENERIC; } @@ -351,7 +358,7 @@ static constexpr __device__ int get_mmvq_mmid_max_batch_for_device() { #endif } -static constexpr __host__ __device__ int calc_nwarps(ggml_type type, int ncols_dst, mmvq_parameter_table_id table_id) { +static constexpr __host__ __device__ int calc_nwarps(ggml_type type, int ncols_dst, mmvq_parameter_table_id table_id, bool small_k = false, bool halve_iters = false) { if (table_id == MMVQ_PARAMETERS_GENERIC) { switch (ncols_dst) { case 1: @@ -454,11 +461,32 @@ static constexpr __host__ __device__ int calc_nwarps(ggml_type type, int ncols_d return 1; } } + if (table_id == MMVQ_PARAMETERS_GB10) { + const int generic = calc_nwarps(type, ncols_dst, MMVQ_PARAMETERS_GENERIC); + // Only worth the wider block when it actually retires the K loop in half the trips (Observation) + if (ncols_dst == 1 && !small_k && halve_iters) { + switch (type) { + case GGML_TYPE_Q4_0: + case GGML_TYPE_Q4_1: + case GGML_TYPE_Q5_0: + case GGML_TYPE_Q5_1: + case GGML_TYPE_Q8_0: + case GGML_TYPE_Q4_K: + case GGML_TYPE_Q5_K: + case GGML_TYPE_Q6_K: + case GGML_TYPE_IQ4_NL: + return 2 * generic; + default: + break; + } + } + return generic; + } return 1; } static constexpr __host__ __device__ int calc_rows_per_block(int ncols_dst, int table_id, bool small_k = false, int nwarps = 1) { - if (table_id == MMVQ_PARAMETERS_GENERIC || table_id == MMVQ_PARAMETERS_GCN || table_id == MMVQ_PARAMETERS_TURING) { + if (table_id == MMVQ_PARAMETERS_GENERIC || table_id == MMVQ_PARAMETERS_GCN || table_id == MMVQ_PARAMETERS_TURING || table_id == MMVQ_PARAMETERS_GB10) { switch (ncols_dst) { case 1: return small_k ? nwarps : 1; @@ -477,8 +505,8 @@ static constexpr __host__ __device__ int calc_rows_per_block(int ncols_dst, int return 1; } -template -__launch_bounds__(calc_nwarps(type, ncols_dst, get_device_table_id())*ggml_cuda_get_physical_warp_size(), 1) +template +__launch_bounds__(calc_nwarps(type, ncols_dst, get_device_table_id(), small_k, halve_iters)*ggml_cuda_get_physical_warp_size(), 1) static __global__ void mul_mat_vec_q( const void * vx_ptr, const void * vy_ptr, const int32_t * ids_ptr, const ggml_cuda_mm_fusion_args_device fusion, float * dst_ptr, const uint32_t ncols_x, const uint3 nchannels_y, const uint32_t stride_row_x, const uint32_t stride_col_y, @@ -495,7 +523,7 @@ static __global__ void mul_mat_vec_q( constexpr int qi = ggml_cuda_type_traits::qi; constexpr int vdr = get_vdr_mmvq(type); constexpr mmvq_parameter_table_id table_id = get_device_table_id(); - constexpr int nwarps = calc_nwarps(type, ncols_dst, table_id); + constexpr int nwarps = calc_nwarps(type, ncols_dst, table_id, small_k, halve_iters); constexpr int rows_per_cuda_block = calc_rows_per_block(ncols_dst, table_id, small_k, nwarps); constexpr int warp_size = ggml_cuda_get_physical_warp_size(); @@ -773,8 +801,8 @@ static __global__ void mul_mat_vec_q_moe( template static std::pair calc_launch_params( const int ncols_dst, const int nrows_x, const int nchannels_dst, const int nsamples_or_ntokens, - const int warp_size, const mmvq_parameter_table_id table_id, const bool small_k = false) { - const int nwarps = calc_nwarps(type, ncols_dst, table_id); + const int warp_size, const mmvq_parameter_table_id table_id, const bool small_k = false, const bool halve_iters = false) { + const int nwarps = calc_nwarps(type, ncols_dst, table_id, small_k, halve_iters); const int rpb = calc_rows_per_block(ncols_dst, table_id, small_k, nwarps); const int64_t nblocks = (nrows_x + rpb - 1) / rpb; const dim3 block_nums(nblocks, nchannels_dst, nsamples_or_ntokens); @@ -782,7 +810,7 @@ static std::pair calc_launch_params( return {block_nums, block_dims}; } -template +template static void mul_mat_vec_q_switch_fusion( const void * vx, const void * vy, const int32_t * ids, const ggml_cuda_mm_fusion_args_device fusion, float * dst, const uint32_t ncols_x, const uint3 nchannels_y, const uint32_t stride_row_x, const uint32_t stride_col_y, @@ -797,7 +825,7 @@ static void mul_mat_vec_q_switch_fusion( if constexpr (c_ncols_dst == 1) { if (has_fusion) { const ggml_cuda_kernel_launch_params launch_params = ggml_cuda_kernel_launch_params(block_nums, block_dims, nbytes_shared, stream); - ggml_cuda_kernel_launch(mul_mat_vec_q, launch_params, + ggml_cuda_kernel_launch(mul_mat_vec_q, launch_params, vx, vy, ids, fusion, dst, ncols_x, nchannels_y, stride_row_x, stride_col_y, stride_col_dst, channel_ratio, stride_channel_x, stride_channel_y, stride_channel_dst, sample_ratio, stride_sample_x, stride_sample_y, stride_sample_dst, ids_stride); @@ -808,7 +836,7 @@ static void mul_mat_vec_q_switch_fusion( GGML_ASSERT(!has_fusion && "fusion only supported for ncols_dst=1"); const ggml_cuda_kernel_launch_params launch_params = ggml_cuda_kernel_launch_params(block_nums, block_dims, nbytes_shared, stream); - ggml_cuda_kernel_launch(mul_mat_vec_q, launch_params, + ggml_cuda_kernel_launch(mul_mat_vec_q, launch_params, vx, vy, ids, fusion, dst, ncols_x, nchannels_y, stride_row_x, stride_col_y, stride_col_dst, channel_ratio, stride_channel_x, stride_channel_y, stride_channel_dst, sample_ratio, stride_sample_x, stride_sample_y, stride_sample_dst, ids_stride); @@ -860,16 +888,18 @@ static void mul_mat_vec_q_switch_ncols_dst( const bool has_ids = ids != nullptr; + // How the K loop divides up at the baseline block width, both decisions below use these. + constexpr int qk = ggml_cuda_type_traits::qk; + constexpr int qi = ggml_cuda_type_traits::qi; + constexpr int vdr = get_vdr_mmvq(type); + const int blocks_per_row_x = ncols_x / qk; + const int blocks_per_iter_1warp = vdr * warp_size / qi; + const auto should_use_small_k = [&](int c_ncols_dst) { // When K is small, increase rows_per_block to match nwarps so each warp has more work to do // Trigger when the full thread block covers all K blocks in a single loop iteration and few threads remain idle. - constexpr int qk = ggml_cuda_type_traits::qk; - constexpr int qi = ggml_cuda_type_traits::qi; - constexpr int vdr = get_vdr_mmvq(type); - const int blocks_per_row_x = ncols_x / qk; - const int blocks_per_iter_1warp = vdr * warp_size / qi; - const int nwarps = calc_nwarps(type, c_ncols_dst, table_id); - bool use = nwarps > 1 && blocks_per_row_x < nwarps * blocks_per_iter_1warp; + const int nwarps = calc_nwarps(type, c_ncols_dst, table_id); + bool use = nwarps > 1 && blocks_per_row_x < nwarps * blocks_per_iter_1warp; constexpr std::array iq_slow_turing = { GGML_TYPE_IQ3_XXS, @@ -902,6 +932,28 @@ static void mul_mat_vec_q_switch_ncols_dst( return use; }; + // Whether doubling nwarps pays off on the ncols_dst == 1 path, where K sets the K loop trip count. + const auto should_halve_iters = [&] { + if (table_id != MMVQ_PARAMETERS_GB10) { + return false; + } + + // Expert rows are gathered per token, so a wider block adds reduction work without reuse. + if (has_ids) { + return false; + } + + const int blocks_per_iter = calc_nwarps(type, 1, table_id) * blocks_per_iter_1warp; + const int iters = (blocks_per_row_x + blocks_per_iter - 1) / blocks_per_iter; + const int iters_wide = (blocks_per_row_x + blocks_per_iter * 2 - 1) / (blocks_per_iter * 2); + + // An odd trip count leaves half the wider block idle for its last iteration, that tail is + // only affordable once the loop is long enough to dilute it to an eighth of the work (observation). + const int idle = iters_wide * 2 - iters; + + return idle * 8 <= iters_wide * 2; + }; + if (has_ids && ncols_dst > 1) { // Multi-token MUL_MAT_ID path - dedicated MoE kernel mul_mat_vec_q_moe_launch( @@ -914,26 +966,34 @@ static void mul_mat_vec_q_switch_ncols_dst( switch (ncols_dst) { case 1: { - constexpr int c_ncols_dst = 1; + // static, else MSVC lambda capture breaks the constexpr uses below + static constexpr int c_ncols_dst = 1; - bool use_small_k = should_use_small_k(c_ncols_dst); + // Tag types keep the flags compile-time, so __launch_bounds__ matches what is launched. + const auto launch = [&](auto small_k_tag, auto halve_iters_tag) { + constexpr bool c_small_k = decltype(small_k_tag)::value; + // Types the table does not promote would compile a second, identical kernel. + constexpr bool c_promoted = + calc_nwarps(type, c_ncols_dst, MMVQ_PARAMETERS_GB10, false, true) != + calc_nwarps(type, c_ncols_dst, MMVQ_PARAMETERS_GB10, false, false); - if (use_small_k) { - std::pair dims = calc_launch_params(c_ncols_dst, nrows_x, nchannels_dst, - nsamples_dst, warp_size, table_id, true); - mul_mat_vec_q_switch_fusion( + constexpr bool c_halve_iters = decltype(halve_iters_tag)::value && c_promoted; + + const std::pair dims = calc_launch_params(c_ncols_dst, nrows_x, nchannels_dst, + nsamples_dst, warp_size, table_id, c_small_k, c_halve_iters); + mul_mat_vec_q_switch_fusion( vx, vy, ids, fusion, dst, ncols_x, nchannels_y_fd, stride_row_x, stride_col_y, stride_col_dst, channel_ratio_fd, stride_channel_x, stride_channel_y, stride_channel_dst, sample_ratio_fd, stride_sample_x, stride_sample_y, stride_sample_dst, dims.first, dims.second, 0, ids_stride, stream); + }; + + if (should_use_small_k(c_ncols_dst)) { + launch(std::true_type{}, std::false_type{}); + } else if (should_halve_iters()) { + launch(std::false_type{}, std::true_type{}); } else { - std::pair dims = calc_launch_params(c_ncols_dst, nrows_x, nchannels_dst, - nsamples_dst, warp_size, table_id); - mul_mat_vec_q_switch_fusion( - vx, vy, ids, fusion, dst, ncols_x, nchannels_y_fd, stride_row_x, stride_col_y, stride_col_dst, - channel_ratio_fd, stride_channel_x, stride_channel_y, stride_channel_dst, sample_ratio_fd, - stride_sample_x, stride_sample_y, stride_sample_dst, dims.first, dims.second, 0, ids_stride, - stream); + launch(std::false_type{}, std::false_type{}); } } break; case 2: { From 8b86400975fbbe29fca7f196ee208ab246a7c29f Mon Sep 17 00:00:00 2001 From: Georgi Gerganov Date: Tue, 18 Aug 2026 10:14:41 +0300 Subject: [PATCH 06/48] ci : create pre-release with change log and nightly link in make-release (#27302) * ci : create pre-release with change log and nightly link in make-release After pushing the tag, create a pre-release using ggml-org/action-create-release. The release description is generated by scripts/make-release-desc.sh: the change log between the current and previous version (one line per commit), a link to the corresponding nightly build when it exists, and a note that semantic versioning is still work in progress. Assisted-by: pi:llama.cpp/Qwen3.8-27B * cmake : bump version to 0.1.2 Assisted-by: pi:llama.cpp/Qwen3.8-27B * ci : find the nightly tag by commit in make-release-desc.sh The nightly release is guaranteed by the release checks to point at HEAD, so instead of reconstructing its name (commit count, branch, hash) just pick the b* tag pointing at HEAD. This also drops the RELEASE_BRANCH env var from the workflow. Assisted-by: pi:llama.cpp/Qwen3.8-27B * ci : resolve the release commit from the version tag in make-release-desc.sh The change log and nightly lookup now use the commit the version tag points at (HEAD when the tag does not exist), instead of always HEAD. This makes the script usable locally for older versions, e.g. ./scripts/make-release-desc.sh v0.1.1. The tag is resolved to a SHA first, since --points-at does not peel annotated tags. Assisted-by: pi:llama.cpp/Qwen3.8-27B * ci : normalize the version argument in make-release-desc.sh Accept the version with or without the leading v (0.1.1 == v0.1.1) and reject anything else, instead of silently treating a bare version as a non-existent tag name. Assisted-by: pi:llama.cpp/Qwen3.8-27B * cont : clean-up --- .github/workflows/make-release.yml | 27 ++++++++++ CMakeLists.txt | 2 +- scripts/make-release-desc.sh | 87 ++++++++++++++++++++++++++++++ 3 files changed, 115 insertions(+), 1 deletion(-) create mode 100755 scripts/make-release-desc.sh diff --git a/.github/workflows/make-release.yml b/.github/workflows/make-release.yml index 9bf289a8c..78d7caa7a 100644 --- a/.github/workflows/make-release.yml +++ b/.github/workflows/make-release.yml @@ -49,6 +49,33 @@ jobs: git push origin "${VERSION}" echo "Created and pushed tag ${VERSION}" + - name: Generate release description + id: desc + run: bash scripts/make-release-desc.sh "${{ steps.checks.outputs.version }}" + env: + GITHUB_REPOSITORY: ${{ github.repository }} + + - name: Create release + if: ${{ github.event.inputs.dry_run == 'false' }} + uses: ggml-org/action-create-release@v1 + env: + GITHUB_TOKEN: ${{ github.token }} + with: + tag_name: ${{ steps.checks.outputs.version }} + # TODO: remove the prerelease flag once the semantic versioning workflow is ready + # ref: https://github.com/ggml-org/ggml/discussions/1579 + prerelease: true + body: | + > [!NOTE] + > Semantic versioning is still work in progress. + > More info can be found in https://github.com/ggml-org/ggml/discussions/1579 + + ${{ steps.desc.outputs.nightly }} + + ## ${{ steps.desc.outputs.changelog_title }} + + ${{ steps.desc.outputs.changelog }} + - name: Dry run summary if: ${{ github.event.inputs.dry_run == 'true' }} run: | diff --git a/CMakeLists.txt b/CMakeLists.txt index 19f14e0d1..5c443c0dd 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -5,7 +5,7 @@ include(CheckIncludeFileCXX) ### llama.cpp version set(LLAMA_VERSION_MAJOR 0) set(LLAMA_VERSION_MINOR 1) -set(LLAMA_VERSION_PATCH 1) +set(LLAMA_VERSION_PATCH 2) set(LLAMA_VERSION_BASE "${LLAMA_VERSION_MAJOR}.${LLAMA_VERSION_MINOR}.${LLAMA_VERSION_PATCH}") # whether this is a development/nightly build diff --git a/scripts/make-release-desc.sh b/scripts/make-release-desc.sh new file mode 100755 index 000000000..100f855b3 --- /dev/null +++ b/scripts/make-release-desc.sh @@ -0,0 +1,87 @@ +#!/bin/bash +# Generate the description of a release: the previous release version, the +# change log and the link to the nightly release corresponding to the commit being released. +# +# Usage: make-release-desc.sh +# : current release version (v.., the leading v is optional) +# +# The previous version is the highest plain semver tag (v..) +# strictly below . The change log lists all commits between the +# previous version tag and the release commit, one line per commit. +# +# The release commit is the commit points at when the tag exists, +# HEAD otherwise. The nightly release is the b* tag pointing at that commit +# (release.yml tags the same commit); the link is only generated when that +# tag exists. +# +# Env (when running in GitHub Actions): +# GITHUB_OUTPUT: previous_tag, changelog_title, changelog and nightly are written here +# GITHUB_REPOSITORY: owner/repo, used to build the nightly release URL (skipped when unset) +set -euo pipefail + +if [[ $# -ne 1 ]]; then + echo "Usage: $(basename "$0") " + exit 1 +fi +VERSION="$1" + +# Accept the version with or without the leading v, reject anything else +if [[ "${VERSION}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + VERSION="v${VERSION}" +elif [[ ! "${VERSION}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "Error: invalid version '${VERSION}' (expected v..)" + exit 1 +fi + +# Make sure all remote tags are available locally (skipped on local runs without origin) +if ! git fetch --tags origin 2>/dev/null; then + echo "Warning: could not fetch tags from origin (local run?)" +fi + +# Release commit: the commit points at when the tag exists, HEAD otherwise. +if ! RELEASE_COMMIT="$(git rev-parse -q --verify "refs/tags/${VERSION}^{commit}" 2>/dev/null)"; then + RELEASE_COMMIT="$(git rev-parse HEAD)" +fi + +echo "Release commit: $(git rev-parse --short "${RELEASE_COMMIT}")" + +PREV="$( { git tag --list; echo "${VERSION}"; } \ + | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' \ + | sort -V \ + | awk -v cur="${VERSION}" '$0 == cur { exit } { prev = $0 } END { print prev }')" + +if [[ -n "${PREV}" ]]; then + CHANGELOG="$(git log --oneline "${PREV}..${RELEASE_COMMIT}")" + CHANGELOG_TITLE="Change log since ${PREV}" +else + CHANGELOG="(no previous release tag found)" + CHANGELOG_TITLE="Change log" +fi + +# Nightly release: the b* tag pointing at the release commit (|| true: no match is not an error) +NIGHTLY_TAG="$(git tag --points-at "${RELEASE_COMMIT}" | grep -E '(^|-)b[0-9]+(-[0-9a-f]{7})?$' | head -n 1 || true)" + +NIGHTLY="" +if [[ -n "${NIGHTLY_TAG}" ]]; then + if [[ -n "${GITHUB_REPOSITORY:-}" ]]; then + NIGHTLY_URL="https://github.com/${GITHUB_REPOSITORY}/releases/tag/${NIGHTLY_TAG}" + NIGHTLY="**Nightly build:** [${NIGHTLY_TAG}](${NIGHTLY_URL})" + echo "Nightly release: ${NIGHTLY_URL}" + fi +else + echo "No nightly release found for commit $(git rev-parse --short "${RELEASE_COMMIT}")" +fi + +echo "Previous version: ${PREV:-none}" +echo "${CHANGELOG}" + +if [[ -n "${GITHUB_OUTPUT:-}" ]]; then + { + echo "previous_tag=${PREV}" + echo "changelog_title=${CHANGELOG_TITLE}" + echo "nightly=${NIGHTLY}" + echo "changelog<> "${GITHUB_OUTPUT}" +fi From 27e345b574dd8c8838e2c06e47699a3135f16ec9 Mon Sep 17 00:00:00 2001 From: Georgi Gerganov Date: Tue, 18 Aug 2026 11:16:51 +0300 Subject: [PATCH 07/48] build : fix xcframework + cmake clean-up (#27304) * xcframework : fix build * mtmd : remove unused include path * vendor : use vendor::hash alias target in cmake CMake reserves "::" in target names for imported/alias targets, so the real target keeps the name vendor-hash and a vendor::hash ALIAS target is added. Consumers (mtmd, llama-gguf-hash) now link against the namespaced alias. Assisted-by: pi:llama.cpp/Qwen3.8-27B * vendor : add cmake targets for all vendored libs with vendor:: aliases Add INTERFACE targets for the header-only vendor libs (miniaudio, nlohmann, sheredom, stb) and ALIAS targets named vendor:: for all of them, including cpp-httplib and hash. Each exposes the vendor/ root so includes are namespaced, e.g. . Consolidate the per-lib add_subdirectory calls into a single add_subdirectory(vendor), keeping the cpp-httplib gate on LLAMA_BUILD_COMMON. Consumers (llama-common, mtmd) now link the aliases instead of relying on raw vendor/ include paths. hash: consumers now include via "hash/hash.h"; the vendor/hash dir is kept as a PRIVATE include so the synced upstream sources compile unmodified. Assisted-by: pi:llama.cpp/Qwen3.8-27B * readme : use foo/bar names in acknowledgements Assisted-by: pi:llama.cpp/Qwen3.8-27B * ocd : fix valign --- CMakeLists.txt | 4 +--- README.md | 6 +++--- build-xcframework.sh | 1 + common/CMakeLists.txt | 3 ++- examples/gguf-hash/CMakeLists.txt | 2 +- examples/gguf-hash/gguf-hash.cpp | 6 +++--- tools/mtmd/CMakeLists.txt | 4 +--- tools/mtmd/mtmd-helper.cpp | 2 +- vendor/CMakeLists.txt | 11 +++++++++++ vendor/cpp-httplib/CMakeLists.txt | 2 ++ vendor/hash/CMakeLists.txt | 9 +++++++-- vendor/miniaudio/CMakeLists.txt | 6 ++++++ vendor/nlohmann/CMakeLists.txt | 6 ++++++ vendor/sheredom/CMakeLists.txt | 6 ++++++ vendor/stb/CMakeLists.txt | 6 ++++++ 15 files changed, 57 insertions(+), 17 deletions(-) create mode 100644 vendor/CMakeLists.txt create mode 100644 vendor/miniaudio/CMakeLists.txt create mode 100644 vendor/nlohmann/CMakeLists.txt create mode 100644 vendor/sheredom/CMakeLists.txt create mode 100644 vendor/stb/CMakeLists.txt diff --git a/CMakeLists.txt b/CMakeLists.txt index 5c443c0dd..8da08ac5a 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -224,12 +224,10 @@ add_subdirectory(src) # utils, programs, examples and tests # -# mtmd needs this even when common is not built -add_subdirectory(vendor/hash) +add_subdirectory(vendor) if (LLAMA_BUILD_COMMON) add_subdirectory(common) - add_subdirectory(vendor/cpp-httplib) endif() if (LLAMA_BUILD_COMMON AND LLAMA_BUILD_TESTS AND NOT CMAKE_JS_VERSION) diff --git a/README.md b/README.md index 1b341e7fb..5960ef684 100644 --- a/README.md +++ b/README.md @@ -119,7 +119,7 @@ The `llama.cpp` project is build on top of the [ggml](https://github.com/ggml-or ## Acknowledgements - [yhirose/cpp-httplib](https://github.com/yhirose/cpp-httplib) - Single-header HTTP server, used by `llama-server` - MIT license -- [stb-image](https://github.com/nothings/stb) - Single-header image format decoder, used by multimodal subsystem - Public domain +- [nothings/stb](https://github.com/nothings/stb) - Single-header image format decoder, used by multimodal subsystem - Public domain - [nlohmann/json](https://github.com/nlohmann/json) - Single-header JSON library, used by various tools/examples - MIT License -- [miniaudio.h](https://github.com/mackron/miniaudio) - Single-header audio format decoder, used by multimodal subsystem - Public domain -- [subprocess.h](https://github.com/sheredom/subprocess.h) - Single-header process launching solution for C and C++ - Public domain +- [mackron/miniaudio](https://github.com/mackron/miniaudio) - Single-header audio format decoder, used by multimodal subsystem - Public domain +- [sheredom/subprocess.h](https://github.com/sheredom/subprocess.h) - Single-header process launching solution for C and C++ - Public domain diff --git a/build-xcframework.sh b/build-xcframework.sh index e8b7247f4..e405a1c0f 100755 --- a/build-xcframework.sh +++ b/build-xcframework.sh @@ -290,6 +290,7 @@ combine_static_libraries() { "${base_dir}/${build_dir}/ggml/src/ggml-metal/${release_dir}/libggml-metal.a" "${base_dir}/${build_dir}/ggml/src/ggml-blas/${release_dir}/libggml-blas.a" "${base_dir}/${build_dir}/tools/mtmd/${release_dir}/libmtmd.a" + "${base_dir}/${build_dir}/vendor/hash/${release_dir}/libvendor-hash.a" ) # Create temporary directory for processing diff --git a/common/CMakeLists.txt b/common/CMakeLists.txt index d6cfc9a00..54691da3f 100644 --- a/common/CMakeLists.txt +++ b/common/CMakeLists.txt @@ -126,7 +126,8 @@ set_target_properties(${TARGET} PROPERTIES MACHO_CURRENT_VERSION 0 # keep macOS linker from seeing oversized version number ) -target_include_directories(${TARGET} PUBLIC . ../vendor) +target_include_directories(${TARGET} PUBLIC .) +target_link_libraries (${TARGET} PUBLIC vendor::nlohmann vendor::sheredom) target_compile_features (${TARGET} PUBLIC cxx_std_17) if (LLAMA_SUBPROCESS) diff --git a/examples/gguf-hash/CMakeLists.txt b/examples/gguf-hash/CMakeLists.txt index 2542074fb..f0fb8232a 100644 --- a/examples/gguf-hash/CMakeLists.txt +++ b/examples/gguf-hash/CMakeLists.txt @@ -2,5 +2,5 @@ set(TARGET llama-gguf-hash) add_executable(${TARGET} gguf-hash.cpp) install(TARGETS ${TARGET} RUNTIME) -target_link_libraries(${TARGET} PRIVATE vendor-hash ggml ${CMAKE_THREAD_LIBS_INIT}) +target_link_libraries(${TARGET} PRIVATE vendor::hash ggml ${CMAKE_THREAD_LIBS_INIT}) target_compile_features(${TARGET} PRIVATE cxx_std_17) diff --git a/examples/gguf-hash/gguf-hash.cpp b/examples/gguf-hash/gguf-hash.cpp index 43de6300d..317a5e342 100644 --- a/examples/gguf-hash/gguf-hash.cpp +++ b/examples/gguf-hash/gguf-hash.cpp @@ -17,15 +17,15 @@ extern "C" { #endif -#include "xxhash/xxhash.h" -#include "sha256/sha256.h" +#include "hash/xxhash/xxhash.h" +#include "hash/sha256/sha256.h" #ifdef __cplusplus } #endif // sha1 is compiled as C++ and lives in a namespace, see scripts/sync_vendor.py -#include "sha1/sha1.h" +#include "hash/sha1/sha1.h" using namespace vendor_hash; diff --git a/tools/mtmd/CMakeLists.txt b/tools/mtmd/CMakeLists.txt index db758395f..95aa853ea 100644 --- a/tools/mtmd/CMakeLists.txt +++ b/tools/mtmd/CMakeLists.txt @@ -78,10 +78,8 @@ set_target_properties(mtmd PROPERTIES ) target_link_libraries (mtmd PUBLIC ggml llama) -target_link_libraries (mtmd PRIVATE Threads::Threads vendor-hash) +target_link_libraries (mtmd PRIVATE Threads::Threads vendor::hash vendor::miniaudio vendor::stb vendor::sheredom) target_include_directories(mtmd PUBLIC .) -target_include_directories(mtmd PRIVATE ../..) -target_include_directories(mtmd PRIVATE ../../vendor) target_compile_features (mtmd PRIVATE cxx_std_17) if (MTMD_VIDEO) diff --git a/tools/mtmd/mtmd-helper.cpp b/tools/mtmd/mtmd-helper.cpp index bce8e38cc..cc966b93c 100644 --- a/tools/mtmd/mtmd-helper.cpp +++ b/tools/mtmd/mtmd-helper.cpp @@ -12,7 +12,7 @@ #include "mtmd-helper-common.h" #include "llama.h" -#include "hash.h" +#include "hash/hash.h" #include #include diff --git a/vendor/CMakeLists.txt b/vendor/CMakeLists.txt new file mode 100644 index 000000000..4479dafca --- /dev/null +++ b/vendor/CMakeLists.txt @@ -0,0 +1,11 @@ +# mtmd needs these even when common is not built +add_subdirectory(hash) +add_subdirectory(miniaudio) +add_subdirectory(nlohmann) +add_subdirectory(sheredom) +add_subdirectory(stb) + +# only used by common +if (LLAMA_BUILD_COMMON) + add_subdirectory(cpp-httplib) +endif() diff --git a/vendor/cpp-httplib/CMakeLists.txt b/vendor/cpp-httplib/CMakeLists.txt index 6a6eefed1..30ae8b47e 100644 --- a/vendor/cpp-httplib/CMakeLists.txt +++ b/vendor/cpp-httplib/CMakeLists.txt @@ -9,6 +9,8 @@ set(CMAKE_POSITION_INDEPENDENT_CODE ON) add_library(${TARGET} STATIC httplib.cpp httplib.h) +add_library(vendor::cpp-httplib ALIAS ${TARGET}) + # disable warnings in 3rd party code if (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC") target_compile_options(${TARGET} PRIVATE /w) diff --git a/vendor/hash/CMakeLists.txt b/vendor/hash/CMakeLists.txt index efdf58e63..1654eb185 100644 --- a/vendor/hash/CMakeLists.txt +++ b/vendor/hash/CMakeLists.txt @@ -16,6 +16,8 @@ add_library(${TARGET} STATIC ${VENDOR_SRCS} ) +add_library(vendor::hash ALIAS ${TARGET}) + target_compile_features(${TARGET} PRIVATE cxx_std_17) # disable warnings in 3rd party code, but keep them for hash.cpp @@ -29,5 +31,8 @@ set_source_files_properties(${VENDOR_SRCS} PROPERTIES COMPILE_OPTIONS ${NO_WARN_ # sha1 lives in a namespace to avoid a clash with boringssl, see scripts/sync_vendor.py set_source_files_properties(sha1/sha1.c PROPERTIES LANGUAGE CXX) -# sha256.c includes "rotate-bits/rotate-bits.h", so consumers get this dir too -target_include_directories(${TARGET} PUBLIC .) +# expose the vendor/ root so consumers can include via "hash/hash.h" +target_include_directories(${TARGET} PUBLIC ..) + +# internal includes of the vendored sources, e.g. sha256.c -> "rotate-bits/rotate-bits.h" +target_include_directories(${TARGET} PRIVATE .) diff --git a/vendor/miniaudio/CMakeLists.txt b/vendor/miniaudio/CMakeLists.txt new file mode 100644 index 000000000..8c706b62f --- /dev/null +++ b/vendor/miniaudio/CMakeLists.txt @@ -0,0 +1,6 @@ +# header-only: interface target exposing the vendor/ root so consumers +# can include via +add_library(miniaudio INTERFACE) +add_library(vendor::miniaudio ALIAS miniaudio) + +target_include_directories(miniaudio INTERFACE ..) diff --git a/vendor/nlohmann/CMakeLists.txt b/vendor/nlohmann/CMakeLists.txt new file mode 100644 index 000000000..630b3748a --- /dev/null +++ b/vendor/nlohmann/CMakeLists.txt @@ -0,0 +1,6 @@ +# header-only: interface target exposing the vendor/ root so consumers +# can include via +add_library(nlohmann INTERFACE) +add_library(vendor::nlohmann ALIAS nlohmann) + +target_include_directories(nlohmann INTERFACE ..) diff --git a/vendor/sheredom/CMakeLists.txt b/vendor/sheredom/CMakeLists.txt new file mode 100644 index 000000000..f0c148500 --- /dev/null +++ b/vendor/sheredom/CMakeLists.txt @@ -0,0 +1,6 @@ +# header-only: interface target exposing the vendor/ root so consumers +# can include via +add_library(sheredom INTERFACE) +add_library(vendor::sheredom ALIAS sheredom) + +target_include_directories(sheredom INTERFACE ..) diff --git a/vendor/stb/CMakeLists.txt b/vendor/stb/CMakeLists.txt new file mode 100644 index 000000000..14ea2f9e0 --- /dev/null +++ b/vendor/stb/CMakeLists.txt @@ -0,0 +1,6 @@ +# header-only: interface target exposing the vendor/ root so consumers +# can include via +add_library(stb INTERFACE) +add_library(vendor::stb ALIAS stb) + +target_include_directories(stb INTERFACE ..) From da786dc23e2bc060856c015dadc53400ff29ecfd Mon Sep 17 00:00:00 2001 From: Georgi Gerganov Date: Tue, 18 Aug 2026 11:28:01 +0300 Subject: [PATCH 08/48] ggml : bump version to 0.20.2 (ggml/1589) --- ggml/CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ggml/CMakeLists.txt b/ggml/CMakeLists.txt index b4d627320..b7110fa12 100644 --- a/ggml/CMakeLists.txt +++ b/ggml/CMakeLists.txt @@ -5,7 +5,7 @@ project("ggml" C CXX ASM) ### GGML Version set(GGML_VERSION_MAJOR 0) set(GGML_VERSION_MINOR 20) -set(GGML_VERSION_PATCH 1) +set(GGML_VERSION_PATCH 2) set(GGML_VERSION_BASE "${GGML_VERSION_MAJOR}.${GGML_VERSION_MINOR}.${GGML_VERSION_PATCH}") list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake/") From 1511ce3bc3f087376c8526b4ad07100bfabb277f Mon Sep 17 00:00:00 2001 From: Georgi Gerganov Date: Tue, 18 Aug 2026 11:29:14 +0300 Subject: [PATCH 09/48] sync : ggml --- scripts/sync-ggml.last | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/sync-ggml.last b/scripts/sync-ggml.last index c001bae1e..8a978f87b 100644 --- a/scripts/sync-ggml.last +++ b/scripts/sync-ggml.last @@ -1 +1 @@ -3834fd814e74e8af277939dabd69ecc780affd21 +8c63e70982c95ceb862e3a1073a2c1beef75d60a From 7acdbb1f191d869bad8c5da9d4a2121defa340af Mon Sep 17 00:00:00 2001 From: BlackFoil <127078112+BlackFoil@users.noreply.github.com> Date: Tue, 18 Aug 2026 18:11:19 +0900 Subject: [PATCH 10/48] mtmd: fix LFM2 image tiling threshold (#27057) * mtmd: fix LFM2 image tiling threshold * refactor testing * fix * fix on windows --------- Co-authored-by: Xuan Son Nguyen --- tests/CMakeLists.txt | 3 ++ tests/test-mtmd-impl.cpp | 88 +++++++++++++++++++++++++++++++++++++++ tools/mtmd/CMakeLists.txt | 3 ++ tools/mtmd/clip-impl.h | 4 ++ tools/mtmd/mtmd-image.cpp | 21 +++++++++- tools/mtmd/mtmd-image.h | 2 + 6 files changed, 119 insertions(+), 2 deletions(-) create mode 100644 tests/test-mtmd-impl.cpp diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 08c6f5a47..3ee51b519 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -310,6 +310,9 @@ llama_build_and_test(test-mtmd-c-api.c) target_link_libraries(${LLAMA_TEST_NAME} PRIVATE mtmd) unset(LLAMA_TEST_NAME) +llama_build_and_test(test-mtmd-impl.cpp) +target_link_libraries(test-mtmd-impl PRIVATE mtmd) + # GGUF model data fetcher library for tests that need real model metadata # Only compile when cpp-httplib has SSL support (CPPHTTPLIB_OPENSSL_SUPPORT) if (TARGET cpp-httplib) diff --git a/tests/test-mtmd-impl.cpp b/tests/test-mtmd-impl.cpp new file mode 100644 index 000000000..41cb53237 --- /dev/null +++ b/tests/test-mtmd-impl.cpp @@ -0,0 +1,88 @@ +#include "testing.h" + +#include "mtmd-image.h" + +#include +#include +#include +#include + +// this test file contains: +// 1. test cases for mtmd helpers +// 2. test cases for internal mtmd components +// internal headers can be included here + +struct test_registry { + using fn_t = void (*)(testing &); + + struct entry { + std::string name; + fn_t fn; + }; + + static std::vector & all() { + static std::vector entries; + return entries; + } + + test_registry(const char * name, fn_t fn) { + all().push_back({ name, fn }); + } +}; + +#define MAKE_TEST(name) \ + static void name(testing & t); \ + static const test_registry test_registry_ ## name(#name, &name); \ + static void name(testing & t) + + +// +// mtmd_image +// + +MAKE_TEST(test_image_preprocessor_lfm2) { + clip_hparams hparams; + hparams.patch_size = 16; + hparams.n_merge = 2; + hparams.set_limit_image_tokens(64, 256); + + // { image size, expected tiling } + const std::vector> cases = { + { { 704, 704 }, false }, + // 720 / (patch_size * n_merge) is exactly 22.5, so this only matches HF + // if round_by_factor rounds half to even (22) instead of away from zero (23) + { { 720, 720 }, false }, + { { 736, 736 }, true }, + { { 1024, 977 }, true }, + { { 1056, 384 }, false }, + }; + + for (const auto & [size, expected] : cases) { + const bool actual = mtmd_image_preprocessor_lfm2::should_tile(hparams, size); + + t.assert_equal( + "tiling for " + std::to_string(size.width) + "x" + std::to_string(size.height), + std::string(expected ? "tiled" : "single"), + std::string(actual ? "tiled" : "single")); + } +} + +// +// main +// + +int main(int argc, char ** argv) { + testing t(std::cout); + t.verbose = true; + + // usage: test-mtmd-impl [filter_regex] + for (int i = 1; i < argc; i++) { + t.set_filter(argv[i]); + } + + for (const auto & e : test_registry::all()) { + t.test(e.name, e.fn); + } + + return t.summary(); +} diff --git a/tools/mtmd/CMakeLists.txt b/tools/mtmd/CMakeLists.txt index 95aa853ea..aa97c768a 100644 --- a/tools/mtmd/CMakeLists.txt +++ b/tools/mtmd/CMakeLists.txt @@ -90,6 +90,9 @@ if (BUILD_SHARED_LIBS) set_target_properties (mtmd PROPERTIES POSITION_INDEPENDENT_CODE ON) target_compile_definitions(mtmd PRIVATE LLAMA_BUILD) target_compile_definitions(mtmd PUBLIC LLAMA_SHARED) + + # export all symbols so that internal components can be tested by test-mtmd-impl + set_target_properties (mtmd PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS ON) endif() set(MTMD_PUBLIC_HEADERS diff --git a/tools/mtmd/clip-impl.h b/tools/mtmd/clip-impl.h index 2c9ea499c..ea8549776 100644 --- a/tools/mtmd/clip-impl.h +++ b/tools/mtmd/clip-impl.h @@ -858,6 +858,9 @@ static std::ifstream open_ifstream_binary(const std::string & fname) { } #endif +// in test-mtmd-impl, we include woth common.h and this file, and these functions are duplicated +// this is a quick fix to avoid compilation errors +#ifndef DIRECTORY_SEPARATOR static std::string string_format(const char * fmt, ...) { va_list ap; va_list ap2; @@ -915,6 +918,7 @@ inline bool string_ends_with(std::string_view str, std::string_view suffix) { return str.size() >= suffix.size() && str.compare(str.size() - suffix.size(), suffix.size(), suffix) == 0; } +#endif // // gguf utils diff --git a/tools/mtmd/mtmd-image.cpp b/tools/mtmd/mtmd-image.cpp index 769b6efe6..0d9db4f62 100644 --- a/tools/mtmd/mtmd-image.cpp +++ b/tools/mtmd/mtmd-image.cpp @@ -1013,14 +1013,31 @@ mtmd_image_preproc_out mtmd_image_preprocessor_lfm2::preprocess(const clip_image return output; } +bool mtmd_image_preprocessor_lfm2::should_tile( + const clip_hparams & hparams, + const clip_image_size & original_size) { + const int align_size = hparams.patch_size * hparams.n_merge; + + const auto round_by_factor = [align_size](float x) { + // see https://github.com/ggml-org/llama.cpp/pull/27057#discussion_r3796264887 + return static_cast(std::nearbyint(static_cast(x) / align_size)) * align_size; + }; + + const int h_bar = std::max(hparams.patch_size, round_by_factor(original_size.height)); + const int w_bar = std::max(hparams.patch_size, round_by_factor(original_size.width)); + + return static_cast(h_bar) * static_cast(w_bar) > + static_cast(hparams.image_max_pixels) * max_pixels_tolerance; +} + mtmd_image_preprocessor_llava_uhd::slice_instructions mtmd_image_preprocessor_lfm2::get_slice_instructions(const clip_image_size & original_size) { mtmd_image_preprocessor_llava_uhd::slice_instructions inst; const int align_size = hparams.patch_size * hparams.n_merge; inst.overview_size = img_tool::calc_size_preserved_ratio( original_size, { align_size, hparams.image_min_pixels, hparams.image_max_pixels, 0 }); - // tile if either dimension exceeds tile_size with tolerance - const bool needs_tiling = original_size.width > tile_size * max_pixels_tolerance || original_size.height > tile_size * max_pixels_tolerance; + + const bool needs_tiling = should_tile(hparams, original_size); if (!needs_tiling) { inst.refined_size = clip_image_size{0, 0}; diff --git a/tools/mtmd/mtmd-image.h b/tools/mtmd/mtmd-image.h index 40dfea7eb..732e27379 100644 --- a/tools/mtmd/mtmd-image.h +++ b/tools/mtmd/mtmd-image.h @@ -148,6 +148,8 @@ struct mtmd_image_preprocessor_lfm2 : mtmd_image_preprocessor_llava_uhd { mtmd_image_preproc_out preprocess(const clip_image_u8 & img) override; slice_instructions get_slice_instructions(const clip_image_size & original_size) override; + static bool should_tile(const clip_hparams & hparams, const clip_image_size & original_size); + private: clip_image_size find_closest_aspect_ratio( float aspect_ratio, From c0296022f361beaab4c6ecf5def391d2fc598dae Mon Sep 17 00:00:00 2001 From: shivamkumard-ctrl Date: Tue, 18 Aug 2026 15:25:45 +0530 Subject: [PATCH 11/48] ci: add Windows ARM64 CUDA support to the manual workflow (#27300) - Add a CUDA 13.4 ARM64 matrix entry. - Build only ggml-cuda for x64 and ARM64. --- .github/actions/windows-setup-cuda/action.yml | 3 +- .github/workflows/build-cuda-windows.yml | 32 +++++++++++-------- 2 files changed, 20 insertions(+), 15 deletions(-) diff --git a/.github/actions/windows-setup-cuda/action.yml b/.github/actions/windows-setup-cuda/action.yml index 31250eda1..917513b85 100644 --- a/.github/actions/windows-setup-cuda/action.yml +++ b/.github/actions/windows-setup-cuda/action.yml @@ -6,8 +6,7 @@ inputs: required: true cuda_arch: description: "CUDA target architecture" - required: false - default: "x64" + required: true runs: using: "composite" diff --git a/.github/workflows/build-cuda-windows.yml b/.github/workflows/build-cuda-windows.yml index 8b59f3975..95843946f 100644 --- a/.github/workflows/build-cuda-windows.yml +++ b/.github/workflows/build-cuda-windows.yml @@ -22,6 +22,7 @@ env: jobs: cuda: + name: windows-cuda (${{ matrix.cuda }}, ${{ matrix.arch }}) runs-on: windows-2022 permissions: @@ -29,7 +30,16 @@ jobs: strategy: matrix: - cuda: ['12.4', '13.3'] + include: + - cuda: '12.4' + arch: x64 + defines: '-DGGML_CUDA_CUB_3DOT2=ON' + - cuda: '13.3' + arch: x64 + defines: '' + - cuda: '13.4' + arch: arm64 + defines: '-DCMAKE_TOOLCHAIN_FILE=cmake/arm64-windows-msvc-cuda.cmake' steps: - name: Clone @@ -39,12 +49,13 @@ jobs: - name: ccache uses: ggml-org/ccache-action@v1.2.21 with: - key: release-windows-2022-x64-cuda-${{ matrix.cuda }} + key: release-windows-2022-${{ matrix.arch }}-cuda-${{ matrix.cuda }} - name: Install Cuda Toolkit uses: ./.github/actions/windows-setup-cuda with: cuda_version: ${{ matrix.cuda }} + cuda_arch: ${{ matrix.arch }} - name: Install Ninja id: install_ninja @@ -54,26 +65,21 @@ jobs: - name: Build id: cmake_build shell: cmd - # TODO: Remove GGML_CUDA_CUB_3DOT2 flag once CCCL 3.2 is bundled within CTK and that CTK version is used in this project run: | - call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 + call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" ${{ matrix.arch == 'x64' && 'x64' || 'amd64_arm64' }} cmake -S . -B build -G "Ninja Multi-Config" ^ - -DLLAMA_BUILD_SERVER=ON ^ - -DLLAMA_BUILD_BORINGSSL=ON ^ - -DGGML_NATIVE=OFF ^ -DGGML_BACKEND_DL=ON ^ - -DGGML_CPU_ALL_VARIANTS=ON ^ + -DGGML_NATIVE=OFF ^ + -DGGML_CPU=OFF ^ -DGGML_CUDA=ON ^ - -DGGML_RPC=ON ^ - -DGGML_CUDA_CUB_3DOT2=ON + -DLLAMA_BUILD_BORINGSSL=ON ${{ matrix.defines }} set /A NINJA_JOBS=%NUMBER_OF_PROCESSORS%-1 - cmake --build build --config Release -j %NINJA_JOBS% -t ggml - cmake --build build --config Release + cmake --build build --config Release -j %NINJA_JOBS% --target ggml-cuda - name: ccache-clear uses: ./.github/actions/ccache-clear with: - key: release-windows-2022-x64-cuda-${{ matrix.cuda }} + key: release-windows-2022-${{ matrix.arch }}-cuda-${{ matrix.cuda }} hip: runs-on: windows-2022 From 9d77fa17254e1dee4b9e92504c91611a60b1359f Mon Sep 17 00:00:00 2001 From: Zijun Yu Date: Tue, 18 Aug 2026 18:02:22 +0800 Subject: [PATCH 12/48] ci : Update OpenVINO to 2026.3, skip nemotron-h rollback test (#27292) * update to ov-2026.3, update device drivers * ci: skip nemotron-h rollback test on OpenVINO The OpenVINO backend does not support SSM_SCAN, so the Nemotron-H recurrent state rollback graph is split and cannot preserve the recurrent cache output shape. Keep the test enabled for other backends and retain the qwen35 OpenVINO rollback coverage. --------- Co-authored-by: ravi9 --- .devops/openvino.Dockerfile | 20 ++++++++++---------- .github/workflows/build-cache.yml | 10 +++++----- .github/workflows/build-openvino.yml | 14 +++++++------- .github/workflows/build-self-hosted.yml | 4 ++-- .github/workflows/release.yml | 8 ++++---- ci/run.sh | 2 +- docs/backend/OPENVINO.md | 12 ++++++------ 7 files changed, 35 insertions(+), 35 deletions(-) diff --git a/.devops/openvino.Dockerfile b/.devops/openvino.Dockerfile index 9b2784b66..7b0249da9 100644 --- a/.devops/openvino.Dockerfile +++ b/.devops/openvino.Dockerfile @@ -1,18 +1,18 @@ -ARG OPENVINO_VERSION_MAJOR=2026.2.1 -ARG OPENVINO_VERSION_FULL=2026.2.1.21919.ede283a88e3 +ARG OPENVINO_VERSION_MAJOR=2026.3 +ARG OPENVINO_VERSION_FULL=2026.3.0.22451.bd8d6542e3c ARG UBUNTU_VERSION=24.04 # Intel GPU driver versions. https://github.com/intel/compute-runtime/releases -ARG IGC_VERSION=v2.36.3 -ARG IGC_VERSION_FULL=2_2.36.3+21719 -ARG COMPUTE_RUNTIME_VERSION=26.22.38646.4 -ARG COMPUTE_RUNTIME_VERSION_FULL=26.22.38646.4-0 +ARG IGC_VERSION=v2.38.2 +ARG IGC_VERSION_FULL=2_2.38.2+22051 +ARG COMPUTE_RUNTIME_VERSION=26.27.39122.11 +ARG COMPUTE_RUNTIME_VERSION_FULL=26.27.39122.11-0 ARG IGDGMM_VERSION=22.10.0 # Intel NPU driver versions. https://github.com/intel/linux-npu-driver/releases -ARG NPU_DRIVER_VERSION=v1.33.0 -ARG NPU_DRIVER_FULL=v1.33.0.20260529-26625960453 -ARG LIBZE1_VERSION=1.27.0-1~24.04~ppa2 +ARG NPU_DRIVER_VERSION=v1.35.0 +ARG NPU_DRIVER_FULL=v1.35.0.20260722-29947505341 +ARG LIBZE1_VERSION=1.28.2-1~24.04~ppa1 # Optional proxy build arguments ARG http_proxy= @@ -170,7 +170,7 @@ RUN --mount=type=cache,target=/var/cache/intel-npu,sharing=locked \ fi; \ DEB=/var/cache/intel-npu/libze1_${LIBZE1_VERSION}_amd64.deb; \ if [ ! -f "$DEB" ]; then \ - wget -q -O "$DEB" https://snapshot.ppa.launchpadcontent.net/kobuk-team/intel-graphics/ubuntu/20260324T100000Z/pool/main/l/level-zero-loader/libze1_${LIBZE1_VERSION}_amd64.deb; \ + wget -q -O "$DEB" https://snapshot.ppa.launchpadcontent.net/kobuk-team/intel-graphics/ubuntu/20260606T100000Z/pool/main/l/level-zero-loader/libze1_${LIBZE1_VERSION}_amd64.deb; \ fi; \ mkdir /tmp/npu/ && cd /tmp/npu/ && tar -xf "$TGZ" && cp "$DEB" .; \ apt-get update; \ diff --git a/.github/workflows/build-cache.yml b/.github/workflows/build-cache.yml index 604f84241..187427a8d 100644 --- a/.github/workflows/build-cache.yml +++ b/.github/workflows/build-cache.yml @@ -40,9 +40,9 @@ jobs: runs-on: ubuntu-24.04 env: - # Sync versions in build.yml, build-self-hosted.yml, release.yml, build-cache.yml, .devops/openvino.Dockerfile - OPENVINO_VERSION_MAJOR: "2026.2.1" - OPENVINO_VERSION_FULL: "2026.2.1.21919.ede283a88e3" + # Sync versions in build-openvino.yml, build-self-hosted.yml, release.yml, build-cache.yml, .devops/openvino.Dockerfile + OPENVINO_VERSION_MAJOR: "2026.3" + OPENVINO_VERSION_FULL: "2026.3.0.22451.bd8d6542e3c" steps: - name: Clone @@ -69,8 +69,8 @@ jobs: env: # Sync versions in build.yml, build-self-hosted.yml, release.yml, build-cache.yml, .devops/openvino.Dockerfile - OPENVINO_VERSION_MAJOR: "2026.2.1" - OPENVINO_VERSION_FULL: "2026.2.1.21919.ede283a88e3" + OPENVINO_VERSION_MAJOR: "2026.3" + OPENVINO_VERSION_FULL: "2026.3.0.22451.bd8d6542e3c" steps: - name: Clone diff --git a/.github/workflows/build-openvino.yml b/.github/workflows/build-openvino.yml index 938cde3f2..ee4268f97 100644 --- a/.github/workflows/build-openvino.yml +++ b/.github/workflows/build-openvino.yml @@ -39,8 +39,8 @@ jobs: env: # Sync versions in build-openvino.yml, build-self-hosted.yml, release.yml, build-cache.yml, .devops/openvino.Dockerfile - OPENVINO_VERSION_MAJOR: "2026.2.1" - OPENVINO_VERSION_FULL: "2026.2.1.21919.ede283a88e3" + OPENVINO_VERSION_MAJOR: "2026.3" + OPENVINO_VERSION_FULL: "2026.3.0.22451.bd8d6542e3c" steps: - name: Clone @@ -81,7 +81,7 @@ jobs: # TODO: fix and re-enable the `test-llama-archs` test below run: | cd ${{ github.workspace }} - ctest --test-dir build/ReleaseOV -L main -E "test-llama-archs" --verbose --timeout 2000 + ctest --test-dir build/ReleaseOV -L main -E "test-llama-archs|test-recurrent-state-rollback-nemotron-h" --verbose --timeout 2000 - name: Test (GPU) id: cmake_test_gpu @@ -89,15 +89,15 @@ jobs: run: | cd ${{ github.workspace }} export GGML_OPENVINO_DEVICE=GPU - ctest --test-dir build/ReleaseOV -L main -E "test-llama-archs" --verbose --timeout 3000 + ctest --test-dir build/ReleaseOV -L main -E "test-llama-archs|test-recurrent-state-rollback-nemotron-h" --verbose --timeout 3000 openvino-windows-2022: runs-on: windows-2022 env: # Sync versions in build-openvino.yml, build-self-hosted.yml, release.yml, build-cache.yml, .devops/openvino.Dockerfile - OPENVINO_VERSION_MAJOR: "2026.2.1" - OPENVINO_VERSION_FULL: "2026.2.1.21919.ede283a88e3" + OPENVINO_VERSION_MAJOR: "2026.3" + OPENVINO_VERSION_FULL: "2026.3.0.22451.bd8d6542e3c" steps: - name: Clone @@ -166,4 +166,4 @@ jobs: call "%OPENVINO_ROOT%\setupvars.bat" cd build - ctest --test-dir ReleaseOV -L main -E "test-llama-archs" -C Release --verbose --timeout 3000 + ctest --test-dir ReleaseOV -L main -E "test-llama-archs|test-recurrent-state-rollback-nemotron-h" -C Release --verbose --timeout 3000 diff --git a/.github/workflows/build-self-hosted.yml b/.github/workflows/build-self-hosted.yml index 0ef202193..fe2ab8154 100644 --- a/.github/workflows/build-self-hosted.yml +++ b/.github/workflows/build-self-hosted.yml @@ -288,8 +288,8 @@ jobs: env: # Sync versions in build.yml, build-self-hosted.yml, release.yml, build-cache.yml, .devops/openvino.Dockerfile - OPENVINO_VERSION_MAJOR: "2026.2.1" - OPENVINO_VERSION_FULL: "2026.2.1.21919.ede283a88e3" + OPENVINO_VERSION_MAJOR: "2026.3" + OPENVINO_VERSION_FULL: "2026.3.0.22451.bd8d6542e3c" steps: - name: Clone diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index eba2bd87b..9de593b9c 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -446,8 +446,8 @@ jobs: env: # Sync versions in build-openvino.yml, build-self-hosted.yml, release.yml, build-cache.yml, .devops/openvino.Dockerfile - OPENVINO_VERSION_MAJOR: "2026.2.1" - OPENVINO_VERSION_FULL: "2026.2.1.21919.ede283a88e3" + OPENVINO_VERSION_MAJOR: "2026.3" + OPENVINO_VERSION_FULL: "2026.3.0.22451.bd8d6542e3c" steps: - name: Set OpenVINO version output @@ -562,8 +562,8 @@ jobs: env: # Sync versions in build-openvino.yml, build-self-hosted.yml, release.yml, build-cache.yml, .devops/openvino.Dockerfile - OPENVINO_VERSION_MAJOR: "2026.2.1" - OPENVINO_VERSION_FULL: "2026.2.1.21919.ede283a88e3" + OPENVINO_VERSION_MAJOR: "2026.3" + OPENVINO_VERSION_FULL: "2026.3.0.22451.bd8d6542e3c" steps: - name: Set OpenVINO version output diff --git a/ci/run.sh b/ci/run.sh index 8046df255..3d1d75b5b 100755 --- a/ci/run.sh +++ b/ci/run.sh @@ -190,7 +190,7 @@ if [ ! -z ${GG_BUILD_OPENVINO} ]; then CMAKE_EXTRA="${CMAKE_EXTRA} -DGGML_OPENVINO=ON" # TODO: fix and re-enable the `test-llama-archs` test below - CTEST_EXTRA="-E test-llama-archs" + CTEST_EXTRA="-E test-llama-archs|test-recurrent-state-rollback-nemotron-h" fi ## helpers diff --git a/docs/backend/OPENVINO.md b/docs/backend/OPENVINO.md index 68b960a41..3cdf631ce 100644 --- a/docs/backend/OPENVINO.md +++ b/docs/backend/OPENVINO.md @@ -237,8 +237,8 @@ chmod +x ubuntu-llamacpp-ov-install.sh # ============================================ set -euo pipefail -OPENVINO_VERSION_MAJOR="2026.2.1" -OPENVINO_VERSION_FULL="2026.2.1.21919.ede283a88e3" +OPENVINO_VERSION_MAJOR="2026.3" +OPENVINO_VERSION_FULL="2026.3.0.22451.bd8d6542e3c" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" OPENVINO_INSTALL_DIR="/opt/intel/openvino_${OPENVINO_VERSION_MAJOR}" @@ -334,7 +334,7 @@ echo " ./build/ReleaseOV/bin/llama-cli -m model.gguf" ``` > [!NOTE] -> The script pins OpenVINO `2026.2.1` via the `OPENVINO_VERSION_MAJOR` / `OPENVINO_VERSION_FULL` variables at the top — edit them to track a different release. +> The script pins OpenVINO `2026.3` via the `OPENVINO_VERSION_MAJOR` / `OPENVINO_VERSION_FULL` variables at the top — edit them to track a different release. @@ -364,8 +364,8 @@ REM ============================================ REM llama.cpp OpenVINO Build Script (Ninja) REM ============================================ -set "OPENVINO_VERSION_MAJOR=2026.2.1" -set "OPENVINO_VERSION_FULL=2026.2.1.21919.ede283a88e3" +set "OPENVINO_VERSION_MAJOR=2026.3" +set "OPENVINO_VERSION_FULL=2026.3.0.22451.bd8d6542e3c" set "SCRIPT_DIR=%~dp0" set "VCPKG_DIR=C:\vcpkg" @@ -547,7 +547,7 @@ endlocal ``` > [!NOTE] -> The script pins OpenVINO `2026.2.1` via the `OPENVINO_VERSION_MAJOR` / `OPENVINO_VERSION_FULL` variables at the top — edit them to track a different release. From any new shell, source the matching `setupvars` script via the junction — `call "C:\Intel\openvino\setupvars.bat"` from `cmd`, or `& "C:\Intel\openvino\setupvars.ps1"` from PowerShell. If `winget` cannot register Visual Studio Build Tools on first run, install them once manually and re-run the script from an elevated **Developer Command Prompt for VS 2022**. +> The script pins OpenVINO `2026.3` via the `OPENVINO_VERSION_MAJOR` / `OPENVINO_VERSION_FULL` variables at the top — edit them to track a different release. From any new shell, source the matching `setupvars` script via the junction — `call "C:\Intel\openvino\setupvars.bat"` from `cmd`, or `& "C:\Intel\openvino\setupvars.ps1"` from PowerShell. If `winget` cannot register Visual Studio Build Tools on first run, install them once manually and re-run the script from an elevated **Developer Command Prompt for VS 2022**. From 169e4a7ff201e666c2325bcd35afb64573e39d09 Mon Sep 17 00:00:00 2001 From: Georgi Gerganov Date: Tue, 18 Aug 2026 14:35:04 +0300 Subject: [PATCH 13/48] readme : update status badges + regen AUTHORS (#27317) * readme : update status badges * authors : regen --- AUTHORS | 463 +++++++++++++++++++++++++++++++++++++++++++++++++++++- README.md | 7 +- 2 files changed, 466 insertions(+), 4 deletions(-) diff --git a/AUTHORS b/AUTHORS index c297f3c21..41c6672ca 100644 --- a/AUTHORS +++ b/AUTHORS @@ -1,8 +1,9 @@ -# date: Mon Feb 2 08:45:04 EET 2026 +# date: Tue Aug 18 14:32:43 EEST 2026 # this file is auto-generated by scripts/gen-authors.sh Нияз Гарифзянов <112617865+garrnizon@users.noreply.github.com> 杨朱 · Kiki +王金旭 <105263726+wjinxu@users.noreply.github.com> エシュナヴァリシア <148695646+eternaphia@users.noreply.github.com> 吴小白 <296015668@qq.com> 源文雨 <41315874+fumiama@users.noreply.github.com> @@ -10,47 +11,70 @@ 도로로도로또 <60079918+dororodoroddo@users.noreply.github.com> 손희준 谢乃闻 +0 <1939455790@qq.com> +0 <56664264+Yunzez@users.noreply.github.com> 0cc4m 0Marble <85058989+0Marble@users.noreply.github.com> 0xspringtime <110655352+0xspringtime@users.noreply.github.com> 20kdc 2114L3 <2114L3@users.noreply.github.com> 2f38b454 +3 a l i <58257628+alielfilali01@users.noreply.github.com> 3ooabkhxtn <31479382+3ooabkhxtn@users.noreply.github.com> 44670 <44670@users.noreply.github.com> 4onen <11580688+4onen@users.noreply.github.com> 65a <10104049+65a@users.noreply.github.com> 708-145 <40387547+708-145@users.noreply.github.com> +A B +a-huk <56552991+a-huk@users.noreply.github.com> a-n-n-a-l-e-e <150648636+a-n-n-a-l-e-e@users.noreply.github.com> +a3894281 a3sh <38979186+A3shTnT@users.noreply.github.com> aa956 Aadeshveer Singh <24b0926@iitb.ac.in> Aadeshveer Singh +aafsmarak <92150196+aafsmarak@users.noreply.github.com> +Aarnav Pai <52203828+arnu515@users.noreply.github.com> Aarni Koskela Aaron Miller Aaron Teo <57927438+taronaeo@users.noreply.github.com> Aaron Teo Aaryaman Vasishta Abheek Gulati +abhijain1204fujitsu <139222713+abhijain1204fujitsu@users.noreply.github.com> +Abhijit Ramesh +abhijitb11 <113058133+abhijitb11@users.noreply.github.com> Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com> +Abhinay Krishna Abhishek Gopinath K <31348521+overtunned@users.noreply.github.com> +abotsis +Abraham Gonzalez Acly Adam +adavyas <121313528+adavyas@users.noreply.github.com> adel boussaken +adgup-qti Adithya Balaji AdithyanI +Aditya Singh <60082699+adityasingh2400@users.noreply.github.com> +Adrian <40185566+adrianisk@users.noreply.github.com> Adrian Adrian Hesketh Adrian Kretz Adrian Lundberg <47256989+alundb@users.noreply.github.com> +Adrien Adrien Gallouët Adrien Gallouët +AesSedai <7980540+AesSedai@users.noreply.github.com> afrideva <95653597+afrideva@users.noreply.github.com> ag2s20150909 <19373730+ag2s20150909@users.noreply.github.com> +agent-enemy-2 +AgoraPete agray3 Ahmad Tameem <113388789+Tameem-10xE@users.noreply.github.com> Ahmet Zeer ai-fonsi +aic0d3r <168572732+aic0d3r@users.noreply.github.com> Aidan <99101158+gSUz92nc@users.noreply.github.com> AidanBeltonS <87009434+AidanBeltonS@users.noreply.github.com> AidanBeltonS @@ -59,6 +83,8 @@ Akarshan Biswas Akarshan Biswas Akarshan Biswas akawrykow <142945436+akawrykow@users.noreply.github.com> +akleine +Al G Al Mochkin <14274697+amochkin@users.noreply.github.com> Alan Gray Alawode Oluwandabira @@ -70,9 +96,13 @@ Alberto Cabrera Pérez Alberto Cabrera Pérez Aldehir Rojas alek3y <44779186+alek3y@users.noreply.github.com> +Aleksander Grygier Aleksander Grygier +Aleksander Grygier Aleksei Nikiforov <103434461+AlekseiNikiforovIBM@users.noreply.github.com> +Alessandro de Oliveira Faria (A.K.A.CABELO) Alessandro98-git <61804547+Alessandro98-git@users.noreply.github.com> +Alex <18387287+wadealexc@users.noreply.github.com> Alex Alex Azarov Alex Azarov @@ -89,6 +119,10 @@ Alex Tuddenham <61622354+AlexsCode@users.noreply.github.com> Alex von Gluck IV Alex Wu alex-spacemit +Alexander Batischev +Alexander Heisler <126129661+heislera763@users.noreply.github.com> +Alexey Dubrov +Alexey Kopytko Alexey Parfenov Alexis Williams alexpinel <93524949+alexpinel@users.noreply.github.com> @@ -108,16 +142,24 @@ amd-lalithnc Amir amirai21 <89905406+amirai21@users.noreply.github.com> AmirAli Mirian <37371367+amiralimi@users.noreply.github.com> +Amos Wong <8733840+amoshydra@users.noreply.github.com> amritahs-ibm +An Long AN Long +Anand Patil <126432639+AnandPatil1@users.noreply.github.com> Ananta Bastola Anas Ahouzi <112881240+aahouzi@users.noreply.github.com> Anav Prasad anavp-nvidia +anchortense Andika Wasisto András Salamon +Andrea Arcangeli +Andrea Richiardi Andreas (Andi) Kunar Andreas Kieslinger <47689530+aendk@users.noreply.github.com> +Andreas Krebbel +Andreas Obersteiner Andrei Andrew Aladjev Andrew Canis @@ -126,9 +168,13 @@ Andrew Duffy Andrew Godfrey Andrew Marshall Andrew Minh Nguyen <40281306+amqdn@users.noreply.github.com> +Andrew Smith andrijdavid Andy Salerno Andy Tai +Andy Williams <8692+sobakasu@users.noreply.github.com> +andyluo7 <43718156+andyluo7@users.noreply.github.com> +Angel Galindo <131726962+AngelGalindo7@users.noreply.github.com> Ankur Verma <31362771+ankurvdev@users.noreply.github.com> anon998 <131767832+anon998@users.noreply.github.com> Anri Lombard @@ -140,7 +186,10 @@ Anton Mitkov Anton Mitkov Antonis Makropoulos Anudit Nagar +Anuj Attri anzz1 +Aparna M P +Aparna M P apaz apcameron <37645737+apcameron@users.noreply.github.com> arch-btw <57669023+arch-btw@users.noreply.github.com> @@ -149,11 +198,13 @@ ardfork <134447697+ardfork@users.noreply.github.com> Arik Poznanski arlo-phoenix <140345165+arlo-phoenix@users.noreply.github.com> Armen Kaleshian +Arsen Arutunan <58118221+limloop@users.noreply.github.com> Artem Artem Zinnatullin Artyom Lebedev aryantandon01 <80969509+aryantandon01@users.noreply.github.com> Asbjørn Olling +asf0 Ásgeir Bjarni Ingvarsson Asghar Ghorbani Ashish <1856117+ashishdatta@users.noreply.github.com> @@ -162,10 +213,12 @@ Ashraful Islam AT at8u <129688334+at8u@users.noreply.github.com> Atharva Dubey +Atomic-Germ <97569476+Atomic-Germ@users.noreply.github.com> Atsushi Tatsuma aubreyli Austin <77757836+teleprint-me@users.noreply.github.com> AustinMroz +AUTOMATIC1111 <16777216c@gmail.com> automaticcat awatuna <23447591+awatuna@users.noreply.github.com> b4b4o @@ -174,6 +227,7 @@ BADR bagheera <59658056+bghira@users.noreply.github.com> Bailey Chittle <39804642+bachittle@users.noreply.github.com> bandoti <141645996+bandoti@users.noreply.github.com> +Bar Haim BarfingLemurs <128182951+BarfingLemurs@users.noreply.github.com> Bart Louwers Bartowski <3266127+bartowski1182@users.noreply.github.com> @@ -184,24 +238,35 @@ BB-fat <45072480+BB-fat@users.noreply.github.com> Behnam M <58621210+ibehnam@users.noreply.github.com> beiller Beinsezii <39478211+Beinsezii@users.noreply.github.com> +Belem Zhang Ben Ashbaugh Ben Chen Ben Garney +Ben Guidarelli +Ben Racicot <1815385+BenRacicot@users.noreply.github.com> Ben Siraphob Ben Williams Benjamin Findley <39356821+Kartoffelsaft@users.noreply.github.com> Benjamin Lecaillon <84293038+blecaillon@users.noreply.github.com> Benni <73313922+BenjaminBruenau@users.noreply.github.com> Benson Wong +Berk Idem <55372926+berkidem@users.noreply.github.com> +Bernard Ladenthin Bernat Vadell Bernhard M. Wiedemann Bert Wagner +Bertay Eren <39909689+bertaye@users.noreply.github.com> +Bhavik Sharda <10757940+BLSharda@users.noreply.github.com> bhubbb <79117352+bhubbb@users.noreply.github.com> +Bill Sideris Billel Mokeddem Bingan <70050083+binganao@users.noreply.github.com> +Bipin Yadav <83943505+bipinyadav3175@users.noreply.github.com> Bizhao Shi <37729561+shibizhao@users.noreply.github.com> Bjarke Viksøe <164612031+bviksoe@users.noreply.github.com> Björn Ganster +BlackFoil <127078112+BlackFoil@users.noreply.github.com> +BlueMöhre bmwl Bo Zheng <368586905@qq.com> bobqianic <129547291+bobqianic@users.noreply.github.com> @@ -223,6 +288,7 @@ bryanSwk <93190252+bryanSwk@users.noreply.github.com> bsilvereagle bssrdf byte-6174 <88070277+byte-6174@users.noreply.github.com> +Caleb DeLeeuw <143902425+SolshineCode@users.noreply.github.com> Calvin Laurenson Cameron Cameron Kaiser @@ -238,6 +304,7 @@ cduk <19917266+cduk@users.noreply.github.com> cebtenzzre Cebtenzzre CentricStorm +Cetarthoriphros Chad Brewbaker Chad Voegele chaihahaha @@ -248,15 +315,20 @@ characharm <123120856+characharm@users.noreply.github.com> Charles Duffy Charles Xu <63788048+chaxu01@users.noreply.github.com> Charles Xu +Chedrian07 <108463785+Chedrian07@users.noreply.github.com> chen fan <350211548@qq.com> Chen Xi Chen Xi +Chen Yuan +Chen Yuan Cheng Shao Chenguang Li <757486878@qq.com> Chenguang Li <87689256+noemotiovon@users.noreply.github.com> +Chipmunk <101038159+CHIPMUNK-T0T@users.noreply.github.com> chiranko <96988916+chiranko@users.noreply.github.com> Chris Elrod Chris Kuehl +Chris Lee Chris Peterson Chris Rohlf Chris Thompson @@ -264,11 +336,16 @@ Christian Demsar Christian Demsar Christian Falch <875252+chrfalch@users.noreply.github.com> Christian Fillion +Christian Hoener zu Siederdissen Christian Kastner Christian Kögler Christian Köhnenkamp +Christian Schmitz Christian Zhou-Zheng <59622928+christianazinn@users.noreply.github.com> +Christopher Albert +Christopher Maher Christopher Nielsen <62156882+mascguy@users.noreply.github.com> +Chyan <163109379+chyan8@users.noreply.github.com> City <125218114+city96@users.noreply.github.com> CJ Pais Clark Saben <76020733+csaben@users.noreply.github.com> @@ -288,12 +365,15 @@ Congcong Cai Conrad Kramer Copilot <198982749+Copilot@users.noreply.github.com> Corentin REGAL +cphlipot <9103367+cphlipot@users.noreply.github.com> cpumaxx <163466046+cpumaxx@users.noreply.github.com> crasm crasm crat0z <11581854+crat0z@users.noreply.github.com> CRD716 CrispStrobe <154636388+CrispStrobe@users.noreply.github.com> +Cristiano Pinto <140563307+crowmoed@users.noreply.github.com> +crsawyer <7572190+crsawyer@users.noreply.github.com> Csaba Kecskemeti Cuong Trinh Manh daboe01 @@ -301,6 +381,7 @@ daghanerdonmez <44506702+daghanerdonmez@users.noreply.github.com> Damian Stewart daminho <37615795+daminho@users.noreply.github.com> DAN™ +Dan Hoffman <43101339+thedanhoffman@users.noreply.github.com> Dan Johansson <164997844+eddnjjn@users.noreply.github.com> Dan Johansson Dane Madsen @@ -308,6 +389,7 @@ DaniAndTheWeb <57776841+DaniAndTheWeb@users.noreply.github.com> Daniel Benjaminsson Daniel Bevenius Daniel Drake +Daniel Elliott Daniel Han Daniel Hiltgen Daniel Illescas Romero @@ -324,9 +406,11 @@ Dave Dave Airlie Dave Airlie Dave Della Costa +Davi Henrique Linhares <38295327+WizardlyBump17@users.noreply.github.com> David Chiu David Friehs David Huang <1969802+hjc4869@users.noreply.github.com> +David Huggins-Daines David Kennedy David Lima David Pflug @@ -334,10 +418,13 @@ david raistrick David Renshaw David Ribeiro Alves David Sommers <12738+databyte@users.noreply.github.com> +David Spruill <62445444+Spruill-1@users.noreply.github.com> David Yang David Zhao <90013954+Your-Cheese@users.noreply.github.com> +David366AI <86212041+David366AI@users.noreply.github.com> davidef DavidKorczynski +davidrhodus Dawid Potocki Dawid Wysocki <62249621+TortillaZHawaii@users.noreply.github.com> ddh0 @@ -345,11 +432,16 @@ ddh0 ddpasa <112642920+ddpasa@users.noreply.github.com> DDXDB <38449595+DDXDB@users.noreply.github.com> Dean +decahedron1 deepdiffuser <112834445+deepdiffuser@users.noreply.github.com> deepsek <166548550+deepsek@users.noreply.github.com> Deins Denis Spasyuk <34203011+dspasyuk@users.noreply.github.com> Derrick T. Woolworth +Dev-iL <6509619+Dev-iL@users.noreply.github.com> +Dev-X25874 <283057883+Dev-X25874@users.noreply.github.com> +Devedse <2350015+devedse@users.noreply.github.com> +Developer-Ecosystem-Engineering <65677710+Developer-Ecosystem-Engineering@users.noreply.github.com> Deven Mistry <31466137+deven367@users.noreply.github.com> devojony <61173062+devojony@users.noreply.github.com> diannao <55k@outlook.com> @@ -365,7 +457,9 @@ Djip007 <3705339+Djip007@users.noreply.github.com> Djip007 dm4 dm4 +Dmitry Atamanov Dmytro Minochkin +Dmytro Romanov Dobri Danchev <12420863+danchev@users.noreply.github.com> DocShotgun <126566557+DocShotgun@users.noreply.github.com> Doctor Shotgun <126566557+DocShotgun@users.noreply.github.com> @@ -375,6 +469,7 @@ Donghyeon Jeong <54725479+djeong20@users.noreply.github.com> Dongliang Wei <121270393+wdl339@users.noreply.github.com> Doomsdayrs <38189170+Doomsdayrs@users.noreply.github.com> DooWoong Lee (David) +DorianRudolph Dorin-Andrei Geman dotpy314 <33351922+dotpy314@users.noreply.github.com> Dou Xinpeng <15529241576@163.com> @@ -383,7 +478,9 @@ Douglas Hanley Dowon Dr. Tom Murphy VII Ph.D <499244+tom7@users.noreply.github.com> drbh +drrros <52050875+drrros@users.noreply.github.com> ds5t5 <145942675+ds5t5@users.noreply.github.com> +dskwe duduta dylan eastriver @@ -395,11 +492,14 @@ Ed Addario <29247825+EAddario@users.noreply.github.com> Ed Lee Ed Lepedus Eddie-Wang +eduardopessin <100053075+eduardopessin@users.noreply.github.com> Edward Taylor eiery <19350831+eiery@users.noreply.github.com> Elaine Elbios <141279586+Elbios@users.noreply.github.com> Elton Kola +Emanuil Rusev +Emil Askerov <56842174+EmilAskerov@users.noreply.github.com> Emmanuel Ferdman Emreerdog <34742675+Emreerdog@users.noreply.github.com> Engininja2 <139037756+Engininja2@users.noreply.github.com> @@ -407,6 +507,8 @@ Equim Eric Curtin Eric Curtin Eric Curtin +Eric Hartford +Eric Hsieh Eric Sommerlade Eric Zhang <34133756+EZForever@users.noreply.github.com> eric8607242 @@ -414,8 +516,10 @@ Erik Garrison Erik Scholz Ervin Áron Tasnádi Esko Toivonen +Ethan Turner Ettore Di Giacinto EugeoSynthesisThirtyTwo +Evan Huus Evan Jones Evan Miller Eve <139727413+netrunnereve@users.noreply.github.com> @@ -434,21 +538,29 @@ Fan Shupei FantasyGmm <16450052+FantasyGmm@users.noreply.github.com> fanyang Farbod Bijary <110523279+farbodbj@users.noreply.github.com> +Fathi Boudra Fattire <528174+fat-tire@users.noreply.github.com> +felix Felix fengerhu1 <2748250768@qq.com> fidoriel <49869342+fidoriel@users.noreply.github.com> +fiesh Finn Voorhees Firat FirstTimeEZ <179362031+FirstTimeEZ@users.noreply.github.com> fj-y-saito <85871716+fj-y-saito@users.noreply.github.com> FK +fl0rianr <226492742+fl0rianr@users.noreply.github.com> +fl0rianr Florent BENOIT Florian Badie Folko-Ven <71110216+Folko-Ven@users.noreply.github.com> +forforever73 <63285796+forforever73@users.noreply.github.com> Foul-Tarnished <107711110+Foul-Tarnished@users.noreply.github.com> Francisco Herrera Francisco Melo <43780565+francis2tm@users.noreply.github.com> +Francois Dugast +franitel Frank Mai FrankHB Frankie Robertson @@ -456,7 +568,10 @@ fraxy-v <65565042+fraxy-v@users.noreply.github.com> Fred Douglas <43351173+fredlas@users.noreply.github.com> Frederik Vogel Fredrik Hultin +fredzillman frob +Frosty40 +Funtowicz Morgan fxzjshm <11426482+fxzjshm@users.noreply.github.com> g2mt <166577174+g2mt@users.noreply.github.com> Gabe Goodhart @@ -468,13 +583,24 @@ GainLee Galunid Gary Linscott Gary Mulder +Gaspard Petit gatbontonpc Gaurav Garg <52341457+gaugarg-nv@users.noreply.github.com> Gaurav Garg +Gautam0507 <110854761+Gautam0507@users.noreply.github.com> Gavin Zhao Genkagaku.GPT +Geo Maciolek +George <35490284+noctrex@users.noreply.github.com> Georgi Gerganov +Geramy Loveless +Gerard Guillemas Martos +Gerard Martinez +Gerben van V +Gezahegne +ghleg Gian-Carlo Pascutto +GiantPrince <90118823+GiantPrince@users.noreply.github.com> GideonSerf Gilad S Gilad S. <7817232+giladgd@users.noreply.github.com> @@ -491,6 +617,10 @@ grahameth <96447521+grahameth@users.noreply.github.com> Gregor Jasny Grzegorz Grasza gtygo +Guanhuai Zhang <67999475+BiReRa@users.noreply.github.com> +Guido Imperiale +Guido Imperiale +Guilherme Quintino Guillaume "Vermeille" Sanchez Guillaume Wenzek Guoliang Hua <32868157+nbcsm@users.noreply.github.com> @@ -499,6 +629,7 @@ Guspan Tanadi <36249910+guspan-tanadi@users.noreply.github.com> Gustavo Rocha Dias <91472747+gustrd@users.noreply.github.com> Guus Waals <_@guusw.nl> Guy Goldenberg +guyfischman <138163913+guyfischman@users.noreply.github.com> gwjr <502526+gwjr@users.noreply.github.com> h-h-h-h <13482553+h-h-h-h@users.noreply.github.com> Haggai Nuchi @@ -506,21 +637,31 @@ Haiyue Wang Halalaluyafail3 <55773281+Halalaluyafail3@users.noreply.github.com> Hale Chan Hamdoud Hakem <90524568+hamdoudhakem@users.noreply.github.com> +Hamish M. Blair Han Qingzhe <95479277+hNSBQZ@users.noreply.github.com> Han Yin HanishKVC hankcs +Hans Florian +Hao-Chen2337 <2113996104@qq.com> Haohui Mai +HaoJun ZHANG haopeng <657407891@qq.com> Haowei Wu Haoxiang Fei Harald Fernengel +Harapan Rachman +Harkirat Gill +HarrisonSec Hatsune Miku <129688334+at8u@users.noreply.github.com> HatsuneMikuUwU33 <173229399+HatsuneMikuUwU33@users.noreply.github.com> Haus1 +hcl Héctor Estrada Moreno +helanfxz <126638465+helanfxz@users.noreply.github.com> HelloKS Helton Reis <47722840+HRKings@users.noreply.github.com> +Hemanth Battu <56206750+hbattu73@users.noreply.github.com> Hendrik Erz Henk Poley Henri Vasserman @@ -534,20 +675,30 @@ Hesen Peng HighDoping HimariO hipudding +Hitesh Chopra <34310832+hiteshchopra11@users.noreply.github.com> hksdpc255 <43977088+hksdpc255@users.noreply.github.com> +hmscider <201289679+hmscider@users.noreply.github.com> Hoang Nguyen hoangmit +hogeheer499-commits +hokanosekai <69720899+hokanosekai@users.noreply.github.com> +Holger Voormann HonestQiao Hong Bo PENG hongbo.mo <352280764@qq.com> +Hongqiang Wang <66336067+wanghqc@users.noreply.github.com> +Hongqiang Wang Hongyu Ouyang <96765450+casavaca@users.noreply.github.com> hopkins385 <98618192+hopkins385@users.noreply.github.com> +hourhl <67227355+hourhl@users.noreply.github.com> Howard Su howlger howlger +hrushitfujitsu Hua Jiang Huang Qi Huawei Lin +Hugo Hugo Roussel Huifeng Ou <79071290+ho2103@users.noreply.github.com> hutli <6594598+hutli@users.noreply.github.com> @@ -555,9 +706,11 @@ hutli hutli hxer7963 hydai +iacopPBK iacore <74560659+iacore@users.noreply.github.com> Ian Bull Ian Bull +Ian Faust Ian Scrivener ibrahim khadraoui <132432132+ibrahimkhadraoui@users.noreply.github.com> Icecream95 @@ -568,13 +721,19 @@ igardev <49397134+igardev@users.noreply.github.com> igarnier IgnacioFDM Igor Okulist +Igor Rudenko Igor Smirnov Ihar Hrachyshka Ihar Hrachyshka +ihb2032 <40718643+ihb2032@users.noreply.github.com> Ikko Eltociear Ashimine Ilia Ilmer +Ilya Ilya Kurdyukov <59548320+ilyakurdyukov@users.noreply.github.com> Imad Saddik <79410781+ImadSaddik@users.noreply.github.com> +iMil +Incarnas <119618389+bit-incarnas@users.noreply.github.com> +Intel AI Get-to Market Customer Success and Solutions intelmatt <61025942+intelmatt@users.noreply.github.com> iohub Ionoclast Laboratories @@ -583,8 +742,10 @@ Isaac McFadyen IsaacDynamo <61521674+IsaacDynamo@users.noreply.github.com> Ishaan Gandhi iSma +Ismail <115064057+AlrIsmail@users.noreply.github.com> issixx <46835150+issixx@users.noreply.github.com> Ivan +Ivan Chikish Ivan Filipov <159561759+vanaka11@users.noreply.github.com> Ivan Komarov Ivan Stepanov @@ -596,6 +757,7 @@ Jack Mousseau Jack Mousseau JackJollimore <130917767+JackJollimore@users.noreply.github.com> jacobi petrucciani <8117202+jpetrucciani@users.noreply.github.com> +Jaden_Mach <88880593+jadenmach2@users.noreply.github.com> Jaeden Amero Jaemin Son Jafar Uruç @@ -604,11 +766,15 @@ jaime-m-p <167997752+jaime-m-p@users.noreply.github.com> Jake Karnes Jakkala Mahesh <155058658+MaheshJakkala@users.noreply.github.com> Jakub N +JamePeng James A Capozzoli <157492257+jac-jim@users.noreply.github.com> +James O'Leary <65884233+jpohhhh@users.noreply.github.com> James Reynolds jameswu2014 <545426914@qq.com> Jan Boon Jan Boon +Jan Ekström +Jan Patrick Lehr Jan Ploski Jannis Schönleber Jared Tweed @@ -620,8 +786,10 @@ Jason McCartney Jason Ni Jason Stillerman jason_w +Jassieluo <130133492+Jassieluo@users.noreply.github.com> Jay Jay Zenith <162098309+JayZenith@users.noreply.github.com> +Jayant Lohia JC <43374599+MrSMlT@users.noreply.github.com> jdomke <28772296+jdomke@users.noreply.github.com> Jean-Christophe Hoelt @@ -633,10 +801,14 @@ Jeffrey Quesnelle Jeremy Demeule Jeremy Rand <244188+JeremyRand@users.noreply.github.com> Jeroen Mostert +jeromew Jesse Jesse Gross Jesse Ikonen Jesse Jojo Johnson +Jesse LaRose +Jesse Posner +Jesus Talavera <145992175+jesus-talavera-ibm@users.noreply.github.com> Jett Janiak Jeximo JFLFY2255 @@ -646,22 +818,28 @@ Jiacheng (Jason) Chen <76919340+jiachengjason@users.noreply.github.com> Jiahao Li jiahao su Jian Liao +Jiang, Fish JidongZhang-THU <1119708529@qq.com> Jie Fu (傅杰) Jie Fu (傅杰) jiez <373447296@qq.com> +Jillis ter Hove +Jim Wu Jinwoo Jeong <33892306+williamjeong2@users.noreply.github.com> Jinyang He +jinzihao Jiří Podivín <66251151+jpodivin@users.noreply.github.com> Jiří Sejkora JJJYmmm <92386084+JJJYmmm@users.noreply.github.com> jklincn <985765408@qq.com> jklincn +JM Robles jneem Joan Fontanals Joan Fontanals João Dinis Ferreira Joe Eli McIlvain +Joe Rowell Joe Todd joecryptotoo <80373433+joecryptotoo@users.noreply.github.com> Johan @@ -670,16 +848,22 @@ Johannes Rudolph John <78893154+cmp-nct@users.noreply.github.com> John Balis John Bean <113509988+johnbean393@users.noreply.github.com> +John Eismeier <42679190+jeis4wpi@users.noreply.github.com> John Smith <67539080+kingsidelee@users.noreply.github.com> +Johnathan Craig Maudlin <13183098+jcmdln@users.noreply.github.com> JohnnyB johnson442 <56517414+johnson442@users.noreply.github.com> jojorne jon-chuang <9093549+jon-chuang@users.noreply.github.com> +Jonas Jankaitis <111707981+John-194@users.noreply.github.com> Jonas Wunderlich <32615971+jonas-w@users.noreply.github.com> +Jonathan <47618606+jbuchananr@users.noreply.github.com> +Jonathan Clohessy Jonathan Graehl <99024+graehl@users.noreply.github.com> Jorge A <161275481+jorgealias@users.noreply.github.com> Jose Maldonado <63384398+yukiteruamano@users.noreply.github.com> Joseph Stahl <1269177+josephst@users.noreply.github.com> +Josh Leverette Josh Ramer Joshua Cogliati Joyce @@ -689,7 +873,10 @@ Judd <4046440+foldl@users.noreply.github.com> Judd Juk Armstrong <69222624+jukofyork@users.noreply.github.com> jukofyork <69222624+jukofyork@users.noreply.github.com> +Julian Pscheid +Julien Chaumond Julien Denize <40604584+juliendenize@users.noreply.github.com> +Julien Jerphanion Julius Arkenberg Julius Tischbein Julius Tischbein @@ -698,9 +885,13 @@ Jun Jie <71215065+junnjiee16@users.noreply.github.com> junchao-loongson <68935141+junchao-loongson@users.noreply.github.com> junchao-zhao <68935141+junchao-loongson@users.noreply.github.com> Junil Kim +Junmo Kim Junwon Hwang Junyang Lin Juraj Bednar +Jürgen Schmied +JusteLeo +Justin Bradford Justin Parker Justin Santa Barbara Justin Suess @@ -709,63 +900,97 @@ Justine Tunney Justine Tunney Juuso Alasuutari Juyoung Suk +JvM jwj7140 <32943891+jwj7140@users.noreply.github.com> k.h.lai +k4ss4n <128936199+k4ss4n@users.noreply.github.com> +Kaben Nanlohy +Kabir Potdar +Kabir08 <62639358+Kabir08@users.noreply.github.com> Kai Pastor kaizau +Kakaru <97896816+KakaruHayate@users.noreply.github.com> kallewoof kallewoof kalomaze <66376113+kalomaze@users.noreply.github.com> +Kamalesh VS <76260512+kkjjkamal123@users.noreply.github.com> Kamil Tomšík kang +Kangjia Gao <145212963+kkkzbh@users.noreply.github.com> Kante Yin +karavayev <192749314+karavayev@users.noreply.github.com> Karol Kontny <82021046+kkontny@users.noreply.github.com> Karsten Weiss Karthick Karthik Kumar Viswanathan <195178+guilt@users.noreply.github.com> Karthik Sethuraman +Kartik Sirohi <99896785+sirohikartik@users.noreply.github.com> +Kashif Rasul KASR Kasumi <90275229+kasumi-1@users.noreply.github.com> +Katostrofik katsu560 <118887472+katsu560@users.noreply.github.com> Kawrakow <48489457+ikawrakow@users.noreply.github.com> kchro3 <62481661+kchro3@users.noreply.github.com> +kdkd <2569413+kdkd@users.noreply.github.com> Keiichi Tabata Keke Han Kenvix ⭐ Kerfuffle <44031344+KerfuffleV2@users.noreply.github.com> Kevin Gibbons +Kevin Hannon Kevin Ji <1146876+kevinji@users.noreply.github.com> Kevin Kwok +Kevin Liu <4396kevinliu@gmail.com> Kevin Lo Kevin Pouget Kevin Wang +Khashayar Ghafouri <43180261+khashayarghafouri@users.noreply.github.com> khimaros +Kilian Hu <90606809+kilian-hu@users.noreply.github.com> +Kilian Krampf kiltyj Kim S. kimminsu <80271594+kimminsu38oo@users.noreply.github.com> +KITAITI Makoto kiwi <122582483+kiwi142857@users.noreply.github.com> klosax <131523366+klosax@users.noreply.github.com> +KokerZhou <111279477+KokerZhou@users.noreply.github.com> Kolen Cheung +kononnable +Konrad Moren +konradmb Konstantin Herud Konstantin Zhuravlyov +Krishna Sridhar <99914379+srikris-sridhar@users.noreply.github.com> krystiancha +kubawoo +kumaal <44551860+kumaal@users.noreply.github.com> kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com> kunnis Kunshang Ji kuronekosaiko +Kusha Gharahi <3326002+kushagharahi@users.noreply.github.com> kustaaya <58045274+kustaaya@users.noreply.github.com> kuvaus <22169537+kuvaus@users.noreply.github.com> +kvc0 <3454741+kvc0@users.noreply.github.com> +Kwa Jie Hao <31984694+kwajiehao@users.noreply.github.com> kwin1412 <42286931+kwin1412@users.noreply.github.com> Kyle Bruene Kyle Liang Kyle Mistele +KyleHagy <59183061+KyleHagy@users.noreply.github.com> Kylin <56434533+KyL0N@users.noreply.github.com> l-austenfeld <53152202+l-austenfeld@users.noreply.github.com> l3utterfly +l8bloom LaffeyNyaa <112215776+LaffeyNyaa@users.noreply.github.com> laik +lainon1 <271530700+lainon1@users.noreply.github.com> Lars Grammel Lars Sonchocky-Helldorf +las7 <98077186+las7@users.noreply.github.com> +Lasse Lauwerys <65569591+Iemand005@users.noreply.github.com> Laura Law Po Ying <30721578+yingying0906@users.noreply.github.com> lcy @@ -779,6 +1004,7 @@ Lennart Austenfeld <53152202+l-austenfeld@users.noreply.github.com> leo-pony Leon Knauer Leonard Mosescu +leonardHONG <2695316095@qq.com> Leonardo Neumann LeonEricsson <70749762+LeonEricsson@users.noreply.github.com> levkropp @@ -788,6 +1014,7 @@ lhez lhez Li Pengzhan <151381994+Lpzhan931@users.noreply.github.com> Li Tan +liminfei-amd <91481003+liminfei-amd@users.noreply.github.com> limitedAtonement Linwei Wang Liu Jia <109258120+Septa2112@users.noreply.github.com> @@ -795,6 +1022,7 @@ Liu Jia liuwei-git <14815172+liuwei-git@users.noreply.github.com> lixing-star <104126818+lixing-star@users.noreply.github.com> lksj92hs <134250687+lksj92hs@users.noreply.github.com> +lnigam LoganDark Loïc Carrère lon <114724657+longregen@users.noreply.github.com> @@ -806,6 +1034,9 @@ ltoniazzi <61414566+ltoniazzi@users.noreply.github.com> Luca Stefani Lucas Moura Belo Luciano +lucy <154630366+lucyknada@users.noreply.github.com> +Ludovic Henry +Ludovic Henry Lukas Straub Łukasz Ślusarczyk <112692748+lslusarczyk@users.noreply.github.com> Luo Tian @@ -815,22 +1046,29 @@ Lyle Dean M-A M. Mediouni M. Yusuf Sarıgöz +M1DNYT3 <42499082+M1DNYT3@users.noreply.github.com> +m1el m3ndax Ma Mingfei Maarten ter Huurne +Maciej Lisowski <39798354+MaciejDromin@users.noreply.github.com> Mack Straight maddes8cht <55592906+maddes8cht@users.noreply.github.com> Maël Kerbiriou MaggotHATE +MagicExists <106458387+gugugiyu@users.noreply.github.com> magicse +Mahdiou Diallo <104755555+mahdiou@users.noreply.github.com> Mahekk Shaikh <118063190+Mahekk357@users.noreply.github.com> Mahesh Madhav <67384846+heshpdx@users.noreply.github.com> mahorozte <41834471+mahorozte@users.noreply.github.com> makomk +manayang manikbhandari Manuel <44313466+makuche@users.noreply.github.com> maor-ps <154728172+maor-ps@users.noreply.github.com> Marc Köhlbrugge +Marcel Petrick Marcello Seri Marco Matthies <71844+marcom@users.noreply.github.com> Marcos Del Sol Vives @@ -838,21 +1076,30 @@ marcoStocchi Marcus Dunn <51931484+MarcusDunn@users.noreply.github.com> Marek Hradil jr. Marian Cepok +Mario <191101255+wariuccio@users.noreply.github.com> +Mario Limonciello +Mario Limonciello Marius Gerdes <141485318+mglambda@users.noreply.github.com> Mariusz Woloszyn Mark Fairbairn Mark Zhuang Marko Tasic +Markus Ebner Markus Tavenrath +Martin Andersson +Martin Chang Martin Delille +Martin Klacer Martin Krasser Martin Schwaighofer Marvin Gießing +Marxist-Leninist <31905382+Marxist-Leninist@users.noreply.github.com> Masashi Yoshimura Masato Nakasaka Masato Nakasaka Masaya, Kato <62578291+msy-kato@users.noreply.github.com> mashdragon <122402293+mashdragon@users.noreply.github.com> +Mason Milburn MasterYi1024 <39848311+MasterYi1024@users.noreply.github.com> Mateusz Charytoniuk Matheus C. França @@ -863,9 +1110,13 @@ Mathieu Nayrolles Mathijs de Bruin Mathijs Henquet matiaslin <45382001+matiaslin@users.noreply.github.com> +Matt Matt Clayton <156335168+mattjcly@users.noreply.github.com> +Matt Corallo <649246+TheBlueMatt@users.noreply.github.com> +Matt Jallo Matt Pulver Matt Stephenson +Matt Thompson <111157855+boondocklabs@users.noreply.github.com> matt23654 <193348153+matt23654@users.noreply.github.com> matt23654 matteo @@ -875,7 +1126,9 @@ Matteo Mortari Mattheus Chediak Matthew Michel Matthew Tejo +Matthias Straka <59084281+matthiasstraka@users.noreply.github.com> Matthieu Coudron <886074+teto@users.noreply.github.com> +Matti4 Mattt Matvey Soloviev Max Krasnyansky @@ -883,12 +1136,18 @@ Max Krasnyansky Max Krasnyansky Maxim Evtush <154841002+maximevtush@users.noreply.github.com> Maxime <672982+maximegmd@users.noreply.github.com> +Maximilian Werk Maximilian Winter mdrokz +meatposes MeeMin <74113151+Meet91721@users.noreply.github.com> +megemini +Mendy Berger <12537668+MendyBerger@users.noreply.github.com> Meng Zhang Meng, Hengyu Mengqing Cao +Mengsheng Wu +Mengsheng Wu Merrick Christensen mgroeber9110 <45620825+mgroeber9110@users.noreply.github.com> Miaoqian Lin @@ -898,41 +1157,56 @@ Michaël de Vries Michael Engel Michael Francis Michael Giba +Michael Grau +Michael Huang <15768500+tehsiuhuang@users.noreply.github.com> Michael Hueschen Michael Kesper Michael Klimenko +Michael Lamothe Michael Podvitskiy Michael Potter Michael Wand +michaeltrabalka-tech Michał Moskal +Michał Piszczek Michał Tuszyński Michelle Tan <41475767+MichelleTanPY@users.noreply.github.com> +Mickael Desgranges midnight Mihai Mike Mike Abbott Mike Abbott +Mikhail Podvitskii Mikko Juola +Mikolaj Kucharski Min-Hua <136287195+Min-Hua@users.noreply.github.com> minarchist Minsoo Cheong <54794500+mscheong01@users.noreply.github.com> Minsoo Cheong Mirko185 Mirror Azure <54669636+MirrorAzure@users.noreply.github.com> +Mishusha <55416420+Mishusha@users.noreply.github.com> MistApproach <98988043+MistApproach@users.noreply.github.com> Miwa / Ensan <63481257+ensan-hcl@users.noreply.github.com> +miyan <1138989048@qq.com> mj-shifu <77107165+mj-shifu@users.noreply.github.com> +mkoker <132301062+mkoker@users.noreply.github.com> mmyjona mnehete32 <33429707+mnehete32@users.noreply.github.com> +Mohammad Athar <157023731+m-atharkhan@users.noreply.github.com> Mohammadreza Hendiani Mohammadreza Hendiani Molly Sophia momonga <115213907+mmnga@users.noreply.github.com> momonga <146910567+mmngays@users.noreply.github.com> MoonRide303 <130458190+MoonRide303@users.noreply.github.com> +MoonShadow MorganRO8 <47795945+MorganRO8@users.noreply.github.com> moritzbrantner <31051084+moritzbrantner@users.noreply.github.com> +mtmcp <141645996+mtmcp@users.noreply.github.com> muggle-stack +Muhammad Salem Murilo Santana Musab Gultekin musoles <135031143+musoles@users.noreply.github.com> @@ -945,6 +1219,8 @@ Natsu Nauful Shaikh NawafAlansari <72708095+NawafAlansari@users.noreply.github.com> Nebula +Nechama Krashinski +neha-ha <137219201+neha-ha@users.noreply.github.com> Neo Zhang <14088817+arthw@users.noreply.github.com> Neo Zhang Neo Zhang Jianyu @@ -959,18 +1235,26 @@ Niall Coates <1349685+Niall-@users.noreply.github.com> niansa/tuxifan niansa/tuxifan Nicholai Tukanov +Nicholas Sparks <157740354+nisparks@users.noreply.github.com> Nick <0x0b4ac@gmail.com> nick huang +Nick Lafleur <55208706+nicklafleur@users.noreply.github.com> +Nick Towle nickp27 +Nicky Mouha +Nico Nico Bosshard Nicolai Weitkemper Nicolas B. Pierron +Nicolas Mowen Nicolás Pérez Nicolò Scipione Nigel Bosch Nikhil Jain Nikita Sarychev <42014488+sARY77@users.noreply.github.com> Niklas Korz +Niklas Sheth +Niklas Wenzel NikolaiLyssogor <59844691+NikolaiLyssogor@users.noreply.github.com> Nikolaos Pothitos Nikolas <127742645+nneubacher@users.noreply.github.com> @@ -982,39 +1266,53 @@ nold nopperl <54780682+nopperl@users.noreply.github.com> nullname Nuno +nuri nusu-github <29514220+nusu-github@users.noreply.github.com> nwyin o7si <32285332+o7si@users.noreply.github.com> +Oğuzhan Akkaya Oleksandr Kuvshynov <661042+okuvshynov@users.noreply.github.com> Oleksandr Nikitin Oleksii Maryshchenko Olexandr88 olexiyb +Oliver Simons Oliver Simons Oliver Simons Oliver Walsh Olivier Chafik Olivier Chafik omahs <73983677+omahs@users.noreply.github.com> +Omer Ozarslan +Omid Azizi Ondřej Čertík oobabooga <112222186+oobabooga@users.noreply.github.com> oobabooga opparco +Ori Pekelman Oscar Barenys OSecret <135510162+OLSecret@users.noreply.github.com> ostix360 <55257054+ostix360@users.noreply.github.com> Ouadie EL FAROUKI +Ozymandias_EBON <112784549+johnkarlhill@users.noreply.github.com> PAB Pablo Duboue Pádraic Slattery +parabelboi Pascal Pascal Patry pascal-lc <49066376+pascal-lc@users.noreply.github.com> +Pasha Khosravi Patrice Ferlet +Patrick Buckley Patrick Peng Patryk Kaminski +Paul Dubs +Paul Flynn Paul Tsochantaris +Pavan Shinde Pavel Zloi +Pavel Zloi Pavels Zaicenkovs Pavol Rusnak Paweł Wodnicki <151604+32bitmicro@users.noreply.github.com> @@ -1028,6 +1326,7 @@ Percy Piper Perry Naseck <4472083+DaAwesomeP@users.noreply.github.com> perserk Peter +Peter Sideris Peter Sugihara Peter0x44 petterreinholdtsen @@ -1037,23 +1336,33 @@ philip-essential <169196560+philip-essential@users.noreply.github.com> Phillip Kravtsov Phylliida Dev piDack <104877312+piDack@users.noreply.github.com> +Piero Evangelista Pierre Alexandre SCHEMBRI Pierrick Hymbert Pieter Ouwerkerk +PikaPikachu Piotr Piotr Jasiukajtis Piotr Kubaj Piotr Wilkin (ilintar) pl752 Plamen Minev +pmaybank <113125070+pmaybank@users.noreply.github.com> pmysl +PMZFX pockers21 <134406831+pockers21@users.noreply.github.com> +Pop Flamingo postmasters Pouya pqnet <119850+pqnet@users.noreply.github.com> Prabod Prajwal B Mehendarkar +Pranav Dhinakar +Pranav Dhinakar +Pranav Uttarkar <122235768+PranavUttarkar@users.noreply.github.com> +Pranesh Gonegandla Prashant Vithule <119530321+Vithulep@users.noreply.github.com> +ProgenyAlpha Przemysław Pawełczyk psocolovsky <50770545+psocolovsky@users.noreply.github.com> pudepiedj @@ -1064,10 +1373,14 @@ Qin Yue Chen <71813199+chenqiny@users.noreply.github.com> qingfengfenga <41416092+qingfengfenga@users.noreply.github.com> qingy1337 Qingyou Meng +qiurui144 <39214303+qiurui144@users.noreply.github.com> qouoq Qu Zongfu <43257352+yancaoweidaode@users.noreply.github.com> +quei <56998528+quei4r@users.noreply.github.com> Quentin Bramas +QuintinShaw qunash +quyentonndbs R R R0CKSTAR @@ -1076,22 +1389,40 @@ rabidcopy RachelMantel Radoslav Gerganov Radosław Gryta +Rafail Giavrimis <47496212+grafail@users.noreply.github.com> Rafal Lewczuk +ragz4125 <65285549+ragz4125@users.noreply.github.com> Rahul Sathe <150351592+rrsathe@users.noreply.github.com> Rahul Vivek Nair <68507071+RahulVivekNair@users.noreply.github.com> +Rail Chabdarov rainred <107027757+gryffindor-rr@users.noreply.github.com> Raj Hammeer Singh Hada +Rajendra Matcha Ralph Soika +Raman Shinde Rand Xie Randall Fitzgerald Random Fly +rankaiyx rankaiyx +RapidMark <32768622+RapidMark@users.noreply.github.com> +Rares Vernica +Rashid Ul Islam <33536561+Ra5hidIslam@users.noreply.github.com> Raul Torres <138264735+rauletorresc@users.noreply.github.com> +ravel7524 <58877666+ravel7524@users.noreply.github.com> +Ravi Panchumarthy +Ray Xu <22774575+RayXu14@users.noreply.github.com> +RealOrko <45273739+RealOrko@users.noreply.github.com> redbeard +redfox <59549776+yaohengxu@users.noreply.github.com> Reese Levine +Reguna +rehan-10xengineer Reinforce-II +Rémy Mathieu Rémy O Rémy Oudompheng +ren <189031187+lathrys-at@users.noreply.github.com> Ren Xuancheng Renat Rene Leonhardt <65483435+reneleonhardt@users.noreply.github.com> @@ -1105,8 +1436,10 @@ Riccardo Orlando Riceball LEE Rich Dougherty Richard +Richard Davison Richard Kiss Richard Roberson +RichardScottOZ Rick G <26732651+TheFlipbook@users.noreply.github.com> Rickard Edén Rickard Hallerbäck @@ -1115,21 +1448,28 @@ Riley Stewart rimoliga <53384203+rimoliga@users.noreply.github.com> Rinne Rinne +Rithik Sharma RJ Adriaansen rmatif <66360289+rmatif@users.noreply.github.com> rmatif rmatif Robert Brisita <986796+rbrisita@users.noreply.github.com> Robert Collins +Robert Esclapez Robert Ormandi <52251610+ormandi@users.noreply.github.com> Robert Sung-wook Shin +robertomeroni <150194833+robertomeroni@users.noreply.github.com> Robey Holderith Robin Davidsson <40024429+R-Dson@users.noreply.github.com> Robyn Rőczey Barnabás <31726601+An0nie@users.noreply.github.com> RodriMora +Roger Chen Roger Meier +Rohan Jain <343499+crodjer@users.noreply.github.com> Rohanjames1997 +Rohit Mahesh <74331568+rohitmahesh1@users.noreply.github.com> +Roj234 <82699138+roj234@users.noreply.github.com> Roland <14355895+rbur0425@users.noreply.github.com> Romain Biessy Romain D <90720+Artefact2@users.noreply.github.com> @@ -1146,17 +1486,20 @@ Rowan Hart rspOverflow <217881046+rspOverflow@users.noreply.github.com> rtaluyev Ruan <47767371+ruanych@users.noreply.github.com> +ruanslv Ruben Ortlam Ruben Ortlam Ruchira Hasaranga Rudi Servo Ruikai Peng +Ruixiang Wang Ruixin Huang <18860020911@163.com> Rune <43761327+Rune-AI@users.noreply.github.com> runfuture RunningLeon RunningLeon Russyyds <161207317+Russyyds@users.noreply.github.com> +Ryan Goulden Ryan Landay Ryan Mangeno <160974989+ryan-mangeno@users.noreply.github.com> Ryder Wishart @@ -1164,7 +1507,9 @@ Ryuei s-goto-11 <206795233+s-goto-11@users.noreply.github.com> s8322 Saba Fallah <10401143+sfallah@users.noreply.github.com> +Saba Fallah Sachin Desai +Sachin Sharma safranowith SakuraUmi Salvador E. Tropea @@ -1173,18 +1518,31 @@ Sam Sam Malayek <12037535+SamMalayek@users.noreply.github.com> Sam Spilsbury Sam/Samuel <57896620+cern1710@users.noreply.github.com> +Samanvya Tripathi +SamareshSingh <97642706+ssam18@users.noreply.github.com> SAMI Sami Farin <3876865+Safari77@users.noreply.github.com> +Sami Kama Samuel Maynard +samuraieng <89817709+samuraieng@users.noreply.github.com> Sandro Hanea <40202887+sandrohanea@users.noreply.github.com> sandyiscool Sang-Kil Park +Sanjay Ahari Sascha Rogmann <59577610+srogmann@users.noreply.github.com> sasha0552 +Satinder Grewal +Satinder Grewal +SATISH K C <157192662+satishkc7@users.noreply.github.com> +Saurabh Dash <111897126+saurabhdash2512@users.noreply.github.com> SavicStefan <50296686+SavicStefan@users.noreply.github.com> Scott Fudally +ScrewTSW +scutler-nv Seb C <47074056+Sebby37@users.noreply.github.com> Sebastián A +Sebastian Dröge +Sebastian Dröge SebastianApel <13675545+SebastianApel@users.noreply.github.com> semidark Senemu <10880819+Senemu@users.noreply.github.com> @@ -1193,17 +1551,25 @@ Sergei Vorobyov Sergey Alirzaev Sergey Alirzaev Sergey Fedorov +Sergey Malinin Sergio López Sergio López +Sergiu <8598216+mzsergiu@users.noreply.github.com> serhii-nakon <57632032+serhii-nakon@users.noreply.github.com> Sertaç Özercan <852750+sozercan@users.noreply.github.com> +seryogakovalyov +Seungmin Kim <8457324+ehfd@users.noreply.github.com> SeungWon Jeong <65549245+redlion0929@users.noreply.github.com> +Seyoung Jeong ShadovvBeast Shagun Bera <141054835+notV3NOM@users.noreply.github.com> +Shahir BIn Zulfiker <119410932+aorko01@users.noreply.github.com> Shakhar Dasgupta +Shakhnazar Sailaukan <101112128+Sailaukan@users.noreply.github.com> Shakil Ahmed <44522075+ahmedshakill@users.noreply.github.com> shalinib-ibm Shane A +Shane Tran Whitmire <64436119+dogunbound@users.noreply.github.com> Shangning Xu <32517059+xushangning@users.noreply.github.com> shani-f Shankar @@ -1211,6 +1577,7 @@ Shanshan Shen <467638484@qq.com> shaofeiqi <109865877+shaofeiqi@users.noreply.github.com> shaofeiqi sharpHL <132747147+sharpHL@users.noreply.github.com> +Shaw Nguyen <49144872+mrshaw01@users.noreply.github.com> Shawn Gu Shawn yang <137684499+Yangxiaoz@users.noreply.github.com> Shelby Jenkins <47464908+ShelbyJenkins@users.noreply.github.com> @@ -1219,15 +1586,24 @@ shibe2 Shijie <821898965@qq.com> Shin-myoung-serp Shintarou Okada +shivamkumard-ctrl Shouyu <65317431+joeldushouyu@users.noreply.github.com> Shouzheng Liu <61452103+lshzh-ww@users.noreply.github.com> Shouzheng Liu +Shreya Jain +Shreya Jain +Shrivas Shankar <86219405+shrivasshankar@users.noreply.github.com> SHUAI YANG Shuichi Tsutsumi shun095 <8069181+shun095@users.noreply.github.com> Shunta Saito Shupei Fan Si1w <139008732+Si1w@users.noreply.github.com> +Sid Mohan <61345237+sidmohan0@users.noreply.github.com> +Sid Shaytay <2595088+SidShaytay@users.noreply.github.com> +Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> +Sigbjørn Skjæret +Sigbjørn Skjæret Sigbjørn Skjæret simevo Simon Redman @@ -1235,6 +1611,7 @@ Simon Willison simon886212 <37953122+simon886212@users.noreply.github.com> Simranjeet Singh <105192966+simrnsingh@users.noreply.github.com> singularity <12184989+singularity-s0@users.noreply.github.com> +Sirui He <143699303+SiruiHe@users.noreply.github.com> sirus20x6 Siwen Yu sjinzh @@ -1248,17 +1625,25 @@ Slava Primenko Slobodan Josic <127323561+slojosic-amd@users.noreply.github.com> Small Grass Forest SmartestWashingMachine +smugman-dot SnA1lGo <44647694+skrandy@users.noreply.github.com> snadampal <87143774+snadampal@users.noreply.github.com> SoftwareRenderer <138734813+SoftwareRenderer@users.noreply.github.com> Someone Someone Serge someone13574 <81528246+someone13574@users.noreply.github.com> +someoneinjd +Son H. Nguyen <33925625+nhs000@users.noreply.github.com> +Song Li +Sophon +Sou-ly <79574807+Sou-ly@users.noreply.github.com> Sourab Mangrulkar <13534540+pacman100@users.noreply.github.com> Spencer Sutton +sprayandwipe SRHMorris <69468379+SRHMorris@users.noreply.github.com> Srihari-mcw <96763064+Srihari-mcw@users.noreply.github.com> Srinivas Billa +srkizer ssweens <1149151+ssweens@users.noreply.github.com> standby24x7 staviq @@ -1267,9 +1652,11 @@ Stefan Sydow Ștefan-Gabriel Muscalu Steffen Röcker Stephan Walter +Stephen Cox Stephen Nichols Steve Bonds Steve Grubb +Steve Lhomme Steven Prichard Steven Roussey stevenkuang @@ -1279,6 +1666,8 @@ strawberrymelonpanda <152940198+strawberrymelonpanda@users.noreply.github.com> Suaj Carrot <72162667+SuajCarrot@users.noreply.github.com> sudhiarm Sukriti Sharma +Sumit Chatterjee <51856136+sumitchatterjee13@users.noreply.github.com> +Sundaram krishnan <104441812+sundaram123krishnan@users.noreply.github.com> SuperUserNameMan Sutou Kouhei Svetlozar Georgiev <55534064+sgeor255@users.noreply.github.com> @@ -1292,6 +1681,9 @@ takasurazeem takov751 <40316768+takov751@users.noreply.github.com> takuya kodama takuya kodama +Talha Adnan +Talha Can Havadar +Tamar tamarPal Tameem <113388789+AhmadTameem@users.noreply.github.com> Tamotsu Takahashi @@ -1304,56 +1696,79 @@ Taylor tc-mb <157115220+tc-mb@users.noreply.github.com> TecJesh Tei Home +Tekin Ertekin tempstudio <49735574+tempstudio@users.noreply.github.com> teo +texasich <101962694+texasich@users.noreply.github.com> texmex76 <40733439+texmex76@users.noreply.github.com> +tha80 <7176001+tha80@users.noreply.github.com> Thái Hoàng Tâm <75922889+RoyalHeart@users.noreply.github.com> Thammachart Chinvarapon <1731496+Thammachart@users.noreply.github.com> Thatcher Chamberlin +thecaptain789 <257642323+thecaptain789@users.noreply.github.com> Theia Vogel thement <40525767+thement@users.noreply.github.com> theo77186 theraininsky <76763719+theraininsky@users.noreply.github.com> +therealkenc Thérence <13496987+Royalphax@users.noreply.github.com> thewh1teagle <61390950+thewh1teagle@users.noreply.github.com> +Thiago Padilha Thibault Terrasson thom-dev-fr <161708450+thom-dev-fr@users.noreply.github.com> Thomas Germer <99991@users.noreply.github.com> Thomas Jarosch Thomas Klausner +Thomas LECONTE <161708450+thom-dev-fr@users.noreply.github.com> Thore Koritzius Thorsten Sommer TianHao324 <854531745@qq.com> TianHao324 Tianyue-Zhao +Tillerino Tim Miller Tim Neumann +Tim Neumann Tim Wang +timkhronos Timmy Knight Timothy Cronin <40186632+4imothy@users.noreply.github.com> Ting Lou Ting Lou Ting Sun +Titaniumtown tjohnman Tobias Lütke +Toby <25832191+aetherbird@users.noreply.github.com> +Todd Malsbary Todor Boinovski Tom C +Tom Hillbrunner Tom Jobbins <784313+TheBloke@users.noreply.github.com> +Tom Overlund +Tom Tan <29201606+intel00000@users.noreply.github.com> +Tom Vaucourt <34662901+T0mSIlver@users.noreply.github.com> Tomas Tomáš Pazdiora +Tomeamis Tony Wasserka <4840017+neobrain@users.noreply.github.com> toyer <2042519524@qq.com> TrevorS +TriDefender triplenom <79777178+triplenom@users.noreply.github.com> Tristan Druyen Tristan Ross Trivikram Kamat <16024985+trivikr@users.noreply.github.com> +Trivikram Reddy <127072883+trivikram-reddy1@users.noreply.github.com> +Ts-sound <44093942+Ts-sound@users.noreply.github.com> tslmy tt <291400568@qq.com> +Tunahan <115956684+tnhnyzc@users.noreply.github.com> Tungsten842 <886724vf@anonaddy.me> Tungsten842 Tushar tv1wnd <55383215+tv1wnd@users.noreply.github.com> +tyronecai ubergarm ubik2 UEXTM.com <84163508+uextm@users.noreply.github.com> @@ -1363,27 +1778,34 @@ uint256_t Ujjawal Panchal <31011628+Ujjawal-K-Panchal@users.noreply.github.com> Ulrich Drepper unbounded +unraido <127105806+unraido@users.noreply.github.com> uvos uvos uvos Uzo Nweke Vaibhav Srivastav Val Kharitonov +ValdikSS Valentin Konovalov Valentin Mamedov <45292985+Inf1delis@users.noreply.github.com> Valentyn Bezshapkin <61702053+valentynbez@users.noreply.github.com> +Valeriy Dubov Vali Malinoiu <0x4139@gmail.com> valiray <133289098+valiray@users.noreply.github.com> vb Vedran Miletić +Vexxie Victor <194116445+dodekapod@users.noreply.github.com> Victor Nogueira +Victor Villar Victor Z. Peng Viet-Anh NGUYEN (Andrew) +viggy <70774793+vignesh191@users.noreply.github.com> vik Ville Vesilehto Vineel Abhinav <131174187+vineelabhinav@users.noreply.github.com> Vinesh Janarthanan <36610342+VJHack@users.noreply.github.com> +Vinicios Lugli Vinkal virajwad <84867530+virajwad@users.noreply.github.com> viric @@ -1396,6 +1818,7 @@ Vladimir Vladimir Malyutin Vladimir Vuksanovic <109677816+vvuksanovic@users.noreply.github.com> Vladimir Zorin +Vladislav Vladislav Sayapin <70110788+v-sayapin@users.noreply.github.com> vmobilis <75476228+vmobilis@users.noreply.github.com> vodkaslime <646329483@qq.com> @@ -1404,26 +1827,34 @@ Volodymyr Vitvitskyi <72226+signalpillar@users.noreply.github.com> vvhg1 <94630311+vvhg1@users.noreply.github.com> vxiiduu <73044267+vxiiduu@users.noreply.github.com> Wagner Bruna +Wallentri Wang Qin <37098874+wangqin0@users.noreply.github.com> Wang Ran (汪然) Wang Weixuan +Wang Zhiyu WangHaoranRobin <56047610+WangHaoranRobin@users.noreply.github.com> wangshuai09 <391746016@qq.com> wbpxre150 <100937007+wbpxre150@users.noreply.github.com> wbtek <171302111+wbtek@users.noreply.github.com> +Wei Wang Weird Constructor Weizhao Ouyang Weizhao Ouyang Welby Seely welix +wencan +wendadawen <130649302+wendadawen@users.noreply.github.com> Wentai Zhang whoreson <139810751+whoreson@users.noreply.github.com> Wilken Gottwalt <12194808+wgottwalt@users.noreply.github.com> +will-lms WillCorticesAI <150854901+WillCorticesAI@users.noreply.github.com> william pan <61359596+wp4032@users.noreply.github.com> William Tambellini William Tambellini +willjoha Willy Tarreau +Winston Ma woachk <24752637+woachk@users.noreply.github.com> wonjun Jang woodx <124784234+woodx9@users.noreply.github.com> @@ -1435,6 +1866,7 @@ wsbagnsv1 Wu Jian Ping Wu Jian Ping wwoodsTM <104587230+wwoodsTM@users.noreply.github.com> +Wyatt Caldwell <218154709+Detensable@users.noreply.github.com> wzy <32936898+Freed-Wu@users.noreply.github.com> xaedes xaedes @@ -1453,27 +1885,43 @@ Xingchen Song(宋星辰) Xinpeng Dou <15529241576@163.com> Xinpeng Dou <81913537+Dou-Git@users.noreply.github.com> xloem <0xloem@gmail.com> +xris99 <79798089+xris99@users.noreply.github.com> Xuan Son Nguyen Xuan-Son Nguyen Xuan-Son Nguyen +y198 <90976397+y198nt@users.noreply.github.com> yael-works <106673277+yael-works@users.noreply.github.com> YaelGitAccount <38328157276@mby.co.il> YaelLogic Yaiko +Yakine Tahtah <96926916+ReinforcedKnowledge@users.noreply.github.com> YangLe yangli2 Yann Follet <131855179+YannFollet@users.noreply.github.com> +Yanzhao Wang +Yarden Tal +YardenTal44 Yaroslav +Yash Raj Pandey <55940078+devYRPauli@users.noreply.github.com> Yavor Ivanov Yazan Agha-Schrader Ycros <18012+ycros@users.noreply.github.com> YehuditE +Yes You Can Have Your Own <188969017+yychyo@users.noreply.github.com> +yggdrasil75 Yibo Cai Yibo Cai +YiChen Lv <63285796+forforever73@users.noreply.github.com> yifant-code +Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com> +yikechayedan <2935171085@qq.com> Yiming Cui Yishuo Wang +Yiwei Shao <44545837+njsyw1997@users.noreply.github.com> ymcki <84055651+ymcki@users.noreply.github.com> +ynankani +Yongmin Yoo 유용민 +Yongyue Sun Yoshi Suhara Yoshi Suhara Yoshi_likes_e4 <104140648+pt13762104@users.noreply.github.com> @@ -1494,19 +1942,27 @@ yuri@FreeBSD Yusuf Kağan Hanoğlu Yuval Peled <31162840+Yuval-Peled@users.noreply.github.com> Yuxuan Zhang <2448370773@qq.com> +yzyyzyhhh <96101183+happyyzy@users.noreply.github.com> Z +Zach Winter +Zack Li <39573601+zhiyuan8@users.noreply.github.com> Zagaj zakkor Zane Shannon Zay <95888118+isaiahbjork@users.noreply.github.com> +zduford Zenix +ZeroV0LT Zhang Peiyuan zhangkaihuo +zhangrunda +zhangtao2-1 <478679312@qq.com> ZHAOKAI WANG Zheng.Deng <32841220+dengzheng-cloud@users.noreply.github.com> zhentaoyu Zhenwei Jin <109658203+kylo5aby@users.noreply.github.com> Zheyuan Chen +Zhihao "Zephyr" Yao Zhiyong Wang <85110830+ravenouse@users.noreply.github.com> Zhiyuan Li Zhiyuan Li @@ -1515,5 +1971,10 @@ zhouwg ZhouYuChen Ziad Ben Hadj-Alouane Ziang Wu <97337387+ZiangWu-77@users.noreply.github.com> +ZihaoMu +Zijun Yu +Zijun Yu +zql <37731799+zqlcode@users.noreply.github.com> zrm Zsapi +zzzzwc diff --git a/README.md b/README.md index 5960ef684..85726c8c0 100644 --- a/README.md +++ b/README.md @@ -7,10 +7,11 @@ LLM inference in C/C++ [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://opensource.org/licenses/MIT) -[![Release](https://img.shields.io/github/v/release/ggml-org/llama.cpp)](https://github.com/ggml-org/llama.cpp/releases) +[![Release](https://img.shields.io/github/v/release/ggml-org/llama.cpp?filter=v*)](https://github.com/ggml-org/llama.cpp/releases?q=tag:v0) +[![Nightly](https://img.shields.io/github/v/release/ggml-org/llama.cpp?label=nightly)](https://github.com/ggml-org/llama.cpp/releases) [![Server](https://github.com/ggml-org/llama.cpp/actions/workflows/server.yml/badge.svg)](https://github.com/ggml-org/llama.cpp/actions/workflows/server.yml) -[![Docker](https://github.com/ggml-org/llama.cpp/actions/workflows/docker.yml/badge.svg)](https://github.com/ggml-org/llama.cpp/actions/workflows/docker.yml) -[![Winget](https://github.com/ggml-org/llama.cpp/actions/workflows/winget.yml/badge.svg)](https://github.com/ggml-org/llama.cpp/actions/workflows/winget.yml) +[![Docker](https://img.shields.io/github/actions/workflow/status/ggml-org/llama.cpp/docker.yml?label=Docker)](https://github.com/ggml-org/llama.cpp/actions/workflows/docker.yml) +[![Winget](https://img.shields.io/github/actions/workflow/status/ggml-org/llama.cpp/winget.yml?label=Winget)](https://github.com/ggml-org/llama.cpp/actions/workflows/winget.yml) [manifesto](https://github.com/ggml-org/llama.cpp/discussions/205) / [ggml](https://github.com/ggml-org/ggml) / [ops](https://github.com/ggml-org/llama.cpp/blob/master/docs/ops.md) / [maintainer PRs](https://github.com/ggml-org/llama.cpp/issues?q=is%3Apr%20is%3Aopen%20draft%3AFalse%20(author%3Argerganov%20OR%20author%3AKitaitiMakoto%20OR%20author%3Adanbev%20OR%20author%3Aaldehir%20OR%20author%3Amax-krasnyansky%20OR%20author%3ACISC%20OR%20author%3Aggerganov%20OR%20author%3Aam17an%20OR%20author%3Abartowski1182%20OR%20author%3Ahipudding%20OR%20author%3AServeurpersoCom%20OR%20author%3Apwilkin%20OR%20author%3Areeselevine%20OR%20author%3Angxson%20OR%20author%3Ajeffbolznv%20OR%20author%3A0cc4m%20OR%20author%3Aangt%20OR%20author%3AIMbackK%20OR%20author%3Aarthw%20OR%20author%3AJohannesGaessler%20OR%20author%3AORippler%20OR%20author%3Aruixiang63%20OR%20author%3Axctan%20OR%20author%3Aallozaur%20OR%20author%3Ayomaytk%20OR%20author%3Aaendk%20OR%20author%3Agaugarg-nv%20OR%20author%3Ataronaeo%20OR%20author%3Aforforever73%20OR%20author%3Alhez%20OR%20author%3Anetrunnereve%20OR%20author%3Afairydreaming)%20sort%3Aupdated-desc) / [compile times](https://github.com/ggml-org/llama.cpp-dev/blob/master/README-compile-times.md) / [lib llama API](https://github.com/ggml-org/llama.cpp/issues/9289) / [llama-server REST API](https://github.com/ggml-org/llama.cpp/issues/9291) From afd439df1f081ce0f6bc76965aa0a0ed762ecc91 Mon Sep 17 00:00:00 2001 From: Thiago Padilha Date: Tue, 18 Aug 2026 10:15:22 -0300 Subject: [PATCH 14/48] unicode : include '~' in collapsed symbol class (#26972) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The collapsed \p{S} class was missing '~', which split " ~" into separate pre-tokens and prevented the Ġ~ BPE merge used by DeepSeek V4. This caused re-tokenized prompts to diverge from sampled tokens and broke KV cache reuse. Assisted-by: Codex --- src/unicode.cpp | 2 +- tests/CMakeLists.txt | 2 ++ tests/test-unicode.cpp | 24 ++++++++++++++++++++++++ 3 files changed, 27 insertions(+), 1 deletion(-) create mode 100644 tests/test-unicode.cpp diff --git a/src/unicode.cpp b/src/unicode.cpp index b02ecdc93..93996f9dd 100644 --- a/src/unicode.cpp +++ b/src/unicode.cpp @@ -1241,7 +1241,7 @@ std::vector unicode_regex_split(const std::string & text, const std { unicode_cpt_flags::LETTER, "\x41-\x5A\x61-\x7A" }, // A-Za-z { unicode_cpt_flags::PUNCTUATION, "\x21-\x23\x25-\x2A\x2C-\x2F\x3A-\x3B\x3F-\x40\\\x5B-\\\x5D\x5F\\\x7B\\\x7D" }, // !-#%-*,-/:-;?-@\[-\]_\{\} { unicode_cpt_flags::ACCENT_MARK, "" }, // no sub-128 codepoints - { unicode_cpt_flags::SYMBOL, "\\\x24\\\x2B\x3C-\x3E\x5E\x60\\\x7C" }, // $+<=>^`| + { unicode_cpt_flags::SYMBOL, "\\\x24\\\x2B\x3C-\x3E\x5E\x60\\\x7C\\\x7E" }, // $+<=>^`|~ }; // compute collapsed codepoints only if needed by at least one regex diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 3ee51b519..0db7fd9ef 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -116,6 +116,8 @@ function(llama_build_and_test source) set_property(TEST ${TEST_TARGET} PROPERTY LABELS ${LLAMA_TEST_LABEL}) endfunction() +llama_build_and_test(test-unicode.cpp) + # build test-tokenizer-0 target once and add many tests llama_build(test-tokenizer-0.cpp) diff --git a/tests/test-unicode.cpp b/tests/test-unicode.cpp new file mode 100644 index 000000000..2347d9000 --- /dev/null +++ b/tests/test-unicode.cpp @@ -0,0 +1,24 @@ +#include "../src/unicode.h" + +#include +#include +#include + +int main() { + const std::vector regex_exprs = { + "[~][A-Za-z]+| ?[\\p{S}]+|\\s+", + }; + const std::vector expected = { " ~", "foo" }; + const auto actual = unicode_regex_split(" ~foo", regex_exprs, false); + + if (actual != expected) { + fprintf(stderr, "unexpected split:"); + for (const auto & piece : actual) { + fprintf(stderr, " [%s]", piece.c_str()); + } + fprintf(stderr, "\n"); + return 1; + } + + return 0; +} From 0882c7bc89074017c6a2a3149eae46929cb32ab3 Mon Sep 17 00:00:00 2001 From: Titaniumtown Date: Tue, 18 Aug 2026 06:21:25 -0700 Subject: [PATCH 15/48] sycl: honor GGML_HINT_SRC0_IS_HADAMARD (#27298) Kernel is a port of `ggml-cuda/fwht.cu` (us/run, median): ``` m x n x k GEMM FWHT speedup 64 x 1 x 64 10.20 2.93 3.48x 64 x 2048 x 64 10.75 2.71 3.97x 128 x 1 x 128 10.33 2.88 3.59x 128 x 32 x 128 9.20 2.77 3.33x 128 x 2048 x 128 16.46 2.76 5.95x 256 x 1 x 256 10.19 2.77 3.68x 256 x 2048 x 256 16.69 3.41 4.89x 512 x 2048 x 512 54.16 12.89 4.20x ``` --- ggml/src/ggml-sycl/fwht.cpp | 119 +++++++++++++++++++++++++++++++ ggml/src/ggml-sycl/fwht.hpp | 12 ++++ ggml/src/ggml-sycl/ggml-sycl.cpp | 13 ++++ 3 files changed, 144 insertions(+) create mode 100644 ggml/src/ggml-sycl/fwht.cpp create mode 100644 ggml/src/ggml-sycl/fwht.hpp diff --git a/ggml/src/ggml-sycl/fwht.cpp b/ggml/src/ggml-sycl/fwht.cpp new file mode 100644 index 000000000..2312b3d13 --- /dev/null +++ b/ggml/src/ggml-sycl/fwht.cpp @@ -0,0 +1,119 @@ +#include "fwht.hpp" + +#include + +template +static void fwht_kernel(const float * __restrict__ src, float * __restrict__ dst, const int64_t n_rows, + const float scale, const sycl::nd_item<2> & item) { + const sycl::sub_group sg = item.get_sub_group(); + + const int64_t r = item.get_global_id(0); + if (r >= n_rows) { + return; + } + + src += r * N; + dst += r * N; + + constexpr int el_w = N / WARP_SIZE; + static_assert(el_w >= 1 && N % WARP_SIZE == 0, "row must be a whole number of sub-group widths"); + + float reg[el_w]; + const int lane = sg.get_local_linear_id(); + +#pragma unroll + for (int i = 0; i < el_w; ++i) { + reg[i] = src[i * WARP_SIZE + lane] * scale; + } + + // Butterflies inside the sub-group. The partner of a lane with bit h clear is the + // lower index of the pair, so it takes the sum and the upper takes lower - upper. +#pragma unroll + for (int h = 1; h < WARP_SIZE; h *= 2) { +#pragma unroll + for (int j = 0; j < el_w; ++j) { + const float val = reg[j]; + const float val2 = dpct::permute_sub_group_by_xor(sg, val, h, WARP_SIZE); + + reg[j] = (lane & h) == 0 ? val + val2 : val2 - val; + } + } + + // Butterflies across registers: h is a multiple of WARP_SIZE, so the partner of + // element i*WARP_SIZE + lane lives in reg[i + h/WARP_SIZE] on the same lane. +#pragma unroll + for (int h = WARP_SIZE; h < N; h *= 2) { + const int step = h / WARP_SIZE; +#pragma unroll + for (int j = 0; j < el_w; j += 2 * step) { +#pragma unroll + for (int k = 0; k < step; ++k) { + const float x = reg[j + k]; + const float y = reg[j + k + step]; + + reg[j + k] = x + y; + reg[j + k + step] = x - y; + } + } + } + +#pragma unroll + for (int i = 0; i < el_w; ++i) { + dst[i * WARP_SIZE + lane] = reg[i]; + } +} + +template +static void launch_fwht(const float * src, float * dst, const int64_t n_rows, const float scale, + dpct::queue_ptr stream) { + constexpr int rows_per_block = 4; + + const int64_t num_blocks = (n_rows + rows_per_block - 1) / rows_per_block; + + // dim 1 is the fastest-varying, so a sub-group is exactly one row's WARP_SIZE lanes. + const sycl::range<2> global(num_blocks * rows_per_block, WARP_SIZE); + const sycl::range<2> local(rows_per_block, WARP_SIZE); + + stream->parallel_for(sycl::nd_range<2>(global, local), + [=](sycl::nd_item<2> item) [[sycl::reqd_sub_group_size(WARP_SIZE)]] { + fwht_kernel(src, dst, n_rows, scale, item); + }); +} + +bool ggml_sycl_op_fwht(ggml_backend_sycl_context & ctx, const ggml_tensor * src, ggml_tensor * dst) { + if (src->type != GGML_TYPE_F32 || dst->type != GGML_TYPE_F32) { + return false; + } + if (!ggml_are_same_shape(src, dst)) { + return false; + } + if (!ggml_is_contiguous(src) || !ggml_is_contiguous(dst)) { + return false; + } + + const int n = (int) src->ne[0]; + const int64_t rows = ggml_nrows(src); + + const float * src_d = (const float *) src->data; + float * dst_d = (float *) dst->data; + dpct::queue_ptr stream = ctx.stream(); + + const float scale = 1.0f / std::sqrt((float) n); + + switch (n) { + case 64: + launch_fwht<64>(src_d, dst_d, rows, scale, stream); + return true; + case 128: + launch_fwht<128>(src_d, dst_d, rows, scale, stream); + return true; + case 256: + launch_fwht<256>(src_d, dst_d, rows, scale, stream); + return true; + case 512: + launch_fwht<512>(src_d, dst_d, rows, scale, stream); + return true; + default: + return false; + } +} diff --git a/ggml/src/ggml-sycl/fwht.hpp b/ggml/src/ggml-sycl/fwht.hpp new file mode 100644 index 000000000..cd238cfaf --- /dev/null +++ b/ggml/src/ggml-sycl/fwht.hpp @@ -0,0 +1,12 @@ +#ifndef GGML_SYCL_FWHT_HPP +#define GGML_SYCL_FWHT_HPP + +#include "common.hpp" + +// Fast Walsh-Hadamard transform, the fast path for a MUL_MAT whose src0 ggml has +// tagged GGML_HINT_SRC0_IS_HADAMARD. src0 is not read at all. Returns false if the +// shape is not one this can serve, in which case the caller must fall through to the +// ordinary mat-mul dispatch. +bool ggml_sycl_op_fwht(ggml_backend_sycl_context & ctx, const ggml_tensor * src, ggml_tensor * dst); + +#endif // GGML_SYCL_FWHT_HPP diff --git a/ggml/src/ggml-sycl/ggml-sycl.cpp b/ggml/src/ggml-sycl/ggml-sycl.cpp index 5416d4f0f..d31df611a 100644 --- a/ggml/src/ggml-sycl/ggml-sycl.cpp +++ b/ggml/src/ggml-sycl/ggml-sycl.cpp @@ -58,6 +58,7 @@ #include "ggml-sycl/backend.hpp" #include "ggml-sycl/common.hpp" #include "ggml-sycl/element_wise.hpp" +#include "ggml-sycl/fwht.hpp" #include "ggml-sycl/gemm.hpp" #include "ggml-sycl/getrows.hpp" #include "ggml-sycl/norm.hpp" @@ -4473,6 +4474,18 @@ static bool can_use_mul_mat_vec_q(const ggml_tensor * src0, const ggml_tensor * static void ggml_sycl_mul_mat(ggml_backend_sycl_context & ctx, const ggml_tensor * src0, const ggml_tensor * src1, ggml_tensor * dst) { scope_op_debug_print scope_dbg_print(__func__, dst, /*num_src=*/2); + + // Handle HADAMARAD hint given from further up the pipeline and pass it to the correct + // kernel. + // + // The op check is not redundant: this backend also routes MUL_MAT_ID through here with a + // stack copy of dst, which carries MUL_MAT_ID's own op_params. ggml_mul_mat_set_hint() + // asserts GGML_OP_MUL_MAT for the same reason. + if (dst->op == GGML_OP_MUL_MAT && ggml_get_op_params_i32(dst, 1) == GGML_HINT_SRC0_IS_HADAMARD && + ggml_sycl_op_fwht(ctx, src1, dst)) { + return; + } + const bool split = ggml_backend_buffer_is_sycl_split(src0->buffer); int64_t min_compute_capability = INT_MAX; From 0596704284bffe8350eef4a9510316258ed6b87a Mon Sep 17 00:00:00 2001 From: Ed Addario <29247825+EAddario@users.noreply.github.com> Date: Tue, 18 Aug 2026 15:22:32 +0100 Subject: [PATCH 16/48] quant : Optimise memory usage by evicting weights after processing each layer (#22877) * Evict weights from memory after processing each layer * Revert changes * Move unmap to libllama * Unmap weights offloaded to backend * Change member's constness * Remove unmap weights offloaded to backend --- src/llama-model-loader.cpp | 5 +++++ src/llama-model-loader.h | 3 +++ src/llama-quant.cpp | 6 +++++- tools/perplexity/perplexity.cpp | 1 - 4 files changed, 13 insertions(+), 2 deletions(-) diff --git a/src/llama-model-loader.cpp b/src/llama-model-loader.cpp index 1ca698704..9b22cb05f 100644 --- a/src/llama-model-loader.cpp +++ b/src/llama-model-loader.cpp @@ -1395,6 +1395,11 @@ void llama_model_loader::get_mapping_range(size_t * first, size_t * last, void * } } +void llama_model_loader::unmap_weight(const llama_tensor_weight & w) const { + if (!use_mmap) { return; } + mappings.at(w.idx)->unmap_fragment(w.offs, w.offs + ggml_nbytes(w.tensor)); +} + void llama_model_loader::load_data_for(struct ggml_tensor * cur) const { const auto & w = require_weight(ggml_get_name(cur)); diff --git a/src/llama-model-loader.h b/src/llama-model-loader.h index d6b31c231..e9fe3592d 100644 --- a/src/llama-model-loader.h +++ b/src/llama-model-loader.h @@ -194,6 +194,9 @@ struct llama_model_loader { void get_mapping_range(size_t * first, size_t * last, void ** addr, int idx, ggml_context * ctx) const; + // release a weight's mmap pages + void unmap_weight(const llama_tensor_weight & w) const; + // for backwards compatibility, does not support ggml-backend void load_data_for(struct ggml_tensor * cur) const; diff --git a/src/llama-quant.cpp b/src/llama-quant.cpp index 7f99e96bc..20252815d 100644 --- a/src/llama-quant.cpp +++ b/src/llama-quant.cpp @@ -1270,7 +1270,7 @@ static void llama_model_quantize_impl(const std::string & fname_inp, const std:: total_size_org += tensor_size; total_size_new += new_size; - // update the gguf meta data as we go + // update the gguf metadata as we go gguf_set_tensor_type(ctx_outs[cur_split].get(), metadata[i].name.c_str(), new_type); GGML_ASSERT(gguf_get_tensor_size(ctx_outs[cur_split].get(), gguf_find_tensor(ctx_outs[cur_split].get(), metadata[i].name.c_str())) == new_size); gguf_set_tensor_data(ctx_outs[cur_split].get(), metadata[i].name.c_str(), new_data); @@ -1278,6 +1278,10 @@ static void llama_model_quantize_impl(const std::string & fname_inp, const std:: // write tensor data + padding fout.write((const char *) new_data, new_size); zeros(fout, GGML_PAD(new_size, align) - new_size); + + // unmap the tensor to free memory + if (ml.use_mmap) { ml.unmap_weight(weight); } + } // no --dry-run } // main loop diff --git a/tools/perplexity/perplexity.cpp b/tools/perplexity/perplexity.cpp index 92f88306c..ba41287d8 100644 --- a/tools/perplexity/perplexity.cpp +++ b/tools/perplexity/perplexity.cpp @@ -2023,7 +2023,6 @@ int llama_perplexity(int argc, char ** argv) { } const int32_t n_ctx = params.n_ctx; - if (n_ctx <= 0) { LOG_ERR("%s: perplexity tool requires '--ctx-size' > 0\n", __func__); return 1; From 04b569142da23d91beca090a99098d592d3f3c80 Mon Sep 17 00:00:00 2001 From: Niklas Wenzel Date: Tue, 18 Aug 2026 16:23:43 +0200 Subject: [PATCH 17/48] common: share thread pools when `n_threads` differ (#27138) --- common/common.cpp | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/common/common.cpp b/common/common.cpp index cea6d3f5c..0f2f01ad0 100644 --- a/common/common.cpp +++ b/common/common.cpp @@ -1750,6 +1750,18 @@ struct ggml_threadpool_params ggml_threadpool_params_from_cpu_params(const commo return tpp; } +namespace { + +bool can_share_threadpool(const ggml_threadpool_params & tpp1, const ggml_threadpool_params & tpp2) { + // n_threads does not matter -> we'll use what's larger + ggml_threadpool_params tpp_comparison = tpp1; + tpp_comparison.n_threads = tpp2.n_threads; + + return ggml_threadpool_params_match(&tpp_comparison, &tpp2); +} + +} // namespace + common_threadpools::~common_threadpools() { if (!free_fn) { return; @@ -1778,7 +1790,9 @@ void common_threadpools::init(llama_context * ctx, const common_params & params) struct ggml_threadpool_params tpp = ggml_threadpool_params_from_cpu_params(params.cpuparams); - if (!ggml_threadpool_params_match(&tpp, &tpp_batch)) { + if (can_share_threadpool(tpp, tpp_batch)) { + tpp.n_threads = std::max(tpp.n_threads, tpp_batch.n_threads); + } else { threadpool_batch = ggml_threadpool_new_fn(&tpp_batch); if (!threadpool_batch) { COM_WRN("batch threadpool create failed : n_threads %d\n", tpp_batch.n_threads); From fdf4c6460477e2761faac4527fc6266e48a893f0 Mon Sep 17 00:00:00 2001 From: Aleksander Grygier Date: Tue, 18 Aug 2026 16:37:26 +0200 Subject: [PATCH 18/48] ui: Stores consolidation refactor (#27238) * ui: Remove dead code from stores - persisted() helper was exported but never used - messageUpdateCallback / registerMessageUpdateCallback were never wired up - conversationsStore.initialize() alias, single caller moved to init() * ui: Merge device, theme and viewport into a single deviceStore All three are reactive browser-environment signals, now exposed as one class store: deviceStore.isMobile, deviceStore.isIOSDevice / isIOSSafari / isWKWebView / isStandalone and deviceStore.systemTheme.isDark. The systemTheme name disambiguates the OS preference from the user theme preference in settingsStore. Drops the unused viewport export (only isMobile was consumed). * ui: Merge build info into version store One VersionStore class with build (llama.cpp build number from build.json) and frontend (PWA version from _app/version.json), matching the class pattern of the other stores. * ui: Colocate context gauge popup state with its components The gauge popup state is local UI state shared only by the ChatFormContextGauge subtree, so it lives next to its consumers instead of the app-scope stores barrel. --- .../ChatFormActionsAdd.svelte | 4 +- .../ChatFormActionModels.svelte | 10 +- .../ChatFormContextGauge.svelte | 7 +- .../ContextGaugeDetails.svelte | 2 +- .../ContextGaugePopup.svelte | 7 +- .../gauge-popup.svelte.ts} | 0 .../ChatFormInput/ChatFormInputBasic.svelte | 4 +- .../ChatFormInput/ChatFormInputRich.svelte | 6 +- .../ChatFormPickerMention.svelte | 4 +- .../ChatMessage/ChatMessage.svelte | 4 +- .../app/chat/ChatScreen/ChatScreen.svelte | 30 ++--- .../app/chat/ChatScreen/ChatScreenForm.svelte | 6 +- .../SidebarNavigation.svelte | 22 ++-- .../SidebarNavigationActions.svelte | 8 +- tools/ui/src/lib/hooks/use-pwa.svelte.ts | 2 +- tools/ui/src/lib/stores/build-info.svelte.ts | 45 -------- tools/ui/src/lib/stores/chat.svelte.ts | 3 - .../ui/src/lib/stores/conversations.svelte.ts | 25 ---- tools/ui/src/lib/stores/device.svelte.ts | 107 +++++++++++------- tools/ui/src/lib/stores/index.ts | 22 +--- tools/ui/src/lib/stores/persisted.svelte.ts | 51 --------- tools/ui/src/lib/stores/settings.svelte.ts | 4 +- tools/ui/src/lib/stores/theme.svelte.ts | 14 --- tools/ui/src/lib/stores/version.svelte.ts | 67 +++++++---- tools/ui/src/lib/stores/viewport.svelte.ts | 9 -- tools/ui/src/routes/(chat)/+page.svelte | 2 +- tools/ui/src/routes/+layout.svelte | 15 ++- tools/ui/src/routes/search/+page.svelte | 4 +- 28 files changed, 181 insertions(+), 303 deletions(-) rename tools/ui/src/lib/{stores/context-gauge-popup.svelte.ts => components/app/chat/ChatForm/ChatFormContextGauge/gauge-popup.svelte.ts} (100%) delete mode 100644 tools/ui/src/lib/stores/build-info.svelte.ts delete mode 100644 tools/ui/src/lib/stores/persisted.svelte.ts delete mode 100644 tools/ui/src/lib/stores/theme.svelte.ts delete mode 100644 tools/ui/src/lib/stores/viewport.svelte.ts diff --git a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionsAdd.svelte b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionsAdd.svelte index 47bdb47a4..b2581f11e 100644 --- a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionsAdd.svelte +++ b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionsAdd.svelte @@ -2,10 +2,10 @@ import ChatFormActionAddButton from './ChatFormActionAddButton.svelte'; import ChatFormActionAddDropdown from './ChatFormActionAddDropdown.svelte'; import ChatFormActionAddSheet from './ChatFormActionAddSheet.svelte'; - import { isMobile } from '$lib/stores'; + import { deviceStore } from '$lib/stores'; -{#if isMobile.current} +{#if deviceStore.isMobile} {#snippet trigger({ disabled, onclick })} diff --git a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionModels.svelte b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionModels.svelte index fad223a98..689ef8c3a 100644 --- a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionModels.svelte +++ b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionModels.svelte @@ -1,6 +1,12 @@ -{#if isMobile.current} +{#if deviceStore.isMobile} import ContextGaugeDial from './ContextGaugeDial.svelte'; - import { useContextGauge } from '$lib/hooks/use-context-gauge.svelte'; import { - chatStore, - conversationsStore, gaugeTriggerClick, gaugeTriggerEnter, gaugeTriggerKeydown, gaugeTriggerLeave, gaugeTriggerPointerDown - } from '$lib/stores'; + } from './gauge-popup.svelte'; + import { useContextGauge } from '$lib/hooks/use-context-gauge.svelte'; + import { chatStore, conversationsStore } from '$lib/stores'; import { untrack } from 'svelte'; const gauge = useContextGauge(); diff --git a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormContextGauge/ContextGaugeDetails.svelte b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormContextGauge/ContextGaugeDetails.svelte index eaaba69de..1153c70fd 100644 --- a/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormContextGauge/ContextGaugeDetails.svelte +++ b/tools/ui/src/lib/components/app/chat/ChatForm/ChatFormContextGauge/ContextGaugeDetails.svelte @@ -1,9 +1,9 @@