mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2026-09-14 18:58:52 +02:00
Daniel Bevenius
3bcfeb700f
* scripts : add initial profiling script (wip) * src : add precompile headers (PCH) for models.h * common : add common.h as PCH * ggml : add PCH for ggml-impl.h * mtmd : use PCH for models.h * scripts : add script to build with Server/Tools/Tests * server : add PCH for common.h * docs: add profiling progress notes (wip) * ggml : add exclude for GCC + SVE on ARM Refs: https://github.com/ggml-org/llama.cpp/actions/runs/33393906061/job/99493756214?pr=28091 * ggml : attempt to fix use of std::hardware_destructive_inference_size Refs: https://github.com/ggml-org/llama.cpp/actions/runs/33396221677/job/99501265689?pr=28091 * squash! ggml : attempt to fix use of std::hardware_destructive_inference_size Add a version check for GCC 12 to conditionally apply the `-Winterference-size` pragma. * editorconfig : exclude profiling reports dir This directory will not be included in the merge later and this commit can be ignore at that point. Just fixing to keep CI happy. * ggml : skip PCH for gcc on non-x86 architectures * tests : add PCH for peg-parser/tests.h There are 7 peg-parser tests that can share one PCH instead of then each parsing the full tests.h. * common : add PCH for chat.h * docs : update linux build profiling full results Just updating after a number of PCH additions. These are not exact figures and will vary a bit from run to run, but they give a general idea of the performance impact of PCH. * cmake : introduce unity build for models This commit introduces a unity build for the models to improve compilation time. The improvements were roughly the following: ```console +------------------------+-----+------------+------------+------------+ | Build | TUs | Frontend | Backend | Total | +------------------------+-----+------------+------------+------------+ | Full, master | 396 | 811.0 s | 692.2 s | 1,503.2 s | | Full, with PCH | 405 | 380.0 s | 664.7 s | 1,044.7 s | | Full, with PCH + UB | 264 | 357.7 s | 635.7 s | 993.4 s | +------------------------+-----+------------+------------+------------+ TU = Translation Unit. Full = includes Server, Tools, and Tests. PCH = precompiled headers. UB = unity build for models. ``` * docs : update linux profiling table with unitiy build results * docs : update mac profiling results to include unity build [no ci] * docs: remove profiling reports * scripts : merge build profile scripts into one script I was lazy before and just copied the first script to enable Tests, Server, and Tools. This now merges them into a single script. * Revert "editorconfig : exclude profiling reports dir" [no ci] This reverts commit 2922a12118a0730d2f7632bcba265b44a0856c59. * src : rename ggml_view_2d_slice to gemma3n_view_2d_slice This is to be consistent with the rename in gemma4.cpp which was required to avoid a name clash. * cmake : add build profile script for windows [no ci] This commit adds a port of the scripts/build-profile.sh script to windows powershell. This was developed on Windows on ARM but should work on X64 as well but needs to be tested there as well.
123 lines
4.4 KiB
Markdown
123 lines
4.4 KiB
Markdown
## Build profiling
|
|
This page is a working document for analyzing the current build and try to
|
|
identify ways to improve the build time.
|
|
|
|
### Requirements
|
|
The profiling script requires clang to be used as the compiler tool chain and
|
|
also requires that ClangBuildAnalyzer is installed.
|
|
|
|
Mac:
|
|
```console
|
|
brew install clang-build-analyzer
|
|
```
|
|
|
|
Linux:
|
|
```console
|
|
git clone https://github.com/aras-p/ClangBuildAnalyzer.git
|
|
cd ClangBuildAnalyzer
|
|
cmake -B build -DCMAKE_BUILD_TYPE=Release
|
|
cmake --build build -j$(nproc)
|
|
sudo cp build/ClangBuildAnalyzer /usr/local/bin/
|
|
```
|
|
|
|
Windows: install LLVM/clang and Ninja (e.g. via the
|
|
[LLVM releases page](https://github.com/llvm/llvm-project/releases) and
|
|
`winget install Ninja-build.Ninja`), then build ClangBuildAnalyzer the same
|
|
way as on Linux:
|
|
```console
|
|
git clone https://github.com/aras-p/ClangBuildAnalyzer.git
|
|
cd ClangBuildAnalyzer
|
|
cmake -B build -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Release
|
|
cmake --build build --config Release
|
|
```
|
|
Then add `ClangBuildAnalyzer\build` to `PATH`.
|
|
|
|
### Usage
|
|
Mac/Linux:
|
|
```console
|
|
$ ./scripts/build-profile.sh
|
|
```
|
|
|
|
Windows:
|
|
```console
|
|
> .\scripts\build-profile.ps1
|
|
```
|
|
|
|
Both accept `--full`/`-Full` (include Server, Tools, and Tests) and a jobs
|
|
override (`-jN` / `-Jobs N`).
|
|
|
|
Note: on Windows, `cmake` defaults to the Visual Studio generator, which
|
|
ignores `CMAKE_C_COMPILER`/`CMAKE_CXX_COMPILER` and silently falls back to
|
|
MSVC. `build-profile.ps1` passes `-G Ninja` so clang is actually used, this
|
|
is required on ARM64.
|
|
|
|
### Linux (Ubuntu 24.04)
|
|
|
|
Environment:
|
|
- Clang: 18.1.3 (Ubuntu clang version 18.1.3 (1ubuntu1))
|
|
- libstdc++: GCC 13.3.0 (Ubuntu 13.3.0-6ubuntu2~24.04.1)
|
|
- Target: x86_64-pc-linux-gnu
|
|
|
|
```console
|
|
+------------------------+-----+------------+------------+------------+
|
|
| Build | TUs | Frontend | Backend | Total |
|
|
+------------------------+-----+------------+------------+------------+
|
|
| Minimal, master | 249 | 468.2 s | 270.3 s | 738.5 s |
|
|
| Minimal, with PCH | 253 | 177.1 s | 265.8 s | 442.9 s |
|
|
| Full, master | 396 | 811.0 s | 692.2 s | 1,503.2 s |
|
|
| Full, with PCH | 405 | 380.0 s | 664.7 s | 1,044.7 s |
|
|
| Full, with PCH + UB | 264 | 357.7 s | 635.7 s | 993.4 s |
|
|
+------------------------+-----+------------+------------+------------+
|
|
|
|
PCH = precompiled header.
|
|
Full = includes building Server, Tools, and Tests.
|
|
UB = unity build for models
|
|
```
|
|
Note that the number of translation units (TUs) increases when using precompiled
|
|
headers — each PCH target adds one extra TU for the precompilation step itself.
|
|
|
|
### Mac (Apple M3)
|
|
|
|
Environment:
|
|
- Clang: Apple clang version 17.0.0 (clang-1700.3.19.1)
|
|
- libc++: ships with Apple clang 17.0.0 (Xcode toolchain)
|
|
- Target: arm64-apple-macosx15.6
|
|
|
|
```console
|
|
+------------------------+-----+------------+------------+------------+
|
|
| Build | TUs | Frontend | Backend | Total |
|
|
+------------------------+-----+------------+------------+------------+
|
|
| Minimal, master | 256 | 154.5 s | 94.8 s | 249.3 s |
|
|
| Minimal, with PCH | 261 | 65.9 s | 90.0 s | 155.9 s |
|
|
| Full, master | 407 | 265.7 s | 209.7 s | 475.4 s |
|
|
| Full, with PCH | 414 | 154.6 s | 197.5 s | 352.1 s |
|
|
| Full, with PCH + UB | 274 | 143.0 s | 192.2 s | 335.2 s |
|
|
+------------------------+-----+------------+------------+------------+
|
|
|
|
PCH = precompiled header.
|
|
Full = includes building Server, Tools, and Tests.
|
|
UB = unity build for models
|
|
```
|
|
|
|
### Windows (ARM64)
|
|
|
|
Environment:
|
|
- Clang: clang version 22.1.8 (LLVM, `C:\Program Files\LLVM`)
|
|
- STL: MSVC STL (Visual Studio 2022 Build Tools 14.44.35207)
|
|
- Target: aarch64-pc-windows-msvc
|
|
|
|
```console
|
|
+------------------------+-----+------------+------------+------------+
|
|
| Build | TUs | Frontend | Backend | Total |
|
|
+------------------------+-----+------------+------------+------------+
|
|
| Minimal, master | 249 | 159.4 s | 82.2 s | 241.6 s |
|
|
| Full, master | 373 | 337.2 s | 167.4 s | 504.6 s |
|
|
| Minimal, with PCH + UB | 113 | 62.3 s | 82.4 s | 144.7 s |
|
|
| Full, with PCH + UB | 240 | 233.0 s | 185.1 s | 418.1 s |
|
|
+------------------------+-----+------------+------------+------------+
|
|
|
|
PCH = precompiled header.
|
|
Full = includes building Server, Tools, and Tests.
|
|
UB = unity build for models
|
|
```
|