Public/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-19 09:15:04 +02:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 552309b1b6 cont : use sparse vec FA for prefill Georgi Gerganov 2026-08-30 18:58:36 +03:00
  • c31b682bd1 metal : fix sparse flash attention row addressing Georgi Gerganov 2026-08-30 18:26:26 +03:00
  • 39a0be844a metal : support n_kv_max sparse mask hint in flash attention vec kernel Georgi Gerganov 2026-08-30 16:47:15 +03:00
  • 06a3b04bc8 fix PDL issues Aman Gupta 2026-08-29 15:57:11 +08:00
  • c2d23183d9 CUDA + ggml: add sparse flash attention Aman Gupta 2026-08-24 06:08:36 +02:00
  • 2d8d612e4c kv-cache : optimize restoring non-contiguous cells (#27991) b10724 itsnotoger 2026-08-31 18:49:58 +02:00
  • 010be9683a opencl: tune the quant paths for Intel Xe-LP GPUs to improve its TG and PP performance (#26438) b10723 Hongqiang Wang 2026-08-31 08:56:22 -07:00
  • 774ee0e200 ui: copy the displayed text of grouped agentic responses (#27832) Pascal 2026-08-31 17:48:43 +02:00
  • 8e53fcefd2 webgpu : avoid crash when offset is not multiple of 4 in WebGPU ggml_backend_tensor_get() implementation (#28045) b10721 fairydreaming 2026-08-31 16:04:38 +02:00
  • f8dbcd6189 ROCm: add radix TOP_K for long rows (#27466) b10720 Jaden_Mach 2026-08-31 09:00:04 -04:00
  • 5d4a3be26d metal : add fa-vec tunings for M1 (#28078) b10719 Niklas Wenzel 2026-08-31 13:58:55 +02:00
  • 41ef91f7c8 CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were restricted to 1 token (#27621) b10718 ynankani 2026-08-31 11:22:28 +00:00
  • a32af33de2 sycl : Enhance to get the free memory of Intel GPU (#27968) b10717 Neo Zhang 2026-08-31 18:33:02 +08:00
  • 580e88d8b7 ci : add check for unzip (#28082) Sigbjørn Skjæret 2026-08-31 12:17:51 +02:00
  • 662a0b0121 spec : fuse the DFlash encoder into the KV cache injection (#27310) b10715 王金旭 2026-08-31 17:19:20 +08:00
  • 2cdae802e4 vulkan: tune mat-vec rows for batched inference on Strix Halo (#27909) b10714 Simon Teixidor 2026-08-31 11:07:53 +02:00
  • 557614e029 ggml : add MUL_MAT to the list of ops that may need additional memory (for WebGPU) (#28071) b10713 fairydreaming 2026-08-31 10:17:23 +02:00
  • daef7b6874 vulkan: top_k radix select for k >= 1024 for Qwen 3.8 Flash Next (#28032) b10712 Ruben Ortlam 2026-08-31 07:04:34 +02:00
  • 9723942adc hexagon: fix CPY fence bug (#28033) b10711 gg/test-vulkan-fail Shenghan Yang 2026-08-31 02:18:24 +08:00
  • bd55e6aae8 metal : add remaining Q4_1/Q5_0/Q5_1 fa-vec tunings for M2 (#28017) b10710 codemonkey 2026-08-31 02:00:10 +08:00
  • a7cc83bbae rpc: avoid serializing buffers from other servers (#26500) b10709 hmirin 2026-08-31 02:26:16 +09:00
  • 6d1479c148 ggml : fix ggml_backend_buft_get_alloc_size() guard (#28038) b10708 Georgi Gerganov 2026-08-30 20:25:15 +03:00
  • 62acc89c26 kv-cells: stop the sequence scan once all sequences are seen (#28011) b10707 Pascal 2026-08-30 17:27:34 +02:00
  • 0190529ec4 ggml: add SWIGLU_CLAMP (#27930) b10706 Aman Gupta 2026-08-30 20:30:02 +05:30
  • 2578138397 llama: improve TENSOR_READ_LAZY handling (#27837) b10705 Xuan-Son Nguyen 2026-08-30 16:59:48 +02:00
  • f1793c1c4e CUDA: use the fast mm_ids_helper path for any n_expert_used (#27978) b10704 Pascal 2026-08-30 16:06:32 +02:00
  • 0b5be7e4a2 hip: tune rdna 3 mmq config (#26284) b10703 itterative 2026-08-30 13:47:21 +03:00
  • e422148047 hip : optimize Q2_0 dot-product path for gfx1201 (#26753) b10702 LunalFresh 2026-08-30 05:18:36 -05:00
  • cc231cb0da dflash: pass missing NVFP4 scales to attention operations (#28000) b10701 JamePeng 2026-08-30 16:34:39 +08:00
  • bebc9350ec common: rename --tensor-read-lazy to --lazy-mode, add -lzm shorthand (#27969) b10700 Georgi Gerganov 2026-08-30 09:18:10 +03:00
  • 73f56d105b ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (#27960) b10699 Georgi Gerganov 2026-08-30 09:17:47 +03:00
  • 742347b2e7 rpc: fix apple rdma error spew on teardown (#27908) b10698 Ryan C 2026-08-30 06:16:26 +00:00
  • 093adb242e metal: add fa-vec tunings for M3 Ultra (#27999) b10697 Nils Gladitz 2026-08-30 08:06:29 +02:00
  • b8b743c3c1 metal : Add fa-vec tuning for M3 Pro (#27963) b10696 Daya Adianto 2026-08-30 06:02:22 +00:00
  • dc7aecf70d vendor : update cpp-httplib to 0.54.0 (#27919) b10695 Alessandro de Oliveira Faria (A.K.A.CABELO) 2026-08-30 03:01:51 -03:00
  • 2bf0415152 rpc : fix pre-rdma macOS versions (#27815) b10694 Ryan C 2026-08-30 05:59:25 +00:00
  • 9e54e687cb hexagon: support for device discovery and create sessions on demand (#27785) b10693 Max Krasnyansky 2026-08-29 22:57:55 -07:00
  • 370cb12e8b sycl: split long rows in TOP_K instead of one work-group per row (#27847) b10692 Titaniumtown 2026-08-29 22:57:08 -07:00
  • d882575cc8 metal : fix null-pipeline crash for F16 src1 mul_mat/mul_mat_id (#25648) b10691 QuintinShaw 2026-08-30 13:56:35 +08:00
  • bdf3955159 memory : copy Hadamard matrix to k_rot tensor only if it has buffer assigned to prevent crashes during context shift of unquantized K cache (#27967) b10690 fairydreaming 2026-08-30 07:47:15 +02:00
  • 57291f2644 ggml: allow passing alloc dependencies in graph_optimize (#27301) b10689 Aman Gupta 2026-08-30 09:04:20 +05:30
  • c589f0ed10 metal : add fa-vec tunings for M2 (#27940) b10688 codemonkey 2026-08-30 07:44:53 +08:00
  • c841aeeb8b opencl: use a better matmul path on two Adreno GPU generations (#27640) b10687 Hongqiang Wang 2026-08-29 10:46:27 -07:00
  • 3173a56471 metal : assert shared memory padding (#27951) b10686 Georgi Gerganov 2026-08-29 17:55:15 +03:00
  • 17252c769a metal : add remaining fa-vec tunings for M4 Pro (#27915) b10685 Niklas Wenzel 2026-08-29 14:50:13 +02:00
  • cc83d7b482 sycl: make --fit respect --fit-target better (#27629) b10684 Nick Farrell 2026-08-29 19:00:09 +10:00
  • c9ca51c1f6 vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs (#27526) b10683 Jeff Bolz 2026-08-29 02:59:48 -05:00
  • 5ea1b124e7 metal : add fa-vec tunings for M1 Max (#27932) b10682 Jhen-Jie Hong 2026-08-29 15:12:23 +08:00
  • 77f132cb1d vulkan: Change mul_mat_id to pad K rather than N (#27925) b10681 Jeff Bolz 2026-08-29 02:09:24 -05:00
  • d7bd3bfcad snapdragon: python SDK setup (Windows) (#27903) b10680 kurquhar 2026-08-28 14:01:59 -07:00
  • cda6f9e3a0 Merge remote-tracking branch 'origin/master' into xsn/tensor_read_lazy_improve xsn/tensor_read_lazy_improve Xuan Son Nguyen 2026-08-28 22:59:01 +02:00
  • 50f068ffff bench: add --tensor-read-lazy (#27881) b10679 Xuan-Son Nguyen 2026-08-28 20:51:05 +02:00
  • f91123d2d0 qwen4exp: fix sparse-attention block selection tmp-q4 Thiago Padilha 2026-08-28 05:33:37 -03:00
  • 6fe7498016 model: qwen4exp: reduce number of graph splits (#27880) b10678 Xuan-Son Nguyen 2026-08-28 19:24:46 +02:00
  • b387ddfd84 vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize (#27812) b10677 Eric A Stalee 2026-08-28 12:12:33 -05:00
  • a43c3986b4 ggml : fix conv_transpose_2d for multiple batches (#26132) b10676 Tekin Ertekin 2026-08-28 20:09:08 +03:00
  • 90c26fcd4b Vulkan: add hoisting support for row IDs and expert count in shaders (#26686) b10675 ravel7524 2026-08-28 10:52:49 -04:00
  • 18f3082e88 metal : rename variable Niklas Wenzel 2026-08-28 16:36:14 +02:00
  • 44a1f10095 metal : fix more leaks due to missing autoreleasepools Niklas Wenzel 2026-08-28 14:13:10 +02:00
  • 8663224818 context : disable non-fused GDN and LID ops (#27877) Georgi Gerganov 2026-08-28 16:34:26 +03:00
  • f5e85d43a0 metal : add fa-vec tunings for M4 (#27875) b10673 Strongtut 2026-08-28 05:37:37 -07:00
  • 511f9c1379 OpenVINO: Update OV to 2026.3.1, whisper.cpp support, Qwen3.5 on NPU, and new ops (#27843) b10672 Zijun Yu 2026-08-28 19:42:07 +08:00
  • b19cbe925b convert: prevent ndarray conversion in LazyChunkedTensor (#27869) Xuan-Son Nguyen 2026-08-28 11:46:30 +02:00
  • 6bfbc89820 convert: prevent ndarray conversion in LazyChunkedTensor xsn/fix_lazy_tensor Xuan Son Nguyen 2026-08-28 11:25:42 +02:00
  • d077b4c214 sycl: use TILE for quantized KV decode on BMG (#26689) b10670 Ozymandias_EBON 2026-08-28 03:58:58 -05:00
  • be876204aa sycl: bind the f16 KV cache in place for the oneDNN SDPA path (#27468) b10669 Titaniumtown 2026-08-28 01:53:31 -07:00
  • 8963a9bdcd metal : add fa-vec tunings for M3 Max, M5 and M5 Pro (#27863) b10668 Georgi Gerganov 2026-08-28 11:52:03 +03:00
  • 6d6b697cd5 metal : add fa-vec tunings for M4 Pro (#27824) b10667 Brad Smith 2026-08-28 04:37:43 -04:00
  • a22c1d344e ui : apply ESLint formatting to models discover components allozaur/feat/models-discover-and-download Aleksander Grygier 2026-08-25 15:27:59 +02:00
  • ed857de31d feat: WIP Aleksander Grygier 2026-08-25 11:43:36 +02:00
  • 2282e8598f feat: WIP Aleksander Grygier 2026-08-25 10:43:40 +02:00
  • 59c905e6ba feat: WIP Aleksander Grygier 2026-08-25 09:30:23 +02:00
  • 3fa85af123 feat: WIP Aleksander Grygier 2026-08-24 16:52:11 +02:00
  • 9ffdcb7865 fix: layout Aleksander Grygier 2026-08-24 16:37:25 +02:00
  • a56fef4aa5 feat: WIP Aleksander Grygier 2026-08-24 16:23:08 +02:00
  • 941d7923c6 feat: WIP Aleksander Grygier 2026-08-24 15:40:13 +02:00
  • 4efb65d24c feat: WIP Aleksander Grygier 2026-08-24 15:01:16 +02:00
  • 84d6a812ca chore: Format Aleksander Grygier 2026-08-22 18:02:25 +02:00
  • fe26a3a59e feat(ui): make base dialog responsive and support sticky headers Aleksander Grygier 2026-08-22 09:55:00 +02:00
  • b7a8df052e chore: Format Aleksander Grygier 2026-08-22 07:10:06 +02:00
  • 3ef212034b ui : open MCP servers in a dialog from the chat form Aleksander Grygier 2026-08-22 06:53:22 +02:00
  • e3e74a1bfb ui : fix eslint issues in chat form and model selector Aleksander Grygier 2026-08-17 11:04:02 +02:00
  • fb20f31a47 ui : add raw model id tooltip to model selector options Aleksander Grygier 2026-08-15 23:26:25 +02:00
  • 3f7e6974d7 ui : add show-org-name-in-trigger display setting Aleksander Grygier 2026-08-15 22:53:48 +02:00
  • f140d848ef ui : keep reasoning submenu visible regardless of model state Aleksander Grygier 2026-08-15 22:53:26 +02:00
  • 4e97ac86eb tests : run test-save-load-state across all architectures (#27755) b10666 Georgi Gerganov 2026-08-28 09:45:19 +03:00
  • ca3d5a3e10 model: add DSpark support for Nemotron3.5 (#27804) b10665 Ruixiang Wang 2026-08-28 01:49:27 +02:00
  • 280e764524 llama: improve TENSOR_READ_LAZY handling Xuan Son Nguyen 2026-08-28 01:29:13 +02:00
  • a7cfd8864f force lazy tensor on cpu if lazy is on Xuan Son Nguyen 2026-08-28 01:06:40 +02:00
  • e70802a01f ggml-hexagon: add HTP unary ops for ABS and LOG (#27786) b10664 cqderek 2026-08-28 06:05:57 +08:00
  • 83d855c5a6 hex-unary: fix RMS_NORM_MUL weight-offset bugs for grouped/broadcast norms (#27798) b10663 Aparna M P 2026-08-28 03:08:02 +05:30
  • f47ff9b250 quantize: row-slab stream to avoid thread starvation xsn/quant_stream Xuan Son Nguyen 2026-08-27 22:42:22 +02:00
  • 18443257a3 server: add ctx-per-slot (--kv-unified-per-slot) (#24124) b10662 Bartowski 2026-08-27 16:39:14 -04:00
  • 32176338a6 ci : build only the ggml-hip backend for windows-rocm release (#27753) b10661 Harkirat Gill 2026-08-27 16:18:34 -04:00
  • 6c84c7d5d8 model: add Qwen3.8-Flash-Next (qwen4exp) (#27742) b10660 Daniel Han 2026-08-27 12:32:31 -07:00
  • 6fdd0ac890 ci : bundle HIP runtime DLLs with Windows ROCm release (#26973) b10659 Slobodan Josic 2026-08-27 19:27:57 +02:00
  • b10f9ca58c spec : add DFlash2 support (local convolution + candidate selector) (#27342) (#27816) b10658 Xuan-Son Nguyen 2026-08-27 19:17:07 +02:00
  • 4e2f54f8f3 revert top-k.cu changes xsn/dflash2 Xuan Son Nguyen 2026-08-27 19:08:52 +02:00
  • 4a6ad487a6 spec : add DFlash2 support (local convolution + candidate selector) (#27342) Zihan Zhang 2026-08-27 10:05:57 -07:00
  • 58546250cf opencl: add bin kernels kernel_gemm_moe_q4_0_q8_1_dp4a_bin, kernel_gemm_moe_mxfp4_q8_1_dp4a_bin (#27768) b10657 Shawn Gu 2026-08-27 09:44:05 -07:00