skills: add ggml-test skill

2026-09-20 01:31:31 +02:00 · 2026-07-27 00:58:39 +02:00
parent d4d057b6dd
commit d29e61d45a
6 changed files with 421 additions and 0 deletions
@@ -48,6 +48,8 @@ Follow HOWTO-add-model.md section 3 for the actual touch points (`src/models/<na

 Skill-specific addition: before writing `src/models/<name>.cpp`, read at least 10 other files under `src/models/` (pick a mix, not just the one reference architecture) to confirm the struct layout, naming, and style you're about to write actually matches current convention - the pattern drifts over time and the HOWTO doc can lag behind it.

+If this model has a non-standard sub-component you're porting independently of the rest of the graph (a custom mixer/state-space block, an AltUp-style block, a non-standard RoPE or interpolation variant, etc.), use the `ggml-test` skill to validate that component's ggml op sequence against a small PyTorch reference before wiring it into the full graph - it's much faster to find a shape/stride bug in an isolated op sequence than in a full forward pass.
+
 ## Step 4 - Optional: multimodal encoder

 Only do this if the contributor flagged a vision/audio encoder in Step 0. Follow HOWTO-add-model.md section 4 and `docs/multimodal.md` for the actual touch points (`MmprojModel` subclass, `clip.cpp`, `mtmd.cpp`, encoder graph in `tools/mtmd/models`, etc.).
@@ -0,0 +1,82 @@
+---
+name: ggml-test
+description: Test a single ggml op or a small piece of a cgraph in isolation, by compiling a minimal standalone ggml program and comparing it against a reference implementation (usually PyTorch). Use when porting a model component (e.g. from HF transformers) and you want to validate it before wiring it into the full graph, or when debugging a specific op/subgraph (custom RoPE variant, interpolation, pooling, an isolated sub-block like AltUp) without running the whole model.
+---
+
+# Test a ggml op / cgraph fragment in isolation
+
+Porting a whole model end-to-end and only then comparing final logits makes it hard to find *which* op is wrong. This skill validates one component at a time: write a tiny PyTorch reference with randomly-initialized tensors, dump its inputs/weights/output to GGUF, write a minimal standalone ggml program that runs the same op sequence, and compare the two outputs numerically. This is a scratch/throwaway harness, not a permanent test -- it lives in `tmp/` (gitignored) and gets deleted once the component is confirmed correct and merged into the real graph-building code.
+
+Good times to reach for this:
+- A new architecture has a non-standard sub-component developed/ported independently (e.g. an AltUp block, a custom mixer/state-space component) before it's wired into the full model graph.
+- Validating one op's semantics against a reference before trusting it inside a much bigger graph (e.g. a multi-dimensional RoPE variant, an interpolation/resize mode, a custom pooling or normalization).
+- Debugging a numerical mismatch in `clip.cpp` / `mtmd/models/*.cpp` / `src/models/*.cpp` where the whole-model comparison (see `examples/model-conversion/README.md`) shows a mismatch but doesn't say where.
+
+This complements, it does not replace, the full logits-verification workflow in `examples/model-conversion/README.md` and the `add-new-model` skill -- use this to isolate a failure or de-risk a component *before* or *while* doing that full-model verification.
+
+## Workflow
+
+### 0. Confirm there's a real prerequisite
+
+You need: (a) the reference implementation's source (usually a HF `transformers` `modeling_*.py` file -- fetch the specific class/method, don't rely on memory of what it does), and (b) the ggml code under test already written (even if unverified), so you know exactly which op sequence to transcribe. If the ggml side doesn't exist yet, write it first (following the `add-new-model` skill's conventions), then come back here to validate it.
+
+### 1. Write a minimal PyTorch reference
+
+Only implement the component under test, not the surrounding model. Use small, randomly-initialized tensors (`torch.manual_seed(...)` for reproducibility) with the smallest shapes that still exercise the interesting behavior (e.g. a handful of patches/tokens, small hidden size) -- this keeps compile/run iteration fast and makes mismatches easy to eyeball. Save every tensor the ggml side will need as an input (weights, indices, etc.) plus the expected output, using the helper in `scripts/gguf_io.py`:
+
+```python
+import sys
+sys.path.insert(0, "skills/ggml-test/scripts")
+from gguf_io import save_tensors
+
+save_tensors("tmp/<component>_in.gguf", {
+    "some_weight": weight_tensor.numpy().astype(np.float32),
+    "some_indices": idx_tensor.numpy().astype(np.int32),
+})
+save_tensors("tmp/<component>_ref.gguf", {
+    "out": expected_output.numpy().astype(np.float32),
+})
+```
+
+Notes:
+- `save_tensors` preserves each array's numpy shape/dtype as-is; gguf-py reverses the axis order under the hood to match ggml's `ne[]` convention (numpy's last axis becomes ggml's `ne[0]`), same as regular model conversion. You don't need to transpose anything by hand.
+- Use `int32` for any tensor that becomes a ggml `I32` index tensor (e.g. positions fed to `ggml_get_rows`/`ggml_rope_ext`) -- ggml has no int64 op support for these.
+- Split "inputs" and "reference output" into separate files (or just separate tensor names in one file) so the ggml program only has to read what it needs to build the graph, and the comparison step only has to read what it needs to check.
+
+### 2. Write the minimal ggml program
+
+Put it in `tmp/` (gitignored). Structure:
+
+1. Load the input GGUF with `gguf_init_from_file(path, {no_alloc=false, ctx=&ctx_data})`. This gives you a `ggml_context` whose tensors already have their data loaded in plain CPU memory (`ggml_get_tensor(ctx_data, name)`) -- no backend buffer juggling needed for a CPU-only test.
+2. Build a *second*, `no_alloc=true` context for the graph, and **copy-paste the actual op sequence from the real source file into it** -- don't reimplement/paraphrase it. Swap struct-member references (`model.foo`, `ctx0`, `hparams.bar`) for local variables holding the same values, but keep the `ggml_*` calls themselves verbatim. This is the entire point: you're testing the exact code that will ship, not a restatement of it.
+3. `ggml_backend_cpu_init()` + `ggml_gallocr_new(ggml_backend_cpu_buffer_type())` + `ggml_gallocr_alloc_graph(...)`, then `ggml_backend_tensor_set(...)` the graph's input tensors from the data loaded in step 1, then `ggml_backend_graph_compute(...)`.
+4. Dump the result tensor to a GGUF file with `gguf_add_tensor` + `gguf_write_to_file` so the comparison step can read it back.
+
+See `references/gemma4v_pos_embd_example.md` for a complete, verified worked example covering all four steps.
+
+### 3. Compile and run
+
+```bash
+skills/ggml-test/scripts/build_and_run.sh tmp/test_<component>.cpp tmp/<component>_in.gguf tmp/<component>_out.gguf
+```
+
+This links directly against the already-built `libggml*` in the project's CMake build dir (default `build/`, override with `GGML_TEST_BUILD_DIR`) -- no need to add a target to the project's `CMakeLists.txt` for a throwaway test. Requires the project to have been built at least once already.
+
+### 4. Compare
+
+```bash
+python3 skills/ggml-test/scripts/compare_tensors.py tmp/<component>_ref.gguf tmp/<component>_out.gguf
+```
+
+Reports, per tensor name, shape, max abs diff, mean abs diff, relative L2 diff, and a pass/fail against `--rtol`/`--atol` (defaults `1e-3`/`1e-4` -- loosen for f16/bf16 or tighten for a pure-f32 op with no reduction). A shape mismatch is reported explicitly since it usually means a transpose/reshape assumption is wrong, not a numerical issue.
+
+### 5. Iterate, then clean up
+
+Fix the ggml side (or discover the PyTorch reference itself was wrong -- re-check against the HF source), re-run steps 3-4 until it passes. Once confirmed, port the validated op sequence into the real graph-building file if it wasn't already there, and delete the scratch files under `tmp/` (they're gitignored, but delete them anyway so they don't linger and get mistaken for still-relevant scratch work).
+
+## Common pitfalls
+
+- Forgetting `ggml_set_input()`/`ggml_set_output()` on the tensors you need to set/read -- without these the graph allocator is free to reuse/overwrite their memory.
+- Testing with shapes so small that a bug that depends on stride/alignment (e.g. an off-by-one in a view offset, a wrong `nb[]` in `ggml_view_*`) doesn't get exercised. Prefer shapes where every dimension has a different size, so a transposed axis or swapped stride shows up as a shape or numeric mismatch instead of silently working.
+- Comparing f16/bf16 ops with the same tight tolerance as f32 -- loosen `--rtol`/`--atol` accordingly, or cast both sides to f32 before comparing if you only care about the op logic and not quantization-induced error.
+- Not resetting `torch.manual_seed(...)` -- without it the reference becomes non-reproducible across runs, which makes it impossible to tell whether a fix actually changed anything.
@@ -0,0 +1,202 @@
+# Worked example: `model.position_embeddings` in `tools/mtmd/models/gemma4v.cpp`
+
+This is a complete, verified walkthrough of the workflow in `SKILL.md`, testing the 2-D lookup-table positional embedding block in `clip_graph_gemma4v::build()` (`tools/mtmd/models/gemma4v.cpp`, the `model.position_embeddings` block):
+
+```cpp
+{
+    const int64_t pos_size = model.position_embeddings->ne[1];
+    const size_t  nb1      = ggml_row_size(model.position_embeddings->type, n_embd);
+
+    // positional embeddings are stored as lookup tables (one for x, one for y)
+    ggml_tensor * tbl_x = ggml_view_2d(ctx0, model.position_embeddings,
+                                         n_embd, pos_size, nb1, 0);
+    ggml_tensor * tbl_y = ggml_view_2d(ctx0, model.position_embeddings,
+                                         n_embd, pos_size, nb1, pos_size * nb1);
+
+    // ggml_get_rows: [n_embd, n_patches]
+    ggml_tensor * emb_x = ggml_get_rows(ctx0, tbl_x, pos_x);
+    ggml_tensor * emb_y = ggml_get_rows(ctx0, tbl_y, pos_y);
+
+    inp = ggml_add(ctx0, inp, emb_x);
+    inp = ggml_add(ctx0, inp, emb_y);
+}
+```
+
+## Step 0: read the reference implementation
+
+The HF reference is `Gemma4VisionPatchEmbedder._position_embeddings` in
+[`modeling_gemma4.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/gemma4/modeling_gemma4.py):
+
+```python
+class Gemma4VisionPatchEmbedder(nn.Module):
+    def __init__(self, config):
+        ...
+        self.position_embedding_table = nn.Parameter(
+            torch.ones(2, self.position_embedding_size, self.hidden_size)
+        )
+
+    def _position_embeddings(self, pixel_position_ids, padding_positions):
+        clamped_positions = pixel_position_ids.clamp(min=0)
+        # position_embedding_table: (2, position_embedding_size, hidden_size)
+        x_emb = F.embedding(clamped_positions[..., 0], self.position_embedding_table[0])
+        y_emb = F.embedding(clamped_positions[..., 1], self.position_embedding_table[1])
+        position_embeddings = x_emb + y_emb
+        position_embeddings = torch.where(padding_positions.unsqueeze(-1), 0.0, position_embeddings)
+        return position_embeddings
+```
+
+`position_embedding_table` has PyTorch shape `(2, position_embedding_size, hidden_size)` -- axis 0 selects the x- or y-table. Saved as-is with `save_tensors`, gguf-py's axis reversal turns this into a ggml tensor with `ne = [n_embd, pos_size, 2]`: a *3-D*, contiguous tensor where the y-table starts exactly `pos_size * nb[1]` bytes in (that byte offset is `nb[2]` of the 3-D tensor). That's why the C++ code above can treat it as two `ggml_view_2d` slices of a nominally-2-D view -- it's relying on the underlying 3-D tensor being contiguous. This is exactly the kind of stride assumption this skill is good at catching if it were ever wrong.
+
+This test only covers `_position_embeddings` (the lookup-and-add), not the padding mask or the surrounding `input_proj`/conv -- keep each test scoped to one op sequence.
+
+## Step 1: PyTorch reference (`tmp/pt_ref_gemma4v_pos_embd.py`)
+
+```python
+import sys
+sys.path.insert(0, "skills/ggml-test/scripts")
+import numpy as np
+import torch
+import torch.nn.functional as F
+from gguf_io import save_tensors
+
+torch.manual_seed(0)
+
+n_embd    = 8
+pos_size  = 6   # config.position_embedding_size
+n_patches = 4
+
+# Gemma4VisionPatchEmbedder.position_embedding_table
+position_embedding_table = torch.randn(2, pos_size, n_embd)
+
+pos_x = torch.randint(0, pos_size, (n_patches,))
+pos_y = torch.randint(0, pos_size, (n_patches,))
+
+# Gemma4VisionPatchEmbedder._position_embeddings (padding path omitted -- not under test)
+x_emb = F.embedding(pos_x, position_embedding_table[0])
+y_emb = F.embedding(pos_y, position_embedding_table[1])
+out = x_emb + y_emb  # (n_patches, n_embd)
+
+save_tensors("tmp/gemma4v_pos_embd_in.gguf", {
+    "position_embeddings": position_embedding_table.numpy().astype(np.float32),
+    "pos_x": pos_x.numpy().astype(np.int32),
+    "pos_y": pos_y.numpy().astype(np.int32),
+})
+save_tensors("tmp/gemma4v_pos_embd_ref.gguf", {
+    "out": out.numpy().astype(np.float32),
+})
+print("wrote tmp/gemma4v_pos_embd_in.gguf and tmp/gemma4v_pos_embd_ref.gguf")
+```
+
+Run: `python3 tmp/pt_ref_gemma4v_pos_embd.py`
+
+## Step 2: ggml test program (`tmp/test_gemma4v_pos_embd.cpp`)
+
+```cpp
+#include "ggml.h"
+#include "ggml-alloc.h"
+#include "ggml-backend.h"
+#include "ggml-cpu.h"
+#include "gguf.h"
+
+#include <cstdio>
+
+int main(int argc, char ** argv) {
+    const char * in_path  = argc > 1 ? argv[1] : "tmp/gemma4v_pos_embd_in.gguf";
+    const char * out_path = argc > 2 ? argv[2] : "tmp/gemma4v_pos_embd_out.gguf";
+
+    // 1. load inputs + weights (data already loaded into plain CPU memory)
+    struct ggml_context * ctx_data = nullptr;
+    struct gguf_init_params gguf_params = { /*.no_alloc =*/ false, /*.ctx =*/ &ctx_data };
+    struct gguf_context * gguf_ctx = gguf_init_from_file(in_path, gguf_params);
+    if (!gguf_ctx) { fprintf(stderr, "failed to load %s\n", in_path); return 1; }
+
+    ggml_tensor * position_embeddings = ggml_get_tensor(ctx_data, "position_embeddings"); // ne = [n_embd, pos_size, 2]
+    ggml_tensor * pos_x_data          = ggml_get_tensor(ctx_data, "pos_x");               // ne = [n_patches]
+    ggml_tensor * pos_y_data          = ggml_get_tensor(ctx_data, "pos_y");
+    if (!position_embeddings || !pos_x_data || !pos_y_data) {
+        fprintf(stderr, "missing expected tensor in %s\n", in_path);
+        return 1;
+    }
+
+    const int64_t n_embd    = position_embeddings->ne[0];
+    const int64_t n_patches = pos_x_data->ne[0];
+
+    // 2. build the graph -- copy-pasted verbatim from the `model.position_embeddings`
+    //    block in tools/mtmd/models/gemma4v.cpp
+    struct ggml_init_params cparams = { /*.mem_size=*/ 16*1024*1024, /*.mem_buffer=*/ nullptr, /*.no_alloc=*/ true };
+    struct ggml_context * ctx = ggml_init(cparams);
+
+    ggml_tensor * pos_x = ggml_new_tensor_1d(ctx, GGML_TYPE_I32, n_patches);
+    ggml_set_name(pos_x, "pos_x");
+    ggml_set_input(pos_x);
+
+    ggml_tensor * pos_y = ggml_new_tensor_1d(ctx, GGML_TYPE_I32, n_patches);
+    ggml_set_name(pos_y, "pos_y");
+    ggml_set_input(pos_y);
+
+    ggml_tensor * cur;
+    {
+        const int64_t pos_size = position_embeddings->ne[1];
+        const size_t  nb1      = ggml_row_size(position_embeddings->type, n_embd);
+
+        ggml_tensor * tbl_x = ggml_view_2d(ctx, position_embeddings, n_embd, pos_size, nb1, 0);
+        ggml_tensor * tbl_y = ggml_view_2d(ctx, position_embeddings, n_embd, pos_size, nb1, pos_size * nb1);
+
+        ggml_tensor * emb_x = ggml_get_rows(ctx, tbl_x, pos_x);
+        ggml_tensor * emb_y = ggml_get_rows(ctx, tbl_y, pos_y);
+
+        cur = ggml_add(ctx, emb_x, emb_y);
+    }
+    ggml_set_name(cur, "out");
+    ggml_set_output(cur);
+
+    struct ggml_cgraph * gf = ggml_new_graph(ctx);
+    ggml_build_forward_expand(gf, cur);
+
+    // 3. allocate + run on CPU backend
+    ggml_backend_t backend = ggml_backend_cpu_init();
+    ggml_gallocr_t galloc = ggml_gallocr_new(ggml_backend_cpu_buffer_type());
+    ggml_gallocr_alloc_graph(galloc, gf);
+
+    ggml_backend_tensor_set(pos_x, pos_x_data->data, 0, ggml_nbytes(pos_x));
+    ggml_backend_tensor_set(pos_y, pos_y_data->data, 0, ggml_nbytes(pos_y));
+
+    ggml_backend_graph_compute(backend, gf);
+
+    // 4. dump the output to gguf so a python script can compare it to the pytorch reference
+    struct gguf_context * out_ctx = gguf_init_empty();
+    struct ggml_init_params out_params = { (size_t) ggml_nbytes(cur) + 1024*1024, nullptr, /*.no_alloc=*/ false };
+    struct ggml_context * ctx_out = ggml_init(out_params);
+    ggml_tensor * out_t = ggml_dup_tensor(ctx_out, cur);
+    ggml_set_name(out_t, "out");
+    ggml_backend_tensor_get(cur, out_t->data, 0, ggml_nbytes(cur));
+    gguf_add_tensor(out_ctx, out_t);
+    gguf_write_to_file(out_ctx, out_path, false);
+    printf("wrote %s\n", out_path);
+
+    gguf_free(out_ctx);
+    ggml_free(ctx_out);
+    ggml_gallocr_free(galloc);
+    ggml_backend_free(backend);
+    ggml_free(ctx);
+    gguf_free(gguf_ctx);
+    ggml_free(ctx_data);
+    return 0;
+}
+```
+
+## Step 3-4: build, run, compare
+
+```bash
+skills/ggml-test/scripts/build_and_run.sh tmp/test_gemma4v_pos_embd.cpp
+python3 skills/ggml-test/scripts/compare_tensors.py \
+    tmp/gemma4v_pos_embd_ref.gguf tmp/gemma4v_pos_embd_out.gguf
+```
+
+Verified output:
+
+```
+[PASS] out: shape=(4, 8) max_abs=0.000e+00 mean_abs=0.000e+00 rel_l2=0.000e+00
+```
+
+An exact match is expected here since every op involved (`ggml_view_2d`, `ggml_get_rows`, `ggml_add`) is exact in f32 with no reduction -- if you see anything above float epsilon on a component like this, suspect a shape/stride bug rather than expected numerical drift.
@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+# Compile and run a single-file standalone ggml test program against an already-built llama.cpp tree, without touching the project's CMakeLists.txt.
+#
+# Usage:
+#   skills/ggml-test/scripts/build_and_run.sh tmp/test_foo.cpp [-- program-args...]
+#
+# Env:
+#   GGML_TEST_BUILD_DIR - path to the CMake build dir to link against (default: <repo root>/build). Must already contain bin/libggml*, i.e. the project must be built once.
+set -euo pipefail
+
+SRC="$1"; shift || true
+
+ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
+BUILD_DIR="${GGML_TEST_BUILD_DIR:-$ROOT/build}"
+BIN_DIR="$BUILD_DIR/bin"
+OUT="${SRC%.cpp}"
+
+if [ ! -d "$BIN_DIR" ]; then
+    echo "error: $BIN_DIR not found - build llama.cpp once first, e.g.:" >&2
+    echo "  cmake -B $BUILD_DIR && cmake --build $BUILD_DIR --target ggml -j" >&2
+    exit 1
+fi
+
+g++ -std=c++17 -O0 -g \
+    -I "$ROOT/ggml/include" \
+    "$SRC" \
+    -L "$BIN_DIR" -lggml -lggml-base -lggml-cpu \
+    -Wl,-rpath,"$BIN_DIR" \
+    -o "$OUT"
+
+echo "compiled -> $OUT"
+exec "$OUT" "$@"
@@ -0,0 +1,58 @@
+#!/usr/bin/env python3
+"""Compare tensors between two GGUF files (e.g. a PyTorch reference dump and a ggml test program's output dump) by name and report numerical differences.
+
+Usage:
+    python3 compare_tensors.py ref.gguf out.gguf [--rtol 1e-3] [--atol 1e-4]
+
+Exits 0 if every tensor present in both files matches within tolerance, 1 otherwise. Tensors present in only one file are reported but don't fail the run on their own -- useful when comparing partial dumps.
+"""
+import argparse
+import sys
+from pathlib import Path
+
+_GGUF_PY = Path(__file__).resolve().parents[3] / "gguf-py"
+if _GGUF_PY.exists() and str(_GGUF_PY) not in sys.path:
+    sys.path.insert(0, str(_GGUF_PY))
+
+import numpy as np
+from gguf.gguf_reader import GGUFReader
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("ref")
+    ap.add_argument("out")
+    ap.add_argument("--rtol", type=float, default=1e-3)
+    ap.add_argument("--atol", type=float, default=1e-4)
+    args = ap.parse_args()
+
+    ref = {t.name: t.data for t in GGUFReader(args.ref).tensors}
+    out = {t.name: t.data for t in GGUFReader(args.out).tensors}
+
+    ok = True
+    for name in sorted(set(ref) | set(out)):
+        if name not in ref or name not in out:
+            print(f"[MISSING] {name}: in ref={name in ref}, in out={name in out}")
+            continue
+
+        a, b = ref[name].astype(np.float64), out[name].astype(np.float64)
+        if a.shape != b.shape:
+            print(f"[SHAPE MISMATCH] {name}: ref={a.shape} out={b.shape}")
+            ok = False
+            continue
+
+        diff = np.abs(a - b)
+        max_abs = diff.max() if diff.size else 0.0
+        mean_abs = diff.mean() if diff.size else 0.0
+        rel_l2 = np.linalg.norm(diff) / (np.linalg.norm(a) + 1e-12)
+        passed = np.allclose(a, b, rtol=args.rtol, atol=args.atol)
+        ok &= passed
+
+        status = "PASS" if passed else "FAIL"
+        print(f"[{status}] {name}: shape={a.shape} max_abs={max_abs:.3e} mean_abs={mean_abs:.3e} rel_l2={rel_l2:.3e}")
+
+    sys.exit(0 if ok else 1)
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,45 @@
+#!/usr/bin/env python3
+"""Save/load plain numpy tensors to/from a GGUF file, for the ggml-test skill's PyTorch <-> ggml comparison workflow. Thin wrapper around gguf-py.
+
+Import from a script placed anywhere under the project (e.g. tmp/):
+
+    import sys
+    sys.path.insert(0, "skills/ggml-test/scripts")
+    from gguf_io import save_tensors, load_tensors
+"""
+import sys
+from pathlib import Path
+
+_GGUF_PY = Path(__file__).resolve().parents[3] / "gguf-py"
+if _GGUF_PY.exists() and str(_GGUF_PY) not in sys.path:
+    sys.path.insert(0, str(_GGUF_PY))
+
+import numpy as np
+from gguf import GGUFWriter
+from gguf.gguf_reader import GGUFReader
+
+
+def save_tensors(path, tensors, kv=None):
+    """tensors: dict[str, np.ndarray]. Array shape/dtype is preserved as-is -- gguf-py reverses the axis order internally to match ggml's ne[] convention (numpy's last axis becomes ggml's ne[0]), and load_tensors() below reverses it back, so round-tripping keeps the original numpy shape."""
+    writer = GGUFWriter(str(path), "ggml-test")
+    for k, v in (kv or {}).items():
+        if isinstance(v, bool):
+            writer.add_bool(k, v)
+        elif isinstance(v, int):
+            writer.add_int64(k, v)
+        elif isinstance(v, float):
+            writer.add_float32(k, v)
+        else:
+            writer.add_string(k, str(v))
+    for name, arr in tensors.items():
+        writer.add_tensor(name, np.ascontiguousarray(arr))
+    writer.write_header_to_file()
+    writer.write_kv_data_to_file()
+    writer.write_tensors_to_file()
+    writer.close()
+
+
+def load_tensors(path):
+    """Returns dict[str, np.ndarray], shape as originally saved."""
+    reader = GGUFReader(str(path))
+    return {t.name: t.data.copy() for t in reader.tensors}