This commit is contained in:
Xuan Son Nguyen
2026-08-17 02:36:33 +02:00
parent 71c7866a16
commit 584b4f93c6
+4
View File
@@ -29,6 +29,7 @@ The required steps to implement for an HF model are:
```python
@ModelBase.register("MyModelForCausalLM")
@ModelBase.example("user/model")
class MyModel(TextModel):
model_arch = gguf.MODEL_ARCH.MYMODEL
```
@@ -37,10 +38,13 @@ or
```python
@ModelBase.register("MyModelForConditionalGeneration")
@ModelBase.example("user/model")
class MyModel(MmprojModel):
model_arch = gguf.MODEL_ARCH.MYMODEL
```
The `example` should point to a valid Hugging Face model that will be used for testing. You can add multiple models if necessary. Prefer a non-gated model, or tiny random weights if no such model exists.
2. Define the layout of the GGUF tensors in [constants.py](/gguf-py/gguf/constants.py)
Add an enum entry in `MODEL_ARCH`, the model human friendly name in `MODEL_ARCH_NAMES` and the GGUF tensor names in `MODEL_TENSORS`.