* Adding support for bailingmoe3
* Adds speculative decoding support
* Make BailingMoE3 safe gate metadata optional
* bailingmoe3: apply trained SwiGLU clamps
* common: fix Bailing V3 tool argument parsing
* llama-model-saver, instantiate float vector metadata writer
* bailingmoe3: support Q-LoRA (Ling-3.0-tiny)
Ling-3.0-flash sets q_lora_rank: None and projects Q directly, so the current
implementation loads a single ATTN_Q tensor. Ling-3.0-tiny sets q_lora_rank: 256
and routes Q through a LoRA bottleneck instead:
q_a_proj -> q_a_layernorm -> q_b_proj
Conversion therefore failed with:
ValueError: Can not map tensor 'model.layers.3.attention.q_a_layernorm.weight'
Add the missing path, mirroring the existing deepseek2 MLA implementation:
* constants.py - add ATTN_Q_A / ATTN_Q_B / ATTN_Q_A_NORM to BAILINGMOE3
* tensor_mapping.py - map model.layers.{bid}.attention.q_{a,b}_proj and
q_a_layernorm
* conversion - emit attention.q_lora_rank when the config has it
* bailingmoe3.cpp - read n_lora_q; create the Q-LoRA tensors and build Q
through the bottleneck when q_lora_rank > 0
Everything is gated on q_lora_rank > 0. Ling-3.0-flash's config has no
q_lora_rank, the converter only emits the key when present, hparams.n_lora_q
defaults to 0, and get_key(..., required=false) leaves the target untouched when
the key is absent - so flash keeps taking the existing direct-Q branch.
The LoRA path produces the same shape as the direct projection, so the
nope/rope split, RoPE application and wk_b absorption downstream are unchanged.
* small mtp change
* bailingmoe3: support separate MTP GGUF and Q-LoRA MTP
* gguf: remove duplicate add_kda_gate_lower_bound definition
---------
Co-authored-by: bloomer <bloomer@booper.brushtail.me>
Co-authored-by: Dyluhn <dylanranejohnston1@gmail.com>
* model: add Hy3 (hy_v3) architecture support
Adds Tencent Hunyuan 3 (HF architecture HYV3ForCausalLM, GGUF arch
hy_v3): a MoE decoder stack with per-head Q/K RMSNorm, a sigmoid
router with expert selection bias, an always-active ungated shared
expert, and leading dense block(s) (first_k_dense_replace).
The base implementation is ported from charlie12345's fork
(https://github.com/charlie12345/ROCmFPX, src/models/hyv3.cpp),
adapted to current mainline APIs (hparams.n_layer(), build_qkv,
build_moe_ffn with fused gate_up + scale tensors, output_s).
Note: blk.N.exp_probs_b is stored without a .bias suffix for
compatibility with existing hy_v3 GGUFs produced by that fork.
Co-Authored-By: charlie12345 <charlie12345@users.noreply.github.com>
Co-authored-by: Piotr Wilkin <ilintar@gmail.com>
Assisted-by: Claude Fable 5
* chat: trim messages sent to StepFun parser (fixes long reasoning loops)
* add regression test; remove duplicate template
* chat: trim StepFun content parts before rendering
The StepFun trim workaround ran on the already-rendered messages, where
typed content parts have been concatenated into a single string, so the
per-part whitespace could no longer be reached. Move the trim ahead of
rendering and apply it to content_parts text as well as the string
content and reasoning_content. Adds a content-parts regression test.
Co-Authored-By: Piotr Wilkin <ilintar@gmail.com>
Assisted-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: tarruda <tpadilha84@gmail.com>
* chat: fix whitespace problems once and for all
* Purge trailing spaces from grammar generation
* Revert "Purge trailing spaces from grammar generation"
This reverts commit b0827ecb7d.
* common : add common_chat_split_by_role
* cont : fix spans to reach end of message
* server: fix checkpoints creation
- extract message_spans from chat templates
- find the prompt token position before the latest user message
- split prompt batching at that position
- create a context checkpoint before the latest user input
- avoid periodic mid-prompt checkpoints when that position is known
- handle multimodal prompts when mapping text/template positions to server prompt tokens
- add --checkpoint-min-step to control minimum spacing between checkpoints
* cont : clean-up
* Support autoparser detection for message barriers
* server: fix message span delimiter and update docs
---------
Co-authored-by: Alde Rojas <hello@alde.dev>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Piotr Wilkin <piotr.wilkin@syndatis.com>
* common : handle incomplete UTF-8 at end of input in PEG parser
* cont : if reached end prematurely, emit needs_more_input to propagate partial output
* cont: refactor peg parse context to add lenient flag
* cont : remove partial flag, keep lenient flag
Toby
Guido Imperiale
Joe Rowell
Satinder Grewal
Piotr Wilkin (ilintar)
jacekpoplawski
Jhen-Jie Hong
Aldehir Rojas