Model: add Tencent Hy 4 (hy_v4) preview architecture support

[Little0o0]

Overview

Adds support for the Tencent Hy 4 model (Hugging Face architecture HYV4ForCausalLM, GGUF arch hy_v4)

This PR adds architecture support for the Tencent Hy4 preview model (Hugging Face architecture HYV4ForCausalLM, GGUF arch hy_v4). The model is published at https://huggingface.co/tencent/Hy4-preview. Some GGUF files with patches were published on https://huggingface.co/AngelSlim/Hy4-preview-GGUF , receiving 80K+ downloads in 2 days.

This PR is developed from official patch and referring branch by @fairydreaming .

Hy 4 is a DeepSeek-V3-style MLA + MoE model with several notable extensions that this implementation supports:

• Independent Hyper-Connections (iHC) — hc_mult parallel residual streams mixed per-layer via hc_attn_fn / hc_ffn_fn (pre/post) and a global hc_head_fn before the final norm. The HC pre/post reduce runs in fp32 to match the reference's numerical behaviour over the full stack of layers.
• Gated MLA — an element-wise sigmoid gate (self_attn.linear_gate) is applied to the decompressed attention output, before o_proj.
• Learnable sink — per-head learnable_sink_param passed into the attention sink term.

Changes by area:

Area	File(s)	Notes
C++ model graph	src/models/hy-v4.cpp	New llama_model_hy_v4 with iHC, gated MLA, learnable sink, MoE, optional DSA indexer
Arch registration	src/llama-arch.cpp, src/llama-arch.h, src/llama-model.cpp, src/models/models.h	Wire up LLM_ARCH_HY_V4
Hyper-parameters	src/llama-hparams.h, src/llama-context.cpp, src/llama-graph.cpp	HC / indexer / MoE params and graph routing
Vocab	src/llama-vocab.cpp, src/llama-vocab.h	LLAMA_VOCAB_PRE_TYPE_HY_V4 tokenizer pre-type
GGUF writer	src/llama-model-saver.cpp	Emit hy_v4 tensors / metadata
GGUF constants	gguf-py/gguf/constants.py, gguf-py/gguf/gguf_writer.py	MODEL_ARCH.HY_V4, tensor names, metadata keys
HF to GGUF conversion	conversion/hy_v4.py, conversion/__init__.py, conversion/base.py	HYV4Model (extends DeepseekV2Model), registered for HYV4ForCausalLM
Model list	convert_hf_to_gguf_update.py	Add hy_v4 entry (BPE tokenizer, tencent/Hy4-preview)
Tests	tests/test-llama-archs.cpp	Extend the architecture unit-test suite with hy_v4

Additional information

Testing

• python convert_hf_to_gguf.py <Hy4 checkpoint> --outfile hy4.gguf converts cleanly
• ./tests/test-llama-archs passes for the new hy_v4 entry
• Generated logits / sampled output match the reference implementation within tolerance

Not supported / intentionally dropped

• MTP / speculative decoding — num_nextn_predict_layers (the model.mtp_layers.* weights) are dropped.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: AI usage disclosure: YES. This implementation is developed from the official patch and real human fork, highly assisted by Claude Opus 5, working under the direction of @Little0o0, who commissioned, reviewed, tested the resulting builds on real hardware. If this PR cannot be merged under policy, it can still be a reference for the Hy4_v4 implementation.

[fairydreaming]

Links here should point to the model main HF repo.

Suggested change

	{"name": "hy_v4", "tokt": TOKENIZER_TYPE.BPE, "repo": "https://huggingface.co/AngelSlim/Hy4-preview-GGUF", "chkhsh": "e6ddf9c6686791c12d698d34c31ab9be1fea9af5a3d9a6909783ab382198ae1c"},
	{"name": "hy_v4", "tokt": TOKENIZER_TYPE.BPE, "repo": "https://huggingface.co/tencent/Hy4-preview", "chkhsh": "e6ddf9c6686791c12d698d34c31ab9be1fea9af5a3d9a6909783ab382198ae1c"},