Tencent Releases Lightweight Hy4 Preview, Compressed to 214GB

Tencent's Hunyuan team launched a lightweight version of Hy4 Preview, compressing the model weights from nearly 1.5TB to about 214GB. The model is based on the open-source MoE large language model released on August 28, with 770 billion parameters, 49 billion active parameters, and support for 1 million token context.

The compression uses Sherry sparse ternary quantization and MIX-STQ1_0 mixed precision, keeping performance stable on major tasks, with long-text understanding nearly matching the original and a slight dip in math ability.

For heterogeneous device inference, Tencent partnered with prima.cpp, achieving 1.02 token/s on a laptop with a single 4090 and a server with four A4000s, about 6 times faster than a single machine. The lightweight version is now on Hugging Face and GitHub, supporting llama.cpp, vLLM, and more.