Tencent's Hunyuan Hy3 gets quantized versions, runs on single GPU

Tencent's Hunyuan team released quantized versions of its flagship Hy3 model (295B parameters), available in 1-bit and 4-bit. The original BF16 weights were nearly 600GB, requiring multi-GPU servers. The 1-bit version shrinks to 85.5GiB, running on a single 96GB GPU; the 4-bit version at 169.9GiB fits on two GPUs.

Performance holds up well. The 4-bit version closely matches the original in output distribution, with strong results in agent tasks, multilingual code, tool use, and long-context understanding. The 1-bit version shows minimal loss in long-text tasks and only slight drops in agent and code areas, enough for daily coding help and document processing.

A custom MTP speculative decoding patch boosts decoding speed by 50% for the 1-bit version and nearly 60% for the 4-bit version, ensuring smooth interaction. The quantized models are packaged as GGUF and work with llama.cpp, lowering the barrier for deploying large models.