Tencent Hunyuan Hy3 Preview Language Model Released and Open-Sourced, Supporting 256K Context Length

Tencent's Hunyuan Hy3 preview language model has recently been released and open-sourced. It is a mixed expert model combining fast and slow thinking, with 295 billion total parameters parameters, 21 billion active parameters, and supports up to 256K context length, making it the most intelligent model in the Hunyuan series to date.

In February this year, Tencent rebuilt the pre-training and reinforcement learning infrastructure for Hunyuan. Yao Shunyu, Tencent's chief AI scientist, stated that Hy3 preview is the first step in the reconstruction of the Hunyuan large model, and the team will continue to expand the scale of pre-training and reinforcement learning to raise the model's intelligence ceiling.