Tencent Hunyuan says smaller models can beat bigger ones

The Tencent Hunyuan team held a 120-minute AMA on Reddit to talk about model size. Asked about rivals pushing 3T to 5T parameter models, the team said Hy3 and Hy4 show that small, well-built models can beat bigger ones and cost less.

The team said Hy4 can still improve, especially with post-training. It is putting a lot of work into large-scale reinforcement learning, better user experience, and model safety, and plans to train larger models. It is considering dense models with 24B to 31B parameters. A small dense model to rival Qwen 27B is possible, but a coding-only model under 100B is probably not.

Yao Shunyu is Tencent's chief AI scientist and leads AI Infra and the large language model team. Tencent has upgraded its model R&D structure and merged its Hunyuan multimodal and language model teams into a foundation model department under him.

SponsoredDomoAI
AI Tools Mentioned

Comments

Loading comments…