Moonshot AI open-sources Kimi K3 with 2.8 trillion parameters

Moonshot AI has officially open-sourced the Kimi K3 model, a massive AI with 2.8 trillion parameters. Built on a hybrid linear attention mechanism, it natively supports visual understanding and a 100 million token context window. The company says this is the first open-source model to reach this scale, and Kimi models have held the top spot for open-source size for nine out of the past 12 months.

Kimi K3 uses a Mixture of Experts (MoE) setup, activating 16 out of 896 experts efficiently. With better training methods and data, it's about 2.5 times more efficient than the previous K2 model. It's great at handling long coding tasks, working with large codebases, and using terminal tools with little human help. It also shines in tasks that mix software engineering and visual reasoning, like game development and front-end design.

The company also open-sourced supporting tech: MoonEP, FlashKDA, and AgentEnv, covering key areas like high-performance communication and distributed reinforcement learning. While Kimi K3 still trails top closed-source models, it performs at a frontier level in tests and beats all other models.