DeepSeek open-sources DeepInfer, its inference framework

DeepSeek has fully open-sourced its internal inference framework, DeepInfer, along with a production-grade token encoding component called deepseek-recipe. The code is now available on GitHub. Earlier, the company had already released several large models and the TileLang compiler tool.

DeepInfer comes with high-performance operators and a deeply optimized engine. It supports distributed inference, speculative decoding, and structured output, and works on both NVIDIA and Huawei Ascend platforms. The deepseek-recipe component matches the official API logic, so developers get the same experience.

DeepSeek already uses DeepInfer to run large-scale inference for DeepSeek V4.1 Flash with high throughput and low latency. Inference code for future models will keep updating in the community.

DeepSeek says the inference framework manages huge computing clusters, and its efficiency directly affects the cost and power use of AI services. The goal is to help developers cut costs and energy use, making AI more affordable for more people.