# DeepSeek open-sources DeepInfer, the engine behind its production services

> DeepSeek released its DeepInfer inference framework under Apache 2.0, supporting Nvidia SM100 GPUs and Huawei Ascend 950 NPUs.

*The Apache 2.0 release hands rivals and cloud providers DeepSeek's serving stack, with support for both Nvidia and Huawei chips.*

By Bez · InsidersFeed
Canonical: https://insidersfeed.com/news/deepseek-open-sources-deepinfer-the-engine-behind-its-production-services

DeepSeek has put the software that runs its own paid services into the open, and it did so under one of the most permissive licences available. Inference efficiency decides margins in model serving, so anyone building a rival service can now study, and reuse, how DeepSeek does it.

On 20 October DeepSeek released DeepInfer under the Apache 2.0 licence. The [DeepInfer repository on GitHub](https://github.com/deepseek-ai/DeepInfer) describes it as "the high-performance inference framework that powers DeepSeek's large-scale production services." The framework targets DeepSeek-V4.1-Flash and supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs.

The kernel list is long: FlashMLA, DeepGEMM, DeepEP, TileKernels and DeepSelect, alongside TileLang. Features include attention data parallelism with balanced context parallelism, and mixture-of-experts expert parallelism with EPLB. It also offers draft-token acceleration with confidence-scheduled verification, called DSpark, an asynchronous zero-overhead batch scheduler and a batch-invariant mode. Constrained generation works by regex, JSON schema or EBNF grammar.

DeepSeek makes a headline throughput claim: about 155 trillion tokens per day per cluster, with a 96% input cache hit rate. These figures are DeepSeek's own and have not been independently verified.

The README says the framework was "originally built primarily on vLLM", and that it draws on ideas from SGLang and TensorRT-LLM. Reports of its popularity differ: one cites 5.5k stars, another more than 8,000 within hours. The repository itself is the place to check.

The chip support follows a recent statement. DeepSeek said last week it plans to pre-train on Huawei Ascend 950 chips by the end of 2026. One report says the launch came a day after a US Department of Homeland Security ban on DeepSeek in federal agencies; that has not been verified.

## The Insider view

The licence is the commercial point. Apache 2.0 lets cloud providers, hosting start-ups and enterprises run the stack without negotiating terms. That favours operators who serve open-weight models and want to cut serving costs, and it raises the bar for closed inference vendors whose edge was in the serving layer.

The dual-chip support matters as much as the code. Shipping Nvidia and Ascend paths together, a week after the Ascend pre-training statement, signals that DeepSeek wants its software to be the default on both hardware camps. Exposed are the frameworks it borrowed from, vLLM, SGLang and TensorRT-LLM, which now face a purpose-built rival. Watch for independent benchmarks of the 155 trillion token claim and for who adopts the code in production.

## Key takeaways

- DeepSeek gives away its production inference stack under a permissive Apache 2.0 licence.
- It claims about 155 trillion tokens per day per cluster; this is unverified.
- Support for Huawei Ascend 950 sits alongside Nvidia, following DeepSeek's Ascend pre-training plan.

## Sources

- [DeepInfer repository](https://github.com/deepseek-ai/DeepInfer) — GitHub (deepseek-ai), 2026-10-20