DeepSeek Opens Inference Framework as AI-Written Kernels Top Two Benchmarks
DeepSeek has open-sourced its inference framework DeepInfer, while an AI-written operator system, ForgeKernel, topped two benchmark leaderboards in the same week, beating a human-tuned DeepSeek operator.
Competition among Chinese large-model developers is moving from the model layer to the compute layer: even a strong model ultimately executes as operators on chips. For the same model and the same chip, a better operator suite can sometimes bring several-fold performance gains. Operator efficiency determines how fast a given graphics card can run and the price per token.
DeepSeek had already released model weights and operator libraries including FlashMLA and DeepGEMM. DeepInfer adds the model runtime chain to that open-source lineup. Among the released work, FlashMLA has a version hand-tuned line by line by engineers on chips; its operator performance reached more than 90% of the hardware limit, which Leiphone described as the limit human hand-tuning has reached on some chips.
That limit was surpassed the same week by ForgeKernel, according to the report. As of Oct. 22, ForgeKernel ranked first on two operator evaluation lists: the combined SOL-ExecBench FlashInfer-Bench leaderboard, on Nvidia's home turf, and the CANNBench A3 hardware leaderboard maintained by the official Ascend CANN repository. Its benchmark speedup was 1.835x, 25% ahead of second place.
Beyond the rankings, ForgeKernel also surpassed DeepSeek's open-source Nvidia version on DeepSeek v4.1 Flash operators through AI iteration from zero, the report said. The two benchmark lists cited were SOL-ExecBench FlashInfer-Bench and CANNBench A3.
Editor's Summary DeepSeek has open-sourced DeepInfer, adding the inference runtime to its previously released model weights and operator libraries. In the same week, Leiphone reported that ForgeKernel, an operator system written entirely by AI, ranked first on both SOL-ExecBench FlashInfer-Bench and CANNBench A3, with a 1.835x benchmark speedup. The results place AI-driven operator optimization ahead of a human-tuned DeepSeek operator that had reached more than 90% of the hardware limit on some chips.