Software Tuning Lifts DeepSeek Inference Throughput 6.87x on PCIe-Only GPUs, METASTONE Says

The company's Meta-Infer deployment engine targets GPUs that sit outside the official validation matrices of mainstream open-source inference frameworks. Such cards can download weights and start a service, but community default settings push operators back to slow generic implementations, keep communication tuning parameters written for NVLink hardware, and apply memory and parallelism defaults meant for other devices. On DeepSeek-V4.1-Flash, the community Day-0 baseline on eight PCIe-only cards returned 1,932 tokens per second of input throughput. After a first stage the company calls model-chip co-optimization, that figure reached 5,850 tokens per second, according to the report.

The first stage works by backfilling kernels the framework never reports as missing. It aligns metadata definitions between hardware and framework, corrects page size settings to unlock native high-performance operator paths, replaces compute kernels written for older devices, and widens the threshold at which the fast communication path takes effect. On DeepSeek-V4.1-Flash the company enabled a sparse-MLA prefill fast path, replaced an FP8 dense GEMM slow kernel and expanded PCIe-IPC fast-path coverage. Applying only this stage to DeepSeek-V4-Flash and GLM5.3 produced throughput gains of 20 to 33 percent, the report said.

Once those paths are open, the bottleneck shifts to communication overhead, parallel scheduling, memory allocation and repeated computation. The second stage fuses attention and projection operators, overlaps computation with collective communication, embeds statistical calculations into communication gaps, and rewrites collective communication logic for PCIe bandwidth. It then sets tensor and context parallelism separately for the prefill and decode phases, adjusts static memory ratios and KV cache capacity, and adds a risk-aware cache reuse mechanism that decides when to reuse or refresh cached state. On the same DeepSeek-V4.1-Flash deployment, these steps raised input throughput from 5,850 to 13,274 tokens per second, with support for a 1 million token context.

Across other models, DeepSeek-V4-Flash rose from 14,546 to 22,584 tokens per second, a 1.55-fold gain. GLM5.3 went from 3,236.78 to 6,222.72 tokens per second, a 1.92-fold gain, while its P95 first-token latency fell from 141.6 seconds to 46.6 seconds and its supported context grew from 270,000 to 1.05 million tokens.

At matched concurrency points, an NVIDIA B300 delivered roughly 4.9 to 5.6 times the input throughput of the eight-card machine on DeepSeek-V4-Flash, with the B300 configured per SGLang cookbook recommendations at 33,937 tokens per second at C8 and 60,486 at C256. On GLM-5.3 the ratio was 3.5 to 4.7 times, with the B300 at 16,384 tokens per second at C8 and 23,503 at C64, according to the report. The company said the remaining gap comes from software stack, caching strategy and model optimization choices rather than from the hardware.

The same approach was applied to MiniMax H3 video generation, combining sparse attention, the risk-aware cache reuse scheme and Turbo LoRA. End-to-end generation speed for a 15-second reference-image-to-video task improved 2.48 times, with peak memory held level with the original dense baseline. On one eight-card machine, text-to-video throughput reached 5.33 times the baseline and reference-image-to-video 4.98 times, without significant quality loss by the company's account. On a unified single-machine basis at 5 seconds, 768P and 16:9, the machine produced 296 text-to-video clips per hour and 131 reference-image-to-video clips per hour, against 439 and 225 for the B300, or about 67 percent and 58 percent of the NVIDIA figure. The comparison used 16 B300 instances with an in-flight setting of one, with text-to-video measured against the B300's FL2VA result. By machine throughput, roughly 2.6 units of the 6000D correspond to one B300 for text-to-video and about 2.9 for reference-image-to-video; with the cache plus LoRA scheme those ratios narrow to about 1.5 and 1.7.

METASTONE also reported validation on domestic GPUs, which likewise fall outside mainstream frameworks' verification lists. For those parts, the attention module has to explicitly enable a platform-specific NSA sparse attention implementation to avoid falling back to a slower path, while Shared Expert fusion features designed for NVIDIA and AMD platforms are switched off. Communication is configured by phase, with DeepEP normal mode for prefill and low-latency mode for decode, and Shared Expert and Routed Expert split into two streams to overlap with collective communication. That stream split alone produced an 11.26 percent throughput gain across 35 paired tests at identical configurations, the company said.

All of the work is contained inside the Meta-Infer engine, with no changes to model weights, model structure or task semantics, according to the report. The company describes its own position as an independent third-party full-stack computing operator that manages more than 20,000P of compute across more than 10 intelligent computing centers, with a stated service level above 99.95 percent availability. The DeepSeek figures cited a lossless NVFP4 quantized deployment of the weights. The article was published by QbitAI as an authorized reprint, with the views attributed to the original author.