Polymathic AI and Princeton Researchers Release FlashMap for Faster LLM Decoding

FlashMap uses a 0.5B-parameter drafter, with 500M parameters for Qwen3-8B and 475M for Qwen3-4B, and drafts blocks of K=16 tokens on the same GPU as the target model. The paper's speedup measurements use one NVIDIA H200 NVL at concurrency 1 in bf16. In matched tests, the authors retrained EAGLE-3 and DFlash on the same data and settings and evaluated nine benchmarks across math, code and chat at temperatures 0 and 1.

In speculative decoding, a small drafter proposes K tokens, and the target model checks all K in one parallel pass and keeps the longest matching prefix, so output remains exactly what the target model would produce. FlashMap maps Gaussian noise to a token block in one step, with each noise sample mapped to one full block and the noise selecting the mode for every slot jointly. The drafter is a 6-block diffusion transformer with embedding dimension 2048 and 16 attention heads. It reads hidden states from target layers 1, 9, 17, 25 and 33, while the embedding matrix and LM head are copied from the target and kept frozen.

Training runs in two stages. A flow teacher is trained as a continuous flow over one-hot token blocks with cross-entropy, and a time reparameterization handles Qwen3's 151,936-token vocabulary. A flow map student then uses semigroup distillation to turn the flow into a two-time denoiser that jumps from noise to data in one step. Both drafters train on 500,000 samples from Open-PerfectBlend, and total compute across all models and ablations was about 4,600 GPU hours on H200s.

The researchers attribute part of the acceptance problem in diffusion drafters to independent per-slot prediction, which they call the factorized approximation. In an example where a continuation could be "new york" or "san diego" with equal odds, an independent drafter proposes "new diego" or "san york" half the time, and the target rejects those at slot 2. FlashMap instead builds on Flow Map Language Models so that noise samples select the full block jointly.

On Qwen3-8B, one-step FlashMap was the fastest drafter tested on every task at both temperatures. Its acceptance length beat DFlash in 15 of 18 settings with about half the parameters. At temperature 0, gains over DFlash were 5.5% on math and 11.1% on chat. A two-step mode beat DFlash on acceptance length in all 18 settings, by 10% on average. EAGLE-3 at temperature 1 had 9% higher acceptance length but ran 35% slower. MarkTechPost calculated average speedups at temperature 0 from the paper's Figure 1(c): 4.36x for FlashMap, 4.07x for DFlash and 2.55x for EAGLE-3.

The best reported result was 6.14x speedup on GSM8K at temperature 0 with an acceptance length of 7.69 tokens; the worst was 2.28x speedup on Alpaca at temperature 1. On Qwen3-4B at temperature 0, FlashMap trailed DFlash on 4 of 9 tasks. Drafting four noise samples as a tree raised speedup by 11% on average. A shuffle test found that at two steps FlashMap recovered most of the joint-structure gain of a 16-step teacher, while at one step it kept only a small fraction of that gain.

For deployment, the flashmap package requires only torch and transformers>=4.56, and a spec_generate call runs the drafter and target together on one GPU. For serving, the project lists vLLM 0.31 with the drafter passed in --speculative-config and num_speculative_tokens set to 16, and SGLang 0.5.20 with --speculative-algorithm DSPARK and the FlashMap draft path; setting num_steps=2 trades some speed for acceptance. The comparison table lists FlashMap weights under Apache-2.0 and serving through Transformers, vLLM and SGLang.