DeepSeek Releases V4.1-Flash With 1M Context, FP4 KV Cache, and MIT-Licensed Weights

The 40-layer backbone is split into a 20-layer causal encoder and a 20-layer decoder. Inspired by YOCO, the decoder does not compute its own global KV; per-layer projection weights derive it from the final encoder hidden state. Prompt tokens therefore stop at the encoder, nearly halving prefill compute, according to the report. Sliding-window attention with a 128-token window still runs in every layer, so decoder SWA states are rebuilt by replaying only the last 128 prompt tokens. The research team calls this Decoder SWA Bounded Replay.

V4.1-Flash uses Compressed Sparse Attention 2, or CSA2, to attack cache size along the layer axis. DeepSeek-V4 had mixed CSA with Heavily Compressed Attention; V4.1-Flash uses pure CSA2. Each CSA2 layer is statically assigned one of three modes. Full computes its own main KV, projects indexer K from it and selects fresh Top-512 indices. Reindex reuses main KV and indexer K from the last Full layer but rescores them with its own indexer Q. Reuse reuses both the main KV and the latest Top-K indices, skipping the indexer entirely. Every layer keeps its own main Q and SWA KV. The 18 CSA2 encoder layers use a compression ratio of 2 in three groups of six, with one Full and five Reuse layers. The 20 decoder layers use ratio 1 in five groups of four: the first is Full plus three Reuse, and the rest are Reindex plus three Reuse. A Hierarchical Sparse Indexer in the decoder lets the Full layer build a candidate pool of up to 16,384 positions, or 2,048 blocks of eight, so later Reindex layers score a bounded set instead of the entire context.

The main KV cache is quantized to E2M1 with one E4M3 scale per 16 channels, following NVFP4 without its global scale. The report said this is introduced through quantization-aware training in post-training and nearly halves storage against V4's FP8 cache. At deployment, SWA KV is no longer persisted to SSD. It lives in a distributed pool carved from 10% of host DRAM with a TTL of minutes, while global KV keeps a guaranteed 72-hour lifetime. On a miss, Encoder SWA Bounded Replay recomputes only 128 tokens instead of layers times window. Other changes include Single-Pass mHC, which shifts input-mixing coefficients by one block so a fused Mega-mHC kernel can halve activation memory traffic; the Engram conditional memory module at layers 1 and 14; DSpark speculative decoding trained after pre-training with the backbone frozen; and head-wise Muon. Single-token decode FLOPs rise by only one-quarter when context grows from 4K to 1M.

Pre-training covers 45T multimodal tokens at a 7:1 text-to-multimodal ratio. Sparse attention is trained from scratch at 64K sequence length with no dense warmup, and context is extended to 1M at 34T tokens. The base model matches DeepSeek-V4-Pro-Base on world knowledge and coding while using one-third of the total and one-quarter of the activated parameters, according to the report. Post-training introduces no new algorithms. Gains come from large-scale synthesis of verifiable agent tasks, reinforcement learning across heterogeneous scaffolds including Claude Code, Codex, OpenCode, Pi, mini-SWE and DeepSeek Harness, and on-policy distillation from over 40 teachers.

Selected max-effort results put V4.1-Flash at 90.6 on Terminal-Bench 2.1, compared with 82.7 for V4-Flash, 89.1 for Opus-5 and 88.8 for GPT-5.6 Sol. On DeepSWE v1.1 it scored 74.2, against 54.4, 74.0 and 73.0. Terminal-Bench 4.0 scores were 31.2, 7.0, 51.8 and 39.9. Automation-Bench scores were 54.8, 37.7, 50.3 and 45.8. GPQA Diamond scores were 90.9, 89.9, 93.4 and 94.1. On Codeforces rating, V4.1-Flash scored 3471 and V4-Flash scored 3289, while Opus-5 and GPT-5.6 Sol were listed as not available. The report said V4.1-Flash beats Opus-5 and GPT-5.6 Sol on Terminal-Bench 2.1 and DeepSWE v1.1 with MIT weights, and that the model and technical report are available.