Cursor's MoK and Meta's Muse Glimmer: Two Approaches to Squeeze More From AI Hardware

Cursor's MoK focuses on Mixture-of-Experts (MoE) layers, where tokens are dynamically routed to experts across GPUs. In a typical MoE forward pass, tokens must be dispatched to the GPU holding the expert, computed, and then combined back. This Dispatch-Combine cycle, repeated in reverse during backpropagation, incurs overhead beyond mere data transfer: systems must count tokens per expert, arrange layouts, synchronize completion, and handle uneven loads. Even with fast libraries like DeepEP, the handover between Dispatch, Grouped GEMM, and Combine creates a scheduling problem—whether to wait for enough tokens to keep Tensor Cores busy or start computing early to hide communication.

MoK's key change is replacing the traditional Push-based Dispatch with Pull-based Dispatch. In Push, each source GPU actively writes tokens to a target GPU, requiring coordination on target addresses so that tokens from many senders don't overwrite each other. In Pull, the GPU holding the expert fetches the tokens it needs from source GPUs, eliminating address coordination and allowing data to be laid out directly for local experts. Although Pull moves slightly more data (172.0 KB vs 159.6 KB for a 256x256 BF16 block), it improves signaling latency from about 103 microseconds with Push to 18 microseconds with Pull in multi-node microbenchmarks, and increases NVLink utilization by up to 29% in unbalanced expert loads.

MoK further integrates communication and computation by splitting GPU SMs into two groups: one handling Dispatch, Combine, and state management, and another executing expert FFNs. A ring buffer reuses memory for incoming tokens, and a key parameter called minibatch controls how many tokens are handed to the expert GEMM at once. If too small, the GEMM lacks enough tasks to keep SMs occupied; if too large, computation starts late. Cursor uses the concept of a wave—when all compute SMs have work—to set a minimum of two waves per minibatch. On a Kimi 2.5-shaped model (hidden size 7168, expert intermediate 2048), increasing minibatch from 512 to 2560 tokens reduced forward time from 5.981 ms to 3.425 ms.

In Cursor's benchmarks on GB300 NVL72, compared to the fastest public baseline for each scenario, MoK achieved up to 2.37x speedup for MXFP8 forward and 1.78x for backward; BF16 forward and backward improved up to 1.92x and 1.58x respectively. In end-to-end training on 512 GB300 GPUs, Cursor's production setup with DeepEP achieved 760.9 tokens per second per GPU; with MoK it reached 1070.2, a roughly 41% improvement. Cursor did not release a complete ablation study, so the exact contribution of each component is unknown, but Pull's improvements to NVLink utilization and signaling latency were separately confirmed. MoK is tailored to Blackwell and NVL72-class NVLink domains, relying on low-latency access to remote GPU memory; optimal minibatch and SM counts vary with model shapes.

Meta's Muse Glimmer addresses a different bottleneck: running a 30B agent with 128K context on a single 24GB GPU. It uses a 52-layer dense transformer with hidden size 6656, 32 query heads but only 2 KV heads, and a pattern of three local attention layers (sliding window of 2048 tokens) followed by one global attention layer. This mix reduces the KV cache theoretically from about 6.5 GiB for full 128K context across all layers to around 1.7 GiB. If it used MHA-style 32 KV heads, the KV cache would be roughly 16 times larger. The model also uses quantization: a 17GB K Quant version for 24GB devices and a 20GB Dynamic K Quant for 32GB devices. Meta reports average accuracy loss of about 1.0% for K Quant 17GB and 0.2% for Dynamic K Quant on 15 benchmarks.

Muse Glimmer includes a 1.8B-parameter ViT G 14 vision encoder that can convert an image into up to 4096 visual tokens, but it only accepts text and image input and outputs text. In an agent workflow, screenshots are repeatedly captured and processed, and keeping all of them can fill the 128K context and create conflicting state. Meta's OSWorld Verified evaluation does not retain the full screenshot history but only recent screenshots, indicating that long context does not replace state management.

The model is distilled from a larger Muse Spark. Training uses logit distillation, where the student learns the teacher's probability distribution over the vocabulary, not just the selected token. Mid training adds long-context and reasoning traces, and post training includes on-policy distillation: the student rolls out and is supervised on the states it actually encounters, including errors, to improve failure recovery. The model supports four reasoning strength levels (low, medium, high, xhigh); higher levels produce more reasoning tokens, which can raise accuracy on complex tasks but also increase decode time.

To reduce decode latency, Muse Glimmer uses DFlash, a speculative decoding component that employs block diffusion instead of an autoregressive drafter. With a block size of 16, DFlash can generate candidate tokens in parallel, and it reads hidden features from layers 1, 13, 25, 37, and 49 of the main model to improve prediction accuracy. The drafter is only 5 layers deep.

Cursor's MoK and Meta's Muse Glimmer show a growing trend: AI startups and large companies are increasingly writing their own kernels and model architectures to squeeze performance out of fixed hardware. Cursor's MoK rethinks how tokens flow through MoE layers, while Meta's Muse Glimmer combines GQA, local attention, quantization, and speculative decoding to make a 30B agent fit on a consumer GPU. Both projects are open-sourced, reflecting a shift where software-level innovation is being used to reclaim performance that current frameworks leave on the table.