Liquid AI ships LFM2.5-VL-DSpark vision drafter, claiming up to 3.13x faster on-device decoding

The drafter uses the same architecture as Liquid AI's text LFM2.5-DSpark drafters. It captures the target model's hidden states at a fixed set of tapped layers and conditions on them to draft a block of k candidate tokens. Image patches and text tokens are projected into a shared representation before those layers, so the drafter operates on hidden-state vectors of identical dimensionality regardless of input modality, and the inference algorithm is unchanged from the text models.

The draft model is a simplified attention-only drafter with four layers and a block size of 9, selected after ablations across three, four and five layers. Liquid AI ran 10 epochs on a mixture of vision-language supervised fine-tuning data weighted toward the workloads it expects the model to serve, measuring acceptance after each epoch; acceptance improved with additional training tokens before reaching diminishing returns. The company recommends a block size of 8 or 9 at inference depending on the hardware. Its components break down as 193.0M parameters in the four-layer decoder stack, 21.0M in the hidden-state projection, 65.5M in the Markov head and 6.4k in norms and the confidence head, for 279.5M total, an 8.9% increase over the 3B target.

Liquid AI evaluated the drafter at a block size of 8 on six vision-based tasks, including general VQA, text VQA, image captioning, chart VQA, complex reasoning and multi-turn conversation, following the MMSpec benchmark. With MLX on an M5 Max, decoding ran 2.30x to 3.13x faster by task and end-to-end latency improved by 1.56x to 2.62x. With llama.cpp on an M3 Ultra, decoding improved by 1.57x to 2.14x and end-to-end latency by 1.30x to 1.77x. On an H100, the same drafter produced end-to-end improvements of 1.64x to 2.27x.

The post gives H100 decoding speedups of "20.4x to 2.66x," a range whose lower figure exceeds the 2.66x maximum cited in its summary of results at the top of the page.

Liquid AI attributes the gap between decoding and end-to-end gains to prefill, which is mostly compute-bound and grows subquadratically with prompt length. In vision-language models, the image first passes through a vision encoder and the language backbone then processes hundreds of visual tokens alongside the text prompt. Edge devices have far less compute than datacenter GPUs, so prefill accounts for a larger share of end-to-end latency, as time-to-first-token and decode measurements on Apple silicon and the H100 show; the M5's per-core GPU neural accelerators narrow that gap. Speculative decoding speeds up only decode, not vision encoding or prefill, so when those stages already consume much of the wall time, even a large decode speedup yields a modest end-to-end gain.

Running the draft models with SGLang requires a build with DSpark support for LFM2 targets (PR #40651), launched with the DSPARK speculative algorithm, a flashinfer draft attention backend, a block size of 9 and the radix cache disabled; the block size is read from the draft's config.json. The llama.cpp path requires its own build (PR #29339) and is started with the draft model attached and spec-type draft-dspark. MLX-VLM requires a separate build (PR #2280), with the block size read from sidecar metadata and n-max clamped to it. Liquid AI says speculative decoding is exact because the target verifies every proposed token, so greedy output equals the target alone, and per-response timings report draft_n and draft_n_accepted.

The vision drafter is available on Hugging Face in Safetensors and GGUF formats. Liquid AI describes the LFM2.5 family as open-weight, supported from day one on llama.cpp, MLX and SGLang, and spanning base models for customization and specialized audio and vision variants on one architecture.