LFM2.5-24B-A2B Brings Reasoning MoE to On-Device Agents

LFM2.5-24B-A2B builds on LFM2-24B-A2B and keeps its architecture: 40 layers, 64 experts per MoE block with top-4 routing, and a hybrid of gated short convolutions and GQA. The context window expands from 32K to 131K tokens, and an in-place tokenizer expansion doubles the vocabulary from 65K to 128K tokens for better non-Latin tokenization. Unlike its predecessor, it is a reasoning-only model that produces an explicit chain of thought before its final answer.

Training scaled pre-training from about 17T to about 33T tokens. Mid-training adds 2T tokens at 32K context, weighted toward reasoning, math, code, instruction following and multilingual text, plus 300B tokens at 131K context, weighted toward long-context data. Tool calls and multi-turn trajectories appear in both phases. Post-training has five stages: supervised fine-tuning, teacher specialization, non-agentic multi-domain on-policy distillation, agentic multi-domain on-policy distillation, and agentic reinforcement learning inside harnesses such as OpenClaw, Hermes Agent, Pi and OpenCode.

The post evaluated the model against Qwen3.6-35B-A3B, Gemma-4-26B-A4B-it and Nemotron-3.5-Lightning across 16 benchmarks for tool use, agentic workflows, knowledge, instruction following, math and coding. It says LFM2.5-24B-A2B leads or matches the competition on five of eight tool-use and agentic benchmarks: AutomationBench, BFCLv4, ToolSandbox, LiveClaw and τ³-Bench Banking. On LiveClaw, it scores 48.09 versus 38.19 for Qwen3.6-35B-A3B; on AutomationBench, 27.69 versus 8.72, more than three times the next-best score. It also leads all three instruction-following benchmarks, with IFEval at 96.75, IFBench at 74.69 and Multi-IF at 84.38. Larger models still lead on knowledge-heavy tests such as GPQA Diamond, HLE and AA-Omniscience, and on coding tests such as SWE-Bench Verified (OpenCode) and SciCode, so the post recommends pairing the model with retrieval or web search for knowledge-heavy workloads. Qwen3.6-35B-A3B also scores higher on PinchBench (89.93 versus 83.18) and WebArena Verified (37.62 versus 26.75), while LFM2.5-24B-A2B is ahead on τ³-Bench Banking (8.04 versus 7.63) and MATH-500 is close (96.04 versus 96.73).

For inference, the post says LFM2.5-24B-A2B is the fastest of the compared MoEs. It decodes 184 tokens per second on an Apple M5 Max and 114 tokens per second on a Ryzen AI Max+ 395, up to 1.7 times faster than the next-best model while using the least memory. On GPU, it reaches over 21K output tokens per second at high concurrency on a single NVIDIA B300. The model ships with day-one support across llama.cpp, MLX, vLLM, SGLang and ONNX; the post instructs users to install transformers version 5.0.0 or later.