Hugging Face Releases Granite Switch 4.1 for Chat, Decisions, and Speech on One Endpoint

The company said the models include a new Decision aLoRA for general-purpose decision-making, such as v1/systemone; a specialized Guardian aLoRA; and Granite-Speech-5.0-Turbo-CTC, which it described as the fastest ASR model on the OpenASR Leaderboard. The Guardian adapter, on average, outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on safety-related decision tasks, according to the blog post.

Hugging Face said all three technologies and the underlying base model are composed into one model. The combined model uses approximately the same amount of VRAM on a GPU as the base model by itself, which the company said means more capabilities with fewer GPUs.

The Switch adapters were built as cache-efficient Activated LoRAs, allowing the KV cache created from a base model generation to be reused for subsequent adapter-based decisions. Combined with vLLM contributions to speed up low-rank adapter inferencing and a co-located ASR encoder, the company said Granite Switch 4.1 models can switch between chat, decision, and spoken requests at high speed.

The release includes a Speech-to-Decisions Hugging Face Space where granite-switch-4.1-8b transcribes audio, generates summaries, screens for harmful content, and tests 15 other user-defined flags in real time on a single endpoint, according to the post.

Hugging Face noted decision models are not new. Its earlier Granite Libraries release included specialized decision model adapters and other task adapters that can be composed into custom Granite Switch models. While Jev and other closed decision models focus on generality, the company said many specialized granitelib adapters can outperform generalists on specialized classification tasks.

Because adapters share the base model's KV cache, adding specialists alongside a generalist decision adapter costs almost nothing, the post said. That setup provides broad coverage from the generalist and higher accuracy from specialists on tasks users care about.

In a "Race" between Granite-Switch-4.1-30B, which uses a frozen base Granite model plus four specialized aLoRAs, and Jev-27B, which uses a frozen base Qwen model plus one generic decision LoRA, the Granite model completed a real-world, five-step RAG workflow much faster than its competitor, according to the blog post.

Granite-Switch-4.1-8B and -30B are released with what Hugging Face called "batteries included," meaning they are compiled with a predefined set of helpful adapters. Users do not have to use that version. Anyone can compose their own Granite Switch model by mixing and matching adapters from Granite Libraries.

The company said it sees opportunity for the community to compose and extend Granite Switch models with custom and third-party adapters. It plans to release an Adapter SDK for training adapters in Mellea, along with tutorials and Granite Switch support. These adapters could serve as specialized classification adapters for decision-based workflows or adapters for specialized language tasks, the post said.

As an example, Hugging Face said a custom 3B parameter Granite Switch model made split-second decisions to navigate the classic game Doom, similar to Jev. The Granite Switch model did so at 60 to 100 decisions per second, compared with Jev's reported 10 decisions per second, according to the post. Unlike Jev, the Granite Switch model can also narrate and respond to questions while playing.