Glonce

Granite Switch 4.1 bundles chat, decisions and speech in one model

modelsThe Glonce editorsHugging Face🔥66Glonce rating66Impact55Novelty60Relevance100Surprise35Players60Credibility85
Granite Switch 4.1 bundles chat, decisions and speech in one model

The Granite team has published Granite Switch 4.1, released as Granite-Switch-4.1-8B and Granite-Switch-4.1-30B. Each model composes several pieces into a single model served from one endpoint: the underlying base model, a new Decision aLoRA for general-purpose decision-making (for example v1/systemone), a specialized Guardian aLoRA, and Granite-Speech-5.0-Turbo-CTC, an ASR model embedded for transcription. The authors describe Granite-Speech-5.0-Turbo-CTC as the fastest ASR model on the OpenASR Leaderboard, and say the Guardian aLoRA, on average, outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions and Clef-Flash on safety-related decision tasks.

The central claim is efficiency. All three technologies plus the base model are composed into one model that takes up approximately the same amount of VRAM on a GPU as the base model would by itself, which the authors summarise as more capabilities with fewer GPUs. The adapters are built as cache-efficient Activated LoRAs, so the KV cache created by a base-model generation can be reused for any later adapter-based decision. Together with the team's recent vLLM contributions that speed up low rank adapter inference, and a co-located ASR encoder, the models can switch between a chat request, a decision request and a spoken request at what the post calls blazing fast speeds. A Speech-to-Decisions demo in a Hugging Face space shows granite-switch-4.1-8b transcribing audio, generating summaries, screening for harmful content and testing 15 other user-defined flags, all on a single endpoint in real time.

The post also argues for specialists over generalists. Decision models are not new: the team's earlier Granite Libraries release holds specialized decision-model adapters and other task adapters that can be composed into custom Granite Switch models. While Jev and other closed decision models focus on generality, the authors say many of their specialized granitelib adapters can outperform generalists on specialized classification tasks. Because adapters share the base model's KV cache, adding specialists next to a generalist decision adapter costs almost nothing, giving broad coverage from the generalist and higher accuracy from specialists.

To illustrate, the post describes a "Race" between Granite-Switch-4.1-30B (a frozen base Granite model plus 4 specialized aLoRAs) and Jev-27B (a frozen base Qwen model plus 1 generic decision LoRA). Granite-Switch-4.1-30B completed a real-world, 5-step RAG workflow much faster than its competitor. No figures are given for the speed gap.

The released 8B and 30B models are shipped "batteries included": a predefined set of helpful adapters compiled into Granite Switch models. Users do not have to use that version. Anyone can compose their own Granite Switch model by mixing adapters from Granite Libraries. The team plans to release an Adapter SDK for training adapters in Mellea, along with tutorials and Granite Switch support, and sees room for custom and third-party adapters, whether specialized classification adapters for decision workflows or adapters for specialized language tasks.

As a playful example, the authors say a custom 3B-parameter Granite Switch model learned to play Doom. Like Jev, it made split-second decisions to navigate the game, but at 60 to 100 decisions per second, compared with Jev's reported 10 decisions per second (the figure comes from Jev's own published post, cited in a footnote). Unlike Jev, the authors say, this model can also narrate and answer questions while it plays. More details and data on Granite Switch are promised in the upcoming weeks. Linked resources include model cards, the Speech-to-Decisions demo, a GitHub repo, the original Granite Switch and Granite Libraries release blog, and a Colab for an aLoRA versus LoRA race.

Key facts

  • Granite-Switch-4.1-8B and -30B combine a base model, a Decision aLoRA, a Guardian aLoRA and the embedded Granite-Speech-5.0-Turbo-CTC ASR model behind one endpoint.
  • The authors say the whole bundle takes up approximately the same VRAM as the base model by itself, and that Activated LoRAs let adapters reuse the base model's KV cache.
  • In a "Race", Granite-Switch-4.1-30B (base plus 4 specialized aLoRAs) finished a 5-step RAG workflow much faster than Jev-27B (Qwen base plus 1 generic decision LoRA); no timings are given.
  • A custom 3B Granite Switch model played Doom at 60 to 100 decisions per second, against Jev's reported 10 decisions per second.
  • An Adapter SDK for training adapters in Mellea is planned, and anyone can compose their own Switch model from Granite Libraries adapters.

Why it matters

Serving chat, classification-style decisions and speech transcription usually means separate models and separate GPU memory. The authors claim one Granite Switch model covers all three at about the VRAM of the base model alone, with the KV cache reused when switching from a chat generation to an adapter decision. If that holds, the saving is in GPU count, which the post puts as more capabilities with fewer GPUs. The post also pushes a design argument: a generalist decision adapter for broad coverage, plus cheap specialist adapters where accuracy matters, set against closed generalist decision models such as Jev.

Who it affects

Teams running decision-style workflows, such as safety screening, routing or multi-step RAG, who want one endpoint instead of several models. Teams that need speech input alongside text are covered by the embedded ASR model. People who want custom behavior are addressed too: anyone can compose a Switch model from Granite Libraries adapters, and a planned Adapter SDK is meant for those training their own adapters in Mellea.

How to use it

The released models are Granite-Switch-4.1-8B and Granite-Switch-4.1-30B, which come with a predefined adapter set. The post links model cards, a Granite Switch GitHub repo and a Speech-to-Decisions demo in a Hugging Face space, where granite-switch-4.1-8b transcribes audio, summarises it, screens for harmful content and tests 15 other user-defined flags. A Colab lets you build your own aLoRA versus LoRA race. To go beyond the shipped set, compose your own model from Granite Libraries adapters. The Adapter SDK is only planned.

How solid is it

This is a first-party announcement from the Granite team, and every performance claim is the authors' own. No numeric benchmark scores are given for the Guardian aLoRA versus the named competitors, only the claim that it outperforms them on average. No numeric speed or latency figures are given for the Race against Jev-27B, only "much faster". The exact VRAM overhead is not quantified beyond "approximately the same". The Doom comparison uses Jev's reported figure of 10 decisions per second, not a measurement by the authors. The authors say more details and data are coming in the upcoming weeks.

Risks and caveats

The headline efficiency and speed claims cannot be checked from this post alone, since it gives no timings, scores or exact memory figures. The Race compares one 30B Granite setup with four specialized adapters against a 27B Qwen-based Jev with one generic adapter, a single workflow chosen by the authors. The Doom result is a showcase for a custom 3B model, not a shipped feature. No license, release date or download figures are given in the text, and no release date is given for the Adapter SDK.

“Anyone can compose their own Granite Switch model, mixing and matching adapters from Granite Libraries to best meet their needs.”

— Granite team, Granite Switch 4.1 post