Cerebras says GPU prefill plus CS-3 decode gives 5x tokens per megawatt

Cerebras says GPU prefill plus CS-3 decode gives 5x tokens per megawatt

Cerebras has announced that its disaggregated inference is now in production. The idea is to split an inference request into its two stages and give each to the hardware that suits it. GPUs handle prompt processing (prefill), and Cerebras systems handle output generation (decode). Cerebras says that in 2026 it saw an insatiable demand for fast inference, both on its own inference cloud and through partners, and built the approach to meet it.

The headline claim is that the combination delivers five times more tokens per megawatt at the same ultrafast Cerebras speeds. In the measured production deployment on the Cerebras Inference Cloud, AMD Instinct GPUs process prompts and Cerebras CS-3 systems generate responses. Cerebras reports that this delivered 5X the throughput per megawatt of a CS-3-only configuration, with no loss in token generation speed. The data behind it is from November 14th, on a shared GPT-OSS-120B endpoint on the Cerebras Inference Cloud, comparing throughput per megawatt before and after enabling disaggregated inference, normalized to the aggregated inference baseline (1x). Cerebras's reasoning is that moving prompt processing to GPUs lets its systems devote their serving capacity to response generation. For inference providers, it argues, that means more capacity and greater revenue potential from the same power budget.

The post explains why Cerebras thinks it is advantaged at decode. Prefill processes the input and produces intermediate model state; decode uses that state to generate output tokens. Decode is autoregressive: the model generates one token at a time, using each token as input for the next, so every step reads model data from memory again and memory bandwidth matters a great deal. Cerebras argues that SRAM-centric chips have an advantage here because they keep that data on-chip and read it much more quickly. It says its wafer-scale chip has tens of gigabytes of SRAM and tens of petabytes per second of memory bandwidth, which it states as 350x the SRAM capacity and 5,400x the memory bandwidth of a typical GB300 GPU. That lets the system fit more concurrent requests and model layers on the chip. Cerebras also notes there are other ways to split inference, such as draft-target and attention-FFN, but says separating prefill from decode is uniquely advantageous for highly interactive inference.

The demand case cited is agentic applications such as Muse, Copilot and Codex, which Cerebras says are scaling rapidly and need highly interactive inference in a power-constrained environment. The post cites a Goldman Sachs Research report dated May 20, 2026 and Goldman Sachs data center capacity estimates reported by Investing.com.

Cerebras also built the solution for partners. Infrastructure partners can deploy disaggregated clusters from a Cerebras-provided reference architecture, co-locating their chosen prefill accelerators and Cerebras systems in a unified network. There are two software routes. The end-to-end stack, suited to partners optimizing time-to-market, has four parts: a disaggregated inference framework that routes requests between prefill and decode pools and manages replica capacity and health; a prefill engine built on open-source engines such as vLLM or SGLang; a Cerebras decode engine; and KV Portal, software that transfers intermediate model state, including key-value caches, between prefill and decode over RDMA. The decode-only SDK, suited to partners optimizing for control, lets them bring their own inference framework and prefill engine to the Cerebras decode engine and KV Portal, and adds development aids such as a mock Cerebras decode replica and a reference integration with an open-source disaggregated inference framework and prefill engine.

Early-access partners, including Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute, already have access to the solution for their deployments, and Cerebras is expanding access to more partners. Looking ahead, it plans to deploy disaggregated inference broadly across hundreds of megawatts in the Cerebras Inference Cloud and in partner deployments by the end of 2027. It frames this alongside its recently announced CS-4, scaled-up manufacturing and a planned 700MW of data center capacity.

Key facts

  • Cerebras says disaggregated inference is in production: GPUs do prefill (prompt processing) and Cerebras CS-3 systems do decode (output generation).
  • In the measured production deployment, AMD Instinct GPUs plus CS-3 delivered 5X the throughput per megawatt of a CS-3-only configuration, with no loss in token generation speed.
  • The 5X result comes from a shared GPT-OSS-120B endpoint on the Cerebras Inference Cloud on November 14th, normalized to an aggregated baseline of 1x.
  • Partners get a reference architecture and a choice of an end-to-end stack (with KV Portal over RDMA) or a decode-only SDK; AWS, Fireworks AI, Gimlet Labs and General Compute have early access.
  • Cerebras plans broad deployment across hundreds of megawatts, in its cloud and partner sites, by the end of 2027.

Why it matters

Cerebras frames the problem as power: agentic applications need highly interactive inference, and the industry must deliver much more inference capacity per megawatt in a power-constrained environment. Its answer is to stop making one kind of chip do both stages. Prefill goes to accelerators suited to compute, decode stays on wafer-scale SRAM-centric hardware. If the claim holds, the same power budget serves five times the tokens at the same generation speed, which Cerebras says means more capacity and greater revenue potential for inference providers.

Who it affects

Inference providers and infrastructure partners are the target. Cerebras is offering the solution so partners can pair their chosen prefill accelerators with Cerebras systems in their own datacenters. Early-access partners named are Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute. Customers of the Cerebras Inference Cloud are affected indirectly, since Cerebras says it can now serve more ultrafast tokens to meet demand.

How to use it

This is a partner offering, not a self-serve product; the post ends by inviting readers to contact Cerebras to discuss results, solutions or deployment opportunities. Partners build clusters from Cerebras's reference architecture and pick one of two integration routes. The end-to-end stack suits those optimizing time-to-market and includes a routing framework, a prefill engine using open-source engines like vLLM or SGLang, a Cerebras decode engine and KV Portal. The decode-only SDK suits those who want control: bring your own framework and prefill engine, and use the Cerebras decode engine and KV Portal, with a mock decode replica for development. Cerebras says it is expanding access to additional partners.

How solid is it

This is a first-party blog post, and all figures are Cerebras's own, with no independent verification of the results. The 5X figure is measured on one endpoint (shared GPT-OSS-120B on the Cerebras Inference Cloud) on a single stated date; no other models or workloads are reported. The baseline is a CS-3-only (aggregated) configuration, not a GPU-only setup, so no comparison with GPU inference is given. No absolute throughput, tokens-per-second, latency, power or cost figures are given, only the relative 5X. The early-access partners are said to have access, not to be running production deployments or to have achieved the 5X result.

Risks and caveats

Read the 5X as a ratio against Cerebras's own previous setup, not as a market benchmark. The post gives no pricing, no customer names for the production deployment, and no count of AMD GPUs or CS-3 systems used. It does not say AMD is a formal partner or that AMD announced anything; AMD Instinct is named as an example of a prefill accelerator and as the GPU used in the production deployment. The broad rollout across hundreds of megawatts by the end of 2027 is a plan, not a current figure, and the 350x SRAM and 5,400x bandwidth comparison with a typical GB300 GPU is Cerebras's own statement.

“Moving prompt processing to GPUs allows Cerebras systems to devote their serving capacity to response generation.”

— Cerebras, company blog post