Glonce

Cerebras explains how Kimi K3 (~3T parameters) maps onto 59 WSE-3T wafers

infrastructureThe Glonce editorsHacker News🔥64Glonce rating64Impact50Novelty45Relevance100Surprise30Players55Credibility70
Cerebras explains how Kimi K3 (~3T parameters) maps onto 59 WSE-3T wafers

Cerebras published a technical blog post answering a question it says it gets often: how does it run huge models on wafer-scale chips? A single WSE-3T wafer holds 44 GB of on-chip memory, while trillion-parameter frontier models need terabytes for weights alone. Cerebras says that by splitting a model across several wafers it can run multi-trillion parameter models at 1000-2000+ tokens per second.

The post uses the open source Kimi K3 as an illustrative example. Kimi K3 has about 3 trillion parameters, 69 linear-attention layers and 24 dense (full-attention) layers. Cerebras cites 88.3% on Terminal-Bench 2.1 (against 88.8% for GPT-5.6 Sol) and 91.2% on BrowseComp (against 90.4% for GPT-5.6 Sol), and says this shows frontier-level coding and agentic ability, even outperforming leading proprietary models on some benchmarks. The example assumes a 256k average request size, 4-bit weight quantization and an 8-bit KV cache.

Step one is layer size. Storing all 3T parameters at 4 bits each and dividing by 93 layers gives about 16 GB per layer. Step two is wafer count. A dense layer must keep keys and values for every token of every active request, so each dense layer gets a wafer to itself, with the remaining SRAM left for its KV cache. A linear-attention layer keeps only a small fixed-size state per request, whatever the context length, so two fit on one wafer (2 x 16 GB = 32 GB of 44 GB). That gives 35 wafers for the 69 linear layers plus 24 for the dense layers: 59 wafers in total.

Step three is connecting the wafers. Cerebras runs models as a pipeline: its compiler maps layers onto regions of wafers, and the runtime maps the path tokens take between them. Because whole layers fit on a wafer, the weights stay in place and layers are not split across wafers in most cases. Only activations cross from one wafer to the next, which Cerebras says can be done over RDMA over Ethernet at roughly 2 microseconds per hop. Each wafer is one stage of an assembly line; in the post's simplified picture, wafer 1 runs layers 1 and 2, wafer 2 runs layers 3 and 4, and so on until the last wafer produces the next token.

The post then addresses the worry that a token passing through wafers in order leaves each one idle, making the system effectively batch size one. Cerebras says no: it keeps the pipeline full with different requests, so while stage 3 handles a token for request A, stage 2 handles request B and stage 1 handles request C. In the example each stage is a layer; in production Cerebras uses much finer-grained stages and many more of them. Each stage can also process a micro-batch, and the number of requests in flight is roughly the number of stages times the micro-batch size. Smaller micro-batches make each request faster; larger ones raise total throughput.

Concurrency is capped by KV cache memory. Every in-flight request needs its own KV cache on each dense-layer wafer, so the space left after the 16 GB dense layer sets the ceiling. Only dense layers count here. To find the number of concurrent requests, you work out how much cache one 256k request takes and divide the free space by it; the post shows this as formulas whose results are not given in the text. Cerebras adds that, unlike a GPU batch, these requests do not wait for one another and stream through the pipeline continuously. It also argues that concurrency and batch size alone do not show how efficient an inference system is; the more meaningful metric is total token throughput per unit of power (for example tokens per second per megawatt) at the same output generation speed.

Finally, Cerebras says the same recipe scales past 3T parameters: compute the memory needed for weights and request state, distribute the model across wafers and connect them in a pipeline. Each added wafer brings compute and on-chip memory. The post closes by pointing readers to its free API at cloud.cerebras.ai and a sales contact.

Key facts

  • One WSE-3T wafer has 44 GB of on-chip memory; trillion-parameter models need terabytes for weights, so Cerebras spreads them over several wafers in a pipeline.
  • In the illustrative Kimi K3 example (~3T parameters, 69 linear-attention and 24 dense layers, 4-bit weights, 8-bit KV cache, 256k average request), each layer is about 16 GB, giving 35 + 24 = 59 wafers.
  • Dense layers get one wafer each, leaving room for the KV cache; two linear-attention layers fit per wafer. Only activations move between wafers, over RDMA over Ethernet at roughly 2 microseconds per hop.
  • Cerebras keeps the pipeline full with different requests at each stage, so it is not running at batch size one; KV cache memory on dense-layer wafers caps concurrent requests.
  • Cerebras claims 1000-2000+ tokens per second for multi-trillion parameter models and says tokens per second per megawatt at equal generation speed is the metric that matters.

Why it matters

Models with trillions of parameters do not fit in the 44 GB of on-chip memory on one wafer, so how they are split determines whether a wafer-scale approach can serve them at all. The post lays out the arithmetic in plain steps: layer size, wafers per layer type, pipeline links, and a memory ceiling on concurrent requests. It also gives Cerebras's view of what to measure: total token throughput per unit of power, such as tokens per second per megawatt, at the same output generation speed, rather than concurrency or batch size.

Who it affects

Mainly teams that run or evaluate large-model inference and want to understand how wafer-scale hardware handles models far bigger than one chip. It is also aimed at prospective Cerebras customers; the post ends with a link to its free API and a sales email. It touches users of open source frontier models such as Kimi K3, which Cerebras picks as its example.

How to use it

The post is an explainer, not a product release, but its method can be reused as a back-of-envelope recipe: compute the size of each layer from parameter count and quantization, check how many layers fit in 44 GB (one dense layer per wafer with KV cache room, two linear-attention layers per wafer), then derive wafer count. For concurrency, subtract one dense layer from the wafer's memory and divide the remainder by the KV cache of one request. Readers who want to try Cerebras can use the free API at cloud.cerebras.ai named in the post.

How solid is it

This is a first-party engineering post by Cerebras, and the numbers are an illustrative calculation, not a measured deployment. The 59-wafer figure follows from the stated assumptions (4-bit weights, 8-bit KV cache, 256k requests, about 16 GB per layer). The benchmark scores for Kimi K3 and GPT-5.6 Sol are as quoted by Cerebras, with no independent source or methodology given. The 1000-2000+ tokens per second figure is a general claim for multi-trillion parameter models, not a measurement for Kimi K3. The post does not say Kimi K3 is actually deployed in production on Cerebras.

Risks and caveats

The post gives no throughput-per-megawatt numbers and no comparison with GPU systems, even though it argues that metric is the one that matters. It does not state how many concurrent requests the 59-wafer configuration supports; the formulas are shown but not the results. Production uses much finer-grained stages and many more of them, and the post does not say how that changes the wafer count. It also gives no pricing, customer or availability date for Kimi K3 on Cerebras. The post itself says some benchmarks favor Kimi K3 and some favor GPT-5.6 Sol: it trails on Terminal-Bench 2.1 (88.3% vs 88.8%) and leads on BrowseComp (91.2% vs 90.4%).

“Unlike a GPU batch, those requests don’t wait for one another to finish, they stream through the pipeline continuously.”

— Cerebras engineering blog post