Glonce

Conditional Program compresses n-gram memory tables to two bits, cutting DeepSeek's table from 203GB to 51GB

researchThe Glonce editorsHugging Face🔥56Glonce rating56Impact40Novelty60Relevance95Surprise25Players12Credibility70
Conditional Program compresses n-gram memory tables to two bits, cutting DeepSeek's table from 203GB to 51GB

Conditional memory lets a language model grow its capacity through selective lookup: instead of using every parameter on every token, the model fetches only the table rows it needs. The catch, as the paper's authors put it, is that the full embedding table must still be kept available for every possible input. For n-gram memory models that table is large.

The paper introduces Conditional Program (CP), a compressed representation that reconstructs only the rows an input requests. Each row is stored as a short code plus a local scale, and a reconstruction program shared across the whole table turns the code back into a usable row. The authors build this with three pieces: a Gaussian trellis, norm restoration, and independently decodable row records. The last one matters for the lookup use case, since a row can be decoded on its own without unpacking the rest of the table.

The headline result is about accuracy at a very low bit width. CP's two-bit code lowers output KL by 41 to 63% relative to 2-bit round-to-nearest, the plain rounding baseline. That is a relative reduction against the baseline, given as a range; the abstract does not break it down by model or memory family.

The authors also analysed three trained n-gram memory families. They report that Gaussian-like coordinate statistics coexist with learned relationships among rows. At matched storage budgets, the shared trellis outperforms the fitted codebooks and the low-rank residual variants they tested.

For DeepSeek, CP reduces table storage from 203GB to 51GB. The authors say this leaves room for larger workloads when the table is kept on GPU.

Key facts

  • Conditional Program (CP) stores each n-gram memory row as a short code plus a local scale, with one reconstruction program shared across the table.
  • CP's two-bit code lowers output KL by 41 to 63% relative to 2-bit round-to-nearest.
  • For DeepSeek, table storage falls from 203GB to 51GB, which the authors say leaves room for larger workloads when the table sits on GPU.
  • The method combines a Gaussian trellis, norm restoration and independently decodable row records, so only requested rows are rebuilt.
  • Across three trained n-gram memory families, the shared trellis beat the fitted codebooks and low-rank residual variants the authors tested at matched storage.

Why it matters

Conditional memory adds model capacity without running every parameter per token, but its lookup table has to be stored in full. Shrinking that table attacks the storage cost directly. The reported DeepSeek figure, 203GB down to 51GB, is the concrete example: the authors say the smaller table leaves room for larger workloads when the table is kept on GPU.

Who it affects

Teams building or serving language models with n-gram memory tables, where the table size is the constraint. The paper's analysis covers three trained n-gram memory families, and the storage example is given for DeepSeek.

How to use it

The abstract describes a method, not a product. It names the ingredients: a Gaussian trellis, norm restoration and independently decodable row records, with a short code and local scale per row and a shared reconstruction program. No code release, publication date or venue is stated in the source text.

How solid is it

This is a preprint abstract, and the evidence is summarised in a few numbers. The 41 to 63% KL reduction is relative to 2-bit round-to-nearest only, and the source gives no absolute KL values and no breakdown of the range by model or family. The comparison with fitted codebooks and low-rank residual variants is limited to the variants the authors themselves tested, at matched storage budgets.

Risks and caveats

The source gives no downstream task accuracy, perplexity or latency results, so the effect on real model quality and speed is not shown. It does not say which DeepSeek model or n-gram memory variant the 203GB table belongs to. It also does not say whether the 51GB figure includes the shared reconstruction program or row-record overhead.

“Conditional memory expands language-model capacity through selective lookup, but the full embedding table must remain available for every possible input.”

— Paper abstract