Glonce

BITCOS breaks the 1.585-bit barrier for ternary LLM weights

researchThe Glonce editorsHacker News🔥78Glonce rating78Impact70Novelty78Relevance95Surprise58Players25Credibility85

A paper submitted to arXiv on 14 September 2026 challenges a number long treated as a hard floor for ternary large language models: 1.585 bits per weight, the information-theoretic cost of a symbol drawn from three equally likely values, log2(3). Ternary LLMs store every weight as one of {-1, 0, +1}, and the deployment format used in practice, five-trit packing (five ternary weights per byte), actually costs 1.625 bits per weight once you account for the power-of-two group sizes real systems use. That format implicitly assumes the three symbols are equally likely. The authors checked that assumption against 29 real ternary LLM models and found it false: zeros make up as much as 51.5% of all weights in the sparsest of them, not the roughly 33% that equal probability would imply. Based on that measurement, they built BITCOS, a storage layout that adapts to the actual distribution instead of assuming a uniform one. It combines a dense bitmap marking which weights are zero with a compacted vector of signs for the nonzero ones, and its storage cost works out to 2 minus z bits per weight, where z is the model's zero density. At the 51.5% zero density measured on the sparsest model, that formula gives 1.485 bits per weight, below the 1.585-bit theoretical reference for a uniform ternary symbol. Across the 29 tested models, BITCOS beats five-trit packing on 26 of them. The authors also built optimized unpacking sequences for BITCOS on AVX-512, AVX2 and Intel Xe2 GPUs, since a more compact format is only useful if it can be unpacked fast. Measured against production state-of-the-art ternary matrix-vector multiplication kernels, at the zero densities real ternary models actually exhibit, BITCOS delivers a realized speed gain of up to 1.28x. In end-to-end LLM inference tests across five platforms, spanning client and server CPUs and integrated and discrete Xe2 GPUs, decode throughput improved by up to 1.18x on CPUs and up to 1.27x on GPUs.

Key facts

  • Measured across 29 ternary LLM models, zero weights account for up to 51.5% of all weights, far above the roughly one-third an equiprobable {-1,0,+1} split would predict.
  • The prevailing five-trit packing format costs 1.625 bits per weight; BITCOS instead costs 2 - z bits per weight, where z is the model's measured zero density.
  • BITCOS stores weights more compactly than five-trit packing on 26 of the 29 tested models, reaching 1.485 bits per weight on the sparsest one, below the 1.585-bit information-theoretic reference for a uniform ternary symbol.
  • Against production state-of-the-art ternary matrix-vector multiplication kernels, BITCOS's realized speed gain reaches up to 1.28x at real-world zero densities.
  • End-to-end inference tests on five platforms (client and server CPUs, integrated and discrete Xe2 GPUs) show decode throughput improving by up to 1.18x on CPUs and 1.27x on GPUs.

Why it matters

Ternary quantization is one of the most aggressive ways to shrink an LLM's memory footprint, and the field has treated 1.585 bits per weight (log2 3) as the natural cost floor because a symbol from three equally likely values needs that much information to encode. This paper shows that floor only holds if the three symbols really are equally likely, and in real ternary models they are not: zero dominates, by up to 51.5% in the sparsest model measured. Once the storage format is built around the actual, skewed distribution rather than an assumed uniform one, the effective cost per weight can drop below the textbook number entirely.

Who it affects

The result is aimed at people building or deploying ternary LLMs and the inference kernels underneath them: model quantization researchers, and engineers writing the CPU and GPU code that unpacks ternary weights at inference time. The paper explicitly targets both ends of that chain, from the storage format's bit cost to optimized unpacking code for specific instruction sets and GPU architectures.

How to use it

This is a research paper, not a shipped product; the source text names no license, library release or product. Beyond the storage layout itself, the authors present optimized BITCOS unpacking sequences for AVX-512, AVX2 and Intel Xe2 GPUs, so the practical path to using the format is through those kernel-level implementations rather than a drop-in library announced in the abstract.

How solid is it

The claims rest on measurements across 29 real ternary LLM models rather than a single synthetic case, and the speed gains are validated end-to-end across five different hardware platforms (client and server CPUs plus integrated and discrete Xe2 GPUs) against production state-of-the-art ternary matrix-vector multiplication kernels, not just a theoretical bit-cost comparison. The visible submission record names only one person, Evangelos Georganas, as the submitter of record; the source text available here does not show the full author list, so the paper's authorship beyond that cannot be confirmed from this text. It is also a preprint: no peer-reviewed venue or publication timeline is given.

Risks and caveats

The gains are not uniform: BITCOS beats the incumbent five-trit packing format on 26 of the 29 tested models, not all of them, and every headline number here (1.485 bits per weight, 1.28x kernel speedup, 1.18x and 1.27x decode throughput) is a ceiling reached on the sparsest or most favorable case, not a guaranteed average across models. The source text does not name which 29 models were tested or which one was the sparsest, and it gives no absolute wall-clock time or memory figures, only bits-per-weight counts and throughput multipliers, so the real-world footprint reduction in gigabytes or dollars is not stated.