GLIE compresses visual document retrieval to four vectors per page

Late-interaction retrieval, the current state of the art for visual document search, represents each page with roughly 1,000 separate vectors, which makes it accurate but expensive to store. Existing ways to shrink that footprint either keep a subset of the vectors or replace them with a local average, and both degrade sharply once the storage budget gets aggressive; the alternatives to those methods require retraining the encoder itself. The authors traced this degradation across three encoders and found two consistent geometric properties of the page vectors: they sit exactly on the unit sphere, and they cluster near a manifold with an intrinsic dimension of only five to six. Two practical results follow from that geometry. First, the k-means centroids normally used to build a compressed index fall slightly inside the sphere rather than on its surface, which systematically underestimates the MaxSim similarity scores used at query time; simply normalizing those centroids back onto the sphere is a free correction worth up to +0.093 nDCG@5 over raw centroids. Second, because the page manifold has so few effective degrees of freedom, the full set of roughly 1,000 vectors per page can be regenerated from only a handful of them. Building on this, the authors introduce Generative Late-Interaction Embeddings (GLIE): a small number of vectors per page, learned from the normalized centroids, that serve both as a lightweight search index and as the basis from which a decoder can regenerate the page's complete embedding set. At query time, the actual search runs only over these few vectors, and a decoder expands just the top candidates back out to the full vector set for exact rescoring. At four vectors per page on the ViDoRe v1 benchmark, GLIE retains nearly 80% of the uncompressed system's nDCG@5, compared with 70% for the best prior post-hoc compression method at the same budget. The GLIE network itself is small: 415,000 parameters, trained in under three GPU-minutes on just a thousand pages. At a matched training budget, fine-tuning the encoder instead does not even reach the accuracy of GLIE's training-free stage (the normalization correction alone), and the full GLIE system beats fine-tuning at every budget tested. The authors report that these patterns hold on a second encoder and on the ViDoRe v2 benchmark as well, not only on the original setup.

Key facts

Page embeddings in late-interaction visual document retrieval sit on the unit sphere and cluster near a manifold of intrinsic dimension five to six.
Normalizing k-means centroids onto the sphere's surface, instead of leaving them where standard k-means places them, is worth up to +0.093 nDCG@5 with no retraining.
GLIE, a 415K-parameter decoder trained in under three GPU-minutes on 1,000 pages, regenerates a page's full ~1,000-vector embedding set from just a few stored vectors.
At four vectors per page on ViDoRe v1, GLIE keeps nearly 80% of uncompressed nDCG@5, versus 70% for the best prior post-hoc method.
Fine-tuning the encoder at a matched training budget does not reach even GLIE's training-free (normalization-only) stage, and full GLIE beats fine-tuning at every budget.

Why it matters

Late-interaction retrieval keeps a separate vector per token or image patch instead of collapsing a document into one embedding, which is why it leads on visual document search, but storing roughly 1,000 vectors per page is the main cost that limits how large a searchable archive can get. Most compression work so far has tried to pick or average a subset of those vectors, and that approach hits a wall under tight storage budgets. GLIE reframes the problem: instead of sampling the vectors that exist, it learns to regenerate them from a compact code, which only works because the authors first showed the vectors live on a low-dimensional manifold rather than filling the full embedding space.

Who it affects

The result targets teams building or operating visual document search systems, such as retrieval over scanned reports, slide decks, or screenshots, where storage cost scales directly with the number of pages and the vectors per page. Anyone already using post-hoc compression on late-interaction embeddings is the direct comparison point, since GLIE is measured against the best prior post-hoc method rather than against uncompressed retrieval alone.

How to use it

Two pieces are separable and independently useful. The centroid-normalization fix requires no retraining and no new model, only projecting existing k-means centroids onto the unit sphere before computing similarity scores, and it is a free accuracy gain on top of any current post-hoc compression setup. GLIE proper is a compact decoder, 415,000 parameters, that the authors say trains in under three GPU-minutes on a thousand pages, making it cheap to fit for a given encoder and corpus rather than something that demands large-scale retraining.

How solid is it

The claims rest on evaluation across two encoders and two versions of the ViDoRe benchmark (v1 and v2), with the headline comparison, four vectors per page on ViDoRe v1, run against the strongest prior post-hoc compression baseline rather than a weaker one. The authors also control for training cost directly by comparing GLIE against fine-tuning the encoder at a matched budget, where GLIE still comes out ahead at every budget level tested. The abstract does not name the three encoders used in the initial degradation study or the second encoder used to confirm the pattern, and it gives GLIE's retained accuracy as 'nearly 80%' without a precise figure.

Risks and caveats

The reported numbers come from the paper's own abstract; independent reproduction, code availability, and any release or deployment plan are not mentioned in the source. The headline 80%-versus-70% comparison is specific to four vectors per page on ViDoRe v1, and behavior at other compression budgets or on document types outside the ViDoRe benchmarks is not detailed here. No authors or institutions are named in the abstract.