Glonce

Glance lets one 9B RAG generator rerank its own passages

researchThe Glonce editorsHugging Face🔥60Glonce rating60Impact45Novelty60Relevance100Surprise35Players10Credibility70
Glance lets one 9B RAG generator rerank its own passages

Retrieval-augmented generation (RAG) systems commonly serve one model to rerank retrieved passages and another to write the answer. The paper behind Glance says this duplicates both model residency and passage processing.

Glance removes the second model. It equips the generator itself with a lightweight decision adapter and a pointer head, so the generator can score candidate passages from a shared prefill. Short relevance probes branch from the cached context and produce one score per passage, so there is no need to prefill each query-passage pair separately. A single resident 9B backbone then supports generation, general decisions and reranking, with approximately 0.1 GiB of additional memory per decision skill. A probe-only activated adapter lets generation continue from the original passage cache.

The reported results come after 39 minutes of MS MARCO fine-tuning. On retrieval, Glance reaches nDCG@10 of 0.741 on SciFact and 0.345 on NFCorpus, against 0.721 and 0.326 for bge-reranker-v2-m3. On 300 HotpotQA questions, the context Glance selects improves answer F1 by 0.045 over context selected by a cross-encoder, with the same base generator (paired bootstrap 95% CI [0.014, 0.076]).

In a scaling study, reranking quality keeps rising up to 200 candidates (43k tokens) scored from one shared prefill. At that size Glance exceeds the cross-encoder by 0.058 nDCG@10.

The authors describe these as initial results that demonstrate generator-based reranking with one resident backbone. Evaluation currently covers two retrieval datasets, one QA dataset and one backbone family.

Key facts

  • Glance adds a lightweight decision adapter and pointer head to a RAG generator so it scores candidate passages from a shared prefill, with no separate reranker.
  • One resident 9B backbone handles generation, general decisions and reranking, at approximately 0.1 GiB of extra memory per decision skill.
  • After 39 minutes of MS MARCO fine-tuning: nDCG@10 of 0.741 on SciFact and 0.345 on NFCorpus, versus 0.721 and 0.326 for bge-reranker-v2-m3.
  • On 300 HotpotQA questions, answer F1 rises by 0.045 over cross-encoder-selected context (95% CI [0.014, 0.076]).
  • Reranking quality keeps rising up to 200 candidates (43k tokens), where Glance beats the cross-encoder by 0.058 nDCG@10.

Why it matters

Serving a reranker next to a generator means two models in memory and passages processed twice. Glance tries to fold reranking into the generator that already holds the passages in its cache. The paper reports that each extra decision skill costs approximately 0.1 GiB, small next to a 9B backbone. If the approach holds up, a RAG stack could drop one model from its serving setup.

Who it affects

Teams that run RAG pipelines with a separate reranker and generator are the obvious audience, as are researchers working on retrieval and reranking. The paper's setting is one backbone family and a handful of benchmarks, so it speaks to them as a research direction rather than a drop-in replacement.

How to use it

The source mentions no code or model release, so there is nothing to download from it. The practical takeaway is the design: score passages with short probes that branch from the cached context, and use a probe-only adapter so generation can continue from the original passage cache. The reported training recipe is MS MARCO fine-tuning for 39 minutes.

How solid is it

This is a preprint, and the authors call the results initial. The gains are modest in absolute terms: 0.741 versus 0.721 on SciFact and 0.345 versus 0.326 on NFCorpus. The HotpotQA improvement of 0.045 F1 comes with a paired bootstrap 95% CI of [0.014, 0.076] over 300 questions. Evaluation covers two retrieval datasets, one QA dataset and one backbone family. No authors or institutions are named in the source text.

Risks and caveats

The comparison with bge-reranker-v2-m3 is reported for SciFact and NFCorpus only; the cross-encoder used on HotpotQA and in the scaling study is not named. No latency or throughput figures are given, only memory and quality, so the speed benefit of sharing a prefill is not quantified. The 200-candidate result is reported as a 0.058 nDCG@10 gap, without the absolute scores. The backbone model is identified only as 9B.