Glonce

Hugging Face details hybrid search built for Papers with Code

infrastructureThe Glonce editorsHugging Face🔥48Glonce rating48Impact40Novelty38Relevance75Surprise22Players48Credibility82
Hugging Face details hybrid search built for Papers with Code

Hugging Face published a technical write-up describing the search system it built for Papers with Code, engineered by people with prior experience building RAG systems at the consultancy ML6. The system today maintains embeddings for more than 110,000 current papers sourced from arXiv and Daily Papers, and it combines PostgreSQL full-text search with pgvector-based semantic search rather than relying on either alone, because the team found hybrid search typically outperforms keyword-only or vector-only systems.

The architecture splits into an offline corpus build and an online query path. The offline side uses Hugging Face Jobs: a batch workload that exports a snapshot of every paper from a repeatable-read PostgreSQL database, streams it into checksummed JSONL shards, and syncs the run directory to a private Storage Bucket mounted directly into the Job's filesystem. Embedding runs on an l4x1 Job, a single NVIDIA L4 GPU with 24GB of VRAM, using Qwen/Qwen3-Embedding-0.6B pinned to an exact model revision. Every paper is encoded as its normalized title plus normalized abstract, and each embedding run records the model revision, output dimension, input-format version, query-versus-document flag, normalization method, and a content hash, treating the embedding format as a versioned API. Production vectors are 256-dimensional and L2-normalized, chosen using Qwen3's Matryoshka Representation Learning (MRL) feature, which lets the team trade retrieval quality against storage and speed by truncating a larger embedding down to a smaller one. In a 5,000-paper pilot, the Job encoded about 75 papers per second at 1024 dimensions on the L4 GPU; the same pass was also deterministically materialized at 512 and 256 dimensions to compare trade-offs without paying for extra inference.

Storage Buckets serve as the handoff layer between the production database, the ephemeral Jobs, and the search index: artifacts live under immutable run-ID prefixes, each covered by a manifest and checksums, so a generation can be validated, indexed, and only then atomically activated, with the previous generation kept available for rollback. An importer rechecks schemas, checksums, dimensions, normalization, and paper IDs before loading vectors into PostgreSQL and building a new HNSW index. On the 5,000-paper pilot, the 256-dimensional index achieved 0.9955 Recall@20 against exact search, with 1.31 ms p50 and 2.21 ms p95 HNSW lookup latency, while its table and index used about 27% of the storage of the 1024-dimensional version at essentially the same recall in that test. The post does not state whether these recall, latency, or storage figures hold at the full 110,000-paper production scale.

On the online side, a query is embedded live through an authenticated Inference Endpoint running Text Embeddings Inference (TEI), configured with a maximum of one replica and able to scale to zero when idle to save cost. Because a cold endpoint takes time to spin up, the query client enforces a one-second production timeout, a non-blocking concurrency limit, response validation, a short result cache, and a circuit breaker; if the endpoint is cold, slow, or returns a bad vector, the system falls back immediately to full-text search rather than making the user wait. For ranking, the lexical branch retrieves up to 50 candidates via weighted PostgreSQL full-text search and the semantic branch retrieves up to 50 candidates via pgvector; the two ranked lists are merged with weighted reciprocal rank fusion (RRF) using equal branch weights and a rank constant of k=60. On top of the fused ranking, the system preserves deterministic behavior for exact titles and arXiv IDs, recognizes navigational queries such as searches for a specific well-known paper, tolerates incomplete titles and minor typos via trigram matching, and abstains on ambiguous fuzzy matches rather than forcing a result. The authors close by noting hybrid search is not always the right starting point: they recommend beginning with a cheap, fast keyword-search baseline and adding semantic or hybrid search only when needed.

Key facts

  • The system maintains embeddings for more than 110,000 current papers from arXiv and Daily Papers, combining PostgreSQL full-text search with pgvector semantic search via reciprocal rank fusion (RRF, k=60, equal weights, up to 50 candidates per branch).
  • Offline embedding runs as a Hugging Face Job on an l4x1 (one NVIDIA L4 GPU, 24GB VRAM) using Qwen/Qwen3-Embedding-0.6B; in a 5,000-paper pilot it encoded about 75 papers per second at 1024 dimensions, with production using 256-dimensional L2-normalized vectors via Matryoshka Representation Learning.
  • Storage Buckets connect the production database, the Jobs, and the search index through immutable, checksummed run-ID prefixes, letting a new embedding generation be validated and indexed before being atomically activated, with the prior generation kept for rollback.
  • A single-replica, scale-to-zero Inference Endpoint (Text Embeddings Inference) embeds live queries behind a one-second timeout and a circuit breaker; if it is cold or fails, the app falls back immediately to full-text search instead of blocking.
  • On the 5,000-paper pilot, the 256-dimensional index hit 0.9955 Recall@20 against exact search with 1.31 ms p50 / 2.21 ms p95 HNSW latency, using about 27% of the storage of the 1024-dimensional version; the post does not confirm these figures at the full 110,000-paper production scale.

Why it matters

This is a rare public, detailed account of production search infrastructure for a large paper corpus, written by the team that runs Papers with Code's search. Rather than a generic RAG demo, it documents the operational discipline (versioned embedding contracts, checksummed artifacts, staged rollout and rollback) that separates a prototype vector search from one running against 110,000-plus documents in production.

Who it affects

It is most directly useful to engineers building search or retrieval systems on top of Hugging Face's Jobs, Storage Buckets, and Inference Endpoints, and to anyone designing hybrid lexical-plus-vector search with PostgreSQL and pgvector. Researchers and developers who use Papers with Code's search or its CLI benefit indirectly, since this is the system that serves their queries.

How to use it

The post includes a concrete hf jobs uv run invocation for launching the embedding Job with a chosen model, revision, dimensionality, and Matryoshka truncation, plus the SQL used for the pgvector cosine-distance query. Teams can adopt the same building blocks directly: Jobs for burstable batch embedding, a Bucket for the input/output handoff with manifests and checksums, and a scale-to-zero Inference Endpoint for query-time embedding with a strict client-side timeout and fallback.

How solid is it

The design choices are backed by concrete pilot measurements rather than assertions: throughput, Recall@20, HNSW p50/p95 latency, and a storage comparison across embedding dimensions are all given as specific numbers. Those measurements come from a 5,000-paper pilot, not the full production corpus of more than 110,000 papers, and the post does not state that the same recall, latency, or storage ratios hold at that larger scale.

Risks and caveats

The Inference Endpoint's cold-start and scale-to-zero behavior means the semantic branch can silently drop out under load or after idle periods, falling back to lexical-only results; the system is built to tolerate this, but it is a real trade-off for cost savings. The post gives no cost figures, team size, timeline, or production query volume, and the pilot's performance numbers may not directly generalize to the full corpus.