Glonce

Turbovec brings Google's TurboQuant quantizer to Rust

toolsThe Glonce editorsHacker News🔥74Glonce rating74Impact70Novelty60Relevance85Surprise40Players60Credibility85
Turbovec brings Google's TurboQuant quantizer to Rust

Turbovec is a newly released open-source vector index written in Rust, with Python bindings, built on Google Research's TurboQuant algorithm. TurboQuant is described as a data-oblivious quantizer with near-optimal distortion and no separate training phase: vectors are indexed as they are added, with no train step, no parameter tuning, and no rebuilds as the corpus grows. The project's headline comparison is memory use on a 10 million document corpus: stored as float32 that corpus takes 31 GB of RAM, and turbovec fits the same corpus in 4 GB. The project also reports that turbovec searches this compressed index faster than FAISS. Across benchmarks run on both ARM and x86 with 100,000 vectors, 1,000 queries and k=64 (median of five runs), turbovec's hand-written SIMD search kernels beat FAISS's IndexPQFastScan in every measured configuration: an average of 3.4 times faster at 4-bit quantization and 23% faster at 2-bit, combined across the eight benchmark cells of each width on both architectures. Broken out by platform, ARM averages 3.5 times faster at 4-bit (3.4 to 3.7 times across cells, using NEON SDOT/SMMLA dot-product kernels) and 26% faster at 2-bit (22% to 29%); x86 averages 3.4 times faster at 4-bit (3.2 to 3.5 times, using an AVX-512 VNNI kernel) and 20% faster at 2-bit (5% to 32%). Insert and remove operations are also benchmarked against FAISS on the same 100K-vector corpus: a single turbovec add() call takes 6.3 to 19.7 microseconds depending on the cell, 7.6 to 13.9 times faster than a single FAISS add, and a 100-vector batch amortizes to 4.6 to 16.3 microseconds per vector, 4.6 to 15.1 times faster than the same batch into FAISS. Removing a vector by id through turbovec's IdMapIndex takes 0.44 to 1.22 microseconds per operation in steady state, versus 0.19 to 1.02 seconds for a single remove through FAISS's remove_ids on an IndexIDMap at 100K vectors, with FAISS's cost doubling as code size grows. On recall, turbovec is measured against FAISS's IndexPQ (LUT256, nbits=8) baseline from the TurboQuant paper's Section 4.4, described by the project as a stronger baseline than the paper's own custom quantizer because FAISS uses a higher-precision lookup table and k-means++ codebook training. With per-coordinate calibration applied (labeled TQ+ in the project's charts), turbovec beats FAISS on R@1 on three of four OpenAI-embedding cells (d=1536 and d=3072) by 0.9 to 2.9 points, trailing by 0.7 points on the fourth (d=1536, 4-bit); both approaches converge to a recall of 1.0 by k=8. On the harder, lower-dimensional GloVe (d=200) dataset, calibrated turbovec leads FAISS at R@1 at both bit widths, by 1.9 points at 4-bit and 0.8 points at 2-bit, though FAISS regains a slim edge from around k=8 onward at 2-bit; calibration recovers that 2-bit deficit at R@1 specifically, reaching 0.572 against FAISS's 0.564. Calibration itself, via a one-time index.calibrate() call on roughly 1,024 sample vectors, is reported to gain up to 2.2 percentage points of recall at R@1 on the cells that drift most from the algorithm's underlying statistical assumption. Beyond raw performance, turbovec supports filtering search results to an id allowlist directly inside its SIMD kernel, at 32-vector block granularity, so that blocks with no allowed slots are skipped before scoring; it supports incremental, crash-safe saves that persist only what changed since the last sync; and it offers drop-in replacements for the in-memory vector or document stores used by LangChain, LlamaIndex, Haystack and Agno, installable as optional extras of the same Python package. The project pitches itself for local, air-gapped retrieval-augmented generation deployments where no vector data leaves the user's machine or network.

Key facts

  • Turbovec compresses a 10 million document vector corpus from 31 GB of RAM (float32) to 4 GB, while searching faster than FAISS.
  • Search benchmarks on 100K vectors, 1K queries, k=64 show turbovec beating FAISS IndexPQFastScan in every tested configuration: an average of 3.4x at 4-bit and 23% at 2-bit quantization across ARM and x86.
  • A single vector add() takes 6.3 to 19.7 microseconds, 7.6 to 13.9 times faster than FAISS; removing a vector by id takes 0.44 to 1.22 microseconds versus 0.19 to 1.02 seconds for FAISS's remove_ids at 100K vectors.
  • With per-coordinate calibration (TQ+), turbovec matches or beats FAISS on recall (R@1) on most tested configurations across OpenAI embeddings (d=1536, d=3072) and GloVe (d=200).
  • It ships as drop-in replacements for the default in-memory vector or document stores in LangChain, LlamaIndex, Haystack and Agno, and requires no separate training step as the index grows.

Why it matters

Vector search memory and latency are a real cost in retrieval-augmented generation systems, and turbovec's core claim is that both can shrink sharply without giving up accuracy or speed: an 8x memory reduction on the project's example corpus, search that beats FAISS's quantized index in every measured configuration, and no training phase to manage as the corpus changes. Because TurboQuant's calibration is optional and additive, an index can skip it entirely and run as plain TurboQuant, or calibrate once up front on a small sample before vectors are added, which keeps the operational overhead of adopting a new quantization scheme low.

Who it affects

The project targets teams building retrieval-augmented generation pipelines where memory footprint, query latency or data locality is a constraint: turbovec is pitched explicitly for local, air-gapped RAG stacks where no vector data leaves the machine or VPC. Because it offers drop-in replacements for the default in-memory stores in LangChain, LlamaIndex, Haystack and Agno, teams already using one of those frameworks can adopt it by swapping an import rather than rewriting their retrieval pipeline.

How to use it

Turbovec installs via pip (pip install turbovec) with a Python API, or via cargo add turbovec for direct Rust use; framework integrations install as extras, for example pip install turbovec[langchain]. A basic index is created with a fixed dimension and bit width (for example TurboQuantIndex(dim=1536, bit_width=4)), vectors are added with add(), and search() returns the top-k results, optionally restricted to an id allowlist or slot bitmask supplied by the caller. IdMapIndex adds stable external ids that survive deletion, with O(1) removal by id. Indexes persist with write()/load() for full snapshots or sync() for incremental, crash-safe saves that write only what changed. Calibration, which improves recall, is applied once via calibrate() on a representative sample of roughly 1,024 vectors before adding data, and index.calibration_state reports whether an index is calibrated or not.

How solid is it

The performance numbers come from the project's own benchmark suite, not an independent third party: 100,000 vectors, 1,000 queries, k=64, median of five runs, run on both ARM and x86 with single-threaded and multi-threaded cells. The project states its FAISS comparison point, IndexPQ with a 256-entry, 8-bit lookup table and k-means++ training, is a stronger baseline than the custom quantizer used in the original TurboQuant paper, and says it reproduces the paper's own TurboQuant numbers on OpenAI-sized embeddings. The project also flags one of its own measurement quirks: on small payloads, a full save-reopen-search round trip can time faster than an isolated post-mutation write, which it attributes to the standalone write step's fsync dominating at small file sizes rather than to any real advantage of the combined path.

Risks and caveats

The visible material does not name an individual or organization as the author of the turbovec project itself, only Google Research as the source of the underlying TurboQuant algorithm, and it gives no release date, version number or software license for the project. No CPU or hardware model is specified for the ARM-versus-x86 benchmarks. All performance figures are self-reported by the project's own benchmark suite rather than an outside party. The captured project text also cuts off mid-sentence during its explanation of Lloyd-Max scalar quantization, so any detail beyond that point in the original page is not reflected here.

“A 10 million document corpus takes 31 GB of RAM as float32. turbovec fits it in 4 GB - and searches it faster than FAISS.”

— turbovec project documentation