Glonce

VectorWare brings Rust's portable SIMD to the GPU

toolsThe Glonce editorsHacker News🔥70Glonce rating70Impact65Novelty80Relevance70Surprise55Players30Credibility80

VectorWare, a startup building what it calls the first GPU-native software company, announced that GPU code can now use Rust's portable SIMD API (core::simd). The company had previously ported std::thread to the GPU by mapping each Rust thread to a GPU warp; that used the GPU's thread-level parallelism but left the parallel lanes inside each warp untouched. The new work fills that gap: NVIDIA's SIMT execution model, where a warp issues one instruction that every one of its 32 lanes runs on its own data, is treated as a form of SIMD, so a portable SIMD vector maps directly onto a warp. A Simd<i16, 32> value, for example, gives one i16 element to each of the warp's 32 lanes, and adding two such vectors compiles to a single warp instruction. The team reports this is a world first: the identical core::simd source that lowers to x86-64 SIMD instructions on a laptop now also lowers to warp instructions on the GPU, without any change to the source, and they show a recording of one program producing the same output on both CPU and GPU. Under the hood, elementwise operations (add, multiply, compare) map to native warp operations; reductions such as reduce_sum use the GPU's warp shuffle instructions; cross-lane shuffles reuse the same warp shuffle primitives; and SIMD masks map to GPU vote and ballot instructions. Because GPU hardware has a fixed lane width (32 on NVIDIA, 32 or 64 on AMD) while a CPU-side Simd<T, N> can use any N from 1 through 64, the mapping is one-to-one only when N matches the warp width; a narrower vector leaves lanes idle and a wider one takes more instructions. To handle work that does not fit neatly onto a warp, VectorWare built what it describes as an intermediate representation (IR) encoded directly in Rust's type system, using generics and trait bounds to type operations such as ballots, shuffles, reductions, scans, gathers, scatters, atomics and strip-mining for vectors wider than a warp, so that many invalid programs cannot be constructed at all. The company says this IR needs no interpreter on the GPU and each operation lowers straight to GPU instructions at zero cost over hand-written PTX; it also built a deterministic reference interpreter, which it compares to Miri, to run and differentially test the same IR on the CPU. The current work targets NVIDIA, but VectorWare says nothing in the design is CUDA-specific and that AMD wavefronts and Vulkan subgroups expose similar primitives, making the IR architecture-agnostic in principle. Portable SIMD in Rust is still an unstable, nightly-only feature (#![feature(portable_simd)]), and VectorWare says it had to change the Rust compiler itself to make the GPU mapping sound alongside other language features, adding that it is not yet confident every case is covered. Looking ahead, the company wants to combine its GPU threads, core::simd and async work together, lower matrix-shaped SIMD onto GPU tensor cores, and explore auto-vectorizing ordinary scalar Rust loops into SIMD operations, work it frames as appealing in part because its team includes members of the Rust compiler team. It also states that while a Rust type shared between CPU and GPU vector code is valuable, today's portable SIMD types may not be the right basis for one.

Key facts

  • VectorWare says it has mapped Rust's portable SIMD (core::simd) onto GPU warp lanes, so SIMD source that compiles to CPU vector instructions now also compiles to GPU warp instructions unchanged.
  • NVIDIA's SIMT model is treated as SIMD: a warp has 32 lanes, and AMD wavefronts have 32 or 64 lanes, while a CPU Simd<T, N> allows N from 1 through 64, so the mapping is zero-cost only when N matches the warp width.
  • SIMD reductions, cross-lane shuffles and masks are implemented with the GPU's native warp shuffle, vote and ballot instructions.
  • VectorWare built a Rust type-system-encoded IR for warp programming that it says lowers to GPU instructions at zero cost over hand-written PTX, plus a CPU-side reference interpreter for differential testing.
  • The work targets NVIDIA today and requires Rust's unstable, nightly-only portable_simd feature; VectorWare says it changed the Rust compiler to make the mapping sound and is not yet confident every case is covered.

Why it matters

Portable SIMD was designed so Rust code written once against core::simd compiles to whatever vector instructions a CPU target has, without vendor-specific intrinsics. VectorWare's claim is that the same abstraction now extends to the GPU: existing Rust code and libraries that already use core::simd become candidates for running on GPU warp lanes without being rewritten, and code that is GPU-aware in other ways can still drop down to architecture-specific intrinsics where needed. It follows the same company's earlier work mapping std::thread onto GPU warps, and together the two are pitched as completing a CPU-style parallelism hierarchy (threads containing SIMD lanes) on GPU hardware.

Who it affects

Rust developers who write performance-sensitive, data-parallel code and want it to target both CPUs and GPUs from a single source are the direct audience. It also matters to anyone building on VectorWare's toolchain specifically, since the mapping is implemented as compiler and IR work inside VectorWare's own stack rather than as a change shipped in upstream Rust. The company frames itself as building GPU-native software infrastructure generally, so the announcement is also a marker of that company's progress rather than a change available to all Rust users today.

How to use it

The feature requires Rust's portable SIMD, which is unstable and only available behind the nightly flag #![feature(portable_simd)], with a surface that may still change before stabilization. Within that, a Simd<T, N> behaves as an ordinary owned Rust value, with the borrow checker, lifetimes and type checking applying exactly as on the CPU; VectorWare says it is not introducing a GPU-specific vector type or new annotations, only mapping the existing portable SIMD API onto GPU execution. The demonstrated mapping targets NVIDIA hardware; VectorWare states the underlying IR is architecture-agnostic and that AMD wavefronts and Vulkan subgroups expose comparable primitives, but the post does not describe those targets as implemented or tested.

How solid is it

The claims come directly from VectorWare's own blog post, with no independent benchmark or third party verification included in the source. The company states its team includes members of the Rust compiler team, and it backs the design with a working demonstration: a program using elementwise arithmetic, a comparison producing a lane mask, a mask-driven select and a horizontal reduction, shown running on the GPU and printing output identical to the CPU run. It also built a deterministic reference interpreter for its IR to run the same warp-level programs on the CPU for differential testing. No throughput, latency or speedup figures are given anywhere in the source to substantiate the 'zero cost' claims, and no specific NVIDIA GPU model, driver or toolchain version used in the demo is named.

Risks and caveats

Portable SIMD remains an unstable, nightly-only Rust feature that may change before it stabilizes. The zero-cost property VectorWare describes holds only when a SIMD vector's width exactly matches the GPU's warp width; a narrower vector leaves lanes idle and a wider one turns a single operation into multiple instructions. Not every cross-lane shuffle maps cleanly to a warp instruction: arbitrary permutations can require several instructions or a trip through shared memory, and horizontal operations like reductions and any/all act as synchronization points that constrain scheduling. VectorWare says it had to modify the Rust compiler to keep the abstraction sound alongside other language features and states it is not yet confident every interaction is covered. No release timeline or product availability is given, and the AMD and Vulkan compatibility claims are architectural rather than demonstrated in the source.