Hugging Face releases 207 WebGPU kernels, 2.57x faster than ONNX Runtime Web

Hugging Face has released @huggingface/kernels, a JavaScript library for loading and running optimized WebGPU kernels straight from the Hugging Face Hub, together with an initial collection of 207 kernels hosted at huggingface.co/webgpu-kernels. The kernels cover operations that show up across most machine learning architectures, such as matrix multiplication, normalization, convolutions, attention primitives, quantization and data layout transforms, the individual GPU operations a browser based model eventually gets compiled down to. Each kernel ships as its own versioned repository rather than a loose shader file: a manifest.json defines the operation's contract (inputs, outputs, attributes, type constraints), a metadata.json records identifiers and provenance, test.json holds correctness cases, bench.json holds benchmark and tuning cases, and the actual implementation lives in parameterized WGSL templates. The whole collection is Apache 2.0 licensed.

Alongside the library, Hugging Face launched Fleet, an in browser GPU benchmarking and testing suite that runs the kernels on a visitor's own hardware and scores them. With the visitor's consent, each run privately contributes performance and correctness evidence back to Hugging Face from devices the company says it could never cover in a conventional test lab, which the team plans to use to catch device specific failures, compare kernel variants and improve selection rules.

On the performance side, Hugging Face benchmarked its kernels against ORT WebGPU (part of ONNX Runtime Web version 1.30.0-dev.20260826-b1f76d586a) on an Apple M4 GPU. The comparison started from 1,756 test cases spanning all 207 operations and kept the 809 cases where both implementations produced matching outputs and reliable timings. Across those 809 cases, Hugging Face's kernels were 2.57x faster by geometric mean and 1.90x faster at the median, winning 629 comparisons, losing 176 and tying 4. Some individual cases showed far larger gaps: a bilinear Einsum case (i,ij,j at size 4096) ran in 0.136 ms with Hugging Face's kernel versus 1,396 ms on ORT WebGPU, over 10,000 times faster, and a row wise CumSum over a [256, 4096] tensor ran in 0.016 ms versus 4.784 ms, a 301x gap. Hugging Face frames these two as unusual outliers rather than typical results, illustrating what happens when a general purpose implementation hits a slow path rather than a number to expect across the board. The timing measured only work done on the GPU itself, excluding setup steps such as loading kernels, creating sessions, uploading inputs, compiling shaders and reading results back, and the company says it is working with the ONNX Runtime team to upstream the improvements so the wider ONNX Runtime Web ecosystem can benefit too.

Hugging Face frames this release as the first layer of a larger effort to make fast local AI inference easier to build in the browser, sitting alongside its existing kernel collections for CUDA, ROCm and Metal on the Hub's kernel pages, with plans to connect the kernel layer to higher level model tooling and keep expanding operation coverage.

Key facts

Hugging Face released @huggingface/kernels, an npm package (currently tagged @preview) that loads and runs WebGPU kernels directly from the Hugging Face Hub, backed by an initial collection of 207 kernels at huggingface.co/webgpu-kernels, all Apache 2.0 licensed.
Each kernel ships as a versioned repository with a manifest.json contract, correctness tests (test.json), benchmark cases (bench.json) and parameterized WGSL shader templates, rather than a single unversioned shader file.
Benchmarked on an Apple M4 GPU against ORT WebGPU from ONNX Runtime Web 1.30.0-dev.20260826-b1f76d586a, across 809 comparable test cases the Hugging Face kernels were 2.57x faster by geometric mean and 1.90x faster at the median, with 629 wins, 176 losses and 4 ties.
Individual outlier cases showed much larger gaps: a bilinear Einsum case ran over 10,000x faster (0.136 ms versus 1,396 ms) and a row-wise CumSum ran 301x faster (0.016 ms versus 4.784 ms) than ORT WebGPU.
Hugging Face also launched Fleet, an in-browser benchmarking suite that, with visitor consent, crowdsources correctness and performance evidence from real-world GPUs to help tune kernel variants and catch device-specific failures.

Why it matters

Running AI models locally in a browser ultimately comes down to a sequence of raw GPU operations, and WebGPU exposes those operations through a portable API across modern browsers. But portability does not guarantee speed: two shaders implementing the same operation can behave very differently depending on workgroup sizes, memory access patterns, vectorization, data types and fusion strategy, and the best choice can shift with input shape, device, browser and available WebGPU features. Hugging Face is treating kernels as the foundational layer of fast browser inference on the theory that higher level runtimes can only be as efficient as the operations they dispatch, so making those operations individually discoverable, testable, benchmarkable and versioned lets the foundation improve without breaking the contract the layers above it depend on.

Who it affects

The release targets developers building browser-based or local-first AI applications in JavaScript, whether they use @huggingface/kernels directly or build their own runtimes and use the kernel repositories as reference implementations. It also affects the ONNX Runtime Web project, since Hugging Face says it is working with that team to upstream the improvements, and it invites anyone with a WebGPU-capable device to contribute benchmarking data through Fleet.

How to use it

The package installs from npm with npm install @huggingface/kernels@preview and requires a browser with WebGPU support, which can be checked in JavaScript with "gpu" in navigator. A developer calls getKernel with a Hub repository ID (for example webgpu-kernels/ai.onnx.Add) and a contract version, then invokes the returned function with typed input data and tensor shapes; the loader derives output shape and data type from the kernel's manifest and allocates the output automatically. The call pattern stays the same for heavyweight operations like matrix multiplication where optimized kernels matter most, and a single operation like elementwise addition can ship multiple variants (equal-shape, vectorized broadcasting, scalar, general broadcasting) that the runtime selects between without changing the application-facing API. The whole collection is Apache 2.0 licensed, so there is no cost or licensing restriction mentioned for using it.

How solid is it

The benchmark is Hugging Face's own, run on a single device type (Apple M4 GPU) against one baseline (ORT WebGPU from a specific ONNX Runtime Web dev build), and it measured pure GPU execution time while excluding setup costs like session creation, shader compilation and data transfer. Of 1,756 initial test cases, only 809 were kept for comparison after filtering for matching outputs and reliable timings, and Hugging Face itself flags the largest individual speedups (over 10,000x and 301x) as unusual outliers rather than typical results, cautioning that the numbers are best read as a useful comparison rather than a promise for every application. Each kernel does ship with its own correctness and benchmark test cases, and the Fleet tool is designed specifically to widen this evidence base beyond one lab's hardware.

Risks and caveats

Performance depends heavily on GPU, browser and driver, and Hugging Face is explicit that exact results will vary across that combination; the benchmark numbers describe individual operations, not complete models, so end-to-end application speedups are not established by this data. The article does not name a timeline for the promised ONNX Runtime upstreaming or for connecting the kernels to higher-level model tooling. Fleet's data collection is described only in broad terms, as privately contributed performance and correctness evidence gathered with consent, without the article specifying exactly what is collected or how consent and storage work in practice.

“These are unusual cases rather than the speedups you should expect everywhere, but they show how much a specialized kernel can help when a general implementation hits a slow path.”

— Hugging Face, WebGPU kernels blog post