Glonce

EdgeKernelBench benchmarks LLM-written kernels for edge backends

researchThe Glonce editorsHugging Face🔥57Glonce rating57Impact40Novelty60Relevance95Surprise35Players10Credibility72
EdgeKernelBench benchmarks LLM-written kernels for edge backends

A preprint listed on Hugging Face Papers takes on a gap the authors say has not been studied systematically: using LLMs to generate kernels for diverse edge backends. Earlier work, they note, shows LLMs can produce kernels with strong performance on server GPUs. Edge deployment is harder because the hardware is heterogeneous, the software stacks are fragmented, and the resource budgets are tight.

The first contribution is EdgeKernelBench. It combines a unified intermediate representation (IR) deployment framework with a hierarchical benchmark for edge workloads. The unified IR captures reusable computation, scheduling and memory decisions, and it connects to each target backend through a compiler interface. The benchmark spans progressively larger program scopes, from individual operators to compact model graphs, across representative edge application domains. The authors instantiate the design with TVM as a reference compiler and validate it across heterogeneous edge backends.

The headline result is a gap between correct and fast. With a single generation attempt, the best model reaches 64.5% Correct@1 but only 25.5% Fast@1.0, a difference of 39.0 percentage points. The authors read this as evidence that functional correctness does not consistently translate into performance gains.

That gap motivates the second contribution, the Layered Edge Kernel Agent (LEKA), which organizes optimization through a layered IR. On the agent evaluation subset, LEKA achieves 43.5% Fast@1.0, against 13.0% for direct generation and 26.1% for Claude Code. That is about 3.3 times the direct-generation figure and about 1.7 times Claude Code's. The authors state that LEKA and Claude Code use the same base model and optimization budget. They conclude that EdgeKernelBench and LEKA together support unified evaluation and layered optimization of LLM-generated kernels across edge backends.

Key facts

  • EdgeKernelBench combines a unified IR deployment framework with a hierarchical benchmark, from single operators up to compact model graphs, with TVM as the reference compiler.
  • With one generation attempt, the best model gets 64.5% Correct@1 but only 25.5% Fast@1.0 on the benchmark.
  • The Layered Edge Kernel Agent (LEKA) optimizes through a layered IR and reaches 43.5% Fast@1.0 on the agent evaluation subset.
  • On that subset, direct generation scores 13.0% and Claude Code 26.1%; LEKA and Claude Code share the same base model and optimization budget.

Why it matters

Per the authors, LLM kernel generation is known to work well on server GPUs, but edge backends have not been studied systematically. Edge targets bring heterogeneous hardware, fragmented software stacks and tight resource budgets. The paper offers a common way to measure the problem and a first agent aimed at it. Its sharpest finding is that code that runs correctly is often not fast: 64.5% Correct@1 against 25.5% Fast@1.0 for the best model.

Who it affects

The work speaks to researchers and engineers who want LLMs to write or tune kernels for edge deployment, and to those building compilers and benchmarks for heterogeneous backends. The benchmark covers representative edge application domains, from individual operators to compact model graphs.

How to use it

The abstract describes the design: a unified IR that links to each backend through a compiler interface, instantiated with TVM, plus the LEKA agent that works through a layered IR. No code or dataset release is mentioned.

How solid is it

This is a preprint abstract with concrete numbers, but the figures need careful reading. The 64.5% and 25.5% results are for the full benchmark with a single attempt. The 43.5%, 13.0% and 26.1% results come from the agent evaluation subset only, so the two sets are not directly comparable. The abstract does not name the best model or the base model used by LEKA and Claude Code. It does not give the size of the subset or the number of benchmark tasks, and it does not define Correct@1 or Fast@1.0, including the speed baseline for Fast@1.0. The comparison with Claude Code is stated to use the same base model and optimization budget, but no budget size is given.

Risks and caveats

The abstract does not list the specific edge backends or hardware tested, so how far the results generalize is unclear. The headline gains rest on a subset of unstated size. The abstract names no authors or institutions. Readers should treat the LEKA result as the authors' own evaluation rather than an independent replication.

“This gap shows that functional correctness does not consistently translate into performance gains.”

— EdgeKernelBench abstract