Hazy Research says AI agents are retiring CUDA kernel DSLs
Hazy Research, the Stanford systems group behind the CUDA kernel library ThunderKittens, published a blog post arguing that CUDA domain-specific languages (DSLs) are heading toward obsolescence because coding agents can now do the job those DSLs were built for.
The group's account: last year, building a working megakernel for Llama models required them to construct a full C++ abstraction layer, and even with that layer in place they spent a couple of months fighting race conditions and deadlocks before it ran fast. This year, building a mixture-of-experts (MoE) megakernel, they deleted the abstraction layer entirely. Instead, they had agents work through the low-level complexity directly and generate target-optimized code from scratch, with no intermediate C++ abstraction needed.
From this, the authors draw a general claim: an abstraction's traditional job is to act as a cognitive offloader, letting a person hold a manageable version of a problem in their head instead of the full complexity. They argue that job is now being taken over by agents, which can take a vague, incomplete prompt and act as a kind of compiler that produces working code from it. Tasks that previously needed no abstraction at all, such as writing an optimized GEMM kernel, are now nearly automated given the right prompt, though the authors still have to specify choices like PTX instructions and warp-specialization design. Tasks that did need an abstraction, such as writing a full megakernel, are not yet automated; the authors say they cannot yet one-shot a megakernel.
By inductive reasoning, the authors put CUDA DSLs, including their own ThunderKittens, on the same retirement trajectory, expecting it "probably next year, or maybe sooner," while saying they do not know how far the trend will extend.
The post frames this within a broader argument about codebases versus prompts: a codebase is precise but brittle, tied to one language, framework and hardware target; a prompt is fuzzy but portable, since a sufficiently capable executor, human or machine, can fill in unspecified gaps correctly. The authors compare this to the historical acceptance of compilers: most engineers no longer write assembly by hand because compilers got to be right more often than people were, and they suggest letting an agent sit even higher above the code is a difference of degree from that, not of kind.
The authors qualify their own claim in three ways. First, an abstraction is also a shared surface for reuse and review, not just a memory aid, so removing it risks fragmenting verification work across teams that no longer share common tile semantics to test against. Second, a layer can only be retired once there is an "oracle" that outlives it, meaning reference implementations, numerical tolerances and tests the authors trust; in the megakernel work, they say they kept that oracle even as they deleted the framework, and in domains where no such oracle exists, they say the scaffolding should stay. Third, they acknowledge being "a single, biased sample" working in a domain they already know deeply, and say they cannot tell whether an agent that fills in gaps for an expert does the same for someone encountering warp specialization for the first time; abstractions also exist to transmit knowledge to newcomers, and that role might still require frameworks even if agents make expert work require fewer of them.
The post concludes that what should survive a retired abstraction is the intent, the invariants, the tests and the domain knowledge currently encoded inside the library, while the specific compiled code becomes disposable, something regenerated rather than hand-tuned, with trust and scrutiny moving up to the spec and the oracle rather than the diff. The authors say they will keep maintaining ThunderKittens regardless, noting that kernels for Nvidia's upcoming Vera Rubin architecture are coming.
Key facts
- Hazy Research (Stanford, maker of the ThunderKittens CUDA library) built an MoE megakernel this year using agents with no C++ abstraction layer, after needing months to build and debug that layer for a Llama megakernel the year before.
- Tasks that never needed abstraction, such as writing an optimized GEMM kernel, are now nearly automated given the right prompt; megakernels still cannot be one-shot by agents.
- The authors predict CUDA DSLs, including their own ThunderKittens, are next to retire, "probably next year, or maybe sooner."
- They argue a layer can only be retired once there is a trusted "oracle": tests, reference implementations and numerical tolerances to check agent-generated code against; without one, the scaffolding stays.
- They plan to keep maintaining ThunderKittens regardless, including kernels for Nvidia's upcoming Vera Rubin GPU architecture.
Why it matters
The post reframes what an abstraction layer is for in systems programming: not a design nicety but a cognitive offloader that lets a person hold an otherwise unmanageable problem in their head. Hazy Research's claim is that agents can now take over that offloading role directly from a vague prompt, which is why a group that has spent years building and maintaining a CUDA DSL is arguing DSLs like it are heading toward retirement. They frame it as a shift in where trust sits: away from scrutinizing a diff or a codebase and toward scrutinizing the spec and the "oracle", the tests and reference implementations that check the agent-generated code.
Who it affects
Anyone who writes or maintains GPU kernels, or builds and maintains a DSL or framework meant to make that writing manageable, most directly Hazy Research's own ThunderKittens users and the wider CUDA-kernel and ML-infrastructure community that relies on hand-built abstraction layers to manage GPU programming complexity.
How to use it
The authors' own account reads as a template rather than a tool: keep the oracle, meaning tests, reference implementations and numerical tolerances, even after deleting the abstraction, because that is what lets you trust agent-generated kernels. Don't retire a layer in a domain where no such oracle exists yet; the authors say they'd keep the scaffolding there. What should survive isn't the code but the intent, the invariants and the domain knowledge currently encoded inside the abstraction, since the compiled implementation itself becomes disposable, a cache of one particular compilation rather than the source of truth.
How solid is it
This is a single team's first-person account of one project, an MoE megakernel built this year versus a Llama megakernel built the year before, published on Hazy Research's own blog with no independent verification. The authors give no performance numbers comparing the agent-built megakernel to the abstraction-built one, do not name the agent or model they used, and do not give exact dates beyond "this year" and "the past quarter." They explicitly call themselves "a single, biased sample" working in a domain they already know deeply, which limits how far the claim can be generalized.
Risks and caveats
The authors' own qualifications are the caveats: an abstraction is also a shared surface for reuse and review, and losing it risks a "Cambrian explosion" of unverified, disjoint kernels across teams that no longer share common test semantics. Retiring a layer requires an oracle that outlives it, and in domains where nobody knows what the oracle is, the argument does not apply. And the authors say they cannot tell whether an agent that fills gaps for an expert does the same for someone learning the domain for the first time, meaning abstractions may still be needed for onboarding even if experts need fewer of them.
“By inductive reasoning, CUDA DSLs are next on the retirement list, including our beloved ThunderKittens.”
— Hazy Research, "Retire the Abstractions"