Glonce

AWS releases TorchNeuron to run PyTorch apps on Trainium, targeting Nvidia's lock-in

toolsThe Glonce editorsTechCrunch🔥70Glonce rating70Impact60Novelty62Relevance100Surprise40Players80Credibility85
AWS releases TorchNeuron to run PyTorch apps on Trainium, targeting Nvidia's lock-in

On Monday, Amazon Web Services released TorchNeuron, a tool meant to let AI startups move their AI apps onto Amazon's Trainium chip. TechCrunch frames it as a direct stab at the software advantage that has made Nvidia the default hardware choice for AI apps.

TorchNeuron works with PyTorch, a free, open source toolkit developers use to build, train and run AI models. According to Amazon, apps written with PyTorch can run on Trainium by changing just one line of configuration, with no code rewrite required. TorchNeuron is adapted from PyTorch, so developers who already know PyTorch already know how to use it. Amazon says about 85% of AI apps are written with PyTorch. Facebook originally developed the toolkit, and since 2022 it has been part of the Linux Foundation. Nvidia has spent years making sure its CUDA software, the infrastructure that lets apps run on Nvidia GPUs and its other hardware, supports PyTorch apps, so its hardware is now deeply embedded in the toolkit, essentially by default.

Amazon is not the first to try this. TechCrunch notes that PyTorch users can also port their apps to AMD GPUs or even Apple's chips with a simple configuration change. Amazon's pitch is that porting to Trainium is easy and also gives startups access to Amazon's entire cloud. AWS claims that is a better choice for startups than neoclouds, many of which are tied to Nvidia hardware, such as CoreWeave, Nebius and Firmus.

Colin Brace, VP of Technology Software at AWS Annapurna Labs, the unit that makes Amazon's custom chips, told TechCrunch by email: "You're building on AWS. You're not betting on a single-purpose compute provider and scrambling to build everything else around it." He added that on AWS the infrastructure to go from prototype to production is already there, so a founder can move fast and scale when needed.

On cost, AWS says the same chip handles both training (teaching the model) and inference (putting the finished model to work, as when it answers a user's question) and is tightly integrated with the cloud infrastructure startups need. Its cloud will therefore save them money, even if AWS does not offer the cheapest per-token price. Brace said Trainium's value is price-performance plus being one platform for both training and inference, which improves the economics across a full workload. As an example he cited Deepgram, an AI voice app provider and a competitor to ElevenLabs: it moved to Trainium using native PyTorch support and is processing 70% more audio per dollar at production-grade accuracy.

Amazon also says it is pushing PyTorch to be less automatically centered on Nvidia. In the blog post announcing the tool, the company wrote that it is contributing changes upstream to PyTorch itself, working with the PyTorch community to remove code that ties the framework to specific chips. The goal, it wrote, is to make PyTorch more hardware-agnostic for the entire developer community, not just Trainium users.

TechCrunch's own view is more guarded. Such a tactic is good for sellers of hardware and cloud services that compete with Nvidia, but one of the points of PyTorch is to hide the difficulties of getting AI code to run on hardware. The outlet says we will have to wait and see whether pushing Nvidia out that far gives developers more choice and more affordable options, as Amazon promises, or just more configuration headaches. If Amazon succeeds in deeply uncoupling PyTorch from Nvidia, though, TechCrunch says the effect on Nvidia could be much bigger than simply dealing with chip competitors.

Key facts

  • AWS released TorchNeuron on Monday; per Amazon, PyTorch apps run on Trainium by changing one line of configuration, with no code rewrite.
  • Amazon says about 85% of AI apps are written with PyTorch, where Nvidia's CUDA support has made its hardware the default.
  • AWS cites Deepgram, which it says processes 70% more audio per dollar at production-grade accuracy after moving to Trainium.
  • Amazon says it is contributing changes upstream to PyTorch to remove code that ties the framework to specific chips.
  • PyTorch apps can already be ported to AMD GPUs or Apple's chips with a simple configuration change, so Amazon is not the first to try this.

Why it matters

Nvidia's hold on AI hardware rests partly on software: years of work making CUDA support PyTorch apps have embedded its hardware in the toolkit by default. TorchNeuron goes after that advantage directly, and Amazon's upstream contributions to PyTorch aim at the same thing from the other side. TechCrunch argues that if Amazon succeeds in deeply uncoupling PyTorch from Nvidia, the consequences for Nvidia could be much bigger than ordinary chip competition. Brace put it this way: "The real story is removing the lock-in that kept developers from having a real choice at all."

Who it affects

AI startups and developers who build with PyTorch are the target audience, since TorchNeuron is meant to let them move apps to Trainium. Neoclouds tied to Nvidia hardware, such as CoreWeave, Nebius and Firmus, are the alternative AWS says is worse for startups. Nvidia is affected most if the upstream changes reduce its default position in PyTorch. Deepgram is named as a customer that moved to Trainium.

How to use it

Per Amazon, an app written with PyTorch needs a one-line configuration change to run on Trainium, and TorchNeuron is adapted from PyTorch, so existing PyTorch knowledge carries over. Porting also brings access to the rest of AWS's cloud for taking a project from prototype to production. No price of TorchNeuron or of Trainium instances is given; TorchNeuron's licence is not stated.

How solid is it

The central claims come from Amazon and AWS: the one-line change, the 85% share of PyTorch apps, and Deepgram's 70% gain. No independent benchmarks or verification of the one-line-change claim or of Deepgram's 70% figure are reported. The 70% is relative to Deepgram's prior setup, which the source does not name. The 85% figure is Amazon's claim, with no underlying study or time frame given. The details on the tool's release and Brace's comments come through TechCrunch, which spoke to Brace by email.

Risks and caveats

TechCrunch says it remains to be seen whether this gives developers more choice and cheaper options or just more configuration headaches. AWS itself says its cloud may not offer the cheapest per-token price; its argument rests on integrated training and inference economics. No timescale is given for the upstream PyTorch contributions, and it is not said whether PyTorch maintainers have accepted them. No Nvidia or PyTorch/Linux Foundation reaction is reported. The source does not say which Trainium generation TorchNeuron supports.

“The real story is removing the lock-in that kept developers from having a real choice at all.”

— Colin Brace, VP of Technology Software at AWS Annapurna Labs