AWS launches TorchNeuron in direct challenge to Nvidia's software lock-in

TorchNeuron works with PyTorch, the open-source toolkit developers use to build, train and run AI models. AWS said applications written with PyTorch can run on Trainium without rewriting code. Amazon estimates about 85% of AI applications are written with PyTorch. Facebook originally developed PyTorch, and it has been part of the Linux Foundation since 2022.

Nvidia has spent years ensuring CUDA, its software infrastructure for running applications on Nvidia GPUs and other hardware, supports PyTorch smoothly. That work has embedded Nvidia hardware deeply in the toolkit, largely by default, according to TechCrunch.

TorchNeuron is adapted from PyTorch, so developers familiar with PyTorch already know how to use it, AWS said. Amazon is not the first company to make it easier to move PyTorch applications to rival chips. PyTorch users can also port applications to AMD GPUs or Apple chips with a simple configuration change.

AWS argues that moving to Trainium gives AI startups access to its broader cloud. It says this is a better choice for startups than using neoclouds, many of which are tied to Nvidia and its hardware, including CoreWeave, Nebius and Firmus. Colin Brace, vice president of technology software at AWS Annapurna Labs, the unit that makes Amazon’s custom chips, said in an email to TechCrunch that developers are building on AWS rather than betting on a single-purpose compute provider and scrambling to build everything else around it. He said the infrastructure to go from prototype to production is already there, allowing a founder to move fast and scale when needed.

AWS said the same chip handles both training, which teaches an AI model, and inference, which puts a finished model to work, and is tightly integrated with cloud infrastructure startups need. Brace said Trainium’s value is price-performance plus being one platform for both training and inference, improving the economics across a full workload. He cited Deepgram, an AI voice app provider and competitor to ElevenLabs, as a customer that moved to Trainium using native PyTorch support and is processing 70% more audio per dollar at production-grade accuracy.

Amazon is also pushing PyTorch to be less automatically centered on Nvidia. AWS said it will contribute code to the main PyTorch project and work with the PyTorch community to remove code that ties the framework to specific chips. The company said the goal is to make PyTorch more hardware-agnostic for the entire developer community, not just Trainium users.

That effort could give developers more choice and lower costs, as Amazon promises, or more configuration headaches, TechCrunch reported. If Amazon succeeds in deeply uncoupling PyTorch from Nvidia, the effect on the GPU giant could extend beyond competition from rival chips. Brace said the real story is removing the lock-in that kept developers from having a real choice at all.

Editor's Summary

AWS released TorchNeuron to let PyTorch AI applications run on Trainium with a one-line configuration change, extending its push into Nvidia’s software stronghold. AWS argues the tool offers access to its cloud and combined training and inference on one chip, while also contributing upstream changes to make PyTorch less tied to specific hardware. The outcome could expand developer choice or add complexity, with potential consequences beyond chip competition.