Google Releases TorchTPU, Joining Amazon in Challenge to Nvidia

PyTorch is the popular open-source toolkit that AI developers and startups use to build, train, and run AI models; by some accounts, it is used by 85% of AI applications. Meta developed PyTorch in 2016, when Nvidia’s GPUs were already the standard hardware for AI research, and many of its features were built to work specifically with those chips. Meta turned the project over to the Linux Foundation in 2022.

Google said TorchTPU makes it easy to move existing PyTorch applications to TPUs and to Google Cloud. Developers can make the shift with a simple configuration change and without rewriting their applications, the company said.

Amazon made a similar announcement on Monday, releasing its own PyTorch version, TorchNeuron, which allows AI applications to be ported to Amazon’s homegrown Trainium chip and to AWS without rewriting. Amazon also said it is submitting code to the main PyTorch project that decouples various features from Nvidia’s chips, making the project more open to any hardware by default.

Google said it worked directly with Meta, Hugging Face, Inferact, the maker of vLLM, RadixArk, the maker of SGLang, and others to ensure that TorchTPU’s features and code libraries work well with those popular AI development tools, as the original PyTorch does. It also launched a TPU agent suite to help developers move their applications to Google’s chips and cloud and fix errors along the way, and updated its LiteRT tool, which runs AI on phones, laptops, and embedded devices, to work better with PyTorch applications.

The efforts do not mean AI developers will automatically abandon Nvidia’s hardware en masse now that they can more easily move to alternative chips and clouds. But Google and Amazon are clearly working to end Nvidia’s advantage, TechCrunch AI reported.