Google unveils TorchTPU in push to loosen Nvidia's grip on AI compute

PyTorch, created by Meta Platforms Inc. in 2016 and open-sourced in 2022 under the Linux Foundation, is used to build, train and run AI models and neural networks. SiliconANGLE reported that more than 85% of AI applications are built with PyTorch, which was designed to work with Nvidia GPUs before large language models drove the current AI boom.

TorchTPU offers an alternative version that lets developers create models and applications for TPUs and run existing apps with a few configuration changes, without rewriting underlying code, according to the report. Google's effort follows Amazon Web Services Inc.'s announcement earlier this week of TorchNeuron, a PyTorch variant for Amazon's Trainum chips. Amazon also said it contributed code to the original PyTorch project to decouple key features from Nvidia GPUs and improve compatibility with other hardware.

Google said it worked with Meta, Hugging Face and startups Inferact Inc. and RadixArk Inc., the developers of vLLM and SGLang, to make TorchTPU features and code libraries compatible with their developer tools. It also introduced a TPU agent suite to help developers move AI applications to Google Cloud and automatically fix errors during migration, and updated LiteRT, a tool for building AI applications that run on smartphones, laptops and embedded devices rather than in the cloud. LiteRT is now compatible with PyTorch-based apps, Google said.

The article said it will take more than the launch to persuade millions of developers to drop Nvidia GPUs and move to TPUs, but the effort indicates Google believes it can reduce Nvidia's lead over time. No adoption figures or independent performance comparisons were provided in the report.