Google and Meta Release TorchTPU, an Open Source Backend That Runs PyTorch Natively on TPUs

Google on October 20, 2026 released TorchTPU as an open source project on GitHub, a native PyTorch backend for its Tensor Processing Units (TPUs) built together with the PyTorch team at Meta. Porting a model is meant to be as simple as changing model.to("cuda") to model.to("tpu"), and the software already runs production inference and training for some of the largest Google Cloud customers, the TorchTPU team said in the PyTorch blog post.

PyTorch is the open source framework many AI developers use to build and train models, and many of its features were built for Nvidia's GPUs, which gives Nvidia an edge in AI chips beyond the hardware itself (TechCrunch). The release came one day after Amazon announced TorchNeuron, its PyTorch backend for Trainium. Google first described TorchTPU's design in a developer blog post on April 7, 2026, where it listed a public GitHub repository among its plans for the rest of the year.

What exactly did Google and Meta release?

Google and Meta released TorchTPU on October 20, 2026 as one of four open source repositories, alongside TPU versions of the vLLM and SGLang serving engines and TPU support in TorchTitan, Meta's reference platform for large scale training. The work started at Google in mid 2025 in collaboration with Meta's PyTorch team, according to the PyTorch blog post.

TorchTPU plugs into PyTorch's own extension point for new hardware, so tensors on a TPU are ordinary PyTorch tensors rather than wrapper types. It supports eager mode, where each operation runs as soon as it is called, and torch.compile, which turns a model into a graph that Google's XLA compiler optimizes. A third mode, DeferAndFuse, groups several operations together and can be 50% or more faster than plain eager execution, the team says. The backend covers about 450 PyTorch core operators and standard distributed training tools such as DDP, FSDP and DTensor.

The previous route, PyTorch/XLA from 2020, needed code changes, compiled slowly and lacked eager mode, the TorchTPU team says. Hugging Face now says its Optimum TPU library is no longer needed for Transformers and Diffusers models, which load on a TPU with device_map="tpu" (Hugging Face).

How well does TorchTPU work so far?

Google reports high but not complete coverage, and all of the figures below are the company's own measurements. Of 4,917 Hugging Face models tested end to end, 4,821 (98%) ran without crashing, and 638 of 657 models chosen to represent distinct architectures (97%) matched GPU results within Google's accuracy thresholds, according to the PyTorch blog post.

On training speed, Google compared PyTorch on TPUs with its own JAX stack, the framework TPUs were built around. Training the Qwen3 235B mixture of experts model on 64 TPU v7 chips reached 1,828 tokens per second with TorchTitan on TorchTPU against 1,937 with JAX on MaxText, which works out to about 94% of the JAX figure. Scaled from 1,024 to 8,192 TPU v7 chips, model FLOPs utilization (the share of the chips' peak math capacity doing useful work) stayed between 26.9% and 28.8%, Google says.

Hugging Face reports that 96.5% of the Transformers library tests that run on TPU pass, and that it has filed 34 issues on the TorchTPU repository so far, 19 of them fixed.

The team also lists limits. Support for dynamic shapes, where tensor sizes change at runtime, is not on by default, and FlexAttention is not supported yet. The default handling of collective operations, which keeps processes in step, can cost 30% to 40% of end to end speed compared with an optional mode that lets the compiler overlap computation and communication, the team says.

What else did Google launch alongside it?

Google launched new vLLM and SGLang inference engines built on TorchTPU on October 20, 2026, developed with Inferact and RadixArk, the companies founded by the creators of those two open source serving projects. Both engines already run in production with preview customers, Google said in a Developers Blog post.

On cost, Google says its TPU v7 (Ironwood) with the vLLM engine reached 2.01 cents per million tokens on the InferenceX 8K/1K benchmark run by research firm SemiAnalysis, down from 3.6 cents at its September debut.

Google also launched a TPU agent suite that helps developers move applications to TPUs and fix errors along the way (TechCrunch).

How does TorchTPU compare with Amazon's TorchNeuron?

TorchTPU and Amazon's TorchNeuron take the same approach, registering each company's chip as a standard PyTorch device, and they were announced on consecutive days around PyTorch Conference in October 2026.

Meta plays several roles in Google's version: it created PyTorch in 2016 before handing it to the Linux Foundation in 2022 (TechCrunch), it co-developed TorchTPU, and it already runs one of the new TPU serving engines in production.

Google counts individual models that run without crashing, while Hugging Face's TorchNeuron figure counts supported architectures, so the two numbers answer different questions.

What we don't know yet

• How TorchTPU performs against Nvidia GPUs on the same models in independent benchmarks, beyond Google's own measurements and its comparison with JAX.
• When FlexAttention support will arrive, and when dynamic shapes and FP8 or FP4 support will be production ready; the published roadmap covers the next 6 to 12 months without dates.
• Whether the new serving engines come with any change to TPU pricing or availability on Google Cloud.