Hugging Face: PyTorch now runs natively on Google TPUs via TorchTPU

Hugging Face published a post on 20 October 2026 saying PyTorch code now runs on Google's Tensor Processing Units (TPUs) through TorchTPU, a native PyTorch backend, and that its Transformers, Diffusers and Kernels libraries support it. Until now, the post says, running PyTorch on a TPU meant going through PyTorch/XLA (torch_xla), where tensors are lazy, small changes can trigger graph recompilation and debugging is hard to reason about, or learning JAX.

What changes in practice

With TorchTPU, TPU becomes another PyTorch device. Hugging Face shows installing it on a TPU virtual machine with one pip command, loading a model with device_map="tpu" in Transformers, and moving a Diffusers pipeline with pipe.to("tpu"). Models run in eager mode by default; torch.compile is optional for speed. TorchTPU ships Pallas kernels tuned for TPUs and supports distributed training.

For models too big for one chip, a TensorParallelConfig passed to from_pretrained shards the model across chips as it loads. Hugging Face says Optimum TPU, its earlier TPU library, is no longer required for Transformers and Diffusers.

The numbers Hugging Face reports

The post's one performance figure is for a Hub kernel that plugs a Tokamax ragged-dot operation into mixture-of-experts models. At batch size 64 on a Cloud TPU v6e-8, Hugging Face reports decoding 21.5 times faster on Qwen3-30B-A3B and 12.8 times faster on Gemma 4 26B-A4B than the default implementation. It adds that the bigger the batch, the bigger the gain. The comparison is against the default code path, not against GPUs.

On reliability, Hugging Face says it runs the Transformers test suite on TPU and that 96.5% of the tests that run there pass. It says the CI results are not all green and that it is still adding a nightly TPU run to the Transformers dashboard. It has reported 34 issues to the TorchTPU repository, 19 of them fixed, and says TPU runs also exposed bugs in Transformers itself, in StaticCache and in Qwen3 tensor-parallel plans, which it fixed upstream.

What the post leaves out

It gives no TPU price, no cost per million tokens, no throughput or latency comparison with GPUs, and no production deployment. The image-generation timings (for example, 3.5 seconds compiled versus 8.0 seconds eager for a 1024-by-1024 image on a v6e-4) show the value of compiling, not how TPUs compare with other hardware. The 96.5% figure counts only tests that run on TPU, so the share of the suite that cannot run there is not stated.

What a team can take from it

If you self-host open-weight models and your capacity is on Google Cloud, the barrier to trying TPUs for inference is lower: the test is a device string rather than a port. Treat it as an evaluation, not a migration. Run your own model at your own batch size, compare cost per million tokens with your current GPUs, and note that the kernel speed-up is reported only for mixture-of-experts models at large batches.