Google's TPUv7 (Ironwood) is now matching—and in some cases beating—Nvidia's B200 and GB300 chips on inference tasks, per SemiAnalysis's latest InferenceX benchmarks.

These are nightly, real-world tests: open-source models, long-context workloads, agentic tasks, tokens-per-dollar economics.

In FP8 comparisons, TPUv7 delivers 1.2× to 2.7× more tokens per dollar than Nvidia parts. In some setups, performance per dollar is up 50%. Raw throughput is often comparable, but Google's lower total cost of ownership—chip plus system—creates the edge. Nvidia still leads on certain FP4 workloads and highly optimized rack configurations.

The gap has closed not because of peak FLOPs, but because TPUs have improved across silicon, software (TorchTPU, vLLM, SGLang support), and economics. Performance still depends on model, precision, and serving setup—but the playing field is leveling.

This matters for anyone tracking AI infrastructure spend, hyperscaler capex, and the margin structure of inference at scale. The narrative that Nvidia owns inference is starting to crack.

$NVDA $GOOGL