TPUs vs. GPUs and why Google is positioned to win AI race in the long term
50 comments
·November 27, 2025mosura
This is the “Microsoft will dominate the Internet” stage.
The truth is the LLM boom has opened the first major crack in Google as the front page of the web (the biggest since Facebook), in the same way the web in the long run made Windows so irrelevant Microsoft seemingly don’t care about it at all.
dana321
That and the fact they can self-fund the whole AI venture and don't require outside investment.
jsheard
That and they were harvesting data before it was cool, and now that it is cool, they're in a privileged position since almost no-one can afford to block GoogleBot.
They do offer a way to signal to GoogleBot that your data is not to be used for training, but AFAIK there is no way to stop them doing RAG on your pages without destroying your SEO.
mlnj
They already have the data. They have the browser. Apps + users + talent. I see no reason to doubt Google with this.
mrbungie
The most fun fact about all the developments post-ChatGPT is that people apparently forgot that Google was doing actual AI before AI meant (only) ML and GenAI/LLMs, and they were top players at it.
Arguably main OpenAI raison d'être was to be a counterweight to that pre-2023 Google AI dominance. But I'd also argue that OpenAI lost its way.
lvl155
And they forgot to pay those people so most of them left.
OccamsMirror
To be fair, they weren't increasing Ads revenue.
paulmist
> The GPUs were designed for graphics [...] However, because they are designed to handle everything from video game textures to scientific simulations, they carry “architectural baggage.” [...] A TPU, on the other hand, strips away all that baggage. It has no hardware for rasterization or texture mapping.
With simulations becoming key to training models doesn't this seem like a huge problem for Google?
ricardo81
It's a cool subject and article and things I only have a general understanding of.
What I'm sure about is having a programming unit more purposed to a task is more optimal than a general programming unit designed to accommodate all programming tasks.
More and more of the economics of programming boils down to energy usage and invariably towards physical rules, the efficiency of the process has the benefit of less energy consumed.
As a Layman is makes general sense. Maybe a future where productivity is based closer on energy efficiency rather than monetary gain pushes the economy in better directions.
Cryptocurrency and LLMs seem like they'll play out that story over the next 10 years.
siliconc0w
Google has always had great tech - their problem is the product or the perseverance, conviction, and taste needed to make things people want.
thomascgalvin
Their incentive structure doesn't lead to longevity. Nobody gets promoted for keeping a product alive, they get promoted for shipping something new. That's why we're on version 37 of whatever their chat client is called now.
I think we can be reasonably sure that search, Gmail, and some flavor of AI will live on, but other than that, Google apps are basically end-of-life at launch.
jimbohn
Given the importance of scale for this particular product, any company placing itself on "just" one layer of the whole story is at a heavy disadvantage, I guess. I'd rather have a winning google than openai or meta anyway.
zenoprax
I have read in the past that ASICs for LLMs are not as simple a solution compared to cryptocurrency. In order to design and build the ASIC you need to commit to a specific architecture: a hashing algorithm for a cryptocurrency is fixed but the LLMs are always changing.
Am I misunderstanding "TPU" in the context of the article?
sbarre
A question I don't see addressed in all these articles: what prevents Nvidia from doing the same thing and iterating on their more general-purpose GPU towards a more focused TPU-like chip as well, if that turns out to be what the market really wants.
timmg
They will, I'm sure.
The big difference is that Google is both the chip designer *and* the AI company. So they get both sets of profits.
Both Google and Nvidia contract TSMC for chips. Then Nvidia sells them at a huge profit. Then OpenAI (for example) buys them at that inflated rate and them puts them into production.
So while Nvidia is "selling shovels", Google is making their own shovels and has their own mines.
1980phipsi
Aka vertical integration.
HarHarVeryFunny
It's not that the TPU is better than an NVidia GPU, it's just that it's cheaper since it doesn't have a fat NVidia markup applied, and is also better vertically integrated since it was designed/specified by Google for Google.
fooker
That's exactly what Nvidia is doing with tensor cores.
bjourne
Except the native width of Tensor Cores are about 8-32 (depending on scalar type), whereas the width of TPUs is up to 256. The difference in scale is massive.
LogicFailsMe
That's pretty much what they've been doing incrementally with the data center line of GPUs versus GeForce since 2017. Currently, the data center GPUs now have up to 6 times the performance at matrix math of the GeForce chips and much more memory. Nvidia has managed to stay one tape out away from addressing any competitors so far.
The real challenge is getting the TPU to do more general purpose computation. But that doesn't make for as good as story. And the point about Google arbitrarily raising the prices as soon as they think they have the upper hand is is good old fashioned capitalism in action.
sojuz151
They lose the competitive advantage. They have nothing more to offer than what Google has in-house.
blibble
the entire organisation has been built over the last 25 years to produce GPUs
turning a giant lumbering ship around is not easy
sbarre
For sure, I did not mean to imply they could do it quickly or easily, but I have to assume that internally at Nvidia there's already work happening to figure out "can we make chips that are better for AI and cheaper/easier to make than GPUs?"
sofixa
> what prevents Nvidia from doing the same thing and iterating on their more general-purpose GPU towards a more focused TPU-like chip as well, if that turns out to be what the market really wants.
Nothing prevents them per se, but it would risk cannibalising their highly profitable (IIRC 50% margin) higher end cards.
numbers_guy
Nothing in principle. But Huang probably doesn't believe in hyper specializing their chips at this stage because it's unlikely that the compute demands of 2035 are something we can predict today. For a counterpoint, Jim Keller took Tenstorrent in the opposite direction. Their chips are also very efficient, but even more general purpose than NVIDIA chips.
clickety_clack
Any chance of a bit of support for jax-metal, or incorporating apple silicon support into Jax?
null
villgax
riku_iki
It's all small products which didn't receive traction.
davidmurdoch
It's not though. Chromecast, g suite legacy, podcast, music, url shortener,... These weren't small products.
riku_iki
chromecast is alive, podcast, music were migrated to youtube app, url shortener is not core business and just side hustle for google. Not familiar about g suite legacy.
bgwalter
Google Hangouts wasn't small. Google+ was big and supposedly "the future" and is the canonical example of a huge misallocation of resources.
Google will have no problem discontinuing Google "AI" if they finally notice that people want a computer to shut up rather than talk at them.
riku_iki
> Google+ was big
how you define big? My understanding they failed to compete with facebook, and decided to redirect resources somewhere else.
Google's real moat isn't the TPU silicon itself—it's not about cooling, individual performance, or hyper-specialization—but rather the massive parallel scale enabled by their OCS interconnects.
To quote The Next Platform: "An Ironwood cluster linked with Google’s absolutely unique optical circuit switch interconnect can bring to bear 9,216 Ironwood TPUs with a combined 1.77 PB of HBM memory... This makes a rackscale Nvidia system based on 144 “Blackwell” GPU chiplets with an aggregate of 20.7 TB of HBM memory look like a joke."
Nvidia may have the superior architecture at the single-chip level, but for large-scale distributed training (and inference) they currently have nothing that rivals Google's optical switching scalability.