Google will let companies run Gemini models in their own data centers
139 comments
·April 9, 2025blitzar
swalsh
The google search appliance might have been one of the worst products I've ever used in my career. If they're going to make a box, I hope they put some effort into it.
dehrmann
My theory is that the heuristics (PageRank and click-through feedback) that made 2008 Google great don't work in corporate environments.
er4hn
I actually mourn the loss of it. It felt so much better than any other accursed on-prem search solution I've seen since.
cyberpunk
fastsearch was pretty good before Microsoft bought it.. Elastic is good enough.
mosura
Strictly speaking Google still make boxes for people just in a different market.
What was so bad about the search appliance though? Physical? Software?
null
stingraycharles
The Netflix appliance is pretty good in my experience. No reason Google couldn’t pull something similar off themselves, unless they’re being very Google about it.
hinkley
Do they only sell those to ISPs or could a housing developer or a hotelier get their hands on those?
ijustlovemath
They have some incredible hardware talent (TPUs, Pixels), but I'm guessing this project will not get the polish of those more public facing products
sorokod
More like the GSA probably.
surajrmal
You're going to need a few specialized racks all wired up together. A single box won't be sufficient.
null
ashoeafoot
He personally stands behind the developed haedware, will you stand with him ?
unixhero
No, Jack Barkers' revamped version is needed.
FirmwareBurner
I miss that show. Too bad it ended right before the AI hype.
j_bum
I miss it too.
I think by the end I was far more invested in the characters rather than the plot though.
fonsai
Wasn't the last season mainly about AI?
anshumankmr
In fact, an AI that went rogue was the major plotpoint, so the satire is still on point.
radicalbyte
Fingers crossed that it'll do an Arrested Development.
candiddevmike
Richard has a student with an idea involving AI and joins his company as an advisor but can't keep his opinions to himself. Ends up ruining the company because everything he touches turns to shit.
next_xibalba
This tactic comes straight out of the Conjoined Triangles of Success playbook. It’s a classic Action Jack Barker move.
hinkley
On paper, Stephen Tobolowsky seems like he shouldn’t be successful enough of an actor to warrant an autobiography. But man do I love Ned “The Head” Ryerson in all his incarnations. What a strange, tall, little man.
That the world does not have a Stanley Tucci, Stephen Tobolowsky buddy comedy trilogy has made it all the poorer. But it’s been a while since someone tried to remake The Odd Couple…
nkassis
His signature could be the size of a full size rack on this one.
nsriv
This might be a great way for them to strengthen their model through federated learning.
djoldman
Google announcement:
https://cloud.google.com/blog/products/ai-machine-learning/r...
mkl
Interesting that the hardware is NVidia Blackwell, not Google TPUs. That means Google will likely have an energy efficiency and cost advantage, and keep their proprietary hardware out of other people's reach.
crowcroft
Getting a whole business set up to build TPU hardware for third parties (design, build, sell, support, etc.) is probably not worth it when there is overflowing demand for TPUs in their cloud already.
Businesses running their own hardware probably prefer CUDA as well for being more generally useful.
bitexploder
Part of the reason for this is likely due to customers preference to have CUDA available which TPUs do not support. TPU is superior for many use cases but customers like the portability of targeting CUDA
alienthrowaway
What are the pros of using CUDA-enabled devices for inference?
WalterGR
Google doesn’t make TPUs available to 3rd parties, right? I assume there would be tremendous reverse-engineering risk if they were to include them?
re-thc
> not Google TPUs
They're in limited supply. Even Google doesn't have enough for their own use.
holografix
This is obvious government contract baiting. Kudos though they might actually move some Google Distributed Cloud this way
noitpmeder
Financial firms with significant on-prem datacenter use will love this as well. My company still stays away from the cloud -- we have 6 DCs in the building, and run everything else out of colocated racks.
brcmthrowaway
Who provides internet
aduffy
I don’t think so. To my knowledge GCP has no approval for classified networks, which is by far the hardest part. Contrast with Azure OpenAI has been approved to run on government networks for over a year now.
This feels like a play for companies in highly regulated industries, GCP has a notable list of biopharma customers.
Maxious
>Today at Google Cloud Next, we're thrilled to announce another significant milestone for Google Public Sector: the authorization of Google Distributed Cloud Hosted (GDC Hosted) to host Top Secret and Secret missions for the U.S. Intelligence Community, and Top Secret missions for the Department of Defense (DoD).
https://cloud.google.com/blog/topics/public-sector/google-pu...
aduffy
You are right, I should’ve RTFA
ZeroCool2u
FedRAMP High is the mark you really want to hit for the US Government and GCP's service coverage is surprisingly broad in that realm.
skybrian
From Google's blog post:
> Our GDC air-gapped product, which is now authorized for US Government Secret and Top Secret missions, and on which Gemini is available, provides the highest levels of security and compliance.
nkassis
Banking as well, this is the kind of offering they've been looking for a while. Google just saw the demand decided to jump in while OpenAI and Anthropic probably calculated they don't have the manpower to deal with the support for this.
null
reaperducer
This is obvious government contract baiting
You don't have to be a government agency to not want your company's data all over the place.
connicpu
With a few exceptions for companies with highly secretive data, you do have to be a government agency or working in a highly regulated government-adjacent area for secured private clouds to be a requirement carved in stone and therefore worth investing a ton of extra money into though.
reaperducer
Just off the top of my head: Healthcare. Banking.
culopatin
They’ll have to fight Microsoft who’s been promising copilot.
rwmj
A bit thin on detail, but will this require confidential VMs with encrypted GPUs? (And I wonder how long before someone cracks SEV-SNP and TDX and pirate copies escape into the wild.)
vasco
At the pace models improve, the advantage of going the dark route shouldn't really hold for long, unless I'm missing something.
miohtama
Access to proprietary training data: Search, YouTube, Google Books might give some moat.
maxloh
We have Common Crawl, which is also scraped web data for training LLMs, provided for free by a non-profit.
hdjjhhvvhga
Are the differences between Google Books and LibGen documented anywhere? I believe most models outside of Google are trained on the latter.
bjackman
> I wonder how long before someone cracks SEV-SNP
https://bughunters.google.com/blog/5424842357473280/zen-and-...
unsnap_biceps
The number of folks that have the hardware at home to run it is going to be very low and the risk of companies for leaking it is gonna make it unlikely IMHO.
RadiozRadioz
It only takes one company to leak it
franga2000
Or one company to get hacked and the hackers leak it
spacebanana7
Realistically the only people able to run models of this size are large enterprises.
Those enterprises won’t take the risk of being sued for using a model without proper permission.
notpushkin
I think home users would be the least of their concerns.
BiteCode_dev
They can get "hacked" and wooops.
NoahZuniga
I'd expect watermarked model weights plus a lot of liability to distinctivise leaking the model.
null
tziki
I don't understand how Google is willing to do this but won't sell TPUs to other days centers. It should be obvious from Nvidia's market cap that they're missing a huge opportunity.
dehrmann
The only reasons I can think of is they see them as their secret sauce, they don't want to support them for customers long-term, or they don't have the foundry capacity.
paxys
It's definitely #3. The GPUs have to first satisfy Google's own computing needs, and only then can they start selling them to others. Given how much training and inference the company is doing and how much demand there is internally it's very unlikely they are able to manufacture loads of extras, especially not profitably.
null
throwaway48476
Like you can with deep seek? Or will it be more complicated and expensive. I don't know who would actually want that.
yoavm
Absolutely many would, especially those with deep pockets. The biggest concern I'm hearing from companies adopting AI, for basically any use case, is data leaving their network. Especially (but not only) in the EU.
throwaway48476
Deepseek is just the model weights. Nothing about it requires network access.
yoavm
Deepseek is not really comparable to Gemini 2.5 Pro.
surajrmal
Folks who would prefer to run deepseek are not in the end customer for this product. Deepseek doesn't provide a service contract.
null
miohtama
Is Gemini tied/benefitting from Google TPU hardware? Because you need hardware in the data center to run this, and I feel it is somewhat specialised.
drdirk
Gemini models are written in Jax which through the XLA compiler can be compiled either to TPU or GPU hardware.
Performance may differ but Google (and Nvidia) are very interested in having good performance on both platforms.
cavisne
The raw computation is just a bunch of matrix multiplications in a row, most of the algorithmic complexity/secret stuff would be around scaling & efficiency.
For training the model the HW is much more important as you need to scale up to as many chips as possible without being bottlenecked by the network.
This would just be inference, and it doesn't need to be very efficient as its for on prem usage not selling API access. So you could strip out any efficiency secrets, and it would probably look like a bigger Gemma (their open source model).
I wonder if they would/could try and strip out stuff like whatever tricks they use for long context + video support (both of which they are a bit ahead of everyone else on).
summerlight
The model itself is likely built upon their own open source system JAX so they should be usable in Nvidia. Of course cost efficiency is going to be a different story.
disgruntledphd2
The Google blogpost notes that it's a partnership with Nvidia, so using cuda rather than TPUs apparently.
stingraycharles
Makes complete sense, as NVidia has a lot more experience building these types of appliances.
ddingus
Some one said it could also mean Google hardware has some advantage they would rather stay inside the G-silo.
MortyWaves
Google abandoned Coral in true Google style.
jaggs
Not a cat's chance in hell that any eu organisation will rush to this offer right now. Or maybe ever in fact.
geodel
Huh, how does that matter? Maybe OVH AI is fine for EU.
aussieguy1234
What is the risk that some hacker could exfliltrate the weights?
cavisne
Seems pretty high, this is an air gapped product so at some point the employees of whatever government they are giving it to would need to SSH into the VM's to load new weights etc. Lots of ways to make it tricky/watermark the weights though.
dist-epoch
Very low if they use confidential VMs (CPU rooted encryption). Just like the Xbox uses and remains unhacked 10 years later.
onlyrealcuzzo
What would be the benefit of hacking the Xbox? What would you get?
DoctorOW
Pirated games, cheating/botting in multiplayer rooms, etc.
roro_7
[dead]
Is The Gavin Belson Signature Edition Box is needed to run these?