AI won't use as much electricity as we are told (2024)

It's been a while, but I don't recall any of the dotcom startups making deals with nuclear energy companies to buy out entire nuclear power stations: https://www.npr.org/2024/09/20/nx-s1-5120581/three-mile-isla...

And that's just an example, there are many power-related deals of similar magnitude.

The companies building out capacity certainly believe that AI is going to use as much power as we are told. We are told this not on the basis of hypothetical speculation, but on the basis of billions of real dollars being spent on real power capacity for real data centers by real people who'd really rather keep the money in question. Previous hypotheses not backed by billions of dollars are not comparable predictions.

Similar concerns were raised regarding the energy used to mine cryptocurrencies. It's basically wasted energy - no doubt about that. But this is different. Crypto's potential has been very limited all along, whereas generative AI has numerous potential uses, and we are seeing more and more companies, as well as ordinary people, utilising it.

> But we have been here before. Predictions of this kind have been made ever since the emergence of the Internet

I don't think I live in the same world as the author. Ever since the emergence of the Internet, "stuff related to IT" has been using more and more energy.

It's like saying "5G won't use as much electricity as we are told! In fact 5G is more efficient than 4G". Yep, except that 5G enables us to use a lot more of it, and therefore we use more electricity.

It's called the rebound effect.

> By contrast, the unglamorous and largely disregarded business of making cement accounts for around 7 per cent of global emissions.

Oh, that's not a good example of the point they're trying to make. The emissions from concrete are a point of major concern and are frequently discussed. A ton of effort is being put into trying to reduce the problem, and there are widespread calls to reduce the use of the material as much as possible.

> But far from demanding more electricity personal computers have become more efficient with laptops mostly replacing large standalone boxes, and software improvements reducing waste.

If only it was true, I reckon we’re using multiple-orders of magnitude more computational per $ of business objectives simply because of the crazy abstractions. For example, I know of multiple small HFT firms that are crypto market makers with their trading bots in Python. Many banks in my country have excel macros on top of SQL extensions on top of COBOL. We’ve not reduced waste in software but rather quite the opposite.

I don’t think this is super relevant to the articles point but I think it’s an under discussed topic.

“You may not know about the issue but I bet you reckon something, so why not tell us what you reckon. Let us enjoy the full majesty of your uninformed ad-hoc reckon” - David Mitchell.

Hopefully the panic continues and we get a lot of extra electricity, ideally via nuclear, wind, solar - and then if AI is a flop at least we get big progress on global warming.

What’s the energy profile of running inference in a typical ChatGPT prompt compared to:

  - doing a google search and loading a linked webpage
  - taking a photo with your smartphone and uploading it to social media for sharing
  - playing Fortnite for 20 minutes
  - hosting a Zoom conference with 15 people
  - sending an email to a hundred colleagues

OpenAI yesterday announced[1] a partnership to deploy computer chips, but chose to denominate the size of the deal in gigawatts (instead of dollars, or some measure of computing capacity, or some measure of capability). They certainly seem to think about this in terms of electricity requirements, and seem to think they require a lot of it.

(I may have the units off a bit, but it looks like OpenAI's recent announcement would consume a bit more than the total residential electricity usage of Seattle.)

1 - https://openai.com/index/openai-nvidia-systems-partnership/

I found this video https://youtu.be/IQvREfKsVXM interesting, especially because it mentions couple of AI studies/papers that argue in favor of much smaller (and more efficient) models. (And I have never heard of them.)

I suspect that yes, for AGI much smaller models will eventually prove to be sufficient. I think in 20 years everyone will have an AI agent in their phone, busily exchanging helpful information with other AI agents of people who you trust.

I think the biggest problem with tech companies is they effectively enclosed and privatized the social graph. I think it should be public, i.e. one shouldn't have to go through a 3rd party to make an inquiry for how much someone trusts a given source of information, or where the given piece of information originated. (There is more to be written about that topic but it's only marginally related to AI.)

I'd be very interested in seeing some kind of aggregated daily demand curve for AI workloads.

It seems like a lot of the hyperbolic angles are looking at this as a constant draw of power over time. There is no reason for a GPU inference farm to be ramped up to 100% clock speed when all of its users are in bed. The 5700XT in my computer is probably pulling a mere 8~12W right now since it is just sitting on an idle desktop. A hyperscaler could easily power down entire racks based upon anticipated demand and turn that into 0W.

> Most of the increase could be fully offset if the world put an end to the incredible waste of electricity on cryptocurrency mining (currently 0.5 to 1 per cent of total world electricity consumption, and not normally counted in estimates of IT use).

I do not accept this. It was once true under Proof-of-Work (typically ~1,000–2,000 kWh per transaction), not so much under Proof-of-Stake (typically 0.03–0.05 kWh per transaction).

Note that proof-of-stake may actually have a lower energy footprint than credit card or fiat banking transactions. An IMF analysis [1] pegged core processing for credit card companies at ~0.04 kWh per transaction (based on data centers and settlement systems), but noted that including user payment means like physical cards and terminals could increase this by about two orders of magnitude—though even then, it doesn't extend to bank branches or employee overhead - an overhead not implicit in decentralized finance.

[1] https://www.elibrary.imf.org/view/journals/063/2022/006/arti...