The predictions are coming in hot. Data centers could grow to consume more than 9% of U.S. electricity generation by 2030, according to EPRI. That’s more than double its current estimated data center load. AI will increase global data center power demand 165% by 2030, says Goldman Sachs. And billions of dollars are at stake. Utilities, megasite developers, and data center operators are all basing major decisions on predictions like these.
But they’re also the kinds of predictions we’ve seen before. In 1999, when the internet was growing fast, a couple researchers claimed it would grow to consume half of all U.S. power generation within a decade — until a team at Lawrence Berkeley National Laboratory debunked it.
Jonathan Koomey was one of those researchers. Although today’s predictions about energy usage are tamer than those in 1999, Jonathan still has questions about the current hype around AI power demand. He’s now the founder and president of Koomey Analytics, which has published multiple papers on the topic, including a recent report for the Bipartisan Policy Center: Electricity Demand Growth and Data Centers: A Guide for the Perplexed.
So what are the assumptions that go into these new predictions? And how do they hold up to scrutiny?
In this episode, Shayle talks to Jonathan about why he questions the hype around AI load growth predictions and why he believes energy constraints will incentivize the AI industry to focus on efficiency. Shayle and Jonathan cover topics like:
- The time lags and proprietary data that hinders precise data center load estimates, both in historical analyses and future predictions
- The difficulty of reproducing the predictions of even prominent institutions like the IEA
- The two basic assumptions that go into predictions: AI demand and AI power requirements
- Why Jonathan believes conventional wisdom relies on questionable sources, like Nvidia’s business plan
- The unexplored areas of AI energy efficiency, like computer architecture, software improvements, algorithms, and special purpose computers
Recommended resources
- Lawrence Berkeley National Laboratory: 2024 United States Data Center Energy Usage Report
- Nature: Will AI accelerate or delay the race to net-zero emissions?
- Joule: To better understand AI’s growing energy use, analysts need a data revolution
- WSJ: Internet Hype in the ’90s Stoked a Power-Generation Bubble. Could It Happen Again With AI?
- Open Circuit: The data center boom: ‘All the cheap power is gone’
Credits: Hosted by Shayle Kann. Produced and edited by Daniel Woldorff. Original music and engineering by Sean Marquand. Stephen Lacey is executive editor.
Catalyst is brought to you by EnergyHub. EnergyHub helps utilities build next-generation virtual power plants that unlock reliable flexibility at every level of the grid. See how EnergyHub helps unlock the power of flexibility at scale, and deliver more value through cross-DER dispatch with their leading Edge DERMS platform, by visiting energyhub.com.
Catalyst is brought to you by Antenna Group, the public relations and strategic marketing agency of choice for climate and energy leaders. If you’re a startup, investor, or global corporation that’s looking to tell your climate story, demonstrate your impact, or accelerate your growth, Antenna Group’s team of industry insiders is ready to help. Learn more at antennagroup.com.
Transcript
Stephen Lacey: Hey, it’s executive editor Steven Lacey. Many of you were loyal listeners to a show I created and co-hosted for eight years called The Energy Gang. Well, we’re back Jigger Shaw, Catherine Hamilton and I are excited to announce our latest project, a weekly news round table called Open Circuit. Every week we’ll break down how major projects come together, how deals and policies get structured, and what it takes to build critical infrastructure at scale, all through the lens of current events. You can subscribe to open circuit wherever you listen to podcasts or follow it@latitudemedia.com,
Tag: Latitude Media: podcast at the frontier of climate technology.
Shayle Kann: I’m Shayle Kann, and this is Catalyst.
Jonathan Koomey: Usually what they’re assuming is NVIDIA’s business plan is our growth forecast, right? They’re just saying, well, Nvidia has this projection for the next few years of how many AI nodes they’re going to sell, and they just use that and say that’s what’s happening. And of course, there’s uncertainty in that
Shayle Kann: Coming up: Injecting a little bit of skepticism into all the hype around data center energy consumption.
I’m Shayle Kann, I lead the Frontier Fund at Energy Impact Partners. New tagline, who dis? Anyway, we’ve talked about this before, but here is what has been going on in the world. There’s a general perception that near and maybe also long-term demand for AI compute is essentially insatiable, at least relative to today’s capabilities to deliver it. And crucially also, there’s I think, generally a belief that access to power is one of, if not the most important bottlenecks in the delivery of that compute and the services that it allows. So we have developers and colos and hyperscalers all out there executing a literal land grab all over the country and much of the world to lock down sites that will be suitable for tomorrow’s data centers. And then meanwhile, forecasters be they governmental, non-governmental companies themselves, they all keep increasing their load growth projections, painting this picture of an electricity sector that will be essentially bursting at the seams for the foreseeable future.
And then Deepsea comes along and for a moment, scrambles that narrative. People ask, will energy efficiency improvements actually slow or stop the rate of load growth caused by ai? And then it seems at least to me, that everyone kind of collectively decides, nah, Jevon’s paradox, the less it costs, the more we’ll use. And we go right back to seeing gigantic proclamations from the likes of Stargate and Meta and many others. I say everyone collectively decides nah, but it’s actually not quite everyone. Jon Koomey in particular is not part of everyone. He has been researching data center energy demand for a very long time, starting way before the AI wave. He worked at Lawrence Berkley National Labs for a long time and then at Stanford before founding his own firm, Koomey Analytics, which focuses amongst other things on this exact question of AI and data center driven electricity load growth. And as I alluded to, he’s more skeptical than most other folks that I’ve spoken to, at least on how much electricity load growth we will actually see from this new wave of AI data centers. It’s a perspective. I don’t think you hear that much these days. So I thought it was worth teasing out and perhaps debating a little bit. So here’s my conversation with Jon. Jon, welcome.
Jonathan Koomey: Thank you. Glad to be here.
Shayle Kann: Excited to talk about data center energy consumption with you and with everyone else in every conversation that I have at the moment. But particularly with you, you’re I think especially valuable to give a little bit of a history lesson to start here, which is can you just talk through pre AI wave before the last couple of years, we had been on an interesting trajectory with regard to the growth of compute as it pertains to energy consumption overall. So you just walk us through the history there, what had been happening for the past couple of decades?
Jonathan Koomey: Yeah, so let’s go back to the dotcom boom. So the late 1990s, there was a lot of focus on computing in various forms, and people were very excited about different things including fiber optic networks and e-commerce and other stuff. And there was a lot of speculation roundabout the year 2000 that computers and related technologies were going to end up using a lot of electricity. There were projections widely cited by people of many different persuasions that the internet was going to use half of all electricity in the next 10 years, this kind of thing. There were a couple of guys running around pushing that whole narrative, and that led to pressure on people who are at the national labs and other places to try to figure out, well, what were the numbers? What was actually going on? I was a staff scientist at Lawrence Berkeley National Lab at that time, and I had done work on computing electricity before that in the early to mid nineties. So people came to me and said, well, do these numbers make any sense? And we ended up doing careful work. We found that those projections were off massively. These guys were claiming that all computing used 13% of all electricity use, and when we actually tallied the numbers, it was 3%. And we figured–
Shayle Kann: That was projections of what would happen or that was what they thought was already happening?
Jonathan Koomey: Yeah, so they were claiming in 2000 that all computers 13% of our electricity, and we actually analyzed those numbers and found that no, it was actually 3% and that there was no way that it could be half of all electricity use in 10 years. So it led to a lot of great research, but it made me pretty skeptical about these kinds of projections because always people with an interest in promoting this narrative about fast electricity use and growth in data centers and all that. So that was the year 2000. We found those numbers, the base numbers are wrong, the projections were wrong. And then we looked at historically in 2008, 2010, well what actually happened? And we did see a doubling of electricity use for data centers from 2000 to 2005. So there was fast growth during that period, and the industry saw that and responded by creating a bunch of efficiency technologies to mitigate that growth.
So it took data centers from about half of 1% of electricity. It was half of 1% to 1% of electricity use by around 2010 or so. And that was about where we were with data centers. So it was fast growth for a bit started to moderate, and then from 2010 to 2018, growth was actually pretty slow in data center electricity use, there was still an increase in the use of data centers. So we had about a sixfold increase in the compute output for data centers from 2010 to 2018, but electricity use went up a grand total of 6% over that period. So pretty modest growth, even though the service demand for data centers was growing pretty fast.
Shayle Kann: So that’s interesting. I hadn’t thought about this in this two periods. So 2000, 2005, electricity consumption from data centers actually did double from a small base, but doubled. So there we weren’t yet seeing, we were seeing a boom in compute. We weren’t yet seeing these dramatic energy efficiency improvements, but partially in response to that in the next big growth cycle for compute, we did see all these energy efficiency improvements. And so the next time compute really took off, we didn’t see the same spike in electricity consumption. Is that the right way to think about it?
Jonathan Koomey: That’s right. Yeah. I think that the growth in the use of computing continued. It wasn’t that it stopped, it was continuing, but the industry focused on efficiency and they figured out a bunch of different ways to improve the efficiency of computing. Some of that involved redesign the silicon, right? So you’re shrinking transistors and changing hardware architectures and this kind of thing. But also it involved moving from very inefficient corporate data centers to much more efficient hyperscale data centers. And we can talk about why that’s important, but basically the hyperscalers have economies of scale and they figured out design tricks to make the efficiency of delivering computing, and sorry, electricity and cooling to the computer is much more efficient and they have much higher utilization of their equipment. And so when you think about it in terms of cost and energy use per compute, these hyperscale facilities are much, much better than the kind of traditional corporate facilities. And there was a big movement towards those hyperscalers in that later period towards 2018.
Shayle Kann: I think that’s the important lesson here is that there has been a historic, even if you just take that whole 20 year period starting in 2000, there has been a gigantic growth in overall computing that has resulted in a much, much more moderate growth in electricity consumption, relatively speaking. And so of course, the operative question in the current moment is, are we in for a repeat of history, either a repeat of the 2000, 2005 history where compute grows alongside electricity consumption? And we do see this big boom, or I guess the bigger part of that 20 year period wherein you saw the six x growth in computing, but only 6% growth in electricity consumption. So that’s the operative question right now. And I guess before we get into answering it, what are you seeing out there in terms of the variability of projections? This is one of the interesting things. Nobody really knows what’s going to happen. And so I think part of the result is that I’ve seen forecasts that are all across the board,
Jonathan Koomey: So there’s huge variability in the forecast. There’s even variability in what we think of as history. So for the year 2022, the International Energy Agency did these projections. They did two of them in 2024, and they started by trying to do history, and they looked at 2022, and the first study they released in January had a number that was 50% greater than the second study that they released in October. And so even the same institution looking at a historical year had this uncertainty. They identified an uncertainty range of 220 to 340 terawatt hours per year for global data center electricity. So that’s a pretty big uncertainty range for a historical year. So there’s a lot of things we don’t understand.
Shayle Kann: Is that because of China or something? Why is it so hard to estimate historically?
Jonathan Koomey: Well, because the data are often held as proprietary information. And so we as analysts, first, we don’t have to the proprietary information, but when there is such information, so these data collection companies, IDC and Gartner and so on, they collect the information, but there’s a time lag for releasing the information. So there’s a lag. It has to be collected and processed and so on. And so by the time we’re getting 2022 data, it’s 2025 or 2024. And so there’s this time lag and it takes time to analyze and so on. So it’s both the inaccessibility of certain data and then the time lags associated with real data and real analysis that leads to this uncertainty in the history. This is distinct from the uncertainty in the projections that you started the question with, but I think it’s important for people to understand that we don’t even know with high precision the historical numbers for 2022 and 2023, and when something grows super fast like ai, which has been growing gangbusters, it’s very hard. Things happen in six months or a year, and it’s two or three years later that we actually get the real data that people have confidence in. So that’s a big uncertainty just in the historical data. And then we can talk about the projections.
Shayle Kann: Yeah. Okay. So let’s talk about the projections then. What’s the range of projections that you’ve seen?
Jonathan Koomey: Well, let’s actually not start with those quantities. Let’s start with conceptually what we think is happening. There’s two drivers of electricity use in data centers or AI data centers or any data centers. There’s the growth in the service demand, the demand for compute, the demand, the AI queries or search queries or something else, some measure of service. And if those measures of service demand are growing, that of course puts upward pressure on electricity use. And then the other big uncertainty is on the efficiency of delivering the service. And if you can deliver the service much more efficiently, then you’ll be able to meet that service demand but not increase electricity use in the aggregate. So those are the two forces. And when people make a projection of what future electricity use will be, they are implicitly or explicitly making assumptions about service demand growth and efficiency of delivering that service demand.
And so that’s the first core level understanding of what’s happening here. When people are making a projection and they’re making assumptions, usually they’re implicit assumptions that are unstated and some are better at explaining what those assumptions are. Like the lb and L report to Congress that came out in December of 2024, that was estimating data center electricity use for the us, and that’s at least reproducible. I can go into that report and I can figure out what they did. Now I was part of that team, but anyone can look at that. Whereas some of the other projections, even from pretty credible organizations like the International Energy Agency, you can’t actually figure out what they did. They don’t release the data, they don’t release the calculations in any explicit way. And so that’s kind of the first question people should ask is can I figure out what these people assumed in making these projections? And usually what they’re assuming is NVIDIA’s business plan is our growth forecast. They’re just saying because for ai, they’re just saying, well, NVIDIA has this projection for the next few years of how many AI nodes they’re going to sell, and they just use that and say that’s what’s happening. And of course, there’s uncertainty in that. It’s not just uncertainty in how many they’ll sell, but it’s also how they will be used and whether there will be innovations that allow people to use that hardware much more efficiently.
You saw the announcement of DeepSeek a few weeks ago, and that was an example. It was that Chinese AI firm that was able to get similar sort of accuracy benchmarks from their runs, but using much less compute and much cheaper.
Shayle Kann: This is I think, the core of what we need to talk about here. Here’s my impression of to the extent that there was conventional wisdom, conventional wisdom in this space. It was that prior to DeepSeek for the past year or two, it was, oh my God, there is going to be, we have such a huge need for new data center capacity, all of the hyperscalers and then OpenAI now at Stargate and all these other things, everybody’s planning to spend hundreds of billions of dollars in CapEx and develop tens of gigawatts of new data centers. And we’re seeing this happen in real time. And so the load projections, load growth projections keep going up and up and up. And in fact, I think the conventional wisdom was becoming the limiting factor on load growth from AI will not be demand for AI or willingness to pay for AI, but rather the grid itself and it’s we’re just going to max out the amount of space we have on the grid for these hyperscale data centers basically.
And then DeepSeek came along and shook everybody on that assumption briefly. Right, and as you said, DeepSeek, from an energy perspective, the main thing that DeepSeek was offering was we could do the same thing with more than an order of magnitude less energy to do the same thing. And so for a moment, everybody freaked out about that, and you could see that reflected in the public equity prices of the constellation energies of the world and things like that. But it was a blip more or less. And since then, all the hyperscalers announced additional CapEx investments and the world sort of resumed back to where it was. And I think it can be boiled down. Why is that? I think it can be boiled down basically to the Jevon’s paradox argument, which Satya Nadella himself made, but many other people have as well, which is basically at the highest level.
And then I’ll have you explain how you think about it. At the highest level, if you make a thing cheaper, if you make a thing cheaper with sort of endless potential demand for the thing, people will just use more of it in the end. And so even if DeepSeek and derivatives thereof turn out to be game changers in terms of energy efficiency, that just means we are going to be able to do ever so much more AI and as a result, from an energy perspective, nothing changed. That’s my sense of the conventional wisdom. I know that is, you have a slightly different view there, but do you think that that is what the world generally thinks? Do you agree with me that that’s the conventional wisdom at the moment?
Jonathan Koomey: I think that the people who are delivering AI who are suppliers of AI, they honestly believe that the demand for their product is infinite, as you described. And the Jevon’s paradox is a complicated thing that we don’t even need to get into, but we can because it’s fun and interesting, because that was used by the head of Microsoft to explain why he thought that demand would not be affected. Electricity demand would continue to grow massively because essentially there’s infinite demand for their product. And that assumption is something that people should examine. I think there’s a lot of people who believe that. And of course in economics, if something gets cheaper, people use more of it. But the question of how much more they use is a question of what is the value being delivered by this technology? And if the value of a technology is very, very high, people will be willing to pay more to have it right?
And so costs may come down and maybe that means they’ll use more of it. But the question is that some point, is there a kind of saturation, kind of a leveling off of this? And so the Jevon’s paradox is also called the rebound effect, and it’s typically used to describe efficiency. The idea is that the efficiency of something like a car goes up, so it’s cheaper to operate the car, so people operate the car more, right? That’s the intuition around it. And in reality, what happens of course is that yes, the car is a little bit cheaper, and yes, people drive a little bit more, but generally these effects tend to be kind of small. They tend to be kind of 10 to 20%. Energy is a small part of the economy. It’s only seven or 8% of the GDP, right? So in the aggregate energy is not a huge fraction of total costs. And so–
Shayle Kann: And that’s also true of data centers.
Jonathan Koomey: Yeah, data centers are a little more–
Shayle Kann: It’s a huge portion of opex, right?
Jonathan Koomey: Yeah. There’s a huge capital cost in data centers. So energy is important. It’s not trivial, it’s not 1% of costs in data centers, but it’s also not the majority. And so tying this to efficiency, I think is a mistake, but I think the general argument is the one that you are describing, which is there’s infinite demand for our products. And even if the products are the cost of delivering the products go down, people are just going to use more because they have this infinite demand. And that’s really the core of the argument. That is what they’re saying is that there’s so much demand that none of these kind of constraints are going to get in the way. And if we reduce the cost, people are just going to use more. So the electricity demand must happen. That is their core argument. And so let’s examine that question because to me, that’s really the assertion that there’s infinite demand is should at least be open to discussion. People should talk about that
Shayle Kann: As it pertains to energy. There’s multiple very hard to predict things. The first hard to predict thing is how much of an energy efficiency improvement, how much of an improvement in watts per flop can we expect realistically, as you made a point before that historically, one of the reasons we got such an energy efficiency improvement was the shift to hyperscale data centers. Well, we already have hyperscale data centers, so that’s not another thing. We can’t replicate that. But then DeepSeek is its own thing. Maybe the way to run these AI models can be a lot more efficient. And similarly, I’ve heard a lot of people talking about, well, look, the hyperscale data centers to run AI right now, it’s mostly training. The world is going to shift predominantly toward inference at some point, and maybe we can, there’s a lot more opportunity for new innovation and inference and we could get more efficiency.
So for me, the thing that’s difficult about this is that you are, as you said, you’re both trying to predict the energy efficiency component and the demand component simultaneously. And both seem highly uncertain to me. But I think that if I can reflect the argument of the masses, it is that there is not necessarily literally infinite demand for AI compute, but the demand is sufficiently higher than where we are today. It’s orders of magnitude, let’s say higher than where we are today, that the limiting factor on electricity growth is not going to be demand, it’s going to be something else. And it could be–
Jonathan Koomey: It’s how fast can we build the equipment?
Shayle Kann: Yes, or chips, right? It’s chips and power and all that kind of stuff.
Jonathan Koomey: And transformers and backup power generators, those kinds of things.
Shayle Kann: Does that argument hold water for you or where would you poke holes in it?
Jonathan Koomey: To me, it’s effectively the same thing. They’re basically saying that constraints are related to how fast we can supply the AI. So effectively, even if it’s not infinite, it’s high enough that they will build as much as they can given these other constraints. And so the question is, is that actually true? I think most people seem to believe it in the industry. I think that’s pretty common. I think most people think that’s true, but is it true? That’s the conversation that I think people need to have because there are issues with these models. We will talk about the efficiency side of it. I have a lot to say about things you can do to do this more efficiently. But on the demand side, there are real questions about accuracy. Traditionally, what has happened, and this is another set of assumptions that the whole industry seems to make, historically, they have gotten increases in accuracy by scaling, throwing more computers at the challenge, and they’ve had whatever it is, 10 or 20 or sometimes 30% increases for a tenfold increase in compute capacity.
And when you have computers improving very rapidly, you can do that over time. But now we’re reaching the point where the scale of the construction challenge for increasing capacity, computing capacity, 10 or a hundred fold, turns out to be much harder when it’s a relatively small industry that’s not that hard. But now we’re getting up to the places where things are a lot harder to do. So that’s one question is continuing to scale in the way that they have in the past, the only or best way to meet this demand for compute. And I think that’s an open question. I think that there are other ways to meet the demand for compute, and that relates to the efficiency conversation. But on the demand side, accuracy is a critical thing. Liability is a critical thing, and companies are liable for the actions of their agents, be they human or computer.
And if your agents are telling people to eat rocks and put glue in their pizza and doing other things that don’t make any sense, ultimately I think that will probably constrain the demand. So it’s really a business question now of there’s all this CapEx. People are trying to sell AI and make money from it. And the question of whether people are willing to pay for the service being delivered is I think an open question because they’re charging open AI right now is just a major, it’s hemorrhaging cash. You can’t believe, right? There’s no obvious way that they become profitable without a massive increase in the demand for their services and the ability to raise the cost. And that is really unclear whether that is possible.
Shayle Kann: Setting aside the model hallucinations stuff, I think there’s definitely an argument there, but there’s also a counterargument of like, well, these models are improving sufficiently. We’re doing better and better at removing the false answers and things like that. But will the revenue show up to match the CapEx is a big question. My friend David Khan at Sequoia wrote this piece a while back called the 600 billion dollar question, which is probably now a 2 trillion dollar question. I dunno what the number is today, but it’s basically like that’s the amount of CapEx that has been announced to spend on AI data centers. And so that is the bare minimum of the revenue that has to show up to support that. And we’re nowhere near that level on that side. So there is a bit of a leap of faith that the market is taking there.
And so I think again, it comes down to this on the demand side alone. And then I definitely want to hear your view on the energy efficiency question. Of course, it is very difficult to predict what is the actual demand if you remove other constraints from the equation, was the actual demand for AI compute. It does seem clear to me that at a minimum we are in for a few years of lots of new applications, model improvement, all sorts of things that are going to lead to lots of additional demand. How high that goes though is obviously I am not equipped to predict it and I’m not sure anybody, I don’t think anyone is.
Jonathan Koomey: Yeah, I don’t think anybody can.
Shayle Kann: But what I do sort of believe is that, again, setting aside dramatic energy efficiency improvements, which we will come back to momentarily, that the infrastructure constraint is a real one. And so you don’t need demand to be infinite in the near or medium term in order to sort of hit the ceiling of what is really possible. Because I do know that infrastructure is a serious constraint, whether it be transformers or labor or power or chips. And so I guess I’ve come to believe that side of the equation, that demand is going to be demand is not going to be the limiting factor. Maybe that’s the way to put it in at least the near term in the long term.
Jonathan Koomey: In the next few years.
Shayle Kann: Right. So if you believe that to be true, then the question is one of how dramatic could the energy efficiency gains be? And there I have no idea, but I want to hear what you think.
Jonathan Koomey: Okay, so I have a lot to say about that. But lemme just first turn back to the question of the hallucinations because I think there’s an overall leap of faith that is a problem that can be solved. And I think it’s at least an open question as to whether this particular technology which relies on probabilities and analyzing past data, whether it’s possible to fix hallucinations given all the weirdness that’s in the training data, this human behavior, and then there’s malicious actors and then there’s just general human stupidity that gets reflected in this. And so I’m sure there are ways to improve the situation, but the question which I don’t think anyone knows the answer to is can it be improved enough so that the liability associated with the errors that these technologies always seem to create are worth it? And that, I don’t know, no one knows the answer, but I feel like that’s a question that needs to be asked because right now people are assuming, oh, it’ll just go away with scale or whatever else, but I don’t think that that’s necessarily true.
It could be true anyway. So that’s another thing on that service demand side that I think people need to question. Okay, so we’ve got the whole service demand question of how much demand will there be? And then there’s the question of how efficiently can you meet that demand? That was where Deep Sea came in and seemed to show orders of magnitude improvement. There’s a whole literature on this until recently, that literature has been around the end of Moore’s Law. So Moore’s law is everyone knows that. It’s like the doubling of transistors on a chip every couple of years and so on. And that went on from 71 when the first, well earlier from 65 when we first started putting these larger ships together till about the year 2000. So they were able to basically double the number of transistors on a chip every couple of years.
And as they did that, they ramped up the clock speed. So that improved the performance, but at the same time, they had to ramp down the voltage. And this was called ARD scaling, a very important concept that worked for 30 years, which is pretty amazing in technology that they were able to do that. But around about 2000 or so, they ran out of the ability to drop the voltage on these chips. You reached a threshold voltage for silicon about one volt, you can’t really go over. And so what that meant was that they needed to come up with new ways to improve performance and efficiency and in practice people who used computers around that time, we’ll know this round about 2005 or so, you saw this proliferation of more than one core on a chip. So instead of having a single chip with a single core, suddenly people started putting lots of cores on these chips.
That was the way that they could improve performance by basically allowing the computer to do computations in parallel. And it usually required some sort of redesign of software reconation, but it allowed them for many applications to really speed things up. And so that was a change that went on. So that was a change in what we call hardware architecture. But it turns out there’s a lot of things you can do. Hardware architecture is one thing, but then you can improve software, you can move to a different kind of software. You can increase the efficiency of the software. There’s also improvements in algorithms. So if you have a much more clever way to do the traveling salesman problem, you can reduce the amount of compute you need to do that calculation. And that’s an example. And those tend to be very domains and problems specific, but a lot of times it’s orders of magnitude improvement there as well.
And then there’s the use of what we could call special purpose computing. So the long-term trends that people talk about, usually these are general purpose computers. They’re good at, they can do many things, but they’re not perfect at doing specific tasks. And so if you design a device or set of devices to accomplish one specific task really well, you can actually speed things up a lot more than what the Moore’s law constraint or any of these other physical constraints would indicate. And so that’s another thing you can do. So you design special purpose devices, you optimize, you co-design the software and the hardware, you optimize ’em to do this thing really, really well. And so between those various improvements that you can get beyond just shrinking transistors and making things closer together, there’s orders of magnitude possibility for improving efficiency. And so my sense is that the focus has been basically building out AI infrastructure and while there was some improvement, and you can see it in the Nvidia chips over time, there’s some improvement, efficiency was not the major focus because these constraints that you’ve been talking about haven’t been as proximate. So people weren’t really focused on it. And this is very similar to that year, 2000 to 2005 situation where people were just putting in as much infrastructure as they could and then they said, oh my God, we’ve doubled electricity use. Maybe we better focus on this. And that’s what I think is likely to happen here as well, is that the industry will actually really focus on this in a way that they haven’t yet.
Shayle Kann: Yeah, you’re making a necessity is the mother of invention type of argument here, where as the infrastructure constraint becomes more and more clear and makes it more and more difficult to build additional data centers, the industry will turn its attention more toward energy efficiency. And we will start to see these more dramatic improvements like what we have seen in the past but have not seen here yet. Do you think that we should be thinking about it? Obviously there’s no way to know exactly, but I just think it’s interesting that historical analog, same amount of time, plus or minus, I dunno, we’re maybe two years in. So call it three more years of just like boom, boom, boom would make a five-year trend after which, and during that time, sort of early r and d on dramatically more energy efficiency and the curve starts to bend a little bit more horizontal in plus or minus three more years, something like that. Or is it going to be 10 or one?
Jonathan Koomey: Yeah, I don’t know. I don’t think anyone can say because it’s partly dependent on the demand question. If people, let’s say they iron out the problems that we talked about in terms of the reliability of these things, then maybe demand is even higher. So while they might be improving, it’s always this struggle between improving efficiency and meeting the service demand. And when service demand grows faster than efficiency grows, electricity use goes up. When service demand is offset entirely by efficiency improvements, then electricity use is flat.
Shayle Kann: So without being able to actually predict the timing of it, is your best guess that the trajectory of electricity load growth from data centers is going to look something like what it did from 2000 to 2020, which is a spike for a period of time where it is growing faster, followed by a longer period of time where it’s growing much slower, if at all, because of some combination of energy efficiency improvements and possibly hitting a ceiling on service demand. Is that the way to think about it?
Jonathan Koomey: Yeah, I think that’s a reasonable way to think about it. I don’t think anyone can know for sure. It does depend on the growth in service demand. But assuming the industry does what it usually does, which is it sees a problem and it fixes it, and the reason why I am optimistic about that is because the industry’s incentives are to reduce the cost of delivering their product, right? And that’s been historically true for data centers is that energy and capital turn out to be huge parts of their total cost. And if they can do things a lot more efficiently, they will because it will save them money and make them money. And so it’s a little different than some other sorts of energy using applications where you don’t have these strong incentives. These players have pretty strong incentives to do things in the most cost effective way.
That isn’t always true in a boom time when it’s the two or three years of building things out super fast. They’re basically just trying to find enough people and enough equipment to put in enough data centers to meet the demand that they perceive. But once they have a little slack in their building, then they can start to think, oh, how can we do this better? Or they see the constraint. There’s all these stories. I worked at Lawrence Berkeley lab for many years and there’s a story of Steve Chu used to be the lab director there, and the supercomputer guy said to him, Hey, we want to build this supercomputer and it’s going to use 40 megawatts. And he said, no, it isn’t because we don’t have that power. And he sent them back to redesign the whole thing, and it turned out to be what, 16 or 18 megawatts and they made it work.
But that’s the kind of thing that happens when there are real constraints, when there aren’t the real constraints, just the world is their oyster. They just build what they can. But then once things start getting big enough and the scale of these data centers is very large, we’re talking tens or hundreds of megawatts, and that’s as much as a small city or an industrial plant. So it’s individual installations can be quite large, and that leads to constraints in transmission and distribution infrastructure and backup power generators. I know the waiting time for backup generators was a couple of years. That was a while ago. It’s probably longer now. There’s been a lot of building and it just takes time. It’s like in the old Star Trek, Scotty would always tell the captain, I can’t change the law as a physics captain. And that’s what we’re dealing with. The world of bits moves really fast. They have all the money in the world. The world of atoms is slower. So it’s like a year for the world of bits, 10 years or 20 years for the world of atoms. And so it is like a mismatch,
Shayle Kann: Which is what you’re saying, provides this enormous degree of uncertainty. But if I’m understanding you right, your high level, I think high level, what you are doing is questioning the conventional wisdom that it is certainly going to be basically the trajectory of load growth from AI or from data centers is just going to be up into the right for the foreseeable future as fast as the grid will. And you’re saying you don’t believe that to be true?
Jonathan Koomey: Yeah, I don’t think that’s right at all. And I think I start from the presumption based on having done energy forecast for 30 years that nobody can predict more than a few years out for computing. And I think it’s a mistake. Nobody has any idea what electricity demand for data centers will be beyond a few years from now. I think as you were describing, we have an idea of what’s in the pipeline. We know what people are planning and all this, and some of those may not be built. You start to see Microsoft, I think started canceling some of their leases and other things like that, clan leases. So you might see some shifts, but you have an idea of what’s being planned for the next two, three years. But that’s about as far out as anyone can have confidence. And even then I think there’s uncertainty. So I think anyone who claims to know what’s going to happen in 2035, the data center electricity use, I think is just making it up. And I think people should be very skeptical whenever they see these longer term forecasts. The problem is that the utilities, they need a longer term forecasts.
Shayle Kann: That’s what I was going to say. And defensive utilities, their entire business is planning around 20, 30 year cycles. This just makes it very difficult for them.
Jonathan Koomey: And so one thing that utilities can do, and the most sophisticated utilities are doing is they can structure the rate designs and incentives in a way that minimizes the risk to existing customers. Because what can happen is a new load comes in and the utility spends a lot of money to meet that load. And if that load goes away, then suddenly all the existing customers are stuck with the costs that were imposed. And so they’re designing the rates and the incentives in a way that reduces that risk to the existing customers. They make the big data center folks pay upfront, and they have take or pay contracts. They do a bunch of other things. The most sophisticated people, like in Dominion, for example, in Virginia, they’ve been dealing with data centers for 10, 15 years. So they actually design things in a better way, but not every place is as sophisticated as they are.
And they can minimize their projection uncertainty by changing the incentives that they offer to these data centers and changing the rates that they offer and making sure that the existing customers are not on the hook. So in a way, you have a forecasting problem, basically impossible to know 10 years from now what will happen, but you also can at least design some of your incentives in a way that will minimize the forecasting uncertainty because the people who don’t want to put up the money for the take or pay contract, they’re not as serious. It’s not necessarily, maybe they can’t justify it, maybe they can’t get their customers to pay for it, they’re speculating or whatever. But basically what happens when you start charging people real money for stuff is that they get a lot more selective. Whereas a lot of times when there isn’t a consequence for speculating, you end up with a lot more of a scattershot sort of approach.
Shayle Kann: Yeah, that’s clearly the direction of travel in utility world, but it is complex and everything is nuanced and location specific in that universe. Super interesting. I’m not going to force you to give me a number, but let me ask you this question. I made a bet a little while ago with a friend where, okay, so the current, this might be a few years dated now, so you can tell me if the number is dramatically different today. You probably know, but the current total data center energy capacity in the United States is something like 20 gigawatts. It’s in that range.
Jonathan Koomey: I think that was true a few years ago.
Shayle Kann: So it’s probably 25 now or so, something.
Jonathan Koomey: It’s probably more so I don’t remember in gigawatts, but the US number from the report to Congress that just came out from LBL was something like on the order of 2% of the electricity in 20 20, 20 21 was data centers. And now in 2023, now 2023, this is the last historically year we have, it’s 4.4%. There was significant aggregate growth. Now that’s still, if you look at that from the perspective of the US as a whole, we’re not seeing that kind of growth. Like the aggregate number 2023 electricity generation in the US was lower than in 2022. Now there’s weather and then there’s other weird stuff. But the point is we’re not seeing explosive demand growth at the top level. So some of it is shifting of data centers, some of it is other uses are getting more efficient. So it’s hard to know.
So the bottom up calculation from LBL is that data centers have gone from like 2% to four and a half percent in a few years. So that’s what they calculated, but you can’t actually see that in the aggregate statistics. And so that’s one of the tensions that we have to try to figure out or uncertainties we’ve got to iron out, which is aggregate generation statistics from EIA, which are added up from all the utilities. They’re not even 2023 commercial sector electricity use was lower than in 2022. Now in 2024, there’s probably a little growth, but 2023 was about the same amount of generation 2018 in the U.S..
Shayle Kann: Overall, not for datacenters. I just want to be clear.
Jonathan Koomey: Overall. Everything. So that includes homes.
Shayle Kann: What you’re saying here is that we have seen data center energy consumption as a share of total electricity consumption double plus or minus in a few years.
Jonathan Koomey: We think based on a bottom up calculation.
Shayle Kann: However, overall electricity generation has not grown during that time. And so the presumption is that the pie was fixed and it’s got shifted around a little bit and data centers have taken more of it and so on. Okay, but that aside, here’s the bet I made with a friend. We don’t have to know the exact number. Whatever the current capacity is of data centers in the United States by 2030, will it double more than double? Less than double?
Jonathan Koomey: What side of that bet did you take? Like how did the bet–
Shayle Kann: I was on the more than double side. I made this bet about a year ago, so now there’s even less time for it to double. But I was on the side of we will see total capacity of data centers as measured in megawatts or gigawatts, I guess really, double by 2030. And I was on the side that we will see it more than double.
Jonathan Koomey: Well, I think it doubled from 2021 to 2024. So maybe it depends on what base year you’re choosing, right? It already doubled.
Shayle Kann: Right. I’m saying let’s say relative to today though, we have five more years.
Jonathan Koomey: Well, so what that means is going from four and half to 9%. Let’s call it five to ten.
Shayle Kann: That assumes everything else is fixed.
Jonathan Koomey: Right, right, right.
Shayle Kann: It may or may not be, may
Jonathan Koomey: It may not be because there’s EVs and other things, but let’s just say it goes from 5% to 10% in five years. So that’s basically 1% increase in demand growth in the aggregate for the US driven by data centers. Right? So you’re saying–
Shayle Kann: That is what that implies.
Jonathan Koomey: So historically in the last 20 years, we’ve had much lower growth in the aggregate. That’s certainly manageable. It’s not what I would call explosive growth. And that’s the narrative that you see in the news is there’s explosive growth and it’s like, well, no, there isn’t. There’s no evidence of explosive growth. There is evidence of regional growth, but in the aggregate, we’re not seeing explosive growth. We’re seeing 1%. And maybe if you add electrification of vehicles and industry and heat, that’s going to add another one or 2% per year. So maybe we’re going to see 3% per year growth versus 0.5%, which is what we’ve been seeing in the last 20 years. And so that to me is the reasonable range here. And so it could be that it doubles again, but my point is simply to argue against the facile narrative that we’re going to see explosive growth because number one, we haven’t seen explosive growth. Number two, there are real constraints coming in to affect the electricity use that are related to the world of atoms and the delivery of power generation, backup generation, and building transmission and distribution and all that stuff. And so those constraints are real, and it would not surprise me if we saw another doubling, but I would say probably that’s about it. I think that’s going to level out and then people are going be more efficient at what they’re doing based on the history.
Shayle Kann: All right. I pinned you down as much as I can.
Jonathan Koomey: Yeah, sorry. I’m squirrely about that. I know. I hate to actually predict the number five, ten years from now.
Shayle Kann: Yeah. No one wants to be stuck with the number that they’ve stated. Jon, this was a lot of fun. Thank you so much for the time.
Jonathan Koomey: My pleasure. This was great. Thanks.
Shayle Kann: Jon Koomey is the founder and president of Koomey Analytics. This show is a production of Latitude Media. You can head over to latitude media.com for links to today’s topics. Latitude is supported by Prelude Ventures, prelude backs, visionaries, accelerating climate innovation that will reshape the global economy for the betterment of people and planet. Learn more@preludeventures.com. This episode was produced by Daniel Woldorff, mixing and theme song by Sean Marquand. Stephen Lacey is our executive editor. I’m Shayle Kann, and this is Catalyst.
