Didn’t catch last week’s episode on Nat Bullard’s mega slide deck on the energy transition? Start there.
This is the second half of our extended conversation with Nat, the former chief content officer at BloombergNEF and current co-founder at data insights company Halcyon.
In this episode, Shayle and Nat dig into topics like:
- Rising solar installations and stagnating wind
- Why we’re wasting so much renewable power amid skyrocketing load growth
- The rise of Chinese plug-in hybrids and exports
- Whether DeepSeek’s efficiency will temper or turbocharge load growth
- The woeful state of transmission buildout, despite demand for it
- Why one quarter of Virginia’s power demand comes from data centers
Recommended resources
- Latitude Media: Does DeepSeek call the data center boom into question?
- Latitude Media: To get data centers online, one Virginia co-op is proposing a new business model
- Latitude Media: A dizzying year at the AI energy nexus
- Catalyst: Demystifying the Chinese EV market
- Reuters: Exclusive: Global solar capacity hits 2 TW on path to climate goal, data shows
Credits: Hosted by Shayle Kann. Produced and edited by Daniel Woldorff. Original music and engineering by Sean Marquand. Stephen Lacey is our executive editor.
Catalyst is brought to you by EnergyHub. EnergyHub helps utilities build next-generation virtual power plants that unlock reliable flexibility at every level of the grid. See how EnergyHub helps unlock the power of flexibility at scale, and deliver more value through cross-DER dispatch with their leading Edge DERMS platform, by visiting energyhub.com.
Catalyst is brought to you by Antenna Group, the public relations and strategic marketing agency of choice for climate and energy leaders. If you’re a startup, investor, or global corporation that’s looking to tell your climate story, demonstrate your impact, or accelerate your growth, Antenna Group’s team of industry insiders is ready to help. Learn more at antennagroup.com.
Transcript
Tag: Latitude Media: podcasts at the frontier of climate technology.
Shayle Kann: I’m Shayle Kann and this is Catalyst.
Nat Bullard: A training hour is GPU use, which is electricity. So that’s it. And if you can do all of this with less, what does that mean for building gigascale data centers?
Shayle Kann: Coming up: Part two of my conversation with Nat Bullard.
I’m Shayle Kann. I invest in revolutionary climate technologies at Energy Impact Partners. Welcome or welcome back rather, if you haven’t listened to last week’s episode, go back and do that first. That was the first half of my conversation with Nat Bullard, where we run through his panoply of slides and data sets on the state of global energy and climate. Here’s the second part, we cover plugin hybrid vehicles, China, electric vehicles, data centers, data centers, data centers, of course AI models and energy consumption. It’s a good one. Here’s Nat.
We’re on to slides. What? 78, 79, 85, a hundred. Okay. This is a sort of a tale of two cities situation. If you’re comparing solar and wind in terms of what’s going on right now, it’s not entirely true. As you point out the 2024 was a record year globally for both solar and wind, but certainly if you look at the US, they’re not trending in the same direction. I think people need to appreciate that. So talk to me about what’s happening with solar versus what’s happening with wind.
Nat Bullard: Yeah, a modest 599 gigawatts of solar installed last year globally, that’s a good number. That is a lot of modules. That is a lot of delivered energy once it’s all fully commissioned. And the wind sector, which I think does get painted with this brush that it’s somehow like a huge laggard did have a significant record year and more than 130 gigawatts that were installed, but nowhere close to the sheer volume of what’s happening in the solar sector. I mean it’s quite something. And if you and I were to backcast this to our analyst selves in 2008 and give us this number, we would say something truly crazy has gone on. Or we’ve ended up with a world global carbon tax of $200 a ton on all economic activity. And solar has also gotten down to only, I don’t know, 50 cents a watt to install. It.
Turns out neither of those things are actually true, but this is the market that we have and that we’ve been given. But in the US it’s definitely not that case. So Rhodium group has some great data that they pull in terms of the actual invested dollars per quarter in both industrial sectors, but then also in the actual deployed energy that they have. And wind is way off where its peak was, which is it got a peak in early, mid, mid 2020 or late 2019, solar is just on its way up. It’s more than 40 billion in the trailing 12 months and storage now invests more than wind does in the states. And partly that speaks, it speaks to a number of things I should say. One is that it’s just going to be simpler and faster to develop solar more than to develop wind in particular in the absence of any particular reform to our transmission system in the United States.
Two is that storage is highly complementary to a high solar penetration market, but also meeting the demand of a more dynamic grid in general and in multiple dimensions providing service in multiple different dimensions. And so we do see this very differing trajectories that won’t necessarily get solved at a pure market basis. They’ll probably staying like this where we should be doing as much wind as we absolutely can. That’s going to be a challenge given interconnection and it’s going to be a challenge given current politics in the us so we will have to wait and see where that goes. But yeah, very much a tail of two markets.
Shayle Kann: But I mean that’s the thing is it was already a tail of two markets prior to the new administration. And so solar was up into the right, wind is down into the right and then the new administration doesn’t seem especially favorable to renewables in general, but mean Trump has always headed out for wind. That’s been a long-term thing. It’s clearly already the case here. So this is likely exacerbating and existing trend, which is unfortunate for the wind and for solar probably. And storage.
Nat Bullard: That’s right. That’s right. And you can see it in the world’s great uncovered or the US is great uncovered power market, which is Ercot in Texas this year. We’re probably going to have more solar installed in toto cumulative than wind. And if you think solar is growing on a rapid trajectory, batteries are going even faster. So you can just see this in the markets where competition is as sort of honest and earnest as it can be and is unconstrained, relatively speaking, as possible. Solar is just in takeoff mode and the shape of the wind growth curve is looking not the good kind of asymptotic. It’s looking like it’s flattening out and you obviously want to reverse that. These are complimentary resources in particular when you add storage to solar and wind in a grid like Texas or really any place that’s sunny and windy, it’s highly beneficial.
Shayle Kann: Well, let’s talk about one of the results of all that solar and wind though, which is interesting data that you put in here in slides 86, 89, 98 on curtailment and negative pricing. So add lots more solar and wind to the grid. You’re going to get periods where you’re oversaturated on them. That’s how things work unless you add enough storage. And it just clearly in the places that we have a lot of penetration, we have not added enough storage yet, or for that matter enough flexible load that can soak up that excess energy to the point where we continue to just dump more and more power literally into the ground, at least in California. And we see negative prices at an alarming frequency in Europe.
Nat Bullard: This is one of our favorites tonight to say our, I think every year we talk about this one.
Shayle Kann: Yeah, it’s always bananas.
Nat Bullard: It’s California ISO data. Every spring, every spring, usually in April, it’s relatively cool temperatures, very sunny in California. Demand is low, demand is low, supply is high, and what you end up with is more than 750 gigawatt hours of solar curtailment in the state of California is
Shayle Kann: Here just to, I mean this is obvious, but at the same time we’re talking constantly about what an electricity supply crunch we have and how big a problem that’s going to be with the growth of AI and all this kind of stuff. And yet we forget that we are curtailing hundreds and hundreds and hundreds of gigawatts of solar every year in California alone.
Nat Bullard: And what’s tricky is that we can’t see the counterfactual in here. We don’t know what this would be without the 10 gigawatts of storage capacity that’s already been built in California. I just imagine what it would be like without that. So that’s California as a case. That’s the story we know.
Shayle Kann: Wait, sorry. Before we move on from California, I want to bring in another of your slides, which is actually, this is an even more alarming statistic about California, which is that I think just as of last year, as of 2024, California has a negative net load for the entire spring and net load for people who are not familiar is basically total demand for electricity minus the intermittent generation that is produced. And so in other words, we are producing more solar and wind, mostly solar in California in aggregate then all electricity demand for the entire spring. Not at every moment in the spring of course, but for the spring
Nat Bullard: Aggregated across exactly aggregated across to a very tiny degree, but nevertheless negative, the minimum net node is negative. Basically California is generating more than it knows what to do from its intermittent resources,
Shayle Kann: Which would be great if it weren’t all intermittent and generating when it’s generating. And as a result, getting curtailed,
Nat Bullard: Right? You mean it would be wonderful to think what would this look like the curtailment and the net load look like in a world where we had an uncapped storage resource, the four hour storage artifact in California is purely a function of written regulation. What would it look like without that? What would the market be doing in terms of longer duration storage or different business models to use that energy? We don’t know because there’s no incentive to know. There’s no incentive to even give it a try right now. It’s hard to do.
Shayle Kann: Yeah, it’s like a market inefficiency that’s just staring you in the face. You got to fix this somehow. You could fix it with policy, you could fix it by figuring out the business model for more storage. You could fix it by figuring out a business model for intermittent load. There’s all these different things you could do, but let’s just remind everybody there hundreds and hundreds of gigawatt hours of completely wasted clean electricity every year, growing every year in California some of the time.
Nat Bullard: Let’s think about the related priced version of that, so to speak, which is in Europe where you’ve got frequent occurrences of negative hourly electricity prices. So Pexa Park has this great data that has how many instances of negative prices there are in one hour basis across Europe, and it was 500 to 550 hours in 2022, and it was more than 9,000 hours last year, which is pretty extraordinary. I mean that’s basically, remember there’s 8,760 hours in the year. So in entire year’s worth of energy across Europe, it was price negative so to speak. Obviously not all of it is being price negative for all the different member states and all the different grids and whatnot, but this is pretty significant. And as ever, the question is what is going to come along and solve this? So ask a European power developer and they will tell you that hybridization putting together plants that are built at the plant level as wind or solar plus storage is one of them.
And there will be new contract structures that come along to do it. But what I would say is that if you ask a good developer, they’ll tell you that this is going to go away because the signal is there to do something with it. Why would I not take advantage of all this? Not only free, but I’m paid to take it electricity at some point and use that in some way if only purely in the power market to basically do some arbitrage and make some money. But yeah, that’s really, I mean 9,000 hours in which there was a negative price, it’s pretty striking. It’s
Shayle Kann: Crazy. Yeah, wild. Alright, before we move on from battery stuff, this is just bit of data in slide 94 that I just thought was cool and had not seen before, which speaks to the availability of grid connected batteries in various weather conditions. Now I just want to frame this as there’s a question that grid operators have been asking themselves for years as to how much reliability value, how much capacity value, for example, do you ascribe to intermittent resources like solar and wind and then increasingly now to batteries. And you can compare the reliability of this stuff up against the reliability of thermal generation, which we’ve seen coal, stockpiles, freeze and all sorts of other issues in weird weather. So how do batteries perform when it’s hot and how do batteries perform when it’s cold?
Nat Bullard: Generally quite well. This is data from Modo, which has a high degree of visibility again into the Texas market, but grid connected batteries have, if you look at the cluster of the DOT plot of all of this in the mid to high nineties, availability as a resource, whether the temperature is below 32 degrees Fahrenheit or above 105, there’s a little bit of tailing off and it becomes very, very hot, but not so much that you would call this unreliable. The lowest plot point that they’ve got for the ultra high temperatures is still above 75% and look above 105 degrees Fahrenheit. All kinds of things start to go wonky. So all kinds of things in the power grid are starting to come off at that point. Not just generation, but power equipment as well lines SAG when it gets that hot, all kinds of things happen. I think it’s an important set of data points in terms of saying that this is a resource that can be relied upon within its confidence interval, so to speak, at these times of days and in these types of markets.
Shayle Kann: So I want to talk about the topic that is the topic Jo, and has been for a while in the electricity world, which is load growth and what that looks like, putting it in context and then is it going to be real? So we’re going to do slides 1 0 1, 1 0 5, and then 180 6 where we get to talk about deep seek, which is the topic du jour of this week. But first, let’s talk about what’s been happening in terms of the expectation of load growth. You had some interesting data there and in terms of what official forecasts have been and how that has trended,
Nat Bullard: This chart probably took me longer than any other one.
Shayle Kann: Oh, so I picked one you worked on. That’s
Nat Bullard: Good simply because nerc, the body that does these official reliability forecasts simply from what I can tell, refuses to put them in any kind of usable format. So this is me going in and hand loading stuff from a bunch of old PDFs, putting it all into a data series, scratching my head for hours as to why it doesn’t line up before realizing that they’ve just sort of had a mulligan in 2012 and put in no data at all. So for those of you who are looking at the chart, there’s a little bit of an Easter egg there, which you’ll see. But yeah, we used to have an expectation of summer and winter mode growth of one and a half to 2%, like go back 20 years. That was the paradigm that the grid was looking at in 10 years time. In the next 10 years, our compound average growth rate is going to be about one point a half to 2% depending on the season. And that steadily eroded down and down and down and down. So by 2020, the expectation for us load growth was it was going to be about a half a percent per year. And that has rapidly unwound back to the point where we’re now above 1% CAGR, ten-year CAGR in the power sector in the U.S.. And it doesn’t sound like much in one sense, 1% a year, except this is a very big system and this very big system that we’ve done a very good job of making it hard to build into.
Shayle Kann: Yeah, I mean what I think is interesting about it is it’s useful to put all the fear and craziness today into historical context, which is we’re expecting all this load growth, it’s causing all these problems, but the current expectations now you could argue the current expectations are behind what’s actually going to happen and it should be a bigger number. But the current expectations are a low growth of let’s bump it up a little and call it one and a half percent annually for the next 10 years, which is up a lot. That’s triple what it was in 2020 and that’s why there’s such a whiplash effect, but it’s actually still well below what it was in the early two thousands. I think that’s the key point in my mind, right? In the early two thousands we were doing 2% annual load growth and so we’re just getting back to the growth rate that we saw in the early two thousands. It’s just the sector had grown accustomed over the past few years to being pretty stagnant, growing at half a percent a year. And so that’s the adjustment we’re making. It’s not like historic load growth, it’s load growth that we did historically.
Nat Bullard:
In fact, it’s funny, this chart is a chart of expectation. The expectation was for half a percent of load growth or 1% load growth, but there was zero. The actual reality was that there was no growth in total electric demand.
Shayle Kann: And it may indeed be that the expectation now is one point a half percent, but it’s actually three, although maybe not for the reason that we’re about to discuss for reasons, right? For reasons and they’re deep seek. So we’re releasing this on Thursday, we’re recording it on Monday. Just to peek behind the scenes, Monday is the day in which Deepsea caused the stock market to crash basically. And every that is either Nvidia and Bacon chips or all the AI providers or the power companies who had been riding high on the expected wave of load growth all crashed in the market today. The reason for that is this deeps seek AI model that appears to be maybe a much, much, much more efficient way to both train AI models and also do inference off of them. So talk to me a little bit about what we know about the efficiency of the Deepsea model and then yeah, does it change your view of this load growth equation?
Nat Bullard: It’s funny, I’ve spent my time at this looking at it actually for several weeks. They released the model that I referenced here right before the new year. So I’ve actually had this slide made for weeks. I was looking at it before, it was cool, put it that way.
Shayle Kann: Congrats.
Nat Bullard: Yeah, good for you. But I was looking at it mostly from a sort of an energy energy perspective, which is to say from a training hours perspective, how much time on the GPU did you need to do this? And deep seek is an open source small model. It was funded with $5.5 million from a Chinese hedge fund. So it’s not funded with billions of dollars of US venture capital. It is operating in multiple dimensions, a constrained environment, not a lot of money, not a lot of access to GPUs, but it is both a very large parameter model and it is very, very efficient. And the sort of benchmark to compare it to would be the NAMA 3.1 model from meta, which had 405 billion parameters and just under 31 million training hours. And Deepsea is getting similar results of V three with 671 billion parameters, but fewer than 2.8 million training hours. So just read that as a lens on efficiency. Like a training hour is GPU use, which is electricity. So that’s it. And if you can do all of this with less, what does that mean for building giga scale data centers?
Shayle Kann: Basically, right. So it’s roughly 10 x more efficient in terms of model training and from my understanding it’s about 30 x more efficient in terms of inference. So of course the question here, right? This is the fundamental question. Let’s just assume for the moment that deep seek and the process that they use to develop this model becomes the standard and the efficiency that they attained becomes the standard as well. There’s two possibilities that you can imagine come out of that at the high level from the energy perspective. One is, oh wait, it turns out we didn’t need all of those gigawatts that we thought we needed. We need a 10th or 30th the gigawatts we thought we needed. And that is exactly what the stock market thinks as of this moment that we are recording. The other possibility, of course, is the jevons paradox version of this, which is, oh wait, now it’s really, really cheap to both train these models and operate them. And so now we’re going to do 10 x 30 x, a hundred x more of it and it doesn’t actually affect the energy equation at all. Or if anything, it causes even more load growth just because of the accessibility of both the training and the inference here.
Nat Bullard: I never imagined that the CEO of Microsoft would reference the Jevons paradox, but he did,
Shayle Kann: Right?
Nat Bullard: Nadella mentioned it specifically.
Shayle Kann: Well, he’s got a certain interest in it, yes.
Nat Bullard: As he said in reference to the Stargate, let’s call it a plan for building all these data centers in the U.S.. I’m good for my 80 billion, he’s good to devote the CapEx to the tune of 80 billion to build all of this sort of stuff. And I think the answer will land somewhere in between. The challenge will be who’s making the money and who’s operating the model? On what basis is that going to accrue value in a market perspective and what does that mean for data center construction? Does it mean that if everybody can go do a very lightweight but very effective model, are they going to go back to hosting it in third party data centers or is it going to be is going to meta that runs everything on their own?
Shayle Kann: Well that and the scale of the individual data center is an interesting, from an energy perspective, it’s like a very salient, very important question. So the trend line has been, okay, we need to train bigger and bigger models. These models need to be in one site, so we need the data centers themselves to get bigger and bigger and bigger is why we’re talking about gigawatt scale and Stargate who knows multiple gigawatt scale individual data centers. And that itself is becoming harder and harder to cite from an electricity perspective. Data centers as circa 5, 7, 10 years ago, you’d see lots of these 20 megawatt scale data centers, not tiny, but way, way smaller. If indeed you get the sort of optimal performance off of an AI model, if you can build it at 20 megawatt scale, it unlocks a very different set of opportunities for where you can cite this stuff and how you can power it. So that’s an important question here. Does the results of this efficiency mean we can build 30 x bigger or more powerful models on gigawatt scale or does it mean we’re building this level of performance but at 20 megawatt scale?
Nat Bullard: A hundred percent. And when it gets down to that, it’s like, okay, 20 megawatts is a not insignificant, but also not insurmountable load interconnection request.
Shayle Kann: No, totally. I mean we have a portfolio company, not to name the company, but we have a portfolio company that needed a new facility, they make hard stuff and they needed a new facility. The place that they found is an industrial property in the middle of the country that just happens to have 40 megawatts of capacity. It’s kind of amazing. They don’t need all of that capacity. They’re not making a data center or anything like that. But that stuff still exists at that scale gigawatt scale. Oh wait, we just found a gigawatt of capacity does not exist anymore.
Nat Bullard: It no longer exists. And in fact, I wonder if it ever actually did. I mean that’s unlikely to have ever been the kind of thing that could have been easily integrated. And if it did, it happened very quietly in a way that was kind of designed to not get a lot of scrutiny. It was designed to make sure that not a lot of people found out about it.
Shayle Kann: Alright, so let’s move on from the question of how much load growth to how are we going to meet this load growth? Slide 1 0 6, I’m going to rename this slide for you. The new name of this slide is build some fucking transmission. That’s the name of the slide. Now, can you describe what the slide is?
Nat Bullard: Sad slide whistle sound here. In 2013 the US built 3,200 miles of 3 45 kv and a transmission. So high voltage, long distance transmission, 3000 miles, whatever, not a lot, but infinitely, not infinitely, but shall we say statistically significantly better than the 125 miles of high voltage going distance transmission that the US built last year, which is pathetic. There’s really no other way to say this. And if you look at the trend of demand growth and you look at the trend of high concentrated sites of new demand that might be far from load center or correction are becoming their own new load center. And you look at the need to decarbonize that electricity insofar as possible, and you say that the answer is to build 125 miles of 3 45 KV and up transmission, it’s not a very good answer. It’s really not an answer at all.
Shayle Kann: No, I mean to the extent that I think that the curtailment data is wild and crazy every year when I see the amount of new transmission that we built in the United States, that’s the craziest number to me. It’s wild to me how little it’s new transmission we’ve been building for a while now. It’s been going down generally from what was already a pretty small number basically since the mid 2010s. But we haven’t fixed it yet.
Nat Bullard: No, and this is again, to some extent an artifact of well load wasn’t growing, so why would you do this? All the, although if you ask any renewable develop prob, they’ll tell you, I know why we do this. Do you want to change the nature of what we generate? Then we need new transmission.
Shayle Kann: Yeah, maybe load wasn’t growing, but the interconnection queues were, I could tell you that
Nat Bullard: Yes, they were right. So there’s no non snarky thing to say about this. I feel like it’s bad and it’s something that needs correcting and it is not because people are unwilling to fund building good transmission. It’s not because there’s not demand to put things into a long distance transmission network. It’s simply because it’s difficult to build or possibly impossible to build. And you might need to reform planning and permission in order to do this, but that’s a policy question, not a market question.
Shayle Kann: All right, moving on. I want to talk about vehicles and one thing that I think is underappreciated about new energy vehicles, which is the rise of plugin hybrids, particularly in China. Predominantly in China actually. But just interesting how strong a position plugin hybrids are in.
Nat Bullard: It’s really interesting because I think we need to do a big reset in terms of what we consider to be a plugin hybrid. I think people’s versions in the United States is a Prius with a plug as opposed to a vehicle with 200 plus kilometers of battery range on its own, plus a small, highly efficient internal combustion engine. What in my part of the world here in Southeast Asia, people refer to as an extended range ev, which is kind of a weird way to say it, but it’s like a vehicle that is capable of running for something like 800 or a thousand kilometers without needing to fuel up the gasoline tank. And these are predominantly a China thing in terms of where they’re made, but they’re being now exported all over the place. Like I was in Thailand last month and there are ads for the extended range EV pickup trucks from China all over the place.
They have the same ads for them in Brazil. BYD is getting ready to ship its extended range EV ute IE pickup truck in Australia. So they’re starting to pick up in places where the infrastructure maybe not be fully built out. The performance advantages of EVs and the economics of them make sense. You do need that. It’s not just a range anxiety thing, but it’s a sort of a range reality kind of thing that you might be long way between electrical infrastructure and so yeah, they’re picking up, but they’re not your grandfather’s Prius and let’s put it that way. These are very different vehicles.
Shayle Kann: Okay, well you just gave me the perfect segue to the other vehicle related thing, which I wanted to talk about, which is slides 1 21 and 1 24, which is here comes the Chinese exports basically, and there go the Chinese imports for that matter. So it’s both are happening at once. But talk to me about Chinese automakers.
Nat Bullard: So we are of the age when we remember the fear that Japan auto and Japan Auto Inc was going to take over the world’s auto manufacturing capacity, the lean production system, the sort of Toyota method of doing things was going to be dominating. China exported 6.4 million vehicles last year. Five and a half million passenger cars, 2 million of those were electric vehicles. This is far above any other country’s export capability. It’s more than Japan exports, it’s more than Germany exports way more than the us. We’re not a big exporter vehicles. And it’s just an extraordinary rise from a couple of years ago. Basically about a million vehicles a year exported for a very long time, and then boom, exporting 6.4 million in the course of just a couple of years, like four years of runup the Chinese auto. There hasn’t been this sort of machine for building machines like this in terms of pure scale ever.
China in 2023 was just a little bit under 40% of the globe’s auto making production, not capacity, but actual production. And what does that mean? It means first you see that in the domestic market where the standalone or even some of the JV foreign companies like the Japanese companies or the German companies are having a hard time moving product, then you see it sort of propagating outwards in terms of those company’s market shares and other markets and where they’re going. But yeah, I mean a good example is GM in China used to sell close to 4 million vehicles a year. Almost all of them were made locally for the record. So they were not exports, GM exports to China, but they were GM companies and brands of GM being sold there. And that was 2017 and now it’s less than a million in 2024 through almost through the end of the year. So it’s really a very, very, very rapid diminution of all of these other companies. And it’s this incredible amount of capability that can go somewhere. And the question is, where is that going to go and how.
Shayle Kann: Yeah, clearly tariffs and import bans aside, in the absence of that, the wave of Chinese vehicles that is going to wash over basically every other country in the world seems imminent and rapid. No question about it otherwise
Nat Bullard: I’m living in it. So the land transport authority in Singapore every month publishes details on the cars that got registered. Admittedly, it’s not a huge auto market, it’s like 40,000 vehicles a year, new vehicles. And in December of 2021 BYD had 0.2% market share. And of December, 2024, it had 14.4% market share, making it the second biggest auto seller in Singapore. Second only to Toyota. And mind you, Toyota’s business has got a hammer lock on all taxi cabs pretty much. So that’s pretty fast to go from nothing to more than 14% and to go from minimus to second in the league table in terms of sales in a market, in a place with not great electrical charging infrastructure, but great product at a good price, new models constantly being refreshed. And there’s others I see routinely in my garage here, brands that you never see and that people in the US have probably never even really heard of Zeer and Xang getting new cars from new companies all the time here and they’re all Chinese and they’re all EVs. Here they come.
Shayle Kann: Okay, we already talked a bunch about load growth and AI and stuff like that, but you have a bunch of interesting data center related slides that I think have data I like. So we’re just going to talk through ’em real quick to slides 1 66 to 1 73. So we’re getting there. Alright, first of all, the first one is just interesting. We spent more on data centers in terms of CapEx last year than hospitals. I didn’t actually realize how much we spent on hospitals to be honest with you. We spent 31 billion on data centers, 27 billion on hospitals. It’s a lot of money to spend on hospitals.
Nat Bullard: It is a lot of money. And remember that’s structures, that’s not mission critical computational stuff. That’s basically the buildings, the buildings in some aspects of the infrastructure that go with it. Basically construction work, not GPUs, let’s put it that way. But yeah, I mean we should be investing a lot in hospitals. Healthcare is a major part of the economy and it’s something for which we have quite a lot of need in the United States. But yeah, the data center figures, which only really begin in 2014, taken off basically, and it’s one of these trajectories too that moves from being a kind of jagged low line to being a very smooth high line in terms of where this trend is going. It’s smoothing itself out into something that really looks like it’s got the beginnings of a real exponential growth to it. We’ll see if that actually prevails or not. But yeah, that’s just construction on buildings. That’s not all of the other aspects of CapEx, which really are the real significant expenditure and Investment.
Shayle Kann: All right. So anyway, we spend a lot of money on hospitals. That’s interesting. We’re obviously now spending more on data centers and that number is going to continue to go up. The other thing we talked about actually for a minute already, which is the portion of data center servers that are in either large co-located data centers or hyperscale data centers as opposed to small on-prem type of stuff. Now I think intuitively everybody knows that number has been increasing, but it’s interesting to see how much has increased and what the timeframe has been.
Nat Bullard: This is exactly my take on this intuitively. We all know that your average tech firm is not running bare metal in their basement the way that they did 25 years ago. But it’s interesting to see where it goes because that has major implications for the infrastructure that gets built. Meeting the demand for Google’s very first server, which was in a building in Palo Alto, like in a cage, basically is very different from meeting the demand for its latest hyperscale data center in terms of what you need to build. But basically a lot of the server, the server mode is now going into these locations that are built really big and they’re built with all kinds of efficiencies within them that are really important to note that are different than being on premise. And we’ll see, does that tap out? Does that reverse? It will be fascinating the context of the deep seek conversation we were having to see if that doesn’t pull backward at some point. Does that actually– do people decide, actually no, I can go on-prem now. It turns out that this is something I can do in a chilled closet as opposed to an industrial park on the edge of town.
Shayle Kann: All right, speaking of industrial parks, the other interesting data you have in here is state level data on data center energy consumption in the United States. Now, I’m sure most of our listeners know that if you ask people what state has been most affected by data center energy or just data centers in general, I think most people probably know it’s Northern Virginia. I wasn’t aware that over a quarter of all electricity consumption in the state of Virginia already as of today is coming from data centers.
Nat Bullard: That’s a big number as of 2023. We don’t even have the 2024 data yet to see what this looks like. So it is really, really significant and it’s a clustering effect that makes a lot of sense in terms of logistics, particularly in terms of fiber access to what’s there. And generally speaking in a place that was like, we don’t have a lot of industrial load where we are able to build, let’s go ahead and serve this. But yeah, Virginia is just under 34 terawatt hours of power consumed by data centers in 2023 and almost 26% of state power are already going to that. And even more important are the designs that companies have in terms of what they want to do and how they want to expand. And it could blow the doors off of these numbers. It could absolutely blow the doors off of these numbers.
Shayle Kann: Speaking of blowing the doors off of these numbers, you have another slide that I really like just because what the hell is going on in Rappahannock Electric Cooperative territory? You want to talk about that a little bit?
Nat Bullard: So Virginia’s really fascinating for this because you got a great big utility dominion, but then you have a lot of co-ops that actually are service providers to all these different communities and co-ops are like, look, we handle rural areas and ex-urban areas. Our main load might be cooling in different times of year. We’re not really built around big industrial consumption. We are not a giant financialized entity either. Obviously we have real finances, but we’re not dominion. And Rappahannock had this filing in Virginia where they basically are saying, look, we are now getting requests for interconnection that in a single location are bigger than our entire peak load right now we have a peak load of 1.2 gigawatts. We have people asking us to interconnect one asset that is bigger than that in some cases, four times bigger than that. I have to say that if somebody’s trying to connect a five gigawatt data center in Virginia, that may have low likelihood of actually going through.
But it’s indicative of the kind of trends that people were wrestling with. And what’s really interesting about this, and something I’ve spent some time looking at is what does this do to what Severn Bernstein at Berkeley says is the duty to serve? What does this do in the market construct of saying, sure, we’ll provide power to you if one company shows up and it can basically blow up the demand profile permanently by just introducing two to three gigawatts of load in a place that has one gigawatt peak load related to that though and equally important. And it’s nice to see that this is entering into the regulatory discussion. What happens if they don’t show up? What happens if I am Northern Virginia Electric Cooperative or Rappahannock Electric Cooperative and I go out there and build on the expectation of getting five gigawatts of data centers? And it turns out that these things got deep sunk and don’t get built at all.
Like what does that do to my rate payers? And we’ve seen similar things happen with a nuclear build out in places like South Carolina with the VC summer plant that was basically a 7 billion worth of pour concrete and nothing more. What would be the equivalent happening there and on? What market structure do you need to account for and accommodate this kind of building? Again, all back to policy and regulation questions. This is not a finance question, it’s not a technology question. It’s a decision question about what our concept of duty to serve is going to be
Shayle Kann: Deep sunk. You just come up with that on the spot. It’s pretty good.
Nat Bullard:. I did. You heard it here first, Shayle.
Shayle Kann: I like it. Okay, we’re going to wrap up with one more thing on AI and energy, which is data that I really like because it drives home an important point. An important point that actually I’ve discussed with Brian, Janice on this podcast before. He talks about this concept to the bit watt spread. And it’s basically the idea that actually the bits that you’re using producing is worth so much more than the watts you pay to produce the bits. And that’s where there’s this sort of limitless willingness to pay to some extent within the data center world. But you have data that is really interesting in here on the share of total cost associated with training an AI model that can be ascribed to energy. And I think people think of it as being such an energy hog thing to do that it would be a big share of the overall cost, but it’s not.
Nat Bullard: So to be very clear, this isn’t my data. I got this from Andy Lubershane, your colleague, our mutual friend. He in turn got it from Epoch AI, but they have these great charts. That’s the breakdown of model costs for training and experimentation in terms of what goes into it and r and d staff and also in this case, they include as they should, the cost of equity is between like 30 and almost 50% of training a model and of doing experiments with it. AI
Shayle Kann: Researchers are expensive.
Nat Bullard: AI researchers are expensive and they demand compensation. The energy component of that comes way after the AI accelerator chips, the other server components, the interconnection costs, and it’s somewhere between 1.7 and 6.3% of the cost of training and experimentation. Now it’s possibly a bigger, and it probably should be a bigger part of inference of actually operating the things that are up and running, and that’s an important caveat. But what Andy says is that energy is everything and energy is nothing. And it’s everything in the sense that it’s binary. I either have it or I don’t in order to run these processes and capabilities that I need. But once I’ve got it, I am not breaking the bank paying for electricity. And this is unusual in the world of very large consumers, like if you are in the business of say, making aluminum or making steel, you are acutely sensitive to the cost of your power input.
Very, very sensitive to it, but it’s not breaking the bank of a model at all. If you had double the cost of electricity, the more important thing is that can you get it right now and can you get it how you want it? Am I able to get it reliably? Am I able to get it with the uptime that I’m expecting? Can I get it quickly? Is this something I can contract for now and have in six months? And it’s very different. It’s a very different kind of buyer than I think we’re used to seeing than grids are used to seeing elsewhere. A big box retailer is very sensitive to the cost of electricity at some point that carries through to the very, very low margin business of selling goods that it does. But we’re not in the scenario where people are jacking up the cost of models that in many cases are still being given away for free because of the electricity cost underneath it. It’s a different paradigm
Shayle Kann: That it is. And with that, Nat, we’ve reached the end of this year’s 200 slides of glory. Thank you so much as always, for letting me cherry pick.
Nat Bullard: Shayle, always a pleasure. Thanks again and I look forward to hearing it live.
Shayle Kann: Nat Bullard is a co-founder at Halian and the former Chief Content Officer at Bloomberg New Energy Finance. This show is a production of Latitude Media. You can head over to latitude media.com for links to today’s topics. Latitude is supported by Prelude Ventures, prelude backs, visionaries, accelerating climate innovation that will reshape the global economy for the betterment of people and planet. Learn more@preludeventures.com. This episode was produced by Daniel Woldorff, mixing by Roy Campanella and Sean Marquand theme song by Sean Marquand. Steven Lacey is our executive editor.
