First, it was a power bottleneck. Then a compute bottleneck. Now, as AI agents burn through tokens faster than anyone predicted, we’re back in a compute shortage. Meanwhile, it’s getting harder than ever to site and build the data centers to alleviate it.
This is shaking up who builds the energy infrastructure to serve it, and how it gets built.
This week, we’re diving into the biggest utility deal in American history: NextEra’s attempt to buy Dominion. If it happens, it would combine the biggest renewable energy developer in the US with the utility serving the world’s largest concentration of data centers.
What does it mean for their power development strategy? We debate the regulatory path, the power mix question, and who actually benefits.
Then we turn to an infrastructure debate. Are we entering a new era of distributed, grid-connected data centers that will overshadow the gigawatt-scale campus model?
And we close with a rapid-fire look at some ideas for solving the compute crunch: home inference hubs, water heaters that serve AI, and wave-powered data centers.
Credits: Co-hosted by Stephen Lacey, Jigar Shah, and Caroline Golin. Produced and edited by Stephen Lacey, Sean Marquand, and Anne Bailey.
Open Circuit is brought to you by FlexGen, a leader in integrated battery energy storage solutions and energy management software. FlexGen helps owners and operators gain greater visibility and control across complex energy systems to maximize performance. Learn more at www.flexgen.com.
Transcript
Stephen Lacey: From Latitude Media, this is Open Circuit. First, it was a compute bottleneck. Then it was a power bottleneck. Now as AI agents start burning through tokens, we’re back in a compute shortage and it’s getting harder than ever to build the data centers to alleviate it.
As both constraints get worse, we’re seeing a shakeup in who builds the infrastructure to serve AI and possibly how that infrastructure gets built. First up this week, NextEra wants to buy Dominion. It would be the largest utility merger in American history. What does it mean if one of the top clean energy builders takes over the utility serving the world’s biggest data center market?
Then, with compute rationing hitting AI companies, a new debate is emerging over the next wave of build out. Will a distributed inference model start to emerge? And we’ll close with some new ideas on how to solve the compute crunch, home inference hubs, wave power data centers, even water heaters that serve AI. That’s all coming right up.
Welcome to the show, everybody. I’m Stephen Lacey, the executive editor of Latitude Media. The three legs of the stool are back. Caroline Golin is the chief growth and policy officer at NRG. She is the leg that keeps us both upright. Welcome back. How are you?
Caroline Golin: I’m great. I am tired. So I’m hoping that you two will carry or have the longer legs, I guess, of the stool today. That analogy doesn’t really work, does it? We still topple, I guess.
Stephen Lacey: Yeah. What does that make you, Jigar? Are you the wobbly leg that you need to fold the piece of paper up today?
Jigar Shah: I’m the one with the adjustable…
Stephen Lacey: Once you get it right, it’s stronger than ever though. How’s it going, Jigar?
Jigar Shah: It’s going great. It’s going great. It is just such a busy week, which is interesting.
Stephen Lacey: What’s busy about it?
Jigar Shah: I mean, we had our good friends at Voltus finally announced their big deal with Google, which is great on demand flexibility. I think that the PJM has decided that they have no choice but to just do all demand flexibility all day. You’ve got huge launches of product around battery storage and other things. And so just keeping up is a full-time job these days.
Stephen Lacey: For sure.
Caroline Golin: Absolutely.
Stephen Lacey: And so let’s turn now to some big utility news that unfolded when we were apart. NextEra wants to buy Dominion. This is the biggest clean energy developer in the country with enormous pipeline of renewables and storage. Wants to take over the utility serving the world’s largest concentration of data centers. It’s an all stock deal that would value Virginia-based Dominion at $120 billion. The combined entity would serve 10 million customers and executives said that they had a combined pipeline of 130 gigawatts of projects, much of it driven by data center growth.
So it raises a bunch of questions. What does it mean for the resource mix? What does it mean for electricity rates? What does it tell us about how AI is shifting the corporate landscape in the power sector? What would this combined entity look like? Let’s start with the basics. Caroline, this merger would not only create the biggest power company in the US, but also America’s third largest energy company behind Exxon and Chevron. What would this entity be able to do that NextEra and Dominion couldn’t do on their own?
Caroline Golin: I’m not even sure that NextEra knows the answer to that. I think NextEra has been in the hunt to expand its IOU portfolio for quite some time and it hasn’t been successful. It’s been tried to buy what is now Dominion, South Carolina several years ago from Santee Cooper and tried to buy Hawaii and is now sort of looking for its next step. I think it knows the Southeast. It likes the Southeast. I think it also knows PJM in the way that it’s developed in its renewables arm. And so it’s a little bit of the devil it knows. I will also say that in terms of the IPP landscape, NextEra has been very aggressive and rightfully so in developing more of a platform business for its data center pipeline. And so I think that putting two and two together at a very basic level, it’s looking to expand its regulated book.
It’s looking to capitalize on the investments it’s made in terms of its data center pipeline. And Dominion operates in a hybrid market, but the truth is Dominion operates much more like a traditional IOU than most entities in PJM. And so it makes sense from sort of checking those boxes. I’m not sure and I’m not privy to it, but I’m not sure that there’s a great big master plan. I think it’s been batting at trying to expand its IOU book. It’s sort of just moved up the East Coast a little bit. It started with the goal-
Jigar Shah: Didn’t they try to buy Duke at one point?
Caroline Golin: Yeah, they did. Yeah. They tried to buy Duke, South Carolina.
Jigar Shah: And Evergy maybe?
Caroline Golin: Yeah. And they tried to buy Santee Cooper. The pushback in South Carolina was that they didn’t want more big corporate. South Carolina is a very interesting state and it’s one of those states like Ohio, like California, like Arizona, which is just sort of unique to its own energy ecosystem culture. And so they pushed NextEra out I think despite some very aggressive persistent bids and we’ll see. I mean, I give it more than a 50% chance that it goes through, but I wouldn’t give it … I wouldn’t say it’s locked down.
There’s a lot of pushback and there’s a lot of populous pushback across the board on all energy issues. And when you are mixing energy with data centers, I think you could stir up a lot of local fervor that maybe five years ago, six years ago wouldn’t have been there. But I think it’s really more just about they’ve been trying for a decade and this was the next one up the East Coast that made sense. And Dominion has a great portfolio for them for what they know in terms of what they know how to operate and what they know how to deploy. Yeah.
Stephen Lacey: Jigar, you’re not one to hold punches. So you called Dominion the worst run utility in America. You’ve also said that this isn’t really an AI story. It’s a competentcy story. Make the case.
Caroline Golin: They’re not the worst run utility in America.
Jigar Shah: If you just look at their stock price since 2019, outside of PG&E and Hawaiian Electric that went bankrupt, Dominion was the worst performing stock of any single utility company in the country. That’s all I said. And it’s true. I mean, when you think about the fact that Bob Blue likes, bless his heart, I mean, the guy was brought in to turn around Dominion in 2020. And so he has had nothing but heartache since he’s been CEO. He had to sell his natural gas assets for a fire sale to Berkshire Hathaway. The amount of money he made there didn’t even pay off the debt that they had taken on there. And so it’s been a tough slog for him to figure out what to do with Dominion. And now they’ve got the offshore wind farm that’s still not completed and probably will be completed in 27.
So you’ve got that overhang because that’s in construction. Until it becomes operational, you don’t get revenues from it. They’ve got a ship that they’re building down in Louisiana for Jones Act compliance. And so it’s a lot. But then I also gave it to NextEra too. NextEra, when you think about it, they were already more valuable than ExxonMobil in 2020 and 2021 during the COVID pandemic. Then they basically were upside down in their yieldco, because interest rates went up and they were long debt. So then they ended up in a situation where their stock price went down by like 35 or 40% a few years ago. Now they’re like climbing back out. They separately had to fire their CEO of Florida Power and Light two years ago for rampant basically corruption in Tallahassee people deserve each other.
Caroline Golin: So listen, first of all, Dominion is not the worst run utility in the country, but what happened that forced Dominion to go from what was a pretty bankable and profitable utility to one that was struggling were a couple of things. First, the BCEA, which everyone rallied as like the biggest climate coup for clean energy at the state level, which was forcing Virginia to go 100% clean energy, but it was doing so in a way that really required a significant amount of confidence in Dominion to build out offshore wind and to build out a lot of resources that frankly probably was not the best path for that utility. And we could probably spend a whole show on that. And the other big thing that happened was that the commission started to pay a bit more attention to Dominion’s returns and tighten the belt. And so when those two things happened in concurrence, you don’t have quite the cash flow that the company had before.
And I think Bob, for what it’s worth, I have a lot of respect for Bob and I think he’s done the best he could with a utility that was probably inevitably going to be carved up given the data center onslaught, the PJM changes, the clean energy requirements and the historical return schedule that they had been used to that all of a sudden they hadn’t got anymore. So I’m not surprised that Bob took Dominion down this path. I do wonder what it means in South Carolina more than actually in Virginia, because if you look at the portfolio, I think that the Dominion South Carolina folks have always wanted to be their own utility. They’ve never particularly been interested in responding to Richmond. And so I think that’s an easier sell than necessarily the whole portfolio, even if everything’s up on the board. I wouldn’t be shocked if at the end of the day Dominion, South Carolina goes to NextEra and Virginia stays Virginia to be honest, but we’ll see. We can make a bet.
Stephen Lacey: Yeah. So Jigar, are there any advantages that this combined entity bring?
Jigar Shah: I mean, to be crystal clear, I don’t know that I care. I mean, in general, I just feel- I mean,
Caroline Golin: It’s a good answer,
Jigar Shah: Right? I mean, it was so weird because I just posted something on LinkedIn about the fact that Dominion had challenges and I think Bob Blue’s great. So I’m not badmouthing him. I think he was brought in at a particularly difficult time and then they had to write off their natural gas pipeline investments and was it mountain state or whatever that didn’t get completed. And so there was just a lot of stuff going on, right? Then NextEra had a lot of stuff going on. So that’s all I posted. I don’t have this desire for all of the utility companies of the country to get aggregated up into one. And if I did have that desire, it wouldn’t be NextEra that would do the work. It would be like BlackRock or JP Morgan’s infrastructure fund that would like buy up everybody and turn us into a copper plate.
I mean, I think the bigger challenge I see right now is that when you think about how many laws have been passed at the state level in Virginia, right? And I was on the other side with DOE with the grid deployment office and loan programs office, et cetera. I mean, Dominion is not known for its extraordinary ability to free up capacity using modern technology for the data center alley folks. They still haven’t deployed grid enhancing technologies fully. They’re still piloting line vision even though they love them, right? They’re still not building out advanced conductors.
When you look at what Gil Quinones is doing in ComEd’s territory, he is deploying all of that stuff at scale super fast and you’re like, why is Dominion not doing that when they have an eight year backlog of data centers? You’re like, “This is literally the easiest goddamn thing to do. And Dominion’s like, “Ugh, this is so weird for us.”
Caroline Golin: Well, I mean, it does beg the question like FP&L is not exactly the hotbed for data center development. It’s a different mountain to climb and it creates a different question of growth and management. I think NextEra made a ton of money on renewables because it had the capital to meet the economies of scale needed to do one of two things, come in under avoided costs for IOU programs, which it did throughout the Southeast very successfully and to meet sort of the threshold for an energy plus rec deal for corporate buyers across US. It was a pretty simple business model and they had the cash and Jigar, you probably know more about this than I do. They had the cash to be able to treat it more like a portfolio of investments, which most developers didn’t have that. They were just deal by deal trying to make ends meet when renewables were taking off.
That’s a different business model than being able to take full deliverability risk, being able to take on capacity and power supply in a way that from NextEra’s renewables arm, I’m not saying they can’t, it just hasn’t been the business model and it hasn’t been the way they’ve made returns. I think they’re moving into that. And in FP&L, they don’t operate, it doesn’t operate in a wholesale market. PJM is what it is and will continue to be the head scratcher of the Mid-Atlantic, but it is going to also continue to be very demonstrative in the way things get built out. So the reason why I think we should care is a question of fit. NextEra has the cash clearly. They have the ability to develop renewables and to sell renewables under a certain business model, but whether or not they have the institutional knowledge and capability to turn around Dominion, I don’t know. I wouldn’t say that what they’ve done historically is- Oh, but –
Jigar Shah: Oh but that’s not what this is based on. Given that there’s four previous failures by NextEra, this is all about their bedside manner. Their bedside mare was terrible in Hawaiian electric. Territory. Can they sweet talk people in Richmond? I don’t know, but they better start learning how to sweet talk people in Richmond. This is not about your capabilities and all this other stuff. It’s about does the governor of Virginia actually think this is good for the citizens of Virginia?
Stephen Lacey: Yeah. So is there a difficult regulatory pathway here then?
Caroline Golin: I think so. I mean, I think so. Like I said, I don’t think this is a slam dunk, but if it does happen, I mean, I go back to the same one, which is like, will they turn Dominion around? Will they do the things that Jigar is talking about?
Jigar Shah: The governor of Virginia is in her first year, right? So she will say, my sense is probably going to be against it, right? But let’s say she was like, “I could be convinced to be for it.” She will say, “You have to finish the offshore wind farm. You have to fully build out grid enhancing technologies. You have to agree to the grid utilization law that we passed. You’ve got to fully build out the 300 megawatt VPP that is in law. You have to do all these things.” And NextEra will say anything you want if we get a permission to-
Caroline Golin: And then she won’t be governor in a year and the legislature will flip and they’ll pass a different law. I mean, that’s what we’re doing.
Jigar Shah: I’m pretty sure that this stuff can all get done in 12 months.
Caroline Golin: Well, maybe, but what I’m saying is that Virginia is notorious for the flippy floppy.
Stephen Lacey: Well, what about rates? I mean, this is an area of contention. NextEra is offering $2.25 billion in bill credits, about $550 per customer as a sweetener. Consumer advocates are skeptical. They see this as a temporary measure. What are the possible scenarios for rates?
Jigar Shah: I think that what’s really important in this moment is just how smart the staff is around the governor. I mean, Abigail Spanberger and Mikie Sherrill for that matter in New Jersey had to run on rates in their governor election. So when you think about all of the people that joined her transition team and then also now she’s elevated the position of energy into the governor’s office, which Asifis who’s joined in … And so I think they’ve got a team that can negotiate a much better deal than for instance, like Mayor Bowser did for the Pepco merger with Exelon where she just gave everybody a one-time $50 credit and just got rolled over by Exelon. But when you think about just how smart the team is around the governor, I think they’re going to ask for a lot and NextEra may balk and say, “Well, we’re not going to give you all that and therefore we’re not going to do the deal.” But ultimately, I think the governor has a plan.
She passed it into law within 90 days of coming into office. And so she is going to demand that they implement that law. That’s what she’s going to say. She’s going to be like, “Great, take your $2 billion and implement the damn law, get utilization rates up, do grid enhancing technologies, deploy advanced conductors, do all the stuff that we put into the law.” And if they say yes and they start doing it within 24 months, great. If they say no, then she’ll tell them to pack.
Stephen Lacey: So one last point here. What does this tell us about consolidation in the power sector around the AI load growth thesis? We saw BlackRock’s Global Infrastructure Partners buy Minnesota Power, BlackRock Consortium is buying AES, now this. What does this mean for sort of the corporate landscape and power generation?
Jigar Shah: I think it’s a lot different when a financial player is buying a company like ALLETE, for instance, Bethany is amazing and I think they’ve already invested $500 million into her utility, which is like almost more money than ALLETE has ever raised as a publicly traded company in its history. When you think about just how important it was for her to switch out her public shareholders to a more deep pocketed infrastructure owner, that was really important for the people who live in elite’s territory. And so I think that similar vein is going through public service of New Mexico. There’s other places as well where these utilities just don’t have the ability to raise the amount of cash necessary to do the things that the governors want them to do on data centers or anything else. And so having a financial owner who has deep pockets is going to be critical.
This is a little bit different. NextEra does have more money than Dominion does, so that’s obvious, but I don’t think they’re viewed as a financial player in the same way that GIP or JP Morgan infrastructure or some of the others would be viewed.
Caroline Golin: So a couple points here. One, I completely agree with Jigar. This is very different. It’s apples and oranges in this case. NextEra is an outlier and they’ve been an outlier for a very long time. And so I wouldn’t say because NextEra can do it, it can be replicated. They have operated differently. To be fair, the firewall between their regulated and deregulated arm has been permeable. And so I think they’ve been able to grow at a level that others have not been able to grow and take that for what it is. The other thing here is that what I’m interested in, and I don’t have any answers to, and maybe one of our listeners will have an answer to, is that the vast majority, and actually Jigar and I talked about this in London a litle bit, was that the vast majority of investors, traditional IPP investors, still want to see fairly traditional deals.
They want to see big centralized power plants built. They want to se long-term offtake. They want IOUs to be building what they know how to build and they want to see load growth with a lot of political and regulatory capture. And we have to just be honest about that. The investment community does not want to evolve, does not want to see a ton of new capital stack needed to launch innovative or what we would say non-traditional returns in this space. So NextEra has been a bit of an outlier in the fact that they’ve proven some of that naysayer wrong. To be honest, when NextEra launched a lot of its renewables, I’m not sure anyone expected it to be as successful as it was. They were sharks in the field to their credit and they were able to grow on a business model that I’m not positive all the IPPs in the country could have done.
So does investment change because NextEra does this? In as much as NextEra proves out new return models, yes. So if NextEra goes and says, “We’re going to invest in distributed, we’re going to invest in grid enhancing technologies, we’re going to invest in utilization and figure out how to commoditize that and return it to shareholders,” then the investment circle may change and then you may see this stir something. If they don’t, I think investment stays the way it is and the margins on conversion for cashflow is going to keep everything pretty clunky in the way that it’s been. And so I hate to say the same thing that the Jigar says, I don’t know how much this matters in the big scheme of things. I think it could matter if NextEra takes a very different approach to building power for Dominion. It could. It could be a game changer because everyone will be watching it. I don’t know that they will though.
Jigar Shah: I was surprised that the Sun Sentinel came out against the merger.
Caroline Golin: The Sun Sentinel comes out against everything. I mean, they come out against Groundhog Day, I think. They don’t like anything, but you’re going to see that. And the truth is if you see two or three more botched big data center investments, I don’t know. I mean, I don’t have a crystal ball on the populace around data center pushback, but it’s bleeding to big corporate, it’s bleeding across the board. And so again, I don’t think it’s a slam dunk.
Stephen Lacey: So that’s the picture for the corporate landscape and serving all this new demand. Let’s actually talk now about how this demand might show up. The last few years, of course, have seen this steady increase in large training focused data centers, but something is shifting as the demand for inference picks up. We’re seeing the explosion of agents, companies are blowing through corporate AI budgets. AI labs are running against their compute limits. And so as a result, we’re seeing this shift in the conversation about how and where we’re going to serve all this new inference. Are we entering a phase where distributed computing could overtake the massive centralized data centers that we’re seeing built out right now? And in that world, how do energy needs change? Jigar’s been firmly arguing that we are undergoing this shift. I’m a little skeptical about how quickly it’s happening, but it’s a great conversation.
I’m dying to get Caroline’s take on this. So let’s roll into it. Jigar, you recently wrote that Google, Meta, AWS are quietly converging on the same conclusion that the gigawatt campus model is hitting limits and that distributed grid connected computing is the future. Make that case.
Jigar Shah: Well, right now I don’t have to make the case. Right now you’ve got 60, 70% of all gigawatt scale data centers that are behind schedule, many of whom haven’t even started construction that were supposed to be completed in 2026. And so they’re just behind. And so what has happened is Google and all the other hyperscalers and frankly, they’re being bypassed directly by Anthropic and OpenAI who’s going directly to other owners of data centers are coming forward and saying, “We have a 200 megawatt data center.” And people were like, “Yeah, we could do 200 megawatts.” Then they’re saying, “We have a 50 megawatt data center.” They’re like, “Ah, that was small, but we’re kind of desperate. We’ll do that deal.” Then they’re coming to 20 megawatt data centers. They’re like, “Actually, we might do that.” Then a startup company called Rune just raised their round of capital and were greatly oversubscribed and they’re putting up five megawatt data centers.
Then NVIDIA signed up with Prologis and Prologis is now signing up five megawatt data centers everywhere and now you’ve got the pilot with Span. And so what’s happening is we had a major chip shortage in the country and NVIDIA was like selling out to 27 and 28. Turns out a lot of those chips are now available on the secondary market because the data centers that they were supposed to go into that were one gigawatt in size are not yet up and running. And so like everybody that I’m supporting on the distributed side is like, “Oh yeah, I just secured Blackwells. I just got more Blackwells. They just got offered to me. I just bought more Blackwells.” And so the GPU prices have not gone up and in fact they’ve stayed pretty stable and folks are just unloading it from a cash on cash basis and then they’re getting their new shipment in whatever it is like six months from now that’ll backfill it.
So I don’t know that people are aggressively saying that they’re against one plus gigawatt data centers, but they are saying that those data centers are not going to come online in time to meet their compute needs. And so they have dropped their standards that they were once sitting on and they are signing contracts for all of this distributed compute now in a way that they were saying that they wouldn’t just like six or nine months ago.
Stephen Lacey: Caroline, is this the same story you’re seeing?
Caroline Golin: Well, I back it up a little bit and say there were some technical advantages to concentrating compute or training, concentrating training, which is that training was a bit of a mess in the first year from a harmonics perspective, reliability perspective, it was blowing through servers in a way that I remember when I saw the first test at Google, I was like, “Oh gosh, if that was at a gigawatt scale, we’d blow the grid.” And so there was actually a lot of thought and logic behind we really need to work out the kinks in this. And if you try to work out the kinks of a new technology across a hundred different point sources, it makes very hard just from a scientific engineering process perspective to do that well. So the idea of concentrating training to work out how training is going to work made sense.
Before you even talk about power and self-supply and all that stuff, it made sense. I think what a number of entities are finding is that, hey, building power plants and operating power plants is hard. So going off grid or even behind the meter and building our own power plants and trying to do that is not our core competency. And unfortunately it’s not a core competency of a lot of utilities in this country anymore. And that’s just because that was the nature of load growth being flat. And I don’t know if I’ve used this analogy before on the podcast. I know I’ve used it in different areas, but I go back to the idea of the military base concept, which is that we are always going to have concentrated large campuses for AI training. Those are going to be your gigawatt plus campuses. Those are going to exist.
I don’t want to send the message that that’s going away. I don’t think that’s going away. Is it going to happen at the scale that we expected? I don’t think so because chips are working themselves out.
And then you have training and then you have your outposts and your outposts can be large or they can be very, very small and strategic. And what Jigar is saying is happening in the field, which is that entities are just desperate to deploy their chips, which they’ve already bought and are depreciated and are on their books and need to generate tokens or else they’re going to go under. That is a major motivational factor. The other motivational factor is that we’re starting to work out demand for AI as a product and you can’t actually work out the value of that until you are able to integrate it at the cloud level. And integrating it at the cloud level requires proximity and that proximity requirement requires flexibility about where you put it. So for the large hyperscalers, the value of AI was always going to be who can build the best cloud product.
But at the end of the day, cloud is geographically important in proximity to its end use. And so that transition, I think this transition was inevitable. What is interesting, like when I was at Google, I mean we always knew that we would switch back and inference would look much more like the early days of cloud where we did have 20, 50, 100, nothing more than a 500 megawatt data center. What I think is so interesting about what’s going on now is that the technology capabilities, and I think we’ll get into this in a second, that allow for even more micro compute is really changing the way that the hyperscalers thought about scale and mass procurement. Because originally I think we thought what does procurement at scale look like? It looks like big chunks of concentrated power. And now they’re being challenged and saying, “Well, potentially procurement at scale could look like a thousand distributed chunks of power.” And that is something that is newer and I think has a lot of potential.
The devil, of course, is going to be in the third party developer and whether some of these development arms are able to manage at that portfolio level and if they are and if the analytics and the grid interoperability and everything is there, great, but there are a lot of holes that have to be or a lot of hoops that have to be jumped through to be able to manage what is an incredibly important and critical asset across thousands and thousands of data nodes, right? So that’s the big question and sort of why I see the shift in the way it is, but I’m for it. I’m like, I’m 100% for it.
Jigar Shah: Well, I mean anyone who knows anything about the physics of the grid is for it.
Caroline Golin: Right. That’s the other thing, right? And this is one thing Jigar and I have always agreed on, which is that the quickest and best way to do this is to figure out how to actually improve the distribution system because the transmission system is clunky and in reality built to be a one-way power flow. It is built that way. But the distribution system has the ability to advance far quicker than transmission does. What I’m more interested in is like which comes first do the entities like Tapestry and whatnot come first to open up this or does this force the improved analytics, improved interoperability, improved load plan? One is going to come before the other. So one’s either going to break the system so the system improves or it’s going to put so much pressure where we just finally start doing the distribution planning that we should have been doing forever.
And you can see like what’s happening in the UK as up on the hills in terms of the onslaught, but like you can see how that regulatory paradigm, how those changes in data analytics allowed for a much more flexible distribution system.
Jigar Shah: That’s why Caroline was actually in Scotland.
Caroline Golin: That’s exactly why.
Jigar Shah: She was siting data centers where the offshore wind farms were coming in. There’s all this extra power there. She was like, “This would be a good place for a data center.”
Caroline Golin: It’s where I rode my horse. I rode my horse to check out more data center sightings. Exactly. No, I mean, but it’s true.
Stephen Lacey: I guess what threw me off about your argument, Jigar, is that it seemed like you were implying that the age of the gigawatt scale data center is coming to an end.
Jigar Shah: Oh, I am more than applying that. Yeah.
Caroline Golin: But I think that’s where we differ.
Stephen Lacey: Yeah, and I don’t agree.
Jigar Shah: No one has to agree with me right now. I mean, the thing that’s interesting- Or ever. I mean, all roads lead to my predictions at some point, but I’m just saying that right now they don’t have to agree with me because the contracts are getting signed. It’s not like they have the choice. The only choice they have if they want near term inference is stuff that can be built in months, which are the smaller formats. And so now the question becomes, what do the investors do? And I’ve talked to them a lot and they are genuinely worried because what’s happening is that when you look at these one gigawatt data centers, I mean, you can imagine that a number of these one gigawatt data center
build outs that have already hit final investment decision is a very large number. It’s like 30 to 40 gigawatts of one gigawatt data centers, right? So now when Mr. Wonderful goes on TV and says, “I feel wronged for my nine gigawatt data center in Utah,” everyone is saying, “I don’t think we need Mr. Wonderful’s nine gigawatt off grid data center.” Because to Caroline’s point, it was so hard to begin with to justify that gigawatt data center and then you were going to build it off grid and then he’s like, “Well, we’re never going to connect it on grid.” And then you’re like, “Wait a second, are these people really that dumb?” And so now you’re starting to see a lot of these frothy one gigawatt data centers starting to lose all traction. All of their investors are pulling out and saying, “You don’t have a hyperscaler that really wants your stuff.” And that’s why Fermi ended up going under.
Caroline Golin: Okay, that is not the reason why Fermi went under.
Jigar Shah: No, but I want to make sure that we’re clear about what Fermi could have done, right? They raised $700 million in the IPO. They had proprietary access to natural gas turbines, but not enough. They could have just built a 500 megawatt data center right away and done exactly what Elon Musk had done at Colossus. They could have said, “Let’s just do a 300 megawatt data center, let’s do a 500 megawatt data center. Let’s just force it. ” And they had 200 megawatts already of grid capacity, but they were too chicken. And so they were like, “We need to get project finance, so we need to get off takers. And to get off takers, we need to do this.” And they never got that stuff done.
And so part of what makes Elon Elon is that he said, “Screw it. I’m not going to do any of that stuff. I’m just going to go and do this merchant” and now he’s got 1.5 billion a month from Anthropic. I just think that the vast majority of these one gigawatt plus data centers are looking for a project finance structure and that project finance structure is proving harder to put together than they thought.
Caroline Golin: I think there’s another layer here, which we don’t talk about a lot, but at the end of the day, when we talk about AI and AI training, it’s a GPU and a TPU world and everyone was building infrastructure to integrate chips divorced from the trajectory of chip evolution because they were advantageous, right? And so there will always be a sector of this market that needs a concentrated area to figure out the evolution of chips and training and development. I don’t think the concentrated space is going away. I think it’s going to get smarter. I hope, and part of what I hope to do with NRG is I hope it gets integrated as an asset to the grid as opposed to just this suck from the grid, but I don’t think that’s going away because I don’t think anyone is going to continually blindly test out training in this distributed manner.
Jigar Shah: But we’re not arguing that, Caroline. Not a single person is arguing that. What everyone is arguing is that we have 30 gigawatts of one gigawatt plus data centers already under construction. They might be behind schedule and when you talk to Epoch AI research, they’re saying that even in 2030 Anthropic is saying its largest, large language model only needs four gigawatts for training. So they’re saying-
Caroline Golin: But that’s because of chip evolution is what I’m saying.
Jigar Shah: I think we’re saying the same thing. The question then becomes, does someone do the 52nd gigawatt of one gigawatt plus data centers and the 64th gigawatt of one gigawatt plus data centers because that’s what’s getting funded right now by land developers called Mr. Wonderful and they’re the ones pissing everybody off at the local level and they don’t care about water and they don’t care about this stuff because they’re just trying to flip these land positions that they have and make way more money. And what I’m hearing from all the hyperscalers is they’re ready to tell all those guys to stop pissing everybody off. We’re not buying your one gigawatt computer anymore. We’re just finishing up the 30 plus gigawatts that we’ve already committed to.
Caroline Golin: I don’t think the shift is completely in the other direction. I don’t think the pendulum has completely swung. I think most of the hyperscalers are looking at the fact that they need options across all of the infrastructure. What I will tell you, what has shifted is that a year ago it was heads down, concentrate compute, build as much as you can in a few choice areas because that was the economies of scale and question that we were all trying to figure out. Now it’s broader than that. And so I think that the bubble, if there is a bubble to Jigar’s point, is around the idea that everything’s going to be concentrated in these gigawatt plus data centers. I think that bubble was all, I think anyone who was actually working in this industry knew that that was always a bit of a bubble. You just have to follow where the revenue stream was going to come from. Google and Nvidia was never going to make revenue just by training. They have to make revenue in cloud.
Jigar Shah: Well, you say that as if it’s obvious, but I’ve been saying this very loudly from the Open Circuit podcast platform and got a lot of hate mail for saying it, even though it’s pretty obvious over here that 90% of all compute- I say it
Caroline Golin: I say it with such a more calm tenor- With grace. And I don’t call people idiots.
Stephen Lacey: I mean, I think we also have a ways to go before we see this market where inference represents 80% of compute, right? It’s five to seven years away.
Jigar Shah: No, I mean epoch AI research is saying 90%. No, they’re saying 90% of all compute in 2030 will be inference. That is like what epoch AI research is saying. And I mean, they’re saying that even for training, they’re saying only the final training runs have to be done in one gigawatt plus data centers, that when they do their R&D training runs, they can do them in batches in 100 megawatt and 200 megawatt data.
Caroline Golin: But that is for existing chip schedules and what I’m saying is that- Well, they’re saying in 2030- Right, which is if your chip’s coming out right now in 2026, you have a three at most four year run rate. So what I’m saying is that that run rate will continue to evolve. It will continue. I think if everyone thinks that Google and NVIDIA are like, “Yeah, we’re good. Our chips are good.”
Jigar Shah: No one’s saying that. And we have a big IPO this last week that proved otherwise, right? Not Cerebras. It’s like Cerebras. I don’t know. How do you say it?
Caroline Golin: That sounds Scottish.
Jigar Shah: Anyway, but whatever it is, but they had a huge IPO, and they do the wafer style chips and 900 times better on the tokens than traditional chip architecture. So I think we’re getting better. I just think that what governors want to know is that we have 30 plus gigawatts worth of data centers already under construction. They’ve already hit final investment decision. Some of them are like four months delayed, eight months delayed, whatever, but they’re already under construction and we’ll be online by 2030, right? Epoch AI research is saying that the largest large language models are going to need four gigawatts of training for 2030. Now after 2030, they might need more, right? So the question is, how many more crazy disruptive one gigawatt data centers do we want to green light this week or do we want to see how this stuff plays out and really focus on the distributed data centers until we figure out how this stuff plays out?
Stephen Lacey: Well, let’s talk about the distributed data centers. There’s a lot of ideas out there about what the nodes will look like, how you build out this network and we have just a few minutes left so I want to turn this into a segment that we are calling short circuit. We’ve got a few examples here.
Caroline Golin: That’s usually a negative thing. Yeah. I was going to say, and none of these technologies were going to short circuit the grid.
Jigar Shah: I want to call it electrocution.
Caroline Golin: Trial by fire. Yeah. Okay.
Stephen Lacey: You got a better name?
Jigar Shah: Lightning round.
Stephen Lacey: Oh, everybody uses lightning round.
Jigar Shah: Okay, got it.
Stephen Lacey: Here we go. Short circuit.
Jigar Shah: Mars shot.
Stephen Lacey: Okay. The first one I want to talk about, which Jigar mentioned was Span’s XFRA Node. This is basically Span’s plan to turn your home into a data center. It was announced at our transition AI conference in April. Span is a smart panel company piloting this thing called XFRA Node, which is this liquid cooled inference GPU module that gets it installed on the exterior of a newly built home. They charge 150 bucks a month for electricity and internet and then they own the hardware and sell the compute to hyperscalers. Is this a compelling concept? Does it seem viable? Caroline, what do you think?
Caroline Golin: Yeah. I mean, I’m a big, big fan of this concept. Where I am personally and professionally most interested is generally evolution of the concept of the smart home, what that means. I think the vast majority of customers, residents, boaters, however you want to denote them in this country do not want to think about their electricity use. In the same way, I don’t want to think about my wifi server. I don’t want to think about any of that stuff. I still don’t even know how to turn my TV on properly because it’s so complicated and it frustrates me that I have to think about which remote to use and which button to press. And so I think the vast majority of households don’t want to think. I think there are some cool techie 2% that are interested in how time of use works interested in sort of solar and storage and all those things.
And so this becomes a very interesting partnership with the general smart home development and where smart home’s going to go. I don’t think this is something new, but Amazon, Meta, Google, a huge part of the growth in AI is also being able to be integral into the way a home operates. So you can imagine being able to pre-program your home from your Gmail or set through Amazon how your home is going to operate for the next month and allowing that bot to be able to do that. If that takes off in a way, this becomes an absolute no-brainer because then each home is its own inference, right and even broader than that. The question for me is, would anyone adopt this if they weren’t already adopting the smart home concept? And that’s where an integration across those two is incredibly interesting and I’m 100% for it.
I just think that there is a customer acquisition model that has to be thought through and there’s going to be a lot of competition in that space over the next couple of years.
Stephen Lacey: Jigar, what do you think?
Jigar Shah: So right now in the early days, they’re able to pay the homeowner $2,000 a month in rent to put this thing at their site, right? Which is very different. There’s a monetary, there’s a monetary reason to do this and they’ll pay their electricity bill and their internet bill for them, right? So there’s a value there. The other piece I’d say, which I didn’t fully appreciate until I got deeper into it, is that we’re just moving away from Claude and ChatGPT and there’s just so many much smaller LLMs that are focused on the medical industry or the dental industry or this industry or that industry. And they’re really easily hosted at these data centers and so these small ones. And so you think about the four gigawatt like Claude LLM in 2030 and then how much memory you’re going to need to be able to like host it and all that stuff, but there’s so many other LLMs that like actually don’t.
Caroline Golin: Yeah. Yeah. They’re more tailored and that tailored approach lends itself to a micro distributor approach. Yeah, I agree.
Stephen Lacey: Yeah. I mean, it’s a super interesting economic case. I think Span’s pitch is that a hundred megawatt data center takes a few years to build, costs $15 million per megawatt and it spans as it can deliver compute across 8,000 homes in six months at $3 million per megawatt. So we’ll see if they can do it. The one thing that stood out to me was like theft, like putting these valuable GPUs on the outside of your house, it seems like it’s ripe for theft.
Jigar Shah: They only live in gated communities.
Caroline Golin: No, I mean, it’s the same customer acquisition as I go back, the issue that you had with the smart home. Jigar’s completely right. Getting paid to do this as opposed to saying it pays for itself in a couple years and then you save 400 bucks a month or a year is very different. But ownership over this long term is a very interesting question because you can imagine, and this is where I think Jigar and I have always agreed, is like the future grid is far more micro, is far more distributed. And with Angentic buying and with the ability to have real time data and treat electrons as they should be physically treated in real time, the whole system changes. And what is interesting to me about this as well as the other technologies is this provides a level of prosumer engagement that we’ve never seen before.
And the question is, what happens to the role of the utility? What happens to the role of the electricity provider? If all of a sudden the customer, the residential customer becomes a net asset to the grid and a net asset to economic development, that’s never happened before. We’ve never seen that happen before. It completely changes the equation and flips it on its head. I’m for it. I don’t think I’m so concerned about things like theft. The bigger thing that I would be concerned about is at the end is insurance. How are you insuring this against residential? I don’t know the answer to that. Maybe some one of our listeners do, but one of the more interesting things for me is like, and I’ve been saying this for a long time, is that if you want to know where the future of data center infrastructure is going, you need to look at forward curves on the insurance market.
Oh, we don’t have forward curves on the insurance market, so no one can really know where this is going, but I’d be interested in the way that insurance sort of appraises this and where that happens.
Stephen Lacey: Well, this was supposed to be a lightning round now Caroline’s talking about forward curves on the insurance market.
Caroline Golin: Personally would have rather talked about this than NextEra buying Dominion. It’s far more interesting to me. Sorry.
Stephen Lacey: I’ll do better next time. I’m sorry.
Caroline Golin: It’s okay. I’m tired.
Stephen Lacey: What do you think of this startup in the UK called Heata? They’re installing a small server in your home that processes certain workloads and then reroutes the waste heat directly into a hot water cylinder, free hot water in exchange for hosting the hardware.
Caroline Golin: No, no, no. No.
Jigar Shah: I’m just saying, pay me $2,000 a month. I don’t want your free heat. I’m just saying like all this like co-jet, I’ll just give you free heating and cooling in exchange for making $50,000 a month off of you. No.
Caroline Golin: I think that the waste heat recycling is probably more a product of the policies and data center requirements that were passed at the EU level that the UK bought into. So my guess is the reason why they started with that is because there was a policy requirement to do that and that was the box they checked.
Stephen Lacey: All right. The third one, Panthlassa, this is a Peter Thiel-backed company. They’re building an 85 meter steel structure that will get towed out into the deep ocean, flipped vertical and it will bob with the swell and there’s compute sealed inside. It’s cooled by seawater. It’s connected to the world via Starlink and this is for like long running inference and reinforcement learning stuff that doesn’t need low latency. This has been a tough space. Marine energy is like a notoriously very difficult space. Can they get the economics right? Jigar, is this a serious contender or science project?
Jigar Shah: Short the stock.
Caroline Golin: I’m so glad you asked Jigar. I have no idea. I mean, I looked at this and I was like-
Jigar Shah: You are not putting your own 401k into this.
Caroline Golin: I am not putting my own 401k into this.
Jigar Shah: No, it is crazy town. First we go to space and then when space didn’t work out then we’re going to the bottom of the ocean.
Stephen Lacey: Well, you’ve been doing this since 2016. They’ve been working on this.
Jigar Shah: I mean, people have been working on all sorts of crazy ideas since 2016.
Caroline Golin: And we still don’t have a lot of floating solar plants either.
Jigar Shah: We still don’t have a lot of space solar that’s like being beamed to earth. I still don’t have like –
Caroline Golin: I think space solar still happens. I mean, I’m not sure it happens at this scale, but I will put my money on space solar before we’re-
Jigar Shah: I think we just tested it. I think we just tested it. I think both China and Japan now have working space solar units for like whatever it is, like $50 a kilowatt hour, but sure.
Stephen Lacey: Sure. Well, if you want to dig deeper into the concept because we don’t have a lot of time to talk about it here, watch the recent episode of Catalyst with Shayle Kann. He talks with the co-founder and CEO of this company and it’s really fascinating. It’s the first concept in the marine environment and marine energy that actually made me prick up my ears and think that they’re being very thoughtful about it and about their design and the limitations of other marine energy devices.
So we shall see and if any of these become the next big solution or the next idea to fizzle, you’ll hear about it right here on Open Circuit. Thanks everyone for being here. Caroline, thanks for joining us, hopping off a plane and coming right here. Good to see you.
Caroline Golin: As always.
Stephen Lacey: Jigar, great to see you too. Are you traveling this week? Are you staying put?
Jigar Shah: I’m staying put. I am just trying to figure out how we actually get my fourth grader graduated on Friday. Oh,
Stephen Lacey: Nice.
Jigar Shah: It’s like a whole production and so he’s going to middle school. It starts in fifth grade now, which I think-
Caroline Golin: What? Why?
Stephen Lacey: Yeah.
Jigar Shah: That’s the way they do it confusing. I mean, no one asks me for my advice on schooling.
Caroline Golin: DC. Y’all always have to do things differently.
Jigar Shah: I know. I know.
Stephen Lacey: Thanks everyone for being here. Hit the subscribe button if you don’t already subscribe to the show on YouTube. Open Circuit is produced by Latitude Media. The show is edited by me, Sean Marquand and Anne Bailey. You can find the audio version of this, of course, wherever you get your podcast and find us on the Latitude Media YouTube page to watch the videos. We will be with you next week. We really appreciate you being here every week with us and we’ll see you next time.
