Data centers are an energy efficiency success story. Even as internet traffic increased exponentially over the last quarter century, the energy demand from warehouse-scale computing remained relatively flat.
But data center efficiency in the age of AI requires reimagining the way facilities are designed and operated, according to Kevin Miller, Amazon Web Services vice president of global data centers.
This week, at its re:Invent customer event, AWS said it is now implementing improvements in data centers designed “to support the next generation of artificial intelligence innovation.” The new components are designed to make data centers both more resilient and more efficient, and they are the product of a “bit of a rethinking of the way data centers are designed,” Miller told Latitude Media. AWS has been working on the improvements for the past couple of years.
“Traditionally, you might think of a data center as a box with a number of highly featured systems, like an uninterruptible power supply system to provide redundant power,” he said. “A UPS system, historically, has been a combination of batteries, electric switch gears, and control logic. And we’ve found a lot of benefit in pulling those different pieces apart and thinking about the data center holistically, as not just a box with some highly featured components in it, but… as a whole.”
Among the changes is a simplification of electrical and mechanical systems, such as by bringing the backup power closer to the rack and reducing the number of fans to exhaust hot air. They also include innovations in cooling, new positioning of racks to reduce stranded power, and the rollout of an Amazon-owned control system, which has reduced the troubleshooting of some failures from hours to seconds.
After chips, cooling is the most energy-intensive part of data center operations and one that has been a “huge focus” for the AWS team, according to Miller. To efficiently train generative AI models, it’s important to have as many GPUs as possible in the same small space, to reduce latency. The high density has required a transition from traditional air cooling to direct-to-chip cooling, or liquid cooling.
“We also use computational fluid dynamics to understand where the air from our fans is flowing,” Miller said. “Fans take energy to run — every time I have a fan, where is that cool air going? Can I be more efficient in where I’m pushing that air to avoid the need to run those fans more?”
These cooling innovations, paired with a more efficient control system, are “expected to reduce mechanical energy consumption by up to 46% compared to its previous design during peak cooling conditions, without increasing water usage on a per-megawatt basis,” according to the AWS announcement.
Gains in energy efficiency for AI data centers are becoming increasingly important, as AI contributes to soaring U.S. electricity demand and creates capacity constraints across the grid. But, according to Miller, efficiency gains can still be realized at the facility level level, which can vary from data center to data center.
“All cloud providers are not alike, and all data centers are not alike,” he said. “I would emphasize that it’s important in these conversations to talk not in general terms about data centers, but about the innovations and improvements we’re making, which are differentiated.”
At the event, AWS also showed off its new chip for data centers, the Trainium2, which could reduce its reliance on NVIDIA’s chips. Together with Anthropic, AWS is building a cluster of servers, containing hundreds of thousands of Trainium2 chips.
AWS also announced a partnership with Orbital Materials, a startup that uses AI models for material research — including materials for data center-integrated carbon removal and cooling technology. The partnership is “promising” and “exciting,” Miller said, but it’s still at the evaluation stage, and the company doesn’t currently “have a specific plan to deploy that technology.”
