How The Massive Power Draw Of Generative AI Is Overtaxing Our Grid

Demand has never been higher for these racks and racks of powerful servers, feeding the internet's insatiable appetite for computing in the cloud. The reality is that the cloud is not up there somewhere. It's right here.

We are in it. You're in the middle of the cloud as we speak. And data centers like this can't ever stop streaming social media, photo storage, and more recently, and requiring much more data training and running chatbots like OpenAI's ChatGPT, Google's Gemini, and Microsoft's Copilot.

And you can feel the heat coming off of these. Thanks to the generative AI race, data centers like this are springing up as quickly as companies like vantage can build them, and that means demand for power to run them and cool them is through the roof, too. If we don't start thinking about this power problem differently now, we're never going to see this dream we have or the potential we have of this amazing technology that can truly change our lives.

One ChatGPT query takes nearly ten times as much energy as a typical Google search, and as much energy as keeping a five watt LED bulb on for an hour. Generating an AI image can use as much power as charging your smartphone. Hyperscalers building data centers to accommodate AI have seen emissions skyrocket, and this problem isn't new.

Estimates way back in 2019 found training one large language model produced as much CO2 as the entire lifetime of five gas powered cars. And even if we can generate enough power, our aging grid is increasingly unable to handle the load. If you look at the peak demand during the summer time, if the data centers don't reduce their load, there could be a blackout.

With the looming question of whether we'll have enough power for the widespread adoption of generative AI, Cnbc visited a data center in Silicon Valley to see those massive compute loads firsthand, and talk to those at the center of the problem to find out what can be done. There are more than 8000 data centers globally with the highest concentration in the US. But it's not enough.

But we suspect that the amount of demand that we'll see from AI specific applications will be as much or more than we've seen historically from cloud computing. The AI frenzy has data center demand rising 15 to 20% every year through 2030. And as companies like Vantage build more and more, getting enough power is key.

Data centers could reach a whopping 16% of total US power consumption by 2030, according to one report, up from just 2. 5% before ChatGPT hit the scene in 2022. That's the equivalent of about two thirds of the total homes in the US.

Natural gas is expected to fuel the majority of this, and utilities will need to invest some 50 billion to support the growth. Data centers that probably around 64MW for the building itself. Many of those are being taken up by single customers.

They'll have the entirety of the space leased to them. And as we think about AI applications, those numbers can grow quite significantly beyond that into hundreds of megawatts. 64mw, 100MW.

How many homes, you know, on average. Tens and tens of thousands of homes per data centers worth of power. Many big tech companies contract with companies like Vantage to house their servers, but the needs of some have grown so much that many have been building their own data centers.

For Google and Microsoft, this has directly translated to soaring emissions. Google's latest environmental report says greenhouse gas emissions rose nearly 50% from 2019 to 2023, in part because of data center energy consumption, although it adds its data centers are 1. 8 times as energy efficient as a typical data center.

Microsoft's emissions rose nearly 30% from 2020 to 2024, due to data centers designed and optimized to support AI workloads. Power needs are so high that some plans to close coal fired power plants are being put on hold, like in Kansas City, where Meta is building an AI focused data center. So you can never really stop the AI computing from growing.

So they will do whatever they can to get the power capacity. One approach: building data centers where power is more plentiful. The industry itself is looking for places where there is either proximate access to renewables whether wind or solar and other infrastructure that that can be leveraged, whether it be part of an incentive program to convert what would have been a coal fired plant into natural gas, or increasingly looking at ways in which to offtake power from nuclear facilities.

Santa Clara, where we visited Vantage, has long been one of the nation's hot spots for clusters of data centers near data hungry clients. Nvidia's headquarters was visible from the roof. Now, Vantage is building in places like Columbus, Ohio, and Atlanta, Georgia.

In Northern California, we're seeing a bit of a slowdown in terms of where data centers are deployed because of the lack of availability of power from the utilities here in this area. On the flip side of bringing data centers where the power is, some AI companies and data centers are experimenting with ways to generate their own power right on site. OpenAI CEO Sam Altman has been vocal about this need.

He recently invested in a solar startup that makes shipping container size modules that include both the panels and power storage in one. Altman's also invested in nuclear fission startup Oklo that aims to make mini nuclear reactors housed in a frame structures and in nuclear fusion startup Helion. Microsoft also signed a deal with Helion last year to start buying its fusion electricity in 2028, and Google's partnered with a geothermal startup that says its next plant will harness enough power from deep underground to run a large data center.

Even data centers themselves are starting to generate their own power. Which Vantage has done, for example, in Virginia, where over the course of the last year, we deployed 100 megawatt natural gas power plant to support a dedicated data center for one of our customers. There, we're self contained and we're delivering power to the data center that we have there, and it doesn't touch the public grid at all.

Even when enough power can be generated, the aging grid is often ill equipped to handle transmitting it. And that's where grid hardening comes in. With concentrations of data centers in particular areas, there tends to put somewhat more pressure on the grid in terms of delivering that much power in those locations.

In an area of northern Virginia known as data center alley servers process an estimated 70% of the world's internet traffic each day. At one point in 2022, the power company there had to pause new data center connections as it struggled to keep up with demand. Basically, during the peak hour, we either ask the the residents to turn off their AC, or we ask the AI company to stop their training.

Vantage, as an example, voluntarily supports a load shedding program such that when the utility knows that they're going to have a constraint based on high temperatures, people need to run their air conditioners. We voluntarily come off the grid. We run our own generators during that time in order to ensure that that power is available for everybody.

The bottleneck often occurs in getting power from the site where it's generated to where it's consumed. One solution is to add hundreds or even thousands of miles of new transmission lines. But those projects, like 15.

$2 billion effort to expand lines to Data Center Alley, have been met with opposition from local ratepayers who don't want to see their bills go up to fund the project. It is a part of the solution to increase the capacity from the grid side, but that's very costly and very time consuming, and sometimes the cost is just passed down to residents in there in term of their utility bill increase. Another solution is to use predictive software to reduce failures at one of the grid's weakest points the transformer.

All electricity generated must go through a transformer, and that's primarily its function connects two sides of electrical circuits together. There are 60 to 80 million transformers in the US alone. The average transformer in the US is 38 years old, so the aging devices have become a common cause for power outages, and replacing them is expensive and slow.

So we have this tiny little sensor. It's two inch diameter, one inch thickness, about half the size of a hockey puck, and we essentially glue it on the outside of a transformer. Software can then predict failures and determine which transformer can handle more load, so it can be shifted away from those at risk of failure.

It's been installed in high power demand areas like part of Mongolia where data centers are coming online. Vi says business has tripled since ChatGPT came on the scene in 2022. And what we're hearing in 2024 that we might double or triple again next year.

A large reason we need more power and a more reliable grid for generative AI is to keep its servers cool. All the servers generate an immense amount of hot air, and cooling them down with air or water keeps them from overheating so they can keep running 24 over seven. The problem is, AI is projected to withdraw more water annually by 2027 than four times all of Denmark.

Everybody is worried about AI being energy intensive. We can solve that when we get off our ass and stop being such idiots about nuclear. Right.

That's solvable. Water is the the fundamental limiting factor to what is coming in terms of AI. I was quite shocked when I saw the number of AI computing's water needs.

Shaolei Ren's been studying data center efficiency for over a decade. His research team found that every 10 to 50 ChatGPT prompts can burn through what you'd find in a standard 16 ounce water bottle. Training takes even more power and generates even more heat as it learns by accessing all the data constantly being added to the entire internet.

Training GPT three in Microsoft's US data centers can directly evaporate 700,000l of clean, fresh water. With global AI demand accountable for up to six point 6,000,000,000m³ of water withdrawal by 2027. More than four times the total annual withdrawal of all of Denmark in drought ridden Chile, the government partially reversed Google's permit to build a data center there following public outcry about water usage, and it's facing similar backlash to planned data centers in Uruguay, too.

There are technologies that are used in certain parts of the industry that do consume water in order to cool data centers through evaporative cooling. That tends to be very efficient from a power perspective, it tends to be very inefficient from a water use perspective. And so vantage is designed from the beginning, really has been to avoid using water at all costs.

Instead, Vantage uses dozens of gigantic air conditioning units on the roof. There's a combination of air handling and cooling towers at the top there, where the condenser coils allow heat. That's coming up from the hot aisles that we were looking at inside the data hall, and then to be converted back into the chilled water loop and brought back down into those critical air handlers.

Microsoft has halted a radical project that tried keeping its servers cool by submerging them under the ocean. Another solution is using cool liquid directly on chips to cool them rather than the inefficient process of cooling air with water. People are working on a whole bunch of things like director chip cooling, for example, which would radically reduce the amount of water that's needed.

But the speed at which this is building makes it very challenging indeed. So there's a lot of talk in the industry right now in support of AI. Moving.

From air, which is moving around us to water, air or liquid that can be brought to the chip itself in order to cool it in a much more efficient way. But for a lot of data centers, that requires an enormous amount of retrofit in our case, advantage. About six years ago, we deployed a design that would allow for us to tap into that cold water loop here on the data hall floor.

There is one broader approach to solving. AI's massive problem of water and power lessen the amount of compute it needs. In other words, get more work per watt.

It all comes back to how do I get more done with the power supply I have? Scale flux makes memory and storage devices for data centers focused on power savings. Data compression can be very slow latency adding and power hungry if you try to do it.

In general purpose cores like Intel or AMD x86 processors, when you put it into a hardware state machine, it can be done orders of magnitude faster and more efficiently. More power efficient. The key alternative to those power hungry x86 cores are ARM based specialized processors.

Arm got its start making low powered chips that maximized the battery life of early mobile phones. Saving every last bit of power. It's going to be a fundamentally different design than when you're trying to maximize the performance.

And so it could be some something as simple as how we access our memory. It can be simple as how we access data, but it every moment of that architecture is thought of as power first. Now ARM makes all sorts of chips, including neoverse for data centers.

And as AI takes off, ARM's power efficiency has made it increasingly popular with tech giants like Google, Microsoft, Oracle and Amazon. I have here AWS graviton and AWS graviton saves 60% power versus competitive architectures. Nvidia's latest AI chip unveiled in March, Grace Blackwell, uses ARM based CPUs that it says can run gen AI models on 25 times less power.

And ARM says a data center filled with its chips can use 15% less power because of lower compute needs. If you think about the scale of data centers, 15% is almost 2 billion ChatGPT queries. Think about that.

You can light 20% of American households with just that 15% savings. And companies like Apple, Samsung and Qualcomm have been touting the benefits of doing AI on device. A huge energy savings for each query kept off the cloud off of servers like these, perhaps giving data centers time to build more and catch up to AI's insatiable appetite for power.

Everybody will build the data centers that they can, and we'll have as much AI as those data centers will support, and it may be less than what people aspire to. But ultimately, there's a lot of people working on finding ways to throttle some of those supply constraints, but definitely a lot of growth ahead in this industry.