How Data Centers Actually Work

0:00Every company says AI will make employees more productive, but most employees are still stuck waiting on IT. Waiting for app access, waiting for password resets, waiting for someone to fix a laptop issue so they can get back to work. That operational drag adds up fast, and IT teams are overwhelmed trying to keep up. Servo was built to automate that work. You describe what you want automated in plain English, and Servo builds it for you. No complicated workflow builders, no consultants, just faster support and fewer tickets slowing everyone down. Serval enables IT teams to build automations using plain English instead of drag-and-drop workflow builders.

0:34The platform is designed to eliminate repetitive tickets so IT can focus on strategic work instead of constant firefighting. Serval positions IT as the AI-powered operational backbone of the company, not just a support function. The company guarantees customers can automate 50 % of IT tickets and backs it up with a free 4-week pilot. Learn more or start a free 4-week pilot at serval.com slash uncanny. That's S-E-R-V-A-L dot com slash uncanny. Serval dot com slash uncanny. NBR believes in your right to be curious about the big questions and the unexpected ones. That's what Shortwave, their science podcast, is all about.

1:15When we stop to ask questions, the world gets more interesting. Shortwave spends each episode answering one big one. Like, why do we have nightmares? How does AI affect my energy bill? And how can trash be good for the environment? Science isn't just for the PhDs of the world. It's for everyone, including those of us who didn't realize we love science. In under 15 minutes, Shortwave follows a question wherever it leads and brings you along for the ride. It's science, but not the kind you remember from school. It's surprising, playful, and full of wonder. News podcasts are great, but sometimes you want to experience the world in a different way.

1:54Shortwave gets you out of your head and in touch with the world around you. Personally, I loved hearing about Janet Jones, horse trainer and neuroscientist, teach me about horse and human relationships throughout history. Follow NPR's Shortwave podcast and exercise your right to wonder. Hey Lauren, how you doing? I'm great. It's so nice to be back in studio with you again because our schedules were not aligning for the past few weeks. Nope. But, you know, the stars and the moon have aligned now and here we are once again. Here we are. And I'm sure all of our listeners have just been sitting here wondering, when are Lauren and Mike getting back together?

2:30When is the band getting back together? And I just went to yet another AI dinner last night. Everyone's talking about AI. Everyone is talking about whether or not we're in an AI bubble. And I think a lot of this is fueled by the news that we've been seeing trickling out about all these AI infrastructure projects. Yes, data centers. Data centers. Big warehouses stuffed with computers. Stuffed with server racks. And for sure, we're going to be touching on all of that on today's show. But before we get into it, we have to welcome our guest, Wired's senior writer and climate energy expert, Molly Taft, is joining us today.

3:07Hello, Molly. Hello. Hi, Molly. Is this your inaugural Uncanny Valley podcast? Yeah, and I'm a fan, so this is a big deal for For me, frankly, I'm very excited to be here talking about my favorite topic and the thing people ask me the most these days about at parties. And what is that? Data centers. It's data centers. I thought you were going to say they're electricity bills. Yeah, I mean, they're related, aren't they? People are fired up about it.

3:40This is Wired's Uncanny Valley, a show about the people, power, and influence of Silicon Valley. Today, we're talking about the AI infrastructure boom. In recent years, tech giants like OpenAI, Amazon, Meta, and Microsoft have invested hundreds of billions of dollars in data centers. These are large warehouses full of servers delivering the huge amounts of computing power needed to run AI models. Recent announcements like OpenAI's new Stargate data center in Texas, or the AI startups deal with the chipmaker AMD, have only added to the hype and the capital surrounding the expansion of these data centers.

4:18But just as the deals keep piling up, the issues around this rapid expansion are becoming more worrisome. We'll dive into how these data centers work, what effect they are already having in some communities, and what they reveal about the current state of the AI industry and our economy. I'm Michael Kalori, Director of Consumer Tech and Culture. I'm Lauren Good. I'm a senior correspondent. And I'm Molly Taft, a senior writer covering energy and the environment.

4:52So I think it's safe to say that all of us, including our listeners, have heard of data centers somewhere in the news, and they have a pretty good idea of what a data center is. But let's talk about why AI companies rely on them. So to put it another way, how does a chat GPT query on my computer end up in a data center? This is an excellent question. And I think we're going to nerd out a little bit. Awesome. Okay. Say you type in a chat GPT, you know, that you want a few options for dinner recipes tonight. Or what am I going to get Mike for his birthday next year? Bless you. You send your request through, and then it goes to OpenAI servers.

5:32It goes through a few checkpoints. First, like there's authentication, are you a valid user? Moderation is the prompt okay, or aligns with their guidelines. And then some load balancing, which is which data center should ultimately handle this. The words or the strings of text that you write are broken down into these little chunks of text called tokens. They're kind of like these puzzle pieces that artificial intelligence models can process. And at this point, the request lands on specialized hardware, in most cases, GPUs. Yeah, the next part is key. GPUs, which stands for graphic processing units, are essential to AI systems.

6:12They're electronic cards that are really good at parallel processing, which basically means they can perform many calculations at once. Data centers are full of metallic rows of servers that contain these GPUs. So once you send your query in, it gets processed by those GPUs. You probably heard some of the most famous ones in tech headlines, like NVIDIA's H100s. Ah, the H100. The kids can't get enough of them. My old friend. And then I guess to bring it back full circle to our, you know, query for what to eat for dinner tonight or Mike's birthday present next year. Once that query arrives at the data center and the servers, the AI model starts working.

6:51And this is called the inference time. The model is going to predict what words in the form of tokens should come next, one after another, until it builds out this full answer. And then finally, the response is sent back through the same network path to your browser or app. This is a very summarized version of what happens, but on a high level, that's why your ChatGPT question or query needs to go through to these data centers. And this all happens in a matter of seconds. It's pretty miraculous. Yeah. Yeah. Well, I love the breakdown. It's also just, you know, for the record, I have to say that I probably would never ask Chai GPT for recipe recommendations.

7:35And Lauren, I really hope that you get a good answer as to what to get me from her birthday because I'm already very excited. You might recall that I did this last year for your birthday, and I think I still haven't gotten you anything. Well, you know, I think if there's a next question that everybody has about data centers, it's about how energy intensive they are. because even a simple query like that, I mean, we're talking about accessing, you know, multiple different servers. You're breaking up your query into different tokens, sort of spraying them into this data center. How much impact do those computing requirements have on the environment?

8:09And Molly, I want to ask you this question because I know you've reported on this a lot. What does the energy consumption of a data center look like? And are we talking about just energy? Are we talking about water? What does it look like? So large data centers, there's a lot going on inside of them, right? They need cooling systems. They need to keep the lights on. They need to run their network equipment. That all creates a lot of energy. They don't run at the same energy level all the time. They often run in cycles based on when there's more queries. You know, they'll take a break at night when the queries are lower.

8:43And the environmental footprint of a data center also depends on what type of energy they're hooked up to. If you're hooked up to a dirtier grid, a grid powered by fossil fuels, you're going to create more emissions. A data center that runs on solar power or wind power will have a lower environmental footprint. You know, it's funny trying to calculate the footprint of a specific data center because a lot of the information about the environmental impacts is proprietary information. So a lot of what we're working with is stuff that companies volunteer themselves. You'll see headlines, you know, Meta is building a data center.

9:18they're calling Hyperion in Louisiana. It's part of their bigger buildout. And that's going to be huge. It says it's going to be about five gigawatts, which is like massive. That's about half the peak power load of New York City. That's the biggest one under construction right now, but there's a lot more out there. There's also regional level approximations. And there are some places in the world that are starting to kind of get worried about how much energy data centers are consuming or are set to consume. In Ireland right now, data centers use more than 20 % of the country's electricity, which is crazy.

9:51Wow. Virginia is also facing a huge cliff, and they're projected to use a lot more over the next couple of years. So as a group, there's definitely an energy trend that's going up. I mean, you mentioned that each tech company, you know, data center company is reporting their energy usage. I'm curious how far down the stack they're going. You know, for example, if they are using a certain number of GPUs and those components from GPUs have to be shipped from different parts of the world or manufactured in different parts of the world, that is also an emissions cost. So are any of them incorporating that?

10:27Yeah. I mean, the problem with climate stuff and the problem with emissions stuff is you can keep going down this rabbit hole forever. Kind of what you're talking about gets into like emissions reporting, which is like, what is the total environmental footprint of what you're doing? And this is something that climate folks and people who like to do math on emissions are constantly trying to figure out when it comes to specific companies. And many companies don't necessarily want to go down that chain. But I think when we talk about power use, we are talking about like on-site power use. We're talking about just what it's going to take to turn that thing on and to run that thing.

11:01The total footprint of these things is probably a lot bigger than we think. And when you mention people who like to do math and figure this out, I think of someone like Sasha Luciani, who, you know, she's pretty vocal out there. She's the climate lead at Hugging Face, and she tends to put it eloquently, call bullshit on a lot of the numbers that these tech leaders put out there. Yeah, and Sasha's been really good at calling out, especially when you have models where it's less transparent as to how much energy is actually going into a query. There's numbers that kind of get tossed around. And Sam Altman put out a blog post this summer where he said the average query for ChatGPT uses about what an oven would use in a little over one second or a high efficiency light bulb would use in a couple of minutes.

11:46It's about 0.34 watt hours of energy. But Sasha like really correctly points out that these kinds of figures don't give us a lot to work on. Like when he says an average query, what does that mean? How many of those queries are happening? what are these data centers hooked up to? Are they hooked up to grids that are mostly renewables? Are they hooked up to grids that use coal? Like, there are so many factors that go into the energy use of these products. And Sasha put it really, really well. She said that Altman pulled that figure out of his ass. That's his direct quote. And, you know, she told me, like, it blows my mind that you can buy a car and know how many miles per gallon it consumes, yet we use all these AI tools every day.

12:25We have absolutely no efficiency metrics, emissions factors, nothing. So just one number doesn't help us get to the bigger picture. And a lot of the times it may just kind of help companies procrastinate on putting out more data. So we don't have any clear indicators about how much energy each query is taking and how much energy these data centers are consuming other than it's a lot and it's a problem. And even knowing that it's a problem, a lot of the big tech companies that are investing in AI infrastructure are continuing to build, build, build. So we've talked about some of the big players, and we should make sure that we mention as many of them as we can.

13:05So who are the other big names in this space? Honestly, I think the biggest ones are the folks we've already mentioned, which is OpenAI, AMD, NVIDIA, and this Stargate project. That's kind of an umbrella term for a bunch of different partnerships. Like the Stargate project is a$500 billion, 10 gigawatt commitment between OpenAI, SoftBank, Oracle, and MGX. All of these interconnected relationships between the different hyperscalers and the chip makers. What's interesting is that there has been a little bit of a shift in kind of the terminology or the way that these investments are being described and that they're being described as gigawatt investments.

13:47So they're kind of staggered investments in many cases. they're kind of based on the assumption that the demand for AI will continue to scale upwards and that eventually more and more and more energy or compute power, I should say, is going to be needed. It's not, you know, some kind of clear cut, well, in exchange for this amount of investment, this company is going to get X million number of GPUs. It's not basic unit reporting. Okay. So there's one thing that you said there that I want to hear more about, and that's hyperscaling. because it sounds like what we're already talking about is hyperscaling, but you're saying that there are companies that are interested in growing even more than what we've already talked about.

14:28Well, they're all interested in growing more. Who among us, Mike? But the hyperscalers refers to this class of major tech companies or cloud service providers. So Meta, Amazon, Microsoft, Google, they're all in that category. Yeah, and I think it's important to remember that these companies have so much money. And they have an ability to raise capital like nobody's business. So they're able to do some really crazy stuff to build quick and to build out really, really big. And they're getting pretty creative because their goals right now are to build these things quickly and get them up and running so they can basically use this physical infrastructure to compete with each other.

15:12I think that's right, Molly. I think there's a lot of frenemy building happening right now, and I would just love to be a part of their group chats when all of these announcements are being made. Yeah. And speaking of frenemies, the other sphere of influence that these companies are operating in is the political sphere. Obviously, in order to build a giant data center somewhere, you need to have the political will to do it, which means you need buy-in from the local residents, the local government, the state, the country. So what's happening in the political sphere with folks who want to build more data centers and people who oppose it, regulation?

15:52How is that playing out? That's a great question. And I think if you look at the national conversation, it's quite different than what's happening on the local level. So, you know, you have Washington. You obviously have an administration that is very friendly to the idea of an American AI empire. Importantly for the energy conversation, the way that the Trump administration has approached this support has been through support of fossil fuels. They would really like for all data centers to be powered with oil and gas, a little bit of nuclear and coal. And this works out great for those industries as well, right?

16:30If you're going to have this massive expansion of power demand, it's really cool to be in the middle of that and be the one that everyone wants to turn to for energy resourcing. And then on the other side, there has been this influx of local opposition to these data centers for a variety of reasons, be it the water use, be it fears about rising electricity rates, be it noise. And some of the really big struggles have kind of catapulted this issue to national conversation. I'm thinking about XAI in Memphis when Elon Musk wanted to get XAI up and running. He installed a bunch of unpermitted gas turbines in order to get XAI working that he installed in a majority black community in Memphis that already had severe issues with air pollution and asthma.

17:22And those folks made themselves known. Earlier this year, there was an attempt in D.C. to impose a moratorium on any state regulation around AI at all. It was an incredibly broad inclusion in the big, beautiful bill that ultimately didn't succeed. But one of the people who opposed it publicly was Marjorie Taylor Greene, who actually mentioned data centers in her opposition. And she compared AI to Skynet, the fictional AI from the Terminator movie franchise. So this is getting some strange bedfellows, you know, in league with each other. I think this kind of contrast between what the administration is trying to push forward and some very powerful energy companies that stand to gain from it versus like some truly grassroots local movements and people concerned about the impacts of what these things are going to do in their communities.

18:21We're going to take a quick break right now, but when we come back, we're going to dive into why AI companies' aggressive bet on scaling might come back to bite them. So stay with us.

18:35This show is supported by OutShift, Cisco's incubation engine. Today's AI agents operate in silos, limiting their true potential. We've been focused on building bigger, smarter models, but scaling up is just one approach. To reach superintelligence together, we need to do more. We need to scale out. And we actually have a blueprint from 70 ,000 years ago. Humans didn't just get smarter individually. The cognitive revolution transformed society because we began sharing knowledge, goals, and innovation. Agents are now at that same inflection point. They can connect, but they can't think together.

19:13That's why Outshift by Cisco is building the Internet of Cognition, transforming AI from isolated systems into orchestrated superintelligence. By creating an open, interoperable infrastructure, Outshift by Cisco is enabling agents and humans to share intent, context, and reasoning. The cognitive evolution for agents is here. Explore the Internet of Cognition at outshift.com. That's outshift.com. Hello, I'm Joseph Cox, host of the 404 Media Podcast. Every week, we break tech news you can't get anywhere else. That includes surveillance. This idea that Ring is going to zero out crime, it's not. AI.

19:53AI is going to stick around in some fashion. That doesn't mean that the vision that these companies are selling us is how it has to be. And big tech companies. This is a serious crime. It's being enabled by Apple and Google through the app stores. Our journalism is by humans for humans. Find the 404 Media Podcast wherever you listen and on YouTube too. Welcome back to Uncanny Valley. Today, we're talking about the AI infrastructure boom. Lauren and Molly, we just talked about why data center investments have been ramping up and what the known impacts are. So now comes another key question. Is all of this aggressive expansion even a good idea?

20:36It depends on who you ask, Mike. AI founders will typically tell you that there's nothing to worry about and that this is necessary to meet the current and future demand for AI. But the big elephant in the room is that for all of this investment, consumer spending on AI is not there yet. And some of the frontier model companies have seen a significant amount of their revenue coming from enterprise, you know, developers who are actually building for their companies and using AI for that reason. You know, I have to imagine, and this is just my own theory, that unless we become a world where consumers essentially are developers, where consumers are using AI to the point where we're like all building things and writing lines of code based on this, that I don't see how the revenues get there.

21:24I also found this really interesting. The Economist reported that AI hyperscalers, which are the cloud service providers I talked about who are the largest spenders on AI, are using some interesting accounting tricks to try to depress the reported infrastructure spending, which then, of course, has the effect of inflating the profits from that. So I think that's partly why there's a lot of people concerned that there could be an AI bubble. It's just it's about supply and demand. And right now we are spending so much money as a nation on supply and just hoping that the demand continues to grow.

21:55Fingers crossed. I think when we talk about the demand for this, you have to understand that behind it, there's a lot more going on than just accurately reading the tea leaves. There was an incident in the late 90s and early 2000s where there was all of a sudden this narrative about how much energy the internet was going to use. And there were a bunch of high-profile articles that made some claims about how by the 2010s, the internet was going to use half of the U.S.'s electricity. Like, the internet was posed as this huge electricity suck. And we were going to need to make more coal-fired power plants to deal with it.

22:36And there's a really smart researcher named John Coomey who dug into this. And, you know, he found that it was really being pushed by the industries that stood to gain from such a build out. And, of course, that didn't end up happening, right? Like efficiency gains were great. The Internet doesn't actually use that much energy. Like I think we're in this weird moment where, you know, U.S. energy use has stayed pretty stagnant. We're getting better at using what we have and we're getting better at being more efficient. And all of a sudden, there's the possibility for a big customer of various industries, not just energy, right?

23:11And so it's hard to tell if we're in a bubble when there are so many people who stand to benefit from people believing that this is really going to happen. And to that point, I mean, when you make this kind of investment in infrastructure, it's a very fixed investment. And you're basically doing it under the assumption that the way the compute happens will remain the same for a certain period of time. And actually, what we're seeing is that some of the more computationally intensive AI models could soon potentially offer diminishing returns compared to smaller models. So, for example, like a frontier model company like OpenAI is currently much better, smarter than a smaller model from an academic lab.

23:52But that may not always be the case. There's a lot of interesting research going on. There are new alternatives to deep learning. There are novel chip designs. There's new approaches like quantum computing. And I think what we saw with DeepSeek out of China and its remarkably or reportedly low-cost model this past January, that's already served as kind of a reality check for the AI industry. What do you think is something that we as citizens, as people who, you know, have the decision of whether or not to use AI tools and we have decisions to get involved in whether or not data centers are going to come to our state or come to our county, what can we do?

24:29Think about this all the time. You know, I'm going to assume, I'm going to go out on a limb that most people listening to this are nerds and enjoy reading and learning. And one thing I, I'm so much fun at parties. One thing I love to tell people at parties is not even get involved, but to learn more about how their local electric utility works. Because that's, you know, if a data center comes to your area and impacts your electric bill, it's because of your utility. And utilities in the U.S. are set up in this really, really strange way. And a lot of them are investor-owned, so still have to make a profit for themselves.

25:08But they also control how much money you're seeing on your electric bill every month. And so I think this kind of problem can feel so big and so outside your control. But the more you dig into the sort of wonkier side of things, the more you will find people who are organizing around even just basic asks to utilities to use more renewable energy, you know, to monitor kind of rate hikes. And they will be the ones that will be up on, if a data center is coming to town, what it might be doing to your electric bill. So I would encourage people to read more about the fascinating topic of electric utilities in the United States.