In short
NVIDIA AI Podcast - Episode 292: Powering the AI Inference Wave with EPRI's Ben Sooter
Podcast Overview The NVIDIA AI Podcast explores groundbreaking technologies and their impact on the world. In episode 292, host Noah Kravitz interviews Ben Sooter, the Director of R&D at the Electric Power Research Institute (EPRI), focusing on the intersection of AI, energy consumption, and the electric grid.
Key Points Discussed
Introduction to EPRI and Ben Sooter
- EPRI Overview: A 501(c)(3) nonprofit independent institute that collaborates with over 400 companies globally to innovate and ensure reliable energy.
- Sooter’s Experience: Over 20 years at EPRI, witnessing significant technological advancements in energy consumption and AI.
AI and Energy Consumption
- Current Challenges: Increased electricity demand driven by AI technologies, particularly from data centers.
- Two Types of Data Centers:
- Mega Data Centers: Built for training AI models, require substantial power and cause spikes in demand.
- Micro Data Centers: Proposed as solutions for inference tasks, providing services close to where users are located to reduce latency.
The Inference vs. Training Energy Dynamics
- Energy Use Statistics: Approximately 80% of a model’s lifetime energy consumption is during inference, compared to only 20% during training.
- Implications: Energy consumption patterns differ significantly, necessitating a rethink of data center placement and design.
Micro Data Centers
- Definition: Smaller, geographically distributed data centers focused on delivering low-latency AI services.
- Location Strategy: Positioned near underutilized substations to take advantage of available energy capacity and reduce load on the central grid.
- Benefits:
- Reduces interconnection delays.
- Leverages existing infrastructure, minimizing societal costs.
- Enhances energy distribution and grid resilience.
Technical Considerations
- Infrastructure: Micro data centers require careful planning for energy needs (e.g., 3-20 MW per center) and connectivity (fiber access).
- Load Management: Strategies need to be developed to balance peak demands and utilize excess capacity effectively.
Future Applications of AI Infrastructure
- Real-World Use Cases: Smart glasses for field workers, real-time translations, and advanced applications in transportation (self-driving cars).
- Grid-Specific Innovations: Enhancing control centers for better efficiency and faster restoration times in utility service.
Conclusion and Future Outlook
- Success Metrics: In the next year, success would look like operational micro data centers that provide valuable insights into grid management and AI service delivery.
- Promising Landscape: Continuous evolution of AI applications and energy consumption patterns presents both challenges and opportunities for the future.
Key Takeaways
- The relationship between AI and energy consumption is evolving, with a significant focus on inference tasks.
- Micro data centers offer a strategic advantage by optimizing energy distribution and improving service latency.
- Collaborations between technology companies and energy providers are crucial for developing sustainable solutions in the energy sector.
Further Resources For more information on EPRI and their ongoing projects, visit:
- [EPRI Official Website](https://www.epri.com)
- Follow EPRI on [LinkedIn](https://www.linkedin.com/company/epri) for updates and news.
Written by AI. May contain mistakes. Listen to the episode to check what was said.
Chapters
Tap a time to open that second in VOUnderstanding EPRI and Its Mission
0:57 to 3:54
Ben Suter explains EPRI's role and his extensive experience in the energy sector.
“Thank you so much for taking the time to join the NVIDIA AI podcast.”
The Evolution of Energy Use and AI
3:54 to 6:46
Discussion on how energy consumption has evolved and the impact of AI on data centers.
“which has obviously just accelerated everything about 10x.”
Micro Data Centers: A New Approach
6:46 to 9:26
Exploring the concept of micro data centers and their relevance for inference loads.
“And so that's going to create another challenge.”
Designing for Inference: Challenges and Solutions
9:26 to 14:00
Discussion on the design considerations and energy implications of micro data centers.
“But all that to say that that paradigm is sort of evolving now and I'm having to change my hypothesis.”
Understanding Data Center Power Needs
14:00 to 18:03
Learn about the evolving power requirements for data centers and the exploration of utilizing existing substations.
“It's just smaller because I don't need as much.”
Distributed Inference and Grid Functionality
18:03 to 22:02
Discover how distributed inference could optimize the electric grid and enhance performance for users.
“And so what we realized was if we go to an opportunity, if we go to a regional area, we go to a city, and we say, hey, are there five data centers that meet this criteria?”
Real-time Applications of AI in Energy
22:02 to 24:08
Explore various real-time applications of AI in the energy sector and their potential benefits for users.
“Continuing sort of along the lines of the applications of all of this, but kind of from the other side of it.”
AI Transforming the Energy Industry
24:08 to 28:00
Understand how AI is revolutionizing the energy industry, from data management to practical applications.
“I don't know what, but are there big examples that kind of jump out in, you know, your own work or what you've seen of how AI is transforming the industry from the inside?”
Reflecting on Historical Data Access
28:00 to 28:48
Explore the excitement around a recent project involving historical data ingestion.
“It rings a bell, but I don't know that I did.”
Envisioning Success in Micro Data Centers
28:48 to 30:31
Discuss the future of micro data centers and their impact on AI services.
“So I'll start with the micro data center, you know, part since we're talking about it.”
Show all 11 chapters
Resources for Learning More about EPRI
30:31 to 31:20
Find out where to learn more about EPRI's initiatives and updates.
“And I'm excited to see how it all unfolds.”
Transcript
Automatic transcript. May contain errors.0:10NVIDIA AI Podcast Host:Welcome to the NVIDIA AI Podcast. I'm Noah Kravitz. Today we're talking micro data centers with Ben Suter, Director of R &D at EPRI, the Electric Power Research Institute. The relationship between AI data centers and energy grids is an increasingly important one, to say the least. In a moment, we'll talk about how micro data centers can help strengthen that relationship. But first, a quick note about GTC San Jose. Join us at the world's premier AI conference. GTC San Jose is online and in person March 16th through the 19th. From physical AI and AI factories to agentic AI and inference, GTC 2026 will showcase the breakthrough shaping every industry.
0:52NVIDIA AI Podcast Host:Learn more and register at nvidia.com slash gtc. Ben Suter, welcome. Thank you so much for taking the time to join the NVIDIA AI podcast. Really glad to have you here.
1:03Ben Sooter:Yeah, great to be here, Noah. Super excited.
1:06NVIDIA AI Podcast Host:So, Ben, to kind of set the table before we dive in, for listeners who don't know EPRI, can you briefly explain, well, first, who you are and what you do, and as part of that, what EPRI is and what EPRI does?
1:17Ben Sooter:Yeah, absolutely. So, EPRI is sort of a unique organization. We're a 501c3 not-for-profit. It's an independent institute focusing on R &D, collaborates with more than 400 companies across more than 40 countries, and really drives innovation to ensure sort of the public has reliable and affordable energy. So really awesome mission statement and been a really exciting place to work.
1:43NVIDIA AI Podcast Host:You've been at Ebre for a couple of decades now.
1:45Ben Sooter:Yeah, I've been here a while. I just crossed over the 20-year mark, which I feel like is ancient times and the way the corporate world works now.
1:55NVIDIA AI Podcast Host:Right. Well, congratulations. And you kind of, this is exactly why I asked, because I'm thinking about data centers and AI, but hearing you talk about things like nuclear and thinking like, man, you've like, must have seen some things and worked on some projects and thinking back, you know, over 20 years and how technology and energy reliance and consumption must have evolved. I don't know if this is a fair question to ask, but can you kind of place our current moment in context to, you know, sort of what you've seen with how the world uses energy and stuff you've worked on over the years?
2:27Ben Sooter:Yeah, yeah, that's a great question, a great way to frame it. And it gets to why I probably have stayed here 20 years, which is that there's just so much, you know, change and a lot of different exciting things that have evolved across the sector in the industry that have sort of landed us here today. So lots of stuff going on. It's been interesting as you come in, and I'm sitting here in Knoxville, Tennessee, and behind me, we actually have a big laboratory that makes up the back half of the building. But pairing with that, over the years here at EPRI, I've seen all kinds of technologies come through, whether it's solar energy, battery storage, electric vehicles, all kinds of things.
3:11Ben Sooter:And what's interesting is you see a lot of it several years, like a lot of times in advance of when it's cool and it's blown up and it's everywhere.
3:20NVIDIA AI Podcast Host:Right, right.
3:21Ben Sooter:And so it's been interesting to just see all these technologies come in and evolve and the challenges that come with them. You know, whether it's, you know, how do we handle the loads of electric vehicles or how do we position the distribution system to handle all the solar capacity?
3:41NVIDIA AI Podcast Host:Right.
3:41Ben Sooter:All these different issues and then meeting those challenges. And so it's been an exciting place. And I've gotten to have several lifetimes here because I've been here for 20 years. And so working through different areas and now kind of in this AI space, which has obviously just accelerated everything about 10x.
4:03NVIDIA AI Podcast Host:Right, right. So let's get into that then. And data center, AI, when I say people, when I think of AI and energy consumption, data centers kind of pop to mind sort of immediately. There's more to that, obviously. But can you kind of set the stage a little bit for, you know, kind of explain what a data center is in this context, and then maybe that can get into what this idea of a micro data center is and how it differs, you know, how those differ from the kinds of things that, you know, people like me usually think of when I hear data center.
4:35Ben Sooter:Yeah, so good question. And I think there are several flavors of data center at this point. And I think kind of the two I'm going to sort of hone in on today, one is the data centers that have been really in the news a lot lately. These multi-gigawatt behemoths that are being built with the objective of providing platforms to train these really exciting AI models. And so, you know, there's been an enormous push to build those types of capacity. Obviously, there's been a big crunch for power in order to meet that demand. And so, lots of exciting research in that area. But all of that's really been directed at making the models that are going to potentially do exciting things for us in the future.
5:25NVIDIA AI Podcast Host:Right. With the training of the models.
5:27Ben Sooter:Exactly. The training of the models. And you mentioned this in the plug for GTC at the beginning, inference. I think people don't realize that while we're so focused on training the models, like there's this huge wave that's coming of once we actually get all these models and we move beyond just chatting with JADGPT and we're doing the real-time translation in our AirPods and we're doing the smart glasses and we're doing all the full self-driving and all these different applications that all those applications all falling into inference sort of using the models is going to sort of accelerate this second compute wave that comes along with all this.
6:11Ben Sooter:in order to have the compute capacity to actually do all this stuff. And there's actually an interesting statistic out there that if you look at the lifetime of a model, so if you look at a GPT 5.1 or whatever, only about 20 % of its compute capacity and thus its power consumption is in the training side. 80 % of it is in the inference side.
6:34NVIDIA AI Podcast Host:80%, okay. Yeah.
6:34Ben Sooter:And so the vast majority is actually in the inference side. So if you think about how much capacity we're building for training, We're going to need a couple of times that to meet the demand for all the inference. Once people start using these things full or close to full max.
6:49NVIDIA AI Podcast Host:Yeah. Yeah.
6:50Ben Sooter:And so that's going to create another challenge.
6:53NVIDIA AI Podcast Host:Thinking about energy consumption, is the distribution of energy consumption during inference as opposed to training, is that just massively different and much more spread out? What does that look like from the perspective of, you know, energy load and consumption and figuring out how to try to balance things? There's a lot of great questions in there. And a lot of, I try to throw 12, 13 of Mechia at once. Yeah. You know, yeah.
7:19Ben Sooter:And a lot of them are things that we're looking at as part of this micro data center project.
7:23NVIDIA AI Podcast Host:Okay.
7:23Ben Sooter:So when the world, when we got into these gigawatt scale training data loads, nobody really realized or thought about the fact that the way the compute and things would happen is the training loads would slam hundreds of megawatts of demand nearly instantly within milliseconds. And they can also fall off once that job is done. And so huge swings of power. And that created some consternation as you had to solve sort of the technical challenge of meeting those demand peaks and spikes and things. You compare that to inference. And when we got into this, we started down this journey about midway through last year.
8:08Ben Sooter:And I was initially imagining this and I'm thinking about, okay, if inference is when I'm using one of these awesome models, I'm using ChatGPT, I'm using Grok, I'm using Gemini. And so it's being the compute tasks are being generated by me. So that's going to give it what we call more low diversity. It's going to kind of smooth it out because it's being randomly generated. Sort of my initial hypothesis. Really interesting discussion last week with someone as we started bringing up just the whole agents and agentic AI that has taken over in just the last few weeks. You know, bring open cloth to house, you know, to take over my world.
8:50Ben Sooter:And I started realizing like, oh man, like that's doing all this work at night now. So when I originally thought this load was going to sort of look like a normal load curve for just people waking up during the day, putting on lights and, you know, air conditioners and stuff. Now, all of a sudden I'm like, well, that completely changes the paradigm because now it's running at night while I'm sleeping and, and is it going to do more? And so to answer your question, this is a really long-winded way to answer it.
9:18NVIDIA AI Podcast Host:No, this is – I want to go deeper and ask you what you're doing with OpenClaw, but maybe that's another podcast.
9:22Ben Sooter:Yeah, that may be another podcast because it's – yeah, that's trying to streamline how you actually survive in the 10x corporate environment. Right. But all that to say that that paradigm is sort of evolving now and I'm having to change my hypothesis. And so when we actually start monitoring these data centers and things and actually building them out and realizing and measuring them, it's going to be really interesting to see what they look like. And I have a feeling you're going to see lots of different loads because it's something that's very consumer-centric, may look different than, yeah, there were some great stories last week of some big financial institutions that were very AI forward and have invested a lot in models and don't have enough compute for their internal models.
10:13Ben Sooter:Right.
10:13NVIDIA AI Podcast Host:Yeah.
10:14Ben Sooter:Which is another, as a whole another, you know, it fits very well into what we're looking at here, but it's a completely different, probably, you know, shape. Yeah.
10:24NVIDIA AI Podcast Host:Well, let's dive into what we can kind of grasp at the moment or, you know, is concrete, I should say, at the moment. And this idea of micro data centers. Can you, you kind of alluded to it in talking a moment ago, but can you talk a little bit more about what they are and why now and what are some of the problems? and these may be some of the examples you're mentioning, are you trying to solve for the power grid as well as for AI users with this idea of micro data centers?
10:50Ben Sooter:Yeah, so great question. So the real thing that we're looking at here, I mentioned everybody's focused on these big giant training data centers. Now we're thinking about how do we create these data centers for inference? And when you actually look at those data centers for inference, and one of the things you start to realize is that having the huge mega data centers that are centrally located don't necessarily make sense for the inference data centers because they are more consumer-centric and user-centric. Positioning them geographically around where the people are tends to make more sense because they can be more latency-sensitive, etc.
11:36Ben Sooter:So you don't necessarily want to have them just in one place in the middle of nowhere. Better to have it broken apart.
11:43NVIDIA AI Podcast Host:I don't know. I may be way off here, but it reminds me of when streaming media centers, you know, started popping up kind of in the whatever period of the aughts, I guess, right? The first dot-com wave when multi-measster become a thing. And yeah, that kind of proximity because it affects performance, as you said.
12:01Ben Sooter:Yeah, exactly. When, you know, the early years of Netflix, where it started off very centrally, and then they realized, hey, if we put a mirror onto the local networks, it becomes a lot easier to distribute. So yeah, it's another thing. And incidentally, the biggest user of geographically dispersed servers, game servers.
12:25NVIDIA AI Podcast Host:Yeah, right, right, right. Through this journey, learn that little tidbit. But can you walk through a little bit what happens in a micro data center in terms of sort of, you know, how do you design and build for an inference load as opposed to a training load? and what does that mean in terms of both the energy usage, but then also like the ripple effect of not housing everything in these central giant megawatt data centers that, as you said, at least for training, they act differently than other big loads on the grid. They come up super quick and I imagine all kinds of other problems that are beyond my knowledge set.
Read the full transcript
13:05NVIDIA AI Podcast Host:But just can you talk a little bit about how they sort of work on that level?
13:08Ben Sooter:Yeah, so a couple things that are kind of in that onion to unwrap. So the first one is sort of on the really underlying underneath construction. It's somewhat similar in the fact that it's still very sort of GPU or TPU-based compute need in order to actually run these models. We're seeing, I think, more chips. Like NVIDIA has more chips, more designed for inference and training now. So there seems to be a little bit of diversification. What was now just sort of one chip initially. And so we're seeing that. So there is some variability, I think, maybe in the underlying chips. But traditionally, it's been sort of the same chip for training and inference.
13:57Ben Sooter:And so from that perspective, it looks similar. It's just smaller because I don't need as much. But, you know, it's sort of the result of, you know, I don't need as much. It's sort of like, how much do I need? And that's been one of the things that we've been looking at. And one of the interesting things that we've been working with technology partners like NVIDIA to really help us understand, you know, what the compute needs of the actually technology companies that are buying these data centers, using these data centers. looking in that, you know, is 3 megawatts enough? Is 5 megawatts enough?
14:34Ben Sooter:Do we need 20 megawatts? And there seems to be, we seem to be coalescing somewhere around this idea of 20 megawatts. But that's actually sort of, I hadn't gotten into some of the electrical aspects of all this, but as we're looking at where to place these microdata centers, 20 megawatts can be the not insignificant ask of just dropping a load somewhere onto the grid. And so there's not a lot of opportunities to drop something of that size. And when EPRI was looking at, okay, our partners are telling us about this coming compute wave, and we want to do what we can to help our utility members be proactive and get ahead of it.
15:20Ben Sooter:Where can we look at opportunities to find power for this type of data center.
15:27NVIDIA AI Podcast Host:Right.
15:27Ben Sooter:One of the things we started looking at was, well, there's substations all over the United States and indeed all over the world. And there's a fair number of them that are actually underutilized. So they've got excess capacity available inside them. And so we started thinking like, well, is there an opportunity there to partner with those substations that have that excess capacity and do something and put these inference data centers near it and maybe directly adjacent is maybe ideal, but close by and make sure we've got everything that is needed in terms of fiber access.
16:04NVIDIA AI Podcast Host:Right, all the infrastructure.
16:06Ben Sooter:All the underlying infrastructure. And so look at all those things and say, does that work? We thought that was a good idea, but the answer is you're probably going to find 3 to 5 megawatts, maybe up to 10 megawatts of available capacity in a single substation. And so then we started thinking about, well, how's that going to work?
16:30NVIDIA AI Podcast Host:Ben, just to interrupt you real quick, sorry, because I keep having a picture in my head of, this is my own ignorance about our electrical grid, of how big one of these existing substations is and where it might be. Is this kind of like suburban as opposed to metropolis? Is that...
16:46Ben Sooter:So it could be both. Okay. So there's a couple caveats in there. So you're right in thinking that your suburban substation may be more likely to have some of that excess capacity.
16:59NVIDIA AI Podcast Host:Okay.
16:59Ben Sooter:That said, we have found that there's interest at the metropolis level too. In capacity. In capacity because there is need. There's people there, so they want to get the compute close to it. And actually, if you see some of the metropolis environments, there's a lot of real estate that's available right now, which equates to load that's not there. So there's opportunity to put load. So that was another hypothesis going in. Yeah, there wasn't going to be interest, but actually it looks like there may be interest and opportunity at that level as well. And so, you know, as you're looking at these data centers and you start to say, well, does three megawatts make sense?
17:41Ben Sooter:And does it make sense for the person that wants to buy it? What we realized was maybe there's an opportunity, and this is the distributed part. We initially kind of called this project distributed inference, truthfully. And while distributed inference seemed to be very technically accurate, it did a really poor job of giving anybody a visual image of like what it was we were talking about. And so what we realized was if we go to an opportunity, if we go to a regional area, we go to a city, and we say, hey, are there five data centers that meet this criteria? And then each data center maybe has five megawatts of capacity.
18:18Ben Sooter:Now we've got five data centers at five megawatts. And now we've got 25 megawatts of capacity. And so actually looking at it as, you know, instead of a single project that's five megawatts, looking at it as a 25 megawatt project that just happens to be distributed across five sites. And so that helps meet the needs of like what the utility grid, you know, has available and sort of meet the economics of what the data center companies need in order to actually make it realistic and viable for them.
18:50NVIDIA AI Podcast Host:Right, right. How does this approach affect the way the grid functions for just, you know, people in general, the city, the region in general?
19:00Ben Sooter:So great question. We've really sort of seen this as a win, a general win for everyone, because the answer is if the existing substations are already kind of sunk cost. We've invested that capital. We've made the investment. We've built it. And so if we can get extra capacity, if we can get extra usage out of existing assets, then that's sort of a win for everyone. Right, of course. If you're at societal cost, if we're not having to put new steel on the ground, then that's helping keep rates lower and things like that. So we really see this as a positive in terms of being able to leverage existing infrastructure.
19:44Ben Sooter:Speed to power, I think, is also a big part of this, where there's a huge scramble for this capability and everything. And so it also means that you no longer have to deal with interconnection cues because you're off the transmission grid and all the things that go along with that. So it definitely speeds up the ability to get to a finished product that's online and serving customers much faster as well.
20:13NVIDIA AI Podcast Host:That's great. Are there clean energy implications?
20:17Ben Sooter:You know, it's interesting you say that. So, definitely there's opportunities to layer all kinds of things on this. So, there's opportunities to layer this with DER and solar, wind, things. And I think there's also a lot of opportunities for energy storage. One of the things we've been looking at getting sort of into the technical weeds, we've been looking at flexibility and how – what you find is that you'll have a substation and it's got excess capacity, but it's actually got quite a bit more capacity, except for July 21st when you have the hottest day of the year.
20:59NVIDIA AI Podcast Host:Right.
21:00Ben Sooter:I'm making July 21st up. That's not the hottest day of the year. Somebody fact check me.
21:04NVIDIA AI Podcast Host:I was like, wait, what AI breakthrough happened on a July 21st?
21:09Ben Sooter:Just been made up a date.
21:11NVIDIA AI Podcast Host:Super hot day. Yeah, yeah, yeah.
21:12Ben Sooter:So if you can engineer it so that you can have flexibility to reduce your load, reduce your demand during those peaks, you actually have a lot more envelope that you could potentially use. And so pairing it with energy storage, backup generators, just working with the technology partners. One of the other nice things about if you have sort of a distributed network of these loads is if there is, you know, possibly like a peak demand issue, I can run down my compute and wait into center and route the calls someplace else. Right, right. And move, smooth things out that way. So there's lots of possibilities.
21:53Ben Sooter:And so that's another thing that sort of makes this exciting and a really neat way that a tool that the utilities could use as well.
22:01NVIDIA AI Podcast Host:Yeah, yeah. No, that's very cool. Continuing sort of along the lines of the applications of all of this, but kind of from the other side of it. And again, you talked about this in reference to, you know, building the data centers, these smaller data centers close to where the users are, the consumers are, and that performance aspect of it. But are there other examples of real-time applications that, as this infrastructure rolls out, you think will be enabled or maybe just kind of accelerated these applications that could directly benefit people?
22:36Ben Sooter:I think there's all kinds of things. And I am certainly not going to claim to have a view into all of those options. I mentioned some, like the translation. Right, right. and self-driving and things. But I think especially as agents develop, as we get smart glasses that can analyze, just here at EPRI, other exciting things we're looking at, and these are going to have applications for everybody, but can you use smart glasses to analyze your poles and transformers and things in a substation and make your line workers smarter, more efficient and safer all at the same time. And so, you know, there's all these applications that everyone's looking at.
23:24Ben Sooter:Can we, you know, again, grid-focused, but can we make the control center of the future smarter? And get smarter about restoration times and all these different things, on and on. I think there's just internally at Epreit, there's a few hundred use cases and things we've identified. And that's very grid-centric. So, you know, obviously the audience is probably not all utility workers and things. But I can only imagine that if the electric industry has identified several hundred use cases, then, you know, around the world, there's got to be just tens of thousands.
24:02NVIDIA AI Podcast Host:We wouldn't be here having this talk on tape, so to speak, if there weren't, right? But kind of, I was just thinking about this as I was listening to you and you spoke to it with examples of like smart glasses, people out, workers in the field, you know, analyzing things. But are there ways that you've seen, you know, and whether you're using them now or maybe things that you kind of see coming that you're excited about, ways that the energy industry has been using AI to, And I don't know if it's like to design better battery storage or to explore, you know, new forms of energy or to, you know, maybe something seemingly more mundane, but still really important, like reorganizing the way that, you know, companies approach different industries.
24:54NVIDIA AI Podcast Host:I don't know what, but are there big examples that kind of jump out in, you know, your own work or what you've seen of how AI is transforming the industry from the inside?
25:02Ben Sooter:Yeah, I mean, I think it's transforming it in all kinds of different ways. And it's one of those things that I think has been really interesting because things do seem to, you know, there's lots of memes about how fast things are going. And I already made some comments about 10xing and things. But it's all sort of the proof is in the pudding. Have we seen, where's that scaled demo? I think there's a lot of proof of concepts that we're seeing pop up around. And really the thing everybody is waiting for is that scaled demo of where there's this application and it's measurable and we've scaled it out to the entire enterprise.
25:46Ben Sooter:So there's definitely a lot of work to do. But I think there's lots of applications as well. Yeah, I'm trying to go through my head. There's just so many different things. But, you know, because everything from understanding, you know, in the utility industry, there's a lot of historical records and things. And a lot of them predate sort of the digital era. And so current models and things can make just ingesting all of that and structuring it into useful structured data sets that you can then use to create new models and create analysis and digital twins and all these things. So I think there's some of the places the existing work is already really useful.
26:32Right.
26:33Ben Sooter:Obviously, all the things we do every day just to accelerate ourselves, you know, with understanding emails and, you know, figuring out how to, you know, have that hard conversation with the problematic coworker.
26:47NVIDIA AI Podcast Host:And that's totally making these up as well. No, no, no. But it's relatable. It's that, well, it's that interesting sort of, there's two layers. Well, there's many layers. The five layer of cake is the iconic layer at the moment. But there's kind of two layers when I'm thinking about it. There's the layer of like the kinds of work that, I don't want to call it knowledge work, but that kind of working with information you just described that is part and parcel of many roles in many industries, right? And then there's kind of the, and AI is helping, you know, helps me day to day in ways you were just describing or, you know, kind of making up and I get you.
27:23NVIDIA AI Podcast Host:And then there's that layer on top, which is specific to the kind of work and the industry that you're doing. And the more people like you, I get to have these conversations with just the more in my mind, I see like, you know, it's both right. And one informs the other, being able to go back and ingest all that old data. You know, we've had a cardiologist or a radiologist on a while ago talking about how much hidden information there is in old analog film scans. Oh, yeah. That, you know, AI image analysis is able to extract now and it's useful, right? And that kind of stuff is, yeah.
27:57Ben Sooter:Did you see the guy with the microfiche, like, repository? It rings a bell, but I don't know that I did. This is a few months ago now, which makes it ancient news. But yeah, there was somebody that had access to this huge repository of Microfish. And I'm old enough. Those of us that are old enough on here will remember looking at it under the little magnifying contraptions in the library. The machine in the library. Yeah, to see the news article from 1942. But he had access to tons of this stuff and started using the models to ingest it all. And just created a monster data set. And it's so cool.
28:33NVIDIA AI Podcast Host:That's amazing. I love stories like that. All right, Ben, as we get to kind of wrapping up here so I can let you go, this is not to put you on the spot because, as you mentioned, these kinds of things are impossible. It's always impossible to predict the future, but when things are moving as quickly as they are, it's harder, right? But if we look ahead to the next year or so, you know, loose timeframe, what does success look like, you know, with micro data centers and even more broadly, I guess, that's what I'm thinking about putting you on the spot, both for the grid and for everyday users of AI powered services?
29:05Ben Sooter:So great question. So I'll start with the micro data center, you know, part since we're talking about it. And I think, you know, hopefully in a year or two years, we've got a pile of these, you know, micro inference data centers built out and we're monitoring and measuring them. And that's helping educate us on what we need to know so that we can continue to build about for all the wonderful things that the industry is going to create. So I think, you know, from the micro data center standpoint, you know, that I think is what I hope what success looks like. And then, you know, I think just in general, you know, I have no idea that everything is so exciting.
29:46Ben Sooter:It's, you know, you mentioned GTC at the beginning. I learned something new from those types of conferences and stuff. Every year, there's new things that come out, completely change things.
29:57NVIDIA AI Podcast Host:I mentioned agents, which are just weeks old, maybe a couple of months old,
30:03Ben Sooter:that we've really sort of delved into that. It's changing the landscape again. So I don't know what it's going to look like, but I'm hopeful. And it's going to be exciting. And there's going to be compute needs. As you mentioned, you know, at the very beginning, sort of the importance of power and stuff. You know, I think, you know, there's still going to be challenges to solve to make sure that we can provide all these awesome things to everybody and really move society forward and everything. So exciting times.
30:32NVIDIA AI Podcast Host:Excellent. Yeah. Well, I'm with you. I'm rooting for you. And I'm excited to see how it all unfolds. Ben, for folks who would like to learn more about the work you're doing, about the work EPRI is doing. Where's a good place for them to go online? Websites, social media accounts, where should they start?
30:50Ben Sooter:Yeah, absolutely. So, websites. So, you can go to epri.com, E-P-R-I.com. It's our official website. So, lots of great information there. Also, very active on LinkedIn. There's lots of, if you're interested in the latest news about exciting AI and data center updates and their adjacentness to the electric sector. Lots of good stuff going over there on LinkedIn. So those are probably the two places to find us.
31:19NVIDIA AI Podcast Host:Perfect. Ben Suter, thank you again for joining the AI podcast and best of luck with everything you and everyone at EPRI is doing.
31:27Ben Sooter:Appreciate it. Great to be here. Great talking with you.
31:39Thank you.
32:07Thank you.
From the publisher
AI is reshaping electricity demand. What does increased demand, and the shape of that demand, mean for the electric grid? Ben Sooter, Director of R&D at EPRI joins the podcast to explain why most of an AI model’s lifetime energy use comes from inference rather than training, and how micro data centers located near underutilized substations can help deliver low‑latency AI services while strengthening grid resilience.
