From superchips to sovereign AI, with Kari Ann Briski

0:00Starting a business comes with its share of ups and downs, which is why staying true to your vision is essential. a non-negotiable for Romeo and Milka Bregali, Capital One business customers and co-owners of Ra's plant-based restaurant in New York. Romeo and Milka took a leap of faith when starting their own restaurant, gutting an empty space and building it from the ground up. Every pipe, every wall, every detail. But building from scratch came with a heavy financial burden, which is when they turned to their Capital One business card. With the flexibility of the card's no preset spending limit, they were able to spend more and earn more rewards while bringing their vision to life.

0:36Today, Raz's success is proof that with passion and the right support, it's possible to make your dreams a reality. Learn more at CapitalOne.com slash business cards.

0:49NVIDIA, the largest AI chip maker in the world, announced some pretty big news to kick off 2025. At CES, the huge annual tech convention in Las Vegas, NVIDIA CEO Jensen Huang debuted the Grace Blackwell Superchip. He also announced a family of new foundation AI models called Cosmos. On this episode, we're going to talk about a different side of NVIDIA's business. But before we get into that, I want to take a moment and talk about how important these new announcements are. First, let's take the superchip. NVIDIA is using the chip to power a new kind of PC, an AI personal supercomputer, one that can fit on your desk.

1:36This computer can run large language models that are up to 200 billion parameters in size. Your average PC today definitely can't train or run these large models locally. For context, widely used LLMs range from tens of billions of parameters to hundreds of billions of parameters. And depending on the exact model size and how complex your prompt is, you may need multiple high-end GPU servers to run these models. And no, the average person won't need this kind of compute power yet. But you know who will? AI developers who are building new projects, or folks who are researching algorithmic innovations.

2:18Then there's Cosmos, the family of new foundation models. This is the product I'm most excited about. We've all seen a lot of large language models that can generate text, images, and even video. But Cosmos is something different. It's trained on real footage of human movement, and that will be used to power embodied AI, like humanoid robots and industrial robots. Of course, a home robot like Rosie from the Jetsons would be awesome. I personally can't wait for a robot to do my laundry. But there are so many other applications of the model. Think about its impact in manufacturing, infrastructure, and even personal health.

2:58So imagine an AI-powered wearable that can recommend the perfect diet for your needs. Or even an autonomous robot that can repair an underwater tunnel. Going into 2025, one of my top predictions is that we will see a rise in embodied AI. 2024 was undoubtedly the year of chatbots, but we live in a physical world. Embodied AI in the form of robots and wearable technology will augment our human abilities, both physically and mentally. As you might already know, NVIDIA is currently the second most valuable company in the world, only slightly lagging behind Apple. The company's AI chips get so much attention.

3:41But actually, NVIDIA is a full-stack company, which means that they offer a suite of AI infrastructure products and tools that allow companies to design, build, and deploy AI systems. That means everything from the hardware, a.k.a. the chips, to AI-powered software. I got the chance to sit down with Keri-Ann Briskey, who oversees generative AI at NVIDIA. On our episode today, Keri is giving us a look under the hood at NVIDIA software business. We'll talk about everything from the competitive advantage of running a full-stack company to the rise of sovereign AI as nations around the world create their own foundation models.

4:22As a heads up, we get into some weedy territory, but this is a great listen for every level of technical knowledge. I'm Rana El-Khalyubi. And this is Pioneers of AI, a podcast taking you behind the scenes of the AI revolution.

4:49Before getting into Keri's work at NVIDIA, I wanted to know more about her background and how she uses AI in her daily life. So before we get into talking NVIDIA, I am always curious about how people use AI in their day-to-day lives. So do you use AI at all outside of work? Yes, and I wish I could use it more. I'll say that. I think being in the industry and you see the power of AI and where things are going, you almost get a little impatient in some day-to-day situations. You're like, oh man, I know this is going to be great in a year or two. You know, I think about just doctor's visits with my kids or I think I just started talking to my car the other day and then I had to giggle because it wasn't listening and I forgot I actually had to touch the wheel to make a command rather than talk.

5:36So yeah. That is actually funny you bring that up because in my former life, I was CEO of Affectiva and we did a lot of work for the automotive industry. And our whole vision was this in-car companion with emotional intelligence that, you know, you can kind of talk to it about like planning dinner tonight. We're not quite there yet, but wouldn't it be cool. Yeah. So you and I are both passionate about getting more women in tech. I'm curious, what was your path to tech? I mean, I think with some people you just know, right? I was always the single girl in the basement of high school taking drafting classes.

6:12And when I went to college, computer engineering was a new dual degree where you could do electrical engineering and computer science. And by the end of my freshman year, I was like, that's what I want to do. I was one of five women that graduated in computer engineering in 2000. And it's funny because I actually used to help coach at local high schools. And one of the girls I coached, she went to computer engineering and she called me and she's like, there's only five girls in my class. Why is that so? So I guess we have to fix it. We can totally do a whole episode on just how to get more women in AI, right?

6:48Yeah. There's a lot more work we need to do there. All right. So most people know NVIDIA for their AI chips work, right? Kind of the hardware microchips or the graphics processing units on which we run AI models, both for training and for inference. But what most people don't realize is that NVIDIA is actually way more than that. It's a full stack company. And I kind of want to take a moment just to explain to our listeners what we mean by full tech stack. especially in an AI tech stack we mean everything from the hardware layer all the way to the application layer and everything in between so that's the infrastructure that's right the model training the model inference the model validation the deployment tools the applications and I think a lot of people don't realize that so why did NVIDIA make this choice or this decision to be a one-stop shop for all things AI I don't know if it's a choice but it was a passion maybe of you know, you want to achieve your dreams and what's the whole purpose.

7:43Not the what are we building, but why are we building it? But I think if I were to take a step back and, you know, talking about full stack, Jensen always talks about there are two simultaneous platform transitions in computing. And the first transition was from general purpose computing to accelerated computing. So accelerated computing not only speeds up applications, but also reduces costs and energy consumption. So this allowed new types of applications to be explored. The accelerated computing powered the gaming industry to start with, right? And it's this idea that you can do a lot of parallel computations on the same chip.

8:16That's right. And so that's kind of the first paradigm shift. And so the second one was enabled by accelerated computing. So the development and deployment of software in a new way, and that's what you were mentioning with generative AI. And so that's about massive AI models that can understand and learn from vast amounts of data. There's data processing and fine tuning and evaluating and optimizing and guard railing of all these models. So to recap, these two shifts in computing, the rise of accelerated computing and the generative AI explosion, are changing the tech landscape. Accelerated computing was initially critical for the gaming industry and also in scientific work where large amounts of data needed to be crunched.

8:57But now with the rise of AI, it's not just the gaming and research industries that need accelerated computing. Every industry does. And NVIDIA, as a full-stack company, is well-prepared to provide that infrastructure. So if you think about it, every back-office enterprise software tool that never needed a GPU before now needs accelerated computing because it's going to be running generative AI as part of its software suite. I think you're also implying that you absolutely need accelerated computing if you are building these models. But it sounds like you also would need them if you're deploying these models for inference as well.

9:36Inference. This is the stage that comes after you've trained your models. It's when the AI model is drawing real-time conclusions or answering questions or synthesizing information. For example, every time you prompt ChatGPT with a question, that's inference at work. We need accelerated computing for inference, and Carrie says that there are two reasons why. One is that even though we're doing a really great job at distilling knowledge and getting large knowledge packed into smaller models, the most accurate models are still the really large models. And we definitely see that larger the models, the more data, the more accurate they are.

10:14So that's one. And I think what people sometimes don't understand is I think people think that you ask a question and you hit one model and it comes back with the answer, and that's absolutely not what's happening. Even today, it's not just a model, it's a full system. and you're hitting at least a dozen LLMs, large language models, to get an answer. And I think in the future, when we have agents and agentic AI, you're going to have agents working on your behalf, talking to other systems agents. And so there's going to be dozens of models that need to work. And so that latency is really going to need accelerated computing.

10:43I love that. And we're going to come back to the agentic AI conversation because that's super important. But you're right. It's not just a call to like open AI chat GPT, right? APIs, it's often a combination of a whole set of models. Yeah, you could have large language model routers because you have an LLM that's fine-tuned for maybe your finance system and an LLM that's fine-tuned for your supply chain management system. And so that all gets coalesced together and then generated for an answer. And so you have a lot of them working for you. Yeah. So you head up generative AI products and product management at NVIDIA, and you've been at NVIDIA for the last eight years.

11:22Can you kind of explain what your role entails, both to a tech-savvy audience, but perhaps also to folks who are not deeply immersed in the tech industry? Yeah. As a product manager, you have to know the why. So why do we need to have reduced precision, right? Why do we need sparsity? Why do we need these features inside our hardware and systems? Because we're trying to build the next speech synthesis or large-scale generative AI or diffusion models. And so we build our own models so that we can kind of push the limits of our hardware systems as well. And so knowing the why. And so having that roadmap of what we're building, why we're building it, positioning it in the market.

12:03You know, at NVIDIA, we really love our partners. We want our partners to succeed too. And NVIDIA has a lot of partners. Some of them are the biggest names in tech, like Meta, Amazon Web Services, and IBM. Yeah. And so a partnership is when you both have something to provide, but also a commitment to each other to help each other succeed. And so knowing your position in the market, how to position with your partners, what's your value that you bring? Not just partner with me because I'm in video, but because I'm bringing value to you and you're going to provide value to me. And so I think that's part of the role of the product manager as well.

12:37I call my product managers T people, so they have to go broad, but they have to be able to go really deep very quickly. Because you're interacting with engineering architects all the time. And so you have to be able to go toe to toe. You have to be able to say, that doesn't make sense. Can we talk about this? And let's maybe change some things. Yeah. I am 100 % going to use that T people example. Genius. But what do these partnerships actually look like? And why do they matter to you? Think big. Like changing the infrastructure of an entire country kind of big. We'll get to that after a short break.

13:43Transcription by CastingWords Video updates in Loom, and now AI agents in Rovo, which connects the dots across your work so nothing gets lost. It's one AI-powered teamwork platform designed for how modern teams actually build. Learn more at Atlassian.com slash TeamChanger. That's A-T-L-A-S-S-I-A-N dot com slash TeamChanger. so today most of the models we recognize are u.s based like open ai's chat gpt or google's gemini or anthropics clod but of course nations around the world want to invest in what we call sovereign ai or ai that is kind of built in-house right they want to build their own infrastructure and leverage their own data and also like hire their own workforce it sounds like this is an area NVIDIA is really kind of doubling down on.

14:39Can you kind of say more about how NVIDIA is helping nations build their own versions of AI? And what does that look like? Yeah, I think what's interesting was that a couple years ago, actually, nations were some of the first movers on understanding the value of AI infrastructure because they knew that they needed models in their language, preserving their national culture, colloquialisms, and just it's better for them to sort of stand on your own two feet with AI is always a good thing, not just for your nation, but also for jobs in your nation, for people to understand how to use AI in your nation.

15:15And then you're using a model that understands you and your culture. And that would be an example of, say, an AI product or an AI software product that's on your roadmap, where you're kind of building these tools to support these nations? Yeah. So what we do is we support the building of these models, right? So just maybe the architectural infrastructure. And then we call this a foundry, actually, because we bring the pillars and the expertise of either community models or our own models, the tools to be able to do it. So the algorithms and to be able to optimize and then the compute necessary.

15:51And then they bring the data and output. The model is theirs. Now, of course, this isn't just at the nation's level, but at the enterprise level, too. So are there any cool examples of how you're helping enterprises build and deploy AI? Yeah, I think that enterprises maybe a year ago were really dabbling with maybe operational efficiency or productivity as the first use cases, especially for customer service. If you might have an account of a customer's call 20 times and you have an agent that gets on the phone, you can quickly summarize what's happening. So I think summarization was a real easy one for enterprises to go to.

16:25And now really everyone's sort of diving into that virtual assistant and agents. And so being able to retrieve information and then be able to generate answers. And so basically it's that sort of search or deep recommender system for your own company that's connected to all your important systems and applying generative AI to that. Yeah. So let's actually double click on the agentic AI. An AI agent is basically an AI that can act on your behalf, whether you're an individual or an organization. and it sounds like this is something NVIDIA is investing in. Can you tell us more? Once you've trained or fine-tuned a model in your domain or a task, you want to go put it to work.

17:04Our concepts, we call them NIMS. Like inference is one word, but we call it NVIDIA inference microservice and just NIMS is like a lot easier to say. And I mentioned to you earlier, it's not just one model, it's multiple models working together. And so how do they work in concert? How does an agent reason about a task? I mean, you use the word artificial intelligence, and I think saying AI has become pretty numb to people. But I think when you really think about the types of models and the evolution of AI, I'm just going to kind of backtrack for a second and geek out. But like first it was around computer vision.

17:35And I do remember someone saying, well, it's all solved. ImageNet happened, we're done. I'm like, well, we're just at the tip of the iceberg, right? For artificial intelligence, like what is intelligence? You have to not only see a vision, but you have to listen. You have to be actively listening. You have to internalize and understand. And if you're given a task, you have to make a plan and go do that task and then check on it and be iterative about it. And then you have to be able to speak and have speech synthesis, right? So all of these different models, being able to put them together, and now we're at that time.

18:04But these agents go off and they need to understand the prompt or the question or the task that was given them. They need to make a plan. They have to go execute on that plan. And when the answers start to come back, they need to say, does this make sense? And so I need to go back again and iterate until it does make sense. You're also underscoring something very important here, which is that AI should be and is fundamentally multimodal, right? To your point, it's bringing all these different channels of information and data, just the way humans do, like with our vision, with our listening, with our kind of interpretation and intention modeling and all of that.

18:39One area I spend a lot of time building is emotional intelligence and building empathy into these interfaces and these technologies. And kind of back to the car example, you kind of want your car to see that you're, okay, you're a little stressed driving the kids to soccer practice. Say more. Situational, right? So like even you want your agent or AI to be able to adapt, adaptive and situational, right? So I might have a different voice that I'm talking with you right now than I am in the car screaming at my kids to get their shoes on and get in the car for work, to get to work in the morning.

19:11Or I might have a different voice if I'm in a serious situation or if you're just out at a bank or in a retail shop ordering, you know, boba tea or whatever the kids do these days. Right. But you definitely want it to be able to adapt. Yeah, absolutely. Now, this also brings me to another question, which is there's a lot of interest in these language models being larger and larger. But NVIDIA released a small language model, which has accuracy comparable to these large language or LLMs. Can you say more? And why is that important? Why do we need these small language models? Well, I think it's dependent on the deployment scenario.

19:47Maybe you're on an edge device or you're in a situation, maybe more classification type scenarios, or you're able to fine tune a smaller model for a certain situation, or you're able to distill a larger model down to a smaller model to get, you know, 90 % of the accuracy, but three times the throughput. People are interested in that, A, for the inference footprint cost of the deployment scenario, but also that that situation might be okay for a loss of accuracy. But that's when I was talking about LLM routers, where if you have a situation where you absolutely need to accuracy, you want to route to the extremely large model that has the highest accuracy.

20:24Yeah. I want to go back to the automotive example. And I love that you started with that because I think we can kind of apply a lot of our discussion to like the car situation. So, you know, if you're using large language models in the cloud, and say you're building an in-cabin sensing solution that detects driver drowsiness or driver distraction, if you're relying on the cloud, you now have to stream all the data to the cloud all the time. And that is expensive. There's latency to your point, like it can slow down the results. But also there's privacy concerns. You may not want your video data in the car being streamed to the cloud.

20:59And that's true for enterprises, right? You know what's happening with your data and where it's going. And you're owning it and you can run it on, like you said, on the edge. So all the processing happens locally and you don't need to send your data anywhere, which to your point, a lot of enterprises will want to prioritize. That's right. With my investor hat on, I do worry that big tech are hogging all the AI chips and that that stifles innovation, both at the startup level, which is, you know, when I have my investor hat on, I want to make sure that startups have access to the latest and greatest chips to build their AIs, but also at the enterprise level.

21:35So how is NVIDIA helping to democratize access to both AI chips and AI software, I'd say? I think that there's a lot of innovation happening at the startup level. I think that that is why we have this inception program for startups who need additional help or get extra attention, sometimes compute, sometimes libraries, sometimes advice. I forget where we are now, but the last time I checked were something around like 18 ,000 inception companies that we work with. I think that what's interesting is that the time has changed, is that you can have a smaller amount of people working in a startup if you have access to compute because you can actually get a lot more done.

22:14And again, when I was talking about productivity gains, you can now take maybe a 10-person engineering company and you can increase their productivity by 10. And now you have a 100-person company where engineers working for you. And then working with our ecosystem partners, right? So we don't just work with the big companies, but we work with our ecosystem partners to make sure that they're integrating our tools and access to our tools and so that they can now come through different channels faster to the enterprise. There is so much more I want to ask Keri about her work at NVIDIA. We're going to take a short break, but more of our conversation in a minute.

23:06Meet Nicole Nicholas, Capital One business customer and co-owner of Ansett Uncles, a plant-based restaurant and community space in Brooklyn, New York, that got its start from a need for unity. The inspiration, it was born from the desire to create a space that felt like home, where we can connect community culture, good food, and come together with family and friends. That's how we birthed aunts and uncles. Nicole and her husband, Mike, were fulfilling their dream of bringing people together out of their home kitchen. But they soon learned that the demand for community was greater than they knew.

23:37It became overwhelming and we were like, we need home, but not in our actual home. We realized that there was also a need in our community for something bigger in our neighborhood. So we had to find a place. Moving from a home operation into a storefront was a huge next step, but Nicole and Mike were able to take it on with the help of Capital One Business. It's not for the weak as a small business. Finding resources is super important because that's the way you'll be able to manage and scale. We would have never done that without having Capital One to be able to help us along the way. The cashback rewards are very helpful.

24:12You know, it just gave us that runway to be able to breathe a little bit. Then you get to focus on the cooking of the food and making the experience great. To learn more, go to CapitalOne.com slash business cards. So today, AI is very energy hungry. I read somewhere that it takes about the entire energy consumption of the whole country of Costa Rica to train an AI model using NVIDIA's H100s, which is, in my mind, not sustainable. This is not a sustainable way to train and deploy AI. Do you see that changing with some of NVIDIA's newer chips? Yeah, I think most people are actually kind of surprised how much we think about energy efficiency at NVIDIA.

24:55You know, a lot of the top 500 green supercomputers are NVIDIA GPUs. And so, again, when you think about the purpose of accelerated computing is that you do more with less. And so you accelerate it and you're actually saving energy in the long run. We want people to have a smaller inference footprint. We want the smaller model with the more accuracy because we know that you're going to need so much more use out of it. So we're always thinking about energy efficiency, not just for training, but also for inference. All right. Final question. If you could have AI do anything for you, what would you have it do?