NVIDIA’s Jacob Liberman on the Power of Agentic AI in the Enterprise - Ep. 250

0:10Hello, and welcome to the NVIDIA AI podcast. I'm your host, Noah Kravitz. Developers are excited about agentic AI, and they're not alone. More and more enterprises are deploying applications with agentic capabilities. But with the excitement comes new questions and challenges. How widespread will adoption of agentic AI become? What should AI teams be thinking about when designing and developing AI agent applications for the enterprise? And what should they be thinking about during adoption? With us live from GTC 2025 to explore the growing role of agentic AI in the enterprise is Jacob Lieberman.

0:44Jacob is a director of product management at NVIDIA, where he leads a team building cloud-native Gen.AI software solutions. Currently, he's focused on building accelerated storage platforms that connect AI agents to enterprise data. And prior to joining NVIDIA, Jacob held product management and engineering positions at Red Hat, AMD, and Dell. Jacob, welcome to the AI podcast, and thanks so much for taking the time to join us. Thank you for having me. I'm very happy to be here. So should we start with the basics? And I'm just going to ask you, what is an AI agent? So no, I'd say an AI agent is the latest evolution in the way people are using Gen AI.

1:20We're kind of in the third era of Gen AI use already, which is crazy when you think about it because it's really been only 18 months or two years since the technology became widespread. But I would say that it started out, people would chat with LLMs and they would use Gen AI as co-pilots and assistants to do their work. Next, they used retrieval augmented generation to attach their LLMs to data and chat about their data. And now people are using large language models to reason and act and do things in the world. So you could think about it this way. With an LLM, you could ask it to plan your trip to Europe.

2:02With an AI agent, you could ask it to plan you a trip to Europe and to book it for you and to give it little cues like, hey, I like castles. And it will do the research and create an itinerary for you, compare prices, and then actually book the trip. Right. So before we get a little deeper, is an agent a large language model with different capabilities? Is it a completely different piece of technology? Is there kind of a concise way, and if there's not, that's fine, to sort of just set the level for the listeners of how agents and LLMs and other models work together? Sure. So I would say that a large language model, when you train it, it's a bit of a generalist and you can give it additional training data to make it more of a specialist.

2:50You can instruction tune it to actually follow directions and call tools and learn how to use the tools. So you can kind of think of an AI agent as, you have a machine shop with a bunch of tools on the wall. You've taken your LLM to school, it has a bachelor's degree, and then you give it a vocational degree, and it knows how to use those tools on the wall now like an expert. Fantastic. And then there are agents kind of popping up all over the place now, and there are agent stores and things like that. Is it a matter of kind of going out and finding the agent that you want, and then do you have to ensure compatibility with the models that you're using?

3:28Or is it designed so that, you know, you can grab an agent and put it into your workflow and these things generally work with each other? That's a great question. So I would say that one of the biggest challenges to AI agent adoption right now in the industry is a lack of standardization. Okay. So things have been standardized in terms of the approach, tuning agents to be able to use tools and to follow instructions. But where we lack standardization is in the way that the agents communicate with one another and the way that they store their actions and represent it in memory. That becomes important whenever you need agents to interact with each other.

4:12It can add a lot of friction if they're communicating with different protocols or storing their conversations in memory in different ways. So typically you can grab an agent from, let's say any vendor off the shelf and apply it to some task. But then when it goes out into the real world and starts interacting with other agents, that's where things become tricky. Gotcha. And so how widespread do you foresee agentic AI becoming? Is this the future or at least the, you know, sort of near-term present future? I think yes. I think that the vast majority of tokens generated by large language models will be to enact and to serve agent communication.

4:51So agents do two things. they reason and they call tools and communicate. I guess that's three things. So that act of reasoning, it's almost like talking to yourself. It's generating a lot of intermediary tokens. So if you look at it in terms of inference, a single agentic workload will usually generate a lot more tokens than a corresponding, let's say, inference chat workload. And the way I think things will evolve will be a lot like computational finance. Not too long ago, the vast majority of stock trades were conducted by humans. But in the world we live in now, probably 75%, 80 of them are conducted by machines with other machines.

5:34And so if you look at that ratio, I would say we're going to see the same thing where the vast majority of LLM inference will be between agents and not involve humans. Right. And what will they be doing? Well, the agents will be doing many of the things that human workers do right now, or maybe that human workers would like to do, but they don't have time to do, or things that human workers don't like to do, which I call toil. Toil is very, let's say, repetitive, error-prone tasks that, you know, they're not creative tasks, they're not necessarily productive tasks, but they take up a lot of our time.

6:11That is probably where we will see the first stages of agent adoption. Right, which I'm all for. Yeah, freeing people from doing the kind of busy work that fills up a lot of their time and leaving them available to do the higher value work. Right. And then we've had, you know, dating back several years now, when I think about it, guests come on to the pod and talk about the metaphor of an orchestra conductor or a trained, you know, train station conductor is used a lot where the human will be assigning those tasks out to the agents, the AI systems, and then almost being like the manager, right?

6:46and you've got a fleet of agents going and doing the work for you and they come back and you review or you kind of prompt it to take a different tack or that kind of thing. Is that still a viable metaphor? You know, I'm not sure. I think it's very supportive of our human egos to believe that we'll be in the best position to kind of conduct the orchestra. That's not clear. So probably what will happen is that there will be teams composed of carbon people and silicon agents, and they're collaborating on tasks. And at various times, the humans will be conducting the orchestra, and at other times, the orchestra will be conducting itself.

7:29And that might be the most efficient way to get the work done. But I do think it's a comforting metaphor. Right, right. No, that's well said, and it was interesting when you said that, It was one of those, I wasn't expecting that response, but it made perfect sense, right? Because if this keeps going the way it's been going, agents are going to get pretty smart pretty fast. They will. And I think that there are unique characteristics of humans. I mean, now I'm getting way outside of my role as a product manager at NVIDIA, and I'm just kind of philosophizing. That's fine. I think you called me a carbon human a few minutes ago, so I'm good with it.

8:06You're carbon, right? You're carbon. You're not a digital human. No, but I do think that, again, if we go back to the finance metaphor and computational finance, there are often algorithms that rebalance portfolios. Right. And maybe a human spot checks them, or maybe the human uses their intuition and experience to recognize some unique situation where they need to stray from that path. Right. And maybe the human knows some information or has some intuition about their individual clients. Well, maybe this risk profile is not right for this client. So there's always – human judgment is critical.

8:46Human strategizing is critical. And there's always room for that. So it's a way to complement the things that we're very good at with some of the things where we could use some help. Yeah. Toil. Toil. All right. So let's kind of flip perspective for a minute and talk about some of the challenges in the enterprise and in an organization when it comes to adopting agentic AI. Yes. So how much time do we have? Well, no, it's a great question. So my primary role is to bring generative AI to every industry. And there are challenges. There are technological challenges and there are social challenges because work always occurs in a social context.

9:28So on the technology side, we touched on this a bit earlier, but the lack of standardization across agent implementations can make agents working together very inefficient. And this becomes a big problem because this work can be costly and enterprises want to extract the most benefit from their investment in the infrastructure. So we have to make those communications more efficient. Standardization is one way to drive that. Another problem is that just like any language model, the work of an agent is not always deterministic. What that means is, you know, there's this notion of hallucination where if a model doesn't know the answer, it might make something up.

10:11And that can happen with an AI agent. And enterprises need deterministic business outcomes because you're, you know, betting your business on it. So that's another area where there's things we can do to increase that determinism and to add checkpoints and whatnot to ensure that it's there. The other source of, I would say, indeterminacy is that a typical LLM interaction, you can more or less judge how much it's going to cost in terms of the amount of tokens it generates. Well, when you combine reasoning, which is kind of an unbounded source of tokens, potentially, with complex problems, now all of a sudden, you can't really predict at times how much that seemingly innocuous question will cost you.

10:59It's like when you travel to Europe and you're using your roaming cell phone and you come back and you have a$5 ,000 bill. And so that's what we want to avoid. And so I think enterprises, they need deterministic outcomes. That's the technological side. On the social side, the autonomy of agents raises a lot of ethical and legal questions. Of course. When I first started working with enterprises, it was IT folks and security folks who would show up in the meetings. But lately, a lot of lawyers have been showing up and HR people have been showing up. So that's just kind of an interesting shift. And we'll have to see where that goes.

11:35Our guest is Jacob Lieberman. Jacob is a director of product management here at NVIDIA, where he leads a team building cloud-native Gen-I software solutions with a focus on accelerated storage platforms that can connect the enterprise and the enterprise data to all of the AI capabilities we've been talking about. Jake, we touched on this a little bit, but I think it's something worth digging into more here. This notion of agentic AI and autonomy and, you know, mentioning some of the systems where the human in the loop maybe checks in less often than some of the others because the system can go faster and the human can see, oh, everything's cool.

12:09And we're going, how far do we take that? How much autonomy should agents have? I mean, technically speaking, theoretically, how much could they have, but how much should they have? And what are some of the conversations that, you know, you've been a part of thinking about this? What's the current thinking about agentic autonomy? So that's another great question. This is a frequent question that we get and a cause of a lot of concern for people. This notion that we're going to kind of turn these agents loose on the world and they'll be able to do whatever they want. So what I usually tell people is that just like human workers, the roles and responsibilities of agents require a range of autonomy.

12:51And in some places, in some roles, the agent needs wide latitude to kind of make decisions. For example, if you're a customer service AI agent and someone calls you up, they might have something missing from their order. They might have a customer service complaint. They might have this. They might want to know what the other options are. Sure. And you really need to be creative in how you address that. And probably the risk of giving that AI agent so much autonomy is relatively low. Right. So, and I guess then it's kind of a, let's say like a plot where you have autonomy on the X axis and risk on the Y.

13:30There are other scenarios. Say you have an AI agent that's responsible for rebalancing your retirement portfolio. Right. There, you don't want it to get very creative. You know, oh, we're going all crypto. No, you want it to kind of follow the tried and true formulas. And you can embed that level of autonomy and determinism into the actions of the agent. Right. So I think that the fear of these autonomous agents kind of running around is, you know, having fun and being wild. I think that is a bit unfounded. But there are situations where you want autonomy. Of course. Frankly. Is there a, I don't know, is there a best practice?

14:09Is there a formula, not formula, but for kind of figuring out when you're meeting with customers, potential customers, how do you kind of find that balance? Well, this is an area that's still pretty nascent. Yeah. So I would say we have a map. We have a roadmap. We know what we need to do because if you look at autonomy in the physical world, we've already seen these questions a bit with robotics. And NVIDIA, of course, we work with robots. We work in simulation and we work in the real world. So if it's an autonomous vehicle you're putting on the road, if it's an AMR that is in a factory, if it's a flight control system, an avionic co-pilot, we have AI agents that assist tractors in the field.

14:58All of those uses are governed by standards. and the standards basically assess the level of risk inherent in the task and the level of risk mitigation that you need. So what I expect will happen is that we're going to learn from AI autonomy in the physical world and apply those lessons to how we deploy AI in the enterprise. So to dig into that a little deeper with what NVIDIA is doing now with agents and working in the enterprise and elsewhere, can you talk a little bit about some of the things NVIDIA is doing in the space? Yeah, so this will be a bit of a shameless plug for the work my team is doing.

15:36Fantastic. Yes. I mean, after all, I'm a human agent and I need to feed my family. So the first thing that we're doing, the first thing NVIDIA is doing and that my team is doing specifically, is we're building blueprints for AI agents. and blueprints are reference architectures implemented in code where we show how you can take NVIDIA software and apply it to some productive task in an enterprise to solve some real business problem. And these blueprints are generally taken by global system integrators and service deployment partners, service delivery partners who take them in, adapt them to their own portfolio, differentiate and then take them out to our customers at scale.

16:20So for example, we have a blueprint for a digital human. The digital human can be made into a bedside digital nurse, a sportscaster, a bank teller with just some verticalization. I'm grateful you said sports and not pod. Podcaster. No, no. That's too far away from our capability. Yeah. And then also where these AI agents start to intersect the physical world, the thing I was just talking about. We also have blueprints for teaching agents how to work in simulation and then deploying them into the physical world. Yeah. So that's very cool. And then the other thing we're working on, of course, NVIDIA, we're an acceleration company.

16:58We make things run faster. The other thing we're doing is we're working with this diverse ecosystem of agent platform builders, and we're trying to make sure that they all run great on NVIDIA software. Both the inference piece at scale and the distributed communications. So it's really those two things. What are you hearing from developers? What are they excited about when it comes to using Agentic AI and the work they're doing? It's very interesting and not surprising that software developers are among the earliest adopters of agentic AI. Now, a lot of the focus of agentic AI up until this point has been on consumer products, but developers have kind of adopted these technologies at a very rapid rate.

17:46And it's really interesting when you watch people program now, they kind of co-develop with the AI agent and they'll say things like, document this for me. How could I do this differently. Okay, take this code and encapsulate it and make it multi-user. So they're actually using human language to ask the AI agent to perform some tasks, whether they can do it themselves or not, is kind of immaterial. It's just more efficient to at least use the agent as a starting point. Right, right. Reasoning has kind of become a little bit of an it-word over the past few months and when it comes to all of this stuff.

18:22What does it actually mean? And we We can stick with the developer context maybe to talk about it. What does it actually add if I'm a developer and I'm coding and I'm using a coding co-pilot to do the things you're just talking about? If the co-pilot has reasoning capabilities versus not having them, what material difference might that make? So reasoning and the use of reasoning with large language models, it's kind of become the dominant paradigm and use case for large language models and agents in enterprise. Is it the default at this point, basically? It's not the default. No. Because the more time you spend thinking about something, the longer it takes, the more latency there is in terms of your reaction.

19:06And in some use cases, it's appropriate. In some, it isn't. But where it's appropriate are doing things like, let's say, biomedical research. We have a blueprint that can be used to simulate developing molecules. And so one of the things our customers can do is take a reasoning model, attach it to all of their private research data, attach it to all the public research data on PubMed on the internet, come up with a unique molecular design, and then pass it into our simulation software to see if they can build it and make sure that it's still in the world. stable. So there you don't need a real-time interactive response.

19:43You're okay if the LLM goes off and thinks for a while before it comes back with an answer. So the actual capability that's emerging is something that we're calling test time compute. Test time compute is system two thinking. It's thinking about thinking. The model will look at the way it's solving the problem and decide if it's doing it in the most efficient way or in the best way, in the optimal way. And it's fairly interesting. You can actually watch the reasoning models think through a problem. And you asked about developers specifically. I saw this one reasoning model interacting with one of my coworkers, and he said something to it like, I'm a product manager at NVIDIA.

20:25Look at this GitHub repo and explain it to me as though I were five. And then the reasoning agent actually thought, well, this will be difficult to explain to a five-year-old, but this guy is a product manager at NVIDIA, so clearly he's not five years old. So I will up my language a little bit to be more appropriate for his level. Oh, that's amazing. Yeah, it was actually interesting to see the reasoning model work this out. Maybe we can dig into some other use cases. You were talking about development, but where else are we seeing, are you seeing agentic AI systems being used in enterprise? Sure.

21:01So we talked about software development. We talked about research. Research can span things like summarization. Let's say you have a bunch of complicated documents, a long email thread, a long Slack thread. You could go through and read each one of those things, or you could ask an AI agent to summarize it for you. You could ask it to prepare structured reports for you. So these are two variations that both require reasoning that are becoming dominant use cases in enterprise. Now, another really interesting thing we're seeing in enterprise is that people are rewriting solved problems, applications that already solve problems through automation in this new format, because they want to take advantage of the natural human interaction.

21:50So for example, let's say predictive failure analysis. You have an offshore oil drilling rig. It's very costly if a component breaks. You have all sorts of telemetry that will let you know if something's about to happen. You could implement that predictive failure analysis with a traditional data science or machine learning approach. But if you use a large language model and if you use an agent, now you have the capability to interact with it in a natural way. So that you can use that to plan the responses, be alerted, trigger all of the logistic actions that might come into place to circumvent that failure.

22:29So I would say those are probably the three. There are very few unique use cases for LLMs, but the technology is so powerful that we're basically re-solving all of the solved problems in order to do it better. To do it better, yeah. You mentioned earlier that the lack of standardization was a challenge in adoption and development when it comes to the agents in general, but also how they communicate with one another. Kind of stepping back from that a little bit, are agents being used kind of more broadly in cybersecurity and risk management applications? They are. So within NVIDIA, we do a lot of work to make sure our software is secure.

23:12Of course. And we use AI. We use AI throughout NVIDIA. We use AI to design our GPUs. We use AI to write our software. We use AI to make sure our software is secure. And then we will often package those approaches and those learnings as blueprints that we can re-deliver to our customers. And one example of that is that we have a computer vulnerability assessment and analysis pipeline. so that if we have a bit of software and we're alerted that there's a vulnerability in the software, the AI agent will actually look and see if our code paths that trigger that exploit are executed. And it will make an assessment of how much risk we're exposed to.

23:50And it will also recommend how to remediate the problem. And so that assists our human worker, Christina, who does all of this work and it should make her work more efficient. So it's a great example of how a very applied thing that's potentially error-prone but important, where we have an expert human who is basically using the agent to give her more leverage. Right, right. So kind of as we start to wrap up here, advice for listeners, I'm sure there are listeners who are in enterprise situations and thinking about, you know, how do we start designing and developing deployings down the line? But thinking about agentic AI for whatever the use case is, you know, for where they're at, what advice would you give them?

24:39Broad advice right now, best practices for designing, developing, deploying agentic AI in these situations? So this is the advice I would give to anyone. And in fact, I try to follow it myself and I give it to my team is that soon this will be the way everyone does everything. We have to start getting familiar with these tools and these approaches just broadly. So are there any that you like to have, you know, consumer facing in a broad sense, but AI tools that you're using regularly at work, not at work that, you know, you're just, you're a fan of? Yeah, so NVIDIA, of course, we have many partners, and I use a lot of their technologies in my personal life.

25:24In fact, I recently presented to a group of energy investors and energy professionals about the sustainability of AI, and I used a lot of the tools, the blueprints that my team built to help me prepare for that conversation. So, for example, we have something on our Blueprints page called PDF to Podcast, and you can give it a bunch of PDF documents, and it will generate an engaging monologue or a dialogue. It could be a conversation or a debate so that you can listen to it in your car and familiarize yourself with the content. With the material, yeah. Right. So the entertainment value is not what you get here, clearly.

26:04Right. But it's useful. You're all weak, folks. Yeah. No, it is though, right? Because you can then, you can, you know, we all have earbuds in our ears all the time anyway. So why not be learning what you need to learn for your next meeting? Right. And so another tool I really like is Perplexity Pro. Perplexity is, it functions a bit as a search engine, but it's also agentic in that it will generate net new content. It's not finding content, it's generating new content. So you can ask it questions. For example, when I was researching sustainability, I asked it to develop a report for me on trends between the top 500 list of the world's fastest supercomputers and the green 500 list of the world's most energy efficient supercomputers.

26:47And are there any crossovers and what are the trends you're seeing? And it prepared, that didn't exist anywhere. It made that report for me. Net new for you, yeah. Yep. And for the developers, you know, I think Cursor is very powerful. cursor will kind of give you a development environment where you have an AI assistant that you can interact with in natural language. And it's kind of a visual basic plugin and it will assist you as you work. So I'd say those are three that I use pretty much every day. Fantastic. Jacob, for listeners who want to learn more Gentic AI, Gentic AI and the enterprise, the work your team is specifically doing, where would you send them?