Capital One’s Prem Natarajan Shares How AI Can Enhance Financial Services and Customer Experiences - Ep. 253

0:10Hello, and welcome to the NVIDIA AI podcast. I'm your host, Noah Kravitz. The financial services industry has long been on the forefront of technological innovation. Today, Generative AI is redefining financial services, from customer service agents to AI factories enabling the next wave of industry innovation. As organizations move beyond testing and experimentation to successful AI implementation, these new technologies and ways of working are driving business results. Our guest is working on the leading edge of bringing the power of AI to financial services. Prem Natarajan is Executive Vice President, Chief Scientist, and Head of AI at Capital One, where he leads the technology strategy, architecture, research, and development for all AI initiatives, as well as all data technologies across the company.

0:56Prem, welcome, and thank you so much for joining the AI podcast. Thank you for having me on the podcast, Noah. Can we start with a little bit, kind of an overview of your role, and then maybe you can talk about your approach, Capital One's approach to using AI? Yeah, delighted to. You know, I started my life, my professional journey after finishing graduate school in a DARPA-sponsored world. It was a company called BBN Technologies, which was one of the pioneers in speech and language technologies at the time. Okay. Which was really the big application of AI and machine learning, if you think about it back, like, you know, it was all tied with speech, language, and computer vision.

1:35Right. These are the three modalities that humans interact in. And it was also big on internet working, but my area was always in AI and machine learning. So after about like two decades in this DARPA-sponsored world, and back in the 90s and the first decade of the 2000s for much of it, DARPA was probably the biggest sponsor of research and development in AI and machine learning. And we were standing on the shoulders of the outcomes of many of those programs that they sponsored. I then went to USC, where I was for several years, University of Southern California. Then I took a leave and I went to Amazon to be part of the Alexa organization.

2:16And at some point, I led the Alexa AI organization for a while and then came to Capital One. So throughout my professional career, though, I've been very close to the intersection of research and transitioning or translating those research advances into products and capabilities that benefit users in their everyday life. Because DARPA itself is very mission-focused research. You're making fundamental advances, but you're also making advances that take that technology towards solving a set of needs, like real-life needs. The real-world use, yeah. Real-world use. And so I've always been inspired by being at that intersection, right?

2:55Like advancing the state of technology, but then advancing it in a way that then humans and other users can benefit from it and it brings some joy and magic to their lives. Journey to Amazon was very similar with Alexa. It really felt like bringing AI into everybody's lives in a way. And now at Capital One, I feel like we have the opportunity to continue on this journey. Capital One has been on this journey for a while. It started with their centricity on data and the value of data in making informed decisions that improve the lives, the financial lives of our customers. And so not getting into all of the details in between, but today we are kind of standing on the shoulders of three decades of work at Capital One and saying, how can we leverage all of that stuff, all of the investments in data, all of the investments in technology and bring to life a new generation of experiences for our customers?

3:47So once again, from a personal perspective, I feel I'm standing at this intersection of how do we advance AI in that last mile to be just optimized for financial services, financial applications? And then how do we bring those advances to life in the form of experiences for customers? And so that's what I'm super excited about right now. And because Capital One is such a platform-centric company, all of our investments go into essentially, metaphorically, a rising tide that lifts all the boats. Where in this ocean, all the boats are the different applications and products and services we want to deliver to our customers.

4:30Fantastic. Okay, so generative AI obviously has been a hot topic in the world, let alone the AI industry for a few years now. And agentic AI and agentic frameworks has really emerged again in the past year or so as kind of a topic of public discussion. I'm sure you've been thinking about it for longer. But can you talk a little bit about how Capital One's using Gen AI, maybe using agentic AI to better serve users? Yeah, you're right. We've been thinking about agentic AI for a while. I mean, in some sense, the only way you get to be at the forefront of a trend is you were kind of working on it before it was a trend.

5:03Sure. And that holds true for our work in leveraging more classical statistical machine learning, which I think has a pretty long life ahead of it still. But then what generative AI has done for us is it's opened up new areas of the waterfront, if you will. So if you think of all these customer interactions of the waterfront and classical machine learning, classical AI might have allowed you to do a much better job of serving your customers in one part of the... But the exciting thing with generative AI is not just that it's improving our ability to do things we've always been doing, but it's opening up.

5:37So there's a little bit of an explorer's feel to it, if you will, right? You feel like you're on this journey of discovery and exploration. When you're on the journey of discovery and exploration, humility is a good virtue to have in particular because part of discovery and exploration is also the openness to learning. because as you discover and you explore, you're going to learn some things that are new things for you. And then you adapt and you figure it out. So in generative AI, our whole approach has been, we start with a test, iterate, refine, test, learn, iterate, refine kind of loop. So I'd say the phase one of this whole journey for us is really been, and I'd say in many ways, in different threads of this activity, we are on that kind of ongoing loop of testing So there's a very thoughtful, deliberate approach to how we're doing.

6:30Now, one of the things that we've seen happening is we've been moving from knowledge gathering or information provisioning type of interactions with customers. And we're seeing the opportunity for actually taking actions in response to customer requests or like going just a little bit further and satisfying the customer's needs. Right. Because not all of their needs are informational. Right. Sometimes they want you to do something for them. And sometimes if you can use AI to do that, you can kind of have the possibility of meeting the customer where they are, when they want to be met, and in the ways in which they want to be met.

7:08So that opens up that set of possibilities. So our first deployment on that front in terms of direct to customer experience is this chat concierge, which is in our auto and financial services division. So we partnered very closely. It was close partnership with the folks in that division and us in the enterprise AI organization. And the goal there was to improve the experience customers have when they're shopping for cars. A worthy goal, generally speaking. It's a worthy goal. Most people buy a car before they buy a home. So at the time that they're buying a car, it's perhaps the biggest financial decision that they have to make.

7:48And it is, you know, in its own way, intimidating. And so anything we can do, I long believe that the noble aim of AI is to transfer the cognitive burden from the human to the system. And that AI, let me put it this way, AI is at its best when it transfers cognitive burden from the human to the system. Right. And allows the human to just have that much more fun or experience that magic. Right. And so the chat concierge allows the customer to come to the website. the dealer's website or the dealer's website you know, discover all the cars they have like interact with it in natural language very natural conversational interaction about finding the kinds of cars they want for their family, etc, etc but also take actions like, you know scheduling a test drive things like that and then we will see, you know, as I said it's a test and learn thing so this is our first agentic AI deployment direct to customer and we chose it in part because of the richness of the interaction, but we also chose it in part because it allows us to start at the lower end of the risk spectrum, if you will, so that we can manage our way through it, understand, learn, as we scale up to even richer interactions just within chat concierge and auto dealerships, but also for other applications.

9:08Sure. And so you develop, Capital One develops AI apps, technologies in-house. You have a department. Can you speak a little bit to, you know, the advantages and kind of the philosophy behind doing that in-house and developing proprietary technologies? Yeah, yeah. So a couple of things, you know, in terms of how we think about things, and maybe some of it is personal as well, but largely how we think about things. One is we're all in the business of serving our customers. Sure. And we want to provide them with the most satisfying, the most useful set of experiences and services we can to improve their everyday financial lives.

9:43Right. It's very hard to provide enduringly differentiated services if you're just going to take a bunch of stuff that's available and integrate it. Right. Because those things leave gaps in how the customer wants to be served because you understand your business best. Sure. And so you're going to build a solution that addresses those needs. And so that's one kind of organizing principle we have. The second is, you know, you talked about we have a department that has apps. Actually, what we have more is a combination of platforms and a community of developers who can develop applications, etc.

10:19So what we're trying to do is provide scalability through platforms. I mean, this is not a novel idea in the technology industry. And given that Capital One is essentially a tech company in a way, maybe the original fintech in a way, it's not surprising that we kind of embody that thing. But it takes tremendous leadership to stay committed to it. So I do have to say that we're blessed with leadership starting from the very top that's kind of committed to that way of operating. So you get efficiencies and then in managing risk, etc. So the third thing I'll say is the proprietary aspect that you mentioned.

10:56I think your data advantage is your AI advantage. And so proprietary data allows you to build proprietary AI that provides enduring differentiated services and products and offerings for your customers. Roughly how many customers does Capital One serve? Oh, 100 million plus customers, which is fantastic, right? Because anything we do, any improvement we make is immediately going to scale to all of these people in the place that is just so central to all of our lives, our financial lives. So much of what we're able to do, the freedoms we're able to exercise, the fun we're able to have, etc. It depends on that.

11:39We're speaking with Prem Natarajan. Prem is the executive vice president, chief scientist, and head of AI at Capital One. And we've been talking about Capital One's approach to developing AI, empowering developers, and as you said, the key to it all, really capturing and making use of all the data that a company like Capital One has from the history and all the 100 million customers you're serving every day. We mentioned agentic AI a little bit. Can we talk a little bit about multi-agentic conversational workflows? And kind of what are the practical applications when we're talking about banking and financial services in particular?

12:16Yeah, maybe this is where we kind of connected to all the general things we've been talking about so far. So we talked about proprietary data and its value, right? So part of what proprietary data allows you to do is to build custom specialized models that are expert in doing the different tasks that your company needs to do in order to serve its customers. You have lots of data about it. Nobody else has that much data about it because these are the things your company is doing for its customers, right? So that's one aspect of the proprietary data and the custom specialized models that we build.

12:48Second thing is, what do we need to do in order to be able to do those custom specialized models? Because we all know building language foundation models from scratch is an expensive business. So we said we need to have open source or maybe a more accurate term is open weights models. So we need to have open weights models that would allow us to do that kind of deep customization that we want to do with our data to make those models that much more performant for our applications. And then the third thing was this move from like kind of knowledge gathering informational things to actually taking actions and completing certain tasks, whether it's for our customers or even for our associates, like our employees, like things that they want to do.

13:31And so we started thinking about how do we bring all of this to life? And that's when we started building out our agentic workflow. Right, right. Now, agentic AI today, you know, what we really mean is that it's a way of bringing to life custom specialized expertise and be able to take actions within the context of the real world, that is the enterprise, to take actions. Good news for us is because we are such like a tech focused company, right? Our tech stack is built to be used by a lot of developers, which also means it's kind of ready for AI to exercise certain portions of it. And because of our focus on data all these years, we have large troves of data that we can use to do deep customization of these models.

14:19So our overall strategy is anchored on open source, open weights models using our proprietary data to build proprietary AI. That proprietary AI starts with these proprietary language models like customized with deep customizations. But then it also gets into this agentic workflows. And so the thing that the workflows adds on top of the data is the ways in which we do these tasks. I mean, it's always every company does similar things in its own way, in its own different ways, using its systems, bringing its values to life, its practices to life, etc. And so the agentic workflows kind of take all of this raw data, the deeply customized models, and then say, here's how we practice our business.

15:01This is how we do things. And so you can then compose different workflows and different task completion trajectories using these agents. And that's kind of the multi-agentic conversation workflow. Conversations, Noah, happen to be one of the things we do. Sure. Right? And it's also one of the most visible things we do from the outside because that's how you interact with customers. So the thing that we kind of bring to life first is this conversational version of it. But the agentic architecture is actually scalable to lots and lots of applications. So in thinking about, I mean, everything you're talking about and the models and the workflows and everything else, were there either big hurdles you cleared or challenges that are common, you know, and may be common to folks you've talked to at other organizations trying to do similar things?

15:49Kind of some learnings you might pass along to folks listening who are trying to set up, you know, at whatever scale in the organization. Yeah, I get what you're asking. Let me first just say, coming from the financial service industry, there's one set of requirements that we deal with that are maybe somewhat unique to this industry. Those are the regulatory requirements, et cetera. But all that means is things that we have to do are actually good things for everyone to do, right? Because we talk about responsible AI or responsible practice of technology. It really comes down to thoughtful, considered analysis of all the implications of what you're doing, right?

16:25Because we tend to fixate on all the great things that will happen. So you have to train yourself to also think of what are the non-great things that could happen. And so we spend a lot of time. So upfront for us, we are thinking of responsibility through design. Like you can't use it as an add-on at the end. You can't say, well, I've built the AI system I want to build. Now let me try to go and make it responsible. Give it a responsible hat. We're all good. So it's part of your design process. It's part of your build process, et cetera. So that's one thing people need to build. Like, how do you build this multi-stakeholder view and getting all the inputs from that?

17:00That's a cultural thing. And so I'll just say that people are starting on this journey. It's an important point. It's a huge point to make, though. Yeah, yeah. And it's crucial. For the advances you make to stick, it's crucial that you do it in that way. But when it comes to data, so one of the challenges is if you have a lot of data, is it ready? Is it curated in the form? Is it clean? Is it reliable? is it? So to me, the AI, the journey that we're on today actually started six or seven years ago with all the investments that were made in the data infrastructure and all of that. So to some extent, if somebody is saying, I want to start my, I'm not thought about, you're like, you find out a way to accelerate those five, six years first, right?

17:44Because otherwise you can't bring your business to life through AI. The second thing is customizing these models with your data is in principle and in concept a very powerful thing. But it turns out there is a lot of fragility around how these things are trained and how they respond to it. So you have to have the talent that knows, that is current in its understanding of deep learning and this new wave of technology, that knows how to design the training algorithms, the optimization functions, et cetera, to get the performance that we need. So what we find is it's not like, oh, I have data, I take algorithm X, I pump it through the data, I get this model and boom, I get this performance.

18:28It turns out there's a fair amount of science invention that needs to happen at the edge, I mean, in the last mile or the last two or three miles. So we've actually invested, I'm very proud of the world-class AI team we've built. And if there are folks who think they belong in the team, they should certainly reach out. Right. But it's a fantastic AI team. All of this agentic stuff, deep customization of these models, they don't just happen off the shelf algorithm. You have creative, talented people working at it day in and day out. So that's the second section. How do you bring the talent that is needed to address some of these last mile challenges?

19:04The third challenge I'll just mention, there's so many more talk about, is how you kind of structure you're thinking around guardrails. And you have to think about both technology guardrails and process or human guardrails, right? Technology guardrails, most people will think about, oh, there's a Lama guardrails models and then NVIDIA's Nemo guardrails, etc. All of which we use and we build and we customize those too. The wonderful thing with open sources is the ecosystem. It's not just the LLM. It's all these other things around it that you can build with it and customize and configure for your use.

19:34But also for a lot of our, for almost all of the things that we do right now, we have a human in the loop. You can scale the engagement of the human in the loop based on the performance of the technology over time. But the human in the loop is a critical risk mitigator for us, especially when we put things in front of customers. We want to make sure. The amazing thing is they become part of this flywheel. Because as the AI produces an answer and the human either tweaks the answer or says this answer is good enough, et cetera, you're getting this reinforcement learning flywheel going, which over time, it's one of the untold magics of the world.

20:10The flywheel. The reinforcement learning flywheel. If you can bring it to life, it's hard to bring it to life. You have to instrument your platforms, et cetera. How do you evaluate the performance of your systems? And are there specific metrics, ways of evaluating, best practices when it comes to AI and finance and banking in particular? Yeah. I think, look, we're so also early in this journey that we look at a variety of things to make sure we're getting it right. So the first thing we look at is the waterfront, as I said, is very large. And you can go rapidly and occupy any part of the waterfront we want.

20:46So what we look at is where are the places where there is potentially a lot of leverage to be had, but the risk profile is such that we know how to mitigate it, that we have high confidence in mitigation. In some sense, I almost want to prioritize things to build where we are very confident in our ability to mitigate risk, but are exploring the level of benefit we might get. Because you want to lead with making sure you're not going to create undesired impacts or consequences. So that's one part of our consideration. Like, how do we look at it? So we go through a pretty formal process of looking at use cases, looking at the risks associated with them.

21:25What might be the learning? So people talk about return on investment, but at this stage in it, one of the returns we're looking for is what will it teach us when we do this about how these things behave? What do we learn from those initial deployments? And how sure can we be in those learnings? We also look at customer impacts. For example, latency, in my mind, of a service, like how long do customers have to wait for something, is a major predictor of their engagement and their satisfaction. We customers get impatient. And increasingly so with, you know, you just have to think about your favorite live sports event and how unhappy you get when it starts buffering.

22:03Right. Right? That's a great analogy. So it's like we don't like latency in anything. It degrades the experience. But also in certain times, in financial services especially, latency is important from the perspective of getting your work done in the moment that it needs to get done. Right. And so we take it very seriously. So how do we manage latency? So we look at what is the latency of this thing? Can we keep it at the levels that we want, that we're happy with, that our customers will be happy with. We also look at metrics like NPS. How happy are our internal users with the things we've built?

22:37How happy are maybe end users with things we built, et cetera? With the fact that we have human in the loop, we have a built-in way to measure how good these things are. Right, right. And so it gives us very good insight into the rapidity with which we're making improvements in performance. So I think it's a collection of those things. The learning, the risk management, the customer experience metrics like latency or the accuracy of the information, et cetera. And then the human in the loop, which allows us to collect it all. Right, right. Are there specific AI-powered, AI-related features that you've heard customers clamoring for?

23:14Customers and internal users, employees. You know, people typically want things that make, I think I mentioned it, like one of the, I think AI is at its best when it's transferring cognitive burden from the user to the system. Right. So in general, as human users of things, We want something that's useful in the moment, right? We want things that reduce our burden in the moment, etc. Whether it comes through AI or something like that, like people are not specifying the ways in which they just have these sense, I want this to be done. But latently, they're clamoring for things. Like, for example, they want strong protection against fraud.

23:50And so we've been using machine learning for fraud defenses for the longest time. And people benefit hugely from it. and we can see how good it is. Now, the thing with these machine learning systems, as opposed to more traditional rules-based systems or something, is that because we learn from data, we can keep them current against current fraud, against evolving fraud practices, et cetera. So their performance remains robust, things like that. So people do clamor for it. People also want things to be available instantly, in a way. So they don't want to wait in queue for their questions to be answered.

24:23So I think we hear, like the clamor of the customer is always a latent one. You have to listen a little bit more carefully, right? It's because they don't come and say, here are my 10 things I'd like you to do. But you can see in the way you measure your satisfaction, where are the areas of dissatisfaction, et cetera. And so we know anytime they have to wait for something, it's not satisfying. So AI can unlock that avenue for them. They can get answers they need or tasks they want done when they want it done, in ways they want it done. So in a way, AI allows us to meet the customer when they want to be met, where they want to be met, in the ways they want to be met, right?

25:01And so that's the second area of like general applicability. Third is with associates. Where are the most burdensome? We are a financial services company, you know. We write a lot of reports. We do a lot of analysis, et cetera. Everybody is trying to do a lot of work. And how can we bring AI? And so they're all saying, how can I make my life not just less burdensome? I want to produce. employees are driven to produce the best quality products they can, right? The best report they're capable of writing. How can AI give them the ability to actually produce the best reports, the best analysis that they can do it?

25:38So those are all areas. And coming back to that agentic thing, I think agentic things are going to help us with all of these. All of these. All right. The new, we talked about agentic before that generative. Factories. This concept of the AI factory. Financial services, you know, Capital One, as you said, in a lot of ways is a technology company, right? Tech center company. Financial services has been, you know, pushing the boundaries of technology for some time now. And so this concept of the factory, centralized platform, hardware, software, tools, and you can create your AI apps. How is Capital One thinking about this?

26:14And you mentioned, you know, it's a platform-centric company. Yeah. Yeah. So does this notion of the AI factory, does that feel like kind of an extension of what you're doing? Is it a different concept? It feels a very natural and organic extension of our way of life in a way. You know, I like how, you know, Jensen like talks about it as a factory in the sense he's simplifying the notion to saying, look, we're all in business to produce something of value to others. And what are the tools of our production? And if you're in man-made products, then the tools of your production of the raw materials and your skills and the implements you use them.