How Visa Is Making Payments Safer and Smarter with AI - Ep. 256

0:10Hello, and welcome to the NVIDIA AI podcast. I'm your host, Noah Kravitz. AI's impact on the financial services industry has already been quite significant. From improving customer satisfaction and loyalty to streamlining operations and reducing costs, artificial intelligence is transforming an industry that's already been at the forefront of technology innovation for quite some time now. Visa is leading the way in leveraging AI to transform payment experiences. And with us to talk about how they're doing it is Sarah Laszlo. Sarah is Senior Director of Visa's Machine Learning Platform, where she's architecting the infrastructure that will power Visa's future.

0:46Before joining Visa, Sarah, well, her resume is really impressive and too long to get into right now, but I have to mention she was on NPR Science Friday, which is pretty cool. Sarah, welcome. Thank you so much for taking the time to join the AI podcast. Yes, thank you so much. So before we get into all the stuff you're doing at Visa, all the stuff Visa's doing, would you tell us a little bit about your own journey, maybe how you got started in AI and how you wound up in the role you're in now? Yeah. So one thing that I always like to remind people of these days is that, of course, everybody thinks of artificial intelligence as something maybe that computer scientists do.

1:22But we need to remember that the backpropagation paper that opened the field back up again after sort of the AI winter was, yes, Jeff Hinton was on there, but it was also co-authored by three psychologists, David Rummelhart. And I came up through that tradition of psychology. Oh, no kidding. Yeah. So my PhD is in psychology and I became interested in computational neuroscience. There were many steps along the way, dot, dot, dot, but now I still do it. Very cool. Did you practice as a psychologist? I was not ever a clinical psychologist or therapist. I was a cognitive neuroscientist. So, you know, I worked with human research participants, but never in a therapeutic context.

2:01Got you. And so was computational neuroscience kind of your pathway leading to where you are now? Yeah. So my postdoctoral advisor, David Plout, was appointed in both psychology and computer science at Carnegie Mellon. And my academic grandfather in that line is Jeff Hinton. So most of the people who were in that line now do this. Because that was sort of, there were not that many people in 2009 that were doing deep learning. And so if you are one of those, you're one of the only 20 people that have 15 years of deep learning experience. Totally, totally. Incredible. I kind of hinted at it and we don't want to talk easy.

2:39We want to talk about what you're doing. How did you wind up with this role? Yeah. So I was working in responsible AI at Google and I did that for a couple of years. After doing that for a couple of years, I decided that I wanted to get out of California. You know, I was 40 years old and I was making a lot of money, but I didn't own a home. My family was in Texas. I really wanted to get to Texas. But I still wanted to be sort of doing hard engineering problems, doing artificial intelligence. And Visa has actually a huge presence of our AI ML team in Austin. So that's how I ended up at Visa. So let's get into that then, into Visa.

3:18What are you doing with Gen AI? What are some of the exciting things, big problems you're solving? What's going on behind the scenes? Yeah. So when Visa generates new technology, like we're always first trying to find ways that we can prevent fraud. There's so much attempted fraud that even though we're very good at it, even marginal improvements in that actually prevent like large dollar amounts of fraud. So we're always looking at, okay, there's a new computational technology. Can we prevent fraud better? That's always number one. Number two is we're starting to get into personalization for our card holders.

3:51So I tell this story. I won't name the card, but I have a card that has an app on my phone. And the offers that it gives me are ridiculous. Like it offered me a Lamborghini one time. I was like, this is, we can do better. We can do better than this. So we would like for your app to be able to like minimally not get offered a Lamborghini, probably for most people. Are we talking free Lamborghinis? No. I don't think, I don't think it was probably like get a free pair of driving gloves if you test a Lamborghini. So I was trying to sell me a Lamborghini, which was ludicrous. So we want those to be better.

4:20We would like to be able to make better recommendations. And then of course, like one of the big themes of NVIDIA GTC this year, everybody's talking about agentic AI. And we are very invested in the concept of agentic commerce, where we can have agents that have access to our credit card credentials that can make purchases for us. So the example I've been giving all day is, hey, agent, I want to go to Greece in August, do it. And it just books it for me. Is the travel example because everybody likes, lots of people like to travel, or is it because of the word agent and we just latched on to travel agent?

4:53I don't know because I didn't think that I, I thought I had come up with this travel example myself, but then I've heard every other exact here using the same one. I think it might be because travel is like, you know, if you're going to take a flight, there's only a certain number of flights you can take. If you're going to stay in a hotel in a city, there's like the space that the agent needs to navigate is pretty constrained and it can know like, do I have Marriott points or do I have Hilton points? It can know that. So it seems like it's something that's tractable as the first problem. I think that's part of it.

5:24That makes sense. So how far along is that development? When can I have an agent book me a trip to Austin and Alambo? Right. So we're still closer to the recommendation part, like recommending offers, recommending products. Right now, what we're working on is trying to figure out the right sort of user embeddings so that the agent can have access to an abstract representation of you, the cardholder, without having access to your private data. Right. And so balancing that is where we are right now. Right. Got it. I know financial services is a heavily regulated industry, so there are lots of challenges that go along with that.

6:02When thinking about and actually developing and deploying Gen. AI apps in this space, what are some of the other unique challenges that folks working in other spaces might not know about when it comes to financial? Yeah. So one thing I've been talking to folks a lot about at GTC is that because we are so heavily regulated and because Visa in particular has such a strong commitment to privacy and security of our cardholder data, we have been slow adopters of cloud. So a lot of other, you know, Visa-sized entities that are trying to adopt Gen AI are very likely going to be getting their GPUs through cloud.

6:38There's a lot of reasons for that. But we don't do that. We own and operate our own data center. And that then puts a lot of constraints on the kind of soft and hard infrastructure that we can have and therefore the kinds of models that we can have. And that is something that other enterprises of our size probably don't face. Right. And so do you get around that by just building things that fit your needs? Yeah. So actually a lot of what my team is responsible for is figuring out how to squeeze what we need out of the GPUs that we have. Yeah. Yeah. That's fun. I love that, actually. That's the funnest part of my job.

7:12Excellent. Do you have any best practices to share around that? Yeah. So one of the things that we have had a lot of success with is we've started using virtual GPUs. So we are able to isolate memory and compute into separate virtual GPU instances on a single card. And the reason this is helpful is that very often our users request more GPU than they actually need. And so if we can actually, instead of giving them a whole GPU, give them a tenth of the GPU, it really improves our cluster utilization with very little impact to the user experience. Right. Is virtual GPUs or are virtual GPUs kind of a standard thing or becoming kind of common practice now?

7:55I think they're getting, I hear a lot of other enterprises of our sizes using that because everyone has this problem. You know, it doesn't take a whole GPU to run a Jupyter notebook, but people use whole GPUs to run Jupyter notebooks. And it's inefficient, it's cost inefficient, and it's wasteful. So I think it's getting to be more standard. Yeah, no, that makes sense. You mentioned personalization, cardholders. That's kind of, you know, the tradeoff, right, when I think about it from the consumer standpoint. And not with Visa or financial services, with anything. Is, okay, if I give you my info, I give you my data, like, how easy are you going to make it for me?

8:28I would imagine you mentioned, you know, that working on the abstract representation so the agent doesn't actually see my card number or what have you, right? How are you thinking about and what's Visa doing and down the road to provide personalized experiences? What are some of the, I don't know, some of the cool things you're working on? Yeah. So one of the best things about Visa and one of the biggest advantages we have is that we have an incredible data set. I can only imagine. You can only imagine. And I think actually maybe more so than people are with the data, like consumers or users are with the data that they provide to other large entities, I think people are pretty comfortable with Visa having their data.

9:04I think everybody sort of uses their credit card and they know that like Visa has been around for 100 years and that we keep the data very safe and that we don't use it for lots of extraneous purposes. And I think people feel comfortable with that. So we have that data set and we are using it to make large models that can have very good abstract representations of consumers. The reason the representations can be good is because of the magnitude of the data that we have. Nobody else, so I come from Google Research, I worked on petascale computing at Google. And so not often did I think I was going to say that I was going to work somewhere else and have access to a data resource that Google doesn't have, that nobody else has.

9:47But we have that. And so that means that the quality of the embeddings can be really high because the magnitude of the data is high, the granularity of the data is high, the computational resources that we can bring to bear to process the data is high. The data scientists that we have are some of the best in the world. And when you put all that together, you can come up with a consumer representation that outmatches what anybody else has. Sure. The challenges with personalization, I think, are we don't ever want to expose the raw cardholder data. So like the raw data is probably like, you know, any situation may be the most powerful because nothing has been removed from it yet.

10:19It's still exactly what it was. And so probably one of the big research challenges right now is figuring out like how to re-represent the data in a way that's effective, but that obscures the private information of the user. Is that as big of a challenge as it sounds like? It's very hard, yes. Because, you know, it's exactly the trade-off. It's like the more obscure the data, the personal information is, like maybe the less effective the representation is. And so this is an exact tradeoff and it's hard to figure out how to, it's hard to figure out what to do. We have a whole research team working on that.

10:48Right. Since we're audio only for the audience, Sarah just made a beautiful graph with her arms. You couldn't see, but it explained everything perfectly. Yes. If you could have seen it, you would understand what I was talking about. What are some of the other things that are either happening or, you know, down the road in terms of personalization? Yeah. So recommendations for offers, also recommendations for product. So one of the things, again, about our data set is we don't just know like what ads you clicked on or like what websites you looked at. And so that means that we can make better product recommendations probably than anyone else can because nobody else knows that.

11:23Right. We were talking a second ago about virtual GPUs. As you alluded to with the data center, right, you're working at incredible scale. What are some of the things that you've had to do innovations you came up with, challenges you got around when it comes to, you know, just scaling up and offering reliable service? Yeah. So especially as we move into sort of generative AI era, the models that need to be trained are larger. That's one part of the problem. But actually, the bigger part of the problem for Visa is dealing with the data hungriness of the new models. So we have, you know, trillions of observations in the petabyte regime size of data.

11:59And we want to be able to bring as much of that to bear as possible, but that's obviously a very high memory intensive process. And so part of the innovation that my team has had to be doing lately is finding ways to like process data efficiently over distributed GPU resources, keeping GPU utilization nice and high and consistent, like trying to stream data where we can, trying to lazy load data where we can, trying to use just-in-time techniques where we can. So as a platform engineer, not only does my team have to figure out how to do that, but then also help our data scientists do it too. And so there's the challenge, there's sort of twofold challenge.

12:38One is how do we do it? And two, how do we get our folks to adopt it? So there are any examples of like big internal wins, kind of a, you know, use case that you can share with us talking about how generative AI has just really made a measurable impact? Yes, absolutely. So, you know, Visa has been doing AI and big data work for 50 years, which means there's some code hanging around in Visa that's like pretty old. And we recently determined that there was a big code base that was still revenue generating, but that was no longer maintainable because it was written in a language, and I won't say which one, it was written in a language that was no longer supportable.

13:15And so we had a big problem because this was still revenue generating. It was no longer maintainable and we wanted to convert it to Python. But because it was written in such an old language, we didn't have anyone in the company who could both know that language and Python. And we were like, oh no, what are we going to do? And this problem was given to my team. And so we ended up using GPT-4 to convert the old codebase into Python. This saved us$5 million. And we had one engineer do it. He converted, I think so far he's done 50 jobs. He did like 50 jobs in a quarter, which is crazy because he was averaging like converting like almost one of these jobs a day at a sustained rate for like at production quality at a sustained rate for a whole quarter because we were able to use these Gen.AI tools.

13:58And as I said, that saved us about$5 billion. Fantastic. Yeah. One guy with a transformer model can do a lot. I'm speaking with Sarah Laszlo. Sarah is senior director of Visa's machine learning platform where she's working on infrastructure, architecture to power Visa's future, which as we've been talking about is big wide feature that touches a lot of people every day. There's this concept of the AI factory and it's picked up some steam recently. What does that mean to you? Yeah. So what that means to me is a single pipeline that goes from a data scientist with an idea about a model they want to build all the way to the model running in production.

14:40That's what it means to me. I like that. Is Visa in on this model? Is it a new name for something you've already been doing? When you think about kind of where this is headed inside of Visa or broadly for that matter, you know, what comes to mind? Yeah. So it's interesting because I hadn't really thought much about this AI factory terminology like I had heard it. But I hadn't really thought it was what I was doing until I came here to GTC and I started hearing other people talking about it. And then I realized, oh, that's what my platform does. So my platform, recently we've adopted what we call the Ray Everywhere strategy.

15:16So we use AnyScale's Ray ecosystem to do the whole thing, the whole shebang. So data conditioning, model training, and model serving, we do all in Ray. And it is intentionally trying to be more of this factory concept where there's not a whole bunch of distinct parts or distinct tools that are living in different places that work differently. It's just one unified, consistent pipeline from start to finish. Right. Is the notion of standardization in an AI world, I've had some guests on the pod before talking about that, and some of them have said, like, this is a big thing we're kind of grappling with now.

15:54What's your take on that? And are we headed towards more standardization or how might this play out? You know, it's an interesting question because I'm sort of of two minds about it. I'm of one mind about it as a platform engineer for a Fortune 500 company. I'm of a different mind on it as a individual who works just because I'm a nerd on AI projects and my side. Like I go home and I do this some more. As a platform engineer, yes, it's a big deal. Like because like in the Visa case, a lot of the models that we already have, so not even looking into future models, the models that we already have that we already have that we already work with big customers for, the faster we can refresh them, the better they are.

16:34Part of the reason for that is that Visa and fraudsters are in sort of a constant cat and mouse game. Like it's, you know, we do something, there's a countermeasure, we have to have a countermeasure for the countermeasure, countermeasure for the countermeasure, and it goes on forever. And so the faster we can refresh things, like the more fraud we can prevent, the better it is. Standardization helps with that. On the other side of it, as an individual practitioner that's been doing this for a long time, like I have been a person that has said many times, like in my courses, that I call it machine learning still, really.

17:05I'm old school. Machine learning is more of an art than a science. And so there's a part of me that's the individual practitioner that likes the sort of artful and surprising and creative endeavor. I find machine learning to be a creative endeavor. And I will miss that. I think it'll make me sad if it becomes a fully factorized kind of process. Right, right, right. Yeah. No, I like that. That's the notion of continuing to do it when you get home from work. I've heard a lot of that lately from the folks we talked to on this show. So what's the adage? If you love it, it's not a job. It's not a job.

17:39And right now is such an exciting time. Like things that I dreamed of doing years ago, I can do now easily. It's like I can't get enough. Like what? Is there a big one that comes to mind? Yeah. So I worked in image generation when I was in Google and just the improvement that has come in image generation from. So I worked on Imagine One. The difference between like the quality of something like Imagine One or Dali One compared to like what MidJourney can create now. It's astounding. It's incredible. Blows my mind. I can't get enough. Awesome. Have best practices emerged when it comes to kind of the broader picture of I'm working in financial services.

18:17is somehow I've been tasked with thinking about gen AI. We got to do it. Everybody's doing it. We need to get in. But like, what do we do? What do we do internally? How do we think about our customers? All of that. You've touched on a lot of this, you know, here and there as we've talked. But do you have kind of learnings that, you know, to share that someone else? Yeah. Yes. People have been asking me about this all day and all week because we are sort of at the front of this. And I hope people will be able to learn from our mistakes. We had a guest recently started an eruption that came to mind who said that, what is it that like trends only happen because some people figured out, like started doing it before it was a trend, right?

18:56So you have to be there. You have to be there. And hopefully we sort of have broken the trail. And so it'll be easier for the people that come behind us. One thing that I've been saying to people is that don't discount open source models. Like using open source for everything that you possibly can, I feel like is a good practice. I was going to ask, is Visa building models? Are you customizing like open weight models or? All of the above. We build our own, we fine tune and rag open models and we use proprietary models, all of these. One of the best practices is that we do use open source for everything that we possibly can.

19:29And that's not just true for AI. That's true. You know, we use open source Kubernetes, we use Red Hat Linux, like we use open source. That's part of our, we're huge actually. Visa is one of the, you wouldn't, maybe your listeners didn't know this. Visa is one of the largest contributors to the Apache Open Source Project. Yeah, like we're huge Apache contributors. So open source as much as possible is a good practice. I think it is very important for governance and tech to have a good relationship from the start. When that is not true, like things can go badly. So I think like making sure that governance and tech are talking to each other all the time from the very beginning.

20:05Like as soon as your organization has an AI governance, an AI governance function, they should be talking to the tech people. Don't wait. Start that right away. I think that's an important practice. I do think that being very aware of what models can and cannot do and what is and is not safe for them to do is important. There needs to be some employee education about like, you know, don't type your tax return into the AI. Right, right. Like people, they're excited to play with the tools and they do all kinds of stuff. But there needs to be some like, here's how you do it safely, I think. I think that's important too.

20:42Sarah, this has been a pleasure. And one of these episodes, I think, where there's just so much that you've said that can be kind of taken and examined and looked at. So this one's going to have a lot of replayability, as we say. So it's fantastic. Thank you. Before we wrap, for listeners who want to know more, is the best place to go just to go to the Visa website? Yeah, Visa website is good. Recently, our CTO, Rajat Taneja, has done some really illuminating interviews with The Wall Street Journal, where he talks about his vision for us in the future. So just look up Rajat in The Wall Street Journal and you can hear about his vision too.

21:15Fantastic. Sarah Laszlo, it was a pleasure to speak with you. Thanks again for coming on the AI Podcast. Thank you.

21:35Thank you.

22:03We'll be right back.