How Walmart is using AI to update key functions like search

0:05Hello, everyone, and welcome to the Modern Retail Podcast.

0:30in terms of what's going on on the product level with Walmart, especially with generative AI, how Walmart is using technology to empower store associates. Those are things I want to dive into. But I also just want to talk about working at such a behemoth job for such a big company, because I imagine you are constantly thinking about dozens, if not hundreds of things at once. We'll get into this very shortly. But John, how are you doing? I'm doing pretty good, man. It's really, really good to be here, Cale. You know, it's, I don't know, we're in the middle of summer and we are approaching, as crazy as it sounds, but I know you know it in the retail industry, we're approaching holiday awfully quickly.

1:09So as a result, on my side, on the product side here at Walmart US, things speed up quite a bit. We have another quarter, maybe quarter and a half to get things done before the holiday season hits in full swing. So we're doing good, but we are definitely running about as fast as we can. Summer is not a holiday for us on the retail side of the world. Yeah, no, it's not. And that's one thing that we always talk about in edit here is where we do a lot of coverage about the holidays or, you know, consumer facing publications do it in October, November. But really, we should be talking with you in August and July because that's when you're actually making your decisions.

1:44That's exactly right. Anyway, so let's just first back up and start with you. Correct me if I'm wrong, but you've been at Walmart since 2022. Is that correct? That's correct. So it's been about two years now. What were you doing before? Let's see, before that, I've done a variety of work, all in the product management realm here in Silicon Valley broadly. Immediately before Walmart, I was working at Varo. I was leading up their product and technology groups. Varo is an online neobank, one of the only that actually has a banking license, so that was an awful lot of fun. Before that, I spent three years leading the ride-sharing teams across product, engineering, science, and the like for Lyft.

2:24and it was a great way to drive one of my passions, which is transactions and how do we match supply and demand of humans at such large scale around the country. And if we go back longer, I spent 13 years at Google where I led a variety of ads teams from search to mobile to shopping and travel, which is ultimately what gave me the passion, the excitement for retail. And it also created some connections between me and Walmart at the early days of my journey there that I've come back full circle to kind of pick up again here. Got it. So now that you're back full circle and you're in retail, how would you describe, what is the mandate of the chief product officer?

3:05Because it seems like such a huge role for a company like Walmart. Well, much like a lot of the other senior or leadership positions in product management, across companies, across the Valley, et cetera, it's our job to really act as a nexus point, to bring together, whether it's folks in design, engineering, the business, science, to solve customer problems that deliver against the business goals and the business outcomes at scale. I talk about our work largely as a multiplexer. We're translators and we bring people together so that the work that we do is much greater than the sum of the parts.

3:42In detail, that means things about understanding resources, prioritization, but it's a lot of looking at things from the lens of the customer? How does the customer look at this? And how does that customer point of view from a horizontal ultimately cut across all of the different verticals that we have in our business? So I have a team of about 500 product managers. All of the product managers in Walmart US don't necessarily sit underneath me, but I lead 500 people directly. And then as the chief product officer, I sort of set how we're going to think about product management, how we bring this point of view of the customer, the more that we can operate consistently as product management within, to your point, this very large organization, the easier it is for us to work with different businesses to make sure that we are building the right reusable components, systems, and ultimately, as I said earlier, solving those problems for our customers to basically make their lives better and help them save money.

4:39So did you have to implement a new project management, I guess, workflow when you started? I imagine things have worked a certain way when you're at a company like Walmart. When you come in, how do you change things or how do you iterate things? It's a great question. So yeah, there were existing, I don't know, processes, procedures set up for how we here at Walmart thought about product development and the overall process. I wouldn't say, I think it's important to know that when you come into a position like this, The last thing you can do is grab the steering wheel and just crank it, you know, 90 degrees or 180 degrees to the right.

5:15But there are some things, some ways to improve that we've wanted to implement over time. So I talk about it as a dovetail. My first year here was really spent listening, learning, listening to our associates, the business leaders who work alongside me, what's working and what isn't. And then I sort of mentioned it earlier. My goal isn't to dictate that this is the way that we do things my way or the highway, but it's how do we figure out the smallest set of things that we can do consistently as product managers across the board, whether you're in my team or you're in one of my sibling teams, so that that consistency drives for better overall customer and business outcomes.

5:54So we've been very thoughtful with the approach that we've been taking. One of the things that we've looked at is how do we better define the units or the segments of work that each of our product management teams is taking on by clearly defining and enunciating charters around these are the goals that we're trying to solve. These are the KPIs that are important. These are the people that are involved. As we understand those charters, we're able to federate more and more across the product management organization at large. So I'd say it's a very thoughtful and careful process. And we're looking to dovetail more than just crank the wheel 90 degrees.

6:30And so given that you're an alum of Google, and maybe this is completely outdated because I have not covered Google in many years, but one of the things that they did there was they focused very much on small teams so that you wouldn't get too many people involved in a certain product. and it would just get bureaucratically overwhelming, I imagine. How does that translate into, is that something that you try to infuse at Walmart? How does that work? Because when you're the biggest retailer in the world, there's a lot of bureaucracy, I imagine. Yeah, I don't know if I'd necessarily say bureaucracy, but there's a lot of points of view.

7:05And one of our jobs as product managers is try to integrate holistically, bring in all of those points of view so that all of the voices are heard. Ultimately, though, decisions have to be made singularly so that we can drive that direction and move forward. So in terms of sort of team size and honing that down, this is one of the things that charters as a tool will give us. It helps prevent overlap in the work that's happening, right? As we bring all of this information together, we can start to identify the distant work that's going on is actually quite similar. So instead of building wholly separate teams and running them separately and then building the same thing over and over again, how do we bring that work closer together so that we can build common, reusable pieces of technology and deploy them in those two separate places?

7:50So it's been a work in progress. I think Walmart's made quite a deal of progress. The scale with which we operate, we do have lots of people working for us. But the way that we're leveraging technology, product management, design at scale to unlock all of the associates that work with us is really quite remarkable. Can you talk a little bit about, because Walmart has many different components, like, you know, you could talk about the retail, you could talk about e-commerce, you can talk about financial service. Like, there's just so many different parts to Walmart, especially since, correct me if I'm wrong, but I was listening to an interview with you earlier, but you mentioned that you deal a lot with e-commerce as a whole.

8:28How do you infuse the e-commerce products that you're working on that necessarily or should be integrated into the retail parts? How do you make those two work together, especially when in-store has been around for so long and it's such an important part of the Walmart experience? We do have separate teams to some extent, right? I have an e-commerce team that's wholly focused on search, the site, everything related to e-commerce transactions. I have a separate store team that is working on managing, you know, whether it's associate tools in-store or things that are going to make customers' lives better in-store.

9:06But I think one of the really exciting things that Walmart has done over the course of the last four or more years, this happened before I got there, was not looking at the world in channels anymore, in e-commerce or online and offline, but really to put everything under one umbrella and talk about things from the point of view of omni-channel. So because we have this omni-channel point of view, this DNA almost, it's the way that not just myself but all of my leads thinks about operating. How do I bring things together in ways that ultimately solve these problems for the customer regardless of whether it's in-store, online, or what have you?

9:42And so we operate them pretty similarly. I keep these teams close together. We share roadmaps and what's going on so that we can constantly, I don't know, we can take what each other is doing and make sure that it's driven in our roadmaps, for example. I do have a great example here is I do have a team that focuses on last mile delivery, right? And this is ultimately consummating, it's bringing to life this physical in-store experience, but also this digital e-commerce experience. So that as a customer, you can buy whatever you would like online at Walmart. And if we have it available in a store, we can get that shipped to you the same day, oftentimes within hours.

10:25And we found that to be an incredible superpower for us that's a really unique differentiator as well. So it's really just about bringing these things together, seeing yourselves as omnichannel. And again, looking at it from the point of view of the customer more than anything else. Yeah. And so, and you sort of answered this, but I want to dive in a little bit deeper because this is a question I've always wanted to ask someone in your position because Omnichannel is, you know, it's a buzzword, but it's also very important. It's what everyone's talking about. But you have these two different teams.

10:52You have stores, you have e-commerce. If we're talking about last mile delivery, if we're talking about buy online, pick up in store, for example, how on an operational level, you said the teams are close together, but I imagine you need members of one team to talk to the other so that the roadmaps go in time so that they know what's going on and they know what the timeline is. So how are you just on an operational level making it so that these two teams work together so well? Yeah, I mean, I don't want to make it sound like it's not a challenge at all, but when you really start looking at things from the point of view of the customer and you start looking at even your high-level metrics just from an omni-channel point of view, you understand very quickly, and we're way past the early days of this stuff, you understand how important the e-commerce piece is to our merchants, for example.

11:43If the merchants want to be able to sell and deliver as much of their product as humanly possible and they just lean into stores, we're going to have a big problem. They're not going to light up anything on the e-commerce side of the world. Likewise, if you just lean into the e-commerce side of the world, you're going to miss out on all these opportunities that you have to deliver to our customers in-store or deliver to them at home very, very quickly. So it is in the way that I talk to my teams, the way that we come together and, again, design solutions for customers, but it's also in the way that Walmart U.S.

12:14talks about this overall, the way that my boss, John Ferner, who's the CEO of Walmart U.S., talks about this. And we have regular meetings and cadence across the week. For example, on Mondays, we have an important meeting called Trade, where we bring everybody together and we look at the health of our overall business and how we're impacting customers positively, sometimes negatively, what we can do to change and make that better. And as we're looking at that, it's all very integrated. It's all super holistic, which is what is so exciting, I think, about Omnichannel and the way that Walmart is approaching Omnichannel, the boundaries between the in-store piece and the online piece really have been broken down.

12:56And it's what not just makes this business and opportunity exciting. It's one of the things that makes it challenging as well, because you have to approach it from this way if you're going to win. Can you talk a little about just the focus on customer experience, especially if you're implementing a new service, a new technology, something that's customer facing? What are the key metrics that you are focused on? Is it just a survey where you talk to this many customers and they say they liked it, they say they didn't? How are you and how is that being translated to the teams as a whole that are iterating these new programs?

13:30Yeah, so it's a great question. I'd say we bring in feedback information from our customers from all angles, all dimensions that we have. So things like customer surveys, feedback forms, absolutely. Things like emails to myself, to the team, to my bosses, to Doug McMillan. All of that stuff gets ingested, pulled in. We take a look at it. We make sure that we are thinking about the challenges that they're either presented with or how we present or how we prioritize some of the gaps they've created or that they've noticed. But I'd say, importantly, we're starting to look at it very nuanced in terms of, or we do look at it very nuanced in terms of metrics as well.

14:13So when we're looking, for example, at optimizing search, it's really not around how do we just optimize the overall revenue or GMV that we get out of search. But how are we delivering against the query, the question, the intent that the customer has put into the search bar? How do we deliver against that in a way that drives trust and drives our customers to come back and ask us those kind of questions over and over again? So it's looking at things like abandonment rates. It's looking at things like frequency or repeat usage, et cetera. And it's also looking at the types of queries that they enter.

14:51Are they refining queries? Are there ways that we can help them refine those queries and get to their ultimate answers even faster? So it's very, very holistic. We also, with our design teams, we have panels of customers. We'll sit in front of customers and we'll watch them interacting with the products. A lot of times in those cases, you see really interesting outcomes, things that you never expected. And you go, oh, well, they thought about that button much different than we had thought that they were going to think about it. So then how do we put that into practice? Importantly, given that Walmart has a very large physical retail presence, one of the other things that we do is we get out in the field.

15:28We get out in the stores very frequently. We do jobs alongside our associates in the stores, whether it's picking for last mile delivery or operating cash registers on the checkout side of the world to understand what works and what doesn't. And we will routinely talk to customers, as long as they're comfortable with it, about what's working and what's not for them in stores. It's one of the really, really exciting parts about the job. Do you say who you are when you're talking to a customer, or are you just like, it's okay? No, we do. We do. No, it's not dark shopping or anything like that. We wear our badges proudly.

16:02And oftentimes, you know, when you walk around with the badge in a Walmart store, you get asked the same kind of questions that our associates get asked, which is what's so exciting about it, right? Like oftentimes, as you could tell as we were starting the podcast, I'm a big fan of a variety of different toys, especially Legos. So I'll go into a Walmart store and I'll spend time in a Lego area. And I will have customers come up and ask me questions about, well, what's your favorite Lego set? Or I'm shopping for a seven-year-old's birthday party. Which of these do you think would be good for him or her?

16:35Or one time I even heard somebody just ask, hey, there's this show that's on TV. I think it's aimed at kids this age. Do you know what show that is? And I think another associate and I were struggling to figure it out. So you pull out your phones, you search, you do whatever you can to try to help the customer. And these are the kinds of things that I think are pretty unique about the way that we make sure that we are bringing in all of the customer feedback that we have here at Walmart. I wanted to switch gears a little bit and just talk about this year specifically. So what are some of the priorities that you set at the beginning of 2024?

17:08What were some of the things that you've been really hoping to get in place or expand upon? So in 2024, I think one of the big focuses for us has been around leveraging artificial intelligence, large language models, all of that emerging technology in ways that really solve problems for our customers in just really magical kinds of ways. Just a couple of examples for you. On the search side of the world, we've been working really, really hard to do a number of things to better understand the intent, what it is a customer is looking for. And we reflect on that in a number of different launches, things that we've brought to life recently, and we can send you more info on this as well.

17:51One of these things is guided search. So instead of making the customer do all of the work of trying to figure out what they want and breaking down their task into an individual query and bringing that to us, instead, just come to the search bar and tell me what it's about, right? I'm planning a birthday party for a seven-year-old who's really interested in dinosaurs. Okay, well, I'm guessing you want some dinosaur-themed cups and plateware, and maybe these would be appropriate balloons, and these could be appropriate toys for others to bring to the party, et cetera. So that's this notion of guided search.

18:25So tell us your task, tell us your broad intent, and we'll help break that down into easy steps that then link to products that we have in our catalog to make it much easier for you to buy. We're also looking at summarizing item highlights, review highlights, almost synopses of this dense information that we have available on our site. And we're able to use generative AI and large language models to bring up the highlights and really pare that down so we bring the information to a customer much more quickly. That's on the customer side of the world. A lot more we're doing there as well. On the associate side of the world, too, it's a really heavy lean-in on the artificial intelligence side of the world.

19:06On the associate tools side of the world, we have this overall framework tool called Miat. So our million-plus associates use Miat as a tool on their phones that helps them with everything that they would need to do to do their job. Everything from payroll to clocking in and clocking out. And what's the next best action that I should take as an associate so that I can deliver that value for Walmart and then free up my own time so that I can spend that time on the floor? And some of the things that we've brought to life there are if we can let you know as an associate that we have just brought in a number of pallets or boxes of, say, somebody's favorite cracker and it happens to not be on the floor and you're out on the floor talking to a customer who wants that cracker, you can say, oh, I know they just got dropped off.

19:58Let me go in the back and grab a box for you and bring them up front. We're also looking at doing a lot in the RFID and the augmented reality space so that we can help associates in the back. So imagine the back of a Walmart is full of just many, many large cardboard boxes. But you need to know what's in the box in order to be able to unload it and be able to bring those things out. So for our associates, it's really just a matter of waving their phone around the boxes. And because of some of the technology that we have employed, you understand what's in each of those boxes, the amount of inventory, etc.

20:36So we're starting to bring all of this stuff together in the artificial intelligence space and unlock it in ways that really help our customers and associates deliver faster, get at what they're looking at faster, and ultimately save time and for our customers, save money as well. Just a few examples of some of the things that we're doing this year. Just a few. Those were a lot. I want to dig into all of them. Please, please. Let's start with the first one, because this is something I've been interested in. It seems like on the consumer-facing side of things for retailers and brands, the way that they are integrating with Gen.AI is, for the most part, not all, there are exceptions, in chatbots or customer assistance like that.

21:15And it seems like this is similar but somewhat different in the sense that this is about, like, would you say this is different than what I'm seeing at other places where it's just about helping them better deal with customer service inquiries? It seems like it's more a wider mandate. Is that incorrect? Yeah. No, that's definitely correct. And I think it's important to understand that, yes, AI, whatever you want to call it, can solve issues or is primed to solve customer support issues, all of that. And we have roadmaps. We're working on that there. Chatbots, the like, assistants and the like.

21:51And we have examples of work that we're doing there as well. But if you look at where a lot of customers are spending the vast amount of time on an e-commerce site, it's with traditional search. It's with traditional product page results, et cetera. So what we've been looking to do is how can we leverage the same technology for solving some of those problems where we're seeing that massive volume of customer demand? So it's in core search. It's in our core product detail pages as well. And really bringing all of that to life. And that's what's so amazing about what we're sitting on on top of AI.

22:32It's not a one-trick pony. AI can unlock a lot of solutions for us at scale as long as, and importantly, we make sure that the customer is kept or the user is kept in the loop and we have a good human understanding of everything that our machines are doing. Well, that's what I wanted to sort of get into more is where it seems like to integrate AI into search makes sense because the customer does. I guess my question is, do you think the customer cares that something is built with AI or not? because it seems like a lot of companies are launching things specifically so that they say they have an AI tool, whereas it should just be something that makes the customer experience better.

23:15And I don't imagine a lot of people are using a chatbot on a website as much as they are searching for a product on an e-commerce platform, right? That is spot on, Cale. And look, from my point of view, it's not important to say, hey, we built such and such product now powered with AI. Yeah. We can solve problems. We can make it easier for customers to find the unique, special product that they are looking for in a sea of hundreds of millions of products in our catalog. AI can help us make your finding or your selection of that perfect ottoman different than my selection of that perfect ottoman or that perfect red shirt or chair or whatever it is that we're looking for.

24:01And if you do it right, you shouldn't be branding it as AI. You should just bring the magic to customers. They'll be delighted and come back to it time and time again. And so I don't think customers care one way or another. I think they just want their problem solved. Got it. I want to talk about the store associate part, too, because something that Walmart has been known for for decades, centuries even, is just the store experience and how Walmart has always had store greeters. It's always been about that they are, you know, there's always someone there. They will help you. And I imagine there is a long history of store associate training that goes along with it to make sure that they are working within the confines of working at Walmart.

24:43When you are adding atop a new layer of technology, how do you make sure that that works so that it fits within the workflow and so that it makes sense and is additive to what store associates are doing? It's a great question, a great topic. First and foremost, it comes through strong partnership, really working together closely. So say my product teams, the engineering teams, design teams with the teams on the operations, on the associate, on the store side of the world. And there's a decent amount of process, almost rules that go through how do we think about what it is that we're going to unleash and launch to a million plus associates, right?

25:29You want to make sure you get that right and you don't have too many hiccups along the way. We also have ways to test and to pilot new things that we're talking about with smaller subsets of stores, groups of associates, et cetera. But I think importantly, we need to remember that AI isn't here. We don't want to massively change what it is that our store associates are doing, Gail. I think I couldn't have said it better. One of the things that makes Walmart really special is the way that you are treated as a customer inside a Walmart store with respect to the associates being greeted on your way in.