Product lessons from Waymo | Shweta Shrivastava (Waymo, Amazon, Cisco)

0:00Are you proactively trying to challenge your own assumptions? It's extremely important, right? As a beginner product manager, as well as a seasoned product leader, if you're not doing enough of that, then I think you might not be listening. If there's no conflict, if there's no contention, then something is missing.

0:20Welcome to Lenny's podcast, where I interview world -class product leaders and growth experts to learn from their hard -won experiences building growing today's most successful products. Today, my guest is Shweta Shrovasthava. Shweta is Senior Director of Product Management at Waymo, which, if you're not familiar with Waymo, they are building self -driving cars that are already live on the streets in San Francisco, LA, and Phoenix. Actually, I got to take a ride in one ahead of this chat, and you'll hear all about that in this episode. Before joining Waymo, Shweta was Chief Product Officer at Nauto and AI started focusing on driver automation safety.

0:54Before that, she was head of product management at Amazon Web Services for their database and analytics services. And before that, she was at Cisco. In her conversation, we delve into what it's like to work as a PM at Waymo, and how it's both different and similar to software -only products. We talk about their KPIs and goals at Waymo, including how they track progress towards a future of self -driving cars, how they build subtle cues and behaviors into the cars to create trust for the writer and also for other cars on the road, plus Shweta's biggest lessons about building products and teams across the many companies she's worked at.

1:27I can't wait for a future of every car being self -driving, and it was super fun to learn about what goes into making this all happen. With that, Avery Yu Shweta, Shrubostava, after a short word from our sponsors. This episode is brought to you by Vanta, helping you streamline your security compliance to accelerate your growth. Thousands of fast -growing companies like Gusto, Com, Quora, and Modern programs and get ready for audits in weeks, not months. By offering the most in -demand security and privacy frameworks such as SOC2, ISO27001, GDPR, GIPPA, and many more, Vanta helps companies obtain the reports they need to accelerate growth, build efficient compliance processes, mitigate risks to their businesses, and build trust with external stakeholders.

2:16Over 5 ,000 fast -growing companies use Vanta to automate up to 90 % of the work involved with SOC2 other frameworks. For a limited time, Lenny's podcast listeners get $1 ,000 off of Vanta. Go to Vanta .com -slash -Lenny. That's v -a -n -t -a .com -slash -Lenny. To learn more and to clean your discounts, get started today. This episode is brought to you by public .com. You want to tell you about their new treasury accounts, which earn a 4 .8 % yield on your cash. That is higher than a high yield savings account, while still being backed by the full faith and credit of the U .S. government. Treasure yields are at a 15 -year high, but buying U .S.

2:56treasuries is super complicated if you go to a bank or navigate an ancient government website, or at least that was the case. Now you can move your cash into U .S. treasuries with the flexibility of a bank account. You can access your cash whenever you want, even before your treasury bills hit maturity. There are no hold periods, no settlement days, just a safe place to park your cash and earn a reliable yield. Public will automatically reinvest your treasury bills at maturity, so you don't have to do anything to continue growing your yield. And you can manage your treasuries alongside stocks, ETFs, crypto, and any alternative assets.

3:32Do all your investing in one place and earn 4 .8%, a higher yield than a high yield savings account, only with a treasury account at public .com -slash -Lenny.

3:46Shweta, welcome to the podcast. Thank you. Great to be here. It's great to have you. I thought we'd start with just a little bit about what it is you do at Waymo today. What are you and your team's responsible for at Waymo? Yeah, my team's responsible for three key areas, I would say. One is building a big part of the software that actually runs on board. The Filiatana's Wacol and that determines the actual behavior and trajectory of the Wacol. Second, we're building the simulation tools and technologies that are required to validate the performance of the system. And then about third, one of the teams focused on commercially scaling our right -haining business, which is one of our key go -to -market applications for the technology we're building.

4:33So as you know, you arranged it right for me in a waymo in San Francisco. She was actually a self -driving car that had no driver sitting in the front. I have a Tesla and I turn on self -driving sometimes, but I've never experienced just sitting in the back and this thing just rides you around. Also, just like a memory head is like the app to call the waymo is like, it feels like Google Maps except instead of just telling you how to get to a place like a car shows up and just takes you there. And then you could change course as you're driving and it's crazy how quickly became normal. I'm just like, all right, we're just riding around San Francisco in this self -driving car and just sitting in the back and telling it where to go.

5:15And so, so anyway, I'm going to ask you a bunch of questions around this, but I guess thank you for arranging that right. That was quite special. No, I'm glad that you were able to do that and on a rainy day, so that's a special bonus because again, you know, the technology has been performing very well, which is being very hard to see. So a few questions along these lines. One thing that I noticed that was really cool is we were trying to turn into a lane and there's cars coming in that lane just like continuing to move. And the car just kind of like subtly was like inching into way out communicating through this like interesting body language thing of just like, hey, I want to can someone let me in.

5:47And it's interesting. There's no eye contact involved. And it's just like this, I don't know gesture that you all have to develop. And so I guess the question here is just like, what, what have you all learned and what I don't know if you even work on that piece of it, but just I'm curious how you think about creating this like body language of the car communication system to help people understand what it's trying to do. We're using a lot of human driving data to train our deep learning models. So it's important to make sure that the behavior of the car doesn't seem robotic, right? It can feel quite unnatural.

6:19And you know, from the gap goal, we focused on building a fully autonomous system. So it's important to have that some of the addit that trust, you know, building with the riders where they're not daunted by technology. They don't feel like they're sitting in our walk, right? It has to feel very human light, but in a good way, right? Making it safer than human driving, but then not making it feel unnatural. And so we have deep learning models that can understand what the other road users intent is. So stuff like which way the pedestrians looking, right? Or what is their point of orientation? Is that could tell you, you know, what, which way they're headed?

6:56That they're old signs or the gestures, right? Somebody is trying to stop the vehicle. The system can understand all those signals. So because we're using deep learning models, trained on human driving, but again, it's a good way, right? We discard the bad human driving data. We can mimic human driving behavior in a good way. And that's why you saw the behavior that you saw yesterday. Now one thing to note is we can't also just completely rely on explicit gestures and signs, right? Because a lot of driving is also social norms, right? If you're in a particular section in San Francisco, you know, maybe it's okay for pedestrians to cross even when they don't have a walk sign, right?

7:43Another city and other intersection might have a different social norm when it comes to pedestrians crossing the four -way stop sign or the crosswalk or what have you. And so the car also has to learn about those social norms and be able to react to it. So it's like I said, you know, it's we don't realize how sophisticated, how interactive and how social driving really is. And with our artificial intelligence capable of using it, we're being able to incorporate a lot of that into our system behavior. So before Waymo, you worked at non -self -driving software companies, working at Amazon, Cisco, a few other companies.

8:21I'm curious what you've found to be the biggest difference working in a company like Waymo versus a traditional software company. Ever is a set earlier. You know, it's a highly complex, technically complex system that we have built in and we're improving. And it's if I may say so, it's the most game -changing product that anybody would ever work on. I don't know, Amazon's pretty cool, but I totally get. Yeah, I've worked at Amazon, I'm a fan, right? And they have been pretty transformational with AWS and on the e -commerce side. But a fully autonomous driving system, you know, it's also very, very hard problem, right?

8:58So it's transformational from that perspective to I would say that the PMs here have to go have to be able to go technically deep, compared to what they would do in other software products. They have to be able to get into the details as much as needed. They have to be okay with uncertainty and ambiguity. Again, you know, I think that is part and parcel of any product management role, but it's even more so here, right? This is a long game and so you have to have that tenacity to play the long game and be continuously improving the product and make this thing a broad reality, right? In future. So those are some of the attributes.

9:43I would also say that there is some some level of self -selection here, you know, you have to be driven by the mission to meet the road safer, right? We have about 1 .35 million deaths that happen every year across the world from traffic accidents. And most of that is attributable to driving errors and driver distraction, right? And I'm sure I've been guilty of, you know, being on checking my text messages, while driving, I've seen other drivers do that, right? With a fully autonomous technology, you know, you don't have that risk. That's the risk we're trying to minimize. So they have to be driven by that mission.

10:19One other thing is that the concept of MVP, right? Which is so widely popular in the SaaS product management world or product management world in general, has a whole new meaning here at Bebo when you work in a product like this because safety is is, you know, the top of mind for all of us, right? And can't really cut corners on safety. There's no you know, there's, there's the MVP bar itself for safety is extremely high for us, right? So the core product management philosophy of sort of, you know, getting an MVP out there and then iterating with the real world deployment. It applies, but it's just a different, it's a different bar on that MVP.

11:03Touching again on safety and like human behavior, I was thinking a little bit as you're chatting about a Siva Tesla, which has self -driving car, self -driving capabilities. And intellectually, I know it's probably going to drive a lot better than I am, but I still feel like I need to like disengage it occasionally when I'm like on a curvy road. I'm just like, I don't know about this. I don't want to like leave room for air if there's something that weird that happens. And I imagine someone designing product for that weird behavior where like I should probably trust it because it's probably a lot better driver than I am, but I don't know.

11:37I feel like I can do a better job. Is there anything you've learned about a human behavior or how to design software for these sorts of experiences that may be surprised you or is that what I thought was really interesting or that was really important? Since you mentioned Tesla, you know, I just want to clarify that. It's a different system that we're building, right? But just for way more, we started by solving the problem of fully autonomous driving without a human driver at the wheel from the get -go, right? It's not a driver's system or a data system, which relies on the human driver taking over when there's a complex situation, right?

12:13So I think that expectation is built into that kind of a product, right? And and so the human, you know, the folks who are using that product were also have that mindset that he actually be ready to take over when the situation demands. Because we built the system from the get -go to work in a fully autonomous mode without a human driver intervention, right? At the wheel, we had to integrate this into our design philosophy from the very beginning that this has to feel credible, predictable, and the writers have to be able to trust the system, right? So that has been sort of the core of the design philosophy.

12:56And so what happens is, and I heard this from you as well, which resonated with me and I've heard this from a lot of our writers that they feel, you know, it's like, for the first five minutes of the ride, it's, wow, you know, this thing really happening, right? But then it starts to feel very natural. And, you know, as if this is how it was always meant to be, right? After the first one, it's like, uneventful. I think that's how it's supposed to feel, but it's not a happenstance that it feels that way, you know, the naturalness, the smoothness, and still sort of, you know, adhering to safety at all times are things that are designed into the system.

13:35And then we make sure that the writer has visibility into what's happening, the, if they're not bearing the seatbelt, the writer's support would call them, right? So then they know, okay, there's a human that they can reach out to if they have an issue, they can look at the monitor in the in the car, right? To understand what the car is seeing. So I think all these little things help develop that trust in the system. On that same note, what's one thing that you've, your teams have built that creates a lot of trust or maybe it was a surprisingly important element in creating trust in the experience, like in terms of the product, especially, and I don't know, either the app or the in in car experience.

14:14I don't know if I can point to one thing, you know, it is again, it's this is such a holistic experience this then. I think it has to be a bunch of all things to make it feel you know, natural, transparent and trustworthy to the writers. And I can give you one example that I don't think I've mentioned in the discussions so far. So again, because the system is designed to be, you know, cautious and defensive, but still making, you know, adequate progress in the absence of traffic, right? It will never go above the speed limit. Right? It doesn't go about the speed limit. It sticks out here so the speed limit when something that a lot of our writers actually appreciate about the system.

14:55Now, it turns out that adhering to the speed limit, even without traffic, sometimes is is not the best thing. You have to go below the speed limit and we realize that for driving in the in sort of the slopes or the the gradients, you know, the street with clients in San Francisco, there are many of those, right? The human brain is trained to or the human drivers are sort of subconsciously they slow down when they go downhill. Yeah. Those slopes, right? The autonomous vehicle doesn't necessarily have to do that. They're right a bit safe. And if it's staying below the speed limit, but we learn that this is the this is a more natural driving experience, right?

15:36And this is what our writers would also expect in terms of the experience. So that's something that we then you know, modify the behavior on. That makes sense. I would I would want to slow down on the other end if I feel like I could trust that I could I wish there was a button to just like crazy mode just go go for it. Kind of digging a little bit into the product teams way of working. What are KPIs that you all use to track progress? I don't know either amongst some of the teams you lead or also just broadly progress on self -driving technology. How do you know you're making progress? Is it just like miles driven or something else?

16:12There tons of these metrics, right? That we analyze on on Zropa Daily basis weekly basis depending on what what the metrics. But if I were to categorize them as we're two broad categories that be the commercial and operational metrics in the system behavior metrics, right? So it's one thing one important thing to note here is that it's not this is we're not in a proof of concept or a pilot phase anymore. This is the service that we are offering to writers, you know, page service and finance and also it's open to public incentives to school, right? So it's an actual service and so we're tracking you know the commercial metrics in terms of the trips where we write the daily or weekly active users and all the final metrics that you can think of.

16:58Also the operational metrics, right? The cost, right? Well how much does this thing question us to operate? So that's I would say all the stuff on the commercial scaling side and then on the on the driver performance, the driver, Vimo drivers, the technology name as I alluded to earlier, the driver performance metrics, you know, base span across safety, the you know, compliance to the road rules, our ability to make adequate progress as in you know, not get unduly stopped or stranded in dense traffic situations as an example. What are just like specific metrics there? Like anything you could share just like what is the actual goal in one of those teams?

17:38The goal here is to be able to drive safer than than humans, right? Now it's we don't really have one standard human driving benchmark, right? Safety benchmark that everybody uses, right? But we do gather enough of that data, right? We have access to enough of that data to form an opinion on or a metric on a benchmark on what does human driving look like? You know, how many collisions as an example human driver would have every 100 ,000 miles? And then we want to make sure that our performance is better than that, right? So that's I'm simplifying, right? And several things go into sort of both calculating the benchmark as well as our performance across against that benchmark, but that at the core of fit, you know, that's what we're trying to do.

18:30So that's one of the safety side and then I would say on the the tops and strandings which is trying to, you know, which goes in the different direction, hey, you can be very safe if you're not moving at all. That's not to make sure that the riders get to their destination on time. So it has to be making it has to be appropriately assertive and be making the right progress. And so again, how how much did the vehicle slow down? Unduly, right? Or in how many instances in a given week did it have to rely on sort of a rescue help, right? Those are the situations that we want to avoid. And then what how much did we slow down the traffic for other users, right?

19:16So we again do extensive benchmarking and look at the priors, et cetera, and really understand what would and adequate performance be there and measure our own against that. Awesome. Today's episode is brought to you by Element. I just recently discovered the stuff actually from another podcast and it is such sweet salty goodness. Element is a tasty electrolyte drink mix with a science -backed electrolyte ratio. And unlike most electrolyte drinks, there's no sugar, coloring, artificial ingredients, gluten, or any other BS. Getting enough electrolytes helps prevent and eliminate headaches, muscle cramps fatigue, sleeplessness, and other common symptoms of electrolyte deficiency.

19:57Element is the exclusive hydration partner to team USA weightlifting and many other Olympic athletes. Also dozens of NBA and NFL teams and players rely on elements as they along with maybe seal teams at the ice sniper teams and the Marines. You can try element totally risk -free. If you don't like it, you can share it with a salty friend and they'll give you your money back. No questions asked. To give it a shot, go to drinklmnt .com slashleni. And you'll get a free sample pack with any purchase, which includes one packet of every flavor. My favorite is watermelon salt. You won't find this offer publicly available, so you have to drinklmnt .com slashleni to take advantage of this offer.

20:36Stay salty. I'm guessing you're not going to even answer to when do we think we'll have fully self -driving level 5 autonomy. So let me ask you a different approach to that question of just what's getting what's most in the way of us getting to full self -driving where we don't have to do anything ever. Is it like miles driven? Is it like tech breakthroughs that stuff to happen? Is it regulation in just cities being like okay it's fine. What's the biggest blocker at this point or bottleneck? So the nebbing share might be known on the end 5. So I think so. So l4, for those who might not be very familiar with this term, you know, n2l3 is still very much drivers.

21:13It gets to some level of autonomy, but then utilize only human driver at the wheel to be able to take over in complex situations. l4 is fully autonomous without a human driver at the wheel and no expectation of a human driver at the wheel. That's what we'd be focusing on. l5 would be in any kind of road, completely unstructured, off -roading, in that kind of an environment, be able to drive without without any map, without any priors, what have you. And we believe that by offering the kind of service that we are offering and finally extend incentives and let's go and then, you know, and they through the rest of the year and other cities in future is it helps realize that the dream of the fully autonomous driving in a big way, right, without having to go to l5.

22:04So I think that and the technology is already there. You know, l5, I'm not sure, you know, maybe that becomes more niche, etc. It solves really specific use cases. In terms of the blocker, I would say, the technology is still needs improvement, especially, you know, we were not able to drive in snow yet, right? Something that we have to tackle in future. l could barely drive in slow. Yeah, and I don't like to drive in snow even tough. Well, I avoid on race, no days. But yeah, you know, that is something that we still have to build as a capability in driving snow, fine. So that's great. Makes sense.

22:40Clearly, there's a lot of little things, a lot of little things, a lot of big things. And it's a really interesting point about, like, we don't need maybe a last question along this track. And then I want to pivot to a different area. So Waymo's been this long -term investment for alphabet. And many pms often try to create buy -in and keep buy -in for a large investment, a large project. I know this like a different scale of investment than what most pms work through. But is there anything you've learned about keeping leaders bought in and excited and continuing to invest in a project for years and years?

23:14And specifically just like tactics that keep people excited and bought into a long -term investment. Yeah, so first of all, we are fortunate that we have backing from alphabet and other investors. And, you know, the autonomous vehicle industry is interesting. And I think the last year has been interesting with more consolidation happening, right? So I think the name of the game here is to is to show progress, right? Show meaningful progress and meaningful progress, not just in terms of technology, but in terms of commercial deployments, right? That is the rubber meets the road if you will phase off of the product.

23:54And the results have to speak for themselves for our investors to have the confidence in us, right? So notwithstanding, you know, what's happening to other AB companies in the industry, you know, it's about what we are doing. And they look at the purpose that we've been making and where we are headed. And the fact that we've been sort of accelerating our milestones and going through our own expectations, I think these are very positive signals to our investors as well. And we're going to, you know, another startup as well. It's not about, you have to do what's the right thing for the business, right?

24:28That's what you're focused on creating value for the customers, creating value for the right leaders, right? You have to build a business that makes sense. And, you know, and the investors see that too, right? Or we're not going to do something unnatural or something that, you know, doesn't align with the business goals in order to gain any short -term, round -eap points with investors, I think it doesn't work that way. And the investors will see through that too, definitely, alpha that, you know, has been our backer for forever, right? So it's really about focusing on doing the building the right business and doing the right thing for the users.

25:03I think that's a great takeaway that if you're finding that there isn't buy -in and continue to support for which you're building focus on momentum and showing success, like it's pretty simple if you think about it. And it's hard to cut something that's just like showing success. Yeah. And so even at the scale of awaymotes, it's a great lesson. So that makes sense. We talked a bit about other companies you've worked at and so I want to kind of zoom out a little bit. And I just want to ask, did you work that Amazon Cisco, Waymo now start up? You mentioned what are just some of the biggest lessons you've learned about successfully building successful teams and successful products?