The future of driving is autonomous with Dmitri Dolgov of Waymo

0:00Wondery Plus subscribers can listen to How I Built This Early Early and Ad Free right now. Join Wondery Plus in the Wondery app or on Apple Podcast. I love traveling with my family. We did an awesome trip this summer. And one of the things that made the trip so special were the Airbnb experiences we did. Immersive tours, cooking classes, a chance to get coffee with a world-class barista. I had so much fun on those experiences that I decided to host my own Airbnb original experience in San Francisco, designed to help you think about how to unlock your next big move in your career or even in your life.

0:41To learn more about my Airbnb original experience, head to Airbnb.com slash Guy. Listening on Audible helps your imagination soar. And no matter what you like, Audible's romance collection has something to make you swoon. Here's your invitation to have it all. Find a book boyfriend in the city and another on the hockey field. Or if nothing on this earth touches your heart, you can always find love in another realm. Hear modern rom-coms from authors like Lily Chu and Allie Hazelwood, the latest romantic-y series from Sarah J. Maas and Rebecca Yaros, and Regency favorites like Bridgerton and Outlander.

1:19And that's only the beginning. Audible has an incredible selection with over 1 million audiobooks, podcasts, and Audible originals all in one easy app. And you can enjoy Audible anytime while doing other things. Household chores, exercising, on the road, commuting, you name it. Audible makes it easy to drop into your fantasies during your everyday routine without needing to set aside extra time. There's more to imagine when you listen. Your first great love story is free when you sign up for a free 30-day Audible trial. Visit audible.com slash built. Remember dongles? Each piece of tech had its own plug and none of it worked together.

1:58That's where a lot of businesses are with AI. Discover AI that can work seamlessly across your data and applications at ibm.com. The AI built for business, IBM. Hello and welcome to How I Built This Lab. I'm Guy Raz. There's no driver. How crazy is this? That's crazy. This is your first time ever. You're going to look at this as an adult when all cars are driverless. And you're going to be like, yep, I did this driverless drive with daddy. So it's probably hard to tell from the sound alone, but what you're hearing is audio from a ride. I recently took in a Waymo autonomous taxi. My son and I were in San Francisco heading across the city.

2:40These taxis are now all over San Francisco and Phoenix. And yes, there is no human driver. There's no human in the car at all. unless you count me and my son, who were passengers. It's a ghost driver, and if I'm being totally honest, they're amazing. The Waymo taxi is now available 24-7 in San Francisco and Phoenix, and it operates similar to Uber or Lyft. You pull out your phone, you fire up the app, order the car, and it magically appears within minutes, except it's fully autonomous. Waymo was spun out of Google a few years ago, but the project to build autonomous vehicles at Google actually goes back to 2009.

3:21One of the engineers on that project was Dmitry Dolgov. He's been with Waymo from the very beginning, and today he's the company's co-CEO. Dmitry was born in Russia, a child of two physics professors. He went to high school in the U.S. and eventually returned to Russia to attend the Moscow Institute of Physics and Technology. Dmitry then came back to the U.S. to get his Ph.D. And in 2006, he took part in a competition called the Urban Challenge, which was sponsored by DARPA, the U.S. defense agency. The idea was to build an autonomous vehicle that could make it through a series of obstacle courses.

4:00In the Grand Challenge, the Grand Challenge was to drive 150 miles through a desert, a completely static environment. But until then, robots were not capable of doing that. So it took two tries in 2004 and 2005 to accomplish that. So the next step was, let's make it a little bit more interesting. Let's make the environment dynamic. Let's add some rules of the road, stop signs, other vehicles, but do it in a controlled environment in that mock city. So that was the challenge. And we built a car to do exactly that. We equipped it with a bunch of sensors, lighters, lasers, radars, cameras, a computer, and then wrote software that would allow it to follow the rule, understand and follow the rules of the road and interact with other dynamic actors, whether human or other robots.

4:45So, I'm looking at one of the vehicles that you helped put together. It's called the Junior. And it's a Volkswagen Passat, I think. And it came in second place. This is like 2007. And was the underlying technology basically what we're talking about today? Was it pretty similar or was it like a crude version? uh you know yes and no it depends on how you talk about like you know big car plus sensors plus computers plus software you know that stays right uh but of course you know it's what almost 18 years and then everything has changed right the the sensing technology has gone a very very long way uh computers you know have evolved software right all the breakthroughs especially in AI.

5:39So, you know, that led to many, many breakthroughs in the area of software. So now the system we have kind of has the same components, but, you know, all of the components and how they work together is, you know, qualitatively drastically different. Yeah, it's amazing how many, you know, these DARPA challenges, how many sort of self-driving car companies they spawned. I mean, we've talked to Kyle Vogt of Cruise. He was in, He competed in the 2004 and 2005 challenges. Dave Ferguson of Neuro, which is another company, was part of the Carnegie Mellon team. Chris Urmson, who also obviously worked with you at Waymo.

6:15He's now with Aurora, was also part of the Carnegie Mellon team. It's amazing how many, like you knew all of these people. And this is, as you say, it was a small community of people working on this huge challenge to basically create vehicles that could be fully autonomous. That's right. That's right. And, of course, I worked very closely for many years with Dave and Chris, and they're great friends. So, yeah, it's a lot of innovation and a lot of companies. And progress came out of those early days of the RPG Grand Challenges. You and Sebastian Thrun and Chris Urmson, you all went to go work for Google.

6:56And you were part of that founding team, which was at the time a secret project. Google's self-driving car project. And at that time, it's amazing to think 2009, because it seems like ancient history now, you were given a charge by Larry Page and Sergey Brin, the co-founders of Google. You had two years to essentially accomplish two things. Do you remember what they were? Yeah, I remember them very well. The first one was to drive 100 ,000 miles in autonomous mode. I was way more than orders of magnitude, more than what anybody has done at that time. And the second one, and actually that one turned out to be more interesting and much more challenging, was to drive 10 routes.

7:45Each one was about 100 miles long. And they were very carefully selected, I think, by Larry and Sergey personally. And they were fairly devious in how they created them to make it interesting. And the goal was to drive each one from beginning to end in full autonomy, so with no human intervention, which at that time seemed almost impossible. I remember we had a lot of people, even experts in the field, laugh at us when we attempted it. Two years to accomplish this. I think you guys finished it with three months left. Like you actually, I mean, going into this project, do you remember thinking we're never going to do this in two years?

8:28I was actually maybe naively, but fairly optimistic. Right. And it was hard. And they're all different. And yes, there are many moments where we would finish one route and we would think ahead of what would it take to do the next one. And we would write some software. We would collect some data and test it. and then we'd actually try it and have, then we would hit a bunch of old crap moments of like, okay, wow, this is way more difficult than we expected in even those early days. So when you did finally accomplish it, were you guys able to have a party and celebrate or was it just like, nice job, now we got to keep it quiet?

9:13Yes, both. We celebrated and it felt like it was a big accomplishment. But yeah, we kept it quiet. But that's what made it so incredibly fun. I guess, you know, that phase was one of my favorite phases of the whole project. It's like the early days of a startup, you're up against what might seem like an impossible goal, but it's very clearly defined, right? You have a very clear milestone, a very clear goal. You are singularly focused on it. You have a small team. You know, everybody's working around the clock 24-7 and, you know, sprinting together. and every day and every hour you are prototyping, right?

9:57You're learning so you can move incredibly fast. And every day of every hour, you're making amazing progress and you're learning your thing. So that was a tool and it was a blast. And every day probably presented a new series of challenges. Do you remember what was one of the hardest challenges that you had to figure out? Something that just took longer than you thought. that you just, it just didn't, it wasn't coming together quickly on working on trying to make this happen. Do you remember something you worked on that you just, it was like such a hard problem to solve? There's one, I mean, we had a number.

10:32Some were more fundamentally challenging. Some were kind of even comical setbacks where, you know, you would do a ride. I remember one of the routes was driving on all of the freeways and crossing all the bridges in the Bay Area. and it's about 100 miles and you go through all of the challenges and we're attempting this drive and the car is doing a good job. It's handling merges. And you or somebody else was sitting in the driver's, behind the wheel, just in case. That's right. That's exactly right. That's exactly right. And what made those early days a lot of fun is that if you do everything, you would be putting some hardware in the car, you would be calibrating the hardware, then the next hour you're writing some software, whether it's tools or something for the car to actually make decisions.

11:19And then you get in the car and you give it a try. So anyways, on this route, we are driving along. It's doing a job. We get almost to the very end. So at this point, you're holding your breath. You're waiting for the last mile or so of that 100 mile. And the way that particular run was supposed to finish is that we're coming down the Golden Gate Bridge into the city. and they're a set of tall booths when you go through the bridge. Yes, as you go into the city. That's right. At the end of the Golden Gate Bridge, and they're narrow because I use them almost every day. That's exactly right. And the one that our car wanted to go through was closed at the time, and it's just not something that we ever encountered and we thought about.

12:09Oh, right, because it couldn't figure out the X, the red X, or the green arrow. like it didn't, you couldn't recognize what those meant? That's right. And there was actually a gate, so, you know, it would stop, but it was like it would not change lanes and pick a different one. Like, oh, you know, my God, now back to square one. Because probably every, you know, it's impossible to imagine every eventuality, but something's going to come up. That's exactly, that's exactly right. And, you know, we would have to deal with, you know, high-speed traffic on freeways. You know, we had one route that went, and kind of took this windy road from the Bay Area to Highway 1 through the mountains.

12:47And we were driving along, and then a bicycle fell off the truck in front of us. So that's not something that that time the car could deal with. We had another route that went through downtown San Francisco and the famous Lombard Street. That's very narrow, very windy, and has some of the most adventurous pedestrians and tourists in the world. So challenges like that, right? So that's what made it so challenging and so interesting that it was kind of the breadth of the experience. And probably every single ride posed a new series of challenges. Like, for example, some lanes are shoulder lanes, but then during heavy traffic, they're open for driving.

13:27But the lines are painted on them in such a way that it doesn't seem like a lane. Like a human could figure that out. But an autonomous vehicle probably at that time was like, wait, this doesn't make sense. The lines don't make sense. They don't align. Maybe the car was confused. That's right. That's right. And that would be not the kind of situation or condition that at that time we were able to solve robustly. The goal there was to learn and do. We had to do the route once. So if you fail at something, you would go improve the system and you would try it again. So it was a very well-scoped milestone because 100 miles is nothing if you want to build a production system.

14:11But in those early days, it was long enough that it actually forces you to very deeply think and tackle some of the most fundamental, most important challenges that exist. We're going to take a quick break, but when we come back, more from Dimitri Dolgob on how the engineers at Waymo decided to go after the big win and create a fully autonomous vehicle. Stay with us. I'm Guy Raz, and you're listening to How I Built This Lab.

14:41If you've been listening to the show these last couple of episodes, you've probably heard me mention the Airbnb original experience that I'm hosting. and I'm so excited to have a chance to connect with listeners in a completely new way. It's called the Reinvention Lab, and it's designed to help you think about how to unlock your next big move in your career or even in your life. I'll help you discover your own story in ways I do on this show with my guests. And attendees will get a chance to take a deeper dive with me on so many lessons I've learned from this show, lessons that have transformed how I work and think about the future.

15:19All proceeds from the event will go to support the Ronald McDonald houses that helps families stay near their children who are being treated at nearby hospitals. It's going to be really fun and I can't wait to meet you. So come join me in San Francisco and take your idea to the next level. To grab your spot, visit Airbnb.com slash guy. It's going to be great. Now a quick break switching topics to one of our favorite sponsors, Vital Proteins. Vital Proteins is the number one brand of collagen peptides in the U.S. By taking collagen peptides every day, you can help support your hair, skin, nail, bone, and joint health.

16:00And now you can try Vital Protein's new collagen and protein shake. It has all the benefits of collagen in a ready-to-drink chocolate protein shake. With 30 grams of protein, it's great for healthy hair, skin, nails, and joints. And it's perfect when I'm on the go. I use it at the gym. I use it after a workout. It's delicious. And the chocolate flavor is awesome. I highly recommend it. Go to www.vitalproteins.com to learn more and where to buy. Get 20 % off your next order by entering promo code BUILT at checkout. This episode is brought to you by Klaviyo, the only CRM built for B2C and the key to making Black Friday and Cyber Monday your biggest wins yet.

16:47When the holidays hit, your competition only gets louder. That's why the most successful brands use Klaviyo to cut through the noise and build personalized relationships that drive more revenue. With Klaviyo, you get marketing, service, analytics, and all your customer data together on one AI-powered platform. It's everything you need to build lasting customer relationships. Send exclusive email offers to your VIP customers. Deliver perfectly timed text to high-intent shoppers. Maximize your ad dollars with precision. Join the more than 176 ,000 brands, including Away, Patrick Ta, and Dollar Shave Club, already growing with Klaviyo.

17:34Make this holiday season your best yet at klaviyo.com. Hey, Guy Raz and HIVT listeners. My name is Richard Crowdy, and my favorite episode as a business professor is the founding of BET, because it summarizes business schools perfectly and keeping revenues up and keeping costs down. It also shows that a complex company like a media company can be founded by an individual that bootstraps it with the proper type of innovation. Bob Johnson is also a legend, and he's hilarious. I've actually been listening to my top 15 to try to come up with a list for my business students. So thank you, Guy Raz, and cheers.

18:11If you want to share your favorite episode of How I Built This, record a short voice memo on your phone telling us your name, where you're from, what your favorite episode is, and why. A lot like the voice memo you just heard. And email it to us at hibt at id.wondery.com. And we'll share your favorites right here in the ad breaks in future episodes. And thanks so much. We love you guys. You're the best. And now, back to the show.

18:45Welcome back to How I Built This Lab. I'm Guy Raz. My guest today is Dimitri Dolgov, the co-CEO of the autonomous vehicle company Waymo. I guess in those early days, the Google strategy was really on creating driver assist technologies, not necessarily fully autonomous cars, but cars that could make it easier for drivers to navigate. Basically, what you've got in like a Tesla now or some other cars that have driver assist technologies. But I guess around 2013, there was a conscious decision to pivot the focus because most of your testing was done on freeways. That's right. But what I think you guys and most people involved in the space realized is that to really create complexity, you've got to test these in cities.

19:32They've got to navigate city centers and stoplights and left turns and intersections. And so what happened in 2013 to kind of get you guys to focus, to shift your focus on building a fully autonomous vehicle instead of just driver assist technology? Right. I would say there are a number of things. As you mentioned, the first phase was just learning and understanding the complexity of the problem. Then we said, let's try to build a product and that was the driver assist system. You know, we tried it. That was actually reasonably successful. We had ran a pilot where we gave these cars to about, you know, a hundred Google employees and they could, you know, use them to commute and take them around on their daily trips.

20:13So that was, you know, around 2011, 2012. So then in 2013, we looked at the whole thing and we made this decision to go after full autonomy. So the reasons were, you know, several. One was what we learned from people using the driver assist system. They would overtrust it. They would put on makeup and text and one guy actually fell asleep. The car handled everything fine, but that is not what we wanted to see. That was one of the reasons. Another one was that we actually made progress on the core, most difficult aspects of this driving task on surface streets. So that gave us a bit more optimism of going after the big prize.

20:59And this also at the same time, the field was moving forward. So if we wanted to really make an impact and make improved transportation globally, I thought, okay, the field, driver assistance systems, we'll develop there. Other companies will be pushing in that direction. But let's play to our strength, right? And let's go after the uncompromising, big win and big unlock in the space, which is full autonomy and actually building what we now call the Waymo driver that's responsible for the whole task of driving beginning end to end with no human behind the wheel. What do you remember about the conversations and the ambitions?

21:38Was it like, hey, let's build the technology for fully autonomous vehicles that we can then maybe one day license? Or was it, no, let's basically become a car company. Let's make fully autonomous technology and purpose-built vehicles and eventually sell these to consumers that they can have for themselves. Was either of those ambitions discussed? I don't think we ever seriously entertained building a car. It just never made sense. to me or others to do that. Like we're not a car company. Building cars is hard. There's many companies that spend 100 years getting very, very good at it. We did in the early days design a low-speed vehicle that we called the Firefly because we needed to take that first.

22:33That was our zero-to-one moment of full autonomy. It was a prototype that you publicly should display in 2014, but that wasn't the goal. It wasn't, we're not going to build cars. But we're going to build the technology that can power any car to become autonomous. That's exactly right. That's exactly right. And it was the evolution of the thing that then crystallized in the Waymo mission, which is to build the Waymo driver. And then over time, deploy it in different products and different commercial applications, whether ride hailing, trucking deliveries, and eventually personally owned vehicles.

23:11And that's the path that we set for ourselves and the path that we've been on since that pivot in 2013. Now, in some ways, the real work begins because between 2013 and let's say 2023, right, which we're going to get to because it's incredibly exciting what is going on now in San Francisco and Phoenix and a couple other places. But that 10-year period, probably all of a sudden, you're back to kind of startup mode because you've got to get these cars, this technology to be absolutely foolproof perfect, but in complex environments like San Francisco, which I think is next to New York is one of the most complex driving environments in the United States.

24:01Tell me a little bit about the process that then began. Is that what happened? Or was it a shift to like, okay, let's see what these can do in cities? That's exactly right. We didn't start with the full complexity of San Francisco in those early days. So we can kind of think of that 10 years as maybe three phases that correspond to three generations of our technology, three generations of our driver. That first one was on that low-speed vehicle, the Firefly. And that's what we called the third generation of our driver. And by the way, just to clarify, the Firefly probably was designed to be like a campus type of vehicle, right?

24:46Like in a university or an office park, not necessarily in the city. That's exactly right. I mean, it was a low speed vehicle. It can only move up to 25 miles an hour. So we're thinking, you know, maybe large retirement communities or campuses. Like a shuttle that you would get off and on. That's exactly right. And in that first phase and that third generation of our driver, the goal was to actually build something that can take a fully autonomous trip. And we actually did that in 2015. And we put a friend of our project, his name is Steve Mann, he happens to be blind. And in 2015, he took the first autonomous ride in Austin, Texas.

25:26The first person, the first public person that's out of Google. That's right. To take a ride. I remember this. This is a YouTube video of it. And yeah, it was in Austin. And yeah, I mean, that was kind of a big deal. Yeah, it was a huge moment when we're able to do this first ride in 2015. It was a big celebration after that. Yeah, I can imagine. Okay, so at that point, you and your team had built this third-generation version of the Waymo driver that could actually take a fully autonomous trip, which is a big deal. And you were essentially betting that this vehicle would keep passengers safe on real city streets, right?

26:12Like what went into the safety design? That's right. That's right. But it was also that early phase is what kind of forced us to start thinking about this very fundamental question of what does it mean for a self-driving, fully autonomous vehicle to be ready? I come and set out on this path in 2013. I said, okay, we're going to need a car. It needs to have a bunch of safety systems. Does that exist? No. Okay, well, let's design one and work with partners to manufacture it. So we put a lot of thought and work into making it safe. It had like a foam front. It had a plexiglass window. The sensor pods were attached with magnets so they could detach if something were to happen.

26:55And then the sensors at that time didn't have the level of reliability or capability that we would need, that we would trust to go to full autonomy. So we built, you know, that generation had our own lidars and, you know, custom sensor suite. And of course, the software that we had to build, none of that existed. So let's talk about the technology for a moment because you mentioned LIDAR and radar and some of these things we know what they are. LIDAR, for example, uses basically light lasers to measure distance. and you've got radar technology cameras all over the car. Can you just kind of break down how they work?

27:40I mean, a lot of people who drive Teslas, for example, I have one. If you use what Tesla calls full self-driving, which is not really full self-driving, but they rely primarily on cameras. They don't use LiDAR or radar technology. Tesla argues that that is the other ones are just redundancies that are unnecessary. Tell me how your technology works and why you think it's better. Well, these sensors are, they kind of have fundamentally different and complementary physical properties. So cameras give you the high resolution and the richness of color. And how many cameras, by the way, on a Waymo vehicle now?