Chris Urmson: The Future of Autonomous Vehicle Technology

0:05Hey, everyone, and welcome to Generative Now. I am Michael Magnano. I am a partner at Lightspeed. For years, people have dreamed about making a fully autonomous vehicle a reality, and now AI is making that dream a reality. And so this week on the podcast, I speak with Chris Urmson, the CEO and co-founder of Aurora, a company that makes software for autonomous vehicles and specifically freight trucks. I spoke with Chris about the challenges of designing AI systems that mimic human-style driving and why he believes that AI is the key to safety and the success of self-driving technology. Hey, Chris.

0:40Hey, Michael. How's it going? Good. How are you? Doing really well. Thank you so much for doing this. I know how busy you must be right now. There's a lot going on in your world. So appreciate you taking the time out of your day to speak to me. Yeah, no, really appreciate it. And of course, Lightspeed has been an investor with us and, you know, appreciate all the support over the years. So thank you. Yeah, no, our pleasure. So look, there's so much to get to. I think, you know, Self-driving, autonomous vehicles, these are topics that so many people in the world are talking about right now, especially given some of the recent advancements on the consumer side, which obviously has a lot of exposure to the American people and people throughout the world.

1:20But you've been at this for a very, very long time and not just necessarily in the consumer sector. And, you know, I thought it would be great to lay the groundwork to some of the broader topics I hope to get into with you, maybe just to give the audience a sense of your background and your journey. Because, you know, for all the people that are just finding out about this technology now, like you've been at it basically since the beginning. And so, yeah, if you wouldn't mind just sharing a little bit about your story. Yeah, happy to. So I kind of stumbled into this space back in 2003, I think it was.

1:55there was this DARPA grand challenge. And the idea was we were going to build robots to race across the desert from LA to Las Vegas. And to me at the time as a grad student, that just sounded cool. I'll be honest. I'd been working on robots that moved at 15, 30 centimeters a second, which if you imagine, you know, your grandmother with a walker, like that's how fast they were moving. And the idea we'd have a truck drive across the desert at 30, 50 miles an hour, just seemed unfathomable and like a lot of fun. And so worked on that. The first year we went out to the desert and our vehicle went the furthest and it went about seven and a half miles and it got stuck up on a berm and basically burst into flames.

2:43And so, you know, it was supposed to go 150 or something like that. And so - 150 miles. It was supposed to go 150 miles and we went about seven and a half. So imagine an Olympic marathon and someone goes two miles and that's the furthest anyone goes in it, right? So, but we did do it at 40 miles an hour. And so the Defense Department said, like, that's pretty exciting progress. Let's take another run at it a year later. And we came back two trucks the next year to these big Hummers. And we ended up finishing second and third. And again, exciting. We were disappointed, of course, to lose to some junior university out here on the West Coast at Stanford, but, you know, it went okay.

3:28And then they said, okay, you've driven across the desert. Let's actually drive on roads. And instead of just being on the road at all, like you have to be on your side of the road, there's going to be other traffic. And this was the urban challenge in 2007. And that was really the first time we started to think about, or at least I started to really think about the implications for people in, you know, and how we can move people through the world more safely. That competition, the first time they held it, we ended up winning it. It was really exciting. There was lots of, you know, exciting robot moments as we had, you know, MIT and Cornell crash into one another.

4:12So I think that was the first ever recorded robot car crash. You know, vehicles driving around. They had stunt drivers out in the course that created extra traffic for them. So really an exciting day and one of those moments where like, oh, this is going to happen. This is coming. And then I spent a couple of years working with Caterpillar automating these giant haul trucks. So imagine a dump truck the size of your house driving at 40 miles an hour, moving ore from one ployer from mine to another. And so really exciting. And then in 2009, Google reached out and said, hey, we're thinking about doing this secret project in making self-driving cars.

4:54And would you like to come out and lead it? How'd they find you? Were you surprised? I knew some of the folks. So I knew Sebastian Thrun, who'd been a professor at Stanford and then had gone on to Google to help to build what's now Street View and did a bunch of work in mapping. And so we had been staying in contact through the competition. I'd been the technical director for the team at Carnegie Mellon. And Sebastian and I had been talking about doing something together, maybe starting a company. And when he was at Google, he talked to Larry and Sergey about it. And they said, hey, why don't you do that here?

5:34And so then the rest has been history. And so I spent about seven years at Google, helped found what's now Waymo, led that for that seven years and ultimately stepped away from that in 2016. Spent a few months trying to figure out what I wanted to go do next and ultimately settled on founding Aurora and found two great co-founders in Sterling Anderson and Drew Bagnell, who had experience launching Model X in the first version of Autopilot and had been one of the kind of really early and deep researchers in artificial intelligence, machine learning for robotics, and Drew, one of the world's experts in that.

6:14So we kind of came together and said, okay, let's go make self-driving vehicles happen for real and use the experience that we built from all of these different places and try and build something special. So, and my understanding is Aurora is mostly focused on freight. Is that right? But you also do passenger vehicles. Like, I would love to hear a little bit about, like, the journey through Aurora to settle on a market and maybe focus on one versus another. Yeah. So, when we started Aurora, we were pretty clear we wanted to build a driver that can drive any trucks or any cars, right? That if you think about it, the driving task is pretty similar, whether you're in a car or a truck.

6:51And in fact, to be able to drive a truck, you have to have a light vehicle permit first, and then you go and get your commercial driver's license. But when we started, from the experience we had, we knew that there wasn't the technology to see far enough down the road to be able to drive a truck safely. So if you think about if you're driving at 10 to 15 miles per hour like you are in San Francisco traffic most of the time, you can kind of stop in the length of a table. And so, you know, there's a lot of complexity in what's moving around you and there's a lot of things to interpret. But you don't actually have to see that far to be able to stop safely.

7:29In contrast, when you are driving down the freeway with an 18-wheeler, you have to see two football fields or more. In fact, in practice, we think you have to see more to do it well to be able to stop. And so, there wasn't any LiDAR technology that could do that. And so in 2017 and 2018, we spent a lot of time trying to find a technology breakthrough that would enable that. But until then, we were focused on cars because that was where this technology could be applied. There's lots of valuable and interesting use cases there. As we found this little company in Montana called Blackmore, and for the last 10 or 15 years, the folks who had founded that had been working on this really special kind of LIDAR technology called frequency modulated continuous wave.

8:20And without getting too into the weeds, what it means is it lets you see basically twice as far. And you're much more immune to noise. So like bright headlights can blind conventional LIDAR. They can't blind this kind of LIDAR. And really interestingly, you can see exactly how fast the thing is moving The same way that when an ambulance drives by, you hear the Doppler effect, you know, the tone shift in the siren. Of course. Well, you can use that to measure speed. That's how police radar guns work. Well, we do that with light. So this little dot of light goes out and we can see how fast the thing that it hits is moving.

9:00And so we can go, we can see further and we see how fast they're moving and we can use that to really understand the world at range. And so when we brought that technology in-house with acquiring Blackmore, we're like, okay, now we have the key parts that would let us to go work in trucking. Should we go do that? And as we thought about the opportunity for this technology, it became clearer and clearer that that was the right first market to enter. And so maybe take a moment, talk a little bit about why trucking, right? So in the U.S., if you look around the room you're in, the room I'm in, everything here moved on a truck, right?

9:40Like our economy is just 100 % dependent on trucking. It's a backbone to our society. And the people who drive truckers, they do an incredibly important job. But it's a really tough job, right? That they will spend days or weeks away from home. uh they're you know they have a lot of more health conditions uh they are 10 times as likely to die on the job it's the average american and so this means that despite it being a really laudable and important profession uh there's not enough people who want to do it and in the u.s we're short somewhere between 60 and 70 000 drivers today the american trucking association expects over the next decade, we're going to need a million new drivers.

10:28And we don't know where they're going to come from. And so there's a real need. We also accept an incredible amount of collateral damage on our freeway system today on the roads in America, right? 40 ,000 Americans die every year in traffic accidents. Of those, about 6 ,000 are accidents that are related to heavy trucks. And so by having a stable supply of drivers that can do this work that don't have, you know, don't get distracted, don't get tired, but also have superpowers, right? That they can see, you know, the Aurora driver can see in front of the truck and behind the truck at the same time.

11:13and it can be paying equal attention to the hazard or vehicle in front of it as it is the hazard or vehicle beside it in a way that I physically can't because my neck and my eyes can only look in one place at a time. And so that there's a real benefit of safety. So we again see a place where there's not enough people available to do the work and there's an opportunity to make the work safer and kind of help society. And then we also see it as economically incredibly interesting. So the U.S., the trucking industry is about a trillion dollar market today. It's trillion with a T off of 200 billion miles traveled.

11:56So it's this incredibly established market that has a real need and where the unit economics are really interesting. We value driving a truck three times as highly as we value driving a car. And so when you think about introducing a new product, you want to be entering a space where there's a desperate need. You want to enter a space where the market ultimately is large enough that you can actually have a real impact in the world. And you don't have to dramatically change customer behavior. And then you want to do it in a place where you have sustainable unit economics, particularly as you bring a product to market where the cogs the cost of delivering it are higher initially and then they can come down over time and so as we kind of piece this together like that's the place we should start and by building a driver that can operate a truck at speed that driver will be that tech intelligence that we build is going to be useful a bunch of other places it'll be useful to drive light vehicles it'll be useful to understand the world and so there'll be a lot of really interesting long-term places to go.

13:02But even if all we do is, you know, become the leading provider of automated driving for trucking, that's going to be incredibly impactful and incredibly valuable. It almost sounds like, and I could be wrong about this, but like, it almost sounds like the truck, the opportunity in trucking may be more valuable than the opportunity in consumer passenger. Is that, is that right? I mean, because like you said, we're actually short drivers. And if these people didn't have to do this, maybe they wouldn't want to do it, right? Whereas I think we all need to get to where we're going in our regular everyday lives as passengers.

13:41But the same doesn't have to be true for the drivers of trucks. Is that right? That's right. And so exactly, we see this as the most obvious place for this technology to come to market. I do think there's a real opportunity in personal mobility, right, that you get a lot of the safety benefits or all of the safety benefits there to be realized as well. the convenience benefits. You could imagine a world where a shared fleet is providing transportation more efficiently for people than owning vehicles. And you see this is, you know, kind of the analog version of this has happened in New York and big cities for time, right?

14:23Over time, like it's inconvenient to have to go find a place to park your car. So you take a taxi or an Uber or you ride the subway and kind of the cost value proposition and this kind of changes. As you bring the cost of delivering that mobility to customers, then it will become more and more, or cost down of delivering that. You'll get more and more people wanting it. And so you can imagine seeing the shift there over time where you move more people from driving their own cars to riding in this shared service kind of model. Right. But in terms of like, so that makes a ton of sense, but in terms of like adoption, it feels like the adoption and the uptake would probably be quicker in trucking because there's a real economic benefit to the companies that rely on these drivers and these trucks to fuel their business.

15:13Absolutely. That's clear, right? That trucking is really a TCO business, whereas the ride hailing market is a lot more emotion involved. Exactly. Right? If you think about, if I'm a carrier, a trucking company, I care that the thing got where it's supposed to go safely, that it got there on time, and that I could make money doing it. Whereas if I'm taking an Uber or a car across town, you know, I want to know that it's taking the right route that I like. When it cuts between cars, was it too aggressive and so I'm scared? Or was it not aggressive enough and I'm frustrated that we're late? Right.

15:56So there's a lot more. And I have options. And you have options. Right. And you may just like driving your car. And so I talked earlier about not having to change the customer's kind of way of working. And so ultimately, if you want to really build a huge business and have a huge impact in the ride-hailing side, you need to change customer behavior from hopping in their car in the garage to getting into a hailed vehicle somewhere. So I think that's going to be interesting. And when it comes to these trucking companies, you talked about the opportunity for them, the financial incentives around this.

16:36And this is where it's really exciting, again, from a business perspective, is that today, if you drive a truck, you can only do it for 11 hours a day. And so if you're someone who owns a truck and pays a driver, then you've got that really$150 ,000 plus asset that sat fallow for half the day. whereas we should be able to double that and that's a you know the opportunity to increase your revenue and make more money as a and serve your customers better yeah yeah that makes total sense yeah there's just like all these probably efficiency unlocks that come with transitioning to a fully autonomous driver you've been in this industry really since the beginning i mean even i remember like the early days of google self-driving program seeing the vehicles on the street um and and and now you're still doing it at your own company and at the forefront of the technology how have things changed maybe from the technology side from when you started to where they are today?

17:28And maybe if you can touch on the fact that there seem to be, if I understand correctly, different types of approaches that are being utilized by the different companies, such as yours, such as Waymo, such as Tesla. There isn't just a sort of single method that you can choose from a technology standpoint. Is that right? You know, I've had the opportunity to kind of ride and experience the Gartner-Hype curve firsthand, right? And so back in 2003 and 2009 when it started Google, it was like no one thought this could happen, right? It was kind of this moonshot crazy idea. And then sometime around 2015, 2016, it was like everyone's going to have a self-driving car tomorrow.

18:14Problem solved. and so there was a lot of investment activities, a lot of folks that piled into the space and that's great, right? That leads to innovation and that kind of fuels an innovation economy. And then over the last five years, we've gone through the dip and the trough of disillusionment is, oh yeah, this isn't that easy and I don't know if it can ever be solved. Now we're starting to come out and I think we're starting to hit that plane of enlightenment or whatever they call it, where the product is really actually useful and starting to kind of live up to the expectations that folks have had.

18:51So that's been, you know, kind of an interesting journey and one I certainly wouldn't trade. On the technology front, you know, if you go back to the 2003 vehicle from that DARPA Challenge Sandstorm, it's like Lucy, right? Or Cro-Magnon Man, kind of the bridge between the old ideas and new ideas. You know, we had massive computers on board. We had cameras, lasers, radars. And for the first time, we had high definition digital maps. And I think, and we started to do some of the early machine learning AI techniques that are kind of the grandparents of what we use in AI today. And so that was kind of the moment where it first all came together.

19:42And if you stand back at 50 ,000 feet, it basically looks the same, right? We use cameras later, lasers, radar, computers, maps to make the really the robust systems work. But along each of those axes, there's been transformational progress, right? You know, whether it's cameras, If you think back to 2003, you might be able to get a one megapixel camera. But with what's happened with the smartphone industry and the explosion of digital photography through that, camera technology is just night and day different. Automotive radar with the broader adoption of emergency collision mitigation braking, right, where your car will beep at you and help you stop.

20:31That's driven a volume in radar that's driven the cost down of that and driven performance up. And so, again, that's night and day from what it was 20 years ago. At the time we first did that 2003 robot, the best lighter in the world was called a SICK lighter. It's a German company. I don't know what SICK means other than not what you normally mean. And it looked like a coffee maker. And it had this, like, Darth Vader lens on it. and it was just a single plane of data that it would generate from it. And if you look today, another technology that's been through that hype curve has been 3D LIDAR, where you get volumetric data back from it instead of a plane of data.

21:14And we've taken that a step further with our first light technology, which sees further and then also sees the velocity of things. So that's taken a step function. And computation is, you know, like it's, everyone laments the death of Moore's law, But, you know, it's, again, night and day what we have on the vehicles today versus what we could put on the vehicles back then. So, on every one of these axes, we've seen major enablements that have allowed us to get to where we are today. And then on the algorithmic side, you know, the explosion in ideas of how to use computation efficiently to do reasoning and to do inference has been just, again, amazing to watch.

22:04Where, you know, decision trees and k-means clustering and some of this stuff was what we were doing back then. And, you know, then we saw the advent initially of deep learning where instead of having to describe the features to the computer, it kind of figured them out as well as learning the relationship between them to now this very deep learning stuff that is in the transformer model and what's happening with generative AI today. You know, it's, again, all of these technologies kind of feed into what's enabling us to actually get traction with the automated vehicles on the road today. Staying on that last point a little bit about the algorithmic improvements and decision trees to now doing something that's more based on artificial intelligence, machine learning, just playing it back to you a little bit, it sounds like what you're saying is previously you almost had to manually map out every condition that the vehicle might encounter.

23:07whereas now the vehicles are effectively just learning on their own in real time. Is that how we should think about it? Yeah. So there's a couple of bits of nuance here. So at Aurora, we kind of saw this wave coming. And you can look back to 2017 and we put a blog post out about how important this idea of data was and using machine learning technology, that's what we called it then, it's now called AI, to make this happen. And part of what we've seen over that time is the importance of, like, one, you cannot solve this problem with a if-then-else kind of reasoning. There's too many cases. There's too many cases, too complicated, too subtle, right?

23:51And so the way we've approached it is an approach we call verifiable AI. So there are some things in the world that you just know to be true. You know that you should stop at a stop sign. You know that when you're driving down the road, you should stay on the road, right? Right. And so we can encode these guardrails and then use AI to actually blend between them to get humanistic, safe, quality driving. And so like concretely an example, if you I don't know if you have kids or how old they are. I do. I do have kids. Yeah. But if they learn to drive at some point, you're going to tell them, you know, keep keep three chevrons or three seconds behind the vehicle in front of you.

24:33right? That's kind of what the driver handbook says. So that sounds like a really straightforward rule, except what happens when another vehicle cuts in front of you, right? It's out of your control at this point. So they're in front of you. Do you hammer the brakes really hard to maximize the gap or do you kind of just let it coast? That's kind of a function of, well, what's that car in front of you doing? What can you see beyond that? Are you planning to make a lane change? Are Are you planning to exit the freeway? There's a lot of factors. What's the vehicle beside you doing, right, that contribute to how you respond to that behavior?

25:09So that's the perfect place to use AI to actually understand the scene and then generate behavior like expert human drivers will. while not giving up the constraints that we want and be able to go and kind of verify that in these cases that are clear cut, that we're going to get the right answer out of the system. And like you said, because of, you know, cameras, the different sensors you may have in the vehicle, you know, the autonomous vehicle could detect far more situations than a human being could at any given moment in time, right? It could be looking at the chevrons and the car next to it And the distance of the vehicle behind it all at the same time and making a decision based on all the information, right, which a human could just never do.

25:56Yeah, it's one of those underappreciated superpowers. And, you know, back when I was at Google, we had an incident where one of our vehicles was driving in downtown Mountain View. And a cyclist came whipping around the corner on the wrong side of the road at night. And the car did exactly what it should have. It stopped and the cyclists went on their way. And so, you know, this became a little clip I would show, like, look at how good it is. It reacted immediately. This person come around the corner and you would have a hard time seeing it at night. And I probably showed that video, you know, a dozen times before I realized that there was actually a person stood on each of the three, three of the four corners at the intersection.

26:43That there was two vehicles lined up at the intersection. Right. And because my natural instinct and my attention got focused on the thing that mattered, that cyclist coming around the corner, the most important thing in the moment. But the car was able to appreciate that, okay, that matters, but there's also a person here I need to care for and that person and this person and the other vehicles I need to pay attention to. And because of the nature of the processing in the system, it didn't become overly foveated and that allowed it to react appropriately. We just saw a situation like this happen on the freeway for us.

27:20So one of our vehicles, one of our big trucks is running on the freeway at 65 miles an hour.

27:29And as we're approaching, you can see that there's a truck stopped in the gore point where the freeway splits. And so that's interesting. And then as you get a little bit closer, there's a person out, you know, servicing the truck or, you know, checking something out in the middle of this gore point. And so as a driver, you know, that's a really dangerous situation. So you're going to pay a lot of attention to that. But right as that's happening, as we're getting close to it, somebody else, a little pickup truck, I think with a trailer, realized they've missed their exit and just sweeps across in front of us to try to get across the exit.

28:06And so as a human paying attention to this, you know, I'm paying attention there. And I'm certain I would not have seen that other vehicle. In contrast, the Aurora driver is able to, okay, I'm going to move over a little bit to make space for that, but I'm going to begin braking because I see this person cutting through and just instantaneously deal with all of this interesting stuff. What happens in a situation where the machine decides there are actually no good options? I thought of that when you mentioned that there are people on all four corners. Do they ever find themselves in that situation?