#182 Dragomir Anguelov: The Role of AI and Machine Learning in Waymo's Self-Driving Cars

0:00In Waymo's charter, we would talk that the Waymo driver, we eventually want to enable many different kinds of use cases and form factors. I think our first kind of flagship application is right-hailing. But we're building a general stack, right? I mean, we discussed that maybe we can transfer a lot of the models or types of models that we build for autonomous vehicles to other mobility robots. But even before that, right, we aspire to build a stack that learns from data and generalizes across these platforms and enables various, I mean, autonomous driving applications, preferably with as similar of a system as possible or maybe the same system.

0:43So it will take a few years to continue this trend until people really feel it all over the place. And to me, mostly, I don't count how many cities we will be in. I want to build the stack that makes it economically great and scalable to do a city N plus one. And so there's a ton of headroom still possible with the technologies that are being developed. And so I hope we can facilitate it. I believe it's not far out, right? Hi, my name is Craig Smith, and this is Eye on AI. In this episode, I talk to Dragomir Angolov, head of research at the autonomous vehicle company Waymo. Drago discusses how Waymo's research team has leveraged machine learning across perception, behavior prediction, and planning to enable 24-7 fully autonomous ride-hailing services in San Francisco and Phoenix, with LA coming soon.

1:40He talked about the Waymo Open Dataset, including the 2024 Waymo Open Dataset Challenge, which kicks off this month, and its impact on advancing AI research beyond autonomous vehicles. Finally, Drago gives his views on when we can all expect to be summoning robotaxis on our phones wherever we live. I hope you find the conversation as exciting as I did. At home, at work, we all know one person who's password challenged. Sticky note reminders, emailing passwords, reusing passwords, using the word password as their password. Because data breaches affect everyone, you need 1Password. 1Password combines industry-leading security with award-winning design to bring private, secure, and user-friendly password management to everyone.

2:36Companies lose hours every day just from employees forgetting and resetting passwords. A single data breach costs millions of dollars. 1Password secures every sign-in to save you time and money. 1Password lets you switch between iPhone, Android, Mac, and PC with convenient features like Autofill for quick sign-ups. All you have to remember is the one strong account password that protects everything else. Your logins, your credit cards, secure notes, or the office Wi-Fi password. 1Password generates as many strong, unique passwords as you need and securely stores them in an encrypted vault that only you have access to.

3:22I use 1Password, and you should too. 1Password's award-winning password manager is trusted by millions of users and over 100 ,000 businesses from IBM to Slack. It beat out 40 other options to become Wirecutter's top pick for password managers. Plus, regular third-party audits and the industry's largest bug bounty keep 1Password at the forefront of security. Right now, my listeners get a two-week free trial at 1Password.com slash IonAI. That's IonAI all run together, E-Y-E-O-N-A-I. That's two weeks free at 1Password.com slash IonAI. 1Password.com slash IonAI I was a tech lead of 3D vision and pose estimation for Street View and got a lot of exposure to all kinds of platforms that the team was collecting data with, including boats, snowmobiles, backpacks, and of course cars.

4:31And afterwards, I got back a bit closer to my machine learning AI roots. I worked on a team and eventually ended up leading a team that developed, even in those early years, 2013 to 2015, large neural networks for understanding images. And so in those times, we had a very exciting set of developments and architectures that we invented. And we won the ImageNet challenges in 2014. That used to be a very popular competition on object understanding and detection and classification. So we won those two challenges in 2014 with the team that I led and some collaborators from Google Brain. And we also worked a lot on launching the backends that would annotate Google Photos.

5:26So Google Photos was first of its kind to offer deep semantic annotation of photos enabling search and all the photos applications that they would do where they suggest you various events and albums and so on and organize them. That was based on backends and models that we developed at the time and that we launched in the Google data centers. and around 2015 when I felt that deep learning has entered a level of maturity that is sufficient to potentially make autonomous driving a reality, I got back into the autonomous vehicle space, into robotics that I had been exposed to in my early lab days and so I joined initially ZOOX for two and a half years.

6:17It's an autonomous driving startup. I led the perception team there helping them build out their essentially 3D understanding and the machine learning around it. And then I came to Waymo, right? So that's my story, more or less. I would say I'm a machine learning person and my initial expertise had been understanding images and 3D data, geometry, and that's generalized to other applications since autonomous driving is a, I mean, it's a complete robotics stack, which includes all the key robotic systems, perception, planning, behavior prediction, simulation, mapping, localization, and so on. So by now I have this maybe more of a cross-functional exposure to all of this through the lens of machine learning.

7:16Your undergraduate was where and then at Stanford you were studying under Daphne Koller. And what was that work about? I actually did my undergrad at Stanford too, but afterwards I continued into a PhD that took six years. So I spent a total of 10 years at Stanford. It was hard to get me to leave for a while. I worked closely with Sebastian Throne because I was Daphne's first PhD student that worked on computer vision perception topics and we wanted access to robots and data from robots and at the time sebastian wasn't even at stanford but daphne somehow made a connection with him and i ended up visiting his lab in carnegie mellon and collecting data with robots and shorting one robot I'm a software person.

8:08Fortunately, there's things I learned the hard way there. But Sebastian was always a commenter of sorts, even though my primary advisor was Daphne. I think I always worked with Sebastian as well, and we have joint works, and he was a key influence, too. Yeah. And so Waymo, when you got involved with the company, where was it in its development? And where are you right now? I mean, I know you have robotaxis operating in San Francisco and Phoenix, and I hear you're going to have them operating in LA. Yeah, we have San Francisco and Phoenix today. You can experience autonomous driving. It exists at scale in these cities.

8:58We give tens of thousands of rides to users in both cities every week. And Waymo overall has given over a million of paid rides, driving over 10 million miles autonomously as well. So that is the two main markets that we have. The Phoenix area that we cover today is around five times the size of San Francisco. And next, we're looking to expand and start actually paid rides in the coming weeks in Los Angeles. We got a permit a couple weeks ago to expand to Los Angeles in charge. And so we will start doing this. And we also started doing driving driverless rides for employees in Austin, Texas. So we're going to be expanding in this fourth market that we have announced as well.

9:54My time at Waymo has been a wonderful time, seeing a transformation of the stack and getting to maturity that I think has been great to experience and working with a talented team. I think when I joined Waymo in 2018 in the summer, Waymo had hit some great milestones. One of them was in 2015, it gave the first full autonomous ride in Austin, actually. The city we're getting back to, the person called Steve Mann, a friend of the company as a blind person, the first ride involved him. and so we had done that and we were working towards our first commercial deployment and so we're evolving the stack to make sure that it will be a safe and scalable deployment and this eventually happened in 2020 when we launched our Waymo 1 service in Phoenix East Valley around Chandler and so that's been the journey so evolved to a first full autonomous service in Phoenix East Valley, then get to San Francisco and then significantly upgrade the stack to be able to handle the complex, dense urban environment of San Francisco.

11:14And now we continue to all these more cities, expanding in Phoenix. We will be working also on tackling more highways, which is important in large areas like Phoenix. Our area, if we don't go on highways, takes almost an hour to cross on surface streets today so uh so yeah that's i think maybe uh the trajectory of the last five and a half years it's been tremendous also in in terms of development of machine learning and machine learning technology so in 2015 when i joined 2018 excuse me when they joined waymo there was already a lot of impressive machine learning work um especially there was a lot of really interesting, unique work on 3D perception using LiDAR.

12:02And this was in collaboration with Alex Krizevsky and Anilie Angelo and other people from Google Brain, Alex Krizevsky being of the 2012 AlexNet fame. So he took a lot of interest in autonomous driving as an application, and he worked a lot with Waymo and developed some really interesting work, including inspiration to a first paper Waymo ever did. Even though we published it in early 2019, it's called ChauffeurNet. It's actually a trained model to predict and drive by imitating others. That was really groundbreaking, unique work at the time. And then that was still relatively early in its development and maturity, but it was very exciting as a concept.

12:52So we published that and we took all these technologies and we worked a ton on expanding them and transforming Waymo stack. I think at this point, Waymo is an AI first company. So we strive to maximize the impact of machine learning and its power across most of our applications. Yeah. I haven't spoken to anybody about autonomous driving for a while. I did an episode with Alex Kendall at Wave AI, but we were really talking about world models. In the tech stack at Waymoor, from what I've read, you have detailed custom maps for navigation. You have this suite of sensors, and then you have neural nets that are kind of wired together to process that data and and guide the car.

13:52How much of that, how much hard-coded stuff is in that stack, and how much is pure machine learning? So big parts of the whole stack of machine learning, and this spans all the core systems of Waymo. This spans things like perception system, and it also covers major models applying machine learning to planning, behavior prediction of others, and simulation, and also lots of machine learning for employed also in evaluation, extracting signal from, for example, simulation or other tests we perform on the car. So at this point, I would say all major parts of Waymo have powerful, large machine learning models underpinning them.

14:42Are all those models on the car, or what's on the car, what's in the cloud? So we have some flexibility. I think especially the whole evaluation system simulator that is in the cloud, right? Because we develop a variant of the stack and then we need to test it. And we test it in the data centers to a large extent, even though we also test it with human safety operators and we won't have specialized areas where we can stage certain rare situations. but testing in the cloud and simulation is key. I think generally we avoid requiring a human to drive our cars at any time. It's possible to ask human occasionally a specific question in some very, very rare situations when preferably the vehicle, or usually I think almost always, the vehicle is stopped at that time.

15:40can be, but that is also extremely rare. And we are trying to minimize this. So the vast majority of the system that drive the car, they're on the car, on the local compute, and they operate fully, I mean, autonomously as advertised. So how many models or neural networks are involved in the driving system? So there are several, right? I think the history of our space is one of consolidating models, right? And traditionally, many players in the space, and also when machine learning was in its relatively early stages, and there was limited compute and data, it has been beneficial to, or at least at the time to partition the tasks and have maybe one neural net to detect bounding boxes and another to segment, say, the scene into semantic segmentations of various kinds and another maybe.

16:40So traditionally that was the history. And now what's changed is we understand the power of... The models have become a lot more powerful. Their scaling properties have become much improved, especially with the advent of transformers. We understand a lot better how to make models do well on many, many tasks at the same time. That's actually been area of research by us and the whole community. And some of these AI trends can be leveraged, train very large models, so transfer knowledge from the internet and fine-tune on your tasks. So we have ways to make models do well on a lot more tasks today.

17:25And there is benefit then in efficiency and simplicity and quality usually to consolidate. So there has been a push to consolidate. That said, it's an ongoing process. There's some very interesting question whether you should have one model or several. There is challenges in having just one model in our case, especially if you need to iterate on it and ensure it satisfies, I mean, hundreds to thousands of requirements, right? You need some way to have, to be able to do work on parallel problems in this domain to ensure safety. And so there is this constraint. And there are some other constraints that make it difficult for our case.

18:11For example, compared to, say, traditional robotics. Maybe you talk to Peter Abiel or Vincent Van Oog. They have a camera on a robot, and then they have a world model, and they can train it end to end. For us, it's more challenging infrastructure-wise because we have, say, over a dozen cameras to two dozens. We have a few LIDAR, we have a few RIDAR, fitting all this data for all-in-one on the compute and scaling it is quite daunting. And that even motivates potentially training things in stages and so on. So I would say that the answer is we're evolving and we're evolving, satisfying both in the learnings in the industry and the requirements in production.

19:01and generally the trend is to consolidate. Yeah, and I know it's a very complicated system, but can you sort of walk us through how the car drives? I mean, they're the sensors and they're feeding the data into a perception model and then the perception, I mean, just kind of walk me through the pathway. So the simplest way to think is of the onboard system, right? And there's a lot of very interesting problems and questions when you build the simulator as well. But let's talk about, which is also a machine learning task to me, by the way. I'm a machine learning person. And ultimately, we want to solve problem with the data we have as much as possible.

19:46But the traditional autonomous vehicle stack takes data from sensors, can be camera, lidar, radar. And then there is a perception stack that processes this data and creates a representation of the world. In this representation of the world, one of the key aspects of it is it wants to be expressive, but also concise, right? And in that representation, then you train your behavior and potentially simulation models. And because that's a representation of the world, and then you need to, in that representation of the world, you need to plan what you do, and you need to also potentially imagine what the world is going to do in response to your plans to validate that your plans can be saved.

20:32So essentially, there's these main two parts to the stack. Perception produces a model of the world. And then in that model of the world, in plan, you validate your plans. That's the behavior system. And the model of the world is very interesting. So there is a lot of interesting question of what it can or should be. Traditionally, we have opted for at least partially interpretable intermediate representation. So one of the things that has been very beneficial in our space, and I think it's generally superior to treating all the sensors as an ordered collection of sensors, is to take their data, fuse their representations and the knowledge provided to them, and create what we call a bird's eye view model of the world.

21:17So essentially you center a grid or implicit grid on the space around a vehicle, and you essentially populate this grid with everything, all the information that you acquire from these sensors. So that's been very popular. And there is certain benefits to having interpretable representations. People understand them. They know what the problems are. They can annotate or correct them. And so the drawback of those is you need to label data to create them, typically. They're representations that we have chosen because they satisfy requirements, interpretability, testability, right? Potentially people can look at the outputs of a system and impose further constraints or corrections.

22:06That's why you asked me, is all your system just a neural net? Our system is hybrid, even though big parts of it are neural net, and we strive to solve the majority of the problem with machine learning. We still build such representation and design where you can control and steer the system by experts that can introspect the results and introduce additional constraints, requirements on top of what the machine learning models do. And that has been very beneficial for robustness and safety. I guess that's what Wave is, where they're different. They're creating a world model, and then everything happens.

22:45All the planning happens in that. The simulation and planning happens within the world model. I actually don't think they're that different, honestly. I think maybe it was oversold. You have a perception system. It produces a compressed version of the world. In our case, some of it is interpretable, but you can also have various embeddings or quantized embeddings of tokens trained end-to-end that can also be produced. It's a question of choices in that representation. And then given that representation, you have prediction models. And these are, you can call them their world models. And there is a question of which exactly world models do you build.

23:27Do you build a model that just dreams all the video of all your cameras? Well, you can, but on board, maybe that is overkill. So then which things do you choose to dream and auto-regress or predict concisely so that you still satisfy your requirements? That's where you have leeway. But we still have major machine learning models that are word models. Just the design of them may be different. And we also can train end-to-end. However, we have also put historically a lot of focus on having intermediate representations because we have found they're very important so far in actually building a fully working system.

24:14Machine learning is great, but it has limitations. There's always cases where eventually you want to impose guardrails or constraints, and intermediate representations are very powerful for doing it. Now, lately, there's been yet one more trend with large language models and the like. Now, there is a lot, I mean, the kind of language-based representations or language-compatible embeddings are very, very popular for robotics because they bring a ton of knowledge from the internet. and that said, the model, the language models were pre-trained on, which makes it, I mean, the system understand a lot more common sense and be able to interact with humans.

24:56So now that is also very interesting to us, right? So language or language compatible representations are great. That said, there is also another learning in our domain. You really need these representations to be spatially correct. So our models, the perception system needs to be very powerful at spatial reasoning, because spatial reasoning, the ability to represent the world and ultimately evaluate your plans in the world is key to ensuring safety. And so these concepts are now overlapping. And I think we have a lot of exciting, both experience, expertise, and of course, we're also experimenting with the best mix of these concepts.

25:39Yeah. Yeah. I mean, I just had Vincent van Hoek on the podcast talking about RT2 and that system. We are very much inspired by their work and follow it. And I think there's a lot of parallels to our domain. There is certain differences where we are a lot more concerned about correctly predicting and modeling the behavior of others just because in our domain, it's so important to interact correctly with all the traffic participants. You know, a lot of people don't think of cars as robots. They're absolutely robots with well-established hardware stack, yes. Is your system, could it be adopted or adapted for other kinds of robots?

26:27I think definitely it can be adapted. I think a lot of the concepts across the various robotic systems over time, I believe, are converging. There is still certain peculiarities in our domain. You can think of our robot maybe as the first manifestation, maybe the most economically, hopefully impactful and application impactful of most robotic applications. At least I believe it. And that's why I'm in this domain. But a lot of these concepts of how to build your stack, how to test your stack, how to ensure robustness, what kind of machine learning models you do. I think we're becoming more and more similar across robotic.

27:08But yet, in our case, safety, especially at high speed, we have a lot more requirements of those kinds. Like you need to have very fast reactions. You need to have very high bar to safety, right? You're interacting sometimes in parts of second with other agents who could behave also irrationally or adversarially. we need to handle all of this. And the other part is we have very many sensors on the vehicle, right? So to see in every possible direction, and in our case, we've opted for a lot of additional safety by adding LiDAR radar, which are active sensors. They ensure that even if machine learning in cameras misses something, we have other ways to capture it.

27:49So when we put all this together, there are certain differences, but if you build the stack for autonomous vehicles, we can build on top of the strengths of this stack. We can adapt it to make. At home, at work, we all know one person who's password challenged. Sticky note reminders, emailing passwords, reusing passwords, using the word password as their password. Because data breaches affect everyone, you need one password. One password combines industry-leading security with award-winning design to bring private, secure, and user-friendly password management to everyone. Companies lose hours every day just from employees forgetting and resetting passwords.

28:33A single data breach costs millions of dollars. 1Password secures every sign-in to save you time and money. 1Password lets you switch between iPhone, Android, Mac, and PC with convenient features like Autofill for quick sign-ups. All you have to remember is the one strong account password that protects everything else. Your logins, your credit cards, secure notes, or the office Wi-Fi password. 1Password generates as many strong unique passwords as you need and securely stores them in an encrypted vault that only you have access to. I use 1Password and you should too. 1Password's award-winning password manager is trusted by millions of users and over 100 ,000 businesses from IBM to Slack.

29:26It beat out 40 other options to become Wirecutter's top pick for password managers. Plus, regular third-party audits and the industry's largest bug bounty keep 1Password at the forefront of security. Right now, my listeners get a two-week free trial at onepassword.com slash ionai. That's ionai all run together, E-Y-E-O-N-A-I. That's two weeks free at onepassword.com slash ionai. Onepassword.com slash ionai. Yeah. And as the system evolves, when you were saying that it's evolved a lot and then there's all this new research coming out with generative AI and large language models, multimodal models.

30:17It's a great time in our space. I think robotics is, I think progress in robotics is accelerating even though, you know, we're an example of already a successful robotics product out there in the real world. But your model, how do you, how do you integrate these new technology? But when do you decide to bring that into the car? The great thing at Waymo is we have the data and the frameworks to test the performance of our stack. And so the history at Waymo always is, you know, we've been around 15 years. And the technology in these 15 years has been dramatically evolving. And every two or three years, there's major advances in our space, right?

31:01And so we've had to continually rethink and redesign our stack to take advantage of the latest technologies. And so the core is, as in most machine learning, is we have the data and the evaluation. We have collected, as you know, we have many hundreds of vehicles out there in the world with very rich sensors. We have a lot of systems that can process this data, collect the interesting cases, potentially automatically annotate things we want. If we, for super rare things we can get, we have systems to annotate human guidance and so on, right? And so we have all this data. Now it's a question of, it's not that hard to try new paradigm or models and compare to the performance of your previous models.

31:46And anytime we can show a significant improvement, that's then we evolve the stack, right? And this has been done many times already. So, you know, we continue doing this. But the data is the key, right? So because you have all this data and you have all these testing frameworks and safety frameworks and very rich evaluations, the key is to be able to tell when you're better, right? And we worked hard to be able to do that. Yeah. There's so many questions I want to ask. One is on corner cases that arise in this long tail. I read something, this is four or five years ago, I think, of a case where an autonomous vehicle, it may have been Waymo, drove up behind one of these big tanker trucks with a highly polished back that was like a mirror, and that confused the system.

32:41How do you deal with that? Do you add that to your training data or do you tweak the algorithm? I mean, how do you deal with those corner cases? So you can annotate your data, right? And you can use it either for training or for testing. And clearly, the very first thing you want to ensure when you see very rare cases is you want a test set to ensure that you got it right. And then you look for more cases like this and you add those to training set to help you get it right. But I think in our case, because we have also radar and have LIDAR and we have camera, we will see the truck. We won't see just a car reflected in the truck.