Ep 107: Off-Road Autonomy & Saving Soldiers' Lives with Overland AI CEO Byron Boots

0:28I can say it for you. out there and repair the robot faster than we can make changes to the software.

0:40Byron Boots is building one of the most exciting new defense companies in the United States right now. His background is in philosophy and computer science, PhD in machine learning, and a professor of machine learning. Byron got involved in self-driving with DARPA and built the most advanced off-road autonomy capabilities. Everyone's paying attention to autonomous driving with companies like Tesla and others that are going to change how our world works. The even harder challenge may be the off-road capabilities and how this can protect soldiers' lives and completely change the nature of ground warfare.

1:09Excited to hear from Byron about the future of warfare and how a philosopher and professor can work with the DOD. I'm Joe Lonsdale. Welcome to the American Optimist. Excited to have here today Byron Boots, the CEO and co-founder of Overland AI. Byron, thanks for joining us. Yeah, thanks for having me. I'm excited to be here. And so, Byron, you have a PhD in machine learning from Carnegie Mellon. You're a professor also of machine learning at University of Washington. Did you ever expect to be building one of the next big military companies? No, not really. So, I got into academia, you know, over 10 years ago.

1:38I was a professor at Georgia Tech and then University of Washington. It really kind of happened organically. We started doing some work with the U.S. Army Research Lab back when I was at Georgia Tech and continued on to work with them and with DARPA at University of Washington. And that kind of just turned into some capability development that I did as part of those programs. And ultimately, we wanted to spin that off and essentially get those capabilities in the hands of warfighters. And so it happened pretty organically. I love it. And so before I go into that, tell us a little bit more about your background and academic career.

2:11You studied computer science then? Yeah, sure. I'll go way back. So I was actually a computer science and philosophy double major as an undergraduate. I went to a small liberal arts school called Bowdoin College. in Brunswick, Maine. After that, I did a little bit of work in robotics as an engineer at a robotics company and then studied neurobiology for a couple of years at Duke University before deciding to go back to grad school. So, at the time, I was very interested in how the human brain worked, but ultimately, I decided I wanted to design intelligence systems and put them out into the world.

2:45And so, I started to study machine learning and robotics in grad school and got my PhD at Carnegie Mellon University doing that. You know, it's interesting. Some of my smartest friends also kind of went from trying to understand their brain to working on machine learning, AI. Is there anything intuitive to you about how the brain works? Is it all helpful for how you think about machine learning and your AI you're doing now? Are they connected? So I almost think of things in the opposite way. So it's really, really hard to understand the nervous system. And people have been studying it for a very long time.

3:14And so I was interested in getting back into computer science and robotics and probability statistics information theory, really trying to understand the framework for intelligence and then try to build intelligent systems based on that. And so ultimately, I believe that these sorts of tools, these mathematical tools and computational tools will allow us to understand the nervous system better rather than kind of thinking about it the other way around. Interesting. So start with math and computer science, build on top, study the emergent properties or whatever, and that teaches you how the brain probably works.

3:44Yeah, that's right. That's right. And what else did you study in philosophy? I want to ask this because, you know, a lot of the people I've worked with, we have this term philosopher builder we like, right? So, so Alex Karp, Peter Thiel, you know, we're very advanced philosophers, Charles Koch, he calls himself as philosopher in chief, the biggest private business in America. So it's cool. You have a philosophy background as well. How does that tie into this? Yeah, so I only have an undergrad degree in philosophy, unlike Alex Karp, who's got a PhD. But, you know, I think ultimately it comes to thinking about, you know, how the human mind works, you know, how humans understand the world and really trying to delve into that.

4:18I actually think computer science and philosophy are pretty closely related. So you might study logic and formal systems in philosophy, and then that's related to the foundations of computation. And then also when you think about philosophy of mind and how that might relate to AI, I think there are some connections there as well. And so as I moved from philosophy into more practical things like computer science and robotics, I still sort of think about what we do as almost like applied philosophy, where you're thinking about these deep questions and you're trying to realize that in the real world through engineering.

4:53Before we move on to all this other stuff and philosophy, were there any favorite philosophers or any philosophy that had a strong impact on you when you were younger? I always liked Hume. He's very sort of straightforward and I always enjoyed reading him. He's a funny guy as well. And so it's a little bit more entertaining reading his philosophy. I love it. I actually, I'm part of my ancestry is Scottish and I've actually also been obsessed with Hume. He's inspired a lot of the ideas for Palantir. So that's very cool to hear that. Well, let's jump forward. So you won this DARPA Grand Challenge, right, when you were first starting to work on this.

5:22And that was about how cars could drive over complicated terrain. How did you first get into that? What were they doing? Sure. So before I started working on the DARPA RACER program, we were working with the U.S. Army Research Lab on developing new approaches to perception planning and control for off-road ground vehicle autonomous systems. So you just think about, like, I have a robot. I want to move it from one location to another. And I want to do that through any type of terrain. So really kind of thinking about undulating terrain, so different ground geometry. I'm thinking about vegetation.

5:54How do you move through that? So after doing that for a few years, DARPA came out with the RACER program. And the idea behind DARPA RACER was to try to move full-scale off-road ground vehicles at high speeds from one point to another. And there were several teams involved in that effort. And we were very fortunate to be selected as one of those teams. So I was the principal investigator for that, leading one of the DARPA RACER teams. What's it mean to be a principal investigator? Yeah, it's basically just the person who, you know, it's like the academic lead for the program. So the person essentially leading the team.

6:28And did you guys win? So that's a it's kind of a complicated question. So, you know, DARPA likes to say that, you know, it's not a competition. What? Are you competitive? But, you know, yeah, I'm a competitive guy. And of course, you know, there were several teams to start. And then there are several phases in the program. And at each phase, there's a down select. So we're still involved in the program and no one else is. And how many years was that later? So we're about three years into the program now. Awesome. So the original Grand Challenge was many years ago. You weren't involved 15 years ago, right?

7:00Yeah, that's right. So the original Grand Challenge was in 2004 and 2005. So these were these challenges which were really designed to kind of incept the technology behind self-driving cars. So actually, back in 2001 in NDAA, there was the idea that one third of ground vehicles in the U.S. military were going to be autonomous by the year 2015. It's pretty impressive that they were thinking ahead like that. And they're a little early. That's right. And I think, you know, it's something which, you know, we're still going after today. But at the time, no one really knew how to actually build autonomous vehicles.

7:38And so the Grand Challenge was really important as a way to get a lot of smart people involved in the development of this technology. And that really started the autonomous driving industry in this country. And many folks came out of the Grand Challenge and ended up working in industry and developing some of the self-driving cars that we see on the roads today. And I have to admit, when I first heard about Overland, I said, wait a second, I thought that was already Solve and Tesla and everyone else the way ahead. But it turns out it's a very different problem in some ways to really perfectly solve driving on just like random train anywhere versus driving on a road.

8:12Right. That's right. So on road self-driving is very hard. The major challenge there is not, you know, how do you actually traverse terrain? I mean, you're driving on roads which are engineered to be a very easy to drive on. The challenge is dealing with all of the different agents that are in the environment. So think about other cars, bicyclists, pedestrians, understanding where they are and their intent. So that makes on-road self-driving very hard. But off-road self-driving has really a different focus, which is on the traversability of terrain. So you have to perceive the terrain around you.

8:47You have to represent that somehow and then find ways for your vehicle to move reliably through the terrain without rolling over or crashing. How far ahead do you have to look? I mean, if there's like a gully 100 feet ahead, I assume you don't want to accelerate towards it versus you might do that in a worse scenario. if it's not like your sensors have to look pretty far ahead at the elevation as well, which is different, I assume, than normal. Yeah. So you're using a variety of sensors on these vehicles. There's cameras and there's LIDAR and they can see a ways ahead of the vehicle. How far does LIDAR work nowadays?

9:16The LIDAR on our vehicles is, you know, about 200 meters, but it's not necessarily, it doesn't give you a detailed picture of the environment at that range. So it's much closer in where it's detailed. And self-driving, like Elon doesn't use LIDAR for Tesla. Sure. But you think maybe for your challenge, it's actually more necessary? I think you can use either camera or LiDAR. And they're both very useful sensors. And so, you know, we use both. We can use either LiDAR or camera alone. So you can use either one. Using both together allows you to get a more accurate picture of the terrain around the vehicle.

9:52But there are situations where you don't want to use an emitting sensor like LiDAR and just want to use a passive sensor like LiDAR. Yeah, you don't want the bad guys to be able to see where the sensors are from. So LIDAR projects into the environment. And so you can see it very easily if you have the right sort of sensors. Interesting. And so let's back up a little bit. Why does ground autonomy matter in national security? What's the strategic significance? Why does the Army care about this? Yeah, so we're already seeing that uncrewed vehicles are super... They're gaining a lot of traction in Ukraine, right?

10:24So because you can basically... Is that because they're running out of troops? But it's partially because you don't have to have a person there, right? So it does save a life. If you are able to move a vehicle forward without actually having a person on board, then that allows you to, first of all, it makes things more safe, but it also allows you to do things tactically, which are a little bit different. With ground autonomy, one of the things that ground autonomy does is it enables one operator to control multiple ground vehicles. So this is teleoperation now, though, but it's not as different.

11:00Right. So so most of the use of uncrewed ground vehicles in places like Ukraine are like teleoperated vehicles. So someone way behind the scenes somehow calling into this vehicle. They have to have line of sight, I guess, most of the time in order to control it. That's right. So it's either teleoperated or RC. So like, you know, remote controlled. And basically, they don't have computers on board which are processing the environment and choosing routes. A human is controlling the ground vehicle at all times. And so that's still useful, right? Because you can move this ground vehicle forward and it still keeps a person out of harm's way.

11:38But you can imagine that if the vehicle is able to move autonomously, if you're able to abstract the control so that a single operator can basically just tell the vehicle where to go, they can start to actually work with multiple systems, right? So the basic idea is that autonomy will enable one-to-many control on the battlefield. It also is useful in environments which are contested on the spectrum. So you can imagine if you have EW in the environment and you don't have a reliable communication link with the robot, then teleoperation and remote control make it much more difficult to actually control that vehicle effectively.

12:19So EW is electronic warfare for our listeners who are not part of this industry. And basically what happens is we've gotten really good on both sides at turning off all the signals. And so you might have all these robots you're controlling and all of a sudden they project something and the robot just stop. They don't know what to do next. Correct. So imagine, yeah, imagine you have like a remote control and you're driving the robot around, all of a sudden you can no longer see where the robot is or control that robot, then it becomes a target. And so if the robot is autonomous, even if you can no longer communicate with it, it can continue to operate and move towards its goal.

12:52So let's step back. So this is basically a new era of asymmetric warfare. We have all this new electronic warfare. We've had obviously Dino on from Saronic. He's hopefully building thousands of these small autonomous weaponized vessels in the water. You have swarms of these things. What does it look like for ground campaigns in just like 10 years from now even if we've done this correctly like was it how's the ground campaign different than it is today for the army yeah so um we're building autonomy which will allow you to just direct vehicles to go from like one location to another that means that you know you have operators which are removed from those vehicles so hopefully that makes the operator safer and then with autonomy and effective command and control work building technology, which will allow an operator to select and task multiple vehicles on the battlefield.

13:43So the basic idea is that a single operator can control multiple vehicles and be a force multiplier on the battlefield. So the way I see it is, you know, even in the last 20 years, the special forces have become more important. You have just very small elite groups that are commanding a lot of information. They're calling in airstrikes. They're able to do a lot of things with a few number of people it seems to me like the whole army is maybe going to go that way where even a small number of people in a platoon or whatever you call it that are that are forward deployed maybe each person in that platoon has multiple different robots around them kind of helping them that they're controlling and so one group of 10 people might even have 50 or 100 different ground robots that each have different abilities to fight with them protect them breach things is that where we're going is it's like it's like you got a lot more force per person in a battle?

14:32So yeah, that's a powerful vision for the future. And we are trying to enable technology like that where a single operator can control multiple vehicles and, you know, therefore project more force on the battlefield. One of the things we were talking about earlier that really stuck with me was these army engineers. Teach our listeners, like, what does an army engineer do in these different scenarios? It sounds like a really risky job. You got to be right at the front dealing with breaching, dealing with complicated things. What's their job? So combat engineers have a pretty wide variety of jobs.

15:01And we've been discussing operations like breaching operations with them, which is extremely dangerous. What's breaching? So breaching is basically forming a path through enemy defenses. So it's an extremely complex operation and also something which is very, very dangerous. So we're working with the combat engineers to provide them with tools, basically autonomous vehicles, which allow them to execute portions of that operation without putting a human in harm's way. So that's the basic idea. Let's talk about how breach team would work normally. So you have a place, there might be all sorts of mines, there's bad guys, there's walls, whatever.

15:37So you could go forward and I think there's these giant vehicles that like throw these explosive ropes forward. What is this called? So those are called Miklix. So you throw these Miklix. And how big an area does Miklix clear potentially during a big breaching operation? So they'll clear up to about 100 meters in front of the vehicle. This is cool. So this is like we can show maybe on the screen what this looks like. But as you're throwing this giant thing forward, it's 100 meters long. It's maybe 10 meters wide, 20 meters wide. Something like that, yeah. And you're blowing a bunch of crap up.

16:04So there's all these mines that hopefully are exploding. You're creating a path. And then before you pick soldiers on that path, you have to make sure a path is cleared. So then the Army engineer has to drive over that path, right? Yeah, that's right. So it's an operation like that. But I'll leave the details to the expert. But basically, what we're trying to do is provide them with autonomous vehicles to make operations like this a little bit safer. I just imagine being the combat engineer whose job is now to drive something over that. Like, I'd much rather have you on my team first. So hopefully these guys are.

16:34And these guys are trying to work with you, I imagine. That's right. And I think just in general, you know, one of the ways to think about this more broadly is that really what we're trying to do is take the most dangerous jobs in the military. and especially when you're in vehicles and maybe replace those roles with autonomous vehicles so that you can take the human out of harm's way. So going back to the founding of the company, what is Overland AI? Give us what is this company? Sure. So we got started with work. I was doing my academic lab at the University of Washington. Like I said earlier, we've been doing some work with Army Research Lab and DARPA on developing new capabilities for ground autonomy, basically trying to develop an autonomy stack that enables a vehicle to move through the sort of complex and contested terrain that you would find in military applications.

17:23And through work with these organizations, with Army Research Lab and DARPA, we developed capabilities that we thought were game-changing. So basically, the idea, you know, basically capability where we can move a vehicle at high speeds from one point to another in a wide variety of terrain and do that reliably. And so once we had a capability like that, we were looking at the military and trying to figure out, OK, how do we actually get this into the hands of the warfighter? And so we decided to form a company to basically take those capabilities, turn it into a product and iterate with warfighters and get it back into their hands so that they have the technology and can actually use it.

18:07And your first products, you've been equipping others. You actually made a huge amount of revenue last year where others have existing vehicles and you're equipping these vehicles to be autonomous, right? That's correct. So in our first several projects with the U.S. military, they have provided vehicles for us, and we've put our autonomy stack on those vehicles. So essentially, taken hardware that someone else has created and turned it into an autonomous asset that you can move around remotely by something called waypoint navigation, where you will drop a waypoint and the vehicle will autonomously go to that location.

18:43Let's go a little bit one level deeper into the technology here. Not every listener is going to be able to keep up, but we'll try to explain it. So, you know, building off-road ground autonomy seems like an extremely difficult challenge because you don't have the same massive data sets, right? All these other car companies doing self-driving. There's these annoying cars that were driving around Silicon Valley for a decade. And they're kind of funny because if you try to pull in front of them, they'd have to stop because they didn't blood block everyone. But these things are driving around forever gathering data.

19:06And they had so much data that I'm told that eventually was able to be key in trading the models. You're going to have different environments every time. It's never going to be quite the same with the land. I mean, is it also just a massive data problem or what else are you doing for this? It's extremely challenging. So a lot of robotics and in particular field robotics is essentially a combination of relying on your knowledge of physics and the environment and data. So you're using machine learning and the perception system in order to try to predict what the terrain looks like. Let's say, you know, behind trees or underneath vegetation.

19:41you're trying to predict areas that you can drive through and areas that you can't. You have to guess like how strong the ground is, I imagine, right? Because some grounds slip away and some grounds not, right? Well, you have to contend with that somehow. And that can be very difficult because it can be hard to tell from a distance. And so, you know, you can only really do as well as a human driver trying to do something similar. And often what we'll do is, you know, if we are moving into terrain, which is a little bit surprising, our planning control systems adjust instantaneously to try to either get traction or, you know, find a different way through the terrain.

20:17Is there ever something, and it's a silly idea, but if there was something that was complicated, would you ever, like, send ahead a drone to look at something to make sure? Because you're like, oh, I don't know what this looks like there. Well, that's a great question. It's actually not a silly idea. I mean, I think that a lot of folks are thinking about things like this. So combining different types of robotic assets, whether they're UAVs or ground vehicles, and using the sensors on drones to provide a better picture of the environment, which will then help the ground vehicle to move through the environment more effectively.

20:45MARK MANDELAVICHERSON, Awesome. And some of these vehicles, they're going 30 miles an hour going around ditches and fallen trees and boulders. Is that kind of stuff? MARK MANDELAVICHERSON, That's right. Yeah, we've actually gone up to about 35 miles an hour on a pretty wide variety of terrain. So this includes off-road terrain, so pure cross country where there's no roads at all, but we also go through double track and dirt roads. So they're pretty high rates of speed for terrain, which is very difficult. And what's your, if we could ask, obviously it's secret, but what's your secret sauce? You don't like to say you won the competition, but you're the only one left.

21:15Everyone I've talked to, I can say it for you, everyone I've talked to in the DOD, has said really great things and is really excited. You guys are the best, they say. So you're iterating multiple times per week with software and hardware, different environments. How come you got to be the best? Yeah, so I think you are starting to kind of hit on some of our secret sauce. I don't mind saying it because it's a hard system to replicate. So basically, you know, we one of the things about robotics is it's basically software and hardware working together. And then field robotics and robotics in some of the domains that we're talking about requires that software and hardware to work really well in complex environments.

21:50People who are just kind of starting work in robotics often think that you can just do things in simulation. But they quickly learned that it is really important to actually run your software on the robot. And then, especially in field robotics, run that robot in the terrain. So just learn quickly a lot. Yeah. And so for us, just kind of getting back to the secret sauce, we started out by building an extremely capable mechanical engineering and field team that allowed us to put robots in the field all the time, to iterate in the field with our software and, crucially, to make mistakes. And so we can run robots very fast through ditches, potentially break the robot, and then have a field team which is able to come out there and repair the robot faster than we can make changes to the software.

22:38So this allows us to iterate just over and over in a wide variety of terrain and just learn a lot about that coupling between software and hardware and terrain and then take those learnings and, you know, build better software and better hardware. And so that really is the secret sauce, that continuous field testing and that sort of field forward, you know, that kind of way of thinking about the problem. I love it. Well, having a top engineering culture that iterates quickly, it's similar to how Palantir and other companies work. Let's talk about the swarming aspect as well. You mentioned command and control earlier called C2 by a lot of people in this sector.