Safer, Smarter Construction Sites with Edge AI and Caterpillar Autonomous Machines - Ep. 285

0:10NVIDIA AI Podcast Host:Welcome to the NVIDIA AI Podcast. I'm Noah Kravitz. With me today is Brandon Hootman. Brandon is Vice President of Data and Artificial Intelligence at Caterpillar. And we're speaking just after CES Week 2026. Happy New Year, everybody. It's a big week for Caterpillar, so we're going to get into all the exciting stuff that's been going on this week, but leading up to it, certainly. But before we do that, Brandon, welcome to the pod. Thank you so much for taking the time to join us. No, awesome to be with you. Big week for Caterpillar. We are excited to be on stage at CES and also excited to announce some things in partnership with NVIDIA.

0:51NVIDIA AI Podcast Host:Fantastic. We'll get into all that in a second. But first, if you would, can you tell us a little bit about Caterpillar? I would think most people listening know the brand, have seen your machines, have seen the logo. But if you can give us a little bit of background about the company, and then also maybe a little bit about your role, just kind of to set up the conversation. I'd be happy to, Noah. So 2025 was a big year for Caterpillar. We celebrated our 100-year anniversary. So we've been helping build a better world in multiple areas for the past century. and really proud of that. And I think a lot of the audience would recognize Caterpillar from just the drives down the interstate and you see our machines doing the work.

1:28And that's one area of our business, but we've also got a pretty significant footprint in other areas that individuals may not know about. We do a lot of work in heavy mining and then we also do a significant amount of work in energy and power generation too, which with AI now really taking off, as you can imagine, has been a huge part of our business. We're really excited by that.

1:48NVIDIA AI Podcast Host:Yeah, for sure. And so what about your role? How long have you been at Caterpillar? If you want, you know, a little bit of backstory of how you landed here. But then maybe you can just kind of describe what you do as VP of AI and data. Yeah, yeah, we'll do. We'll do. So I'm a software engineer by my education and by most of my time in the workforce. I've been with Caterpillar for 27 years. It doesn't really feel like that, but I've been here now for 27 years. And over the course of my 27 years, well, it feels like you could look at that and say, well, you've been in one spot for quite a while.

2:24I probably had six or seven different careers over the course of that 27 years. So it's been pretty cool to do. I've spent time in our supporting our dealer network and the systems that they use to run their business and then our manufacturing systems. And now, of course, I'm in our digital organization where we're building an ecosystem of digital capabilities that complement the machines that we sell to market. And our goal with that is just to make our customers more efficient, more effective, more productive, more safe with our equipment than any other in the market.

2:53NVIDIA AI Podcast Host:So how is Caterpillar thinking about AI when it comes to reshaping modern manufacturing? We'll start there. Yeah, that's a great question. So in addition to my role that I play in CatDigital, I also serve another role for the enterprise. It's a recently established function called the AI Accelerator. Okay. And it truly is about accelerating the adoption and the value that we get from AI across the enterprise. As you can imagine with our business, NOAA, manufacturing and supply chain is a huge component of that. With Caterpillar having been around for 100 years, we've had, you know, probably for 60 of those 100 years, we've had systems and data produced.

3:32And, you know, the concept of transforming your enterprise in those areas used to be that you go through a big systems deployment, ERP deployment. What we found with AI is it's kind of breaking through the traditional barriers of accelerating, getting to those big transformations without the heavy lift that you'd have of kind of ripping out the core of your systems to be able to do that. We're looking at AI from, and I would say, Noah, we're starting with the foundation. The concept of a digital twin that can truly blueprint your manufacturing facilities, each of the work centers that you have, machining centers, your supply chain that complements that, that first basis then reinforced with real-world data opens up a world of opportunities.

4:17opportunities. Predictive and preventative maintenance on key assets on your shop floor, being able to simulate in real-time supply chain constraints and then optimize your build schedule for what you're going to be building in the next day, next week, next month. All of those scenarios that felt like they were really hard to go achieve, we're finding that AI is allowing us to break through those walls pretty quickly.

4:43NVIDIA AI Podcast Host:That's fantastic. Yeah, we've spoken in the past months to folks using digital twins. As you said, in factory manufacturing situations, automotive comes to mind. You guys are in the business of creating heavy machinery. And as you said, have your reach into other industries and kind of areas as well. You're collaborating with NVIDIA on the AI transformation. Can you dig in a little bit more and kind of talk about what that looks like day-to-day and whether that's for cat customers, dealers of your equipment, or maybe even your own employees. Yeah, I'd be happy to. I guess the first thing I would highlight is that the reason that we're working closely with NVIDIA is the comprehensive approach to an AI ecosystem, not just hardware, not just some libraries and frameworks, but it's a really well-thought-out ecosystem that allows you to simulate, train, and deploy.

5:43Those are the core fundamentals of getting AI right, irrespective of where you're at in an enterprise like Caterpillar, whether we're doing AI to make our customers more efficient in the way they work with our machines, or whether we're making our supply chains more efficient, or whether we're looking at core processes like finance transformation, etc.

6:01NVIDIA AI Podcast Host:So those concepts in and of themselves are almost immutable, Noah. But very specifically, when we look at the work that we're doing, and I'll just zoom in a little bit on specifically our customers and some of what we announced this week at CES. Great. There's a lot of, especially when you get into what we would traditionally call blue-collar work, there truly is a shortage of skilled and qualified labor in the marketplace. And it really doesn't matter whether for us, it might be within our factories, operators, people that actually work within our factory. You go to our dealers, it will be service technicians, and you go to our customers as people to operate the machine.

6:40So we're thinking about AI as a very pragmatic way to solve those problems in a very tangible, very outcome-based way. And starting with our customers, it's simple as making sure that when an operator steps into the cab of our machine, they can understand the operations of the machine. They can get feedback on how they're operating the machine. They can get coaching. The advanced technology features that we have on our machines are very accessible in a natural language way. So by bringing our Cat AI assistant into the cab of our machine, it really allows that operator to get access to our entire digital ecosystem.

7:21The hundred years of collective knowledge that we've built, and then the ability to get real-time interaction with that machine in a natural way where you're not having to open a cell phone or look at a laptop screen to go do these things, it's just there, it gets out of the way, and it kind of helps them do their work. And we're excited by that. A big enabler of that, and part of the reason that we're working with NVIDIA is the edge compute platforms that NVIDIA provides. The Thor ecosystem is class-leading, both in terms of compute power, but also the software frameworks, the libraries, and the ability to quickly deploy.

7:56What we saw at CES that we were able to demonstrate with AI in the cab of an excavator, no, that wasn't a five-year research project that we jointly did. That culmination of that work would be measured in months, not years. So it just shows our ability to take these technologies at scale and rapidly integrate them with some of the best machines in the world. And we're really excited by that.

8:23NVIDIA AI Podcast Host:To go back to what you were describing about, to me, it sounded like this co-pilot scenario or metaphor that's used a lot when we're talking about, you know, humans using AI tools in whatever line of work or whatever they're doing to kind of get that assist. Yeah. What's it like in the cab of a cat machine? What is the operator? And I'm sure they're all different. But when you're talking about, you know, the AI systems being there to help, what is the operator seeing? Yeah. So, and we're still working on what that final experience will be like. You know, the opportunity to demonstrate at CES. Shortly after, you know, coming up here quickly will be Con Expo, Construction Expo.

9:05It's a really big industry event for the construction industry. That'll be in Las Vegas, where our customers will get an opportunity to go more hands-on with that and get to see that. So we're looking for feedback on what that final experience will be. But one of the things that we wanted to make sure that we did with that, Noah, is that this was never about the AI being front and center. It was about helping our customers get the work done.

9:29NVIDIA AI Podcast Host:Yeah. And allowing them to get to all of the advanced capabilities that we've either built with on the machine or across our digital ecosystem in a way that felt natural for them. So we want them focused on digging, moving, hauling, not playing with another screen. And we do have a screen that, in our concept machine that we've showed where things come up. So as an example, If you have questions about operating procedures, that screen will visualize those operating procedures and show them to you. But other than that, you could have a voice interaction with the assistant and get everything done that you need to and never have to worry about taking your eyes off of the job site.

10:08Again, my compliments to NVIDIA. We were very quickly able to do that with NVIDIA's Reva voice services and Parakeet models, etc.

10:17NVIDIA AI Podcast Host:Glad to hear that, but it's always about the collaboration, right? Right. And as you said, it's for all of the advances, all of the speeds and feeds and benchmarks and everything we talk about when it comes to AI and data and everything that goes into it. As you said, it's about getting the work done. And really, the more the system can stay out of the way, but give the operator what they need to focus. Yeah, that's what it's all about. That's great to hear. You know, and that's such a salient point, Noah. And I would say as we've really hit stride with AI, that's what we've found. its ability to naturally help you just get things done in a way that isn't intrusive, that doesn't require you to have deep knowledge on systems and menus and where data is at.

11:01It just allows you to naturally get to that outcome. That's the game changer. And it's not AI instead of people. It's AI with people to help them be more efficient at the work that they're doing and be better at the work that we're doing. And we're really excited about that.

11:17NVIDIA AI Podcast Host:Absolutely. Kat's using NVIDIA's AI factory and your manufacturing digital data platform to automate things like forecasting and scheduling. Can you speak maybe to a manufacturing process that's really been overhauled that just looks completely different now because of AI? Yeah, and I had hinted at this a little bit earlier, Noah, but let me double click on it. And for your audience, I'll apologize. I'm going to go a little bit deep into how factories are ran here. So just bear with me for a minute. Go deep. That's why we're here, yeah. But if you think about the complexities of running an operation like what we do at Caterpillar across a really diverse product line, it's a multi-layered supply chain where you may see final assembly happening in a factory, but there are a lot of predecessors that are feeding into that.

12:05Either parts coming from suppliers or other facilities that we've got that are doing work on components. So as we are getting our orders coming in from our dealers and customers, you want to be able to ensure that you have everything available to be able to produce those orders. So that process, and I think this is probably an industry term, but we use it pretty heavily at Caterpillar. It's called clear to build. It's very simple. In a 30-day window, in a 14-day window, in a 5-day window, am I clear to build? Do I have all of the parts and components, the supply that I need, either in the network or available at the factory?

12:40Do I have the labor that I need to be able to do it? Now, think about Caterpillar at the scale that we are and all of the data and the disparate systems and the signals from your supply chain that have to go into calculating that. It's really hard. And without everybody being on a singular system or a singular platform, you're dealing with kind of an asynchronous environment of multiple signals coming in and being able to parse and process those at scale is really, really difficult and has required a lot of really smart people to do in the past. And even with that, it's been difficult to calculate on a long-term window.

13:15So we have found that with, you know, first with Omniverse, building a good digital twin that shows an accurate representation of your factory, the capacity of the factory and the supply chain, and then taking some of the NVIDIA models and the inference services, Qopt as an example, we were able to take the clear to build process and calculate that for a 30-day window in 100 milliseconds.

13:39NVIDIA AI Podcast Host:Wow. Yeah. That's remarkable. Yeah. Now, granted, we're doing it at a facility level right now, and we're working through the rest of the enterprise to be able to do that. But that just shows you the power of, once you have your arms around the data, once you understand the platforms and working within the NVIDIA ecosystem and the AI factory, the power of what you can do. And these aren't models that we had to go develop on our own, compliments to NVIDIA. Things like Ku-Opt are really good, and they work really well. So with a little bit of tuning on our side, we were able to make those things work at a pretty impressive scale.

14:14NVIDIA AI Podcast Host:I'm speaking with Brandon Hootman. Brandon is Vice President of Data and Artificial Intelligence at Caterpillar. CAT, as the logo says, and people know you, as you said, for, you know, you're driving down the highway and you see the machines. But so much more than that, as we've been discussing. Speaking of the machines, when operators are out, they're working on sites, and they're using the machinery and the CAT systems that now have or are starting to have these AI guidance, you know, sort of co-pilots, as you were describing. How does AI at the edge come into play? How do edge AI systems enable real-time decision-making and kind of just adapting to what must be incredibly unpredictable environments when you're out on a construction site or in a mine?

15:00Yeah, you nailed it, Noah. And you think about the dynamic nature of a construction site. Day to day, they're never the same. Things are moving around, different work is being done, things are being completed, materials being brought to the job site. The very nature of it isn't, it's not like our factories where things are relatively predictable and you've got a line that things are moving down and it looks the same day in and day out. So if you think about what it takes to do some level of automation and bringing AI to a job site, even as performant as networks and cloud systems are today, there is a level of processing that has to occur at the edge.

15:36And that level of processing doesn't just require CPU or GPU compute. It requires a level of storage. It requires a level of memory to be able to process these things in as close to real time as you can get. Situations with safety can sometimes be just a matter of seconds between something being avoided or something really serious happening. So for us, as we're thinking about what we bring to the machines and what we bring to the job site, this partnership with NVIDIA and the architecture and capabilities that Thor brings is really what we see as the game changer that's going to allow us to bring the capabilities that we've had in pockets for a while to now start to bring those to customers and machines en masse.

16:27And the way that we're doing this now is Thor allows us to bring a lot of the sophisticated AI and ML models that we had built that run across our data platform that powers digital today. We can now bring those to the machine. And in the event that we have network connectivity, in the event that the scenario supports it, we might invoke a cloud agent. But if there's too much network latency, we've now got the ability to revert back to something that can run on the machine at a fidelity and a level of insight that still allows the customer or the operator to be effective in the tasks that they're doing.

17:07NVIDIA AI Podcast Host:Could I ask you, Brandon, to dig in a little to what some of those capabilities are kind of at the machine level in whatever scenario, construction, mining, whatever it is that makes the most sense? Yeah, you bet. So just think about a perception-based warning of needing to understand maybe an unsafe scenario on a machine or thinking about a machine operating. Let's just say we've got a machine operating on the side of a road. Okay. Think about the dynamic nature of traffic going by, power lines being above, and let's say this machine's an excavator. Sure. So if you need to set a wall on how far that excavator arm might swing out, you want to know that if I set that wall, I want to know that the agent that's responsible for setting that wall on how far my excavator arm can swing out actually sets it.

18:02Because the last thing we want to happen is an operator be in the machine and all of a sudden that arm swings out in oncoming traffic for an excavator. So that example, NOAA would be one where, you know, now with the power of Thor and the ability to run these AI agents locally, we can do all of that calculation on board the machine and invoke that machine command to be able to set those parameters on the machine to perform those functions.

18:31NVIDIA AI Podcast Host:Kind of a naive question on my part, but what's network connectivity generally like? And again, I know there's a huge diversity of scenarios that that equipment is working in, but is it generally pretty unstable? Is it like akin to using cell phones? Do modern construction sites usually have networks set up? What does that look like these days? Yeah, I would say it really depends on the area where our machines are operating. One of the decisions that Caterpillar made over a decade ago was that every machine that we produce will come with connectivity. So that is there. And that connectivity can be cellular connectivity.

19:10It can be satellite connectivity. We're also looking at the next generation of connectivity of high-speed satellite connectivity, how 5G is going to play into it. So that infrastructure is there, but then also, too, our machines end up operating in areas where the cell network just isn't great and there's not great coverage. And what we want to do ensures that when we bring these AI capabilities to the machine that when the customers need to count on them, they can count on them. And by now having the storage capacity, the compute, the memory bandwidth on machine, we're now able to take a lot of what we've had to do in the back office and now bring that local to the machine, which is super exciting.

19:47NVIDIA AI Podcast Host:Yeah. No, that's fantastic. Yeah. Well, how do you think about safety when you're building autonomous equipment, when you're building, you know, heavy construction machinery and mining machinery in the age of AI? Like, how do AI and safety go together? How do you guys think about it kind of, you know, starting at the beginning of the design process and working through? Yeah, and this is this corner work turning really quickly. If you think about autonomy systems of the past, they were very, very interesting in the way that they worked, but they were also much more deterministic than they were probabilistic.

20:20NVIDIA AI Podcast Host:Right. So, you know, what that means is that, you know, if you're going to put autonomous machines on a job site, you would almost pre-program routes into them. You would then have a complementary suite of sensors on them that were able to detect an object moving in front of them and then appropriately take action to avoid a safety incident. Now with AI, you're kind of going away from pre-programming things in a deterministic autonomy, and now you're going to giving a robot a task. Right. And it now is using various subsystems and agents and visual processing models to then determine the action that it needs to take.

21:02Safety for us, it's priority one. Before we put anything into market, the amount of rigor and the pace that we put it through to ensure safety is second to none. Doing this in a physical world is very time-consuming, as you can imagine.

21:20NVIDIA AI Podcast Host:Yeah. So for us, the pivot to AI-based autonomy in machines is predicated by a robust simulation environment. the ability to take real-world data, a digital twin of a job site, a digital twin of the machine, the models that we would have driving that AI subsystem or the machines making decisions, and ensure that we run that through millions of hours of simulation and test scenarios. And honestly, no worst-case scenarios in our simulation environment. And then obviously that's followed by a series of physical tests too. But now that simulation environment, now when you're getting into, you know, Omniverse and the combination of that, you know, Isaac and Cosmos and, you know, we now have the capability of being able to do that at scale.

22:12And I think that's what's going to make it feasible for companies like Caterpillar now to make this pivot pretty quickly from the way we've been doing autonomy for the last 20 years to what the future of it's going to look like. Right.

22:24NVIDIA AI Podcast Host:You must have an incredible amount of data on just the different types of job sites and the different, you know, natural materials and building materials and all of just all of the different physical world, you know, components that go into the work that these machines are used for. We do. We do. And that data, when you think about what it takes to make AI really work is the foundation models that are out there today are good foundation models, but those foundation models haven't been built with the specificity of the way a construction site works or the way a mining site works or the way query operations work.

23:05So we have the opportunity to take those foundation models now and do some level of reinforcement learning and post-training on those models with the specificity of the way our customers work and operate. We do think that's an area of differentiation for Caterpillar is that we want our customers to know that when they get an AI-assisted or an AI-autonomous machine from Caterpillar, it's backed by the quality of Caterpillar, but it's also backed by the 100 years of experience and the decades of data that we've got to be able to do this the right way.