Agentic AI for Drone & Robotic Swarming

0:03Welcome to the Practical AI Podcast, where we break down the real-world applications of artificial

0:23We'll be right back.

0:48Welcome to another fully connected episode of the Practical AI podcast. In these episodes where we don't have a guest, it's just Chris and I, we take some time to deep dive into some interesting AI topics or explore some things in the news related to AI and hopefully give you a little bit of learning resources or information to help you level up your machine learning or AI game. I'm Daniel Whitenack. I am CEO at Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris? Hey, doing great today, Daniel.

1:28How are you today? I'm doing well. Yeah, I just took a, tried to take a run before it gets really, really hot here. So hopefully, you know, that worked out well. It was still kind of hot. Hopefully I don't end up with a sunburn, but. Absolutely. Yeah, it's been hot here in the U.S., especially in the southern U.S. where we both are right now. And so, yeah, you definitely don't need you collapsing from heat exhaustion right before we get into a show here. Yeah, yeah, for sure. I will definitely say there's fewer people out. There was not in any way a swarm of people running anywhere that I saw.

2:11So yeah, I guess that leads a little bit into what you teed me up to talk about today, Chris, which I'm excited about, which is an area of your expertise and interest, which is swarming, which I guess could be related. you know, people might be thinking of animal swarming, robot swarming, other swarming, maybe just starting there. What do we mean by swarming? So before I get into that from my side, let me set that up a little bit because you actually mentioned several things that knowing me and also, you know, just the general topic as we've kind of prepped to do this a little bit, you know, you mentioned animal swarming.

2:56And so It's funny, in the AI world, we have been talking about robotics and autonomy and unmanned aerial and ground vehicles for various things. You hear about Amazon drones and Walmart drones are out there now and various other applications. Even though my background may be in military applications because I'm in the defense industry, we're really talking about it in a general say. And one thing I should say up front just to cover myself is I'm only representing my only only my own personal viewpoint and not that of my employer or any other organization. Point taken. Point taken, which is kind of funny because when you talk about swarming, if you put 10 people into a room and ask them each to define swarming, you will end up with about 17 different definitions of what swarming is.

3:48It's one of those types of terms. Yeah, you mentioned the delivery drones around where I'm at. I live right by Purdue University. And actually for quite a few years now, we've had these little box robots that will deliver you food. So they like drive along the sidewalks and, you know, go through intersections and deliver food to you through through an app and and all that stuff. So that one definitely hits a little bit close to home. I think for the most part, I have seen them, you know, successfully navigate the terrain and and intersections and all of that. Although I will say I did see one at one point.

4:39I didn't see the collision, but it was definitely run over on the road. It was in pieces at an intersection. Yeah, somebody didn't get their dinner that night. Yeah, exactly. Somebody called up hungry. Where's my food? You know, which, you know, and it was the delivery driver once upon a time. Hopefully that person would not have gotten hit. But yes, having the some sort of automated vehicle bringing the stuff. So I wanted to talk a little bit about it. And probably before we get into, you know, talking about what swarming is, kind of distinguishing it because it's one of those autonomy words.

5:15And, you know, autonomy is another one of those words. And we have things like drone and robot and UAV, UGV, UXV, X covering kind of all the things. And so I don't know what a lot of those things are. OK, so ground, ground, sky. You got a ground air. So like UXV would be unmanned or uncrewed is a little bit more modern term for it. The X signifies kind of what the domain it's operating in, air, ground, whatever, and then vehicle. And if you've seen UXS, that would be an unmanned or uncrewed whatever system. And so those are really common lingo things, and they're not specific to military. You see those in commercial and industrial applications and stuff.

6:02And for the most part, we'll probably, for simplicity's sake, mostly talk about drones. In this conversation, we tend to stay away from jargon. And maybe robots, you know, for ground stuff. And that way, people are not trying to get through acronyms while we're talking and stuff. Just to simplify things a bit. But we're having, we're definitely living in this age, as you just pointed out from personal experience, where we're starting to see these things, which are, you know, physical embodiments of some sort of AI or other algorithmic driven, you know, movement around the physical world for various activities and stuff.

6:40And it sounds like, and I don't know the specific technology for the one that's bringing you the food on how they're approaching that, because there's a bunch of different approaches, but that is either a semi-autonomous, presumably, or fully autonomous, uh you know ground vehicle that's bringing you the stuff uh whereas going back to our topic swarming implies numbers first of all and so but it doesn't just imply numbers it implies the way numbers are working together and collaborating and i think that's where a lot of people get in trouble with all sorts of different definitions and it's super popular to talk about swarming now So yeah, it's a crazy buzzword these days.

7:22Yeah, I almost wonder, like there could be some confusion. We hear a lot of talk these days about multi-agent systems. And of course, we've had episodes where we discuss agents specifically and what that term agents mean. But as soon as you as soon as you bring in that kind of multi-agent side, people might be confused. Is that what we're talking about? So I guess one question would be like swarming, does it imply like physical, I guess, physical AI or physical autonomy in terms of things that are operating in the physical world, not the digital world? We've talked a lot about drones and robots in the beginning of this.

8:08So is that is that part of that definition or not really? It is. OK. It is, you know, and we've had across a lot of our episodes recently, especially our fully connected episodes like this, where it's you and I discussing a topic, you know, we've talked about, we're kind of, you know, we've had over the history of the show, we've seen the evolution of AI, and there tend to be specific topics that get really hot. And right now agents are really hot. But there's also the notion of agents applying themselves in collaboration with various other AI technologies, whether they be LLMs or reinforcement learning or computer vision.

8:46And I think if you take a step farther into this world of autonomy, especially modern autonomy in 2025, agents are a big part of that. And having multiple agents collaborating and they're using these other models to get these tasks done. So you have agents operating with LLMs for different purposes. And, you know, so you're able to go and grab the right model for the right task. And you're putting a bunch of tasks together to go do something which which I might call a mission. But I don't necessarily mean that in a military-esque way. Could be. But it could be something that you're that is what the purpose of your company's autonomy is to do.

9:27Maybe like a goal or an objective or outcome. That's right. Could be synonyms. That's right. I'm not that great at the English language, but. Those are all loose synonyms that could apply. Yeah. And so, and we're seeing this, you know, we're seeing these individual drones and robots that are starting to do tasks in the commercial and industrial space. Certainly it's been that way in the industrial space and warehouses and robot. I guess the first robot I worked on was 2000 that personally I was doing were two of them that were autonomous in 2018. So this is not new in that capacity. How it's doing it has changed over time.

10:07Yeah, maybe it just taking a moment kind of looking back a little bit because I remember what year was it that the, I forget the year of the Beijing Olympics when they had the the, it was the first one I had seen where they had a kind of, I guess we could call it a swarm. I don't know. You can correct me if I'm wrong. Swarm of drones that were sort of in the sky in the, I guess, I think it was the opening ceremony. I'm trying to remember, but that was the first time I had seen that. Of course, I've seen it in other places, you know, after that. But yeah, what's kind of the... The definitions, right?

10:52Yeah. Well, how would you represent the history of kind of coming to the point where we're at now in these... So I'm getting from you, there's these autonomous... So multiple objects in the physical world that are autonomous or multiple vehicles or robots or whatever in the physical world that are autonomous trying to accomplish a goal or mission. Now, in the Beijing case, I don't know if those were autonomous or not. But yeah, help me kind of parse through maybe a little bit of that. So I should say ahead of time that I don't have any special knowledge of that particular configuration. So I'm making educated guesses on that.

11:35So I would suggest that's not what I, and I'll define a swarm in a moment as a follow-up, but I would not call that a swarm. I would call that a large number of individual aerial platforms that are operating in a predetermined, coordinated manner. Yeah, it's like synchronized. They're synchronized from the ground control system. So a notion in flying autonomous vehicles is that you have a ground control station, which might be like a laptop. It might be a bigger thing inside a truck or something, but it has, you know, comms and it has ways of communicating. And so for shows like that, you would use, you know, you would use things like GPS and they would have a three.

12:19Each drone would have a three dimensional path that it's following. It's a little bit different from all the others. But when you put them all up there at the same time, they look like a big coordinated show, but they're not actually coordinated. It's the human on the ground programming in their path. It is. They are not determining their coordination. There you go, which is really important. So, yeah, you went right to the heart of it. And so there is that which is really not intelligent at all. It's pre-programming these drones to do something. And the visual impact is that you have this thing happening among a bunch of things.

12:56But it's sort of an illusion. It's really a bunch of individual things on their path. And then there's a whole continuum of how those kind of vehicles can operate. one kind of popular approach today that I that that a lot of people would call swarming and which I don't and I'll define it right after I say this is the notion of like giving plays to a group of vehicles that go out and do a task sort of like some people the analogy will be a football play is go go to this location and do this and there might be some communication with the other platforms in your area that you're doing something with.

13:37But I also don't consider that a swarm because I think it comes back to the notion of what is swarming. It's swarming is not an autonomous thing. It's a type of behavior exhibited. And we see that in nature. And so I will I'll give the definition and then I'm actually going to go away from technology for one second and talk about what we see in Mother Nature that's kind of consistent with that. So my definition sounds a little military-like, but don't take it too much that way. It could easily be applied to commercial or industrial, but I think it gets the gist of how I see swarming. Mine is, swarming occurs when numerous independent, fully autonomous platforms exhibit highly coordinated locomotive behaviors in any domain, be it air, ground, sea, undersea, or space, functioning as a single, independent, logical, distributed, decentralized decisioning entity for purposes of command, control, and communications with human operators on the loop to implement actions that can achieve strategic, tactical, or operational effects in the furtherance of a mission.

14:52A little military sounding, but if you translate those words into commercial and industrial, they still apply. You might choose a synonym in some of those areas. But that is a pretty high bar right there. And as we go forward, we can talk a little bit about kind of what in that definition turns something from a collection of autonomous vehicles into a swarm of autonomous vehicles.

15:33Well, friends, if you want to build the future of multi-agent software, check out Agency. That's A-G-N-T-C-Y. It's an open source collective building the Internet of Agents. It is a global collaboration layer where AI agents can discover, connect, and work across frameworks. That means better tools for you, standardized agent discovery, seamless interagent communication, and modular components to scale your multi-agent workflows. And they're teaming up with Crew AI, Langchain, Cisco, and many more, dropping real code, specs, and services with no strings attached. Start building alongside engineers who care about high quality multi-agent software.

16:16Learn more at agency.org. Again, that's A-G-N-T-C-Y dot org. That's agency.org.

16:28Well, Chris, I really appreciate your definition. Maybe that's even something we can pull out and put into the show notes for people so that they can read that if they want to. I'm actually reminded as we're talking about this and as I'm hearing your definition of these things in the animal kingdom. Actually, I'm remembering a movie that I watched with my wife, The Murmuring. I think it was on Netflix and very interesting movie. but kind of at the core of the movie were these, I don't know if I'm using the right terminology, but murmurations of swarms of birds. Starlings, you're famous for it.

17:15Exactly. Yeah. So you can see in the sky, I'm sure many people have seen this, swarms of birds that almost form kind of patterns and synchronizations in the sky and move around together. Now, according to your definition, each of those birds, right? Now, I guess in this sense, you know, there's no connection to a, obviously a human command and control type of scenario, but certainly they are autonomous, but they're accomplishing something together, which is maybe a little bit mysterious in this case, although I'm sure it's a topic of study. But yeah, that's what came to my mind is that movie and that kind of phenomenon.

18:06Yeah. And that murmuration, if you will, is literally, it's called flocking in that case, which is obviously a bird term. But that is definitely consistent with what I would consider to be a swarm. And if you break that down, And there are other animals we can talk about, too, that have different types of swarming behavior, but that are consistent with that. If you talk about starlings in particular, you know, they're beautiful when you see those massive. And there could be thousands of starlings moving across and you see the waves within the system, you know, within the flock as they're doing that.

18:41And so you kind of go, you know, what's going on there? We have thousands of individual beings that are communicating and sensing each other. They have a unified purpose. They're all going someplace together. But there's no one master general or CEO starling that's going, you guys are going to go that way. Everyone do what I tell you to do. They are all subscribing to the mission, if you will. But each one has a position, if you will, or a place in that mission. where they are communicating and the way they do their sensing of each other and their communications helps them a not collide and, and, you know, fail by, by through collisions and such.

19:25It keeps them together, but they also have a way of, of agreeing, if you will, to go do something. They're going to, to move from one place on the earth to another place on the earth. and that kind of comes down to the crux of it is that swarm behavior is where you have that a bunch of fully autonomous beings in this case they're functioning in that murmuration as a single independent logical distributed decentralized decisioning entity it's a long mouthful that I use because all of those have a part in that describing it but they're all working together in that way Do you want to pick apart maybe each of those words?

20:09So just like why, why use those multiple words kind of distributed, decentralized, some people might kind of not know why each of those is, is important. Sure. So, so single implies that the joint actions together about how they're communicating and sensing give a rise to one generalized process. Kind of an emergent or combined process. That's a great way of putting it. And that's kind of that emergent, you know, we said mission or goal kind of thing is coming out of that. And that process is independent of any one bird. And it's independent of any controlling authority that's saying from afar, y 'all shall go do this thing, you know.

20:56And so you have that independence. You have that single emergent characteristic. logical meaning it's also kind of uh it's kind of think about it's arising a an emergent intelligent notion or a processing notion that's all happening by the unity it is distributed across the entire flock or murmuration it is decentralized there's no one special bird or group of special birds that are the birds that are in charge of everybody else and you guys have to do what we say And finally, it's a decisioning entity that by this emerging thing happening, choices are made collectively for the whole thing. And yet no one bird is saying everyone's going to go do this thing.

21:43So it's all these kind of characteristics about that emergent. I may add the word emergent into the definition that I have there because I think that's really a great way of describing that. But it's that unified collection to do this. Now, that's a far cry from what we're currently doing in the autonomy space at large. I'm not saying that doesn't exist in little pockets, academic pockets and other places, labs, whether they be government labs or university labs and stuff like that. But you don't see widespread deployment of drones or robots that are acting in this manner as we sit here recording today.

22:24Okay, that makes sense. Yeah. Yeah. Part of my what I'm thinking and my questioning in my mind, which maybe you're going to get to as you kind of get get through some of these things is obviously we have a kind of inspiration or a model there from the animal kingdom. in the case of, let's say, robots in a warehouse or a manufacturing facility or autonomous vehicles, we have in probably things that people are familiar with the idea of self-driving things or autonomy. So like the Waymo cars or that sort of thing. And there's certainly obviously AI at play there in terms of computer vision. and maybe reinforcement learning and decision making.

23:17So assuming that people might know at least or be exposed to that, in terms of the connection to AI specifically, what about this kind of swarming or robots or these entities working together to do a task? what is necessary for AI to do in that swarming scenario that's not necessary in just a single like self-driving car, like a Waymo car or something like that? That's a great question. There's a lot of nuance there. So the self-driving car is having to navigate an environment in which it's both sensing and to some degree communicating potentially, although that is evolving over time, with other actors in that environment around it.

24:10And there is emergent behavior on what to do. There's kind of the classic cases of kids playing on the sidewalk and the ball bounces into the street and reacting to various unexpected events like that. But at the end of the day, that vehicle is still, which may be fully autonomous, is still only having to decision on its own behalf. And so it has an onboard inference system, the onboard computers, the onboard models, or potentially that in combination with some remote that it's doing over some sort of connection, radio connection or cellular connection. But it's still decisioning for itself. The difference, according to my definition of a swarm, is that the emergent quality of the decisioning is arising from participation in many of the platforms that are participating in that unified mission or activity.

25:07But it has to decide for all of them, and it may have to decide on platform A. We're looking at something, we're evaluating it, we're sensing it, we're processing based on reinforcement learning, and our agents are then saying platform A, you go do this activity, platform B, you go do that one, platform C, you go do that one. And those are all slightly different activities as those platforms collaboratively work together based on the sensing and inferencing that's happened across the swarm to get an activity accomplished. Does that clarify a little bit? Yeah, that helps. And I'm trying, Maybe this is the decentralized part of what you're talking about, but part of maybe what is a challenge in my mind to think about is how such a thing would happen without a central kind of.

26:04So to say like, oh, let's just take a very simple, maybe it's simple. In my mind, it seems complex as well. But we have 20 robots in a manufacturing facility trying to do something. In one model, I could see where each of those robots detects certain things on a number of sensors, maybe cameras or temperature sensors or force sensors or something like that. All of that is communicated back to a central processing system. And it's really the single entity, the kind of master brain that makes the decision and communicates out a next step to all the robots. But in the decentralized way, you talked about how there's not this kind of central.

26:55There may be a central command where maybe there's oversight or something, but things somehow happen in a decentralized way. That's what I'm trying to connect in my in my brain, maybe. No, that's a great nuance that you're pointing out there. So in the warehouse, that's not really that's definitely a centralized environment. And you don't even have to have onboard compute for that because your robots can have a highly reliable system of communication over Wi-Fi, you know, with a master server that's there. And the distances involved are close enough to where you can have them interact with each other without having to do that.

27:34And according to the definition of the swarm that we're using here, that would not be a swarm, even though you may have many platforms working collaboratively. they're being controlled by a local centralized agent. And in swarming generally, according to this definition, you're really looking at an environment where that's not possible. You're looking at what we would traditionally call edge, or far edge is another term that people would use, basically where you can't count on either local or cloud environments to provide the compute. And so you have to have the compute on board, including all the activities that will arise from that compute.

28:17Yeah, I'm almost imagining moving still kind of in my mind living in the industrial world if I think more to like oil and gas and like certain of the environments that are part of part of working in that industry, whether it be deep under the ocean or in very disconnected remote places where you might not even want to want to or even could expose humans to those environments. That's right. But they're also very hard in terms of connectivity and that sort of thing. You might want to accomplish tasks in that sort of environment. Am I getting to the right kind of scenarios? Like one good one that is commonly used is the notion of disaster recovery, where you might go into an area that is geographic, you know, it may be geographically, you know, remote or may not be.