#234 Tyler Xuan Saltsman: How AI is Shaping the Future of Combat & Warfare

0:00Our logistics agent is used with a meta model fine-tuned on a bunch of military logistics doctrine And then now that model now thinks like a military logistician. And what we do is we build these adapters. They're called LORAS, which stands for a low rank adaptation of a large language model. And then now what you do is that adapter freezes 99 % of the parameters of the model. And then now the model only speaks to the information pertinent in that adapter. So rather than a generalized chat model, which is what we see today that are pretty much unusable, it's now specific to a logistician. Tyler Saltzman here.

0:33I'm founder and CEO of Edgerunner. We're building generative AI for the warfighter. And what that means is we're building domain-specific intelligence that's occupation-specific for each warfighter's role, whether it's logistics, condition-based maintenance, whether you're a fighter pilot, you operate a battleship or a submarine. We're building AI that'll augment all of our warfighters to ensure national security. and what also makes us different is our ai is open and auditable meaning you can understand the training data and the biases that went into the ai unlike our peers today that's interesting and and what are the primary models uh and use cases uh i mean when you say logistics i mean specifically or someone managing a weapons platform?

1:34Yeah, so right now we're in a crawl, walk, run. The crawl phase, we're actually excited to partner up with Meta. So we have clearance to use Meta's Lama models for DoD applications. So our logistics agent is used with a Meta model fine-tuned on a bunch of military logistics doctrine. And then now that model now thinks like a military logistician. And what we do is we build these adapters. They're called LORAS, which stands for a low-rank adaptation of a large language model. And then now what you do is that adapter freezes 99 % of the parameters of the model. And then now the model only speaks to the information pertinent in that adapter.

2:16So rather than a generalized chat model, which is what we see today that are pretty much unusable, it's now specific to a logistician. And this is a fine-tuned model. You're not using a RAG system. We also use RAG. So I like to think of what we're doing is we're building a combination of a RAG pipeline, of LoRa adapters, and with small language models that all work together synergistically. So for example, you would use RAG on things that are static and that don't change. And then you would use a small language model with Allura on something that's dynamic and that you need to be creative. So here's a good example.

3:00Let's say you're a fighter pilot. And let's say I need to know everything about my fighter jet. How do I maintain it? How does it perform under certain circumstances? An interactive manual on steroids, if you will, that I can talk to. That would be a RAG pipeline because RAG, retrieval augmented generation, is just retrieving all the relevant information of that fighter jet. But now let's say I have a bespoke mission where I'm doing something that's not common. Let's say I have to do an air to ground mission and I'm flying low to the ground with an F-22 or an F-16. That would then be a small language model with a LoRa adapter attached to it.

3:39So now I'm using a combination of RAG when I need to interact with my fighter jet. And then I have my small language model with my LoRa for the bespoke mission that's specialized. Yeah. Talk about how that small language model with LoRa would work. And is that on the edge? I mean, is that in the aircraft? That's right. So for example, what's great about LoRa is it's access fine tuning on the edge because we're freezing basically 99 % of the entire model and we're only speaking to what's in that adapter. So that pilot would then have an adapter that's specific to the terrain, the demographics, the mission here she is executing, the target that they might need to go take down, or maybe it's a reconnaissance mission.

4:25Basically, all the situational awareness you'll need will be in that LoRa. And then the small language model from Meta, for example, would then be augmented by it. So we would build that LoRa, we would run that meta model, we'd also provide the Rack pipeline, and that would all work right on a device on the edge that doesn't need the internet. And how does the pilot interact with the model then? Right now, the pilot, they would type in the device, but we're in the process of making this voice-enabled, kind of like Jarvis to Iron Man. Mm-hmm. And what's an example of what a pilot would be asking the model for?

5:09Yeah, let's say the pilot's noticing something, an unusual sound within the aircraft. Hey, what is this? Is this a problem? Do I need to divert and ground the aircraft? That could be something that comes up or let's say there's an enemy on the horizon. Based on my payload, what I have, do I engage the enemy? What do I do? What kind of evasive maneuvers can I make safely based on how that fighter jets configured? Something like that. And again, I don't know much about flying, but I could get into more logistics, which is what I know. I used to be a logistician for the Army. And we also prepared for you a logistician demo of our technology.

5:57Sure. Yeah. Well, let's talk about that because logistics is a big piece of the military. That's right. We see in Russian-Ukraine conflict right now, Ukraine sometimes gets the upper hand because of better logistics. And so, of course, if you run out of fuel or life support or ammo, you name it, your fighting force is down. And so we have to make sure that we're optimizing for that. So for example, with logistics, it can help me like which ammo goes with which ammo on which truck? What happens if this truck goes down? What alternate routes do I have? If I get ambushed, what are my friendly units around the area?

6:37um you know if i'm hearing a weird noise in this truck what how do i troubleshoot it what's wrong with it so ai can detect all of this and help me on the fly right away um but i'll pause there colton can bring up the demo let's let's walk through that and show you sure the actual working prototype and again this works locally on device without ever needing the internet so we're going to show that we're going to turn on or turn off wi-fi of course, this is just a demo, but here you can see a very comprehensive prompt. For example, if you asked chat GBT this or any kind of model, it wouldn't know this because, again, these models are too generalized.

7:18Now, what we're able to do is drill into this exact mission ask or the prompt on how do we properly plan a logistics mission of this magnitude and how do we mitigate risk? What's the equipment we're going to need? You know, what does the planning phase look like versus the execution phase, you name it. So we can get very granular. And this helps a logistician officer like me better plan and, you know, and do my job. And what are the PDFs listed below? So what we do is we actually cite all the sources that we trained on. So now if I say, okay, this is an interesting line item and the recommendation the AI gave me, where did that come from?

8:03Now you can drill into that, the doctrine of where it came from. Yeah. Yeah, this is remarkable. And for audio listeners, we're looking at Edge Runner, the logistics agent built on, how do you say that? It's an instance of Edge Runner or a use case of edge runner. That's right. And what this is, so we're actually showing you all the different sources we train on. And we train this on thousands of different PDFs related to the Army and logistics. So it's very specific. uh and it's giving a breakdown of um day by day you know morning uh midday late afternoon exactly what has to happen exactly like this this what it just generated for me in literally 30 seconds can take could take me hours or even or even a couple days um and so now this can augment and this is based on like the army doctrine way.

9:17And then of course, you know, things in the battlefield aren't always going to be by the book or by doctrine, but at least now it's giving me a very good baseline of what the mission should look like. And before something like this, or currently, it's all done by hand or are there systems that are doing this without generative AI? So this is completely not done by hand at all. But what you can do is now that we have like this baseline, we can then go whiteboard and have an art of the possible. And maybe we want to take out day two so then we can just quickly modify it via hand jamming. Right. And what's the uptake at this point?

10:07I mean, where are you guys in your – give me a little background on the company when you formed and where you are. Yeah, so the genesis of us – so we're all pretty much X-stability AI. And stability was known for stable diffusion, which is one of the most popular media models in the world for text-to-image modality. and from you know in in working locks up with all these scientists you know we started because I was I was head of supercomputer when I was at stability and working with these scientists we realized we had a core competency of being able to train and inference models across any different types of chip architecture and hardware but also to make these models much smaller and then they and when they're smaller they can live right on the device and not need the internet I realized we had this core competency, you know, it's like, let's go build this for the military, you know, and given my background as an ex-army officer and also meeting these scientists, it just made a lot of sense for us to work together.

11:13And so it was challenging to add stability because we weren't really creating products that added value to customers. It was more just a cool thing, but there was no real value creation. I think that's the problem with AI today. It's not actually solving anyone's problem and it's not really doing anything useful other than making cool things. So now we're solving for that by actually building products that our customers, the DOD, that they could use. So we're only about eight months old. We raised five and a half million. Madrona is the biggest VC on our cap table. They're an amazing partner. Matt McElwain has been an awesome supporter of us as well as John Turow.

11:54And then of course we have some angels So, yeah, we have a full team of 18, and we're starting to ramp up to scale early next year. You know, I worked on the National Security Commission on AI for a couple of years, the two years that it ran. and there was a lot of talk about integrating AI into, I mean, that was the focus of the commission, really, how to accelerate that integration. Is there anything, I mean, how does this work? So you guys, you know, you have a company and a product and relationships with the DoD. It's so complex, the acquisition process. How do you manage that? How do you navigate that?

12:59You know, we're going to be going strictly to the channel. Right now, we're on Kerasoft. They're one of the biggest channel distribution partners of the DoD. So we're a prude vendor on Kerasoft. we're on nasa soup as well as uh ites sw2 contract vehicles that the army uses but the dod uses nasa soup so we're we're good to go we're on that vehicle so when when you need us you can procure us directly through them and then of course this will expand go ahead and and then it's up to the individual unit commander or who makes that decision about whether to uh to onboard edge runner Yeah, so that'll come way up at the J-level, even at the Pentagon.

13:45Typically, it'll come down from the top if there's some sort of AI mandate or policy. Then there'll be what we're going after is called broad area announcements. There'll be broad area announcements of AI to do X, Y, Z. And then we'll then bid on it. But what helps us is we qualify as a small business and we're veteran owned and I have a PTSD disability rating. So we're a strategic partner for these bigger partners that are required to work with smaller guys like us. And of course, we have core competencies of building AI that's personalized and completely air gapped, which actually segues into our computer vision technology.

14:23So not only are we building agents and assistants on device, but let's show you our computer vision technology. Sure. And what we're solving for is right now in Ukraine, the life expectancy when you're spotted by a Russian drone is only seven minutes. Wow. Until you find cover and concealment. And so how do we neutralize these Russian drones? Well, we're building drones that can see now rather than requiring a pilot. Because when you require a pilot, it's easy to jam these drones and disconnect the pilot from the drone, rendering the drone useless. But now we can put vision on swarms of consumer grade drones, which are very cheap, that then carry a payload via a kamikaze suicide drone style.

15:10So now we'll show you what that computer vision technology looks like. And it works today.

15:22And that's the problem that we're solving for is it's easy to jam these rounds, but now when they can see like what you're seeing now, the drones are autonomous and they can fly right into that tank and with a payload and neutralize it. And how do you ensure that it's not going to fly into a piece of Ukrainian armor? No, that's a great question. And what we're doing is we're building the neural network. Just like a human, if a human can discern the difference between a Ukrainian tank and a Russian tank, AI can as well. You do that by training the model on hundreds of thousands of images. Now, to your point, if they look identical, then you're right.

16:09We can't. So if Russia now rolled out tanks that are identical to Ukrainian tanks, that would be problematic. We'd have to find another solution. But right now, it's very easy to discern what a Russian tank looks like versus an allied tank. Yeah. Okay. And so again, for audio listeners that don't want to go over to youtube to see the video we're looking at footage from a drone or from a series of drones hitting russian armor is this training data or is is you don't have these drones fielded these yet do you correct so the drones aren't deployed in real life combat zones yet Right now we're testing.

16:54And this is an overlay of our technology from drones that can't see. Now we're showcasing that they could see. So this is more of the art of the possible. But the technology does work and we're in the process of bringing this to life with our partners. So we're working with service capital as well as the drone company is called Rapid Flight. And their core competency is they build 3D. It's easy to print drones via a 3D printer. And then these drones are one-way drones that are cheap to make. They can make them quickly. And then, of course, they can carry a payload. And where are those drones made?

17:33They're made here with our partner called Rapid Flight. I see. Are you familiar with what Eric Schmidt's doing with White Stork? I think it's called White Stork. I know Eric Schmidt's got some really cool projects going on that I'd love to partner with him on. um so i'm not familiar with that exact project but i know that eric has a keen interest in in what we're working on as well so if you happen to know him please give him a shout yeah well i'm hoping to have him on the podcast soon so i'll mention it to him but he's very interested in in drone technology just uh and how it's uh changing uh the nature of warfare i mean it really has, I mean, what do you think?

18:19All the analysis I've read is that, you know, armor is pretty much obsolete at this point. That's right. I mean, as drones become ubiquitous, it's definitely going to change how we fight wars. Now we're going to have drone-on-drone combat. Also, what will be interesting too is AI, large language models still aren't intelligent yet. And I think Largely, it's because they don't understand the physical world. Well, how do we understand the physical world? We've put BLMs on drones, and drones now scan the battlefield and are describing everything that they're seeing. Now we can take that new synthetic data and reinforce the neural network of the large language model, which then makes the on-device agents better.

19:02And now we get another step, another great leap forward of AI. Now, I don't think this will create AGI, but I think it'll make generalized AI much better. yeah the um and and you'll forgive me tyler i i don't remember whether we spoke about this uh were we talking about uh coordinating uh swarms last time we spoke a little bit um and and i think the the the cornerstone the conversation was around that humans will always be in the loop Right. We'll never want AI to be the shock collar, if you will. I think AI should be great at the expert assistant that's advising you objectively and fairly, right?

19:50But never actually making the decision. Kind of like, again, Jarvis in Iron Man. If you watch the Marvel movies, Tony Stark's always ignoring Jarvis, even though Jarvis is probably right. But he still has the ability to ignore Jarvis because I think there's still the human gut element that we need to trust. But of course, AI is going to make us better at what we do. Yeah. Although human in the loop, how do you explain human in the loop with a drone that is vision enabled and a kamikaze drone that's navigating its way on its own? I think what we do is so we can train drones to see hand and arm signals.

20:36So I can wave to my drone to come here or stop or go things like that. So now we call this being context aware. So normally a drone can just see a human and it's not recognizing the symbols I'm making with my hands. But now that the drone's context aware, I can I can effectively control it. You know, kind of like soldiers. I can't talk to my soldier, but we need to go attack. I can do certain hand and arm signals and we can go engage without talking. But yeah, so the video that we just saw, a drone that identifies a target and flies into it.

21:18How is the human watching what the drone is seeing and can abort the attack if it feels like it's making a mistake? Jake? Of course, a human could. Also, we can take this a step further. Let's say rather than trying to blow up the tank entirely and killing the tank operator, we just want to hit the track of the tank and just disable it. We can do things like that as well. Just like with moving trucks on convoys, why don't we just disable the truck? Now the convoys is hitting duck. We don't have to kill the enemy. We can actually just neutralize their supply chain and we can get smarter. and control the collateral damage and the chaos.

22:01Oh, that's interesting. So that's part of what you're doing with the vision system. That's right. Being able to identify components or areas on a vehicle. That's right. And I think a lot of this, too, is the economics of warfare. If you look at the invention of the Barrett.50 cal, it's a single man operated.50 caliber rifle, but it was really intended as an anti-material rifle to take down radars and grounded aircrafts. And it's a$10 bullet and you can disable a multimillion dollar aircraft. So a lot of this of Warfare 2 is going to be economics based. These consumer grade drones are very cheap, but they can also take out very expensive assets.

22:50And then, of course, that'll help you win the fight as you bankrupt Russia. Yeah. How is this going to affect air warfare or surface warfare on the ocean? You know, I think eventually human pilots will be a thing of the past. I think AI will become much better at flying and much better at dogfighting capabilities. And I think humans, again, will be in the loop of sort of, again, orchestrating and quarterbacking. But the days of, you know, the Top Gun style dogfights, I think those will come to an end. yeah uh and uh how how quickly do you see this moving because we certainly saw i mean i've said this on probably to you and and uh to others on the podcast that at the beginning of the russian invasion there was that 40 mile line of armor headed toward kiev yeah and had there been enough drones that they would have taken out that whole line.

24:09And that was just what, two years ago? I mean, it's now drones are ubiquitous on the battlefield. So it's moving very quickly. How quickly do you think... Another thing is Mark Milley, I heard him at a conference and he said, you know, the aircraft carrier that they're working or building today is going to be obsolete by the time it's in the water because you're not going to use aircraft carriers. Everything's going to be unmanned vehicles and that sort of thing. So yeah, how quickly do you see this moving into actual battlefield situations? I think the entire transformation process, I think, will be less than a decade.

25:03I think in 10 years, we'll have drones. I mean, we already have drones today that can approach supersonic speeds. But, I mean, to your point about completely unmanned, doing everything, yeah, I'd say we're less than a decade out. Just like the Jarvis experience here, Iron Man, that technology is nearly here, and we're building that. That's also less than 10 years out. Now, when people are saying AGI will take over the world in 2028, I think they're insane. And some people we know have said that. I think AGI is at least 20, if not 30 years away. So it's a long ways away. But I think AI will become very sophisticated where it'll resemble reasoning in AGI, but it's not quite there because, again, humans will always be in the loop.

25:51And so, again, that's our philosophy, is rather than one big mega model controlling everything, it's going to be swarms of agents. And these different agents have different core competencies, just like humans do. But again, the agents will always work for us. And that's the point of it. Just like Jarvis always works for Tony Stark, he never goes rogue. He sticks with Tony. And that's sort of how we envision AI. It sticks with you. Yeah.

26:20Hang on a second. There was a question I was going to ask. Yeah. Do you follow what China's doing at all? Because certainly they've been very active, particularly in the drone space. Yeah. I think what's sort of a sobering moment is us losing the DJI protocol of manufacturing to China. And so we lost that and we can't lose the AI race to China. And China is very formidable. Their Alibaba models out of China, the coin series are phenomenal. And so are they ahead of us? You could argue they are, but just the fact that it's even an argument is concerning. Like we should be way ahead of China and we're not.

27:06So, yeah, I think we need to put our foot on the gas. We need to take this seriously. And we need to really ensure that folks that are building AI uphold national security and that protect Western principles and values. Because, again, how do we make AI think like an American? How do we make AI culturally aware? If you take a model and then translate it to Hebrew, that doesn't mean it's thinking like an Israeli. It's just a translation. So like as AI gets more advanced, it'll learn your, it'll learn our culture. And it's important that we, that we build AI that magnifies our culture and that protects it.

27:42Yeah. I mean, I have to ask on this computer vision enabled drone, could it be deployed with facial recognition technology? It could be, but then there's the argument of it's unethical to capture people's faces and store it. But yes, the technology is there and you could do it. Absolutely. I know China is doing that. I know with GDPR, they're trying not to do that. So if you want to track a human, you wouldn't even need to track their face. You can actually analyze their gait, the way they walk. I walk differently than you do. So my gate can be a unique identifier, even though you're not looking at my face.

28:33So things like that is how you get creative and have a workaround. Yeah. I just remember the, you know, the hunt for Osama bin Laden. If there had been face recognition enabled drones, that whole thing might have ended much earlier. You mentioned at the very beginning the seven seconds that a warfighter has from the time that it's identified by a drone or when a drone locks onto the human. Yeah, seven minutes. Seven minutes, yeah. sorry when it sees you to when you need to get cover and concealment and it used to be much longer i mean maybe it's seven eight i don't know ten minutes max it's very but it's very the window's very short but the point is it used to be hours and now that you know drones are ubiquitous now and getting more dangerous it's now shrunken down to seven minutes around yeah is there anything on the defense side that you guys have looked at that soldiers on the ground can use when they're identified by a drone?

29:59You know, I can't speak to that, but what we're building, if let's say you are in the field and you have a drone, call it a FOB that has 3D printing, we can quickly assemble the drones and zip them out there. But I would imagine fleets and swarms of these cheaper drones that if I am seen, we can just deploy them right away in my AO. And then if a drone's coming at me, these suicide drones will then fly into it. And so that really, to me, is the solution. I know that there are other companies that are using AI to actually operate a machine gun. And a machine gun will then shoot the drones down.

30:40But the problem with that is with ballistics, your MOA, your minute of angle is off. So if I'm shooting something 1 ,000 meters out and it's a 5.56 round, there's a variance of where that bullet will actually be, even though I'm right on target, especially with windage and the air density, you name it. So it's still not very effective because you would want a shotgun to shoot a drone, not a machine gun. or I think the best solution will be cheaper suicide drones that'll lock on and get right to it. And again, and that'll save money and it won't give away my position. There's lots of advantages to doing it this way.