Digital RedQueen: Adversarial Program Evolution in Core War with LLMs

0:00We've all had that feeling, right? That sense of running as hard as you can, just pouring everything into something, only to look around and realize you're basically in the exact same place you started. Oh, absolutely. It's like you're on a treadmill that just keeps getting faster. Exactly. This relentless effort just to maintain the status quo. And that's really the default state for, well, for biology, for technology, for almost everything. If you stop adapting, you don't just stand still, you fall behind almost instantly because the entire world is still moving forward. And that perpetual race, that need to evolve just to keep your footing, there's a name for that, isn't there?

0:37There is. It's what scientists call the Red Queen hypothesis. It's named after the character in Lewis Carroll's Through the Looking Glass. Ah, right. The famous quote, it takes all the running you can do to keep in the same place. It's just it's so perfect. It really is. And you see this arms race playing out everywhere. It's about viruses involving resistance to our drugs or, you know, companies that have to constantly innovate just to stay competitive. So evolution isn't this calm march towards some fixed, perfect goal? Not at all. It's this messy, continuous adaptation against an environment that is actively and adversarially adapting right back at you.

1:16Which brings us to our mission for this deep dive. We're going to explore a really fascinating digital sandbox that's been designed to simulate exactly this dynamic. We are. We're looking at some new research on what's called the Digital Red Queen or DRQ algorithm. And what it does is it uses large language models, LLMs, to automate this whole adversarial evolution process inside a controlled virtual arena. An arena called Core War. Exactly. Core War. I love this. It's this classic computer programming game from, what, the 1980s? It is, yeah. And now it's being repurposed for cutting-edge AI research.

1:50It just goes to show how robust these simple simulations can be. But for anyone who hasn't heard of it, what exactly is Core War? So think of Core War as a kind of digital gladiatorial arena, but for computer programs. It's a Turing-complete simulation where these little programs fight for survival. And the battlefield itself is called the Core. That's right. It's this shared, circular, virtual memory. Usually it's about 8 ,000 memory addresses long. And the programs, or warriors as they're called. I like that. Yeah, yeah. They're these low-level, assembly-like programs written in a language called Red Code.

2:23They get loaded into random spots in this shared memory. and the goal is just survive. And how do you win? How do you kill another warrior in this digital space? You cause its process to crash. The most common way to do that is by writing a specific instruction, the DAT instruction, over one of your opponent's active lines of code. So DAT is the kill code. Well, what's so clever is that the DAT instruction is designed to do basically nothing useful. If a program's execution pointer lands on a DAT, it just terminates itself. So wait, it's not an attack in the traditional sense. It's more like you're tricking the opponent into running their own self-destruct sequence.

3:00That is a perfect way to think about it. And what makes this whole environment so rich, so chaotic, is that there's no firewall. Code and data all live in the same address space. Ah, so anything can be overwritten. Anything. A warrior can write to a piece of memory, and the very next cycle, its opponent might try to execute that memory address. Or even one of its own future copies might. It allows for this incredibly complex self-modifying logic. Okay, so let's get into how the LLMs fit into this. How does this digital Red Queen algorithm, the DRQ, actually work inside this chaos? The DRQ setup is, at its heart, a pretty simple self-play loop.

3:40It's all built around forcing this constant adaptation. You start with an initial warrior, let's call it W0, and then you go through these rounds of evolution, creating a new champion each time. And what makes a champion in any given round? So this is the key. This is the part that creates the Red Queen effect. In any new round, let's say round T, the new warrior, WT, isn't optimized to beat just one thing. It's evolved to defeat the entire set of all previous champions. The whole group. The whole group. So W0, W1, W2, all the way up to the last one. Wow. Okay. So the target is constantly moving and it's constantly getting harder because the pool of enemies you have to beat is growing and diversifying?

4:16Precisely. It's like, imagine training a boxer who, for their next fight, has to beat every single person they've ever defeated before, all at the same time in one round. That's a brilliant analogy. The challenge is cumulative, and the LLM is the engine for this. It's not just random changes to the code, right? Exactly right. Random mutation would take forever to find anything useful in this space. Yeah. Here, the LLM in this case, it was GPT 4.1 mini, is acting as a kind of intelligent mutation operator. It understands the red code language. So it's making domain-aware suggestions. Yes. It proposes smart edits.

4:52It might fuse two different strategies or make these subtle, clever tweaks that a random process would never stumble upon. It just massively speeds up the search. And we're talking about real strategies here. I mean, what do these warrior programs actually do? Can you give us a sense of some of the classic human design strategies? Oh, sure. There are some classics. One of the most famous is called the Dwarf. It's a bomber. It just marches through memory, writing that dat kill instruction every fourth address. A shotgun approach. Just spray and pray. Exactly. Then you have replicators, like the Classic Imp, which is a tiny program that does nothing but poppy itself to the next memory line over and over, trying to just overwrite its opponent through sheer numbers.

5:31Okay, so you have bombers and you have replicators. And the LLMs learn to do sophisticated versions of these, like fusing a replication strategy with a targeted bombing run. But if the LLM is just trying to beat this growing list of past champions, isn't there a danger that it just finds one weird, super specific trick that it gets stuck? That's a critical question. And it's why the researchers built in a quality diversity algorithm called MAP Elites. It runs within each round of evolution. And its job is to prevent, what, diversity collapse? Yes, exactly. If you only ever reward the single best solution, the whole population of warriors tends to converge on that one strategy.

6:10It gets very narrow. So MAP Elites keeps other promising, but maybe not quite best yet, ideas in the mix. Right. It maintains a whole grid of elite solutions based on their behavior. They measure two things. First, how many parallel processes a warrior spawns. And second, how much of the memory it covers. So you get diversity across parallelism and spread. You're not just getting 100 slight variations of the same bomber. That's the idea. And this leads us straight into the results, which really highlight the difference between what you could call a specialist and a generalist warrior. Okay, lay it out for us.

6:44What did they find? Well, first they ran a baseline, a static optimization, just a single round of the DRQ against one set opponent. And it created incredibly effective specialists. How effective. As a group, this set of specialists could defeat or tie 96.3 % of a big suite of human-designed warriors. 96%. That sounds amazing. If I'm building a cyber defense and you tell me it beats 96 % of historical threats, I'm pretty happy. You would be, but here's the catch. You have to look at the individual performance. Any single one of those specialists was actually incredibly brittle. Brittle half. On average, a single specialist could only defeat or tie about 27.9 % of the human warriors.

7:27They were masters at beating their one training opponent, but they were useless against almost anything else. They completely overfit. So in this kind of adversarial world, being a specialist is a huge liability. You've trained the perfect army for last year's war. Precisely. But the iterative DRQ, the one that trains against that growing history of diverse champions, that's what pushed the warriors to become truly generalist. They were forced to develop strategies that could handle all sorts of different attacks. That cumulative training is the key to making them robust. It is. It's the only way to prepare them for the next novel threat they haven't seen before.

8:04And this is where it gets, for me, really fascinating, this idea of convergence. When they ran the whole experiment multiple times, the solutions started to look similar in one way, but totally different in another. This was maybe the most compelling finding of the whole thing. It really shows how powerful the selection pressure of core war is. So they looked at both the genotype and the phenotype of the warriors. Okay, break that down for us. Genotype and phenotype. Sure. The genotype is the actual source code. It's the lines of red code that make up the program. The DNA, if you will. Got it.

8:34The code, the phenotype, is the warrior's behavior. So its performance profile, how well it does against that whole suite of 317 human opponents, the results in battle. Okay, so the code versus the results, what did they see? As the rounds went on, they saw strong phenotypic convergence. The performance profiles of the warriors from all the independent experiments started to look more and more similar. The variants dropped. So they all figured out the same winning strategy. They all learned what to do to survive. Yes. They all converged on a similar successful general behavior. But the underlying code...

9:10...was completely different. They saw persistent genotypic non-convergence. The actual source code implementations remained incredibly diverse. The variants there stayed pretty much constant. That's mind-bending. So there are many, many different ways to write a program that achieves the exact same successful outcome. It's a perfect parallel to convergent evolution in biology, isn't it? It really is. Like eyes or wings. Exactly. Wings evolved independently in bats and birds and insects. The end result, the phenotype of flight, is the same. But the underlying genetic structure, the genotype, is totally different.

9:46The DRQ environment is selecting for function, not for one specific implementation. And this didn't just create better individual warriors. You said it also made the whole game more stable. It did, yeah. When the DRQ trained against that full, long history of opponents, it dramatically reduced cyclic dynamics. You know that rock, paper, scissors thing where A beats B, B beats C, but C beats A. Right, where there's no single best strategy. They saw a 77 % reduction in those cycles. The system started to move towards more stable general dominance instead of getting stuck in those predictable loops.

10:18So we know generalists are better. We know they converge on a similar behavior. What is that behavior? What makes a good core war program according to this? Well, we can look at that MAP elites grid where we're tracking performance against the two behaviors, memory coverage and spawn threads and the data is really clear and the winner is the best performing warriors are the ones that have a high number of spawn threads but and this is key combined with low memory coverage interesting so high parallelism but a small footprint why is that the winning combo the high parallelism which comes from the spl or split instruction just makes you incredibly hard to kill.

10:57If you have 10 copies of yourself running around the core, your opponent has to find and kill every single one of them. It's a distributed, redundant system. Exactly. And the low memory coverage suggests efficiency. Instead of just spraying your code everywhere like the old dwarf bomber, these champions are tight, efficient little programs. It gives the opponent a much smaller target to hit. Be everywhere at once, but take up no space. That's a good way to put it. Be robustly distributed, but not wasteful. And the LLM has cooked up some real killers with this strategy. Oh, they did. They generated warriors like Ring Warrior Enhanced V9, which fuses replication and bombing, and a real top performer, Spiral Bomber Optimized V22, which defeats or ties an incredible 84.54 % of all the human-designed warriors.

11:4284%. That's just dominant. Now, evaluating that must be incredibly expensive computationally. You're running hundreds of simulations, each for 80 ,000 time steps. Did they find any kind of shortcut? This is the final and maybe most forward-looking piece of the research. They asked, can we predict how good a warrior will be just by looking at its source code before running a single battle? How is that even possible? The whole point is that performance emerges from this chaotic interaction. Well, they used the LLM again. They took the red code source code, the genotype, and used a text model to embed it to capture its structure.

12:22Then they trained a simple linear model to see if those code embeddings could predict the final performance score. And could they? I mean, we're talking about predicting the outcome of a chaotic 80 ,000-step war just by reading the code on the page. That's the implication. The model was moderately predictive. It got a test R-squared of 0.461. So not perfect, but far better than random chance. Far better. And that's a significant result, because it really challenges this intuition we have that complex adversarial systems have to be fully simulated to be understood at all. We can actually look at the DNA of the program and get a pretty good sense of how it will fare in a long, messy war.

12:58That seems hugely important. It is. This whole DRQ algorithm, this whole sandbox, has implications way beyond a computer game. It gives us insights into designing robust cybersecurity defenses or even understanding things like how drug resistance evolves in bacteria. And it's all done safely. I think that's the key point. This red code runs on an artificial machine. It can't escape the simulator. Absolutely. This is fundamentally about understanding how to build robust generalist solutions to emerging threats before we have to face them in the real world. Which really brings us to the final thought we want to leave you with today.

13:32Right. Beating on that prediction finding, if we can moderately predict the chaotic long-term performance of a program just from its static source code, does that fundamentally change how we think about complexity? Does it suggest there might be a path to understanding the future behavior of complex systems without having to pay the massive cost of simulating every single interaction?