#323 David Ha: Why Model Merging Could Be the Next AI Breakthrough

0:00Let's say you have 10 agents and you give them the same problem and then you come up with solutions. What are you combining at that point? And is it being done in pairs? I mean, does each offspring have two parents or can you combine more than one?

0:16David Ha:Our first paper on model merging does require access to the weights of the model. It's an impractical assumption because in reality, most of the frontier models are closed. And even if we did have access to the waste, they probably do not share architectures. This episode is brought to you by Tasty Trade. On Eye on AI, we talk a lot about how artificial intelligence is changing how people analyze information, spot patterns, and make more informed decisions. Markets are no different. The edge increasingly comes from having the right tools, the right data, and the ability to understand risk clearly.

0:57That's one of the reasons I like what Tasty Trade is building. With Tasty Trade, you can trade stocks, options, futures, and crypto all in one platform with low commissions, including zero commissions on stocks and crypto, so you keep more of what you earn. The platform is packed with advanced charting tools, backtesting, strategy selection, and risk analysis tools that help you think in probabilities rather than guesses. They've also introduced an AI-powered search feature that can help you discover symbols aligned with your interests, which is a smart way to explore markets more intentionally.

1:43For active traders, there are tools like Active Trader Mode, One-Click Trading, and Smart Order Tracking. And if you're still learning, Tasty Trade offers dozens of free educational courses, plus live support from their trade desk reps during trading hours. If you're serious about trading in a world increasingly shaped by technology, check out Tasty Trade. Visit tastytrade.com to start your trading journey today. I'm going to myself. Tasty Trade Inc. is a registered broker-dealer and member of FINRA, NFA, and SIPC. I'm interested in evolution, neuroevolution, which in your co-author on a book that just came out, I've talked to Risto and Sebastian, but I'm interested.

2:46I don't want to talk about the book so much. I want to talk about neuroevolution or evolutionary strategies and what you're doing at Sakana or Sakana in those realms. And it seems like you've got a few layers. I mean, you had the model merging, which we can talk about, and then evolutionary AI generally. But then AI, a scientist, takes those principles and applies it to research. So can we start by talking about evolutionary AI or evolutionary strategies and your work in artificial life, which is part of all of that, and how you came to that area, particularly since you came from finance, which doesn't sound related.

3:53Yeah.

3:55David Ha:Oh, it is quite related, I think. I feel like having lived, went through the financial crisis in 2008, it really made me learn about extreme events in the world. I think anything can happen. I think people are usually underestimating these sort of a tail effects. And I think my experience at that time kind of shaped my views about how our life, how our civilization has came to be, how our intelligence has came to be. And that in turn has shaped my views on how artificial intelligence can be developed. So if we take a step back, when I first, I studied neural networks when I was an undergraduate in the University of Toronto.

4:51David Ha:Some of the earliest, I did my thesis back then on using neural nets to do computer vision, analyzing whether there's these steganography marks and images back then. But after entering the finance world, I did come back to the AI world when I first stumbled onto a few papers and articles about neuroevolution. This was in particular the works and the presentations by Kenneth Stanley and Risto, who is my co-author of the book. So back then, around more than 10 years ago, people were quite excited about image recognition. You have these GPUs, now they can recognize images and winning Alex nets and so on.

5:44David Ha:So that was fine and dandy. People can recognize cat from a dog. But for me, I was more interested in more general AI. And then I was browsing and learning more about what's going on in AI because it seems to be coming back. And I encountered a slide deck actually published by Risto. It was for a workshop, a tutorial. I think it was one of the earlier evolution conferences, like a gecko conference, where they were using neural networks for playing games, having characters move around, evolving the brains of these artificial creatures that can work together. So on one hand, you have a bunch of neural nets that can recognize computer digits or images.

6:35David Ha:And on the other hand, you have this area where you're evolving the brains of these artificial creatures that can play in games. And I thought that was really cool. So it's like, how come the rest of the world are focused on training these neural network models for kind of these narrow classification tasks? And you have this other field that no one talks about that are using, that are evolving these neural networks to create art, to actually have multiplayer games, and to kind of evolve the morphology of neural networks as well. So that really got me in and the more i studied about neural evolution the more i understand that's you know like the it's a broader view of how uh of of ai you know like we had machine learning back then where we're essentially training predefined neural networks to do a particular task uh and i think uh in the field of neural evolution or evolutionary computation there are many ideas that are broader which appeal to me uh there's the idea of you know not not just sticking to systems where you can calculate the gradients uh to do something because i think in in our world like many things uh many problems are such that there's actually no particular gradient signal and the idea that if you just chase the gradient sometimes you can get stuck sometimes you kind of have to wander around the solutions to find an even better solution i like the idea in evolution of this open-ended discovery or open-ended search.

8:14David Ha:So rather than having one particular objective in mind, your objective is to find new objectives, find new novelty. So I think this idea is not just, it came out of the neuroevolution community, but it's not like implicitly an evolutionary thing. It's basically defining goals that haven't been done before. And this idea has actually moved beyond the evolution community into broader AI. And I also like the idea of using evolutionary computation to evolve new neural network architectures and morphologies. So this was actually an idea that was formed in the evolutionary area. And I think when I joined Google in 2016, some of these ideas has been incorporated into deep learning.

9:04David Ha:So now, when evolution, actually the core concepts has been embraced within like Google DeepMind and Google Brain when I joined in 2016. So people then took these ideas to develop them further. Now we ended up into things like neural architecture search, using evolution to optimize how these connections of the transformers were connected. So I think right now, looking back, many of the ideas in evolutionary computation in the last decade has been embraced by the broader AI community. And we're seeing some good progress because of this as well. Now, there's many groups working on open-ended discovery rather than just a fixed optimization.

9:56David Ha:many groups are using evolution to evolve architectures of neural networks, evolving in the LLM world, evolving the prompts or the relationship between agents. And, you know, getting back to your earlier question, I became interested in AI because of evolution, but I'm not stuck just in evolution. I joined Google, you know, in their Google brain research team and i was also really excited about the deep learning revolution as well so all of these fantastic progress about large neural networks that can generate images generate videos and so on so when i joined google in 2016 i wasn't just working on evolution i worked on some of the earlier works in generative ai uh i with my collaborator juergen schmidt huber uh We actually extended some of his ideas from the early 90s.

10:55David Ha:And we brought the world models concept back to the modern era. So I conceived of some of the earlier video generation models. So at the time, the goal was not just for creating video. That's more like a specific application. but is to build world models for general agents that can learn inside of their imagination so that they can be more sample efficient. They can kind of like have a model of how to plan actions without actually trying them in the actual world and to actually have a framework for building knowledge. So I was actually quite excited about deep learning in general, in particular world models, all this generative AI.

11:46David Ha:Now we call them generative AI, but before it was more like generating the data distribution of what you learn and how to combine generative AI or deep learning with concepts in artificial life and evolution. So I think as I became more of a professional researcher, even though I started getting interested in artificial life and evolution, I worked on the core deep learning for several years. And many of my works, I would say it's a bridge between neuroevolution and deep learning. For example, when I was working on world models, I would use evolution components to help find adversarial weaknesses of the world models.

12:33David Ha:How to actually evolve different strategies to trick the world models. I was using evolution to evolve the morphologies of these virtual robots, which would then use reinforcement learning to train the robots. So kind of I view evolution as an outer loop of how things progress and deep learning as the inner loop. And now, fast forward many years when I started the Sakana AI company, we continue to combine the frontiers of generative AI and deep learning with evolution. One of the things that we worked on, as you mentioned earlier, is developing the technology to combine frontier models, whether they're open source or closed source.

13:25David Ha:because I think more fundamentally or philosophically, our intelligence as a species is not like a giant big brain. We are a collective intelligence of like 8 billion people walking around with 30 watts in this thing here. And we talk to each other and exchange ideas. So this is how our human artificial or whether it's an artificial or biological intelligence is formed. And I think this is also a good model of how artificial intelligence could be developed. So in the AI space, we have many, you know, our observation was we have like a large ecosystem of open source models developed enthusiastically by many people working on AI models.

14:15David Ha:and we have closed models developed by all these great companies like Google, OpenAI, Cohere, Anthropic, and in China, DeepSeek, and Alibaba, and so on. So each of these models have their own unique strengths and weaknesses. And it kind of makes sense for me if we wanted to build a model with the very, very best performance on a particular thing is to conceive of a scheme to combine different models and the different strengths of all these models. So one of the first experiments we did is we use an evolutionary algorithm and we use evolution strategies to merge different open models so that we can create custom capabilities.

14:59David Ha:If we wanted to have an open Japanese language model that can do math a bit better or coding a bit better, we couldn't do that. And recently at NeurIPS this year, we also presented another paper called AB MCTS, where we use a Monte Carlo tree search, which is kind of a form of like a tree based approach of combining closed models, where we can combine open AI models, Google's models, DeepSeq models to get the very, very best performance on like an ARC AGI to some of these hard challenges. So one of our theme is to try to build systems to bring the best out of our foundation models. And I think evolution is a fairly strong candidate in that domain.

15:47David Ha:The other parts that our company works on, I would say like this collective intelligence part is more like our developing a foundation layer, developing the very, very best foundation layer technology to get high performance on a particular task. The other observation that we've been seeing is with the advent of these frontier models. I think large language models can, you know, they hallucinate a lot. But at the same time, just like humans hallucinate a lot, some of these hallucinations lead to great ideas. Sometimes we just hallucinate and, you know, some of the best inventions come out of serendipity.

16:29David Ha:You know, we didn't really plan for this stuff to happen. And our company explored this, our lab explored this last year by using evolution combined with LLMs for the purpose of idea generation. So last year, we presented a paper at NeurIPS in 2024 called DiscoPup. And in that paper, I believe we're the first or one of the first published papers to show that LLMs can actually conceive of new LLM training algorithms. So the key idea behind DiscoPop is you take a frontier model back then. It could be like ChatGPT or it could be like Gemini or it could be like Lama. And you would use that model to produce thousands of ideas on how to produce a better algorithm to train an LLM so that we can train LLMs more efficiently.

17:29David Ha:So we actually called this idea internally called LLM squared. It was a bit ambitious. And we would use evolution to find some of the best solutions that were hallucinated. And this solution has to be tested on actual code. They actually have to run as a Python code on PyTorch. And evolution would combine these ideas for the next iteration and next generation. And then we ended up with a pretty good state-of-the-art algorithm for training and fine-tuning LLMs. So that was one of the test cases of using evolutionary computation to select and breed the best algorithmic ideas generated using LLMs to train another LLM.

18:16David Ha:And I think that was really neat because ultimately this technology is consuming so much resources in the world that it makes sense to use AI itself to make AI more efficient. So after that work, we went a bit further. We thought that it was quite promising that LLMs can conceive of LLM training algorithms. Then we proceeded to propose the AI scientists, which is a general framework for a system of agents to conceive of new scientific ideas. They have to, you know, for us, we limit ourselves to computational sciences because we may not have the resource to run a wet lab in our company. And we would have this system of agents run experiments on our Humboldt cluster.

19:03David Ha:they have to actually collect and refine the experiments and perhaps even publish a you know write write up an academic paper using latex and we would also train another llm agent for the purpose of reviewing these papers so it's kind of you kind of have an adversarial thing going on you have your scientists and then you have reviewers that were calibrated on scientific review and then you can have an evolutionary system in the loop to kind of select the best ideas and iterate on. So I feel like using evolution combined with the search process of LLMs makes a lot of sense, because rather than having just one LLM telling us what to do, it's much more reliable to have LLMs tell us a thousand different things and have an evolutionary algorithm or a search algorithm go through each of them to let us know what is the best thing and then iterate the ideas from there.

20:05David Ha:So evolution for us has been a really good tool of, you know, in summary to combine the frontier models to get the very best performance. It's also been a remarkable tool for you using this technology and repurposing them for the purpose of AI driven discovery. So we've been, you know, I think, you know, our lab is known for these two themes. And we've been able to, you know, grow as a company to build on these two R &D developments. And, you know, in the last six months or eight months, using these tools, we've been actually able to grow, grow a healthy and sustainable enterprise business in Japan, Japan of all places, to deploy systems.

20:53David Ha:So yeah, things are going well. So we're looking forward to next year and beyond as we grow this thing. Yeah. I just want to ask your work on world models. I just had Fei-Fei Li on talking about Marble and her world labs. and in the past I've spoken a lot to Yan Lakun about Jepa and his world model work and there's a fundamental difference between what Fei Fei is doing and what Yan is doing. Fei Fei is creating an internal representation that then outputs an explicit model of the world of whatever world you want to output. Yan is working on building an internal representation of the world as it exists in order to reason on that model.

21:54And which side is Sakana on and how do you build your world models?

22:02David Ha:Yeah, I mean, this is... I was actually quite proud of the concept of world models being an actual thing. in 2025. There's just many companies working on this and many startups. And I think like both Fei Fei Li and Yan Le Kun are great researchers on their own right. They're probably exploring things that are worth exploring. My impression, you know, as you mentioned, I think Fei Fei Li's startup, World Labs, focuses on more like developing a representation for realistic 3D environments. You can tell by many of the demos that they look like, you know, 3D games. This is similar to Google's genie efforts as well.

22:43David Ha:You look at many of the generations and they look kind of a 3D generated. So I think the emphasis is to create a representation of a realistic 3D world where the principle behind that is if you're able to harness the compute and scale, you're able to create and hire and hire the fidelity models where you can then do many things inside of the world models, even training entire agents inside the world models to learn new skills. So this is not just Fei-Fei Li's work, but recently, SEMA, which was released by Google, also exported concepts of high fidelity world models that is hyper-realistic. And I think Yan Le Koon's view is more like developing more of the abstract representations for world models.

23:39David Ha:where these abstractions are required to learn reasoning concepts. You want to have these thought vectors inside the world models, and this is how you would do things. So both are very actually required directions for this technology to be done. One is scaling, and one is more like investigating the representation. For me, when I started the world models project with Schmidhuber around 2015, 16. I think Schmidt Hooper's earlier concepts is actually more, I would say it preceded Yannick Kuhn's ideas. I mean, usually that's the case. But a lot of the motivation behind the world models work that I worked on is on developing representation.

24:33David Ha:Like I didn't really care whether the world models would output a really realistic rendering of the real world. And in fact, we showed that the more realistic it is, the higher fidelity it is, the easier for evolution or the agent to exploit some really weird bug in your simulation. So I think this is more like a paradox. the larger your model is and the more detailed it is, it's actually easy to actually find some particular small thing for your agents. If your agent is to beat that game that's simulated in your role model, it'll figure out how to move in such a way to get unlimited scores right away.

25:25David Ha:But what we found is if you make your model noisier, where there's actually a bigger gap between your simulated reality and the real game or the real environment, your agent actually is forced to learn real, like more important skills or it has to actually fend for itself in this really nasty game. So imagine your representation is not so perfect, but it's actually more tricky. It's more tricky to navigate in the dream than the real world. It actually presents some of a difficulty for your agents to develop the skill. So I think it's an open question. I think it's a balance. Of course, you cannot have a pure latent model with no grounding in reality.

26:18David Ha:Then that would be foolish and it would not learn anything useful. But I'm really interested in studying this balance. I think as humans, I'm sure that we don't have the same type of capability as Fei Fei Li's raw models or genie model. We cannot explicitly, at least me, maybe not everyone on earth, cannot draw out a photorealistic version of a video of what I'm imagining in my head. But somehow we can reason in this latent representation. And I'm actually quite sympathetic to Jan LeCun's direction. Maybe his future company would focus on more of this latent, higher order representation of world models.

27:08David Ha:And to be fair, this is also things that Juergen Schmidhuber has been conceiving since the early 90s. yeah yeah he has been working on like actually even using getting agents to dissect directly into the hidden states of the world model so you you don't actually use the world models hidden latent representation directly the agents would basically examine directly into all the hidden connections of your world model to able to figure out intuitively what it should do even skipping the planning part. So there's a lot of work to do. And, you know, for me as a startup guy, I really am excited about the world model's direction.

27:54David Ha:But we have to really focus on our, what we're building as a startup. So we decided to focus mainly on on collective intelligence, combining existing models, and using these models for scientific discovery. You know, Because although deep inside me, I'm quite passionate about this model's direction. And I think we do want to get back into it at some point. Yeah. On the paper that was presented at NeurIPS, I understand model merging, where you have essentially two code bases. and the evolutionary system decides which, you know, how to combine the two code bases, what to keep, what to throw out, how to fit it all together.

28:48I didn't understand how you do that with proprietary models where you don't have access to the weights. and it's you're talking about kind of a metalayer that that uh well explain to me what what the paper that was presented that that easier than me it's getting really interesting because our

29:12David Ha:first paper on model merging is uh it does require access to the weights of the model yeah i think like a it's it it'll be imp it's an impractical assumption impractical uh because uh in reality most of the frontier models are closed and even if we did have access to the waste they probably do not share architectures uh with each other so uh what we decided to do uh and it's also you know as these models become bigger the context length also becomes longer as well So there is a duality between the prompt and accessing the weights of the model. The more context you can put in, it's in a way influencing what the model is doing.

30:03David Ha:So in this ABMCTS work that we presented at NeurIPS this year as a spotlight, We use this scheme to prompt or search through the reasoning techniques of multiple models. You would actually use the sort of the banded approach to give the variations of the same questions for multiple different models, get the responses and decide based on the responses how to actually ask the next iteration of responses in a tree-like fashion. So rather than using evolution to combine the weights, we're using like an evolution type scheme, in this case, Monte Carlo Tree Search, to actually conceive of different conversations with the model.

30:52David Ha:So how to drive the conversation with these batch of frontier models to get really great answers for a particular task. And so you're prompting a range of models and you're taking the outputs and then creating a new set of prompts from those outputs and then re-prompting that same series of models? Yes, that's correct. And it's done away in a tree search method. So you would actually go down the rabbit hole of the most interesting dialogues. So it's like if you have a party with 100 people, you would find the five most interesting conversations, and then you would like continue to only talk to those five people and then you keep on going and going.

31:46David Ha:So that's one work we're doing in our company we're actually exploring multiple different ways of combining frontier models as well so we we will have some exciting work coming up in the next few months where we we actually have better results for for this direction so i i think it's quite exciting for for combining different models when you say in the coming you're talking about working on the context and not not you're not talking about using open weight models and combining code bases and creating new merged models is that right you're talking about this other strategy yes when you're saying that yeah yeah yeah uh and can you you know with ai scientists i wrote a little article in that when you're that paper was written by the model and in our system and uh submitted i think it was the icml was it or i i clear i can't remember but submitted i believe we submitted it's it's quite early work i mean the paper is not like a particular good or anything uh but we we know i know yeah it was kind of uh we had to do a few things because right now using generative AI to produce papers at an academic conference, it is like a, it's an issue that we have to guide very carefully.