Why the Future of AI Isn't Just Bigger Models. It's Models That Evolve | Risto Miikkulainen of Cognizant

29:32And we don't, it's unlikely that we get it from these architectures. Instead, we probably have to look at the brain and biological neural networks, circuitry there and organization, and strive towards something like that. Now, how do we actually get that to work? That's a neuroevolution problem. We set the constraints, we set the elements, the primitives, and we let evolution discover architectures that, say, answer questions about their own performance, metacognitive questions, or put them into an environment that's continuously changing and requires this continuous adaptation. And architecture should evolve that actually cope with that.

30:10And that's exactly what, say, Sebastian is working on. And we're starting to work on the metacognition part. And I think this is... And it might actually require also actual neuroscientists from which we can get ideas about what the directions are and what the constraints are. Because neuroscience has solved neural networks, positive neural networks know very well how to do this. And we don't really have any other examples. So it would make sense to look at neuroscience and other disciplines related to that. And then bring in neuroevolution in order to start small and not too ambitious, but build towards those architectures that duplicate that kind of behavior.

30:49Can you explain sequence of experiments that you would undertake to reach that goal? So you start small. Anything that you do, I always tell my students that your first experiment should be so small that you don't want to tell anybody that you did it. It's embarrassing that you do something that small. So, for instance, you just simply try to train a network to answer a question and then answer a question where you write in that. Is that correct? Do you actually know this fact or you just made it out? And you can do it. I mean, and then you may be fine that the standard neural network architecture doesn't handle it, or maybe reinforcement learning wouldn't.

31:29But if you evolve the circuitry, you might get it. This is an experiment that hasn't been done yet, and I'm more or less proposing it. But how do you evolve it? What are your, again, elements, primitives, constraints? There you have to talk to a neuroscientist that might tell you that you need feedback. You need reverberating circuits. You need spiking neural networks. I don't know. We'll have to think about it. But it's not just one thing. We can do exploration, pretty wild exploration. We can open up all kinds of parameters for evolution to optimize, beyond what we can even think of, perhaps.

32:07And neuroscience is also somewhat limited on what they consider as computation, because it depends so much on what we can see and measure. So you have to have electrodes, you have MRI, you have a couple of tools, and our theories always depend on what we can see and measure. So it might even be useful to let evolution explore a wider area and even give ideas for the neuroscientists of what might be going on. What are the inputs? So you have this evolutionary, neuroevolutionary model, and you wanted to explore architectures that would understand its own outputs or that would not forget as it learned new things.

33:00That's a model. Yeah. Is that right? And then you're feeding in inputs. You have multiple instances of that model and you're feeding them different inputs. I mean, how is the experiment? You have to have a couple of things. You have to have a domain of interaction, and you have to have a way of measuring how well you're doing. I mean, you still need that feedback. You have to decide which ones of your candidates are good and which ones are not so good. So you can run this recombination crossover and mutation. But there are probably places that are better starting points. So if you look at neuroscience, memory systems, hippocampus, is a pretty good start because we know a lot about the circuitry.

33:43in hippocampus and we know a lot about like place cells for the states and it's also involved in memory space and memory um so we have a task of perhaps navigation and remembering where things are um and then you can start asking questions about what you really know you really remember something that's there or did you just make it up um so we can we can then pose the question of do you actually know your own knowledge in in that constrained area of hippocampus and memory systems And then there's, of course, really big questions like how do memories transfer from hippocampus to the rest of the cortex for the lifetime because they are no longer in hippocampus.

34:22And we could also expand that way and find out what kind of circuitry allows you to do that transfer or indexing or whatever it is. So we don't have to stop there. But a starting point, neuroscience motivated. I think, as I said, such a starting point, hippocampus and the memory system is a good start. But how is it inputted? I mean, is this, do you have an LLM interface that you can provide the data in natural language? Or are you, yeah. Yeah. So initially, perhaps not because, and the benefit of having hippocampus is that there's the space representation. So it will be probably navigation task of some kind, location-based.

35:04And like I said, some object in a location, go there, you know, find it. Go there again. Remember where it is. So we don't actually have to start with language, which is on the top of the cognition in humans. But we can start with something that's even very simple animals know how to do. I mean, hippocampus is one of the oldest areas of the brain. But there's a lot of the elements that we are missing in current LLMs are actually there. So we can address the really challenging questions, continual learning, as well as the introspection and metacognition in that domain of navigation, spatial representations, and memory.

35:40And that's why I'm proposing that that would be a good starting point. It seems pretty greenfield that you developed these tools and it's now a matter of seeing how well they work on different problem spaces. Computation is a big part of it. I mean, we were working on these problems of natural language processing in the 80s and 90s. And the ideas were pretty good, but we just did not have the data, we did not have the compute, and we didn't even understand that if we did, it would actually work. And now we're kind of in the same situation with evolution computation, that we've had a lot of ideas but haven't been able to scale them up.

36:22And what would happen if we really did and took advantage of this compute? We could run these experiments on evolving a hippocampus, for instance. And even beyond that, my pet project would be understanding how language evolved. Now, what are the... because now we can have simulations of complex environments, world simulations, and we can put these agents in there and have them involved and develop abilities and eventually maybe communication and even language. So that's a big, big challenge for longer term, but we can now, we have a compute and we have a way of simulating this environment so things like, questions like that could be addressed.

36:59You've been working on this for a long time. Why have Have you not or have you worked at that scale, scaled up? Sure, we tried. I mean, there are many studies by us and others on evolving communication in virtual creatures. But that's always where it stops. I mean, you can signal and they can read each other's signals and behave in a meaningful manner. But language is still a big step beyond that because you have to have linguistic structure, grammar, and flexible structure and rolls and fillers and so on. And we know that some animals are pretty good at that and they can even adopt features of human language if you really train them, but they don't do it naturally.

37:46But animals communicate, but not with language, not with grammar. And they can adopt a lot of that if you really, really train them. So the question is, why did that evolve and how did it evolve? Why only humans? And what other communication systems are possible and is this the best one? And very interesting questions become up. Now that we have the compute and we have an ability to simulate, we can start asking those questions. Scale it up. Yeah, and is that what Cognizant's doing? Is that what Cigana is doing? Maybe someday. We hope to get there, yes. Is it a financing issue? It is. There's no immediate, necessarily an application of it, because it's really a fundamental scientific issue.

38:30It's one of those fundamental challenges of science is how the language evolves. Well, I'm not talking about that particular project, but just scaling. Oh, absolutely. Yeah, absolutely. And there are immediate applications. Decision-making, anything decision-making in society, business, healthcare, medicine, science, we talked about, is an application now. And we can take many steps before we get to these questions that are really kind of fly in the sky. Yeah. I mean, I can see with finance, it's a data-rich domain. This could be applied to anything, but you need the data to evolve. You need to be able to evaluate how well you're doing.

39:16It doesn't need to be data-based. Like I said, simulation could be just as good. So you simulate behavior and you evaluate how well they perform. Well, in stock market, you could actually trade stocks and see how well it works. In healthcare medicine, you could see whether your treatments have good effects. But you don't want to do that in the real world. You have to have some kind of a surrogate model of the world. That's where you need the data. If you have data, you can train a surrogate model. If you have a surrogate model, then you can discover the system strategies. And that, I think, is a really good approach that we are ready for.

39:51Is that why a lot of the people involved in neuroevolutionary AI are also focused on world models? I think the reason is perhaps that now we are transferring or transforming or making progress and moving from imitation, like models that just imitate the statistics of the world that we already know, to agentic AI. I mean, that's a big word right now. Agents that interact with the world. They make decisions. They affect the world. The world changes. And that's where you need a world model. So I think it's more of a result of us going from modeling to agentic AI, and we need these world models to do it.

40:34Does Cognizant work on world models at all? I know Sakana is doing it. We are not developing right now a world model ourselves, but we are definitely using them in order to do agentic AI. In whose world models are you using? Minecraft has been one of those that's existed until now as perhaps the most versatile such world model. But now they are just now, like within the last few months, things have changed and more miles are coming out. We'll have to take a look and see what we can do with that. Have you seen? I have Fei-Fei Li, who, from what I understand, could generate sort of information. Yeah, that's a great opportunity to take advantage of that.

41:11Yes. So where are you guys going from here? Well, let's talk about the book first. The book is kind of the intent is to create an authoritative guide or textbook to neuroevolution in much the way that reinforcement learning by Sutton and Bartow did. Yeah, very much like that. And there hasn't been one to date? There's been books about evolution and computation, but not neuroevolution. And is the book intended, is your hope that you'll generate a following and a shift the research community? Yeah, attract more attention. I mean, we are providing not just a book, which is, first of all, historical overview where all these ideas came from, but also what's currently happening, what's breaking through.

42:16like these combinations of neuroevolution and reinforcement learning and deep learning and biology and generative AI, LLMs, all of these are like breaking out right now. And if we can give people the background where they came from, if they understand the strengths of these different approaches, then they can contribute. And I think that's the role of the book, like give everybody the basic knowledge so that they can take it further. And we believe that this is a time in science where this is happening, that there will be a lot of expansion. We want to encourage that. And beyond the book, we also make a lot of software and digital resources available.

42:59So there's lots of demos that are inspiring, exercises for students. And then we have a GitHub site for community to provide software, to provide pointers to their new papers, new developments. So we're really trying to build a community that could work together, taking advantage of that. And they would have a common foundation. Yeah. What I find interesting about this is I've been talking to people about, I had Carl Fristen on talking about free energy principle and post-transformer architectures. There's another group in New York working on post-transformer architectures. Fei Fei is working on her sort of explicit world models.

43:41Yan Le Koon is working on kind of internal world models. This is almost a horizontal technology, neuroevolution, that can apply to all of these sub-problems. Is that right? I mean, that's what strikes me is it's not domain-specific. Right. There's actually a lot of different things that can be helped or advanced. And I don't mean domain-specific. I mean, it's not specific to one school of AI. Right. It can be applied to all of them. Not the solution for everything. I mean, there are certain areas where neuroevolution can really help, and others where you have other methods that are better. And indeed, if you have a lot of data and you are interested in modeling the statistics of it, Keep learning is perfectly fine.

44:36And it's really good. It's really those areas where there's opportunities for finding creative solutions, where we have little idea of what's going on and what the right solutions should be like. Like I said, green field. It's a great method for discovery in such fields. And your association with Sakana, I mean, do Cognizant and Sakana work together at all, or are you very independent and it's just for the purposes of the book? It was a book, really. I mean, we could perhaps in the future, but it was really just all people coming together because we all provided a different compatible perspective on neuroevolution and found out that actually if we put those together, it's a pretty good foundation for the field and for the community.

45:22And so Cognizant, what's Cognizant's, tell us the history of Cognizant, what's its mission statement, who funds it? Well, Cognizant itself is a very large company, 350 ,000 people, and they have been doing staff augmentation as well as consulting, and now it's turning to an AI company. But the idea is to take AI to the world. They already have a structure to help digital transformation in many different companies. So now it's becoming an AI transformation. And our role, Coins and AI Lab, is to provide the technology to do that. I mean, they can use any technology they want. And there's many, many providers of those.

46:04But we're also spearheading the development of multi-agent systems in particular and neuroevolution as a decision-making system strategy, decision strategies for business, for instance. And there are many businesses that benefit from that. Like I said, some tasks are good for it, not others. But you can make decisions, for instance, how to allocate resources in marketing or in research itself or something like that where you really have to be creative in order to be effective. it. In other areas, other techniques might be used. But multi-agent systems is one area which is also very strongly coming.

46:46We talked about deep learning, big data first, and deep learning, and now then LLMs, and now agentic AI, and multi-agent systems, multi-agent AI perhaps, I believe will be in the future. So now we have multiple agents that are talking to each other and solving problems together. And it's flexible, it's modular. You can have agents whose job is to evaluate other agents and make sure that the system stays in check. And there is absolutely a central role for some agents that are creative, that implement and learn those decision strategies in such a context. There might be a role for new evolution to fine-tune those models, so they become very skilled at certain kinds of behavior.

47:28Otherwise, they are just LLMs that are using RAG, they're using documents, but they can also become real experts on how to reason about certain kinds of, I don't know, medical knowledge, for instance. So that is what we are doing in a time that they are lab. It's multi-agent system is currently a big push, and then neuroevolution as an element in those multi-agent systems is a role for this technology, yes. And so your basic research on neuroevolution is taking place at Austin? Yes. Well, both. I mean, I'm also a professor at UT Austin, so I do have a research group there as well and people who are working on mostly right now it's mostly on cognitive science which i also have been working on for a long long time so that is modeling say human behavior like patients who suffer the stroke or dementia how to help them we've looked at the visual cortex before but it's all coming together now because we're really starting to see that evolution which we used mostly to create behavior, now has to look at these cognitive aspects of behavior.

48:39So I believe that in the end, they will come together. We will build systems that are cognitive using neuroevolution. Yeah. I'm just curious, how do you manage all of these different, I mean, they're related, but all of these different directions of research. Yeah, it's to say that they're all very different directions, but they're all related in my head. But I mean, do you have a team that's working on this and you spend a few hours with them working out a roadmap and then you switch and you have some people working on something else? Sure, but there's this dynamic self-organization. Teams form, if it really works, teams form around an idea.

49:26That's an opportunity. and I'm lucky to have some of these senior colleagues on Collin State, for instance, Shin Chu was mentioned and Elliot Merrison, for instance, who have their own teams and working on different aspects. And we talk all the time and also internships are great. So I have students who are at UT and they come for internships at Collin State A-Lab and learn a lot and maybe stay or maybe they back and do their dissertations on topics related to that. In general, in the bigger picture, I think what is really wonderful right now is this interaction between industry and academia. So we're all here in the same conference, and we publish papers, and we talk the same language.

50:08And this openness of industry and the possibility of academic people to have a one-footed industry, that, I believe, is a big factor of why AI came what it is now. Is that partly because industry has the financial muscle to provide the... Absolutely. And you can still, you can kind of still see a little bit of specialty for both. So industry has finances and means resources, just computational resources, but resources otherwise too. Resources to access, to get data and to have people who build systems that actually work. You know, and academia, well, maybe if you don't have quite the resources and don't have the engineers, you think and be more creative and test ideas that are further along.

50:58So we are actually evaluating or testing or generating, exploring architectures that are not right now feasible. While industry, more or less, will have to have architectures that are feasible. So there is still a little bit separate role, timescale-wise and exploration-wise and resource-wise. But I think right now, it's a very synergetic relationship. Yeah, that's fascinating. that you're testing ideas. I mean, the whole process of scientific research fascinates me, that you're coming up with hypotheses that you don't know to work. You're right. Industry may not want to waste money on it, but academia.

51:45A lot of times I say this is the best time ever to be alive. I mean, this is so much excitement now. You have opportunities, you have an idea, and it has a very good chance, if it's a good idea, to actually affect the world and make the world a better place, which is what we always used to say, but now it's really reality. Yeah. Where do you see neuroevolution going? I mean, it sounds so powerful and it can be applied to so many things. Is developing it a matter of compute or is it a matter of data? Because you need good data and presumably a lot of data. I mean, you were talking about decision-making in the different problem spaces.

52:38I always ask people about the Ukraine-Russia war. If you had enough data, would an evolutionary system be able to find an optimal outcome by trying all the concerns of one side, the concerns of another? I mean, maybe that's not a good example. Oh, no, I understand what we're getting at. And this is really something I often very much like to end my talks with. as this opportunity for the society in that we have AI now, and it's indeed discovering decision strategies is a big part of it because we can talk about decision-making in society. And the dream is that we decide what we want the society to be like.

53:35Do we want to maximize profit or production or scientific progress, or do we want it to maximize equality, distribution of wealth, protection of environment. If we decide what those goals are, then using this kind of technology, we can come up with decision strategies that get us there. And it's important because they are objective. They don't really care about, I don't know, pork and personal agendas and other things that often get in the way in a society, right, or making really good decisions. So we have, I believe we have the technology, if we have the will to take advantage of it. Now, I'm not quite sure of really big challenges like Russia, Ukraine war, or the Middle East, because there are so many of these other agendas and things that come into play.

54:25But if we can actually identify some part of society where the goals are clear and agreed upon, I believe now we could do a much better job by utilizing technology. Do you think that you need, as you said, the will to use it? Is this something that governments could take advantage of? And are they? Well, yes, absolutely. So one example, pandemic. Everybody wanted to do something about the pandemic when it started, and we also developed a decision-making system for non-pharmaceutical interventions. Should you close the schools? Stop the buses? wear masks, do contact tracing. There are many different dimensions that you can try to establish in order to help or prevent the pandemic from spreading.

55:16And it was based on data, and I was lucky that the data was available, the cases and deaths and hospitalizations, but also what governments were doing all around the world. And they were doing different things, so there was diversity, and we could learn from it. And it was quite fascinating that we could come up with these suggestions, just training the system overnight. The next morning, the system would make suggestions for any country in the world where we had data. Now, it was really difficult at the time to get anybody to listen. And it wasn't just our group. There are other groups who have been studying.

55:51U.T. Austin, for instance, Lauren Meyer's group has been studying pandemics for decades and had very good understanding. They had actually an audience in Austin, in the city, that listened. And Austin did really well compared to other cities because they listened to science. And our goal was really the whole world, the different countries. It was much harder to call up Modi in India and say, this is what you should do. We did have one piece of success, and that was Iceland. So in the fall of 2021, they were considering what they should do when schools open after the summer. and we had a contact there to Raspalund, who was an academic and had contacts with the government all the way to the health ministry and even prime minister.

56:39And now we could run models that were relevant to them and run these scenarios, make suggestions and that actually worked. As far as I know, they did communicate to the ministers and they did follow some of those because they did well at that point. So it is possible. But it was a much smaller country than India, for instance, or US. And it is still a big challenge, communication of science. It's magic to many people. And you really have to work hard on getting the message through. I think the technology is ready, but the communication is not. But is the technology ready? I mean, it sounds like it's still in the development phase.

57:23But if you pick a topic that is a good match, yes, I think it's ready. Like the pandemic, it would have been, it was. In your work with Cognizant, can you talk, what's the biggest application or problem that you've applied a neuroevolutionary AI? The most immediate ones are things like marketing, budget allocations, transportation, design of clinical trials, tend to be a little smaller. domains where either the current state of the art is relatively uninformed, so you can't really go wrong, or at least there's a comparison and you can immediately tell that there's value. Now, and this applies to AI in general, why it's really hard to get into the world is because a lot of times AI is very ambitious and AI would just replace what's already there.

58:25And this is a very hard decision to make. So we have to build AI that maybe runs alongside or replaces a small piece that doesn't really matter yet. And you gradually gain confidence and you gain ground that way. And humans are always working alongside with this AI. And it's empowering these humans to make better decisions. So that's how we have to approach, and that's what we are doing too. That you find these small wins, small victories first, And then you gradually expand when everybody gets more comfortable and the value becomes clear. I had on the podcast a while ago a company that's Aerotechnologies is the name, and they're building these decision-making co -pilots.

59:12and the idea is a CEO would have this model in his office accessible and it could help him work through decisions of what the optimal is. Yes. It sounds like you guys… Very good idea. That's exactly what we can build and we actually had the prototype already and pandemic decision-making was exactly that. It wasn't a CEO. it was for health officials. And one important aspect of that is that when AI makes you a suggestion, you get immediately a prediction of what its effect is. So you can see the economic cost and pandemic, you can see the economic cost, and you can see the number of cases. But not only that, you've got to give the CEO or the pandemic decision maker an ability to change those decisions and see the effects.

1:00:07So they can convince themselves that even though I tried a couple of things that I think might work, they don't work as well. This is the best solution. That's how you convince people and you make them more empowered. They have a better idea that what they're doing is right. Yeah. And you think this kind of decision making will eventually spread through the society? I certainly hope so. And I think it will spread when people see the value in it. So when people, at first, maybe they're afraid of AI and that they don't understand it, and then maybe afraid in the sense that they might think that it rephrases them.

1:00:43But when they see that they can use it to their advantage to do their job better, understanding better, get more reward and satisfaction, I think that's how it starts spreading. Right. And Cognizant, is it implementing this largely on a bespoke consultancy basis, or are you building products? Yes, both. Well, it's still starting it. So we have developed basic technology and now we are building teams that can actually go and implement these for customers. And this will expand. So we'll have more teams and maybe at some point also a product that customers can then self-use. But AI is still difficult to actually deploy.

1:01:28So we have to have core teams that really know what they're doing and work together with domain experts. And that's where we are now, at this space where it requires expert knowledge that we are building or we have, and we can start with a few successful applications and then scale. On this combining or evolving code or self-recursive improvement of code, how does that work? I mean, if you have two code bases and you evaluate the outputs and decide that there's some good from this one, some good from that one, let's combine them. Yeah, that doesn't work. Yeah. I mean, obviously, first of all, it has to be written in the same programming language.

1:02:26Or is that not a problem? Well, different levels. But I mean, traditionally, genetic programming as a discipline works with certain language. And early on, it was Lisp because it's very easy to combine and recombine Lisp. But now there are languages that are designed for precisely to be evolved. And the idea is that they are compositional. You can take parts from one and another part from another. Now, how you do that, there's a big chance part of it is randomness. So you do it randomly. Okay. But it doesn't mean that it's just a random search. Not at all. Because you are taking parents that are good.

1:03:06And if you recombine them, chances are pretty good that your offspring is good too. But the random recombination is the creativity part. You want it to be something that is not just like, hmm, I think that's good and that's good. Because then you are putting your preconceived notions in the play and you are limiting your search. But if you do it randomly, anything is possible. And that's how you make those big discoveries that you wouldn't otherwise make. So the idea is a little different from this is good and that's good, so I put them together. No, we don't do that. We say that that parent is good and that parent is good.

1:03:40So they both have good aspects. I wonder if I can recombine something even better. And you can do it many times. You can have lots of offspring and some of them are better than the parents, then you make progress. Combining, you said, is composable. Combining functions or units of code? You have to have some way of getting an offering that's functional, that actually is a viable individual, yes. Okay, well, I hope we can, I don't want to keep you longer.