#302 Karl Friston: How the Free Energy Principle Could Rewrite AI

0:00We change our mind 100 milliseconds by 100 milliseconds as we engage with the world and we continue making sense making sense making sense making sense making sense. And that's a process of inference. It's not necessarily learning, which will be a much slower process. And if the way that our brains work can be cast as inference, that means if our brains don't work properly, as in mental disorders, then we can understand that as false inference. The brain is in the game of basically trying to predict what it's sensing. So if there's a prediction error, that's newsworthy. So it can now use the prediction error to update, to revise its explanation, its representations of the latent causes.

0:39So in the moment, I literally change my mind. I literally change my mind on the basis of the prediction errors that have been elicited by comparing what I predict and what I actually saw. Hi. The transformer algorithm created by a team at Google in 2017 has been the core of generative AI. But scaling transformers has hit practical theoretical limits. New releases are providing only incremental improvement, and as a result, many researchers are looking into new architectures, aiming for better memory use, better generalization, and longer context reasoning. Among those researchers is the team at Versus, a startup implementing the ideas of Carl Friston, who is the most highly cited neuroscientist globally and one of the most cited living scientists overall.

1:40Tristan is the creator of the Free Energy Principle, a sweeping attempt to explain how all living systems, including brains, maintain order in a chaotic world. His work has inspired entire branches of computational neuroscience and new directions in artificial intelligence. Versus, where he serves as chief scientist, was started by some of his students. Its core project is called Axiom, a groundbreaking AI architecture built on the principles of Friston's work, taking inspiration from how the human brain predicts and adapts to its environment, not by memorizing data like today's models, but by constantly minimizing what Friston called surprise, which is a measure of the difference between prediction and reality.

2:37It's an ambitious attempt to create machines that learn and reason the way living systems do, grounding intelligence in physics rather than statistics. In fact, this is my second conversation with Friston because I was completely lost the first time around, and he says himself that he has trouble speaking in the vernacular. So while some listeners will be able to understand what he's saying, most are better off letting the jargon wash over them, focusing instead on the higher concepts and how they're implemented in an AI model. You may not understand everything, but you'll come away with a picture of how and why post-transformer architectures are being developed.

3:26I hope you find the conversation as fascinating as I did. But first, I want to give a shout out to our sponsor. Build the future of multi-agent software with Agency. That's A-G-N-T-C-Y. Now an open source Linux Foundation project, Agency is building the Internet of Agents, a collaborative layer where AI agents can discover, connect, and work across any framework. All the pieces engineers need to deploy multi-agent systems now belong to everyone who builds on agency, including robust identity and access management that ensures every agent is authenticated and trusted before interacting. Agency also provides open, standardized tools for agent discovery, seamless protocols for agent-to-agent communication, and modular components for scalable workflows.

4:31collaborate with developers from cisco dell technologies google cloud oracle red hat and more than 75 other supporting companies to build next generation ai infrastructure together agency is dropping code specs and services no strings attached visit agency.org to contribute that's A-G-N-T-C-Y dot O-R-G. My name's Carl Friston. I'm a professor of neuroscience at University College at London and chief scientific officer of Verses, a cognitive computing company in Canada and America. Verses is using your free energy principle in its models. And to talk about free energy, I thought we would start with physics, move on to neuroscience, and then discuss how it's implemented in, or the free energy principle is implemented in machine learning.

5:40My understanding is that free energy is a measure of the usable energy in a system. Can you talk about that? It's a very confusing concept. it gets more confusing when it moves when you take that principle to neuroscience but can you talk first of all in layman's terms what free energy is in physics yes i can or perhaps i should qualify that i'm not sure i can do that in layman's terms i have a reputation for nothing so you're gonna have to hold my hand in in trying to um make this accessible um so the free energy we're talking about here is closely related to the thermodynamic free energy that you're talking about, which is the amount of available work in a system that is not, if you like, entailed or trapped or locked in because of the entropic or the disorder of the system.

6:44However, the particular mathematical free energy, the variation free energy we're talking about, is a purely information theoretic construct. It actually inherits from Richard Feynman's work on quantum electrodynamics, and he was trying to solve a problem, which in fact Versys is also trying to solve in one sense, which is finding the most efficient paths, the most efficient courses in his context. These were the sort of paths of small particles, for example. So trying to find a probabilistic description of the paths of least action. And he solved that almost intractable problem by turning an impossible marginalization or integration problem into an optimization problem.

7:34And what he did to do that was introduce this notion of variational free energy. And then that was taken up decades later by people like Geoffrey Hinton and David McKay and the like. And they noticed that this information theoretic quantities mathematical information theory quantity that has exactly the same components as a thermodynamic free energy as an expected energy that's supplied by some world model or some probabilistic specification of the thing that you are trying to model minus the entropy was exactly the same that you would need to optimize things from the point of view of um crypto analysis and um with a nod to the russian notion of algorithmic complexity and that led through to universal computation in in terms of crypto analysis um and also was exactly what you needed if you wanted to make something like a Helmholtz machine work.

8:40So if we go right back to sort of, you know, the 1990s, you'll find sort of prototypical variational free energies at the heart of the things like the Helmholtz machine, like the sleep-wake algorithm. And indeed, you can track that right through to things like variational autoencoders. So this is like a universal objective function, and it has two bits to it um one bit is the entropy term and the other bit is this constraint or expected energy which is this basically um something that enforces um an accurate account if you're trying to model something say you're taking a sort of a neural network that was trying to predict the next word or trying to compress information in an image to predict the image and thereby generate new kinds of images, then you're going to want to do that as accurately as possible.

9:42But by including the entropy term into the free energy, then you're effectively applying a kind of Occam's razor. you're sort of not committing to a particular explanation but you're including or you're trying to um um not overfit the data because you're trying to maximize the dispersion or the uncertainty about your particular explanation for this content or for the uh for these data i slipped that in uh again this is not vernacular and i apologize but there's a beautiful link between uh minimizing free energy which is just a sort of expected energy minus entropy which means that you're implicitly under constraints maximizing the entropy which of course is james's maximatory principle which is a cornerstone of measurement physics and you know it is that sort of if you like built into your cost function it is that sort of uncertainty aspect that means you preclude overfitting and elude many of the problems that we see in current uh say deep neural networks um optimize using reinforcement learning so this free energy principle is a first principle account of how to make neural networks or neural networks used as agents or as classifiers or inference machines as efficient as possible, going right back to the path integral formulation of Richard Feynman in terms of finding the paths of least action, the paths of maximum efficiency, the paths of least effort that include, if you like, this cost of overfitting, technically a sort of complexity cost.

11:37So I don't have to change my mind too much in order to account for these data, which means that I don't want to commit to a particular explanation for these data. Does that make sense? It does, only because I've done a lot of reading. I'm thinking for listeners, one of the confusing things for me when we first spoke was the word energy when you apply it to machine learning. What you're in machine learning, energy is, is the, tell me if I'm wrong, but it's the, it's the difference between an input and the prediction is it's, it's an, it's in effect an error function. Is that right? Yeah, no, absolutely.

12:30Indeed, you can sort of look at this first principle approach to optimization in terms of error correction or error minimization. Indeed, in some parts of the life sciences, you get things like predictive coding, which is basically sort of scores the accuracy of your machine in terms of the prediction error. And that's an important part of this free energy. And indeed, you could actually read free energy in the context, in the setting of predictive coding as a precision-weighted prediction error. It's just the mismatch. So that's absolutely right. So just to address that, sort of unpack the notion of energy in this information theory context.

13:21And energy is just a potential. and a potential is just the sort of negative log probability so it's just a measure of the implausibility of something so if something is very implausible or very surprising or very unlikely or very uncharacteristic and then that has a very high potential it has a high energy and you really want to minimize that you know that energy um in order to produce an unsurprising account or render this content unsurprising given what you might have predicted if you knew the causes of this particular content yeah and and in neuroscience i mean you're a neuroscientist uh you saw this principle as uh as you know sensory input coming into the brain and and the brain trying to fit that in its model of the world.

14:17And if it's surprising, there's a lot of energy there, free energy, and the brain then has to update its model of the world to match the sensory input that's coming in. Is that right? No, that's absolutely right. That's a perfect description of predictive coding formulations of this kind of free energy minimization. that the brain is in the game of basically trying to predict what it's sensing, the sensory content. And if it's got the perfect prediction, clearly the free energy will be zero and there will be no prediction errors. So if this is a prediction error, that's newsworthy. So it can now use the prediction error exactly to say to update, to revise its explanation, its representations of the latent causes of these data.

15:12So, one can describe that in terms, if you're a statistician, that would be called Bayesian belief updating. If you're a philosopher, it would be sort of resolving prediction errors. It does really emphasize a sort of key point about this biomimetic principle of how we make sense of things and indeed act upon the world. We do so in a very sort of inside-out way. We generate predictions based upon hypotheses about what could have caused this content, what could have caused this sensory input. Just use the mismatch, the ensuing free energy flight, the implausibility in relation to your predictions of your actual sensory content to update online and over time.

16:00So now sort of introducing a distinction between inference. So in the moment, I literally change my mind. I literally changed my mind on the basis of the prediction errors that have been elicited by comparing what I predict and what I actually saw. But also, over time, the much longer times I can learn in the spirit of machine learning to be a better predictor given this kind of world. A couple of things about the neuroscientific view that I've found fascinating is your view is that this is an instantaneous, continual loop, that the brain is constantly doing this, and that some mental illnesses may be caused by an error or an inability of the brain to update correctly or to minimize the free energy principle.

16:56Absolutely. And in part, it was exactly that observation that motivated much of this work in the context of neuroscience and in particular computational psychiatry. And you could even go further and say that all mental illness could be construed as some kind of false inference. So you made the point that this process is ongoing, it's continuous. So it is something that we change our mind millisecond by millisecond, or at least 100 milliseconds by 100 milliseconds as we engage with the world and we continue making sense, doing our sense-making of the world. And that's a process of inference. It's not necessarily learning, which will be a much slower process.

17:44and that if if the way that our brains work can be cast as inference then that means if our brains don't work properly as in mental disorders then we can understand that as false inference and that makes a lot of sense in this you know and by false inference i mean exactly what you would have done at school when i don't know if you remember doing type one and type two errors when doing t-tests you know you can make a type 1 error and infer something is there when it's not of course that's a very apt description of an hallucination or a delusion but you can also make type 2 errors type 2 false inferences inferring something is not there when it is and of course that's a precise explanation for many neglect syndromes in psychiatry dissociative syndromes, what used to be called hysterical syndromes.

18:35So nearly everything in mental health and mental disorders can be construed as inferring properly or some failure of inference. And then getting into the weeds of the actual mechanisms of this predictive coding is really quite informative and sort of pointing to the particular failures of message passing entailed by this view of the brain. So I'm generating predictions in an inside sort of way, and then using the outside-in prediction errors to correct and update and perform this kind of inference. And another thing that fascinated me from what I read, that this cycle or this process matches the anatomy of the brain.

19:19I mean, that there are layers like in the visual cortex, and information is passed both back and forth, up and down, which is one of the reasons why backpropagation has never been validated as a process that happens in the brain because there isn't that kind of pathway for updating weights going down. Am I wrong on that? No, and I think you've hit a really central argument. and focus of both the machine learning community and people invested in predictive coding formulations. So just to sort of highlight the importance of that question, it is certainly the case that the functional anatomy, the computational anatomy of the brain has a hierarchical structure.

20:16And indeed, you could argue this is where deep learning comes from. It just means that the world model, the generative model, that is entailed by the structure of our brains has a hierarchical depth and of course then that you know naturally translates into deep neural networks and deep rl for example however there's a fundamental distinction between back propagation on a deep neural network and the kind of optimization and message passing between the hierarchical levels in the brain exactly as you say it is all local so as you move from one hierarchical or say layer in the neuronal brain architecture you are in exactly the way you describe sending messages to the layer below which are the predictions and then receiving messages from the layer below which are the prediction errors that revise the representations equipped with uncertainty that then generate the predictions so there's lots of local message passing crucially the objective function that describes the dynamics implied by that message passing is local so now you get into a message passing scheme which is much more sympathetic you know um well the message passing scheme um under predicted coding has this locality and it now acquires a biological plausibility but also a massive increase in efficiency which you don't get with back propagation in terms of sending all the way to the top and then all the way back again which is a very non-local sort of a violation not just of the principles of biological brain function but also the principles of least action that we started with because if you can do it locally you're doing it very very efficiently and you know interestingly in the past five years i would imagine people in machine learning have been looking towards predictive coding as an alternative to backpropagation of errors and certainly establishing that in terms of the efficiency it is at least as good as if not more efficient than backpropagation there are issues about scaling it up to the enormous dimensionality of deep rl but from a mathematical perspective this is the right way to do it because it is just much more efficient it is just so much and it just relies upon uh local um you know local optimization well the beliefs in this context um are used in a sort of technical sense of bayesian beliefs um so just to try and um you know not make that vernacular but certainly um qualify the notion of belief updating in the sense of basically updating so beliefs are read in my world or in the world of active inference and the free energy principle as um conditional probability distributions so are there sufficient statistics so they're not sort of propositional beliefs not something you can talk about these are just sort of mathematical descriptions of a representation which is equipped with uncertainty so there's content and the confidence that you have about this so that's another bright line between the kind of networks that we have in our brain and indeed that are employed in active inference for example versus axiom where the nodes actually represent beliefs in terms of both the content and the uncertainty or the inverse uncertainty which would be which would be the precision as opposed to a neural network which just has a value it just has a cascade of functions that you can interpret as representing content, but it is not equipped with uncertainty.

24:05So once you've got one of these, I repeat, biomemetic kind of neuronal networks in play, of the kind that you might find in predictive coding, then you can talk about the changes in the values of the neural network in terms of belief updating and of course beliefs about what but beliefs about the causes of your content beliefs about the causes of what is observable um so now there's a there's a distinction between the latent causes the unobservable causes of observable data sensations images um you know whatever you know whatever is can actually be explicitly measured or observed so when we talk about belief updating, what we're talking about now is a base-optimal way of updating your beliefs about the causes, the unobservable causes of your sensations.

25:04Of course, we can only sense a very small part of the entire universe. So we have to, on the basis of some very sparse sampling, build beliefs and update those beliefs continually about the states of affairs out there beyond the brain-bound skull or indeed beyond my deep neural network. So inference just is a description of the process of Bayesian belief updating. And Bayesian belief updating is just mathematically something that can be described as minimizing your uncertainty, minimizing your prediction error, minimizing your free energy in the right kind of way that includes this measure of uncertainty.

25:49Another, when you move into machine learning, this Bayesian active inference, it's happening continually. You're working with probability distributions, not weights in a, at least in, I mean, you're working with versus. And at least in Versus's implementation, you're working with the parameters in the model are probability distributions, not weights of individual nodes or neurons that influence the outcome more or less. Am I off there? No, no, no, I don't know. You're spot on. Yeah. So just to sort of contrast. So a deep RL or a standard neural network, including things like transform architectures that underwrite large language models or most of generative AI, these are basically just universal function approximators.

26:58They just map through a series, a cascade of functions, values afforded by the input to generate some transformed values, which are the output. Those could be predictions of the next token. There could be some action. That is not how the brain works. It is not how active inference works. And it's not, as instantiated, say, in Axiom, which was used to demonstrate the fundamental increases in efficiency if you do it using belief updating. So the equivalent architecture is certainly in place. There is a deep structure to it. I mentioned sort of world models and degenerative models before. The structure of the network determines the structure of your model and the implicit conditional dependencies and all the contingencies in there.

27:57But as you say, each node now stands in for a probability distribution, a belief, and therefore what you're passing around, the influence of one node or another node, is determined by and driven by the optimization of your probabilistic beliefs or representation. And that's really important because, for example, if I don't have a representation of uncertainty, then I don't know what I don't know. And more importantly, I don't have any way of judging or evaluating the goodness of an action in terms of its ability to reduce my uncertainty, to reduce my prediction errors, to reduce my expected free energy.

28:56but if i've now got a representation at each and every level of a deep generative model a deep world model i can work out if i went and solicited or looked over there so for example you and i are extremely skilled at this you know every 250 milliseconds every quarter of a second we deploy our visual apparatus by looking over here foveating foveating over here. That's an incredibly skillful act, simply because we are choosing to look at the parts of the visual scene that will have the greatest information gain or resolve the greatest amount of uncertainty, given what we believe at the moment, in this personal Bayesian sense.

29:43So what we're talking about now is a mechanics of optimal smart data mining, knowing where to go and get the right kind of information that enables you to resolve your uncertainty so that you can now have a better model of the world in this probabilistic sense. And Axiom showcased that in the sense that it spoke to what could be construed as two of the main problems with the current tech on offer with generative AI, which is basically the inefficiency and the lack of reliability. so the inefficiency can be resolved by appeal to these first principal accounts as exemplified by things like predictive coding or other implementations of active inference because now you've got this ability to resolve uncertainty in the most efficient way in accord with these principles of least action you can now not only outperform deep RL on the benchmarks as you know i think there's a 60 improvement in performance uh demonstrated by axiom but more importantly you do it much more efficiently so um i think there's a you know using three percent of the compute and that translates into efficiency in many many different ways so not only is it informationally more efficient or statistically more efficient it also is It's thermodynamically more efficient, so you're using less power.

31:23And it's also sample efficient, so you need a fraction of the data. So this is, if you like, by appealing to these biomimetic principles, or from a physicist's point of view, principles of least action, that sort of underwrite the way that we deal with our world and make sense of our world, You can resolve the efficiency problems that are currently plaguing or preventing the right kind of enterprise update of the opportunity to be AI, for example. Where you'd expect it to be deployed, it's not being deployed. And of course, the reliability is, you know, you can't get hallucinations if you've got an explicit uncertainty quantification.

32:11if you're not sure, you're uncertain about a particular recommendation or a particular classification, then because you quantified it, then you inure yourself or you protect yourself against making overconfident predictions or classifications and the like. Build the future of multi-agent software with Agency. That's A-G-N-T-C-Y. Now an open source Linux Foundation project, Agency is building the Internet of Agents, a collaborative layer where AI agents can discover, connect, and work across any framework. All the pieces engineers need to deploy multi-agent systems now belong to everyone who builds on agency, including robust identity and access management that ensures every agent is authenticated and trusted before interacting.

33:19Agency also provides open standardized tools for agent discovery, seamless protocols for agent-to-agent communication, and modular components for scalable workflows. collaborate with developers from cisco dell technologies google cloud oracle red hat and more than 75 other supporting companies to build next generation ai infrastructure together agency is dropping code specs and services no strings attached visit agency.org to contribute that's A-G-N-T-C-Y dot O-R-G. I should have said at the beginning, Axiom, that you've mentioned a couple of times, is versus is the startup that you're working with, and Axiom is the model implementing the free energy principle and Bayesian active inference and all of that.

34:28Is that right? Yeah, yes. it's a sort of a minimal implementation just to you know just to demonstrate what you could achieve and the problems you can solve if you commit to you know to these biomimetic principles you know and there are many aspects that we could talk about you know there's one of them interesting which people sort of latch on to and indeed speaks to one of your other questions about sort of the architecture of the brain i mean the brain is not just you know a deep neural network it has a lot of factorial structure it has a lot of modularity some parts do sort of perception other parts do planning other parts do um memory um so all of these different aspects of uh inference and sense making and decision making all um imply and require a certain kind of structure so your axiom was built to show that if you put this kind of structure, agentic kind of structure, into a world model, into a generative model, and then you apply these variational principles of least action to the local updating, the local optimization, then you can get this kind of performance enhancement.

35:46And to my mind, probably the more important aspect is this efficiency. you could have guessed that from the mean that you and I can drive a car on 20 watts but if you have to train a large language you may need a nuclear power station this is to my mind this is the important part of the solutions to the current problems that are inherent with the current direction of travel with Jared and Alec. Yeah. And a couple of things that fascinated me is this is a dynamic model. It expands and contracts depending on the data that it's looking at. Is that right? So, and it learns continuously because the probability distributions are being refined, but they're not being overwritten.

36:49And that's a problem with neural networks that new data coming in can, that's why they can't achieve continuous learning because of catastrophic forgetting. And can you talk about that, the promise of continual learning in this kind of a system? Yes, I can. But you've already summarized the bullet points. So I'll just speak to those. No, absolutely. So we do not suffer from catastrophic learning. We will grow our models, our world models, in a way that is sufficient to account and optimally explain what's going on. And in growing them, we're now talking about something you implicitly implied, that the gerontine model in and of itself has an optimal size.

37:44For example, it could have, if you were just using a simple layered deep neural network in this instance, how many layers do I have? And in each layer, how many hidden units? And in my world, how would you factorize or modularize within each layer? So all these architectural aspects determine the structure of the model, which just is a specification of the cause-effect structure statistically in the world you're trying to model. so if you remember where we started with the free energy and the importance of that putting entropy into the objective function to implement sort of occam's uh razor what that means is there is an optimum complexity of the structure for any given set of content or world in which you are navigating predicting classifying actor acting in so you need to find the right sort of complexity of the structure, the right depth of the model, the right sort of factorial structure that is sufficiently complex to provide an accurate account, but not too complex.

38:52So this is exactly Einstein's, you know, keep everything as simple as possible, but no simpler. So there's a sweet spot. And in principle, and indeed in practice, one, and indeed Axiom actually demonstrated this under the hood though it was never it wasn't sold as one of the key issues but to my mind it was a really interesting aspect that it grew itself and just to the right level of complexity before if you're like opening up new slots or new um parts of the model for objects that it had never you know it had never seen so you've got this notion now that you can um increase or decrease the structural complexity of the model in exactly the right way so you don't overfit and this contrasts with deep rl because in deep rl you just start with something that's too complicated you start with billions of parameters and then fondly hope you can reduce the number of parameters say connection strengths or weights by some kind of say pruning or dropout or putting noise in and doing mini batching in a way that eludes those sharp minima that you get from the overfitting, simply because you've got too many parameters, too many connection weights in your model.

40:11So conventional approaches use this sort of top-down approach. They start off with an overly expressive model with too many parameters, 99.9 % of which will be redundant, and then try and find some engineering heuristics to eliminate the redundant, to reduce that complexity in a way that you get for free if you'd use the free energy. But to use the free energy, you have to encode the uncertainty. And of course, deep RL doesn't have that. It doesn't have the capacity to represent the uncertainty unless you're dealing with a variation autoencoder. So this is another motivation for using the free energy as your ultimate objective function because it now gives you a way of scoring not just the quality of your current beliefs in the moment or the parameters of your generative model the connection strengths as you assume a simulation accumulate data and build better and better models but also it scores the quality of the structure of your model and that speaks to the possibility of growing models uh so growing models from scratch just so you can get to the right level of complexity apt for the kind of content that you're that you're you're dealing with and this is you're not it is a fascinating and i think really important possibly vexed um problem uh at the moment which in the life sciences would come under the rubric of structure learning learning the right kind of structure automatically in the face of data by optimizing your model with respect to this free energy score that can also be well provides a bound in machine learning i should say this is the elbow this is the evidence lower bound in fact it's another bound because it's a negative in physics.

42:10And therefore, you can understand this sort of model growing and shrinking and adapting to new worlds and new contingencies as maximizing model evidence. Or if you treat your model as an agent, then the agent looks as if it is gathering evidence for its own model. And philosophers sometimes call this self-evidencing, a little twist on the philosophical, simply because you're trying to optimize the model evidence that is bound by the elbow or the variational free energy so this notion of you know gathering evidence can be played right through not just to beliefs in the moment beliefs about states of affairs in the world or hidden states but also the parameters of your model that would be optimized through learning but also the very structure.

43:07In statistics, we call that Bayesian model selection. So it's selecting the model, the structure, that has the highest evidence in the face of the content or the data which it's trying to explain. And if you pursue that notion, what you've now got is a mathematical image of natural selection. So you can now look at natural selection as nature's way of doing Bayesian model selection where you and I are the hypotheses and the structures that provide evidence that this is the kind of thing that can live in this eco-niche. So again, we come back to biological principles. Nature's already done that.