969: The Laws of Thought: The Math of Minds and Machines, with Prof. Tom Griffiths

22:29Yeah. So in fact, what's happened is that, you know, people who work with these models have noticed that they have this kind of deficiency. And so if you ask a modern model to do something like this, so in our paper, we focused on what we call deterministic problems, problems where there's exactly one answer based on the question that you ask. And we show that for those deterministic problems, the models still produce a probabilistic answer, right? They're influenced by the expectations they have about the distribution of language. So if you're asking them for something which is a low probability answer, they're less likely to produce it.

23:00And so that characteristic still holds for modern models. We can show that they're still sensitive to the probability of the output. It's been a little bit attenuated, but if you actually interact with a model through the chat interface or something like that, and it does all its thinking and does all these other things, one of the things that it'll tend to do if you give it one of these deterministic problems is write a piece of code to check its answer. So it has been trained that for certain kinds of problems, it shouldn't rely on its sort of probabilistic backbone. It should do something which is appropriately deterministic in order to then give you the answer back.

23:37But that is also something that I think parallels something that we see in the history of cognitive science. So when people were working with rule-based models in good old-fashioned AI, they realized that they needed to have some kind of structure around this in order to make these models be able to do the kinds of things they wanted them to do. So this was building what were called expert systems in the 1980s based on production systems with these sort of rule-based models. So these had sort of big lists of if-then rules. If this happens, then do this. If this happens, then change your goal to this.

24:10If this happens, if your goal is this, then do this. And so it had all of these instructions that they would follow, and they realized they needed to have some kind of architecture around that, that would then allow this system to make good decisions by using all of those rules together. And you can see that being echoed in what's going on with modern language agents, where a language agent is really building that same kind of cognitive architecture around the core technology, which is the base language model.

24:38Jon Krohn:Really fascinating. I love how you tied it there to the historical expert systems. something else in your book that straddles both human and machine cognition that we found fascinating in our research was in your chapter probability is the new logic you showed how dramatically conclusions can change based on prior probabilities and you used an orangutan example that was easy to follow do you want to tell us about that sure i mean uh let me let me maybe start one step back right so uh one of the things you mentioned was that people started to realize that logic wasn't going to do all of the things that they wanted to be able to do.

25:14And so part of that is that when you think about the kinds of problems that human minds have to solve, you can divide them up into two kinds. One kind is what we could call deductive problems, right? Where there are problems where you have enough information given to you that you're able to work out exactly what the answer is, right? You have all the information you need in order to solve the problem, right? So if I give you a math problem, like, you know, two plus three, you've got all the information that you need in order to work out what the solution is. Or even if I give you a problem with an unknown in it, so like X plus two equals five, pretty easy for you to work out what X is there.

25:54And that's an example of a deductive problem. And so those symbolic rule-based systems were really good at solving deductive problems. You can think about that as you have algebra, algebra, logic, chess, you know, these were sort of things that demonstrated the power of these kinds of systems. Where they didn't do so well was another class of problems, which are what we could call inductive problems. And these are problems where you get some information, but it's not enough to know what the answer is with certainty, right? And it turns out a lot of what humans do is induction, where, you know, if it's perceiving the world, right?

26:33Photon hits your retina. What happened in the external world? You've got some data that came in, but you don't have enough information to know exactly what it was that, you know, is the three-dimensional structure in the world that provided that information to you through your retina. If you're hearing somebody say something, you're hearing some noisy signal, you're forming an interpretation of it. You're trying to think about what someone was thinking based on the actions that they took. You have some data in the form of their behavior, or you're making guesses about what their mental states are.

27:04If you're even something like learning a language, you might have some hypotheses about the structure of a language. Your data are the things that you've heard in that language, and you're trying to use the data to constrain those hypotheses. And so all of those are inductive problems. And that's kind of where that logic, you know, symbolic rules approach at the wall is trying to explain these different important aspects of human cognition. And so probability theory is the tool that we can use for, you know, for solving these inductive problems, because it gives us a way of talking about how data should influence our beliefs, even when we don't have enough information to be certain.

27:45And it does that by essentially allowing us to assign degrees of belief to different possible worlds that we might be in. And then as we get data, those data help us to narrow down those possible worlds in a way that allows us to update our beliefs, reallocate those probabilities across those possible worlds, and be able to draw some kind of conclusions, even if we're not able to reach certainty. And so the example I give in the book is sort of a, it's from a old detective story. I don't know if this is the story, the murders in the room org, right, which is one of the first detective stories.

28:27I don't want to give any spoilers, but I guess I have to. So if you haven't read the story and you want to read it, you should skip ahead a little bit. But the key thing that happens is that the detective, Monsieur Dupin, figures out that the culprit is an orangutan, right? And does so based on some clues like somebody heard a voice, but they didn't recognize the language. there was some tufts of orange hair that were found, and then that the room had to be accessed via a window that was high up, right? That would be hard to climb to. And so none of those facts gives you certainty that the culprit is an orangutan, right?

29:16No detective is necessarily gonna go to a crime scene and see those things and say immediately, oh, orangutans again. But each of them gives you a little bit of evidence for that conclusion. And then probability theory is a tool that we can use for saying how we should integrate those pieces of evidence in order to update our beliefs.

29:34Jon Krohn:Right. And so basically some of the things there that would be very unlikely for a human to be able to do, like being able to get into that window, to be able to get into the room while they also have red hair. So kind of all of these probabilities multiplied together,

29:50Jon Krohn:the most likely conclusion you can draw is that it's an orangutan. Yeah. And the way that we express this in probability theory is via what's called Bayes' rule, which is named after the 18th century minister, Thomas Bayes, who kind of first made the suggestion that the math of probability theory is the way of describing how it is that we should go about changing our beliefs. And Bayes' rule says, you start out believing something to some extent, right? And we'll sort of call that the prior probability of that hypothesis. And then in order to work out how much you should believe it after seeing some data, what's called the posterior probability, you take that prior probability, multiply it by how likely those data would be if that hypothesis were true, and then do that for all of your hypotheses, and then just sort of divide them by the sum of those quantities just so that you end up with something that ends up being a probability distribution.

30:33But the key idea is take the degree of belief you had before and then multiply by the probability of the data if that hypothesis were true. So if it's something that a human would find very hard to do, it's very unlikely that a human could have reached that window, but it's very likely that an orangutan could, then that's going to increase your belief in it being an orangutan, even if your prior probability of orangutans was very, very small.

30:53Jon Krohn:Now that kind of taking advantage of prior probabilities, it seems like the way that we train our large language models today, whether it's the pre-training supervised learning or the post-training reinforcement learning, it doesn't seem like we're taking advantage of priors in any way. Yeah. And in many ways, that's a consequence of a lesson that people learned in machine learning that I think is potentially going to come back and bite us soon, right? So the first point here is that this is exactly right in terms of if we want to characterize why it is that human children can learn language in five years, right?

31:30And our AI system requires more like 5 ,000 or 50 ,000 years of language input to reach the same state. That difference comes down to priors, right? To having different expectations about what the structure of language are like. So inside our human heads, there's a whole bunch of stuff that makes it easier for us to learn the kinds of languages that we end up speaking, and that stuff is not inside our neural networks. So there's a sense in which that's by design, right? Which is that prior to about 2012, the dominant approach in machine learning was one where a human would engineer, say, features that went into your machine learning system, And then you would have a machine learning system that operated on top of that.

32:16And you can think about those features that were being engineered as a source of priors, a source of inductive bias that was sort of setting up the kinds of solutions that the model could find. And that was really because we didn't have a lot of data. And so you needed to have, you know, engineer in those biases in order to make the system work. And then in 2012, I sort of tied things to that point because that's when the AlexNet, this sort of convolutional neural network won the ImageNet challenge, right? And that was a model which was a very large, complicated neural network, although small by modern standards, that was trained on a very large amount of data, although again, relatively small by modern standards, right?

33:02But here, the insight was that you didn't need to hand engineer the things that were going in. You didn't need to hand engineer those inductive biases. If you had enough data, you could learn to solve this problem in an end-to-end way. The neural network could actually find a better solution than the human engineer could if you had enough data in order to get there. And so since then, machine learning has been operating in this paradigm of very weak priors, very weak inductive biases. The only kinds of biases you put in the system are through the choice of architecture you make, right? Maybe use a transformer or a convolutional neural network or something like this.

33:36And then you just throw enough data at it that it's able to end up solving the problem. And so that paradigm is very different from the human one, right? You can kind of think about the machine learning engineer's problem is I'm going to start out here. I need to get here. So I need to get enough data to get me from here to here. The cognitive scientist problem is I know people start, you know, learn from this much data. So I have to figure out what all of the things are that are filling up the rest of that gap, right? And we can actually quantify that. So if it takes you 5 ,000 years of speech to train your large language model and the human child lends it from five years of speech, then there's some prior distribution, some information, some inductive bias that accounts for 4 ,995 years of speech.

34:19That's a lot of data that has to be encapsulated there. And so part of what we do in my lab is try and figure out what those biases are on the human side and try and develop technologies that can help us to make neural networks that express those. So one of the techniques we use is something called meta learning, which is a way of learning the initial weights that you put in your neural network in a way that isn't a hard bias like that previous machine learning approach of constraining the kinds of features that the model would get. it's a soft bias because it says, I'm going to start you out in a particular, you know, space, particular part of the space of solutions.

34:56And if I can start you out close to the solution you need to find, you're going to find it easier to then discover that particular solution you want. And so that meta learning technique tries to find good initial weights for a neural network by creating neural networks that are able to solve a bunch of learning problems where they're learning from much less data for each of those problems.

35:14Jon Krohn:Fascinating. Yeah. You probably have a perspective on where and when this dramatic shift might happen. But it's obvious to me that the current approach that you described there of say, taking 5 ,000 or 50 ,000 years worth of training data for an LLM to be able to have the capabilities of a five-year-old human, there is this huge gap. And it sounds like something like meta-learning or the other things that you're working on in your lab could be the solution. And I guess it could kind of sweep us out of nowhere, just as the transformer architecture did and is now ubiquitous. And so it'll be interesting to see where that goes and the part that you play in it.

35:54Jon Krohn:Actually, my next question that I had already pre-planned follows really nicely from this organic discussion that we've been having now for the past five or 10 minutes. And so in your research, you frame human intelligence as rational adaptation under tight limits. So explaining biases, heuristics and few-shot learning, so learning from a few examples, which is what LLMs don't do, as optimal use of limited time, limited computational resources, and limited bandwidth for communication. So it seems like that, I didn't even really get to a question there, but the question that I was going to have, you kind of already started now getting into, but maybe with that context, you can develop on your answer even further.

36:36Jon Krohn:The question is, how should those constraints guide the design of the AI systems that we now build? I think one thing is it's not necessarily the case that they have to. So I would think about those constraints as a good way of characterizing what the difference is between human minds and our AI systems. right so human minds solve a set of computational problems that are characterized by these constraints um we live only you know a few decades you know give or take um we uh have to do all the things we're going to do using just a couple pounds of you know neurons right and we can only communicate with one another by you know making weird honking noises or wiggling our fingers on a keyboard, right?

37:21Like those are the kinds of constraints that humans operate under, right? And so as a consequence, we have to be able to learn from small amounts of data because we're just not going to live long enough to see lots of data, right? It's not like AlphaGo, which is trained on many human lifetimes of Go games, right? You just can't do that. Human Go players are going to be limited by what it's possible to learn in a human lifetime. You have to be able to recognize the structure in problems so you can use previous solutions that you found or come up with good sort of efficient strategies for solving those problems using the limited computational resources you have.

38:03Right. So like unlike Deep Blue, a human chess player can't search 100 ,000 positions per second. They can search much less than that and they have to focus on finding the good ones to search. and you have to you can't overcome those constraints by you know being able to transfer the data that we've experienced or pool our computational resources directly because we have to operate under those bandwidth limits and so we have to come up with good conventions for allowing us to do things like work together and to pool data things like writing you know language in the first place but writing you know science starting companies right these are all mechanisms that we can have for being able to coordinate the resources that are available to us.

38:49And so that set of things really characterizes in some sense what it is to be a human as an intelligent system. And our AI systems are not subject to any of those constraints, right? They can learn from many human lifetimes. They can add more compute when more compute is needed. They can transfer state or share the data that they train from. All of those things are much more straightforward for your AI systems. So one consequence of that is that you might expect that the AI systems are just going to be a different kind of intelligence from human intelligence, right? Because they're not operating under the same constraints.

39:22They're not solving the same computational problems. They're going to be able to solve problems potentially really well. And in some cases, they're going to solve those problems better than humans, but they're not necessarily going to solve them in ways that are recognizable to humans or ways that are the same as the solutions that humans find. And so that sets up sort of two possibilities. One possibility is we say, that's fine. We just want to engineer the best systems we can. We should just keep on doing what we're doing, keep going in that direction. And we're going to get this kind of divergence between what human intelligence is like and what AI is like, and that's okay.

39:56The other approach you could think about is one of saying, oh, well, actually we want some of those attributes of human intelligence to be inside our AI systems. In particular, it might help us make AI systems that just make more sense to us, right? In the kinds of solutions that they find. And so if we want to do that, then we really need to understand the things that are going on on the human side and think about how it is that we transfer those over. And so we talked about meta learning as a trick for building systems that have inductive biases that are maybe more like humans. That's certainly an avenue that you can go down if you want to do that.

40:27But engineering inductive bias is hard. Scaling is much easier than engineering inductive bias. And that's the reason why we're in the current sort of moment that we're in an AI is that that's a technique that sort of works and it's an engineering problem. We know how to solve it. But engineering inductive bias really requires having an understanding of the problems that we're solving and what reasonable inductive biases for those problems are. And it's something that people are still trying to figure out how to do well.

40:55Jon Krohn:Yeah, let's talk about this inductive bias thing more, because I think this is a really important point. And it's something that will probably be obvious to a lot of our listeners as we explain it a bit more. When we think about the LLMs that we have today, they can't, as far as I know, maybe you can correct me on this, but I think they are quite limited in the extent to which they can induce some kind of conclusion beyond something that is in their training data. So that is quite unlike human cognition, where we don't just depend on the information given. We can use inductive biases, the priors we were talking about earlier, assumptions, to make some kind of guess as to what's happening in situations that are quite different from any that we have ever encountered in our life experience before.

41:47Jon Krohn:And as far as I know, any of the AI systems that we have today really struggle in that kind of circumstance. I think in some ways, I would say this is a little more of an open question, right? To what extent this is possible. The things I can say about it are there are certain kinds of generalization that we know large language models can do. So some of my colleagues here at Princeton have a nice demonstration of this where you can sort of get them to compose ideas together, right? So you can sort of give them a set of properties and then they can produce something new, which is the consequence of combining those things together.

42:21And it's certainly not something that ever existed because those things hadn't been combined together before.

42:25Jon Krohn:For sure. That is something that kind of that blending of content and style is very impressive. So something like, you know, give me, you know, explain this Shakespeare sonnet in the style of Snoop Dogg or something like that, which for sure you end up with an output that is unlike anything we've ever seen before. Well, I mean, that's sort of challenging this idea that they can't do something that's not in their training data. Right. So I think that's not strictly true. They're able to do this kind of, we call it compositional generalization, right? They're able to compose things that they've seen and make new things and they can do that up to some depth, right?

42:57I think the interesting question is, are they able to kind of like produce things that are genuinely novel in the sense of maybe not even being recognizably composed out of things that you've seen before, right? Like come up with a completely new artistic style rather than something which is a blend of styles in a way that's novel or something like that. And I think that's a question where it's still much more of an open question. We in my lab have looked at some questions related to this. So I have a postdoc, LLU, who's been looking at some related things, looking at creativity in these models.

43:42And I can tell you sort of hot off the presses some of our results there. But one of the things that she's found is that if we ask LLMs to come up with sort of novel product ideas, they actually do slightly better than people in terms of producing ideas that people rate as novel. And interestingly, a strategy that works for people, which is asking them to sort of do this by forming an analogy, right? So you say, you know, come up with a new idea for a toaster where it's using principles that come from, how octopus suckers work or something. That's a good way of making humans extra creative, but it turns out not to work for large language models.

44:22And one reason why that might be the case is that the language models are already operating with such a broad set of concepts that they don't need that extra external stimulation in order to come in. And so there are ways that it seems like they can do some kinds of creativity. I think the thing that we haven't seen is something where they're sort of like genuinely coming up with something that we can recognize as a sort of breakthrough. One of my favorite, you know, thing to do whenever a new model comes out is like ask it to come up with ideas for scientific, you know, studies that we could do.

44:53And, you know, sometimes they get a little bit interesting, but I think it's also worth remembering that because the models are trying to produce things that have high probability, right? They're trying to produce high probability sequences of tokens. They're going to be pretty conservative in the things that they do. So they're not going to suggest something that's completely wacky Because they've not been trained to do that. They've been trained to produce things that are like with high probability based on the things that they've seen before. And so if you think about an important part of creativity is just like being completely out of the box, that completely out of the box behavior is something which is maybe inconsistent with the way in which they're trained.

45:28Jon Krohn:Right. We're back to that higher probability of the answer of 30 as opposed to 29. And it reminds me of a friend of mine who's one of the bestselling authors of our time. I won't name him because this is kind of getting into a behind-the-scenes conversation with him. But he was interested in, he is interested, I suppose, in using LLMs to unearth rare information in his field. And so, you know, kind of the stuff that's common has already been in his books or other, you know, best-selling authors in his field. and so he wants to be able to dig up rare he's and and it's very hard to get you know you can kind of one way of kind of getting there is by just asking for more and more and more and more and more and more examples and then eventually maybe some of the examples start to become relatively rare but it's very hard to get that kind of output and that's right that's what we would expect because it sort of pushes against the way in which they're trained one trick that we use for actually getting this to work is um they they because what they're doing is calculating a conditional probability right so they're they're taking the information they've seen before they're conditioning on that information right doing their belief updating about what the the things that would come next would be um you can push them into lower probability parts of the space by doing something where you're creating prompts compositionally right so it's much less likely to sort of come up with something spontaneously, that's an unusual thing.

46:56But if you write a little bit of code that sort of procedurally generates prompts, where you're putting in, you know, an unlikely combination of things, then you say, okay, tell me about this, find an example of this. And then it will then be able to say, oh, yes, a good example of this is this, right? And so you can actually use it to search the space of improbable by things by constructing prompts that are pushing it out into the improbable corners of that space.

47:21Jon Krohn:That is interesting. Good tip there. For a lot of us in ways that we're trying to engineer LLMs to do these kinds of rare outplays that we might want for whatever reason, including for creative reasons. I'm going to get into curiosity in a moment because that's a fascinating topic. But just before we get there, I want to draw the line under something you were talking about maybe 10 minutes ago. And we've kind of gone off on other parts of conversation. but something that you talked about, which is something that's so obvious to me now that you've said it out loud. And I guess in some ways, I've always kind of known this to be true, but I've never perceived it so clearly as when you were talking about it is this idea that it's kind of silly for us to be so focused.

48:07Jon Krohn:Many of us, you know, me, I come from a biological neuroscience background and then that, you know, over the course of my PhD, it became more and more AI focused. And so myself, I think a lot of people look for biological inspiration, particularly in human mind for how we can be designing AI systems. But because the constraints are so different for an AI system versus a biological system in terms of the example you gave perfectly there with Deep Blue of being able to think about hundreds of thousands of possible next moves in under a second would certainly be possible with today's systems. that's obviously something that a human can't do.

48:47Jon Krohn:And so I guess it probably makes sense. We might as well look to human intelligence or biological intelligence for inspiration in some ways, but we've got to remember that we, yeah, have completely different kinds of constraints in the system that we're applying it to. And so what ends up working really well, you know, it could be something like scaling. You know, it's like, it is, it's mind blowing to me. What scaling has been able to achieve, taking this transformer architecture, scaling it up with more data, more compute, And it's when I finished my PhD in 2012, if somebody had asked me if we could have AI systems with the kinds of capabilities that we have today in 2026, I would have said, that sounds like science fiction to me.

49:28Jon Krohn:I doubt it. Yeah. No, I think a lot of people would have agreed with you. It's very easy. This is one of the challenges of the field of AI is that people pretty rapidly adapt to normalizing whatever the last big advance was. So, you know, some AI practitioners sort of say, oh, the definition of AI is whatever the thing is that we can't do yet. So you keep moving the compost. Exactly. So, you know, I think this sets up an interesting challenge, right? So, you know, you have these two paths. You can keep building systems that are different from us and leaning into the ways in which, you know, we can make use of all of the properties of computers.

50:12Or you can try and, you know, build systems that are maybe more like us and sort of think about how it is that we navigate the constraints that we navigate. So I think you could think about this as being a little like, as a sort of classic analogy that's made in AI, right? If you think about like, you want to build a plane, then going and looking at how birds fly is useful. but there's going to be some point where you're going to say, okay, now we've got the basic principles and now what matters more is building like faster jet engines, right? And you're not going to learn how to build faster jet engines by looking at how birds fly, right?

50:50And so that's that route of saying, okay, we've taken inspiration from biological intelligence. We kind of figured out what the basic principles are and now let's build this thing. I think the way that I think about that is that one of the most fundamental similarities between what birds are doing and what planes are doing is that they're dealing with the same laws of nature, right? That there's a set of principles of aerodynamics, which are shaping what's going on in the bird and are going to shape what's going on in your plane. And understanding those principles of aerodynamics is something which is fundamental if you want to build better airplanes.

51:24And so that's the way that I think about what we do as cognitive scientists and how it's useful for thinking about AI is that if we're able to figure out what are the broader principles that are relevant to characterizing intelligence? What are the laws of thought? Those laws of thought are the things that we're going to expect to be in parallel between natural intelligence and the artificial intelligences that we're building. And having expressed things in those mathematical terms gives us the opportunity to say, oh, here's where an insight can transfer in one direction or the other. But there are different levels at which those insights can transfer, right?

51:59That can be the most abstract thing of, you know, what are the problems we have to solve and what's the right way of solving those problems, where things like logic and probability theory and Bayes rule show up. Or it can be at the level of, oh, how do we actually build a system that produces that intelligent behavior? And that's where things like artificial neural networks and so on are useful. But I think it gives us the framework that we need in order to then work out, yeah, how do we have this conversation turn into a productive one that can go in both directions? And how do we recognize when it's not useful to necessarily use those biological principles?

52:33Jon Krohn:That's a great framework for trying to understand when we should be using biological inspiration in our AI systems. Related to that, so curiosity. One of, if not the most famous industrialists of our time, created his now near the frontier AI lab through this idea of maximizing curiosity. And in a paper from 2020, you model curiosity as rational information seeking. In your book, using Herbert Simon's parable of the ant on the beach, you suggest that much of intelligent behavior's complexity comes from the environmental complexity rather than the agent's internal sophistication. So I've talked about a bunch of different things there, but the kind of the common thread is that some people think that curiosity and modeling that in AI systems could be a key driver of getting to big breakthroughs in AI systems.

53:36Jon Krohn:And you've obviously spent a lot of time thinking about curiosity and what it means. And so I'd love to just have your insights on that. Yeah. So my work on curiosity, and this was with my former graduate student, Ratchet and Dubai was looking at how we can think about what the computational problem is that curiosity solves. And the way that we set that up is essentially what you're doing when you're being curious about things is you're gathering information, which is potentially going to increase the value of the actions that you're able to take in the future. So this turns out to be a reasonably good model of human curiosity is that our curiosity gets captured by those things where if you think about it, it's not necessarily something that, you know, like you're not necessarily just like getting distracted by things that you've never seen before because you constantly are seeing things that you've never seen before.

54:29Right. And you're not, you know, getting distracted by things that you've seen a lot because, you know, like you've seen those things a lot and they're no longer interesting. What you, what you're sort curious about is often things that you've seen a few times and you think, oh, maybe I should be paying attention to that. You can explain why you have this inverted U-shaped curve of curiosity from this perspective of you're trying to find the things which are going to maximize the value of information for the actions you take in the future because something that you never see, you're also never going to see again.

55:04If you've never seen it before, you probably will never see it again. So it's not really worth learning anything about it because this might just be a one-off. And something that you've seen a lot, you've already had lots of experience with. And so you've already learned how to deal with that thing. But if you see something a few times, then that's an indication that that thing is going to occur again in the future. And because you've only seen it a few times, it's probably not something you know a lot about. And so for that reason, it's something that should capture your curiosity. And we show that this actually gives us a pretty good model of how it is that people actually choose what they're going to engage with when they're learning about new things.

55:36In the AI setting, there are people who have explored curiosity as a way of defining an objective function for an AI system, where traditionally you might think about building an AI system by saying, here are some rewards that you can get. You get a reward when you solve a problem, and you get a reward when you navigate the maze, or you get a reward when you get the football in the net, or whatever it is that you're trying to do. And there's an alternative way of doing that, where instead of making the rewards dependent on things that happen in the external world, you make the rewards dependent on things that happen internally.

56:05So you get a reward for learning something about a thing that you may not have seen, that you've seen a few times. You build in a reward function, which aligns with that relationship where it's like, you're sort of decreasing the entropy of your beliefs, the sort of decreasing the uncertainty that you have about things in proportion to how often you you see those things and then it produces that same kind of inverted U-shape. And so that's a reasonable way of thinking about creating an AI system that's going to go out and explore on its own, because it's going to be driven by trying to identify those pieces of information that are likely for it to be using in the future.

56:47And so I think that's the reason why you see that same concept manifest across these different places in psychology and computer science. Really interesting.

56:56Jon Krohn:I'd love to be in your lab doing the research that you're doing. This is such cool stuff. Getting near to the end of the technical questions that I have for you, I think this is the penultimate topic area that I have for you. And so this is around evaluating systems. So this is a bit of a long question. But as you leave the Princeton Lab for AI, one of the concerns is to ensure these systems are aligned with human values and that we have robust collaborative systems that understand the nuances of human judgment. You previously argued that LLMs and humans show the same core interpretability challenges, so opaque training data, inaccessible mechanisms, complex internal structure.

57:43Jon Krohn:And so LLMs and humans alike, our internal life is best studied through behavior. I mean, that goes back to right at the beginning of this episode talking about the way that you evaluate intelligence through surveys and that kind of thing in humans. However, you caution against anthropomorphizing AI systems because they're likely to be alien in various ways, as we've talked about in this episode, of things like different kinds of constraints mean that they're going to behave in different ways than us. And so how can we think about designing our benchmarks for evaluating AI systems so that we have some better sense that we're actually getting an understanding of what's going on as opposed to pattern matching?

58:30Jon Krohn:So there was a really long question, but you're nodding your head, so it seems like you follow the thread through. The first thing I'd say is I'm not a big fan of benchmarks, right? Benchmarks are an engineering device rather than a scientific device. If you're making a benchmark for an AI system, normally what that means is you're saying, here's some capacity that I want my AI system to have, and I'm going to make a set of tasks that I can use to measure whether the system has that capacity. Then I'll be able to say, okay, it got 80 % of those tasks right, so we'll say it's scored 80%, and that's how well it's doing in that particular capacity.

59:07um that's a little different from the way that we approach studying llms in my lab where we're more interested in what are the targeted experiments that we can do that are going to give us information about you know what's going on inside them what the kinds of mistakes that they make are you know sort of deeper understanding of that system and so that's less about you know making a big set of tasks and you know seeing what percentage of them they get right and it's more about designing a set of problems where the particular problems that they get right or wrong are then diagnostic of something which is going on inside that system.

59:45And so that's the difference between, like I said, like sort of approaching this as an engineer and approaching it as a scientist. Really, we use a lot of the methods of cognitive science and psychology to study these systems where we're conducting experiments that are designed to elucidate what the principles are that are working under the hood.

1:00:01Jon Krohn:Right. Well, great. From human cognition to understanding AI systems to benchmarking them. This has been a fascinating episode. Much of what we talked about in today's episode is from your brand new book, The Laws of Thought, which at the time of us recording isn't even out yet, but at the time of publication, this episode's publication, it should be live. It should be available for order in most markets. And based on how well your previous book, Algorithms to Live By, did, you can expect that this is going to be a mega bestseller just like that one was. I don't have insights into exactly how many copies it sold, but it has over 6 ,000 reviews on Amazon.

1:00:40Jon Krohn:So you can expect orders of magnitude more in sales because of that kind of figure, which is just wild for, you know, in our field, for these relatively technical books to be selling like that. I mean, that is really remarkable, Tom. Congratulations on that. And so I just also wanted to talk a tiny little bit about your book, Algorithms to Live By here at the end of the episode. From your early days as an avid fencer to co-authoring that book, you have long applied computer science to the messiness of real life. Your book, Algorithms to Live By, reframes everyday human procedures as computational strategies.

1:01:16Jon Krohn:So optimal stopping, explore-exploit trade-offs, computational kindness. And growing up, you described fencing as involving interesting computational problems, even trying to break complex moves into sequences. So how did those early problem-solving experiences shape the way you now think about cognition and algorithms? For me, it's very natural to think about the world in computational terms. So the other formative experience I had growing up was playing a lot of tabletop role-playing games. And you can think about what you're doing in a tabletop role-playing game is essentially defining a system which is supposed to somehow approximate real life, right?

1:01:55Right. So, you know, it has all of these things that you're putting in there about what the chances are succeeding at taking various actions and, you know, how you get better at doing things and so on. So you can actually think about, you know, your role playing game is encapsulating some kind of psychological theory of cognitive, you know, cognitive progress as you're learning new things and so on. And so I think the thing that we did in Algorithms to Live By was really saying that those computational principles have much broader applicability than people might expect. Right. So I think we had two kinds of readers for that book.

1:02:34part of what accounted for it selling relatively well was that we had people who we originally wrote the book for which is people who are used to thinking in computational terms and us telling them oh here's how you can apply this to everyday life and then we had another audience of readers who kind of understood how everyday life worked but wanted to learn a little bit more about computer science and so i think it was really the the synthesis of those two kinds of things that um that really you know caught people and you know coming at it from these these different perspectives And the intersection of psychology and computer science that that encapsulates is very much the same space as the current book, but it's sort of coming at it in a different way, where it's the same thing of trying to think about how do we formalize something, but you can think about in algorithms to live by a little bit more, we were trying to formalize something about the external world.

1:03:28And the new book is about how do we formalize our internal world?

1:03:32Jon Krohn:Right. the big question that has been the focus of your research this whole time. Really fascinating. So again, algorithms to live by, of course, the mega bestseller that you can grab. But most of today's interview was getting into these internal processes of both humans and machines. And so if today's conversation was fascinating for you, you'll want to grab Tom Griffith's brand new book, The Laws of Thought, The Quest for a Mathematical Theory of the Mind. Thank you so much for taking all this time with us today. Before I let you go, I asked all of my guests the same last two questions. And with a book author like you, this second last question seems almost silly, but do you have any book recommendations for us other than your own books?

1:04:19Yeah. So what I would recommend is my co-author on Algorithms to the By, Brian Christian, wrote a wonderful book called The Alignment Problem, which touches on other things that we mentioned here about how it is that you create AI systems that are aligned with people in terms of the values that they have and the actions that they're going to take. And I think it's a really good introduction to both some of the principles behind modern AI and some of the challenges that we face in deploying it.

1:04:42Jon Krohn:Perfect. Thanks for that recommendation. And for folks who love this conversation today, in addition to buying the laws of thought, how can they be following you for your thoughts in real time in social media and that kind of thing? I'm not very good at social media. Well, this podcast will be a rare opportunity for folks to get in behind your brain. I have a LinkedIn page and you can find my lab on Twitter and Blue Sky. It's the Computational Cognitive Science Lab. It's cocosci underscore lab on Twitter and Blue Sky. Nice. We'll have a link to those all in the show notes. Of course, Professor Tom Griffith, thank you so much for taking the time out of your, no doubt, very busy schedule to record this fascinating episode with us.

1:05:31Jon Krohn:And yeah, when you release your next book, we'd love to have you on again. Thanks for having me. This was a great conversation.

1:05:40Jon Krohn:Exceptional episode today with Professor Tom Griffiths. In it, the Princeton professor covered how the laws of thought are mathematical principles for describing our internal mental world, how psychologists tend to view humans as irrational decision makers because they focus on our biases and mistakes, while computer scientists find us impressive because we accomplish remarkable feats with extremely limited computational resources. He talked about how LLMs are trained to predict high probability token sequences, which means they're more likely to output 30 than 29, say, in a counting task. How a human child learns language from roughly 5 years of speech input while LLMs require the equivalent of thousands of years, a gap representing the inductive biases and priors built into human cognition.

1:06:23Jon Krohn:and how just as airplane engineers took inspiration from birds, but eventually focused on jet engines rather than feathers, AI researchers should understand when biological principles transfer usefully and when machine-specific approaches like scaling offer a better path forward. As always, you can get all the show notes, including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Tom's social media profiles, as well as my own, at superdatascience.com slash 969. Thanks to everyone on the Super Data Science Podcast team, our podcast manager Sonja Bredovic, media editor Mario Pombo, partnerships manager Natalie Zajski, researcher Serge Massis, writer Dr.

1:07:03Jon Krohn:Zara Karche, and our founder, of course, Kirill Aromenko. Thanks to all of them for producing another stellar episode for us today. For enabling that stellar team to create this free podcast for you, we're deeply grateful to our sponsors. You can support the show by checking out our sponsors' links, which are in the show notes. And if you'd ever like to sponsor the show yourself, you can find out how at johnkrone.com slash podcast. Otherwise, help us out by sharing this episode with someone who'd love to learn from it, review the show on your favorite podcasting app or on YouTube, subscribe, obviously, if you're not already a subscriber, but most importantly, just keep on tuning in.

1:07:39Jon Krohn:I'm so grateful to have you listening. And I hope I can continue to make episodes you love for years and years to come. Till next time, keep on rocking it out there. And I'm looking forward to enjoying another round of the Super Data Science Podcast with you very soon. Thank you.