1. Podcasts
  2. Machine Learning Street Talk (MLST)
  3. What It Would Actually Take for a Machine to Have Agency (Michael Tomasello)
Machine Learning Street Talk (MLST)

19 Nov 2026 · 1h 7m · 23 chapters

Episode artwork

What It Would Actually Take for a Machine to Have Agency (Michael Tomasello)

In short

What it would actually take for a machine to have agency, using Michael Tomasello’s “control-system” framework and a four-level taxonomy of agency, culminating in uniquely human shared agency and culture.

Guest backgrounds

Michael Tomasello is a developmental psychologist by training, long associated with comparative primate research (Emory’s Yerkes Primate Center; 20 years at Max Planck in Germany; later Duke University). He studies differences between children and great apes using shared experimental setups.

Key claims

  • Agency is for handling ambiguity in unpredictable environments, especially other agents.
  • A Turing test is insufficient for agency.
  • Agency requires goal-directed control plus executive processes; stronger agency adds metacognitive monitoring.
  • Learning is important but not the essence; organization/control is.
  • Humans differ via shared agency: joint goals, role coordination, joint attention, and perspective-taking; cultures then institutionalize conventions.

Notable examples

  • Thermostat and traffic light as minimal control architectures.
  • Robotic vacuum with perception and goal.
  • Squirrels planning/hypothesis testing (e.g., obstacle-course behavior).
  • Squirrels outsmarting humans with bird feeders.
  • Chimp vs child collaboration: chimps stop when they get reward; children keep coordinating.
  • Children invent gestures when deprived of speech, then conventionalize them.

Written by AI. May contain mistakes.

Chapters

1:00

Understanding Decision-Making in Unpredictable Environments

Explore the need for decision-making agents in unpredictable environments.

“The living things are made of the same basic elements as non-living things.”
1:56

Human Differences from Other Primates

Learn about the cognitive and social differences between humans and primates.

“I'm by training a developmental psychologist, but I've always been interested in evolution.”
3:05

The Nature of Agency and Evolution

Understand agency as a lens for adaptive behavior in unpredictable environments.

“So I think the TLDR of this book is that agency is an extremely important lens to understand adaptive behavior, not only in humans, but just across a spectra of life.”
4:25

Defining Agency and Control Mechanisms

Delve into the complexities of defining agency and its control systems.

“So prey, predators, but also conspecific competitors, conspecific cooperators.”
6:07

Hierarchies of Agency in Organisms

Examine different levels of agency and their implications in organisms.

“But I use a control system like a thermostat as a concrete model, because I know a lot of the discussions of free will and all that kind of stuff, they get a little far away from psychology.”
8:55

Agency in Machines and Automation

Discuss the potential for agency in machines and the importance of executive processes.

“is to just take some ordinary dumb mechanism and say, how would you make it smart?”
14:00

Understanding Executive Function in Machines

Explore the missing executive processes in machines that prevent true agency.

“But now we're in a new world where we have self-driving cars and drones and things like that, which are doing something very, very similar.”
17:44

Emergence of Cognitive Capabilities

Discuss the idea of emergent cognitive capabilities and their implications for AI.

“And we just see this huge collection of kind of cognitive atoms that weren't evolved towards a particular capability, but the capability just suddenly appeared in that milieu.”
21:31

The Role of Learning in Agency

Examine the relationship between learning processes and the functioning of agency.

“different way and so i'm kind of drawing a general analogy to um uh say here and that's why i focus on the control systems analogy is the key is a new kind of organization.”
26:59

Modules and General Processes in Decision Making

Understand the balance between specialized cognitive modules and general processes in decision making.

“across all domains, things like perception, attention, learning, memory.”
Show all 23 chapters
28:00

Understanding Emotions in Decision Making

Explore how emotions influence human decision-making from an evolutionary perspective.

“The birds are off on a different track and they have a different explanation.”
29:10

Agency vs. Intelligence: A Philosophical Distinction

Delve into the differences between agency and intelligence as discussed in AI contexts.

“So feeling enables the system to navigate unpredicted environments.”
30:48

Critique of General Intelligence Constructs

Understand the critique against the concept of general intelligence and its historical roots.

“But, you know, squirrels can remember more hiding locations than we can.”
33:06

The Taxonomy of Agency in Evolution

Learn about the four levels of agency as they evolve from reptiles to great apes.

“So you're speaking about the evolution of agency and you have four clear levels.”
36:34

Shared Agency and Human Culture

Examine the concept of shared agency and its significance in human culture and collaboration.

“is metacognitive or rational agency, which I see for sure in Great Apes.”
42:00

The Concept of Shared Agency

Exploration of shared agency as a unique human trait, contrasting it with animal behavior.

“So this final type of agency is a little bit off to the side.”
45:01

Cognitive Evolution and AI

Discussion on cognitive evolution, AI's potential for agency, and the role of social constructs.

“And first of all, Leonardo da Vinci said that from constraints comes freedom, which I thought was quite nice.”
48:42

Perspectivalism and Communication

How perspectives in communication shape language, conventions, and understanding.

“So what you've described here is a picture where essentially complex cognitive capabilities can be factorized in a collective.”
56:00

The Foundation of Agency and Virtualization

Exploring the importance of foundational infrastructure for machine agency and the evolution of cognitive virtualization.

“I hold something up and show it to you for a reason.”
56:45

Cultural Fictionalization and Its Implications

Discussing how cultural constructs like money and corporations are fictional yet powerful societal elements.

“Now, we were speaking before about this cognitive evolution.”
58:53

The Dark Side of Cooperation

Analyzing the potential negative consequences of group cooperation and collective identity in modern society.

“virtualization is one of the things that culture does best and most powerfully, is it creates new entities.”
1:02:26

Emergent Phenomena in Social Systems

Examining how unintended outcomes arise from individual decisions in markets and society.

“And to what extent are these collective agencies a convergent or natural phenomenon?”
1:04:43

Agency in Machines: Philosophical Perspectives

Debating whether machines can exhibit agency and the philosophical implications of such capabilities.

“And I mean, personally, my intuition is the best lens of analysis is to think of this thing as being an agent rather than a collection of individual rational agents.”
A man walking through an airport, asking his phone a question

Ask about this episode

Get the gist, explore an idea or ask about something said.

Claude

On Machine Learning Street Talk (MLST), what are the key points of this episode?

Searching your listening

Transcript

Automatic transcript. May contain errors.

0:00This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed Sponsored Jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed Sponsored Jobs. Get a$75 sponsored job credit at indeed.com slash podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed Sponsored Jobs.

0:31It's the last week to beat the Black Friday crowd. Lowe's. Get 25 % off select pre-lit artificial Christmas trees. Plus, my Lowe's rewards members get same-day delivery for free. Shop now. Beat the rush. Thank us later. Black Friday deals are on now at Lowe's. In-store and online. Valid to$11.25. $25 minimum and loyalty membership required. Order eligible items by 2 p.m. Subject to weight and size restrictions and availability. Terms at lowes.com slash shipping terms. When you need a decision-making agent is when the environment is unpredictable. The living things are made of the same basic elements as non-living things.

1:08They're just organized in a different way. What makes humans different from other great apes is that we can form shared agents. Our shared agency and shared intentionality can adapt to this new world before we blow ourselves up. By the way, a Turing test is not even close to sufficient, not even sort of close. Hi, I'm Michael Tomasello. So I've just read your book, The Evolution of Agency. Let me just show that to the camera. I must admit, my head is exploding because I have so many things in my mind, so many things that I want to discuss with you. And I know we're probably not going to do it justice.

1:45So I do recommend that folks read this book. It's actually quite a short read and it's just so information dense. But Professor Tomasello, would you mind just giving us a bit of an introduction? I mean, how did you come to write this book? I'm by training a developmental psychologist, but I've always been interested in evolution. And my first job was at Emory University, and they had the York East Primate Center. And I went out and started watching the chimpanzees and said, my goodness, they're often like two-year-olds. In a lot of ways, they're like young kids, and in a lot of ways, they're different.

2:17And that was fascinating of the both similarities and differences. So I ended up zeroing in on the question of what makes humans different from other primates, It's what makes their cognitive and social capacities different from others. So I ended up 20 years at Max Planckia in Germany, where we had our own grade 8 facility and did 20 years worth of comparative studies. We kind of pioneered studies where you put children and grade apes in the same experimental situation and see what the subtle differences are. And the differences are often quite subtle, and you need experimental methods to see them.

2:55And then they had mandatory retirement in most of Europe for academics. And so I came back to my alma mater, Duke University, 10 years ago. Wonderful. So I think the TLDR of this book is that agency is an extremely important lens to understand adaptive behavior, not only in humans, but just across a spectra of life. And the way you describe it, roughly speaking, is it's the ability of a living thing to deal with ambiguity and to handle novel situations. Can you put a bit more flavor on that? When something is evolutionarily very important and the environment is predictable, nature tends to build things in.

3:39So think breathing and swallowing, sexual behavior, whatever. These are things that are evolutionarily critical and the environment is more or less predictable. And then there doesn't need to be a decision-making agent. When you need a decision-making agent is when the environment is unpredictable. And so what Mother Nature builds is a mechanism, is an architecture enabling the individual to assess the situation and make a context-sensitive decision with flexible behavior. So people think, oh, gosh, you know, all situations are at least at some level predictable. So for me, the most unpredictable are other agents.

4:25So prey, predators, but also conspecific competitors, conspecific cooperators. So those are there. when you have two agents, they really can't predict one another. So in any case, the idea is it's for unpredictable situations. There's a mechanism to have somebody on the ground, so to speak, to make the decisions because Mother Nature can't predict every possible contingency ahead of time. And I suppose we should talk about how to define agency. Now, this is a really vexed issue. We have many arguments in our community about it, and God knows we've spent a lot of time on the Stanford Encyclopedia of Philosophy, looking at different definitions and so on.

5:09So when we interviewed Friston, Carl Friston, he was kind of talking about it in terms of a similar kind of thing to where you're coming from. So some kind of a control loop, some kind of a boundary, some kind of active sense-making. But roughly speaking, there seems to be a couple of dimensions. So there's the dimension of the will. So, you know, am I the cause of my own actions? Am I doing things for reasons? Am I pursuing goals or intentions? Am I acquiring goals? Maybe at the strongest, do I have free will? So I'm completely almost doing things appropriate to the situation, but not caused by it.

5:42And then there's this causal efficacy dimension, which is quite interesting. So the ability to control the future to kind of realize my goals. And you see the deflationary definition is very much like, oh, I've just got this outer boundary and it's a thing that sort of senses and acts and does something. And then one step up, I have a target like a thermostat, and I'm trying to make the environment like something. And you see there's a kind of hierarchy to this agency. I mean, does that gel with how you think about it? Yes, in general. But I use a control system like a thermostat as a concrete model, because I know a lot of the discussions of free will and all that kind of stuff, they get a little far away from psychology.

6:21And the book on agency, I actually wrote it for animal behavior people. And the AI people have picked up on it. But a lot of them still kind of operate implicitly with a stimulus response, which is the classic. And you still hear them talking. They would say, oh, I'm not a behaviorist. But they still talk about stimuli and responses. No, you have an active organism with goals that they're actively pursuing. And when they aren't reaching their goals, they try something different. they have something that you didn't mention they have executive function executive processes and metacognitive processes some animals which i consider part of executive functioning all of these are different depending on different animals that's part of what the book is about from mammals on you have executive function and so they are second guessing themselves and inhibiting things that they were intending to do and deciding if they need more information and so agents are in control that's the point they're in control of of of of of what they're doing of the causal effects they're having in the environment i mean obviously they're not totally free i say in like the second paragraph we're not talking about free will untethered from biology we're talking about within the context of the organism's biological adaptations there's a space there's there's there's space in your for example pursuit of predators there's space for making certain decisions i mean i mean chasing prey there's space for sort of making certain decisions so um um so yes the way you described it is fine but i i just think that if we the the reason i'm so attached to these control systems cybernetic mechanisms is because it's a concrete manifestation of agency in its simplest form and with its necessary elements.

8:19This episode is supported by CyberFund. If you're building at the frontier of AI, they want to hear from you. CyberFund believes the future belongs to AI natives who want to achieve the impossible. And that's why they're introducing the monastery for AI native founders. It's an environment of pure focus and rapid execution for founders operating at AI native speed. And they're offering teams$2 million each to participate. Apply now at cyber.fund. And one of the things that one of the things I like to do when I'm in it with students is to is to just take some ordinary dumb mechanism and say, how would you make it smart?

9:05And my favorite example is a traffic light, which just works by a timer. And it's not sensitive to the fact that there are no cars coming the other way, but you still have to stop. How would you make it smart? And of course, there's only one solution. First of all, it has to have a way of perceiving the cars. It has to have a goal about what should the result be that, for example, equal numbers of cars at each road to the intersection or something like that. So it has to have a goal. It has to have perception. It has to have a decision-making mechanism that given what I'm perceiving and given my goal, here's what I have to do to make the world conform to my goal.

9:43So you can have that kind of organization that is completely different from stimulus response reacting to prepotent stimuli and even a thermostat or something captured the basics of that, not the entire thing. And in fact, the substance of my book is about different levels of this type of agency. Yes. And there was a beautiful bit in your book where you said you can have an ant colony and it can still have incredibly sophisticated behavior. But the bright line between our kind of agency is that we actually are in control of what we're doing. Yes, that's exactly right. I think I say right in the beginning, a lot of people, a lot of AI people, I actually was in an AI symposium where they were talking about being smarter has got to be good.

10:32So there has to be selection for being smarter. But actually, there are a lot of dumb creatures out there that do quite well evolutionarily because they're adapted for their environment. So it's really not about complexity, like building a spider web is quite complex, but it's about control and being able to control your flexible actions on the basis of your perception of the immediate situation. Yeah, I mean, one thing that concerns me a little bit, because, yeah, so control is important. And also, you've said doing things for reasons are important. So you sketched out a couple of examples. There's the thermostat, which has a target temperature, and it can kind of, you know, modulate the furnace.

11:10And there was another example you gave of a robotic Hoover system, I think, which would kind of move around and it had a camera. And I guess my one concern is that I don't know the extent to which you're kind of human chauvinistic about agency, but it kind of feels that, yes, that has this control feature that you're talking about, but it doesn't, I wouldn't say that it had agency. I mean, to me, something is missing there. Okay. I'm perfectly happy with that. For me, I was coming from a slightly different direction, which is that in the cognitive science community, AI is the subgroup of that, but in the cognitive science community in general, often when you talk about agency, they start worrying about a homunculus.

11:50you know you you have some other person in your brain who's uh making and they don't and they don't like homunculating so i'm trying to say look all i'm talking about is a kind of organization whose basics can be seen in a thermostat or or um you know an automatic hoover machine uh and and that's just the basic organization no i would not call them agents i would not but um uh it's just that it's not about a miraculous homunculus or immaculous mental substance or anything the old philosophers used to think about. It's about an organization between action and perception and cognition and goals and desires.

12:32But do you think the strongest form of agency could, in principle, be replicated in machines? I mean, just as a tangential thing, I read an interesting book the other day called The Irrational Decision, and this was a professor at Berkeley, and he spent his entire career doing control theory. And he was kind of saying that we've gone a bit too far with over-optimization. So after the Second World War, you know, we developed algorithms like the simplex algorithm and powerful computers. And immediately we tried to solve the optimization problem of what is the optimal diet, you know, so can we sort of minimize the costs and get the appropriate calories and get the right types of foods and whatnot.

13:08And what we did was, you know, there's this whole kind of field of rationalism, mathematical rationalism, which is spearheaded by people like Steven Pinker. And, you know, is there a similar thing here? Because the thought occurs, it begs the question that, well, if we are making decisions, surely we could make a computer program that makes even better decisions than we do. We could make a computer program which has more agency than we do. I mean, do you subscribe to that? I have to say that I equivocate on that to some degree. In the old days, the idea of artificial intelligence was, one of the main criticisms was it's like a brain in a vat because the machine itself doesn't have a goal.

13:49It only, you know, and it doesn't perceive the world. And they were trying to just model just the computational parts, assuming perceptions and goals and actions and whatever. But now we're in a new world where we have self-driving cars and drones and things like that, which are doing something very, very similar. And I would say structurally very similar. And so obviously the question then becomes what's missing. And I'm not sure. I actually think that one of the keys is, which I mentioned before, is this executive processes where you know what you're doing. And I think the machines have not really embodied that in the right way.

14:35It's not just a separate little skill. It's a whole other level of organization. I call it a tier of organization. And the executive tier observes, perceives, pays attention to, if you will, what's going on at the attention and action level and monitors it and can proactively avoid mistakes by anticipating them, by planning and all of that. And you need an executive tier for things like planning, hypothesis testing. It requires imaginative representations, as I call them, that you have to imagine the way a world could be. I think animals like reptiles, snakes and turtles and things, I think they don't have those.

15:16I don't think they can imaginatively imagine what might happen and plan a strategy and stuff like that. So now you're getting a little bit more, even with the animals, I call that the beginning of consciousness where I'm conscious of what I'm doing. I'm perceiving and attending to what I'm doing. And then another tier that I think comes with great apes and humans, humans being a type of great ape, is metacognitive, which is another form of executive functioning, where I reflect on my executive planning and all of that. And I say, is that a good plan? Could I have a different plan? Do I need more information to have a good plan?

16:01Am I putting my resources in the right place or am I wasting effort? So these kind of metacognitive things is another tier. It's not a separate little skill over off to the side. It's another tier of functioning. So if you could have drones or self-driving cars that have these executive levels of functioning, perceive the world, have goals. If the goals are built in by humans, that doesn't bother me totally because, of course, the human goals are pretty much given by Mother Nature, at least the most basic ones of eating and procreating and stuff. They're already built in. You'd really want them to be able to create novel goals in situations where they don't know what to do to think, well, okay, here's a novel goal I've never had before that might help get me to this novel situation.

16:54So I just think, I guess I'm not afraid that AI is going to take over the world. I've been here since the 70s, and they've been claiming that it's going to take over the world since the 70s. I think of it as a really important tool, and I don't see any reason why we shouldn't just keep pushing the envelope and trying to make them more agentive. And that's my suggestion is look at natural agents and how they work and use that as a model and see if that will help us get further in that direction. I suppose what's interesting about AI agency is it's something that wasn't, we didn't explicitly train them to be agentic, yet it seems to be some kind of an emergent thing.

17:35And actually, I think the first quotation in your book was from Darwin. and he was saying in the distant future psychology will be based on new future mental powers that are acquired by gradation and isn't it weird because i don't know whether i'm missing maybe i'm misinterpreting you but but you seem to be saying that there are there's almost like geological strata you see these kind of cognitive capabilities that one builds on top of the other and certainly with ai and maybe in other places as well what we see when when we kind of zoom out and we're being externalists rather than internalists.

18:07And we just see this huge collection of kind of cognitive atoms that weren't evolved towards a particular capability, but the capability just suddenly appeared in that milieu. I mean, could that be something that's happening in AI? I'm not technically sophisticated enough to have an informed opinion about that, but that's what it would need to do. It would need to be able to, as I say, formulate new goals, formulate new strategies in evolution, what happens all the time is what they call pre-adaptation. So the textbook example is the opposable thumb was for climbing, and then we come down onto the savanna, and it's useful for picking up things and whatever.

18:50So you have something adapted for one thing, and then now you're in a new context, and the organism, the agent, figures out creatively how to apply it in a new way. Yeah, I know that a lot of AI mechanisms do some surprising things, but again, I'm just not technically knowledgeable enough to know exactly what's possible there. I will say that I do think that a lot of the AI models focus almost exclusively on learning. Everything's learning, learning, learning, learning. But learning only exists to help the agent make adaptive decisions. So learning is a sub-process. It's important. Obviously, it's really important.

19:37But you could, in theory, have an organism like a thermostat that just behaves in a situation without learning. Now, it turns out that pretty much all organisms we know save their experience from when they do something, they save it for future reference. But learning is not the be-all and end-all. And that's one of my things like reinforcement learning. That's one of my sort of gripes about it is it does have a little agentic quality, but it's mainly focused on learning. And I don't know if you're planning to come to this topic at some point, but large language models have a different problem.

20:25And the large language models have the problem that they're starting at the end point. They're starting at the pinnacle of human achievement and thinking we can build artificial general intelligence from that. And it's theoretically possible. I wouldn't rule it out, I guess. But for me, language is built on all of these other more basic cognitive capacities. And can you just skip them and skip to the final? I'm not so sure. But yeah, so I guess the one thing I would, there's an analogy that I have in the book about back at the beginning of the 20th century, there was an argument in biology about vitalism.

21:10and vitalism you know what distinguishes living things from non-living things and there was a school of thought that there was some substance some special elan vital that was a special substance that made living things different and through painstaking research we find that it's the living things are made of the same basic elements as non-living things they're just organized in a different way and so i'm kind of drawing a general analogy to um uh say here and that's why i focus on the control systems analogy is the key is a new kind of organization. And again, the book lays out all the details for this sequence of growing complexity to humans to try to say some of the kind of explicates, specify some of the details of that story.

22:04Yeah. And that really stuck with me in your book when you were saying that you don't actually need to have learning per se. Like a lot of behavioral adaptation can be explained just through this agentic lens. That was very fascinating. But I'm kind of interested. I mean, there's so many directions to go from here. Like, for example, individual agency is something that is valorized, particularly in American culture. And, you know, you gave the example of Richard Dawkins. He was talking about the selfish gene. And what he was basically saying is that there's some kind of a super agent. And actually, postmodernists make a similar argument.

22:37There are machinations of power, and they parasitize us. And they kind of... There was actually a really funny example. I was watching an interview with Noam Chomsky and Andrew Marr. He's a British journalist. How can you know that I'm self-censory? I don't say you're self-censory. I'm sure you believe everything you're saying. But what I'm saying is if you believe something different, you wouldn't be sitting where you're sitting. Do you see what I'm saying? So where do these goals come from? So there have been two kinds of determinism. If you want to think classically, historically, Darwin introduced this idea that you're naturally selected, that sort of nature is in control, at least that some people can construe it like that.

23:17And then Karl Marx is the one who said the culture. All right. No, no, no. You're born into a social class. You have an ideology. There are issues of power. And so he was saying that it was the culture. And I believe that the notion of agency is that we should, of individual agency, is we should think of those as powerful forces that influence the way we decide things. but they don't determine it. And I mean, like Tubian Cosmetes, classic evolutionary psychologists, they always give the example of like, you know, we're attracted to sugary foods and they give an example of why we're attracted to sweet foods and stuff.

24:02But it's not absolutely determined that I'm going to eat ice cream tonight. I'm on a diet and I think I don't want to have the ice cream. So I'm attracted to it. I don't control that attraction. That's a biological given. or I'm attracted to some young, cute woman, but I'm married. And so I'm not going to do that. And we have competing, sometimes we have competing interests. So in the more social realm, I made a promise to somebody, but now my daughter was in a car accident. So I break my promise. I have to decide whether to keep my promise or go help my daughter. So one of the main arguments in the book is against this tendency to deterministically.

24:47Think in terms of influences, factors that are—nature puts her finger on the scales in various ways of your decision-making, and your sociology puts its finger on the scale in various ways as well. But you, the agent, have to resolve moral dilemmas where, as I say, you made a promise, but now something more important comes up. Or you'd like to eat the sugary food, but you're on a diet. We need an agent to adjudicate between those different competing influences. Yes, yes. And I suppose it's interesting that you said we should think more deterministically, but still in the realm of ambiguity and chaos, that you can still end up doing very, very different things in different situations.

25:30But it's really cool that you brought up Tubi and Cosmedes, because I took a note that you quoted that they said that the brain is not a general learning or computational device. So you were talking about, you know, it being very specialized and akin to a Swiss army knife. Could you explain that? So they just focused on the fact that we have a number of things where there's a very strong biological signal. They call them innate modules in their classic thing. They've softened up a little bit, and they're independent of one another. That's why they call them modules. So there's a computational module for mate selection.

26:12And for males, males are attracted by signs of youth and fertility in females, and females are attracted to this in males, but it's a little module and it has nothing to do with foraging for food. Foraging for food is a completely different problem and it has different heuristics. So you're looking for the most amount and the most nutrition and whatever it is. And so we have these little modules and I don't have any real objection to the existence of those. I think they do exist, but they have a hypothesis of what they call massive modularity, that everything is modular. And that's where I differ is because when you have an agent, the agent has to use general processes that cut across all domains, things like perception, attention, learning, memory.

27:10All these things are not specific to, they're not modules. They're not specific to specific domains. They are used by the general decision-making process in the moment. And as I say, what do you do if you have two competing modules at the same moment? You have an attractive sexual partner and attractive food, and both modules are going. What am I going to do? So I don't object to the existence of little special-purpose modules. and the animal world is full of them, where you have little special purpose, you know, squirrels cashing nuts and et cetera, et cetera. There are all kinds of special purpose modules.

27:54So I'm happy with that. But I think with creatures, at least from mammals on in the human line, I'll leave the birds out of it. The birds are off on a different track and they have a different explanation. But along the line to humans, at least the mammals are clearly agentive making decisions influenced by emotions. What are emotions? In an evolutionary sense, the emotions are nature putting its finger on the scale of your decision making. So nature makes you incredibly afraid of snakes. And you see a snake and you drop everything else. You drop all your other goals and leave. But I would say if somebody had a gun to your head and say, don't move, you might let the snake crawl over your foot who knows so you have these um these uh um emotions that uh um attract you to other people uh make you afraid of certain things make you happy in certain situations and make you angry when certain things happen and that is that is nature's way of influencing your decision, but it's still you making the decision.

29:06Yeah, that's so fascinating. I interviewed Mark Soms. I don't know if you're familiar with him, but he said that affect was the felt sense of unpredictability. So feeling enables the system to navigate unpredicted environments. In other words, to manage uncertainty. Okay, that's good. Yeah, that's quite an interesting theory. I wanted to talk about the distinction between agency and intelligence. Now, in your book, whenever you wrote the word intelligence, you put it in air quotes. And I'm just trying to understand because a lot of folks, certainly in the AI world, they talk about intelligence in similar terms.

29:42So it's basically about the adaptation to deal with novelty. And so there seems to be a relationship between the two, but you were a little bit scathing about it kind of saying that, no, it's actually far more specialized and it's misunderstood and so on. How do you distinguish I don't mean to get philosophical on you here, but the word intelligence, the concept of intelligence was created by Alfred Benet in a French laboratory where the French government gave him the task of figuring out who's ready to go to university and who should go off in trade school. They do this in high school level. And so he's basically studying school type intelligence, verbal, abstract thinking.

30:30And then people started giving IQ tests and they reified IQ into intelligence. So I have a kind of a knee-jerk reaction to the notion of general intelligence. And the reason I put it in square quotes is because the reason what people, when they use the word intelligence, They often think about general intelligence. But, you know, squirrels can remember more hiding locations than we can. And a lot of birds can probably discriminate more different sounds than we can. And, you know, chimps use tools better than children do anyway when children are outstripping them and other things. So there is a lot of specialization.

31:11And so I'm just not, we acquire knowledge about the world. we apply it in specialized ways, etc. I just think it's too aimed at verbal intelligence of a type that's tied to school and that kind of thing. And I think if you're thinking more in terms of the natural world and other animals and stuff, general intelligence is not the best construct. Yes, I suppose that's true. I mean, I guess I think about it in terms of it's the ability to crystallize skills. So for example, you learn to drive and I don't know how to drive. And then over time, I crystallize it into a skill and now I can do it automatically.

31:51The only reason I bring this up is could there be a similar charge of agency here that we're also reifying agency and we're reducing it in an oversimplified way? I guess that's possible. But again, that's why I have this sort of control system thermostat that kind of model, because then it gets real. You can touch it. It's a physical mechanism. And it's that organization. And I think that's the starting point. And then I don't think there's an issue of reification. Whereas intelligence, there's nothing like that that you can point to.

32:32And I'm fine about being adaptable and flexible and all that. But I just prefer, because of the baggage that the term intelligence carries of the type that I was just talking about, I prefer talking about cognitive capacities, knowledge gained from experiencing the world and learning. So knowledge, cognition, planning, foresight, hypothesis testing, all the language of cognitive science, I prefer that to the language of intelligence just because of its historical associations. And can we talk through the core taxonomy you have? So you're speaking about the evolution of agency and you have four clear levels.

33:13Can you explain that? So I'm only talking about the lines to humans. Okay, again, the birds are off to the side. And I think with the first land vertebrates, which is where we have good data, let me say The method is a method used in comparative biology in general of having model species. That is, we don't know who the first land vertebrates were, but we have current day reptiles, some of whom are posited by biologists to be very similar to the first reptiles. And we can do experiments with them. So it's not like we can't go back at all. Yes, we have to be careful. The model species is not exactly like the original, but we can do that.

33:56And so experiments with reptiles used as models for the first land vertebrates, they are organized like a control system. They are making decisions, but only very simple decisions. And so I think of them as basic control systems. Then with mammals, you get an executive layer of control, and everything is focused on the decision-making process. and to make the kind of decisions that they make, a squirrel in a tree, you know, starts, you can just see them trying to decide, should I jump to that branch or not? I saw one while I was writing the book. I think I might've given the example. They look like they're going to jump and then he crawls down and crawls up the other one because he can imagine himself failing.

34:42So they are in their planning and hypothesis testing. They have to have imaginative, what I call imaginative representations, representing states of the world that don't currently exist. And they have to have executive level to do the planning and the hypothesis generation and so forth. Their learning is Bayesian because now you can have a hypothesis generation and probabilities and whatnot. So that's the second level is, I guess I call the first level sort of basic agency and then the goal-directed agency. And then this level I call intentional agency, because when you form intentions and plans, you can abort them before you actually enact them.

35:24I think Daniel Dennett said at one time that with intelligence or something like that comes that my hypothesis dies instead of myself. Right. So, you know, I see, whoops, that's not going to work. And so I saved myself the problem of failure. So I think mammals are acting in that way. You can look at your pet dog acting like that all the time. You can look at squirrels. If you want to see some hilarious examples, look on the YouTube under squirrels outsmarting humans with bird feeders in their backyard. They try to protect their bird feeders. And clearly these are novel, but the squirrels are inventing novel strategies for that.

36:11And some people have built obstacle courses for squirrels based on this kind of thing. So they're flexible, creating novel behavior, hypothesis testing, planning, all of that stuff needs an executive tier of organization. And then the third level of individual agency, and the highest level of individual agency is metacognitive or rational agency, which I see for sure in Great Apes. um we don't have enough data on non on other primates but for sure in great apes and they do things like they make a decision but the result doesn't happen right away and then they get more information and they change their they they look for more information so they can make a better decision um and uh they and they have various strategies for when the problem is very difficult and they know they're going to make a mistake uh they can opt out and do something similar, more high probability.

37:11So this metacognitive functioning is another level of flexibility. And in each case, I propose, like reptiles are mainly solitary. So most of their unpredictability comes from their prey. They mostly prey on insects and other little creatures. Mammals all of a sudden have to compete with group mates because they're group living. And so they want to get food, they have to compete with. So they have to make a faster, better decision than their group mates or they're going to die. And great apes have a lot of contest competition where they have to outsmart the other one. Most mammals mainly do, well, they do a lot of scramble competition where they just get their first.

37:53Whoever gets their first gets it. But then great apes have a lot of contest competition where they're having to outsmart one another. And then you were saying four levels. it's slightly off to the side, but the humans, and this is what most of my work for most of my career has focused on, the agency stuff is more recent, but what I focus on for most of my career is that what makes humans different from other great apes is that we can form shared agents. And so this, I know that there are AI, that there are various kinds of AI, collective agents, what's the word they use for? Oh, sort of like AI societies of agents or something like that.

38:38But they mostly react to one another. And what a shared agency is, is we put our heads together. We have a shared goal. And you have your role and I have my role. And we both know that. And the role has certain standards. So if we're going to be successful, you have to do your role this way. I have to do my role this way. And we both know that. And if you don't do your role the right way, I say, hey, what are you doing? Come on. Your bad behavior over there is going to make us fail. And so there's a kind of a self-regulatory mechanism that in this case is social. It's a shared commitment that we have.

39:12There's a postdoc in our department, Xiaojia Cheng, who will be coming to England soon, who is trying to model this kind of shared agency with a joint commitment.

39:27and what that does is cognitively is we have joint attention. So we're trying to pursue some goal together cooperatively as a joint agent. And if we're going to do that effectively, we have to both jointly attend to possible obstacles or opportunities as we're going along. And we have to jointly attend. We have to both know we see them so that we can have a unified strategy as we go. Um, and, uh, so that joint attention, which I've been studying for a very long time in young children, creates the notion of perspective that I see, we're both seeing the same thing, but I see it this way and you see it that way.

40:09And this is one of the most powerful, uh, powerful parts of uniquely human cognition is the idea that the same object can be viewed from different angles. The same object can be a dog or an animal or a pet. We take that for granted. Well, of course it can, we say. It just depends on how you look at it. Well, guess what? Other creatures don't have this. It just depends on how you look at it. You know, they see it as whatever they see it as, but they can't just flexibly switch perspectives. And I come from a long tradition of people like George Herbert Mead and the pragmatist, Donald Davidson philosopher, people who say that this is all social, that this is the notion of perspective is only if you were if you were a child raised on a desert island which is one of my favorite um examples i've been using for over 20 years um wouldn't get the notion of perspective because they'd be going around looking at the world but there wouldn't be a contrasting way of looking at it that is right there with them at the same time so um uh humans have a unique form of sociality uh it's at its basic is this joint agency but then And ultimately, what really, you know, the booster rocket is we have these collective agencies known as cultures.

41:22And cultures give you, you're born into a culture, and it gives you conventions like linguistic conventions and norms about ways you ought to behave or the way you ought to think about things in education. That's what you're taught when you were saying that Noam Chomsky was saying, you know, it starts in school. They're telling you to think. That's what they're teaching you in school, how you ought to think. This is a better way of thinking. We're telling you about it. And so we're born into cultures, which are kind of collective agencies with conventions, norms, and institutions. And they are both constraining, as Chomsky was focusing on, but they're also empowering.

42:02Because when you get a language and you get social norms and you get conventions that you yourself didn't have to invent, those are given to you by your culture, assuming you have the ability to learn them, then you can write poetry with the language. You can do new things with it. So this final type of agency is a little bit off to the side. It's not just another step up. It's a new version, is the shared agency. And I personally believe it's unique to humans. Yes, there are other creatures that cooperate with one another, but we've done experiments, many experiments with chimpanzees compared to human children.

42:37And you can see that children share the spoils of a collaboration differently. The chimps don't correct one another when one of them doesn't play their role the right way. When kids are collaborating and one of them gets their reward first, they keep collaborating until the other one gets theirs. The chimp, as soon as they get their reward, they're done. So there are demonstrable empirical observations that one can make that I would argue show that the human shared agency, including both collaboration among individuals and culture among humans as members of cultural groups, is what makes humans different.

43:20And if I can say, this is one of my issues with the large language models, which I use every day, incredible tool, you know, I'm not a bit. bit. Humans not only use language, they created languages. They created them. We have studies with children showing that when we don't let them talk to one another, we have them going through video monitors. This was back at the time where it was Skype and not Zoom. And the kids knew about talking to their grandparents on Skype. And then we say, oh, the sound went off. So you've got a gesture to one another. And they make up novel gestures and they conventionalize them.

44:00The ones using one another start using the same one and they start adapting one another. So we create linguistic conventions. There's 6 ,000 different languages in the world. We create linguistic conventions and maybe you don't need to have that capacity, but large language models, to my knowledge, don't create linguistic conventions. They don't have discourse around topics and so forth. So I'm waiting for that. But I think this notion of shared agency and shared intentionality are the sort of infrastructure of the creation and use of linguistic conventions. And as I say, it's possible we can skip that step.

44:44But I would say I think it's more likely that we're going to get to more useful and more human-like machines if we start with those steps and have language be the sort of ultimate outcome or the sort of pinnacle. Yeah. Oh, that was absolutely fascinating. And first of all, Leonardo da Vinci said that from constraints comes freedom, which I thought was quite nice. But yeah, I mean, just to backtrack a little bit. So you were talking about this cognitive evolution and a few things you said really jumped out at me. So, you know, like the ability to imagine and simulate and delete strategies and do epistemic foraging, you know, all of this is fascinating.

45:24But certainly the thing that interests me the most, though, is the final step you spoke about when we talk about having shared agencies and where did the goals come from and all of that kind of stuff. Now, isn't it interesting that you spoke about AI and at the moment they don't do this kind of constructive, you know, like coming up with conventions and it's a linguistic agency, basically, is the technical term for it. But maybe they will in the future because, you know, in the world of evolution, we have this kind of physical evolution and we have, you know, sort of ontogenic accumulation of learning.

45:57And then we have this cultural accumulation of learning. And the speed goes up as you as you kind of go up the hierarchy. The speed goes up and maybe AI agents will be able to do that when they can talk with each other and create conventions. But do you think that that's a reasonable way to think about it? Do you think AI could be kind of socially constructive in the same way we are? That's why I say the postdoc, Xiao Zhe Cheng, is working on something like that. And so in my theoretical books before this one on agency, which were on shared agency, that evolutionarily, probably the originating evolutionary ecological challenge was coordination problems.

46:39I don't know if you're familiar with game theory, but Thomas Schelling coordination problems, because coordination problems require a certain kind of logic. It's not just being smart. OK, it's taking the perspective of someone else, taking your perspective while you're taking their perspective. And the stag hunt is a classic coordination problem where we each have our own sort of low level resources that we're foraging. And then something much bigger and better comes along. But I don't know if you're going to go for it. I only want to go. We have it takes both of us to capture it. and so I only want to go if you go because otherwise it's futile and you only want to go if I go and so it's like I only want to go if you go you only want to go if I go it's like Abbott and Costello walking through the door after you no after you no after you so so I do think that evolutionarily the originating parts are not linguistic they are about putting your mind together to solve these coordination problems where we each have to do our part in common knowledge with the other one.

47:48And then David Lewis, the philosopher David Lewis, has a book called Convention, and his argument is that convention is a coordination device, that a convention, it was created conventions, linguistic conventions were created by humans to help coordinate their activities together. So again, I would never make an in-principle argument that we could skip that kind of thinking and go straight to language. Maybe we can. But I would say a reasonable, I would think somebody ought to, at the same time that other people are just working on mature adult language, some other people ought to be working from the bottom up and saying, let's see what it's like to put your head together and what it's like to create a convention as a coordination device.

48:34and then we'll have that infrastructure on which we can start putting the linguistic skills. Yeah, and can we explore a little bit more on this perspectivalism? So what you've described here is a picture where essentially complex cognitive capabilities can be factorized in a collective. And we've spoken about agency, but I also love this idea that even just the epistemic, the representations, the knowledge part require factorization, because I have an intuition that but it's quite path dependent. So we have different perspectives of the elephants, and it seems like there's something strictly better about forming a collective of different perspectives than having one sort of very intelligent agent doing it all on their own.

49:19The philosopher Donald Davidson was basically, he says something like, this kind of engagement with another human being is what raises the possibility that you could be wrong. Because it happens all the time. In science, it happens all the time. I think I'm right, and somebody else shows me something else, and I go, whoops. And if it's just you yourself, yes, you can learn from your mistakes, but learning from your mistakes is not quite the same as seeing things in a different way, as you can sort of take the perspective of another person. So, yes, I think that's underappreciated because if I talked long ago to artificial intelligence people before the modern wave of stuff, they'd say, oh, perspectives like that, that's simple.

50:12Okay, we build one node for animal, one node for pet, one node for dog, and it's no problem. But in the natural world, you have a single perception. you have this thing in front of me and I can see it as a dog or an animal or a pet. And, and I would say that it's, it's, it's the social that in, in language, why do I call it? I can call it a dog, an animal, a pet, or I can call it it or she. When do I call it those things? It's because I'm adapting for my listener and a context where in certain contexts, the only thing is important is that animals are not allowed in here. And in another case, it's I want a pet what kind should I get you should get a dog and another kind it's like should I have some a companion with me or not should I get a pet so depending on your common ground and your perspectives of the person you're talking to we have to construe things in a different way and I think that's an underappreciated aspect of linguistic communication is that it is in its core perspectival.

51:13When people talk about grammar, computationalists talk about grammar, and I know grammar is a topic that bores most people, so I'll be brief. But when you talk about grammar, the computationalists want to talk about combinatorial possibilities and combining and all that. But if you look at the fanciest grammatical constructions, take what in English is called a cleft construction, it was Fred that broke the window. Think about that. It's a very strange construction. It was Fred that broke the window. Why didn't I just say Fred broke the window? The reason is because you think somebody else broke the window and I'm trying to correct you.

51:52So I say it was Fred. It was not Mary. It was Fred that broke the window. So the origin of that construction is in communicative situations where you have a mistake. I believe you have a mistaken impression and I'm trying to correct it. So it's a perspectival problem. And that's just that dimension of grammar is not appreciated. So it's not just words as conventions. Grammatical constructions are conventions that we have created to view things a certain way. The dog is chasing the cat. The cat is being chased by the dog. The passive construction. Actives and passives come from the perspective of the dog or from the perspective of the cat.

52:36We find that trivially easy. but again that's it's all about taking the perspective of the other person so all i'm trying to emphasize here is that language as we use it depends on these more fundamental abilities and and i guess one of the surprising things about the large language models is they seem to do that a lot better than i would have thought they possibly could and there's something in the textual material, the connection between sentences, the logical connections, the changes of perspective within the text. And I actually asked in a recent AI workshop kind of thing, I asked, what if you took all of the stuff, everything on the internet, and considered each sentence individually, and you randomized the order, so you no longer have any texts with coherence, no causal or logical or discourse coherence, but you just have a group of random sentences.

53:34What would the LLM look like if it used that material? And of course, nobody knows, but the consensus was it would be pretty bad. That you need those, one of the reasons it's able to do all these intelligent things is the inherent logic in a discourse, which goes from one sentence to the next one that follows it, et cetera. And so all of that is language building on this ability to have a common perspective with others and have different perspectives. Ironically, on that one, this is something that Chomsky says quite a lot, that the reason he doesn't like LLMs is that they can learn impossible languages.

54:20So you could just randomize all of the words and the LLMs would learn it just fine. Obviously, it would just be garbage to us. But one thing LLMs can't do is, you know, a really good writer, they use agentic storytelling and they write in this perspectival way that you just articulated. So, you know, in terms of me, my experience, my goals, conflicts, things that stop me from getting to my goals and so on, LLMs don't really do that. Yeah, I think that's right. And I think, again, that's the infrastructure. I've used the word infrastructure. I have a book on the evolution of language, and I use that as the infrastructure.

54:57And for example, I don't know whether this is fair, LLMs or not, but what children learn to do before they learn to talk is they learn to use the pointing gesture. So when I point for you to something, I mean, I have practically a whole book on the pointing gesture. So I point over there, and the way I just did it, you would say, huh, what? I did that right. But in the right context, my point can mean anything. There's the thing we need. There's your sister. The library is open. I mean, my pointing finger can mean anything because we are in a shared context where we both know what the other one is thinking and we can take the other one's perspective.

55:40And basically the pointing gesture, one way to gloss it is, look over there and you'll know what I mean. Well, how do you know what I mean? Well, because we are in some kind of communal thinking about things and taking one another's perspective. And as I'm saying, it's not exactly fair to say an LLM can't interpret a pointing gesture. But I believe that that's part of the infrastructure leading to language is the ability to sort of know what somebody means just by the fact that they're pointing something out to you. I hold something up and show it to you for a reason. and again maybe that infrastructure can somehow be skipped and it certainly can be skipped if what you want is a useful tool like llms are right now but they're but but if you want them to be agentic then i think you're going to need that infrastructure and and the methodological prescription then is try to build something that starts from the bottom up that starts from animals coordinating and taking perspectives and maybe communicating in some simpler ways before they build conventions.

56:44Fascinating stuff. Now, we were speaking before about this cognitive evolution. And one way of couching it is that it's about increasing virtualization away from the physical. So, you know, at the beginning, we can imagine and simulate. And then at some point, you're saying we actually have this kind of social agency formation where we can basically coordinate and we can come up with goals. And the more you virtualize, because if you kind of think about it as increasingly kind of propagating storing information because culture is this magical hard drive in the sky. So I could die tomorrow and whatever organize, you know, let's say my podcast might survive after I die because someone else would take it over.

57:21But are there any pitfalls or degeneracies with this kind of virtualization? So I know you've done a lot of research around things like collusion and bad things that happen in these collective agencies. um i i just before i get to the bad thing i just want to when you say this virtualization there's another element to it have you read um harari's book on um homo sapiens on sapiens oh yeah many years ago yes yes so what he wants to point about in the virtualization is he um he says we live in a in an imaginative world where he's talking about things like dollars or pounds uh presidents prime ministers.

58:02Those are all things that only exist because we agree they exist. And if all of a sudden, you know, World War II happens, or I went to East Germany, and East German marks were money, and then one day they weren't money anymore. They were pieces of paper. So this virtualization in culture comes from us creating what John Searle, the philosopher, calls status functions. That is, we make a person into a president, we make a piece of paper into money, and the most powerful forces on the world, Harari argues, is like corporations. What is the Apple Corporation? And what is Meta? These are what he calls fictions.

58:43They're made up. Corporations are fictional individuals that are legally individuals, but obviously not in a direct sense. So this virtualization is one of the things that culture does best and most powerfully, is it creates new entities. Searle is quite strong on this. It is a fact that, unfortunate fact, that Donald Trump is president of the United States. That's a fact about the world. And now it's a human relative fact. That is, it only exists because humans have got this concept, created this concept of president and so forth. But it's still a fact about the world. And the corporations may be fictions, but they're the most powerful things happening on the earth today.

59:31Okay, so now your question is, can this fictionalization, can this sort of have the bad side to it? And I think George Orwell, my daughter was just reading George Orwell for her high school, and I was trying to remember parts of it. And Donald Trump is a master at creating this alternative reality by playing on certain tricks, repeating things often enough, He always says, you know, they say I'm hearing that, which cues into the natural thing is that we want to know that everybody else is thinking about things a certain way. And then we think, oh, well, everybody in our culture is thinking a certain way.

1:00:11That's probably a good way to think. So he tries to make it look like everybody thinks like this. And again, repeating it over and over again is the same thing. And so there are these sort of ways of kind of co-opting the natural system. system. I mean, lying, lying only works because we're cooperative. Chimpanzees don't lie to one another because it would never occur to them that you would tell the truth if it didn't benefit. So the point of lying, I mean, the reason lying works is that we're all gullible. We all expect everybody to be cooperative unless they have some ulterior motive. If I tell you it's raining outside, it would never occur to you that I'm lying about that.

1:00:49I don't have any ulterior motive, of nothing. So lying works because of our general cooperative nature, and these things that politicians pull off work because we generally expect people to be cooperative. And yes, it can be used for nefarious ends. I mean, a lot of times people say to me about, you know, you're talking about how humans are so cooperative, forming shared and collective agencies, but what about all the strife in the world? Well, almost all of it is they are attacking us. It's we versus them. And so the human version of the collective intentionality, collective agency and intentionality is our group.

1:01:32And now in the modern world, we've got to find a way to cope with the fact that all the different groups are interdependent with one another and live in the same place and everything like that. But that's part of the dark side of cooperation. is that our evolutionary history is about people that we live in a group with people that look like us, talk like us, eat the same food, dress the same way. That's our adaptation is that's the we that we live in. And the other we's over there on the other side of the river, they look funny, they talk funny, they eat disgusting things. It's not that I hate them, it's that I'm just not sure about them.

1:02:09But in the modern world, modern world meaning since agriculture and cities and lots of migration and stuff, we live in a different world. And that's what we hopefully, our shared agency and shared intentionality can adapt to this new world before we blow ourselves up. And to what extent are these collective agencies a convergent or natural phenomenon? A lot of people talk about the market as if it is a sort of super organism. Yeah, I'm not a big fan of talking about inanimate things as agents. I mean, you could talk about capital as capital tends to flow toward this or that. But I would say it's the capitalists who make the decisions about where it flows and stuff.

1:02:52And there are these sociological level phenomena like the invisible hand and also collective action problems are of the same type. Collective action problems, nobody intends the bad outcome, but everybody has intentions. Everybody's trying to get home as fast as they can, and it creates a traffic jam. Nobody intended the traffic jam. It's an emergent phenomenon, but it emerges from the intentional actions of individuals. And the invisible hand is supposedly, you know, that prices get fixed at their maximal level because individual agents are making decisions to pay the lowest price for the best value, et cetera, and the sellers are doing the opposite.

1:03:32So those emergent phenomena, sometimes called phenomena of the third kind, are kind of where sociology begins. These are forces that are not due to individual agents making decisions, but they are things that emerge. Inflation. Nobody intends inflation, but it's a product of people's intentional decision. Nobody intends a stock market crash, but it's an outcome of people's intentional decision making. So these kind of negative things are the things that policymakers and politicians and stuff need to be aware of and need to be aware of the connection between individual decision making and the outcome.

1:04:12So stock market crashes, for example, one could have policies around risk taking and so forth that would try to mitigate that. And if you had stricter traffic laws, you might not get, I don't know what they would be. But they're the kinds of things that politicians and policymakers need to see these patterns and see the connection to individual decision making and make some kinds of rules or laws that regulate individual decision making in a way that these unfortunate things don't happen. Yeah, I think about this a lot because you were saying that in the market, it's supervenient on, it's like a superorganism, supervenient on rational, intentional agents.

1:04:52And I mean, personally, my intuition is the best lens of analysis is to think of this thing as being an agent rather than a collection of individual rational agents. And yeah, I mean, what's your argument to sort of go one level down rather than one level up? you were saying it's a useful way to talk about it. And that's fine. If you're saying it is an agent, I start to get a little nervous. But yes, I think it's fine if you're sort of aware. But does capital have a goal? Does capital make adjustments if it's not reaching its goal? I'm not so sure. Maybe this is just I'm a psychologist, and that's the sociological level of analysis and is different for me.

1:05:38But yeah, it's a useful way to talk. Absolutely, fine. But I'm sure I like really thinking of it in reality as an agent. Wonderful. And final question. Is it possible for a system to act as if it has agency, but not actually have agency? That's Daniel Dennett, what he calls the intentional stance. Yes, yes. uh uh yeah boy i don't know now we're getting metaphysical um i don't i don't i don't know i mean i i i would say if it if it walks like a duck and it quacks like a duck it's a duck and so um uh i think if we built something that really had all these qualities that and by the way a turing test is not even close to sufficient not even not even sort of close But if we had a machine that really could mimic all these kinds of things, then, yeah, I guess I would have to feel like it was some kind of an agent.

1:06:38You could say, well, it's not a human agent. I don't know. I mean, I saw a talk by the philosopher Ned Block, and I think the title was, Does Meat Matter? Meat meaning, you know, does the flesh, the brain, the hormones, and all that, does that really matter? because they're never going to have the, well, at least the way we're going now, they're not going to have emotions and hormones and neurons of the type we have. And does it matter? And I'll just say there was no consensus in the room, and I don't have any brilliant insights into that either. Amazing. Professor Michael Thomas-Allow, it's been such an honor and a pleasure having you on.

1:07:19Thank you so much. Thank you, Tim.

1:07:25Thank you.

From the publisher


Michael Tomasello is a developmental psychologist who spent twenty years at the Max Planck Institute in Germany putting children and great apes into the same experiments, and is now back at Duke University. His book The Evolution of Agency asks what it takes for a living thing, or a machine, to act for its own reasons.


SPONSOR:

---

Cyber Fund built the Monastery to help founders ship products that were impossible a year ago.

Apply now: https://cyber.fund

---


His answer starts with unpredictability. Where the environment is predictable, nature builds the behaviour in; where it isn't, it builds a decision maker. Tim and Tomasello test that on a dumb traffic light, a thermostat and a robot vacuum, then follow the line to humans: reptiles running simple control loops, a squirrel that imagines itself falling and takes another branch, and great apes that go looking for more information before they commit.


The second half is about shared agency. Humans form joint goals, attend to the same things together, and learn that one object can be seen from different perspectives. Children who lose the sound on a video call invent gestures and turn them into conventions. Large language models, Tomasello argues, started at the endpoint and skipped that infrastructure. On the way: the stag hunt, why lying only works among cooperators, whether capital is an agent, why a Turing test isn't close to sufficient, and Dennett's intentional stance.


TABLE OF CONTENTS:

0:00 Cold open: agency begins with unpredictability

0:29 Michael Tomasello and The Evolution of Agency

0:59 From chimpanzees to two-year-olds

2:05 Agents exist because the world is unpredictable

3:56 Defining agency: control loops, will and causal power

7:19 Sponsor: Cyber Fund

7:55 How would you make a traffic light smart?

9:57 Thermostats, robot vacuums and what is missing

11:32 Could a machine have more agency than we do?

16:26 Darwin, emergence and the limits of learning

21:04 Selfish genes, Chomsky and where goals come from

24:17 Modules, emotions and nature's finger on the scale

28:25 Why Tomasello avoids the word intelligence

32:06 Four levels of agency: reptiles, squirrels, great apes

37:04 Shared agency, joint attention and perspective

42:25 Why language models do not create conventions

44:08 The stag hunt and conventions as coordination devices

47:41 Perspective, grammar and why discourse matters

53:12 Pointing and building agents from the bottom up

55:44 Money, presidents and status functions

59:26 Lying, gullibility and the dark side of cooperation

1:01:27 Is capital an agent? Emergence and the invisible hand

1:04:48 Acting like an agent: the intentional stance

1:06:16 Closing


BOOK:

Michael Tomasello, The Evolution of Agency: Behavioral Organization from Lizards to Humans (MIT Press, 2022)

https://mitpress.mit.edu/9780262047005/the-evolution-of-agency/


KEY REFERENCES:

[11:41] Benjamin Recht, The Irrational Decision (Princeton UP, 2026) https://doi.org/10.1515/9780691272467

[28:07] Mark Solms, The Hidden Spring (Profile Books, 2021) https://profilebooks.com/work/the-hidden-spring/

[37:42] Michael Tomasello, The role of roles in uniquely human cognition and sociality (2020) https://doi.org/10.1111/jtsb.12223

[38:15] Cheng, Gao, Tomasello & Kushnir, The Agency Gap (CogSci 2026) https://escholarship.org/uc/item/8mv6v0s1

[38:59] Tomasello, Melis, Tennie, Wyman & Herrmann, Two Key Steps in the Evolution of Human Cooperation (2012) https://doi.org/10.1086/668207

[40:28] Michael Tomasello, The Normative Turn in Early Moral Development (2018) https://doi.org/10.1159/000492802

<truncated, see rescript/YT>


SUPPORT MLST:

Patreon: https://www.patreon.com/mlst

Discord: https://discord.gg/HNnAwSduud