#126 Noam Chomsky: Decoding the Human Mind & Neural Nets

0:00What's called AI today has departed to basically pure engineering. It's designed in such, the large language models are designed in such a way that in principle they can't tell you anything about language learning, cognitive processes generally. They can produce useful devices like what I'm using, but the very design ensures that you'll never understand, they'll never lead to any contribution to science. That's not a criticism, any more than I'm criticizing captions. This week I talked to Noam Chomsky, one of the preeminent intellectuals of our time. Our conversation touched on the dichotomy between understanding and application in the field of artificial intelligence.

0:56Chomsky argues that AI has shifted from a science aimed at understanding cognition to a pure engineering field focused on creating useful, but not necessarily explanatory tools. He questions whether neural nets truly mirror how the brain functions and whether they exhibit any true intelligence at all. He also suggests that advanced alien life forms would likely have language structured similar to our own, allowing us to communicate. Chomsky is 94, and I reached him at home where he appeared with a clock hanging ominously over his head. I hope you enjoy the conversation as much as I did. Well, thanks.

1:50You're in California? Actually, I'm in Arizona, which is on California time. Yeah, yeah. Oh, wonderful. Yeah. So, you know, I wanted to talk to you because you have the, you know, one of the few people with a deep understanding of linguistics and natural language processing that has the historical knowledge of where we are, how we got to where we are, and what that might mean for the future. I understand your criticisms of deep learning and what large language models are not in terms of reasoning and understanding the underpinnings of language. but I thought maybe I could ask you to talk about how this developed.

2:54I mean, going back to Minsky's thesis at Princeton, when he was, you know, before he turned against the perceptron, when he was talking about nets as a possible model for biological processes in the brain, And then, you know, how did how you see that things developed and what were the failures that didn't get to where presumably you would have wanted that research to go? And then and then I have some other questions, but but is that enough to get started? Well, let's take an analogy. Suppose you're interested in figuring out how insects navigate biological problems. So, one thing you can do is say, let's try to study in detail what the desert ants are doing in my backyard, how they're using solar azimuth, so on and so forth.

4:09Something else you could do is say, look, it's easy. I'll just build an automobile which can navigate fine, does better than the desert ants. So who Who cares? Well, those are the two forms of artificial intelligence. One is what Minsky was after. It's now kind of ridiculed as good old-fashioned AI, go-fi. We're past that stage. Now we just build things that do it better, okay? like an airplane does better than an eagle, so who cares about how eagles fly? It's possible, but it's a difference between totally different goals. Roughly speaking, science and engineering, it's not a sharp difference, but first approximation, Either you're interested in understanding something, or you're just interested in building something that'll work for some purpose.

5:20They're both fine occupations. Nothing wrong with… I mean, when you say I'm criticizing the large language models, that's not correct. I'm using them right now. I'm reading captions. captions are based on deep learning clever programming very useful i'm hard of hearing so they're very helpful to me no criticism but if somebody comes along and says okay this explains language you tell them it's kind of like saying an airplane explains how eagles fly the wrong question. It's not intended to lead to any understanding. It's intended to be for a useful purpose. That's fine. No criticism. And what's called AI today has departed to basically pure engineering.

6:19It's designed in such a way. Large language models are designed in such a way that in principle, they can't tell you anything about language, learning, cognitive processes generally, they can produce useful devices like what I'm using. But the very design ensures that you'll never understand, they'll never lead to any contribution to science. That's not a criticism any more than I'm criticizing captions. Jeff Hinton says that his goal was to understand the brain. the brain works and he talks about ai as we know it today uh supervised learning and and generative ai as useful byproducts but that that are not his goal are not the goal of uh cognitive science or computational biology.

7:27Was there a point at which you think the research lost a bead, or is there research going on that people aren't paying attention to that is not caught up in the usefulness of these other kinds of neural nets? Well, first of all, if you're interested in how the brain works, the first question you ask is, does it work by neural nets? It's an open question. There's plenty of critical analysis that argues that neural nets are not what's involved, even in simple things like memory. Actually, there's arguments that go back to Helmholtz that neural transmission is pretty slow as compared with ordinary memory.

8:24There's much sharper criticism by people like Randy Gallistel, cognitive neuroscientist, is given pretty sound arguments that neural nets in principle don't have the ability to capture the core notion of a Turing machine, computational capacity. They just don't have that capacity. And he's argued that the computational capacity is in much richer computational systems in the brain. Internal skills, where there's very rich computational capacity, goes way beyond neural net. Some experimental evidence support this. So if you're interested in the brain, that's the kind of thing you look at. Not just saying, can I make bigger neural nets.

9:24It's okay if you want to try it, but maybe it's the wrong place to look. So the first question is, is it even the right place to look? That's an open question in neuroscience. If you take a vote among neuroscientists, almost all of them think that neural nets are the right place to look. But you don't solve scientific questions by a vote. yeah yeah the uh i mean one of the things that's obvious is is neural nets you know they they may be a model or they may mimic a portion of brain activity but but there are so many other structures there's all kind of stuff going on in the brain yeah way down to the cellular level There's chemical interactions, plenty of other things.

10:16So maybe you'll learn something by studying neural nets. If you do, fine. Everybody will be happy. But maybe that's not the place to look if you want to study even simple things like just memory and associations. there's now already evidence of associations internal to large cells in the hippocampus internal to them which means maybe something's going on at a deeper level where there's vastly more computational capacity yeah those are serious questions so there's nothing wrong with trying to construct models and seeing if you can learn something from them if you can fine The building larger models, which is kind of the rage in the engineering side of AI right now, does produce remarkable results.

11:13I mean, what was your reaction when you saw chat GPT or GPT-4 or any of these models, that it's just a sort of clever stochastic parrot or that there was something deeper going on? If you look at the design of the system, you can see it's like an airplane explaining flying. It has nothing to do with it. In fact, it's immediately obvious, trivially obvious, not a deep point, that it can't be teaching us anything. The reason is very simple. The large learning models work just as well for impossible languages that children can't acquire as for the languages they're trained on. So it's as if a biologist came along and said, I've got a great new theory of organisms, lists a lot of organisms that possibly exist, a lot that can't possibly exist, and I can tell you nothing about the difference.

12:26I mean, that's not a contribution to biology. It doesn't meet the first minimal condition. The first minimal condition is to distinguish between what's possible from what's not possible. You can't do that. It's not a contribution to science. If it was a biologist making that proposal, you'd just laugh. Why shouldn't we just laugh when an engineer from Silicon Valley says the same thing? So maybe they're fun, maybe they're useful for something, maybe they're harmful. Those are the kinds of questions you ask about pure technology. Take large language models. There are something they're useful for.

13:16In fact, I'm using them right at this minute. captions. It's very helpful for people like me. Are they harmful? Yeah, they can cause a lot of harm. Disinformation, defamation, relying on human gullibility, plenty of examples. So they can cause harm, they can be of use. Those are the kinds of questions you ask about pure engineering, which can be very sophisticated and clever. I mean, the internal combustion engine is a very sophisticated device, but we don't expect it to tell us anything about how a gazelle runs. You know, it's just the wrong question. Yeah. Although, you know, I talk a lot to Jeff Hinton, And, you know, he'll be the first to concede that backpropagation is not, there's no evidence of that.

14:21And in fact, there's a lot of evidence that it wouldn't work in the brain.

14:29Reinforcement learning, you know, I've spoken to Rich Sutton. That's been accepted by a lot of people as an algorithmic model for brain activity in part of the brain, in the lower brain. So in terms of exploring the mechanisms of the brain, it seems that there is some usefulness. I mean, it's as you said, there's on the one hand, people look at the principles and then they built through engineering, just as the analogy of a bird to an airplane, they've taken some of the principles and applied it through engineering and created something useful. But there are scientists that are looking at what's been created, like Hinton's criticism of backpropagation, and are looking for other models that would fit with the principles they see in cognitive science or in the brain.

15:46And I mentioned this forward-forward algorithm, which you said you hadn't looked at, but I found it compelling in that it doesn't require signals to be passing back through the neurons. I mean, they pass back, but then stimulate other neurons as you move forward in time. But I mean, is there nothing that's been learned in the study of AI or the research of neural nets? but if you can find anything it's great nothing against search you know but it's just but we have to remember what you asked about chatbots what do we learn from them zero for the simple reason that the systems work as well for impossible languages as for possible ones So it's like the biologist with the new theory that has organisms and impossible ones and can't tell the difference.

17:12Now, maybe by the look at these systems, you'll learn something about possible organisms. Okay, great. All in favor of learning things. But there's no issues. It's just that the systems themselves... There are great claims by some of the leading figures in the field. We've solved the problem of language acquisition, namely zero contribution, because the systems work as well for impossible languages. Therefore, they can't be telling you anything about language acquisition, period. period. Maybe they're useful for something else. Okay, let's take a look. Well, maybe for the audience that this is going out to, you know, I understand what you mean by impossible, impossible, but could you just give a brief synopsis of what you mean by impossible languages for people that haven't read your work?

18:21Well, I mean, there are certain general properties that every infant knows, already tested down to two years old, no evidence, couldn't have evidence. So one of the basic properties of language is that the linguistic rules apply to structures, not linear strings. So, if you want to take a sentence like, instinctively, birds that fly swim. It means instinctively they swim, not instinctively they fly. Well, the adverb instinctively has to find a verb to attach to. it skips the closest verb and finds the structurally closest ones. That principle turns out to be universal for all structures, all constructions, and all languages.

19:30What it means is that an infant from birth, as soon as you can test, automatically disregards linear order and disregards 100 % of what it hears. notice, because all we hear is words in linear order. But you disregard that, and you deal only with abstract structures in your mind, which you never hear. Take another simple example. Take the friends of my brothers are in England. Who's in England? The friends or the brothers? the friends, not the brothers, the one that's adjacent. You just disregard all the linear information. It means you disregard everything you hear, everything, and you pay attention only to what your mind constructs.

20:25That's the basic, most fundamental property of language. Well, you can make up impossible languages that work with what you hear. Simple rule, take the first relevant thing, associate them. Friends of my brothers are here. Brothers are the closest things, and the brothers are here. Trivial rule, much simpler than the rule we use. You can construct languages that use only those simple rules that are based on the linear order of what we hear. Well, maybe children, people could acquire them as a puzzle somehow using non-linguistic capacities, but they're not what children, infants, reflexively construct with no evidence.

21:17There's many things like this, in possible and impossible languages. Nobody's tried it out because it's too obvious how it's going to turn out. You take a large language model, apply it to one of these systems that uses linear order, of course, it's going to work fine. Trivial rules. Well, that's a refutation of the system. Meaning that if you trained it on an impossible language, it would produce impossible language. How would you mean? You don't even have to train it because the rules are simple. Yeah. Rules are much simpler than the rules of language. Like taking things that are, take the example, the friends of my brother are here.

22:06The way we actually do it is we don't say, take the noun phrase that's closest. We don't do that. That would be trivial. We don't do it. What we say is first construct the structure in your mind, friends of my brothers, then figure out that the central element in that structure is friends, not brothers, and then let it be talking about the head of it. It's a pretty complicated computation, but that's the one we do instantaneously and reflexively. And we ignore, and we never see it, hear it, remember. We don't hear structures. All you hear is words in linear order. What we hear is words in linear order, we never use that information.

22:56We use only the much more looks-like complex. If you think about it computationally, it's actually simpler, but that's a deeper question, which is why we do it. To move to a different dimension, there's a reason for this. The reason has to do with theory of computation. You're trying to construct an infinite array of structured expressions. Simplest way to do that, the simplest computational procedure is binary set formation. But if you use binary set formation, you're just going to get structures, not order. So what the brain is doing is the simplest computational system, which happens to be very much harder to use.

23:47Nature doesn't care about that. Nature constructs the simplest system, doesn't care about it if it's hard to use or not. I mean, you know, nature could have saved us a lot of trouble if it had developed eight fingers instead of ten, then we'd have a much better base for computation. But nature didn't care about that when it developed ten fingers. If you look at evolution, it pays no attention to function. It just constructs the best system at each point. There's a lot of misleading talk about that. But if you just think about the physics of evolution, say a bacterium swallows another organism, the basis for what became complex cells.

24:40Nature doesn't get the new system. It reconstructs it in the simplest possible way. It doesn't pay any attention to how complex organisms are going to behave. It's not what nature can do. And that's the way evolution works all the way down the line. so not surprisingly nature constructed language so that it's computationally elegant but dysfunctional hard to use in many ways not nature's problem just like every other aspect of nature you can think of a way in which you can do it better but it didn't happen stage by stage two questions from that So your view is that artificial intelligence, as it's being called, and particularly generative AI, doesn't exhibit true intelligence.

25:44Is that right? I wouldn't even say that. It's irrelevant to the question of intelligence. It's not its problem. A guy who designs a jet plane is not trying to answer the question, how do eagles fly? So to say, well, it doesn't tell us how eagles fly is the wrong question to ask. It's not the goal. Except that what people are struggling with right now, you've heard the existential threat argument. that these models, if they get large enough, they'll actually be more intelligent than humans. That's science fiction. I mean, there is a theoretical possibility. You can give a theoretical argument that in principle, a complex system with vast search capacity could conceivably turn into something that would start to do things that you can't predict, maybe beyond.

26:56But that's even more remote than some distant asteroid maybe someday hitting the Earth. Yeah, it could happen. I mean, if you're reading serious scientists on this, like Max Stegmark, his book on the three levels of intelligence, he does give a sound theoretical argument as to how a massive system could, say, run through all the scientific discoveries in history, maybe find out some better way of developing them and use that better way to design something new which would destroy us all yeah it's in theory possible but it's so remote from anything that's available that it's a waste of time to think about it yeah so your view is that whatever threat exists from generative ai it's it's it's the more mundane threat of disinformation and just disinformation, defamation, gullibility.

28:13Gary Marcus has done a lot of work on this, real cases. Those are problems. I mean, you may have seen that there was a, sort of as a joke, people, somebody developed a defamation of the Pope, put an image of the Pope. Somebody could do it for you. duplicate your face so it looks more or less like your face, pretty much duplicate your voice, develop a robot that looks kind of like you, have you say some insane thing. It would be hard. Only an expert could tell whether it was you or not. It's like this was done already several times, but basically it's a joke. when powerful institutions get started on it.

29:08It's not going to be a joke. Yeah. And another argument that's swirling around these large language models is the question of sentience, of whether if the model is large enough, and this goes a little bit back to how there's a lot more going on in the brain than the neural network of the cerebral cortex, but that there is the potential for some kind of sentience, not necessarily equivalent to human sentience. These are vacuous questions. It's like asking, does a submarine really swim? You want to call that swimming? Yeah, it swims. You don't want to call it swimming? It's not a substantive question.

30:05Well, in the sense that it supports the view that there's no separation between consciousness and the material activities of the brain. There's a separation. That hasn't been believed since the 17th century. John Locke, after Newton's demonstration, said, well, leaves us only with the possibility that thinking is some property of organized matter. That's the 17th century. Yeah. Okay. But the belief in a soul and consciousness is something separate from the material biology. It persists. People who believe in all kinds of things. But within the rational part of the human species, once Newton demonstrated that the mechanical model doesn't work, there's no material universe in the only sense that was understood.

31:15Locke took the obvious conclusion and said, well, since matter, as Mr. Newton has demonstrated, has properties that we cannot conceive of, they're not part of our intuitive picture, since matter has those properties, organized matter can also have the property of thought. This was investigated all through the 18th century, ended up finally with Joseph Priestley, a chemist, philosopher, late 18th century, gave pretty extensive discussions of how a material, organized material object could have properties of thought. You can even find it in Darwin's early notebooks. It was kind of forgotten after that.

32:06rediscovered in the late 20th century as some radical new discovery. Astonishing hypothesis. Matter can think, yeah. Of course it can. In fact, we're doing it right now. But the only problem then is to find out what's involved in what we call thinking, what we call sentience, what are the properties of whatever matter is. We don't know what matter is, but whatever it turns out to be whatever constitutes the world. What physicists don't know, but whatever it is, is something. Organized elements of it can have various properties, like the properties that we are now using, properties that we call sentience.

32:56Then the question whether something else has sentience is as interesting as whether airplanes fly. If you're talking English, airplanes fly. If you're talking Hebrew, airplanes glide. They don't fly. It's not a substantive question. Yeah. Just what metaphors do we like? But what you're saying then is that neural nets may not be the engineering solution, but that eventually it may be possible to create a system outside of the human brain that can think, whatever thinking means. Can do what we call thinking. Thinking, yeah. But whether it thinks or not is like asking the airplanes fly, not a substantive question.

33:57we shouldn't waste time on questions that are completely meaningless going back to the history then uh you know minsky was very interested in the possibility of nets neural nets as a as a computational model and minsky's time it looked as if neural nets were the right place to look. Now I think it's not so obvious, especially because of Gallus' work, which is not accepted by most neuroscientists, but seems to me pretty compelling. Can you talk a little bit about that? Because I haven't read that, and I'm guessing our readers haven't, our listeners have. Gallison's not the only one. Roger Penrose is another Nobel Prize-winning physicist, but a number of people have pointed out, Gallison mostly, that have argued, I think, plausibly, that the basic component of a computational system, the basic element of essentially a Turing machine, cannot be constructed from neural nets.

35:16So you have to look somewhere else with a different form of computation. And he's also pointed out what in fact is true, that there's much richer computational capacity in the brain than neural nets. Even internal to a cell, there's massive computational capacity intracellular so maybe that's involved in computation and then there's by now some experimental work I think giving some evidence for this but it's a problem for neuroscientists to work on and I'm not an expert in the field I'm looking at it from the outside so don't take my opinion too seriously, but to me it looks pretty compelling.

36:06But whatever it is, neural nets or something else, some organization of them, of whatever is there, is giving us the capacity to do what we're doing. So if you're a scientist, what you do is approach it in two different ways. One is you try to find the properties of the system. What is the nature of the system? That's first step, kind of thing I was talking about before with structure dependence. What are the properties of the system that an infant automatically develops in the mind? And there's a lot of work on that. From the other point of view, you can say, what can we learn about the brain that relates to this?

36:54Actually, there is somewhere. So there is neurophysiological studies which have shown that for artificial languages that violate the principle that I mentioned, this structure-dependent principle, if you train people on those, the ordinary language centers don't function. You get diffuse functioning of the brain, which means they're being treated as puzzles, basically. So you can find some neurological correlates of some of the things that are discovered by looking at the nature of the phenotype. But it's very hard for humans for a number of reasons. We know a lot about the physiology of human vision, but the reason is because of invasive experiments with nonhumans, cats, monkeys, and so on.

38:00You can't do that for language. There aren't any other organisms. It's unique to humans. So there's no comparative studies. You can think of a lot of invasive experiments, which teach you a lot. You can't do them for ethical reasons. So, study of the neurophysiology of human cognition is a uniquely hard problem. In its basic elements, like language, it's just unique to the species. And in fact, a very recent development in evolutionary history, probably the last couple hundred thousand years, which is nothing. So, you can't do the invasive experiments for ethical reasons. You can think of them, but you can't do them, fortunately.

38:52And there's no comparative evidence. So, it's much harder to do. You have to do things like looking at blood flow in the brain, and MRI type things, electrical stimulation, looking from the outside, it's tough. It's not like doing the kind of experiments you can think of. So it's very hard to find out the neurophysiological basis for things like use of language, but it's one way to proceed. And the other way to proceed is learn more about the phenotype. It's like chemistry for hundreds of years. You just postulated the existence of atoms. Nobody could see them. Why are they there? Because unless they were atoms with Dalton's properties, you don't explain anything.