#426 – Edward Gibson: Human Language, Psycholinguistics, Syntax, Grammar & LLMs

27:06Well, you can just mess with different things in each language. And so in Russian, you have case markers. Right. On the end, which is these endings on the nouns, which tell you how it connects each noun connects to the verb, right? We don't have that in English. And so when I say, um, Mary kissed John. I don't know who the agent or the patient is except by the order of the words, right? In Russian, you actually have a marker on the end. If you're using a Russian name in each of those names, you'll also say is that, you know, a nominative, which is marking the subject or an accusative will mark the object.

27:38And you could put them in the reverse order. You could put accusative first as you could put subject, you could put the patient first and then the verb and then the the the subject. And that would be a perfectly good Russian sentence. And it was still mean Mary, I could say John kissed Mary, meaning Mary kissed John. As long as I use the case markers in the right way, you can't do that in English. And so I love the terminology of agent and patient and yeah, and the other ones you use those are sort of linguistic terms, correct? Those are those are for like kind of meaning, those are meaning and subject and object are generally used for position.

28:15So subject is just like the thing that comes before the verb and the object is when it comes after the verb. The agent is kind of like the thing doing it. I that's kind of what that means, right? The subject is often the person doing the action, right? The thing itself. Okay, this is fascinating. So how hard is it to form a tree in general? Is there a procedure to it? Like if you look at different languages, is it supposed to be a very natural like is it automatable? Or is there some human genius involved? I think it's pretty automatable at this point. People can figure out the words are they can figure out the morphemes, which are the technically morphemes are the the minimal meaning units within a language.

28:50Okay? And so when you say eats or drinks, it actually has two morphemes in English. There's there's the there's the root, which is the verb and then there's some ending on it, which tells you, you know, that's this third person, third person singular. So more things are more things are just the minimal meaning units within a language. And the word is just kind of the things we put spaces between English and they're a little bit more. They have the morphology as well. They have the endings, this inflectional morphology on the endings on the root. They modify something about the word that adds additional meaning.

29:19They tell you yeah, yeah. And so we have a little bit of that in English, very little, even much more in Russian, for instance. And but we have a little bit in English. And so we have a little on the on the nouns. You can say it's either singular or plural. And you can say same thing for verbs, like simple past tense, for example, it's like, you know, notice in English, we say drinks, you know, he drinks, but everyone else is I drink you drink, we drink it's unmarked in a way. And then but in the past tense, it's just drank for everyone. There's no morphology at all for past tense. It's if there is morphology, it's marking past tense, but it's kind of it's in irregular now.

29:53So we don't even, you know, it drink to drink, you know, it's not even a regular word. So in most verbs, many verbs, there's an ed, we kind of add so walk to walk. We add that to say it's the past tense. That I just happened to choose an irregular because the high frequency word and the high frequency words tend to have irregular as an English verb. What's an irregular? It's just there's there's in a rule. So drink to drink is an irregular drink drink. Okay, I love versus walk walk talk and there's a lot of irregular irregular in English. There's a lot of irregular in English. The frequent ones, the common words tend to be irregular.

30:27There's many, many more low frequency words and those tend to be those irregular ones. The evolution of the irregular is fascinating. It's essentially slang that's sticky because you're breaking the rules and then everybody use it and doesn't follow the rules and they say, screwed to the rules. It's fascinating. So you said it morphemes, lots of questions. So morphologies, what the study of morphemes? The morphologies is the connections between the morphemes onto the roots, the roots. So in English, we mostly have suffixes. We have endings on the words, not very much, but a little bit and as opposed to prefixes, some words, depending on your language, can have mostly prefixes, mostly suffixes or both.

31:09And then even languages, several languages have things called infixes where you have some kind of a general form for the root and you put stuff in the middle, you change the vowels. So in general, there's what two morphemes per word, one or two or three. Well, in English, it's one or two. In English, it tends to be one or two. There can be more. In other languages, a language like Finnish, which has a very elaborate morphology, there may be 10 morphemes on the end of a root. And so there may be millions of forms of a given word. Okay, I'll ask the same question over and over. But how does it just sometimes to understand things like morphemes?

32:01It's nice to just ask the question, how does these kinds of things evolve? So you have a great book studying sort of the, how the cognitive processing, how language use for communication, so the mathematical notion of how effective language is for communication, what role that plays in the evolution of language. But just high level, like how do we, how does a language evolve with where English is two morphemes or one or two morphemes per word, and then finish as infinity per word. So what, how does that, how does that happen? Is it just people? That's a really good question. Yeah, that's a very good question.

32:41It's like, why do languages have more morphology versus less morphology? And I don't think we know the answer to this. I know, I think there's just like a lot of good solutions to the problem of communication. So I believe, as you hinted that language is an invented system by humans for communicating their ideas. And I think we, it comes down to we label things we want to talk about. Those are the morphemes and words. Those are things we want to talk about in the world and we invent those things. And then we put them together in ways that are easy for us to convey, to process. But that's like a naive view.

33:24And it's naive. I think it's simple. Simple. Yeah. I think naive is, naive is an education that's an incorrect somehow is a trivial to too simple. I think it could very well be correct. But it's interesting how sticky it feels like two people got together. It just feels like once you figure out certain aspects of a language that just becomes sticky and the tribe forms around that language, maybe the language, maybe the triophones first in the language of alls. And then you just kind of agree and you stick to whatever that is. I mean, these are very interesting questions. We don't know really about how words, even words get invented very much about, you know, we don't really, I mean, assuming they get invented, they, we don't really know how that process works and how these things evolve.

34:11What we have is kind of a current picture, a current picture of a few thousand languages, a few thousand instances. We don't have any pictures of really how these things are evolving really. And then the evolution is massively confused by contact. So as soon as one language group, one group runs into another, we are smart. Humans are smart. And they take on whatever is useful in the other group. And so any kind of contrast, which you're talking about, which I find useful, I'm going to start using as well. So I worked a little bit in specific areas of words, in number words and in color words. And in color words, so we have in English, we have around 11 words that everyone knows for colors.

35:03And many more, if you happen to be interested in color for some reason or other, if you're a fashion designer or an artist or something, you may have many, many more words. But we can see millions. Like if you have normal color vision, normal tri -chromatic color vision, you can see millions of distinctions in color. So we don't have millions of words. The most efficient, no, the most detailed color vocabulary would have over a million terms to distinguish all the different colors that we can see. But of course, we don't have that. So it's somehow, it's been, it's kind of useful for English to have evolved in some way, to such as 11 terms that people find useful to talk about, you know, black, white, red, blue, green, yellow, purple, gray, pink, and I probably miss something there.

35:51Anyway, there's 11 that everyone knows. And depending on your, and but you could have different cultures, especially the non -industrialized cultures, and there'll be many fewer. So some cultures will have only two, believe it or not, the Dani and in Papua New Guinea have only two labels that the group uses for color. Those are roughly black and white. They are very, very dark and very, very light, which are roughly black and white. And you might think, oh, they're dividing the whole color space into, you know, light and dark or something. And that's not really true. They mostly just only label the black and the white things.

36:25They just don't talk about the colors for the other ones. And so, and then there's other groups. I've worked with a group called the Chamani down in Bolivia in South America. And they have three words that everyone knows. But there's a few others that are that that several people like, and that many people know. And so they have me, it's just kind of depending on how you count between three and seven words that the group knows. Okay. And again, they're black and white. Everyone knows those. And red, red is, you know, like that tends to be the third word that everyone that cultures bring in. If there's a word, it's always red, the third one.

37:03And then after that, it's kind of all bets are off about what they bring in. And so after that, they bring in a sort of a big blue green space group group. They have one for that. And then they have, and then, you know, different people have different words that they'll use for other parts of the space. So anyway, and it's probably related to what they want to talk and what they, not what they, not what they see, because they see the same colors as we see. So it's not like they have, they don't, they have a, a week, a low color palette and the things they're looking at. They're looking at a lot of beautiful scenery.

37:37Okay. A lot of different colored flowers and berries and things. And, you know, and so there's lots of things of very bright colors. But they just don't label the color in those cases. And the reason probably, we don't know this, but we think probably what's going on here is that what you do, why you label something is you need to talk to someone else about it. And why do I need to talk about a color? Well, if I have two things which are identical, and I want you to give me the one that's different. And the only way it varies is color. Then I invent a word, which tells you, you know, this is the one I want.

38:12So I want the red sweater off the rack, not the, not the green sweater, right? There's two. And so those, those things will be identical because these are things we made and they're dyed. And there's nothing different about them. And so in, in industrialized society, we have, you know, everything, everything we've got is pretty much arbitrarily colored. But you go to non industrialized group. That's not true. And so they don't, it's not only they're not interested in color. If you bring bright colored things to them, they like them just like we like them. Right colors are great. They're beautiful.

38:43They are, but they just don't need to, nobody to talk about them. They don't have. So probably color words is a good example of how language evolves from sort of function when you need to communicate the use of something. I think so. Then you kind of invent different variations. And basically, you can imagine that the evolution of a language has to do with what the early tribes doing. Like what, what they want it, what kind of problems are facing them? And they're quickly figuring out how to efficiently communicate the solution to those problems. Well, there's aesthetic or functional that kind of stuff running away from a mammoth or whatever.

39:19But you know, it's so I think what you're pointing to is that we don't have data on the evolution of language because many languages have formed a long time ago. So you don't get the chatter. We have a little bit of like old English to modern English because there was a writing system. And we can see how old English looked. So the word order changed, for instance, in old English to middle English to modern English. And so it, you know, we can see things like that. But most languages don't even have a writing system. So of the 7 ,000 only, you know, a small subset of those have a writing system.

39:52And even if they have a writing system, they it's not a very modern writing system. So they don't have it. So we just basically have for Mandarin for Chinese. We have a lot of a lot of evidence from from for long time and for English and not for much else. Not for men German a little bit, but not for a whole lot of like long term language evolution. We don't have a lot. We can have snapshots is what we've got of current languages. Yeah, I you get an inkling of that from the rapid communication and certain platforms like on Reddit. There's different communities. And they'll come up with different slang.

40:23Usually from my perspective, during by a little bit of humor, or maybe mockery or whatever it's, you know, just talking shit in different kinds of ways. And you could see the evolution of language there. Because I think a lot of things on the internet, you don't want to be the boring mainstream. So you like want to deviate from the proper way of talking. And so you get a lot of deviation. Rapid deviation. Then when communities collide, you get like just like you said, humans adapt to it. And you can see it through the lines of humor. I mean, it's very difficult to study, but you can imagine like a hundred years from now.

41:05Well, if there's a new language born, for example, we'll get really high resolution data on. I mean, English changing, English changes all the time. All languages change all the time. So, you know, as a famous result, but the Queen's English. So as opposed to be, you know, originally the proper way for the talk was sort of defined by whoever the Queen talked or the King, whoever was in charge. And so if you look at how her vowels changed from when she first became Queen in 1952 or 1953 when she was current, the first, I mean, that's Queen Elizabeth, who died recently, of course, until 50 years later, her vowels changed, her vowels shifted a lot.

41:48And so that, you know, even in the sounds of British English, in her, the way she was talking was changing. The vowels were changing slightly. So that's just in the sounds there's change. I don't know what's, you know, we're, we're, I'm interested. We're all interested in what's driving any of these changes. The word order of English changed a lot over thousand years, right? So it used to look like German. You know, it looks to be a verb final language with case marking and it shifted to a verb medial language. A lot of contact. So a lot of contact with French. And it became a verb medial language with no case marking.

42:22And so it became this, you know, verb verb initially thing. So, and so that's evolving. We had, it totally evolved. And so it may vary, well, I mean, you know, it doesn't evolve maybe very much in 20 years is maybe what you're talking about. But over 50 and a hundred years, things change a lot, I think. Well, now have good data. Yeah. Yeah. Which is great. Sure. Yeah. Can you talk to what is syntax and what is grammar? So you wrote a book on syntax. I did. I think you were asking me before about what, you know, how do I figure out what a dependency structure is? I'd say the dependency structures aren't that hard to generally.

42:53I think it's a lot of agreement of what they, of what they are for almost any sentence in most languages. I think people will agree on a lot of that. There are other parameters in the mix such that some people think there's a more complicated grammar than just a dependency structure. And so, you know, like, no, I'm Tromsky, he's the most famous linguist ever. And he, he is famous for proposing a slightly more complicated syntax. And so he, he invented phrase structure grammar. So he's well known for many, many things. But in the 50s and early 60s, like, but late 50s, he was basically figuring out what's called formal language theory.

43:33So, and he figured out sort of a framework for figuring out how complicated language, you know, a certain type of language might be so -called phrase structure grammars of language might be. And so he, his idea was that maybe we can, we can think about the complexity of a language by how complicated the rules are. Okay. And the rules will look like this. They will have a left hand side and it will have a right hand side. Something will, on the left hand side, we'll expand to the thing on the right hand side. So we'll say we'll start with an s, which is like the root, which is a sentence. Okay.

44:10And then we're going to expand to things, like, a noun phrase and a verb phrase is what he would say, for instance. Okay. And s goes to an NP and a VP is a kind of a a determiner and a noun, for instance, and a verb phrase is something else is a verb and another noun phrase and another NP, for instance, those are the rules of a very simple phrase structure. Okay. And so he proposed phrase structure grammar as a way to sort of cover human languages. And then he actually figured out that well, depending on the formalization of those grammars, you might get more complicated or less complicated languages.

44:48And so he could, he could, he said, well, you, these are, these are things called context free languages, that rule that he thought, you know, human languages, attend to be what he calls context free languages. And they, but there are simpler languages, which are so -called regular languages, and they have a more, a more constrained form to the rules of the of the phrase structure of these particular rules. So he, he basically discovered and kind of invented ways to describe the language. And then those are phrase structure, a human language. And he was mostly interested in English initially in his, his work in the 50s.

45:23So quick questions around all this. So formal language theory is the big field of just studying language formally. Yes. And it doesn't have to be human language there. We can have a computer languages, any kind of system, which is generating a, some set of expressions in a language. And those could be like the, the, you know, the statements in a, in a computer language, for example. So a formal, it could be that work, it could be human language. So technically you can study programming languages. Yes. And heaven, I mean, heavily studied using this formalism. There's a big field of programming languages within the formal language.

46:00Okay. And then phrase structure grammar is this idea that you can break down language into this S and P, V, P, type of thing. It's a particular formalism for describing language. Okay. So and, and Trump's, he was the first one. He's going to figure that stuff out back in the 50s. And, and, and, but he, and that's equivalent. Actually, the context for grammar is actually is kind of equivalent in the sense that it generates the same sentences as a dependency grammar would. You know, as the, the dependency grammar is a little simpler in some way. You just have a root.

46:37And we just have connections between words. The free structure grammars are kind of a different way to think about the dependency grammar. So it's slightly more complicated, but it's kind of the same in some ways. So to clarify, dependency grammar is the framework under which you see language in your case that this is a good way to describe language. That's correct. And, uh, no, no, no, jomsky is watching. This is very upset right now. So let's, uh, just kidding. But, uh, what's the difference between, uh, where's the, the place of disagreement, um, between phrase structure grammar and dependency grammar?

47:13They're very close. So phrase structure grammar and dependency grammar aren't that aren't that far apart. I, I like dependency grammar because it's more perspicuous, it's more transparent about representing the connections between the words. It's just a little harder to see in phrase structure grammar. You know, the, the place where jomsky sort of devolved or went off from, from, from this is he also thought there was, um, something called movement. Okay. And so, and that's where we disagree. Okay. That's the place where I would say we disagree. And, and, and, and I mean, well, maybe we'll get into that later.

47:44But the idea is, if you want to, do you want me to explain that? I would love to explain movement. Okay. So, it's saying so many interesting things. Yeah. Okay. So here's the movement is, jomsky basically sees English. And he says, okay, I said, um, you know, this is, we had that sentence earlier, like it was like two dogs entered the room. It's changed a little bit, say, two dogs will enter the room. And he notices that, hey, English, if I want to make a question, a yes, no question, from that same sentence, I, I say instead of two dogs will enter the room, I say, will two dogs enter the room?

48:15Okay. There's a different way to, you know, to say the same idea. And it's like, well, the auxiliary verb that will thing. It's at the front as opposed to in the middle. Okay. And so, and he looked, you know, if you look at English, you see that that's true for all those modal verbs and for other kinds of auxiliary verbs in English, you always do that. You always put an auxiliary verb at the front. And what he, when he saw that, so, you know, if I say, um, I can win this bet, can I win this bet? Right. So I move a can to the front. So actually, that's a theory. I just gave you a theory there. He talks about it as movement.

48:49That word in the decalitines, the declarative is the root is, is there's a sort of default way to think about the sentence, and you move the auxiliary verb to the front. That's a movement theory. Okay. So, and he's, he just thought that was just so obvious that it must be true. That, that, that there's nothing more to say about that. That this is how auxiliary verbs work in English. There's a movement rule such that you're move like to get from the declarative to the interrogative, you're moving the auxiliary to the front. And it's a little more complicated as soon as you go to simple, simple, simple present and simple past because, you know, if I say, you know, John slept, you have to say, did John sleep, not slept, John, right?

49:27And so you have to somehow get an auxiliary verb, and I guess, underlyingly, it's like slept is, it's a little more complicated than that, but his, that's his idea. There's a movement. Okay. And, and so a different way to think about that that isn't, I mean, the, then he ended up showing later. So he proposed this theory of grammar, which has movement. There's other places where he thought there's movement, not just auxiliary verbs, but things like the passive in English and things like a, questions, WH questions, a bunch of places where he thought there's also movement going on. And, and, and each, each one of those, he thinks there's words, well phrases and words are moving around from one structure to another, which he called deep structure to surface structure.

50:05I mean, there's like two different structures in his, in his theory. Okay. There's a different way to think about this, which is there's no movement at all. There's a, a lexical copying rule such that the word will or the word can these, the auxiliary verbs, they just have two forms. And, and, and one of them is the declarative and one of them is interrogative. And you basically have the declarative one. And, oh, I formed the interrogative or I can form one from the other, it doesn't matter which direction you go. And, and I just have a new entry, which has the same meaning, which has a slightly different argument structure, argument structure, it's a fancy word for the ordering of the words.

50:43And so if I say, you know, it was, um, the, the dogs, two dogs can or will enter the room. The, the, the, there's two forms of will. One is will declarative. And, and then okay, I've got my subject to the left. It comes before me. And the verb comes after me in that one. And then the will interrogative is like, oh, I go first. Interrogative will is first. And then I have the subject immediately after and then the verb after that. And so you just, you can just generate from one of those words, another word with a slightly different argument structure with different ordering. And these are just lexical copies that don't, they're not necessarily moving from one to another.

51:21There's no movement. There's a romantic notion that you have like one main way to use a word. And then you could move it around. Right, right. Which is essentially what movement is applying. Yeah, but that's, that's the lexical copying is similar. So then, so then, then we, we do lexical copying for that same idea that maybe the declarative is the source. And then we can copy it. And so in advantage, for, well, there's multiple adventures of the lexical copying story. It's not my story. This is like, I even saw linguists, a bunch of linguists have been proposing these stories as well, you know, in tandem with the movement story.

51:56Okay. You know, he's, he's, I even saw a dialogue ago, but he was a one of the proponents of the non -movement of the lexical copying story. And so that is that a great advantage is, well, Chomsky, really famously in 1971, showed that the movement story leads to learnability problems. It leads to problems for how language is learned. It's really, really hard to figure out what the underlying structure of a language is. If you have both phrase structure and movement, it's like really hard to figure out what came from what? There's like a lot of possibilities there. If you don't have that problem learning, the learning problem gets a lot easier.

52:36Just say there's lexical copies. Yeah. Yeah. Well, we say the learning problem. Do you mean like humans learning a new language? Yeah, just learning English. So baby is lying around listening to the crib, listening to me talk. And you know, how are they learning English? Or, or, you know, maybe it's a two -year -old who's learning, you know, interrogatives and stuff or one, you know, how are they doing that? Are they doing it from like a figuring out or like, you know, so Chomsky said it's impossible to figure it out. Actually, he said it's actually impossible, not not hard, but impossible. And therefore, that's where a universal grammar comes from.

53:10Is that it has to be built in. And so what they're learning is there's some built -in movement is built -in in his story, is absolutely part of your language module. And then you are, you're just setting parameters. You're, you're said, depending on English, it's just sort of a variant of the universal grammar. And you're figuring out, oh, which orders does English do these things? That's the, the non -movement story doesn't have this. It's like much more bottom -up. You're learning rules. You're learning rules one by one. And oh, there's this word is connected to that word. A great advantage.

53:46Another advantage is learnable. Another advantage of it is that it predicts that not all exileries might move, like it might depend on the word, depending on whether you, and that turns out to be true. So there's words that that don't really work as a exhilarated, they work in declarative and not an interrogative. So I can say, I'll give you the opposite first. I can say, aren't I invited to the party? Okay. And that's an interrogative form. But it's not from I aren't invited to the party. There is no I aren't. Right. So that's, that's interrogative only. And, and then we also have forms like, um, ought, uh, I, I ought to do this.

54:28And, and I guess some British old British people can say, I, exactly, it doesn't sound right. Does it? For me, it sounds ridiculous. I don't even think ought is great. But I mean, I totally recognize I ought to do it. I do not do bad. Actually, I can say I ought to do this. That sounds very good. Yeah. If I'm trying to sound sophisticated, maybe, I don't know. It just sounds completely up to me. I, anyway, it's, it's, so there are variants here. And a lot of these words just work in one versus the other. And, and that's like, fine, under the lexical copying story. It's like, well, you just learn the usage.

54:58Whatever the usage is, is what you, is what you do with this, with, with you with this word. But, um, it doesn't, it's a little bit harder in the movement story. The movement story, like that's an advantage. I think of lexical copying in all these different places, there is, there's all these usage variants, which make the movement story, um, a little bit harder to work. So one of the main divisions here is the movement story versus the last time, copy story that has to do about the auxiliary words and so on. But you rewind to the phrase structure grammar versus dependency grammar. Those are equivalent, in some sense, in that for any dependency grammar, I can generate a dependency, phrase structure grammar, which generates exactly the same sentences.

55:41I just, I just like the dependency grammar, uh, formalism, because it makes something really salient, which is the dependent, the lengths of dependencies between words, which isn't so obvious in the phrase structure. In the phrase structure, it's just kind of hard to see. It's in there. It's just very, very, it's opaque. Technically, I think phrase structure grammar is mapable to dependency grammar. And vice versa. In vice versa. Yeah, yeah. There's like these like little labels, S and P, V, P. Yeah. For a particular dependency grammar, you can make a phrase structure grammar, which generates exactly those same sentences and vice versa.

56:17But there are many phrase structure grammars, which you can't really make a dependency grammar. I mean, there, there, you can do a lot more in a phrase structure grammar, but you get many more of these extra nodes, basically. You, you can have more structure in there. Uh, and some people like that. And maybe there's value to that. I, I, I don't like it. Well, for you, so, it was a clarifies of, so dependency grammar is just, uh, well, one word depends on only one other word and you form these trees. Yes. And that makes, it really puts priority on those dependencies, just like as a, as a tree that you can then measure the distance of the dependency from what to work to the other, they can then map to the cognitive processing of the, of the sentences, how well, how easy it is to understand and all that kind of stuff.

57:03So it just puts the focus on just like the mathematical, um, uh, distance of dependence between words. So like, it's just a different focus. Absolutely. Just continue on a thread of jobs, because it's really interesting, because it, as you're discussing this agreement to the degree, there's this agreement, you're also telling the history of the study of language, which is really awesome. So the matching context free versus regular. Does that distinction come into play for dependency grammars? No, not at all. I mean, the regular, regular languages are too simple for human languages. They are, they, it's a part of the hierarchy, but human languages are in, in the phrase structure world are definitely, they're, at least context free, maybe a little bit more, a little bit harder than that.

57:54But, uh, so there's something called context sensitive as well, where you can have, like, this is just the formal language description in a context free grammar. You have one, this is like a bunch of like formal language theory. We're doing here, but I love it. Okay. So you have, you have a left hand side category, and you're expanding to anything on the right is a, uh, that's a context free. So like, the idea is that that category on the left expands in independent of context to those things, whatever they are on the right, it doesn't matter what. And, and a context sensitive says, okay, I actually have more than one thing on the left.

58:30I can tell you only in this context, you know, I have maybe you have like a left in a right context or just a left context or a right context, I have two or more stuff on the left tells you how to expand that those things in that way. Okay. So it's context sensitive. A, a regular language is just more constrained. And so it doesn't allow anything on the right. It allows very, it allows, it's a one very complicated rule is kind of what a regular language is. And so it doesn't have any, um, I'll just say the long distance dependencies. It doesn't allow recursion, for instance, there's no recursion.

59:05Yeah, recursion is where you, which is human languages have recursion. They have embedding and you can't, well, it doesn't allow center embedded recursion, which human languages have, which is what center embedded recursion within a sentence, within a sentence. Yeah, within a sentence. So here we're going to get to that. But I, you know, the formal language stuff is a little aside. It was, Chomsky wasn't proposing it for human languages, even he was just pointing out that human languages are context free. And then he was most in for for human, because that was kind of stuff we did for formal languages.

59:32And what he was most interested in was human language. And that's like the movement is where we, we, we, where he sort of set off in on the, I would say, I'm very interesting, but wrong foot. It was kind of interesting. It's a very, I agree, it's kind of, it's a very interesting history. So there's this, so he proposed this multiple theories in 57 and then 65. They, they all have this framework, though, was phrase structure plus movement, different versions of the, of the phrase structure and the movement in the 57. This is the most famous original bits of Chomsky's work. And then 71 is when he figured out that those lead to learning problems, that, that there's cases where a kid could never figure out which rule, which set of rules was intended.

1:00:14And, and so, and then he said, well, that means it's an eight. It's kind of interesting. He just really thought the movement was just so obviously true, that he couldn't, he didn't even entertain giving it up. It's just obvious that that's obviously right. And it was later where people figured out that there's all these like subtle ways in which things, which, which look like generalizations, aren't generalizations and they, you know, across the category, they're, they're words specific and they have, and they, they kind of work, but they don't work across various other words in the category. And so it's easier to just think of these things as lexical copies.

1:00:47And, and I think he was very obsessed, I don't know, I'm just guessing that he, he just, he really wanted this story to be simple in some sense. And language is a little more complicated in some sense, you know, he didn't like words. He never talks about words. He likes to know about combinations of words. And words are, you know, look up a dictionary. There's 50 senses for a common word, right? The word take will have 30 or 40 senses in it. So, there'll be many different senses for common words. And he just doesn't think about that. It doesn't think that's language. I think he doesn't think that's language.

1:01:21He thinks that words are distinct from combinations of words. I think they're the same. If you look at my brain on in the scanner, while I'm listening to a language I understand, and you compare, I can localize my language network in a few minutes, in like 15 minutes. And what you do is I listen to a language I know, I listen to, you know, maybe some language I don't know, or I listen to muffled speech, or I read sentences, and I read non words. Like I can do anything like this. Anything that's sort of really like English, and anything that's not very like English. So I've got something like it and not, and I got a control.

1:01:56And the voxels, which is just, you know, the 3D pixels in my, in my brain, that are responding most are, are, is a language area. And, and that's this left lateralized area in my head. And, and wherever I look in that network, if you look for the combinations versus the words, it's, it's, it's, it's, it's the same. That's fascinating. And, and so it's like hard to find, there are no areas that we know. I mean, that's, it's a little overstayed right now. At this, at this point, the technology isn't great. It's not bad, but we have the best, the best way to figure out what's going on in my brain when I'm listening or reading language is to use FMRI, functional magnetic resonance imaging.

1:02:38And that's a very good localization method. So I can figure out where exactly these signals are coming from. Pretty, you know, down to, you know, millimeters, you know, cubic millimeters are smaller. Okay, very small. We can figure those out very well. The problem is the when, okay, it's, it's measuring oxygen, okay, and oxygen takes a little while to get to those cells. That's what takes on the order of seconds. So I talk fast. I probably listen fast and I can probably understand things really fast. So a lot of stuff happens in two seconds. And so to say that we know what's going on, that the words right now in that network are best guesses that whole network is doing something similar, but maybe different parts of that network are doing different things.

1:03:22And that's probably the case. We just don't have very good methods to figure that out right at this moment. And so since we're kind of talking about the history of the study of language, what other interesting disagreements, and you're both at MIT or were for a long time, what kind of interesting disagreements their attention of ideas are there between you and no chance game. We should say that known was in the linguistics department. And you're, I guess for time, we're affiliated there, but primarily a brain and cognitive science department. Which is another way of studying language. And you've been talking about FMRI.

1:03:59So like, what is there something else interesting to bring to the surface about the disagreement between the two of you?

1:04:12Or other than the two of you? And so I think that's the biggest difference between me and no, is that I gather data from people. I do experiments with people and I gather corpus data. Whatever corpus data is available and we do quantitative methods to evaluate any kind of hypothesis we have. He just doesn't do that. And so, you know, he has never once been associated with any experiment or corpus work ever. And so it's all thought experiments. It's his own intuitions. So I just don't think that's the way to do things. That's a, that's a, you know, across the street. They're across the street from us kind of difference between brain and cox -ci and linguistics.

1:05:12I mean, not all linguists, some of the linguists, depending on what you do, more speech -oriented. They do more quantitative stuff. But in the, in the meaning words and, well, it's combinations of words in texamantics. They tend not to do experiments and, and corpus analyses. So, in the linguistic exercise, probably, well, the, but the method is a symptom of a bigger approach, which is sort of a psychology philosophy side unknown for you. It's more sort of data -driven, so almost like mathematical approach. Yeah, I mean, I'm psychologist. So I would say we're in psychology. You know, I mean, in brain cognitive sciences is MIT's old psychology department.

1:05:51It was a psychology department up until 1985 and it became the brain cognitive science department. And so I, I mean, my training isn't in psychology. I mean, my training is math and computer science, but I'm a psychologist. I mean, I mean, I don't know what I am. So data -driven, psychologist. Yeah, yeah, you are. But I'm having to call the linguist. I'm having to be called a computer scientist. I'm having to be called psychologist. Any of those things. But in the actual, like how that manifests itself outside of the methodology is like these differences, these subtle differences about the movement story versus the lexical copy story.

1:06:23Yeah, those are theories. Right? So the theory is like the theories are, but I think the reason we differ in part is because of how we evaluate the theories. And so I evaluate theories quantitatively and no, I'm doesn't. Got it. Okay, well, let's explore the theories that you explore in your book. Let's return to this dependency grammar framework of looking at language. What's a good justification? Why the dependency grammar framework is a good way to explain language. What's your intuition? So the reason I like dependency grammar, as I've said before, is that it's very transparent about its representation of distance between words.

1:07:06So it's like, all it is is you've got a bunch of words. You're connecting them together to make a sentence and a really neat insight, which turns out to be true, is that the further apart the pair of words are that you're connecting the harder it is to do the production, the harder it is to do the comprehension. It says harder to produce, hard to understand when the words are far apart, when they're close together, it's easy to produce and it's easy to comprehend. Let me give you an example. Okay, so we have in any language, we have mostly local connections between words, but they're abstract.

1:07:41The connections are abstracted between categories of words. And so you can always make things further apart if you add modification, for example, after a noun, so a noun in English comes before verb. The subject noun comes before verb, and then there's an object after, for example. So I can say what I said before, you know, the dog entered the room or something like that. So I can modify dog. If I say something more about dog after it, then what I'm doing is indirectly, I'm lengthening the dependence between dog and entered by adding more stuff to it. So I just make it explicit here if I say the boy who the cat scratched cried.

1:08:27We're going to have a mean cat here. And so what I've got here is I get the boy cried. It would be a very short, simple sentence. And I just told you something about the boy, and I told you it was the boy who the cat scratched. So the cry is connected to the boy. The cry at the end. Yeah. It's connected to the boy in the beginning. Right. And so I can do that. I can say that. That's a perfectly fine English sentence. And I can say the cat, which the dog chased ran away or something. Okay. I can do that. But it's really hard. So it's really hard now. I've got whatever I have here. The boy who the cat now let's say I try to modify cat.

1:09:06Okay. The boy who the cat, which the dog chased scratched ran away. Oh my god, that's hard, right. I can, I'm sort of just working that through my head how to produce and how to and it's really just horrendous to understand. It's not so bad. At least I've got intonation there to sort of mark the boundaries and stuff. But it's that's really complicated. That's sort of English in a way. I mean, that follows the rules of English. But so what's interesting about that is is that what I'm doing is nesting dependencies. There I'm putting one. I've got a subject connected to a verb there. And then I'm modifying that with a clause, another clause, which happens to have a subject in a verb relation.

1:09:46I'm trying to do that again on the second one. And what that does is it lengthens out the dependence, multiple dependence actually get lengthened out there. The dependencies get get longer longer on the outside ones get long. And even the ones in between get kind of long. And you just so what's fascinating is that that's bad. That's really horrendous in English. But that's horrendous in any language. And so in no matter what language you look at, if you do just figure out some structure where I'm going to have some modification following some head, which is connected some later head. And I do it again, it won't be good.

1:10:19It guaranteed like 100 % that will be uninterpretable in that language. In the same way that was uninterpretable in English. So clarify the distance of the dependencies is whenever the boy cried, there's a dependence between two words. And then you counting the number of what morphines between them. That's a good question. I just say words. Your words are morphines between don't we don't know that. Actually, that's a very good question. What is the distance metric? But let's just say it's words. Sure. Okay. So that and you're saying the longer the distance is that dependence the more no matter the language except legalese.

1:10:58Even legally. We'll talk about it. Okay. Okay. Okay. But that the people will be very upset that speak that language. Not upset, but they'll either not understand it. There'd be like this is they'll their brain will be working in overtime. Yeah. They would have a hard time either producing your comprehending it. They might tell you that's not their language. You know, it's sort of the language. I mean, it's following their like they'll agree with each of those pieces as part of the language. But somehow that combination will be very, very difficult to produce and understand. Is that a chicken or the egg issue here?

1:11:29So like is well, I'm giving you an explanation. Right. So the egg, I mean, I mean, I didn't there's I'm giving you two kinds of explanations. I'm telling you that centrum betting that's nesting. Those are the same those are synonyms for the same concept here. And I'm the explanation for what those are always hard centrum betting and nesting are always hard. And I give you an explanation for why they might be hard, which is long distance connections. There's a when you do centrum betting, where you do nesting, you always have long distance connections between the dependence you just it. And so that's not necessarily the right explanation.

1:11:59It just happened. I can go through reasons why that's probably a good explanation. And it's not really just about one of them. It's so probably it's a pair of them or something of these dependents that you get get long that drives you to like be really confused in that case. And so the behavioral consequence there, I mean, we this is kind of methods like how do we get at this? You could try to do experiments to get people to produce these things. They're going to have a hard time producing them. You can try to experiments to get them to understand them and you get you see how well they understand them.

1:12:30Can they understand them? Another method you can do is give people partial materials and ask them to complete them. You know, those those centrum betted materials and they they'll fail. So I've done that. I've done all these kinds of things. So what I mean it's so so central betting meaning like you take a normal sentence like boy cried and inject a bunch of crap in the middle. Yes. That separates the boy in the cried. Okay, that central betting and nesting is on top of that. No, nesting is the same thing. Central betting are totally equivalent terms. I'm sorry. I sometimes use one or sometimes got it.

1:13:04They don't need anything different. Got it. And then what you're saying is there's a bunch of different kinds of experiments you can do. I mean, I like to understand anyone is like have more embedding more central betting. Is it easier or harder to understand, but then you have to measure the level of understanding I guess. Yeah, yeah, you could. I mean, there's multiple ways to do that. I mean, there's there's the simplest ways just ask people how good is it sound? How natural is this sound? That's a very blunt but very good measure. It's very very reliable. People will do the same thing. And so it's like, I don't know what it means exactly, but it's doing something such that we're measuring something about the confusion, the difficulty associated with those.

1:13:38And those like those are giving you a signal. That's why you can say them. Okay, what about the completion of this central bet? So if you give them a partial sentence, say I say the book which the author who, and I ask you to now finish that off, I mean, either say it. Yeah, yeah, but you can just say it's written in front of you. And you can just type and have much time as you want. They will, even though that one's not too hard, right? So if I say it's like the book is like, oh, the book which the author who I met wrote was good. You know, that's a very simple completion for that. If I give that completion on online somewhere to a, a crowdsourcing platform and ask people to complete that, they will miss off of a verb very regular.

1:14:24Like half a time, maybe two thirds of the time, they'll just leave off one of those verb phrases, even with that simple, so to say the book which the author who, and they'll say, was, they won't, you need three verbs, right? I need three verbs or who I met wrote was good. And they'll give me two. They'll say who was, who was famous was good or something like that. They'll just give me two. And that, and that'll happen about 60 % of the time. So 40%, maybe 30, they'll do it correctly, you know, correctly, meaning they'll do with three verb phrases. I don't know what's correct or not, you know, it's hard.

1:14:59It's a hard task. Yeah, I get, actually, I'm struggling with it in my head. Well, it's easier when you when you look, when you look there at it, if you look at a little easier, then listening is pretty tough because you have to, because there's no trace of it, you have to remember the words that I'm saying, which is very hard auditorily. We wouldn't do it this way. We do it written. You can look at it and figure it out. It's easier in many dimensions in some ways, depending on the person. It's easier to gather written data for, I mean, most sort of cycle, I work in cycle linguistics, right? Psychology of language and stuff.

1:15:29And so a lot of our work is based on written stuff because it's so easy to gather data from people doing written kinds of tasks. Spoken tasks are just more complicated to administer and analyze because people do weird things when they speak. And it's harder to analyze what they do. But they generally point to the same kinds of things. So, okay, so the universal theory of language by Ted Gibson is that you can form dependency, you can form trees from any senses. That's right. You can measure the distance in some way of those dependencies. And then you can say that most languages have very short dependencies.

1:16:10All languages. All languages. All languages have short dependencies. You can actually measure that. So, a next student of mine is guys at University of California Irvine, Richard Futrell did a thing a bunch of years ago now, where he looked at all the languages we could look at, which was about 40 initially. And now I think there's about 60 for which there are dependency structures. So, there are meaning that it's got to be like a big text, a bunch of texts, which have been parsed for the dependency structures. And there's about 60 of those, which have been parsed that way. And for all of those, you can, what he did was take any sentence in one of those languages.

1:16:48And you can do the dependency structure. And then start at the root. We were talking about dependency structures. That's pretty easy now. And he's trying to figure out what a control way you might say the same sentence is in that language. And so we just just like, all right, there's a root. And it has, let's say as a sentences, let's go back to two dogs entered the room. So, entered is the root. And entered has two dependents that's got dogs and it has room. Okay. And what he does is like, let's scramble that order. That's three things, the root and the head and the two dependents and into some random order, just random.

1:17:23And then just do that for all the dependents down the two. So now look, do it for the, and whatever was two in dogs and for room. And that's, you know, that's not a very short sentence. When sentences get longer and you have more dependents, there's more scrambling that's possible. And when he found what so that so that that's one, you can figure out one scrambling for that sentence. He did it like a hundred times for every sentence in every corp in every one of these texts, every corpus. And then he just compared the dependency lengths in those random scramblings to what actually happened with what the English or the French or the German was in the individual language or Chinese or what all these like 80 like, you know, 60 languages.

1:18:01Okay. And the dependency lengths are always shorter in the real language compared to these, this kind of a control. And there's another that he, it's a little more rigid his control. So the way I described it, you could have crossed dependencies like that by scrambling that way. You could scramble in any way at all. Languages don't do that. They can not to cross dependencies very much. Like so the dependency structure, they just, they tend to keep things non -crossed. And there's a, you know, there's a technical term they call that projective, but it's just non -crossed is all that is projective.

1:18:36And so if you just constrain the scrambling so that it only gives you projective sort of non -crossed is the same thing holds. So it's so the you still still human languages are much shorter than these, this kind of a control. So there's like, it what it means is that that we're in every language, we're trying to put things close relative to this kind of a control. Like there, it doesn't matter about the word order, some of these are verb final, some of them use a verb, media like English. And some are even verb initial. There are a few languages of the world, which have VSO, world order, word order, verb subject, object language.

1:19:10I haven't talked about those. It's like 10 % of the, and even in those languages, it's still short dependencies. Short dependencies is rules. Okay, so what, what are some possible explanations for that, for why languages have evolved that way? So that's one of the, as opposed to disagreements, you might have Wichowski. So you consider the evolution of language in terms of information theory. And for you, the purpose of language is ease of communication, right, in processing. That's right. That's right. So I mean, the story here is just about communication. It is just about production, really. It's about ease of production is the story.

1:19:53Will you say production? Can you? Oh, I just mean each of language productions. Easier for me to say things when the, come, when I'm doing, whenever I'm talking to you, is somehow I'm formulating some idea in my head and I'm putting these words together. And it's easier for me to do that to put, to say something where the words are closely connected and it depends the, as opposed to separated by putting something in between and over and over again. It's just hard for me to keep that in my head. Like that's the whole story. Like the story, it's basically the dependency grammar sort of gives that to you, like just like long, long as bad, short as good.

1:20:29It's like easier to keep in mind because you have to keep it in mind for probably for production, probably matters in comprehension as well. Like also matters in comprehension. Some both sides of the production and the but I would guess it's probably evolved for production. Like it's about producing. It's about what's easier for me to say that ends up being easier for you also. I, that's a very hard to disentangle this idea of who's it for. Is it for me, the speaker or is it for you, the listener? I mean part of my language is for you. Like the way I talk to you is going to be different from how I talk to different people.

1:21:00So I'm, I'm definitely angling what I'm saying to who I'm saying, right?

1:21:10It's audience. But how does that, does that work itself out in the, in the dependency length differences? I don't know. Maybe that's about just the words that part, you know, which words I select. My initial intuition is that you optimize language for the audience. Yeah. But it's just kind of like messing with my head a little bit to say that some of the optimization might be, maybe the primary objective of the optimization might be the ease of production. Yeah. We have different senses, I guess. I'm like very selfish. And you're like, I don't think it's like it's all about me. I'm like, I'm just doing this easiest for me.

1:21:46I don't want to, I'm like, I'll, I mean, but I have to of course choose the words that I think you're going to know. I'm not going to choose words you don't know. In fact, I'm going to fix that when I, you know, so there it's about, but, but maybe for for the syntax for the combinations, it's just about me. I feel like it's, I don't know though. It's great. But the purpose of communication is to be understood is to convince others and so on. So like the selfish thing is to be understood. It's about the list. It's a little circular there too, then. Okay. Right. I mean, like the ease of production helps me be understood.

1:22:18Then. I don't think it's circular. So I think the primary objective is to be understood is about the listener. Because otherwise, if you're optimizing for the ease of production, then you're, you're not going to have any of the interesting complexity of language. Like you're trying to like, it's control for what it is I want to say. Like I, I'm saying, let's control for the thing, the, the message control for the message. I mean, the message needs to be understood. That's the goal. Well, that's the meaning. So I'm still talking about the form. Just the form of the meaning. How do I frame the form of the meaning is all I'm talking about?

1:22:52You're talking about a harder thing, I think, is like, how am I like trying to change the me? I like, let's, let's keep the meaning constant. Like which you got it. You have you keep the meaning constant. How can I phrase whatever it is I need to say? Like I got to pick the right words. And I'm going to pick the order so that it's, so it's easy for me. I think I'm still tying meaning and form together in my head. But you're saying if you keep the meaning of you're saying constant, what the optimization, yeah, it could be the primary objective of that optimization is the, for production. That's interesting.

1:23:29I'm struggling to keep constant the meaning is just so, I mean, I'm such a human, right? So for me, the form without having introspected on this, the form and the meaning are tied together, like, deeply, because I'm a human. Like for me, when I'm speaking, because I haven't thought about language, like in a rigorous way about the form of language. But look, for any event, there's, there's an unbounded, I don't want to see infinite, but sort of unbounded ways of that I might communicate that same event. This two dogs entered a room, I can say in many, many different ways. I can say, hey, there's two dogs.

1:24:12They entered the room. Hey, the room was entered by something. The thing that was entered was two dogs. I mean, there's, I mean, it's kind of awkward and weird stuff. But those are all similar messages with different forms, different ways I might frame. And of course, I use the same words there all the time. I could have referred to the dogs as, you know, a Dalmatian and a Poodle or something. You know, I could have been more specific or less specific about what they are. And I could have said, been more abstract about, about the number. There's like, so I, like, I'm trying to keep the meaning, which is this event constant.

1:24:46And then how am I going to describe that to get that to you? It kind of depends on what you need to know, right? And what I think you need to know. But I'm like, let's go check control for all that stuff. And not, and, and the way I'm just like choosing, but I'm doing something simpler than you're doing, which is just forms. Yes, just words to use specifying the species of the breed of dog and whether they're cute or not is changing the meaning. That might be, yeah, yeah, that would be changing. Well, that would be changing the meaning for sure. Right. So you're just, yeah, yeah, yeah, yeah, that's changing the meaning.

1:25:17But say, even if we keep that constant, we can still talk about what's easier hard for me, right? The listener and the, and the, right, which phrase structures I use, which combinations, which, you know, this is so fascinating and just like a, a really powerful window into human language. But I wonder still throughout this, how vast the gap between meaning and form, I just, I just have this like, maybe romanticized notion that they're close together, that they evolve close to like hand in hand, that you can't just simply optimize for one without the other being in the room with us. Like, it's, well, it's kind of like an iceberg.

1:26:00Form is the tip of the iceberg and the rest, the, the meaning is the iceberg, but you can't like, set, but I think that's why these large language models are so successful, because they're good at form and form isn't that hard. In some sense, and meaning is tough still. And that's why they're not, they're, you know, they don't understand what they're, we're going to talk about that later, maybe, but, like we can distinguish in our, forget about large language models, like humans, where maybe you'll talk about that later too, is like the difference between language, which is a communication system and thinking, which is meaning.

1:26:33So language is a communication system for the meaning. It's not the meaning. And so that's why, I mean, that, and there's a lot of interesting evidence we can talk about relevant, relevant to that. Well, I mean, that's a really interesting question. What is the different, what is the difference between language written, communicated versus thought? What to use the difference between them? Well, you or anyone has to think of a task, which they think is, is a good thinking task. And there's lots and lots of tasks, which should be good thinking tasks. And whatever those tasks, let's say it's, you know, playing chess or that's a good thinking task or playing some game, we're doing some complex puzzles, maybe, maybe remembering some digits that's thinking, remembering some, a lot of different tasks we might think, maybe just listening to music is thinking, or there's a lot of different tasks we might think of as thinking.

1:27:27There's a woman in my department at Federico and she's done a lot of work on this question about what's the connection between language and thought? And so she uses, I was referring earlier to MRI, FMRI, that's her primary method. And so she has been really fascinated by this question about whether what language is. Okay. And so as I mentioned earlier, you can localize my language area, your language area in a few minutes. Okay. Like 15 minutes, I can listen to language, listen to non -language or backward speech or something. And we'll find areas left lateralized network in my head, which is especially, which is very sensitive to language as opposed to whatever that control was.

1:28:07Okay. Can you express how I mean by language, like communicating language? Just sentences. You know, I'm listening to English of any kind, story, or it can read sentences, anything at all that I understand, if I understand it, then it'll activate my language network. So right now my language network is going like crazy when I'm talking and when I'm listening to you, because we're both communicating. And that's pretty stable. Yeah, it's incredibly stable. So I've, I happened to be married to her this woman at FMRI, so I've been scanned by her over and over and over since 2007 or six or something.

1:28:39And so my language network is exactly the same, you know, like a month ago as it was back in 2007. It's amazingly stable. It's astounding. And with that, it's, it's a really fundamentally cool thing. And so my language network is, it's like my face. Okay. It's not changing much over time inside my head. Can I ask a quick question? Sorry, is a small tangent? Or at which point in the, as you grow up from baby to adult, does it stabilize? We don't know. Like that's a, that's a very hard question. They're working on that right now because of the problem scanning little kids, like doing the, trying to do local, trying to do the, the localization on little children in this scanner where you're lying in the FMRI scan.

1:29:20That's the best way to figure out where something's going on inside our brains. And the scanner is loud and you're in this tiny little, you know, area, your claustrophobic. And it doesn't bother me at all. I can go sleep in there. But some people are bothered by it. A little kids don't really like it. And they don't like to lie still. And you have to be really still because you move around that, that messes up the coordinates of where, where everything is. And so, you know, try to get, you know, your question is, how and when are language developing? You know, how, when, how does this left lateralized system come to play?

1:29:49Where's, you know, and it's really hard to get a two year old to do this task. But you can maybe, they're starting to get three and four and five year olds to do this task for short periods. And it looks like it's there pretty early. So clearly when you lead up to like a baby's first words before that, there's a lot of fascinating turmoil going on about like figuring out like, what are, what are these people saying? And you're trying to like make sense. How does that connect to the world? No, like kind of stuff. Yeah, that might be just fascinating development that's happening there. That's, yeah, hard to introspect.

1:30:21Anyway, you, we've we're back to the scanner. And I can find my network in 15 minutes. And now we can ask a, we can ask find my network find yours find, you know, 20 other people do this task. And we can do some other tasks. Anything else you think is thinking of some other thing. I can do a spatial memory task. I can do a music perception task. I can do programming task. If I program, okay, I can do what where I can like understand computer programs. And none of those tasks tap the language network at all like at all. There's no overlap. They do, they're highly activated in other parts of the brain.

1:30:57There's a, there's a bilateral network, which I think she tends to call the multiple demands network, which does anything kind of hard and sort of anything is kind of difficult in some ways will activate that multiple demands network. I mean, music will be in some music area. You know, there's music specific kinds of areas. And so, but they're, but but none of them are activating the language area at all unless there's words. Like so if you have music and there's a song and you can hear the words then then then you get the language area. We're talking about speaking and listening, but are, or we also talking about reading.

1:31:30This is all comprehension of any kind. And so, that is fast. So what this this this network doesn't make any difference if it's written or spoken. So the the the thing that she calls, Federico calls the the language network is this high level language. So it's not about the spoken the spoken language. And it's not about the written language is about either one of them. And so we're so when you do speech, you're sort of list you're you either you're listening to speech and you're you're subtract away some language you don't understand and so we're just or you subtract away back, backward speech which signs sounds like speech, but it isn't.

1:32:01And and then so you you take away the sound part altogether. And so and then if you do written, you get exactly the same network. So for just reading the language versus reading sort of nonsense words or something like that, you'll find exactly the same network. And so it's about high level. The comprehension of language. Yeah. In this case and the same thing happened. The productions a little harder to run the scanner, but the same thing happens in production. You get the same network. So productions a little harder. You have to figure out how do you run a task, you know, in the network such that you're doing some kind of production.

1:32:31And I can't remember what they've done a bunch of different kinds of tasks there where you get people to control produce things. Yeah, figure out how to produce. And the same network goes on there. It's actually the same place. And so you wait, wait, so if you read random words, yeah, you need things like like jibberish. Yeah, yeah, Lewis Carroll's, it was brilliant. Geberwaki, right? They call that Jabberwaki speech. The network doesn't get activated. Not as much. There are words in there. Yeah, because it's like there's function words and stuff. So it's lower activation. Yeah, yeah. So there's like the more language like it is, the higher it goes in the language network.

1:33:06And that network is there from when you speak from as soon as you learn language. And and it's it's there. Like you speak multiple languages. The same network is going for your multiple languages. So you speak English, you speak Russian. The the both of them are hitting that same network. If you if you're fluent in those languages, programming, not at all. It's not amazing. Even if you're a really good programmer, that is not a human language. It's just not conveying the same information. And so it is not in the language network. And so that as mind -blowing as I think that's pretty cool. That's weird.

1:33:39That's weird. It's amazing. And so that's like one set of day. This is hers like shows that what you might think is thinking is is not language. Language is just the seek just just this conventionalized system that we've worked out in in human languages. Oh, another fascinating little bit to bit is that even if they're these constructed languages like Klingon or I don't know the languages from Game of Thrones. I'm sorry, I don't remember those languages. But there's a lot of people offending right now. There's people that speak those languages. They they really speak those languages because the people that wrote the languages for the shows, they did an amazing job of constructing something like a human language.

1:34:19And those that that lights up the language area. That's like because they can speak, you know, pretty much arbitrary thoughts in a human language. It's not a it's a constructed human language. It's probably it's related to human languages because the people that were constructing them were making them like human languages in various ways. But it also activates the same network, which is pretty really cool. Anyway, sorry to go into a place where you maybe a little bit philosophical, but is it possible that this area of the brain is doing some kind of translation into a deeper set of almost like concepts?

1:34:53Mr. It has to be doing. So it's doing in communication, right? It is translating from thought, whatever that is, it's more abstract and it's doing that. That's what it's doing. Like it is that that is kind of what it is doing. It's kind of a meaning network, I guess. Yeah, like a translation network. Yeah. But I wonder what is at the core at the bottom of it? Like what are thoughts? Are they are thoughts? To me, like thoughts and words, are they neighbors or are is it one turtle sitting on top of the other? Meaning like, is there a deep set of concepts that we? Well, there's connections right between those what what these things mean.

1:35:30And then there's probably other other parts of the brain that what these things mean. And so, you know, when I'm talking about whatever it is I want to talk about if it's some it'll be represented somewhere else that that knowledge of whatever that is will be represented somewhere else. Well, I wonder if there's like some stable nicely compressed encoding of meanings that's separate from language that link, you know, I guess I guess the implication here is that that we don't think in language. That's correct. Isn't that cool? And that's so interesting. So people, I mean, this is like hard to do experiments on, but there is this idea of inner voice.

1:36:08And a lot of people have an inner voice. And so if you do a poll on the internet and ask if you you hear self -talking when you're just thinking or whatever, about 70 or 80 % of people will say yes, most people have an inner voice. I don't. And so I always find this strange when so when people talk about an inner voice, I always thought this was a metaphor. And they hear I know most of you, whoever's listening to this thinks I'm crazy now because I don't have an inner voice. And I just don't know what you're listening to. I just it sounds so kind of annoying to me, but that to have this voice going on while you're thinking.

1:36:42But I guess most people have that. And I don't have that. And we don't really know what that connects to. I wonder if the inner voice activates that same note or I don't know. I don't know. I don't know. But I mean, this could be speechy, right? So that's like the you hear. Do you have an inner voice? I don't think so. Oh, a lot of people have this sense that they hear other people they hear themselves. And then say they read someone's email. I've heard people tell me that they hear that other person's voice when they read other people's emails. And I'm like, wow, that sounds so disruptive. I do think I like vocalize when I'm reading, but I don't think I hear a voice.

1:37:18Well, that's probably don't have an inner voice. Yeah, I don't think I have an inner voice. People have this strong percept of hearing sound in their heads when they're just thinking. I refuse to believe that's the majority of people. Majority? Absolutely. What? It's like two thirds or three quarters. It's what I I would never ask class. And I went internet. They always say that. So you're in a minority. It could be a self report flaw. It could be. You know, when I'm reading. Yeah. Inside my head, I'm kind of like saying the words, which is probably the wrong way to read. But I don't hear a voice.

1:37:54There's no percept of voice. I refuse to believe the majority people have. Anyway, it's a fascinating human brain. It's fascinating. But it still blew my mind that the language does appear comprehension does appear to be separate from thinking. So that's one set. One set of data from Ther group is that no matter what task you do, if it doesn't have words and combinations of words in it, it won't light up the language network. It'll be active somewhere else, but not there. So that's one. And then this other piece of evidence relevant to that question is it turns out there are these groups of people who've had a massive stroke on the left side and wiped out their language network.

1:38:41And as long as they didn't wipe out everything on the right as well, in that case, they wouldn't be cognitively functionable. But if they just wiped out language, which is pretty tough to do because it's very expansive on the left. But if they have, then there are these, there's patients like this, so -called global effasius, who can do any task just fine, but not language. They can't talk to them. I mean, they don't understand you. They can't speak. They can't write. They can't read. But they can do, they can play chess. They can drive their cars. They can do all kinds of other stuff. They can do math.

1:39:14They can do all, like, so math is not in the language area, for instance. You do arithmetic and stuff. That's not language area. It's got symbols. So people sort of confuse some kind of symbolic processing with language. And symbolic processing is not the same. So there are symbols and they have meaning, but it's not language. It's not a, conventionalized language system. And so math isn't there. And so they can do math. They do just as well as their control, age -match controls and all these tasks. This is Rosemary Varley over in University College London, who has a bunch of patients, who she's shown this, that they're just, so that sort of combination suggests that language isn't necessary for thinking.

1:39:52It doesn't mean you can't think in language. You could think in language, because language allows a lot of expression, but it's just, you don't need it for thinking. It's, it's just that language is separate, is a separate system. This is kind of blowing my mind right now. It's cool, isn't it? I'm trying to load that in because it has implications for large language models. It sure does. And they've been working on that. Well, let's take a stroll there. You wrote that the best current theories of human language are arguably large language models. So this has to do with form. It's a kind of a big theory.

1:40:25But the reason it's arguably the best is that it does the best at predicting what's English, for instance. It's, it's like incredibly good. You know, it's better than any other theory. It's so, you know, but you know, we don't, you know, there's, it's not sort of, there's not enough detail. What's opaque? Like there's not, you know, no, what's going on? No, what's going on? It's another black box. But I think it's, you know, it is a theory. It's a definition of a theory because this is a gigantic, this is gigantic black box with, you know, a very large number of parameters controlling it. To me, theory usually requires simplicity, right?

1:41:00Well, I don't know. Maybe I'm just being loose there. I think it's a, it's not, it's not a great theory, but it's a theory. It's a good theory. And in one sense, and then it covers all the data. Like anything you want to say in English, it does. And so that's why it's, that's how it's arguably the best is that no other theory is as good as a large language model and predicting exactly what's good and what's bad in English. You know, you, now your thing is in a good theory. Well, probably not, you know, because I want a smaller theory than that. It's too big. I agree. You could probably construct mechanism by which you can generate a simple explanation of a particular language, like a set of rules, something like a, you could generate a dependency grammar for a language, right?

1:41:43Yeah. You could probably, you could probably just ask it about itself. Well, you know, that's, I mean, that presumes, and there's some evidence for this that that that that some large language models are implementing something like dependency grammar inside them. And so there's work from a guy called Chris Manning in colleagues over at Stanford in natural language. And they looked at, I don't know how many large language model types, but certainly Bert and some others were, and we're, where you do some kind of fancy math to figure out exactly what the sort of what kind of abstractions of representations are going on.

1:42:23And they, and they were saying it does look like dependency structure is what they're constructing. It doesn't, like so it's actually a very, very good map. So kind of a, they are constructing something like that. Does it mean that, you know, that they're using that for meaning? I mean, probably, but we don't know. You write that the kinds of theories of language that LLM's are closest to are called construction based theories. Can you explain what construction based theories are? It's just a general theory of language such that there's a form and a meaning pair for, for lots of pieces of the language.

1:42:58And so it's, it's primarily usage based is a construction grammar. It's just, it's trying to deal with the things that people actually say, actually say and actually write. And so that's, it's a usage based idea. And what's the construction? All the constructions, either a simple word, so of like a more theme plus its meaning or a combination of words, it's basically combinations of words, like the rules. So, but it's, it's, it's a, un -specified as to what the form of the grammar is under, underlinedly. And so I, I would, I would argue that the dependency grammar is maybe the, the right form to use for the types of construction grammar.

1:43:38Construction grammar typically isn't kind of formalized quite. And so maybe the formalization, a formalization of that, it might be a dependency grammar. I mean, I, I would think so, but I mean, it, it's up to people, other researchers in that area if they agree or not. So, do you think that large language models understand language? Are they mimicking language? I guess the deeper question there is, are they just understanding the surface form? Or do they understand something deeper about the meaning that then generates the form? I mean, I would argue they're doing the form, they're doing the form, doing it really, really well.

1:44:17And are they doing the meaning? No, probably not. I mean, there's lots of these examples from various groups showing that they can be tricked in all kinds of ways. I really don't understand the, the meaning of what's going on. And so there's a lot of examples that he and other groups have given, which just, which show they don't really understand what's going on. So, you know, the Monte Hall problem is this silly problem, right? Where, you know, if you have three door that is, it's less make a deal is this old game show. And there's three doors and there's a prize behind one and there's some junk prizes behind the other two and you're trying to select one.

1:44:55And if you, you know, he knows, Monty, he knows where the target item is, the good thing, he knows everything is back there. And you're supposed to, he gives you a choice, you choose one of the three. And then he opens one of the doors and it's some junk prize. And then the question is should you trade to get the other one? And the answer is yes, you should trade because he knew which ones you could turn around. And so now the odds are two thirds. Okay. And then you just change that a little bit to the large language, the larger thank you from all just seen that, that, that explanation so many times that it just, if you change the story, it's a little bit, but it make it sound like it's the Monte Hall problem, but it's not.

1:45:30You just say, oh, there's three doors and one behind there is a good prize and there's two bad doors. I happen to know it's behind door number one. The good prize, the car is behind door number one. So I'm going to choose door number one. Monte Hall opens door number three and shows me nothing there. Should I trade for door number two, even though I know the good prize in door number one and then the large language mall say yes, you should trade because it's, it just goes through the, the, the, the, the, the form that it's seen before so many times on these cases where it, yes, you should trade because, you know, your odds have shifted from one and three now to two out of three to being nothing, it doesn't have any way to remember that actually you have a hundred percent probability behind that door number one.

1:46:12You know that that's not part of the, of the scheme that it's seen hundreds and hundreds of times before and so you can't, you can't, even if you try to explain to it that it's wrong that they can't do that, it'll just keep giving you back the, the problem. But it's also possible that the large language model would be aware of the fact that there's sometimes over representation of a, of a particular kind of formulation and it's easy to get tricked by that. And so you could see if they get larger and larger models be a little bit more skeptical. So you see over representation. So like you, it just feels like form can be trained on form can go really far in terms of being able to generate things that look like the thing understands deeply the underlying world model of the kind of mathematical world, physical world, psychological world that would generate these kinds of sentences.

1:47:16It just feels like you're creeping close to the meaning part, easily fooled all this kind of stuff, but that's humans too. So it just seems really impressive how often it seems like it ununderstands concepts. I mean, you don't have to convince me that I'm, I am very, very impressed, but does it, does, do, I mean, you're, you're giving a possible world where maybe someone's going to train some other versions such that it'll be somehow abstracting away from types of forms. I don't, I mean, I don't think that's happened. And so, well, no, no, no, I'm not saying that. I think when you just look at anecdotal examples and just showing a large number of them where it doesn't seem to understand, yeah, it's easily fooled.

1:48:03That does not seem like a scientific, um, data driven, like analysis of like how many places is a damn impressive, oh no, in terms of meaning and understanding and how many places is easily fooled. And like, that's not the inference. Yeah. So I don't want to make that, the inference I don't, I wouldn't want to make was that inference. The inference I'm trying to push is just that is it, is it like humans here? It's probably not like humans here. It's different. So humans don't make that error. If you explain that to them, they're not going to make that error. You know, they don't make that error.

1:48:34And so that's something is doing something different from humans that they're doing in that case. Well, what's the mechanism by which humans figure out that it's an error? I'm just saying the error there is like, if I explain to you, there's a hundred percent chance that the car is behind this case, this door, well, you do want to trade. If you will say no, but this thing will say yes, because it's so true. It's a that trick. It's so wound up on the form that it's that that's an error that a human doesn't make, which is kind of interesting. Less likely to make I should say. Yeah, less likely because like humans are very.

1:49:08Oh, yeah. I mean, you're asking, you know, you're asking humans to you're asking a system to understand a hundred percent, like asking some mathematical concepts. And so like. Look, the places where large language models are the form is amazing. So let's go back to nested structure, center embedded structures. Okay, if you ask a human to complete those, they can't do it. Neither can a large language model. They're just like humans in that. If you ask, if I ask a large language model, that's fascinating. Yeah, by the way, that essential embedding. Yeah, essential embedding is struggles with just like human exactly like you exactly the same way as humans.

1:49:44They get and that's not trained. So they do exactly so there so that is the similarity. So but then it's it's that's not meaning, right? This is form. But when we get into meaning, this is where they get kind of messed up when you start to saying, Oh, what's behind this door? Oh, it's, you know, this is the thing I want. Humans don't mess that up as much, you know, here, they, the form is it's just like the form of the match is amazing, similar without being trained to do that. I mean, it's trained in the sense that it's getting lots of data, which is just like human data, but it's not being trained on you know, bad sentences and being told what's bad.

1:50:21It just can't do those. It'll actually say things like those are too hard for me to complete or something, which is kind of interesting. Actually, kind of how does it know that? I don't know. But it really often doesn't just complete sense. They get off very often says stuff that's true. And sometimes says stuff that's not true. And almost always the form is great. Yeah. But it's still very surprising that with really great form is able to generate a lot of things that are true. Based on what it's trained on and so on. Yes. Yeah. So it's not just it's not just form that is generating. It's mimicking true statements.

1:51:04That's right. That's right. From the internet. I guess I guess now underlying idea there is that on the internet truth is overrepresented versus falsehoods. I think that's probably right. So but the fundamental thing is trained on you're saying is just form. I think so. Yeah. Yeah. I think so. Well, that's a sad. If that's to me, that's still a little bit of open question. I probably lean agreeing with you, especially now you just blown my mind that there's a separate module in the brain for language versus thinking. Maybe there's a fundamental part missing from the large language model approach that lacks the thinking, the reasoning capability.

1:51:47Yeah. That's what this group argues. So the same group, Federanko's group has a recent paper arguing exactly that. There's a guy called Kyle Mahwell who's here in Austin, Texas actually. He's an old student of mine, but he's a faculty and linguistics at Texas. And he was the first author on that. That's fascinating. Still to me an open question. Yeah. What do you have the interesting limits of all of them? You know, I, I don't see any limits to their form. Their form is. Impressive. Yeah. Yeah. Yeah. It's pretty, I mean, it's close to what you said ability to complete central embeddings. Yeah.

1:52:25It's just the same as humans. It seems the same. But that's not perfect. Right. It should be. That's good. No, but I want to be like humans. I'm trying to, I want a model of humans. But but you all way, so perfect use is as close to humans as powerful. I got it. Yeah. But you should be able to, if you're not human, you're like, you're superhuman, you should be able to complete central embedded senses, right? I mean, that's the mechanism is, if it's modeling, some I think it's kind of really interesting that it's really interesting. It's more like, like I think it's potentially underlyingly modeling something like what the the way the form is processed.

1:53:01The form of human language. The way that how and how humans process the language. Yes. Yes. I think that's plausible and how they generate language, process language and general language as fast as anything. Yeah. So in that sense, they're perfect. If we can just linger on the center embedding thing, that's hard for our allows produce and that seems really impressive because that's hard for humans to produce. And how does that connect to the thing we've been talking about before, which is the dependency grammar framework and which you language and the finding that short dependencies seem to be a universal part of language.

1:53:38So why is it hard to complete center embeddings? So what I like about dependency grammar is it makes the cognitive cost associated with longer distance connections very transparent. Basically, there's some, there turns out there is a cost associated with producing and comprehending connections between words, which are just not beside each other. The further apart they are, the worse it is, the according to, well, we can measure that. And there is a cost associated with that. Can you just linger on what do you mean by cognitive cost? Sure. And how do you measure? Oh, you can measure it in a lot of ways.

1:54:16The simplest is just asking people to say whether, you know, how good a sentence sounds, which is asked. It's one way to measure and you try to like triangulate then across sentences and across structures to try to figure out what the source of that is. You can look at reading times in controlled materials, you know, and certain kinds of materials when the, and then we can like measure the dependency distances there. We can, there's a recent study which looked at, we're talking about the brain here, we could look at the language network, okay? We could look at the language network and we could look at the activation in the language network and how big the activation is depending on the length of the dependencies and turns out in just random sentences that you're listening to, if you're listening to, it turns out there are people listening to stories here.

1:55:05And the bigger, the longer the dependency is, the stronger the activation in the language network. And so there's some measure, there's a different, there's a bunch of different measures we could do. That's kind of a neat measure actually of actual activations. Activation in the brain so that you can somehow in different ways convert it to a number. I wonder if there's a beautiful equation connecting cognitive cost and length of dependency. E equals mc squared kind of thing. Yeah, it's complicated, but probably it's doable. I would, I would guess it's doable. I tried to do that a while ago and I was reasonably successful, but some, for some reason, I stopped working on that.

1:55:42I do, I agree with you that it would be nice to figure out. So there's like some way to figure out the, the, the cost. I mean, it's complicated. Another issue you raised before was like, how do you measure distance? Is it words? Is it, it probably isn't, is the part of the problem? Is that some words matter than more than others. And probably, you know, meaning like nouns might matter depending, and then maybe depends of which kind of noun is it a noun we've already introduced or a noun that's already been mentioned, is it a pronoun versus a name like, like all these things probably matter. So probably the simplest thing to do is just like, let's forget about all that.

1:56:15And just think about words or more themes. For sure, but there might be a, like, there might be some insight in the kind of function. Yeah. Yeah. That fits the data, meaning like quadratic, like what? I think it's an exponential. So we think it's probably an exponential such that the longer the distance, the less it matters. And so then, then it's the sum of those is my, that, that was our best guess a while ago. So that you've got a bunch of dependencies. If you've got a bunch of them that are being connected at some point, that's like at the ends of those, the cost is the, is some exponential function of those is my guess.

1:56:53But because the reason it's probably an exponential is like, it's not just the distance between two words, because I can make a very, very long subject verb dependency by adding lots and lots of noun phrases and prepositional phrases. And it doesn't matter too much. It's when you do nested, when I have multiple of these, then, then things get go really bad, it goes south. Probably somehow connected to working memory. Yeah. Yeah. That's probably the function of the memory here is, is the access is trying to find those earlier things. It's kind of hard to figure out what was referred to earlier.

1:57:27Those are those connections. That's, that's the sort of notion of murky as opposed to a storagey thing, but trying to connect, retrieve, retrieve those earlier words depending on what was in between. And then, then we're talking about interference of similar things in between. That's the right theory probably has that kind of notion and it is an interference of similar. And so I'm dealing with an abstraction over the right theory, which is just, you know, it's count words, it's not right, but it's close. And then maybe, maybe you're right, though, there's some sort of an exponential or something on the, on the, on the, to figure out the total.

1:57:58So we can figure out a function for any given, for any given sentence and any given language. But, you know, it's funny, you know, people haven't done that too much, which I do think is, I'm interested that you find that interesting. I really find that interesting and a lot of people haven't found it interesting. And I don't know why I haven't got people to want to work on that. I really like that too. No, that's a, that's a beautify in the underlying idea is beautiful that there's a cognitive cost that correlates with the length of dependency. It just, it feels like it's a deep, I mean, language is so fundamental to the human experience.

1:58:28And this is a nice, clean theory of language where yeah, it's like, wow, okay. So like we like our words close. Together. Yeah. Dependent of words close together. Yeah. That's why I like it too. It's so simple. Yeah. The simplicity is very simple. And yet it explains some very complicated phenomena. If you, if I write these very complicated sentences, it's kind of hard to know why they're so hard. And you can like, oh, nail it down. I can do like a, give you a math formula for why each one of them is bad and where. And that's kind of cool. I think that's very neat. Have you gone through the process?

1:59:01Is there like a, if you take a piece of text and then simplify sort of like there's an average length of dependency and then you like, you know, reduce it and see comprehension on the entire, not just single sounds, but like, you know, you go from James Joyce to Hemingway or something. No, no, no, simple answer is no. That does, if there's probably things you can do in that, in that kind of direction, that's fun. We might, you know, we're going to talk about legalese at some point. And so we may, we will talk about that kind of thinking with applied to legalese. Let's talk about legalese because you mentioned that as an exception, we just take tangents and upon tangents.

1:59:39That's an interesting one. You give it as an exception. It's an exception. That you say that most natural languages, as we've been talking about, have local dependencies with one exception, legalese. That's right. So what is legalese, first of all? Oh, well, legalese is what you think it is. It's just any legal language. I mean, like I actually know, no, very little about the kind of language the lawyers use. So I'm just thinking about language in laws and language in contracts. Got it. So the stuff that you have to run into, we have to run into every other day, every day, and you skip over because it reads poorly.

2:00:18And or, you know, partly it's just long, right? There's a lot of texts there that we don't really want to know about. But the thing I'm interested in, so I've been working with this guy called Eric Martinez, who is a, he was a lawyer, who was taking my class. I was teaching a psycho linguistics lab class at, I have been teaching you for a long time at MIT. And he's a, he was a law student at Harvard. And he took the class because he had done some linguistics as an undergrad. And he was interested in the problem of why legalese sounds hard to understand, you know, why, and so why is it hard to understand?

2:00:51And why, do they write that way? If it is, so it's hard to understand. It seems apparent that it's hard to understand. The question is, why is it? And so we didn't know. And we did an evaluation of much contracts. Actually, we just took a bunch of random contracts because I don't know, you know, there's contracts and laws might not be exactly the same, but contracts are kind of the things that most people have to deal with the most time. And so that's kind of the most common thing that humans have, like humans, that adults in our industrialized society have to deal with a lot. And so, so that's what we pulled.

2:01:26And we didn't know what was hard about them, but it turns out that the way they're written is very centri -medded, has nested structures of them. So it has low -frequency words as well. That's not surprising. Lots of texts have low, it does have surprising, slightly lower -frequency words than other kinds of control texts, even sort of academic texts. Legalese is even worse. It is the worst that we were fined as a fine. You just, you just reveal the game that laws are playing. They're optimizing it different. Well, you know, it's interesting. That's like, now you're getting at why. And so, and I don't think, so now you're saying it's, they're doing intentionally.

2:02:00I don't think they're doing intentionally. But let's, let's, let's, let's, let's, let's get an emergent phenomenon. Yeah, yeah, yeah. We'll get to that. We'll get to that. And so, but we wanted to see why. So, so we see what first as opposed, so like, because it turns out that we're not the first to observe that legalese is weird. Like back to Nixon had a plain language act in 1970 and Obama had one. And boy, a lot of these, you know, a lot of presidents have said, oh, we've got to simplify legal language, most simplified. But you don't know how it's complicated. It's not easy to simplify it. You need to know what it is you're supposed to do before you can fix it.

2:02:35Right. And so, you need to like, you need a cycle linguist to analyze the text and see what's wrong with it before you can like fix it. You don't know how to fix it. How am I supposed to fix something? I don't know what's wrong with it. And so, what we did was just, that's what we did. We let's look. Okay. We just a bunch of contracts had people and we encoded them for the, the, a bunch of features. And so another feature of the people, one of them was the centrum betting. And so, that is like, basically how often a, a clause would, would intervene between a subject in a verb, for example, that's one kind of a cent, centrum betting of a clause.

2:03:08Okay. And turns out they're massively centrum betting. Like, so I think in random contracts and in random laws, I think you get about 70 % or 80 something 70 % of sentences have a centrum betting clause, which is insanely high. If you go to any other text, it's down to 20 % or something. It's, it's, it's so much higher than any control you can think of, including, you think, oh, people think, oh, technical academic text, no, people don't write centrum betting sentences in, in technical academic text. I mean, they do a little bit, but much, it's on the 20 % 30 % realm as opposed to 70. And so, and so there's that.

2:03:44And there's low frequency words. And then people, oh, maybe it's passive. People don't like the passive, passive, for some reason, the passive voice in English has a bad rap. And I'm not really sure where that comes from. And there is a lot of passive in the, there's much more passive voice in the, in the, in legal East. And there is in other places. In passive voice accounts for some of the low frequency words. No, no, no, no, no, no. Those are separate. Those are separate. Oh, so passive voice sucks. These are low frequency words. That's different. So these are different. Yeah, yeah. Yeah, pass the drop the judgment.

2:04:15It's just like these are frequent. These are things which happen in legal East Texas. Then we can ask the dependent measure is like how well you understand those things with those features. Okay. And so then it turns out the passive makes no difference. So it has zero effect on your comprehension ability, on your recall ability. No, it does nothing at all. That has no effect. Your, the words matter a little bit. They do a low frequency words are going to hurt you in recall and understanding. But what really, what really hurts the centrum bed that kills you that is like that slows people down that makes them that makes them very poor and understanding that makes them they can't recall what was said as well and nearly as well.

2:04:52And we did this not only on lay people, we did have a lot of lay people. We ran on 100 lawyers. We recruited lawyers from a, from a wide range of, of sort of different levels of law firms and stuff. And they have the same pattern. So they also like why when the, when they did this, I did not know it would have not that maybe they could process their use to legally say they process just as well as it was normal. No, no, they, they, they're much better than lay people. So they're much, they can much better recall much better understanding. But they have the same main effects as, as lay people, as lay people exactly the same.

2:05:28So they also much prefer the non -centre. So we, we, we constructed non -centre embedded versions of each of these. We constructed versions which have higher frequency words in those places. And we did, we did un un un un passivized. We turned them into active versions. The passive active made no difference. The words made a little difference. And the unscentre embedding makes, makes big differences in all the populations. Unscentre embedding. How hard is that process by the way? For society, don't question. But how hard is it to detect center embedding? Oh easy to detect. You just look at long dependencies or you serve you can just, you can, so there's automatic parsers for English, which are pretty good.

2:06:06And they can detect center. Oh, yeah, very, very, very, I guess, necessarily perfectly. Yeah, you've learned, yeah, pretty much. So you, you're not just looking for long dependencies. You're just literally looking for center embedding. Yeah, yeah, we are in this case, in these case. But long dependencies are, they're highly correlated. So like a center embedding is a, is a big bomb you throw inside inside of a sentence that just blows up the, that makes sure. Can I read a sentence for you from these things? I see. I mean, this is just like one of the things that this is just my eyes, my glaze over in middle mid sentence.

2:06:36No, I understand that. I mean, legal, legal, is this card? This is a go. It goes in the event that any payment or benefit by the company, all such payments and benefits, including the payments and benefits under section 3a here of being here at here and after referred to as a total payments would be subject to the XI's tax. Then the cash severance payments shall be reduced. So that's something we pulled from a regular text, from a contract. Wow. And the center embedded bit there is just for some reason, there's a definition. They throw the definition of what payments and benefits are in between the subject and the verb.

2:07:09Let's, how about don't do that? How about put the definition somewhere else as opposed to in the middle of the sentence? And so that's, that's very, very common. By the way, that's, that's what happens. You just throw your definitions, you use a word, a couple words, and then you define it, and then you continue the sentence. Like, just don't write like that. And you ask, so then we ask lawyers, we said, oh, maybe lawyers like this. Lawyers don't like this. They don't like this. They don't want to, they don't want to write like this. They, they, we asked them to rate materials, which are with the same meaning with, with, with unscentred, bed and centred, and they much preferred the unscentred, bed versions.

2:07:45On the comprehension, on the reading side. Yeah, well, and we asked them, we asked them, would you hire someone who writes like this or this, we asked them all kinds of questions. And they always preferred the less complicated version, all of them. So I don't even think they want it this way. Yeah, but how did it happen? How did you have it? That's a very good question. And, and the answer is, they still don't know. But I have some theories. Well, our, our best theory at the moment is that there's, there's actually some, kind of a performative meaning in the center embedding in the style, which tells you it's legalese.

2:08:16We think that that's the kind of a style, which tells you it's legalese. Like that's a, it's a reasonable guess. And maybe it's just, so for instance, if you're like, it's like a magic spell. So we kind of call this the magic spell hypothesis. So when you give them, when you tell someone to put a magic spell on someone, what do you do? They, you know, people know what a magic spell is and they, they do a lot of rhyming. You know, that's what, that's kind of what people will tend to do. They'll do rhyming and they'll do sort of like some kind of poetry kind of thing. Abracadabra type of thing. Yeah.

2:08:45And maybe that's, there's a syntactic sort of reflex here of a, of a magic spell, which is centrum embedding. And so that's like, oh, it's trying to like tell you this is like, this is something which is true, which is what the goal of law law is, right? It's telling you something that we want you to believe as certainly true, right? That's what legal contracts are trying to enforce on you, right? And so maybe that's like a, a form which has, this is like an abstract, very abstract form centrum embedding, which has a, has a, has a meaning associated with it. Well, don't you think there's an incentive for lawyers to generate things that are hard to understand.

2:09:25That was one of our working hypotheses. We just couldn't find any evidence of that. No, lawyers also don't understand it. But you're creating space why you, I mean, you ask in a communist Soviet union, the individual members, their self report is not going to correctly reflect what is broken about the gigantic bureaucracy, the least of Chernobyl or something like this. I think the incentives under which you operate are not always transparent to the members within that system. So like, it just feels like a strange coincidence that like there is benefit if you just zoom out, look at the system as opposed to asking individual words that making something hard to understand is going to make a lot of people money.

2:10:14Yeah. Like there's going to, you're going to need a lawyer to figure that out, I guess, from the perspective of the individual. But then that could be the performative as it could be as opposed to the incentive driven to be complicated, it could be performative to where we lawyers speak in this sophisticated way and you regular humans don't understand it so you need to hire a lawyer. Yeah, I don't know which one it is, but it's suspicious. Suspicious that it's hard to understand and that everybody's eyes glaze over and they don't read. I'm suspicious as well. I'm still suspicious. And I hear what you're saying, it could be kind of, you know, no individual and even average of individuals, it could just be a few bad apples in a way which are driving the effect in some way.

2:10:56Influential bad apples at the sort of, yeah, then everybody looks up to whatever they're like essential figures in how, you know, it turns, it is kind of interesting. That's among our hundred lawyers, they did not. They didn't want to. They really didn't like it. And they weren't better than regular people at comprehending it or they were an average better, but they had the same difference. The same, the same difference. Exactly. Same difference. So they, but I, they wanted it fixed. So they, they also, and so that, that gave us hope that because it actually isn't very hard to construct a material, which is unscentrumbed and has the same meaning, it's not very hard to do.

2:11:37It's basically in that situation, just putting definitions outside of the subject verb relation in that particular example. And that's kind of, that's pretty general what they're doing is just throwing stuff in there, which you don't have to put in there. There's extra words involved. Typically, you may need a few extra words to sort of to refer to the things that you're defining outside in some way, because if you only use it in that one sentence, then there's no reason to introduce extra, extra terms. So we might have a few more words, but it'll be easier to understand. So I mean, I have hope that now that maybe we can make legal ease less, less convoluted.

2:12:15So maybe the next president in the United States can, yes, have a saying generic things say, exactly. I ban center embeddings and make Ted the the language is the guy you should really put in there. But center embeddings are the bad thing to have. That's right. So you get rid of that. They'll do a lot of it. That'll fix a lot. That is so fascinating. And it's really fascinating. I'm many fronts that humans are just not able to deal with this kind of thing. And that language because of that involved in the way you did. It's fascinating. So one of the mathematical formulations you have when talking about languages communication is, let's say, do you have noisy channels?

2:13:01What's the noisy channel? So that's about communication. And so this is going back to Shannon. So Shannon, Claude Shannon was a student at MIT in the 40s. And so he wrote this very influential piece of work about communication theory or information theory. And he was interested in human language. Actually, he was trying to, he was interested in this problem of communication, of getting a message from my head to your head. And so, and he was concerned or interested in what was a robust way to do that. And so that assuming we both speak the same language, we both already speak English, whatever, whatever the language is, we speak that.

2:13:46What is a way that I can say the language so that it's most likely to get the signal that I want to you? And so, and then the problem there in the communication is the noisy channel is that there's, I make, there's a lot of noise in the system. I don't speak perfectly. I make errors. That's noise. There's background noise. You know, you know that as, like, literal, literal background noise. There is like white noise in the background or some other kind of noise or some speaking going on that you're just, you're at a party. That's background noise. You're trying to hear someone. It's hard to understand them because there's almost other stuff going on in the background.

2:14:23And then there's noise on the communication on the, on the receiver side so that you have some problem maybe understanding me for stuff that's this internal to you in some way. So you've got some other problems, whatever, with understanding for whatever reasons. Maybe you're, maybe you've had too much to drink. You know, who knows why you're not able to pay attention to the signal? So that's the noisy channel. And so, so that language, if it's communication system, we are trying to optimize in some sense the, the passing of the message from one side to the other. And so it turned, I mean, one idea is that maybe, you know, aspects of like word order, for example, might have optimized in some way to, to make language a little more easy to be passed from speaker to listener.

2:15:10And so, Shannon's the guy that did this stuff way back in the 40s. He was very interesting, you know, historically, he was interested in working in linguistics. He was an MIT. And he did, this is his master's thesis of all things, you know, it's crazy how much, how much he did for his master's thesis in 1948, I think, or 49 or something. And he wanted to keep working in language. And it just wasn't a popular communication as a, as a reason source for what language was wasn't popular at the time. So, Trump's, he was becoming, it was moving in there. He was, and he just wasn't able to get a handle there, I think.

2:15:41And so, and so he moved to Bell Haps and worked on communication from a mathematical point of view and was, you know, did all kinds of amazing work. And so he's just more on the signal side versus like the language side. Yeah. Hi, I would have been interesting to see if you pursue the language side. Yeah. That's really interesting. Yeah. He was interested in that as examples in the 40s are, are kind of like, they're very language -like, like, related things. Yeah. We can kind of show that there's a noisy channel process going on in when you're listening to me, you know, you're, you can often sort of guess what I meant by what I, you know, what you think I meant given what I said.

2:16:20And I, I mean, with respect to sort of why language looks the way it does, we might, there might be sort of, I said, alluded to, there might be ways in which word orders is somewhat optimized for, for, because of the noisy channel in some way. I mean, that's really cool to sort of model. If you don't hear certain parts of a sentence or have some probability of missing that part, like, how do you construct the language that's resilient to that? That's somewhat robust to that. Yeah. That's the idea. And then you're kind of saying, like, the word order and the syntax of language, the dependency length, are all helpful to do.

2:16:54Yeah. Well, the dependency length is really about memory, where I think that's like about sort of what's easier or harder to produce in some way. And these other ideas are about sort of robustness to communication. So the problem of potential loss of it, loss of signal due to noise. And so that, that, that, there may be aspects of word order, which is somewhat optimized for that. And, you know, we have this one guess in that direct. And these are kind of just so stories. I have to be, you know, pretty frank. They're not, like, I can't show this is true. All we can do is like, look at the current languages of the world.

2:17:24This is a, like, we can't sort of see how languages change or anything because we've got these snapshots of a few, you know, 100 or a few thousand languages. We don't have, we don't really, we can't do the right kinds of modifications to test these things experimentally. And so, you know, so just take that this with a grain of salt, okay, from here, this, this stuff, the dependency stuff I can, I'm much more solid on. And like, here's what the lengths are. And here's, and here's what's hard. And here's what's easy. And this is a reasonable structure. I think I'm pretty reasonable. Here's like, why, you know, why does the word order look the way it does is we're now into shaky territory, but it's kind of cool.

2:17:57But we're talking about just to be clear. We're talking about maybe just actually the sounds of community, like you and I are sitting in the bars, very loud. And you, yeah, you model with a noisy channel, the loudness, the noise, and we have the signal that's coming across that, and you're saying word order might have something to do with optimizing. That, yes, presence of noise. Yeah. Yeah. Yeah. I mean, it's really interesting. I mean, to me, it's interesting how much you can load into the noisy channel, like how much can you bake in? Well, you said like, you know, cognitive load on the receiver end.

2:18:29We think that those are, there's three, at least three different kinds of things going on there. And we probably don't want to treat them all as the same. Sure. And so I think that you, you know, the right model, a better model of a noisy channel would treat, we'd have three different sources of noise, which, which are background noise, you know, speaker speaker, inherent noise and listener inherent noise. And those are not this, those are all different things. Sure. But then underneath it, there's a million other subsets, like what that's true on the receiving. I mean, I just mentioned cognitive load on both sides.

2:19:00Then there's like speaking, speech impediment and sort of add just everything. World view, I mean, on the meaning, we start to creep into the meeting realm of like, we have different world views. Well, how about just form still though? Like just just what language you know? Like, so how well you know the language? And so if it's second language for you versus first language and how maybe what other languages you know, these are still just form stuff. And that's like potentially very informative. And, and you know, how old you are, these things probably matter, right? So like, child learning a language is, is a, you know, as a noisy representation of English grammar, you know, depending on a old they are.

2:19:39So maybe when they're six, they're perfectly formed. But you mentioned one of the things is like a way to measure the language is learning problems. So like, what's the correlation between everything we've been talking about and how easy it is to learn a language? So it's like short dependencies correlated to ability to learn a language. Is there some kind of, or like the dependency grammar? Is there some kind of connection there? How easy is to learn? Yeah, well, all the languages in the world's language, none is right now. We know is any better than any other with respect to sort of optimizing dependency lengths, for example, they're all kind of do it, do it well.

2:20:21They all keep low. It's, so I think of every human language is some kind of an optometrism sort of an optimization problem, a complex optimization problem to this communication problem. And so they've like they've solved it, you know, they're just sort of noisy solutions to this problem of communication. There's just so many ways you can do this. So they're not optimized for learning. They're probably less for communication and learning. So yes, one of the factors which is, yeah, so learning is messing this up a bit. And so, so for example, if it were just about minimizing dependency lengths and that was all that matters, then we, you know, so then then we might find grammars which didn't have regularity in their rules.

2:21:02But language is always have regularity in their rules. So what I mean by that is that if I wanted to say something to you in the optimal way to say it was, it really mattered to me, all that mattered was keeping the dependencies as close together as possible. Then I then I would have a very lack set of free structure rule or dependency rule. It wouldn't have very many of those. I would have very little of that. And I would just put the words as close the things that refer to the things that are connected right beside each other. But we don't do that. Like there are, like there are word order rules, right?

2:21:32So they're very, and depending on the language, they're more and less strict, right? So you speak Russian, they're less strict than English. English is very rigid word order rules. We order things in a very particular way. And so why do we do that? Like that's probably not about communication. That's probably about learning. I mean, then we're talking about learning. So I probably easier to learn regular, regular things, things which are very productive all and easy to, so that's probably about learning is my, is our guess? Because that can't be about communication. Can it be just noise? Can it be just the, the messiness of the development of a language?

2:22:06Well, if it were just a communication, then we should have languages which have very, very free word order. And we don't have that. We have free error, but not free. Like there's always, well, no, but what I mean by noise is like cultural, like sticky, cultural things, like the way, the way you communicate, just there's a stickiness to it. It's an imperfect, it's a noisy optimist, it's the castic, yeah, the function over which you're optimizing is very noisy. Yeah. So because I don't, it feels weird to say that learning is part of the objective function, because some languages are way harder to learn than others, right?

2:22:43Or is that, that's not true? That's interesting. I mean, that's the public perception, right? Yes, that's true. For a second language, for a second language, but that depends on what you started with, right? So it really depends on how close that second language is to the first language you've got. And so yes, it's very, very hard to learn Arabic if you've started with English or it's harder to, you know, Chinese, I think, is the worst in the, there's like, defense language institute in the, in the United States has like a list of how hard it is to learn what language from English. I think Chinese is the first.

2:23:18But that's the second language. You're saying babies don't care. No, no evidence that there's anything harder or easier, but any baby, any language learned, like by three or four, they speak that language. And so there's no evidence of anything harder, or easier, but any human language, they're all kind of equal. To what degree is language? This is returning to Chomsky a little bit is, is a Nate. You said that for Chomsky, he used the idea that language is some aspect of the language or a Nate to explain away certain things that are observed. But to how much are we born with language at the core of our mind, brain?

2:23:56I mean, I, you know, the answers I don't know, of course, but the, I mean, I like to, I'm an engineer hard, I guess, and I sort of think it's fine to postulate that a lot of it's learned. And so I'm guessing that a lot of it's learned. So I think the reason Chomsky went with the Nateness is because he, he hypothesized movement in his grammar. He was interested in grammar and movements hard to learn. I think he's right movement is a hard, it's a hard thing to learn to learn these two things together and how to interact. And there's like a lot of ways in which you might generate exactly the same sentences and it's like really hard.

2:24:32And so he's like, oh, I guess it's learned. Sorry, sorry, I guess it's not learned as to Nate. And if you just throw at the movement and just think about that in a different way, you know, then you, you get some messiness. But the messiness is human language, which it's actually fits better. It's that messiness isn't a problem. It's actually a, it's a valuable asset of, of, of the theory. And so, so I think I don't really see a reason to postulate much, much innate structure. And that's kind of, I think, these large language models are learning so well is because I think you can learn the form, the forms of human language from the input.

2:25:12I think that's like, it's likely to be true. So that part of the brain that lights up when you're doing all the comprehension that could be learned. That could be just, yeah, you don't need, you don't need any. It doesn't have to be an eight. So like lots of stuff is modular in the brain that's learned. It doesn't have to, you know, so there's something called the visual word form area in the back. And so it's in the back of your head when you're the, you know, the visual cortex, okay? And that is very specialized language. Sorry, very specialized brain area, which does visual word processing if you read, if you're a reader, okay?

2:25:46If you don't read, you don't have it, okay? Guess what? You spend some time learning to read and you develop that brain area, which does exactly that. And so these, the modularization is not evidence for inateness. So the modularization of a language area doesn't mean we're born with it. We could have easily learned that. We might have been born with it. We just don't know at this point. We might very well have been born with this left lateralized area. I mean, there's like a lot of this kind of argument. So some people get a stroke or something goes really wrong on the left side. Where the left language area would be and that isn't there.

2:26:25It's not available. And it develops just fine the right. So it's no longer, so it's not about the left. It goes to the left. Like, this is a very interesting question. It's like, why is the, why are any of the brain areas the way that they are and how, how, how do they come to be that way? And, you know, there's these natural experiments, which happen where people get these, you know, strange events in their brains at very young ages, which wipe out sections of their brain. And they behave totally normally and no one knows anything was wrong. And we find out later, because they happen to be accidentally scanned for some reason.

2:26:58It's like, what, what happened to your left hemisphere? It's missing. There's not many people who've missed their whole left hemisphere, but they'll be missing some other section of their left or their right. And they behave absolutely normally. We'd never know. So that's like a very interesting, you know, current research. You know, this is another project that this person, in Federico is working on. She's got all these people contacting her because she's scanned some people who have been missing sections, one person missing, missed a section of her brain and was scanned in her lab. And she, and she happened to be a writer for the New York Times.

2:27:30And it was an article in New York Times about, about the, just about the scanning procedure and about what might be learned by sort of the general process of MRI and language in a different language. And because she's writing for the New York Times, and there's all these people started writing to her, who will also have similar, similar kinds of deficits because they've been, you know, accidentally, you know, scanned for some reason and, and found out they're missing some section. And then they, they volunteer to be scanned. These are natural experiments. They're kind of messy, but natural experiments kind of cool.

2:28:07She calls them interesting brains. The first few hours, days, months of human life are fascinating. Like, yeah, well, inside the womb, actually, like that development, that machinery, whatever that is, seems to create powerful humans that are able to speak, comprehend, think, all that kind of stuff, no matter what happened, not no matter what, but robust to the different ways that the brain might be damaged and so on. That's really, that's really interesting. But, what would Chomsky say about the fact that the thing you're saying now that language is, is, seems to be happening separate from thought?

2:28:48Because as far as I understand, maybe you can correct me, he thought that language underpins. Yeah, he thinks so. I don't know, that's right. Absolutely. And it's pretty mind -blowing to think that it could be completely separate from thought. That's right. But, so, you know, he's basically a philosopher, philosopher of language in a way, thinking about these things. It's a fine thought. You can't test it in his methods. You can't do a thought experiment to figure that out. You need a scanner, you need brain damage people, you need something, you need ways to measure that. And that's what, FMRI offers.

2:29:30And patients are a little messier. FMRI is pretty unambiguous, I'd say. It's like very unambiguous. There's no way to say that the language network is doing any of these tasks. You should look at those data. There's no chance that you can say that those networks are overlapping. They're not overlapping. They're just completely different. And so, you know, you can always make, you know, it's only two people. It's four people or something for the patients. And there's something special about them. We don't know. But these are just random people. And with lots of them, and you find always the same effects.

2:30:06And it's very robust, I'd say. What's the fassing effect? What's the, you mentioned Bolivia. What's the connection between culture and language? You've also mentioned that, you know, much of our study of language comes from WERD, Weird People, Western Educated, Industrialized, Rich, and Democratic. So, when you study, like remote cultures, such as around the Amazon jungle, what can you learn about language? So, that term Weird is from Joe Henrich. He's at Harvard. He's a Harvard evolutionary biologist. And so, he works on lots of different topics. And he basically was pushing that observation that we should be careful about the inferences we want to make when we're talking in psychology or yeah, most things psychology, I guess, about humans if we're talking about, you know, undergrads at MIT and Harvard.

2:31:11Those aren't the same, right? These aren't the same things. And so, if you want to make inferences about language, for instance, there's a lot of very, a lot of other kinds of languages in the world than English and French and Chinese, you know. And so, maybe, for language, we care about how culture, because cultures can be varied. I mean, of course, English and Chinese cultures are very different, but, you know, 100 -gatherers are much more different in some ways. And so, if culture has an effect on what language is, then we kind of want to look there as well as looking. It's not like the industrialized cultures aren't interesting.

2:31:48Of course, they are, but we want to look at non -industrialized cultures as well. And so, I worked with two, I've worked with Chimani, which are in Bolivia and Amazon, both in the Amazon, these cases. And there are so -called farmer foragers, which is not hunter -gatherers. It's sort of one up from hunter -gatherers, and that they do a little bit of farming as well, a lot of hunting as well, but a little bit of farming. And the kind of farming they do is the kind of farming that I might do, if I ever were to grow tomatoes or something in my backyard. It's not like, so it's not like big field farming.

2:32:22It's just a farming for a family, a few things you do that. And so, that's the kind of farming they do. And the other group I've worked with are the Pieter Ha, which are in, also in the Amazon, and it happened to be in Brazil. And that's with a guy called Dan Everett, who was a linguist, anthropologist, who actually lived and worked in the, I mean, he was a missionary, actually, initially, back in the 70s, working with, trying to translate languages so they could teach them the Bible, teach them Christianity. What can you say about that? Yeah. So, the two groups I've worked with, the Chimane and the Pieter Ha, are both isolate languages, meaning there's no known connected languages at all, like just like on their own.

2:33:07Yeah, there's a lot of those. And most of the isolates occur in the, in the Amazon, or in Papua New Guinea, and these places where the world has sort of stayed still for a long enough, and they're ha, like so there aren't earthquakes, there aren't, well, certainly no earthquakes in the Amazon jungle. And the climate isn't bad, so you don't have droughts. And so, you know, in Africa, you've got a lot of moving of people because there's drought problems. And so, you've got to move because you've got no water, then you've got to get going, and then, then you run into contact with other other tribes, other groups.

2:33:53In the Amazon, that's not the case. And so, people can stay there for hundreds and hundreds and probably thousands of years, I guess. And so, these groups have, and the Chimane and the Pieter Ha, are both isolates in that, and they just, I guess they've just lived there for ages and ages with minimal contact with other outside groups. And so, I mean, I'm interested in them because they are, I mean, I, you know, in these cases, I'm interested in their words. I would love to study their syntax, their orders of words, but I'm mostly just interested in how languages are connected to their cultures in this way.

2:34:29And so, with the Pieter Ha, they're most interesting. I was working on number there, number information. And so, the basic idea is I think language is invented. So, I get from the words here, is that, I think language is invented. We talked about color earlier. It's the same idea. So, that what you need to talk about with someone else is what you're going to invent words for. And so, we invent labels for colors that I need, not that I, that I can see, but that, but that things I need to tell you about so that I can get objects from you or get you to give me the right objects. And I just don't need a word for teal or a word for acrimarine in the, in the Amazon jungle for the most part, because I don't have two things which differ on those colors.

2:35:10I just don't have that. And so, and so numbers are really another fascinating info source of information here where you might, you know, naively, I certainly thought that all humans would have words for exact counting. And the Pieter Ha don't, okay? So, they don't have any words for even one. There's not a word for one in their, in their language. And so, there's still not word for two, three or four. So, so that kind of blows people's minds on. Yeah, that's going my point. That's pretty weird. How are you, how are you going to ask, I want two of those? You just don't. And so, that's just not a thing you can possibly ask in the Pieter Ha.

2:35:48It's not possible. That is, there's no words for that. So, here's how we found this out, okay? So, so it was thought to be a one, two, many language. There are three words for quantifiers for sets. But, and the people had thought that those meant one, two, and many. But what they really mean is few some and many, many is correct. It's few some and many. And so, and so the way we figured this out. And this is kind of cool is that we gave people, we had a set of objects, okay? And these were having to be spools of thread, doesn't really matter what they are identical objects. And, and I sort of start off here, I just give, you know, give you one of those and say, what's that?

2:36:26Okay, so you're a Pieter Ha, speaker and you tell me what it is. And, and then I give you two and say, what's that? And, and nothing's changing in this set except for the number, okay? And then I just ask you to label these things. We just do this for a bunch of different people. And, and frankly, it's a, I did this task and it's a weird, it's a little bit weird. So, you say, the word that they thought that we thought was one, it's few, but for the first one. And then maybe they say few, or maybe they say some for the second. And then for the third or the fourth, they start using the word many for the set.

2:36:55And then five, six, seven, eight, I go all the way to 10. And, and it's always the same word. And they look at me like I'm stupid because they told me what the word was for six, seven, eight. And I'm going to continue asking them at nine and 10. I'm like, I'm sorry, I just, I just, they understand that I want to know their language. That's the point of the task is like I'm pronouning their language. And so that's okay. But it does seem like I'm a little slow. What, because I, they already told me what the word for many was five, six, seven. And I keep asking. So it's a little funny to do this task over and over.

2:37:26We did this with the guy called Dan was the, our translator. He's the only one who really speaks Pieter Ha, fluently. He's a good, bilingual for a bunch of languages, but also in English and in Pieter Ha. And then a guy called Mike Frank was also a student with me down there. He and I did these things. And so you do that. Okay. And everyone does the same thing. All, all, all, you know, we asked like 10 people and they all do exactly the same labeling for one up. And then we just do the same thing down on, like random order. Actually, we do some of them up, some of them down first. Okay. And so we do, instead of one to 10, we do 10 down to one.

2:38:02And so so I give them 10, nine, eight. They start saying the word for some. And then down, when you get to four, everyone is saying the word for few, which we thought was one. So it's like it's the context determined what word, what, what that quantifier they used was. So it's not a count word. They're not, they're not, count words. They're just approximate words. And they're going to be noisy when you interview a bunch of people with the definition of few. And there's going to be a threshold in the context. Yeah. Yeah. Yeah. Yeah. I don't know what that means. That's, that's going to be 10 on the context.

2:38:30I think it's for English, too, right? If you ask an English person what a few is. I mean, that's going to spend completely on the context. And that might actually be at first hard to discover. Because for a lot of people, the jump from one to two will be few, right? So it's the jump. Yeah, it might be. It might still be there. Yeah. I mean, that's fascinating. That's fascinating. The numbers don't present themselves. Yeah. So the words aren't there. And then and so then we do these other things. Well, if they don't have the words, can they do exact matching kinds of tasks? Can they even do those tasks?

2:39:01And and and the answer is sort of yes and no. And so yes, they can do them. So here's the tasks that we did. We put out those spools of thread again. Okay. So I mean, I put like three out here. And then that we gave them some objects and those happen to be uninflated red balloons. It doesn't really matter what they are. It's just their bunch of exactly the same thing. And it was easy to put down right next to these spools of thread. Okay. And so then I put out three of these and your task was to just put one against each of my three things. And they could do that perfectly. So I mean, I would actually do that.

2:39:35It was a very easy task to explain to them because I have I did this with this guy, Mike Frank, and he would be my I'd be the experiment or telling him to do this and showing him to do this. And then we just like just do it. He did. You'll copy him. All we had to I didn't have to speak pure huh, except for know what copy him. Like do what he did is like, oh, we had to be able to say. And and then they would do that just perfectly. And so we'd move it up. We'd do some sort of random number of items up to 10. And they basically do perfectly on that. They never get that wrong. I mean, that's not accounting task.

2:40:04Great. That is just a match. You just put one against it. It doesn't matter how many I don't need to know how many there are there to do that correctly. And and they would make mistakes, but very, very few and no more than MIT undergrad. Just going to say like this, there's no, these are low stakes. So you know, you make mistakes. Counting is not required to complete the matching. That's right. Not not at all. Okay. And so and so that's our control. And this guy had gone down there before and said that they couldn't do this task. But I just don't know what he did wrong there because they can do this task perfectly well.

2:40:34And you know, I can train my dog to do this task. So of course they can do this task. And so you know, it's not a hard task. But the other task that was sort of more interesting is like, so then we do a bunch of tasks where you need some way to encode the set. So like one of them is just a I just put a a a opaque sheet in front of the other things I put down a bunch of set of these things and I put no big sheet down. And so you can't see them anymore. And I tell you do the same thing you were doing before, right? You know, and it's easy if it's two or three. It's very easy. But if I don't have the words for eight, it's a little harder like maybe, you know, with practice went well, no.

2:41:16Because you have to tell us for us it's easy because we just we just count them. It's just so easy to count them. But they don't they can't count them because they don't count. They don't have words for this thing. And so they would do approximate. It's totally fascinating. So they would get them approximately right. You know, you know, after four or five, you know, because you can basically always get four right three or four. That looks that's something we can visually see. But but after that you kind of have it's approximate number. And so then there's a bunch of tasks we did and they all failed as I mean failed.

2:41:46They did approximate after five on all those tasks. And it kind of shows that the words, you kind of need the words, you know, to be able to do these these kinds of tasks. Because there's a little bit of a chicken and egg thing there. Because if you don't have the words, then maybe they'll limit you in the kind of like a little baby Einstein there won't be able to come up with a counting task. You know what I mean? Like a the ability to count enables you to come up with interesting things probably. So yes, you develop counting because you need it. But then once you have counting, you can probably come up with a bunch of different inventions.

2:42:26Like how to I don't know. What kind of thing they do matching really well for building purposes, building some kind of hut or something like this. So it's interesting that language is a limiter on what you're able to do. Yeah, here's language is just is the words. Here is the words like the words for exact count is the limiting factor here. They just don't have them. Yeah, and this is what that's what I mean. Yeah, yeah, yeah, the limit that limit is also limit on the society of what they're able to build. That's going to be true. Yeah. So it's problem. I mean, we don't know this is one of those problems with the snapshot of just current languages is that we don't know what causes a culture to discover slash invent a counting system.

2:43:11But the hypothesis is the guess out there is something to do with farming. So if you have a bunch of goats and you want to keep track of them and they're you have saved 17 goats and you go to bed at night and you get up in the morning, boy, it's easier to have a count system to do that. You know, if I have that's an abstract abstraction over a set. So they don't have like people often ask me when I talk to tell them about this kind of work. And they say, well, don't these people, I don't think I have kids don't have a lot of children. I'm like, yeah, they have a lot of children and they do they often have families of three or four or five kids and they go, well, don't they need the numbers to keep track of their kids?

2:43:45And I always ask the person who says does it do you have children? And the answer is always no because that's not how you keep track of your kids. You care about their identities. It's very important to me when I go, I think I have five children. It's doesn't matter which, yeah, it matters which five. It's like if you replaced one with someone else, I would, I would care. Goat maybe not, right? That's the kind of point. It's an abstraction. Something that looks very similar to the one wouldn't matter to me probably. But if you care about goats, you're going to know them actually individually also.

2:44:17Yeah, you will. I mean, Kyle's goats, if there's a source of food and milk and all that kind of stuff, you're going to actually do it right. But I'm saying it is abstraction such that you don't have to care about their identities to do this thing fast. That's, that's the hypothesis. Not mine. From anthropologists, this is a guessing about where words for counting came from is from farming maybe. Yeah. Do you have a sense why universal languages like Esperanto have not taken off? Like why do we have all these different languages? Yeah, well, my guess is that the function of a language is to do something in a community.

2:44:53And I mean, unless there's some function to that language in the community, it's not going to survive. It's not going to be useful. So here's a great example. So what I like language death is super common. Okay. Languages are dying all around the world. And here's how here's why they're dying. And it's like yeah, I see this in, you know, in it's not happening right now. And either the Chimane or the or the Pieda ha, but it probably will. And so there's a neighboring group called Mostitan, which is I said, I said that it's a isolate. Actually, there's a dual. There's two of them. Okay, so it's actually there's two languages which are really close, which are Mostitan and Chimane, which are unrelated to anything else.

2:45:32And Mostitan is unlike Chimane in that it has a lot of contact with Spanish. And it's dying. So that language is dying. The reason it's dying is there's not a lot of value for the local people in their native language. So there's much more value in knowing Spanish, like because they want to feed their families. And how do you feed your family? You learn Spanish. So you can make money. So you can get a job and do these things. And then you can, and then you make money. And so they want Spanish things. They want, and so, so Mostitan is, is in danger and is dying. And that's normal. And so basically the problem is that people, the reason we learn language is to communicate.

2:46:11And we need to, we use it to, to make money. And to do whatever it is to feed our families. And if that's not happening, then it won't take off. It's not like a game or something. This is like something we, like why is English so popular? It's not because it's an easy language to learn. Maybe it is. I don't really know. But that's not why it's popular. But because it's the giant, the United States is gigantic economy and therefore, yeah, it's big economies that do this. It's all it is. It's all about money and that's what, and so, you know, there's a motivation to learn Mandarin. There's a motivation to learn Spanish.

2:46:46There's a motivation to learn English. These languages are very valuable to know because there's so, so many speakers all over the world. There's less of a value economically. It's like kind of what drives this. It's not a, it's not a, you know, it's not just for fun. I mean, there are these groups that do want to learn language just for language is sake. And they want, and then, and there's something, you know, to that. But those are rare. Those are rareities in general. Those are a few small groups that do that. Not most people don't do that. Well, if that was the primary driver, then everybody was speaking English or speaking one language.

2:47:17There's also attention. That's happening. And that, well, we're moving towards fewer and fewer languages. We are. I wonder if you're right. Maybe, maybe, you know, this is slow. But maybe that's what we're moving. But there is attention. You're saying a language that defringes. But if you look at geopolitics and superpowers, it does seem that there's another thing attention, which is a language is a national identity sometimes. Oh, you're certain nation. I mean, that's the, the war in Ukraine, language, Ukrainian language is a symbol of that war in many ways, like country fighting for its own identity.

2:47:54So it's not merely the convenience. I mean, those two things are attention. Is the, the convenience of trade and the economics and be able to communicate with neighboring countries and trade more efficiently with neighboring countries, all that kind of stuff. But also identity of the group. I completely agree. As languages the way, for every community, like dialects that emerge are a kind of identity for people. Sometimes a way for people to say FU to the more powerful people. That's interesting. So in that way, language can't be used as that tool. Yeah, I completely agree. And there's a lot of work to try to create that identity.

2:48:36So people want to do that. Speak, you know, as a cognitive scientist and language expert, I hope that continues because I don't want languages to, I want languages to survive because I, because they're so interesting for so many reasons. But I mean, I find the fascinating just for the language part, but I think they, you know, there's a lot of connections to culture as well, which is also very important. Do you have hope for machine translation that can break down the barriers of language? So while all these different diverse languages exist, I guess there's many ways of asking this question, but basically how hard is to translate in an automated way for one language to another?

2:49:20There's, there's going to be cases where it's going to be really hard, right? So there are concepts that are in one language and not another, like the most extreme kinds of cases are these cases of number information. So exactly like good luck translating a lot of English into Pira Ha. It's just impossible. There's no way to do it because there are no words for these concepts that we're talking about. There's probably the flip side, right? There's probably stuff in Pira Ha, which is going to be hard to translate into English on the other side. And so I just don't know what those concepts are. I mean, you know, the space, the world space is a little is different from my world space.

2:49:56So I don't know what, like, so that the things they talk about, things are, you know, it's going to have to do with their life as opposed to, you know, my industrial life, which is going to be different. And so there's going to be problems like that always. You know, there's like, it's not maybe it's not so bad in the case of some of these spaces. And maybe it's going to be harder than others. And so it's pretty bad in number. It's like, you know, extreme, I'd say, in the number space, you know, exact number space. But in the color dimension, right? So that's not so bad. There's, I mean, but it's a problem that you don't have ways to talk about the concept.

2:50:29And there might be entire concepts that are missing. So to you, it's more about the space of concept versus the space of form, like form, you can probably map. Yes. Yeah. But so you were talking earlier about to translation and about how translations, there's good and bad translations. I mean, now we're talking about translations of form, right? So what makes writing good? Right? You know, there's some music in the form. Right. It's not just the content. It's, you know, it's how it's written. And translating that, I, you know, that's, that sounds difficult. We should, we should say that there is like, I don't know, it has a day to say meaning, but there's a music and a rhythm to the form.

2:51:10When you look at the broad picture, like the Prince of the East, the Yaskin, Tolstoy, or Hemingway Bukowski, James Joyce, like I mentioned, there's a beat to it. There's an edge to it that it's like, is in the form. We can probably get measures of those. Yeah. I, I don't know. I'm optimistic that we could get measures of those things. And so maybe that's manageable. I don't know. I don't know though. I have done that. I would love to see translating. Translation to Hemingway is probably the lowest, I would love to see different authors, but the average per sentence dependency length for Hemingway is probably the shortest.

2:51:54That's your sense. It's simple sentences. Simple sentences. Yeah. Yeah. Yeah. I mean, that's one, if you have really long sentences, even if they don't have center, like they can have longer connections. Yeah. They can have long connections. They don't have to. Right. You can't have a long, long sentence with a bunch of local words. Yeah. Yeah. But it's, but it is much more likely to have the possibility of long dependencies with long sentences. Yeah. I met a guy named Azaraskin who, who does a lot of cool stuff. Really brilliant. He works with Tristan Harris and a bunch of stuff. But he was talking to me about communicating with animals.

2:52:29He co -founded Earth Species Project where you're trying to find the common language between whales, crows, and humans. And he was saying that there is a, there's a lot of promising work that even though the signals are very different, like the actual like, if you have embeddings of the languages, they're actually trying to communicate similar type things. Is there something you can comment on that? Like, where is there promise to that? And everything you've seen in different cultures, especially like remote cultures, that this is a possibility? No. Like we can talk to whales. I would say yes.

2:53:10I think it's not crazy at all. I think it's quite reasonable. There's this sort of weird view, well, odd view, I think, that to think that human language is somehow special. I mean, it is, maybe it is. We can certainly do more than any of the other species. And maybe our language system is part of that. It's possible. But people do have often talked about how human, like Chomsky, in fact, has talked about how human only human language has this compositionality thing that he thinks is sort of key in language. And the problem with that argument is he doesn't speak whale. And he doesn't speak crow, and he doesn't speak monkey.

2:53:57They say things like, well, they're making a bunch of grunts and squeaks. And the reasoning is like, that's bad reasoning. I'm pretty sure if you asked a whale what we're saying, they'd say, well, I'm making a bunch of weird noises. Exactly. And so it's like, this is a very odd reasoning to be making that human language is special because we're the only ones who have human language. I'm like, well, we don't know what those other, we just don't, we can't talk to them yet. And so there probably is signal in there. And it might very well be something complicated like human language. I mean, sure, with a small brain in lower species, there's probably not a very good communication system.

2:54:37But in these higher higher species where you have what seems to be abilities to communicate something, there might very well be a lot more signal there than we might have otherwise thought. But also if we have a lot of intellectual humility here, somebody formerly from my teen area, oxman who I admire very much has talked a lot about, has worked on communicating with plants. So like, yes, the signal there is even less than, but like it's not out of the realm of possibility that all nature has a way of communicating. And it's a very different language, but they do develop a kind of language through the chemistry, through some way of communicating with each other.

2:55:23And if you have enough humility about that possibility, I think you can, I think it would be a very interesting in a few decades, maybe centuries, hopefully not, a humbling possibility of being able to communicate not just between humans effectively, but between all of living things on earth. Well, I mean, I think some of them are not going to have much interesting to say. But you just still, we don't know. We certainly don't know. I think if we're humble, there could be some interesting trees out there. Well, they're probably talking to other trees, right? They're not talking to us. And so to the extent they're talking, they're saying something interesting to some other, you know, you know, conspicuous, as opposed to us, right?

2:56:08And so they probably is, there may be some signal there. I, you know, so there are people out there actually is pretty common to say that language, that human language is special and different from any other animal communication system. And I, I just, I just don't think the evidence is there for that claim. I think it's not obvious. We just don't know what, because we don't speak these other communication systems until we get better. I do think there's, there are people working on that as you pointed out, though people working on whale speak, for instance, like that's really fascinating. Let me ask you a wild out there sci -fi question.

2:56:45If we make contact with an intelligent alien civilization, and you get to meet them, how hard do you think of, like how surprised you be about their way of communicating? Do you think it would be recognizable? Maybe there's some parallels here to when you go to the remote tribes? I mean, I would want Dan Everett with me. He is like amazing at learning foreign languages. And so he, like, this is an amazing feat, right, to be able to go. This is a language, which has no translators before him. I mean, there were, he was measuring with that. Well, there was a guy that had been there before, but he wasn't very good.

2:57:20And so he learned the language far better than anyone else had learned before him. He's like good at, he's just, he's a very social person. I think that's a big part of it is being able to interact. So I don't know, it kind of depends on these, these, this species from outer space, how, how much they want to talk to us. Is there something you can say about the process he follows? Like, what, how do you show up to a tribe and socialize? I mean, I guess colors and counting is one of the most basic things to figure out. Yeah, you start that. You actually start with, like, objects and just say, you know, just throw a stick down and say stick.

2:57:53And you say, well, you call this and stick to this feature. And then they'll say the word, whatever. And he says, a standard thing to do is to throw two sticks at two sticks. And then, you know, he learned pretty quick that there weren't any count words in this language because they didn't know this was an interesting thing. I mean, it was kind of weird. They'd say some or something the same word over and over again. And so, but that is a standard thing. You just like try to, but you have to be pretty out there socially, like willing to talk to random people, which these are, you know, really very different people from you.

2:58:21And he was, and he's, he's very social. And so I think that's a big part of this is like, that's how, you know, a lot of people know a lot of languages that they're willing to talk to other people. That's a tough one. We just show up knowing nothing. Yeah. Oh, God. That's a beautiful, beautiful, like, humans are able to connect in that way. Yeah. Yeah. You've had an incredible career exploring this fascinating topic. What advice would you give to young people about how to have a career like that or a life that they can be proud of? When you see something interesting, just go and do it. Like I do that.

2:58:54Like that's something I do, which is kind of unusual for most people. So like when I saw the Peter, like Peter, how was available to go and visit? I was like, yes, yes, I'll go. And then when we couldn't go back, we had some trouble with the Brazilian government, there's some corrupt people there. It was very difficult to get go back in there. And so I was like, all right, I got to find another group. And so we searched around and we were able to find the Chaman because I wanted to keep working on this kind of problem. And so we found the Chimane just go there. I didn't really have we didn't have content.

2:59:22We had a little bit of contact and brought someone. And that was, you know, we just, you just kind of just try things. I say it's like a lot of that's just like ambition. Just try to do something that other people haven't done. Just give it a shot. It's what I, I mean, I do that all the time. I love it. And I love the fact that your pursuit of fun has landed you here talking to me. This was an incredible conversation that you're, you're, you're just a fascinating human being. Thank you for taking a journey through human language with me today. This is awesome. Thank you very much. Lexus. When pleasure.

2:59:55Thanks for listening to this conversation with Edward Gibson. The support that's podcast, please check out our sponsors in the description. And now let me leave you with some words from Woodkenstein. The limits of my language mean the limits of my world. Thank you for listening and hope to see you next time.