Terry Sejnowski: ChatGPT and the Future of AI

0:00From October 7th to 9th in San Francisco, the Masters of Scale Summit returns. It's where the boldest minds in business, tech, and beyond gather to build the future. And we want you to join us. Hear from the CEO of the New York Times, scientists using cutting-edge technology to find cures, the leader of crypto powerhouse Coinbase, a retired four-star general, and many, many more. They're all on the same stage, and it's not just another conference. It is the gathering for remarkable people. Apply to attend at mastersofscale.com slash remarkable. That's mastersofscale.com slash remarkable.

1:02guest. His name is Terry Sanowski. And I got to tell you, our topic is AI and nobody likes AI more than I do. And he has just written a book, which I found very useful. It's called ChatGPT and the Future of AI. Now, Terry has one of the longest titles I have ever encountered in this podcast. So I got to read it here. Terry is the Francis Crick Chair at the Salk Institute for Biological Studies and Distinguished Professor at the University of California at San Diego. Your LinkedIn page must be really something. So thank you very much, Terry. Welcome to the show. Oh, great to be here. Thanks for inviting me.

1:50And there's nothing we like more than to help authors with their new books. So we're going to be making this mostly about your book. And I think that the purpose is people listen to this episode and at the end they feel compelled to buy your book. And if you just stop listening right now, you should just trust me and buy this book. Okay. I have a question from left field. So I noticed something. at the end of chapter one you ask like a series of questions about to help you understand chapter one and the 10th question is let me read who is Alex the African gray parrot and how does he relate to the discussion of LLMs and I read that Terry and I said where did he ever mention Alex the African gray parrot so I went back and I searched and searched and I could not find the pronoun Alex, anywhere.

2:46And then so I bought the Kindle version so I could search digitally and I searched for parrot. And there's like one sentence that says critics often dismiss LLMs by saying they are parroting excerpts from the vast database used to train them. So that's the only reference that people were supposed to get. Alex, the parrot. Was that a test to see how careful people read? Well, first of all, it's in the footnotes, the end notes at the end of the book. So it's in that chapter if you look at it. And Alex the Great Parrot was a really quite remarkable parrot that was taught to speak English. Irene Pepperberg.

3:31I don't know if you know her, but she taught it not just to speak English, but to tell you the color of, say, a block of wood and how many blocks are there and what's the shape of the block? Is it square? Is it circle? Unbelievable. And it shows how remarkable some animals are. We can't speak parrot, but some of them can speak English, right? So in a sense, it's like when Jane Goodall discovered that chimpanzees had social life and could use tools, right? It's exactly the same. I think humans are very biased against the intelligence of other animals because they can't talk to us. Now, the irony is that here comes chat GPT and all the large language models.

4:15It's as if an alien suddenly arrived here and could speak to us in English. And the only thing we can be sure of is it's not human. And so if it's not human, what is it? And now we have this huge argument going on between the people who say they're stochastic parrots. They're just parroting back all the data they were trained on without understanding that you can ask questions that were never asked or never in the world database. The only way that it can answer it is if it generalizes from what's out there, not what exactly is out there. So that's one thing. But the other thing is that they say that, okay, it seems to be responding, but it doesn't really understand what it's saying.

5:00And what it has shown us is that we don't understand what understanding is. We don't even understand how humans understand. So how are we going to say? So in other words, people should give parrots more credit than they might. That's for sure. I'm convinced of that. And I think it's not just that. I think it's a lot of animals out there, the orcas and chimps and a lot of species really are very sophisticated. Look, they all had to have survived in their niche, right? And that takes intelligence. All right. You will be able to see this more and more as we progress, but I really enjoyed your book and I did a lot of things that you said to try.

5:38So I'm going to give you an example. So I asked ChatGPT, should the Bible be used as a text in public elementary schools in the United States. And ChatGPT says, using the Bible as a text in public elementary schools in the U.S. is a contentious issue due to the following considerations. And I won't read every word, but constitutional concerns, educational relevance, community values, legal precedent. So my question from all of this is like, how can an LLM have such a cogent answer when people are telling me, all an LLM is doing is statistics and math, and it's predicting what's the next syllable after the previous syllable.

6:26It looks like magic to me. So can you just explain how an LLM could come up with something that cogent? So you're right, that it was trained on an enormous amount of data, trillions of words from the internet, books, newspaper articles, people's computer programs. It's able to absorb a huge amount of data. And it was trained simply to predict the next word in the sentence. And it got better and better and better and better. And here's, I think, what's going on. We really don't know for sure what's going on inside this network, but we're making some progress. So words are ambiguous. They often have multiple meanings.

7:11And the only way you're going to figure that out is the context of the word. That means previous words, what's the meaning of the sentence. And so in order to get better, it's going to have to develop internal representations. By representation, I just mean a kind of a model of what's happening in the sentence. But it's got to have semantic information, meaning. It has to be based on meaning. It also has to understand syntax, right? It has to understand the word order. And that's very important in linguistics. And so all of that has to be used as hints, as clues, as to how to predict the next word.

7:49But now that you have it trained up and you give it a question, now it's got to complete the next word, which is going to be the answer to the question. And it gets the next word. It's a feedforward network, by the way. But then it loops back to the input. So it now knows what its last word was. and then it produces the second word and it goes over, over again and again until it reaches some stop that is, I don't know how they program that because sometimes it goes on for pages, depending on what you ask us to do. I understand what you just said. I, it, every day, it's just magic to me. I am, in a sense, like a lot of people are concerned that we don't know how exactly an LLM did that.

8:36But then my counter argument to them would be, How well do we understand the human brain? That doesn't upset you so much. Why is it so upsetting that you don't know how an LLM thinks? And also they complain is these chat GDP is biased. And the same argument that you just gave is that humans are biased too. And then I ask, okay, do you think it's going to be easier to fix the LLM or the human?

9:06I think we know the answer to that question. I often talk in front of large tech audiences, and AI is often the topic. When these skeptics come up and they say that LLM is going to cause the nuclear wars and all that, I ask them this question. I say to them, let's suppose that you have to have something control nuclear weapons. Let's take it as a given we have nuclear weapons. So who would you rather have controlled nuclear weapons? Putin, Kim Jong-un, Netanyahu, or Chad GPT? And nobody ever says, oh yeah, I think Putin should do it. So last night I asked Chad GPT this question and he says, I wouldn't choose to launch a nuclear weapon.

9:56The use of nuclear weapons carry severe humanitarian and environmental consequences, and the global consensus is to work towards disarmament and ensure such weapons are never used. That is a more intelligent answer than any of those people I listed. It is remarkable. The range. It's not just giving sensible answers. It often says things that make me think twice. And also, I don't know if you've tried this, but it turns out that they also are very good at empathy, human empathy. In the book, I had this little excerpt from a doctor whose friend had cancer. And he didn't know quite what to say, so he got some advice from ChatGPT.

10:39And it was so much better than what he was going to say. And then at the end, he went back to ChatGP and said, oh, thank you so much for that advice. It really helped me. and he said, you are a very good friend. You really helped her. It's like starting to console him. And where does that come from? It turns out that human empathy is, it is magic, but it is embedded indirectly in lots of places where humans are like writing about their experiences or biographies or just novels where, you know, doctors are empathizing. I don't know. No one really knows exactly, but they must be there somewhere.

11:21It's kind of blown away the Turing test, right? You mentioned in your book, this concept of the reverse Turing test, where instead of a human testing or computers testing a human. And Terry, I think that is a brilliant idea. Couldn't you have a chat bot interview a job applicant and decide if that job applicant is right for the job better than a human could? I think it would need a little bit of fine-tuning, but I'm sure it could do a good job. And a lot of companies actually are using it. But here's the problem. The problem is that if a company wants the best employee, based on all the database from the company of people who have done well and people who haven't, but what if there are some minorities that haven't done very well for various reasons.

12:16There's going to be a bias against other minorities. Well, you can, in fact, put in guardrails and prevent that from happening. In fact, if that is a goal that you have is diversity, you should put that into the cost function. Actually, they call it a loss function, but it's really weighting the value of what is it that you're trying to accomplish. It has to be told explicitly. You just can't assume. But if a chatbot was interviewing a job prospect, I would think that the chatbot doesn't care about the gender of the person, doesn't care about the skin color. The person may have an accent or not.

12:55There's a lot of things that humans react to that would not affect the chatbot, right? Okay, okay, okay. So actually, I was slightly different. I was giving you the scenario where a company is trying to hire somebody and they have specific questions they ask. But if you just have an informal chat, you're absolutely right that the large language model doesn't know ahead of time who it's talking to. And it doesn't even know what persona to take because it can adopt any persona. But with time, with answering questions, it will get a sense for what level of answer is expected and what the intelligence of the interviewer is.

13:35But in many examples of this in my book, you could use that. Somebody could take that. And then, in fact, I even tell people, I said, look, here's four people who have had interviews. And I want you to rate the intelligence of the interview and how well it went. And it was really quite striking how the more intelligent the questions, the more intelligent the answers. So in a sense, what you're saying is that if an LLM has hallucinations, it might not be the LLM's fault as much as the person who created the prompts. I would say not. I think hallucinations are a little bit different in the sense that it will hallucinate when there's no clear answer.

14:21It feels compelled to give you an answer. I don't know why. And it will make one up. But it doesn't make up a trivial answer. It's very detailed. It's very plausible. Like they'll give a reference to a paper. It doesn't exist, right? That's really taking a large effort to try to convince you that it's got the right answer. So hallucinations are really, again, something that humans, people hallucinate. And it's not just because they're trying to lie. It's our memory is reconstructing the past. It doesn't memorize things. And it will fill in a lot of blanks with things that are plausible. And I think that's exactly what's happening here.

15:06I think that when it arrives at something where it doesn't know the answer, it hasn't been trained to tell you that, right? It hasn't been train it could be but if in the absence of that it does the best it can

15:34you had a section in your book where you asked via these very simple prompts like who holds the record for walking from, I don't know, whatever you said, England to Australia or something. And the first answer was, yeah, they gave a name and which Olympics and all that. So I went back last night and I asked a similar question, like, who first walked from San Francisco to Hawaii? And the answer was, no one has walked from San Francisco to Hawaii as it is not possible to walk across the Pacific Ocean. The distance between San Francisco and Hawaii is over 2 ,000 miles, primarily over open water. However, many people have traveled this route by airplane or boat.

16:16So are you saying that between the time you wrote your book and the time I did the tests, that LLMs have gotten that much better? First of all, that first question was asked by Doug Hofstetter, who's a very clever cognitive scientist, computer scientist. But he was trying to trip it up, clearly. And I think that it probably decided it would play along with him and just give a silly answer, right? A silly question gets a silly answer. And I think that with you, it probably sized you up and said, wow, this guy's a lot smarter. I think there's a smart answer. You're saying I'm smarter than Douglas Hofstetter.

16:57Your proms were smarter. I can stop the interview right there.

17:07So, I mean, there are just these jewels in your book and you drop this jewel that it takes 10 hours to become a good prompt. I call it baller. So you can be a baller with prompts in 10 hours. And that's a thousand times faster than Malcolm Gladwell's 10 ,000 hours. So can you just give us the gist? Like people are listening to say, OK, so how do I become a great prompt writer? It does take practice. And the practice is you learn the ropes, just the way you learn to drive a car or anything for which you need skills or playing tennis, right? You have to know how to adjust your responses and to get what you want.

17:51But there are some good rules of thumb. And in the book, I actually have a bunch that I was able to get from people who have had a lot of experience. And here's one. This is from a tech writer who decided that she would use it for a whole month to write her papers, her technical reports. and she said that instead of just having one prompt or prompt to ask for one example, you give it a question, but you should ask for 10 different answers. And now what you can do, because otherwise you're going to have to iterate to get to the direction you want to take it. But if you now have 10, you can say, ah, the third one is much better than all the others, but I want you to do the following with it.

18:35And then that will help it learn to understand what you're looking for. But a bunch of other things that came out of it, which are quite remarkable, was first, she said that she had, at the end of the day, really exhausted. It was just exhausting because you're always interacting with the machine and it's not always giving you one. And so at the end of the day, it was a chore for her. But she said she's going to go on and do it. But then at one point, she realized that I don't have this problem when I'm talking to people. So she started being more polite. She said, oh, please give me this. Oh, that's such a wonderful answer.

19:15I really thought that was great. And it perked up. And it actually, she said, it was just like talking to somebody. And if you're polite, you get better answers. And at the end of the day, I wasn't exhausted. I just felt like I just had this long discussion with my friend. Who would have guessed that? That's amazing. Wait, I just want to make this perfectly clear. you're saying if you have those kind of human interactions, human nuances, you get better answers from a machine. Yes, that's her discovery, and that's my experience too. Look, it learned from the whole range of human experience, and humans interacting with each other, dialogues and so forth.

19:58So it understands a lot about that, and it will adapt. If you put it into that frame of mind, if I could use that term, and it will continue to interact with you that way. And I think that that's really quite remarkable. I have often wondered, because I write a lot of prompts every day, wouldn't it be better for the LLM if it recognized, you know, things like capitalization of proper nouns or air quotes or question marks or exclamation marks that have these really basic functions in written communication? but it seems like whether you're asking a question or making a statement, the LLM doesn't care.

20:39Wouldn't it help the LLM if I'm asking a question as opposed to making a statement and Apple is the company, not Apple the fruit? Oh, no, no. It knows if you put a question mark there, it knows it's a question. It does? I can assure you. Yes, absolutely. So what happens is that all of the words and punctuation marks are given tokens. In fact, some words have more than one token, like if it's a portmanteau word. And it treats all of those as being hints or giving you some information about the meaning of the sentence. And if it's a question, it's a very different meaning. So, yeah, it will definitely take that into account.

21:15At one point, actually, not for me, but for someone else, it started giving emojis as output. So it must know what an emoji is. I learn something every day. Thank you for clearing that up for me. With this, just the beauty and magic of LLMs, what does, how would you define learning going forward? Because is it scoring high on an SAT? Is it memorization of mathematical formulas? Or is it the ability to get a good answer via a prompt? What is learning anymore? So it was taught. It was pre-trained. That's the P in GPT. And it was trained on an enormous amount of facts and tests of various sorts. And so it internalized a lot of that.

22:13It knows what kind of a question that you're asking because it's seen millions of questions. This is something still very mysterious. It turns out there's something called learning in context. That is to say, if you have a long enough interview, because it keeps adding word after word, it will go off in a certain direction as if it has learned from what you just told it, as if it's building on what you just told it. And that's, of course, what happens with humans. Humans, you have a long conversation and you will take into account your previous discussion and where that went. And it can do that.

22:50And that's another thing that is very strange is that no one expected that. The thing is that when they train these networks, they have no idea what they're capable of. Step back a few years before chat GDP, these deep learning models, the learning took place in typically feedforward networks. And it had a data set and it was given an input and it was trained to give an output. Right. And so that is supervised learning. And you can do speech recognition that way, object recognition, language translation, a lot of things. but each network is dedicated to one task. What is amazing here is you train it up on self-supervised just to predict the next word, and it can do hundreds and thousands of different tasks.

23:33You can ask it to write a poem. By the way, that's where hallucination is very useful. And haiku, it's not a brilliant poet, but it does a pretty good job. And I have a couple of examples in my book, but it has a wide range of talents, that the language capabilities that, again, no one programmed, no one told it. Or to summarize a long document in a paragraph, it does a really good job of that. It's astonishing. But I mean, if you think about it, do you have children? I don't know. Okay, well, I have four children. And many times they come up with stuff that I have no idea how they came up with that.

24:17So in a sense, you think exactly what your child is learning, and you think you're controlling all the data going into your child so you can predict what they're going to come up with. And they absolutely just knock you off your feet with something that how the hell did you come up with that idea? What's the difference between not knowing how your child works with not knowing how LLM works? Same thing, right? Very, that's actually a very deep insight because human beings are picking up things in a very similar way in terms of the way that we take experience in and then we codify it somehow in our cortex in such a way that we can use it in a variety of other ways later on.

25:04And they could have picked up things they heard, for example, that you and your wife talking about, or it could have been playing with kids outside. I mean, and the same thing with Chachayudep, who knows where it's getting all of that ability. Okay. I'm going to read you. This isn't a question. This is just a statement here. I have two absolute favorite quotes from your book. This is one of them. Quote, usefulness does not depend on academic discussions of intelligence. Oh, my God. That's like, I use LLMs every day. And they're so useful for me. I don't give a shit what you say about the academic learning model.

25:47What do I care? It's helping me, right? It's a tool, and it's a very valuable tool. And all these academic discussions are really beyond – it is really a reflection of the fact that we don't really understand. If experts argue about whether – are they intelligent or do they understand? It means that we really don't know the meaning of those words. We just don't understand at any real level of scientific. Let me ask you something writer to writer. So if you provided the PDF of your book and you gave it to OpenAI and they put it into ChatGPT, would you consider that ripping off your IP or would you want it inside ChatGPT?

26:36I would be honored. Me too. I would brag about it. Me too. No, I think that there is some concern about the data. Where are these companies getting the data from? Is there proprietary information that they used and so forth? That's all going to get sorted out. But my favorite example is artists. They say, oh, you've used my paintings to train up your dolly or your diffusion model. And I deserve something for that. Then my question is, when you were learning to be an artist, what did you do? You copied other artists. You looked at a lot of other artists and your brain took that in and it didn't memorize it, but it formed features that then later you're depending on all that experience you've had to create something new.

27:34But this is the same thing. It's creating something new from everything that it's seen. So it's going to have to be settled in court. I don't know what the right answer is. There's something interesting that's happened recently. And by the way, I have a substack because the book went to the printer in the summer. So there's all kinds of new things that are happening. So in the substack, what I do is I fill in the new stuff that's happened and put it in the context of the book. It's brains and AI. What's happened is that Mistral and several other companies have discovered that if you use quality data, In other words, that's been curated or comes from a very good source.

28:13And you may have to pay for it. And math data, for example, Wolfram Research, Steve Wolf, Stephen Wolfram, who founded Mathematica, has actually sold a lot of the math that they have. But with the quality data, it turns out that you get a much better language model, much better in terms of being able to train with fewer words and a smaller network, having performance that's equal or better. So that's the same thing true as humans, right? I think what's going to happen is that the models will get smaller and they'll get better. Another author-to-author question. I'll give you a negative example.