Leslie Valiant : The Importance of Being Educable

0:00From October 7th to 9th in San Francisco, the Masters of Scale Summit returns. It's where the boldest minds in business, tech, and beyond gather to build the future. And we want you to join us. Hear from the CEO of the New York Times, scientists using cutting-edge technology to find cures, the leader of crypto powerhouse Coinbase, a retired four-star general, and many, many more. They're all on the same stage, and it's not just another conference. It is the gathering for remarkable people. Apply to attend at mastersofscale.com slash remarkable. That's mastersofscale.com slash remarkable.

0:49I'm Guy Kawasaki and this is Remarkable People. We are on a mission to make you remarkable. Helping me in this episode is Leslie Valiant. He is a professor of computer science and applied mathematics at Harvard. Valiant received a B.A. in mathematics from King's College, a diploma of Imperial College from Imperial College. That makes sense. And a Ph.D. in computer science from the University of Warwick. Prior to joining Harvard in 1982, he held faculty positions at Carnegie Mellon University, the University of Leeds, and the University of Edinburgh. Valiant has made foundational, remarkable contributions to computer science, including the theory of probably approximately correct learning, the concept of P-completeness in complexity theory, and the bulk synchronous parallel processing model.

1:47And if you believe I understand all of what I just said, you are in for a big surprise. Valiant is the recipient of the Turing Award, the highest honor in computer science. The award committee said this about him, quote, Rarely does one see such a striking combination of depth and breadth as in Valiant's work. He is truly a heroic figure in theoretical computer science and a role model for his courage and creativity in addressing some of the deepest unsolved problems in science, end quote. Valiant's book, The Importance of Being Educable, explores the unique ability of humans to absorb and apply knowledge.

2:30He argues that understanding our educability is crucial for safeguarding our future, especially with the rise of artificial intelligence systems. I'm Guy Kawasaki. This is Remarkable People. And now, here is the remarkable Leslie Valiant. What exactly is wrong with the term intelligence? It is that we don't know what it means. It's got no definition. No one can tell you exactly how to recognize an intelligent person. What's the behavior you're supposed to look for? One maybe trivial symptom of this, which may not be that important, but just as a symptom. So there's much discussion about the SAT test, which many American students do.

3:17and I was surprised to find out recently what the letter A stands for. So certainly S stands for standard and T for test. But what does the A stand for? So apparently the A used to stand for aptitude, standard aptitude test. They changed that. Then it was standard assessment test. They changed that. Then it was SAT colon reasoning. They changed that. So now officially the A stands for nothing. It's just a brand. And so whatever it tests for, the society is buying without any label on what it's meant to do. So if you buy food in a store, it says more than that there's a correlation between you being less hungry afterwards and eating this.

4:00There's a correlation between these SAT tests and something, and that's all that's promised. So the problem with intelligence is that it's widely used, but it's got no definition. Isn't that fixable? Can't we get MacArthur Fellows and Turing Award winners and all you guys together say, can you just please define intelligence? How hard could that be? So in the book, I quote some reports of the American Psychological Association some time ago where they report on some other group saying that they asked 24 experts in intelligence and they all gave somewhat different answers. So there's some concepts for which we have a word, but we don't really know what it means.

4:41seeing as how you're saying that there's no sort of agreed upon consistent definition of the word how are we seemingly freely talking about ai so much right now how can we talk about artificial intelligence if we don't even know what intelligence is exactly so in fact i think using word intelligence didn't help the ai field to progress because there was no goal there In my view, it progressed because AI adopted a more specialized definition, or rather used one, and this is learning. So learning is something which you can define much better. So learning from examples, after you've seen some examples, you can classify future examples better than you could before.

5:24So learning from examples is something that's a very specific ability. It can be defined. The computer programs are doing quite well. And in practice, these large language models do exactly that. Okay, so there's some evidence that they do something interesting. So AI, in my view, has progressed by trying to realize by computer tasks for which there is a definition. In a sense, what you're saying is that intelligence is such a quagmire that you're just going to go outside the box and you're going to create this new standard. I'm not saying I want to create a new standard, though that's not how I start.

5:58But I'm saying how I phrase it in the book is that in some sense it's a very old question, of course, is what's the intellectual capability which humans have which other species on Earth do not and the capability which enabled us to create our complex civilization, science, and everything else. And although this may seem an overly ambitious question, in some sense it's not because in evolutionary terms we evolved pretty fast. in the short term this must have evolved. So I'm trying to write down this capability. And the first part of this is exactly what I was describing with learning from examples.

6:39So it's something which we understand well in computers. Everyone sees how it works with these large language models in the last year or so. And then I add a few things on. So I think I'm doing a natural science. I'm trying to explain human capabilities. And I think it's something real, which has a definition. Whether it turns out to be useful, that remains to be seen. And in some sense, it is a replacement for intelligence, but in a special way. So I'm curious that in the creation of your book and the title and really the writing of the entire thing, did you ever think about the difference between educatable versus educable?

7:20Did you ever think of the nuances between those two words or now in modern use, they're equivalent, so it doesn't matter? I'm not quite sure how you mean any of those. So at one point, these terms were used in the sense of yes, no. So which children could be educated in a regular school as opposed to having to be sent out? So I think that's the use these terms had maybe 50 years ago. Maybe that's the way in which you mean educatable, right? But what I'm trying to capture is to define more ground up the capability, which I think all of us have to some extent. And I'm not trying to distinguish whether some of us have more of it than others.

7:59Just generally, we're trying to distinguish us from other species. So it's not to be confused with the other senses in which this word may have been used in the past. It's like a new technical term, if you like. Okay. Now, from your point of view, is this level of educableness, if that's the word, is it a person's or a machine's innate ability or is it also subject to the methods and resources and environments around that person or is it just the person? So the capability itself, there's some of it there at the beginning. The question is whether it can be enhanced by some sort of life experience.

8:38I don't know. So I think that's a good subject for investigation. Certainly if you can enhance the rate at which you can learn, for example, that would be fantastic for society. But I think that's a subject for research. I should add that most capabilities can be enhanced by some sort of training and the right environment. So the default assumption has to be that this also can benefit from suitable interventions. But I don't know. You're saying, theoretically, if we could do this controlled experiment and we took you or we took Stephen Wolfram or we took Neil deGrasse Tyson or, I don't know, Freeman Dyson or something, and we stuck you in the middle of the Amazon jungle with no internet, no books, no nothing.

9:20Clearly, you're still the same person. If we go back 20 years later, are you running the jungle? Are you the leaders there? How much did where you are in modern society play into the fact that you're such overachievers? Well, first, I'm not equating. So let's suppose that you can measure educability. Different people have different levels of educability. Let's assume that. I'm not even certain of that, but let's assume that. What it correlates with in society, again, I don't know. It's not obvious. the most educable people may not be the ones who are most successful in politics or academia or whatever podcast it i'm not sure i don't want to speculate but it would be if you put someone in in the jungle then educability would cause one to how fast they can learn what's going on there relative to what they know obviously maybe they know so little that's relevant to how to survive that the educability isn't enough to survive.

10:18But it's a measure of how fast you can learn and understand what's going on. If I had to put my money on you or Stephen Wolfram to figure out Kurare before the random person in the jungle, I think that's a pretty good bet. So I don't want to speculate, but certainly there's some people. It's maybe the entrepreneurs who are very good at picking up lots of information, sitting through lots of information, understanding what's going on fast. So I'm not quite sure which profession exhibits educational the most. For example, scientists obviously need it, but they may not be the most extreme because they're also good at concentrating for a long time to pursue a direction.

10:58Educability may be more where you keep finding out new things all the time and relating everything to each other and running with it.

11:20Are you familiar with the work of Carol Dweck and the growth mindset? No. She's a professor at Stanford and she, in the early 2000s, pioneered this dichotomy between the growth mindset and the fixed mindset. The fixed mindset basically says that you think you are what you are. You cannot be anymore. You cannot learn new skills. it also means you think you cannot deteriorate. So if you're a genius, you believe you'll always be a genius. And Carol Dweck's theory is that the growth mindset means you can learn things, you can do things. And it seems to me there's a great deal of parallel between your theory of educable and the growth mindset.

12:03So I thought that maybe great minds thought alike and you knew each other. No, but I mean that theory may be more about one's attitude. But I think this educability is something which kind of everyone has. So the fact that most people are gathering information all the time, they're watching movies for example, they're reading novels, they're looking at their cell phones, they're all soaking in information. Much of the information is of no relevance to them, it's got no benefit to them, but that's what they do, they soak in information. so I think it's this distinction whether you regard your ability to soak in information whether you're going to do it useful to yourself to benefit yourself to find a new profession I think that's not exactly the same thing I think what I'm describing is something which we freely all have quite generously I think In your book you mentioned this example of chimps escaping from their cage in a zoo using a fallen tree to jump over the wall.

13:02And then the other chimps watched that and learned and also jumped over the wall. And so are you saying then that in this case, these animals are educable? Is that the conclusion to draw that they passed the test for educable? No, I'm saying that they look pretty smart and they can solve problems. And they can obviously, the first one to jump over the wall, figured out what to do, assuming that hadn't seen something like that being done, and the rest could copy. But so by educable, described in great length in the book, so it's a combination of learning from experience, being able to train together what you've learned, and some kind of reasoning.

13:42And then the third part is being able to learn from someone else, describing too explicitly what should be done. So in some sense, copying behavior is like that, but in humans, it's much more general that we sit in. If you go to college, you sit in a lecture room, and someone One tells you the laws of quantum mechanics, and you sit there for 30 lectures, and at the end, you can do a lot of stuff. So I think it's only humans who have this ability to be able to soak in a lot of information given explicitly. So it's like other people having had the experience, other people having done the experiments, but they can tell you the conclusions, and you can internalize it yourself and use it as if you had the experience yourself.

14:21Have you made observations yet about the presence of educableness in life? Is it normally distributed? Is it distributed differently to genders? Does your educableness, does it grow or decay chronologically through life? Have we had some longitudinal studies and cross-cultural studies, cross-gender studies to understand more about this concept? No, we haven't had any studies. It's a new concept. We haven't had any studies. So to answer any of the questions, you'd have to develop some test, which you plausibly think does measure educability. And these tests would have to be of the nature that you're – like if you do a one-hour test.

15:05In principle, you could have a test which is like a one-semester course, or you could have a one-hour test. But what you'd be measuring is how much you've learned during that one hour. So you're given some questions where it wouldn't help you to have previous knowledge to answer the questions. So it's related to current IQ tests, but it would have a different emphasis. So it's a new kind of question. But the only related point I'd make is that if one accepts this notion that what characterizes humans is its educability, which means an extreme ability to learn and soak up information, then this also illustrates why it's very difficult to answer the questions you ask i mean to try to prove differences between groups in abilities and other abilities is difficult just because you know if you make some measurement of the performance of a group today somewhere else at a different time the outcome may be different because the group may have had different experiences so we are being so subject so extremely subject to outside influences throughout educability, that we're the least promising objects of scientific study, if you like.

16:15By doing surveys on humans in different places, you shouldn't be surprised that these results aren't transferable, generalizable from different places at different times, just because we're so prone to change. The point of educability is that we're so prone to change, so prone to be influenced by our environment that it's very hard to answer any of the questions you ask. if you think about how would you possibly control all the variables in an experiment like that right although i gotta tell you if i had to guess i would tell you that women are more educable than men there's no doubt in my mind but i don't have any objective proof for that just my life experiences so if we're at such like the starting point of all of this let's say somebody's listening to this and say, yeah, you know what, really, I want myself, I want my kids, I want my company to be more educable.

17:07What can I do now? Give me some tactical and practical stuff, Leslie. Help me out here. I don't think I can. I'm sorry. I think the only rational thing is to do research and develop some tests which plausibly measure this kind of thing. So if we can't measure then there's not much we can say about it. So I think it's a long-term project. The question of whether we can improve our eduquability, I think it's a long-term project. Of course, many people have ideas about learning to learn and teaching to learn, but to validate any of these things is difficult without a measure of when you can declare yourself to be successful in having enhanced learning.

17:50If you were to just take a social welfare perspective and step back, it's hard to imagine more things that could be more important than this. This could be the key to the survival of mankind to figure out how to make people more educable. Personally, I agree. I do. So I think that's, yeah, I think exactly. I think it's a public issue for discussion. Listen, let me give you some shallow thinking. Let me see if this metaphor will work for you. Is it fair to say that intelligence, even though we can't really define it, intelligence is like the chip speed and educableness is programmability? Is that a fair statement?

18:41No, I don't think so. I think educability is, supposing let's talk about a computer rather than a human, educability is like a discussion of what the capabilities are of this thing. and intelligence, I don't know what that is. I really don't know what intelligence is. Some people say intelligence is what a standard intelligence test measures, which is a bit circular. But again, I don't quite know what that is. Many cognitive capabilities are correlated. If you're good at one thing, often you're good at something else. It's often a weak correlation, strong correlation. So these intelligence tests are just something maybe arbitrary, which is correlated with many things.

19:23So colleges may want to find this out because it may be correlated with how well students do. But many other tests would also. So I really don't know what intelligence is. Personally, I'm really... So I don't think I go around and say, oh, this person, my neighbor's intelligent. I don't think I would naturally say that. Orthogonal to this, is there no concept that we need to take in account something like morality, right? So that morality, you could be the most educable person in the world. Not that I believe this, but let's just suppose for a second that we think Donald Trump is educable, but I would say he has zero morality, right?

20:05You could take an educable person who could learn from others and extrapolate and do all these great things, but what if that person is fundamentally evil, then what? Then it's unfortunate. But certainly, educability only defines exactly, as they say, one's basic cognitive capabilities. And it's about how Faisal's book is. I talk about beliefs, that you learn beliefs. Other people have beliefs. They tell you their beliefs. And so, in that sense, the theory is about capabilities. So, in one sense, it's morally totally neutral. It doesn't discuss morality because, as you say, the beliefs could be good or could be bad.

20:45But what one's reaction to this should be is it shouldn't be moral neutrality. As you say, some people have different beliefs and some we think are evil and some we think are good. And just because our capabilities are neutral to morality, it doesn't mean that we should be. We should still certainly fight for what you believe is right and against things which you believe are evil. But a fact that because our basic capability is kind of neutral on morals, totally bad beliefs can spread through society. We don't seem to have any good defense against that. If people are listening to this podcast and they're struggling with this educable concept, do you have any people that might be well-known in the valiant hall of fame of educableness that you can say, oh, I understand that now he cites this person as educable?

21:39Is there anybody like that you can say, think of him or think of her when you think about educability? You mean someone who shows educability in the extreme? Yeah. Who's your hero? Who's in your hall of fame of educableness? I don't know, but I sincerely believe that this is something which we all have. I'm not trying to define something which will separate us. I'm really trying to define something which we all have and which unifies us. So I think the education is very important. Otherwise, I wouldn't have spent so much time on it. But I don't know, as we said before, what this correlates with exactly, what human performance correlates best with, what kind of people exhibited the most.

22:20Yeah, so at this point, I just don't have a good answer to your question. It's a natural question, but it's not the direction in which I'm looking. Okay. If we can shift gears slightly towards artificial intelligence, I know I just used the I word, but that's what everybody used, right? At this point, when people say artificial intelligence, do you think it means that it refers to what a machine can do like a human or what a machine cannot do like a human? Like what is artificial anymore? I think the meaning of the term has changed in history, but at the moment in the media, it clearly means the kind of things which current machine systems can do.

23:02Okay, so by AI, people seem to mean large language models and things for which you can download some software which they can do. So it's in the area of machine capabilities and manipulating language pictures. So I think that's what it means now. So it's artificial in the sense that it's a machine doing what humans can do, and that's what makes it artificial? Yeah, the artificial part always means that it's machines doing it. Now, in your opinion, is ChatGPT educable? No, no. Basically, ChatGPT does one of the three requirements of educability, which is learning from examples. So it's trained to predict the next syllable of text.

23:45That's what it's trained for doing, basically. It predicts syllable by syllable texts from being given billions of examples. Now, I understand at a technical standpoint, but from the outside looking in, if you just give it a series of prompts, I don't think most people would look at it like, oh, here comes syllable after syllable very rapidly. It looks very cogent and salient to me. What's going on there? How does that thing work then? It's trained on a very large number of sentences. So the next syllable will be very likely to come from some sentence or some phrase which has been uttered before many times.

24:23And also it works on very large windows of text. So it predicts the next thing on hundreds of characters. So it does give the impression of some sort of stream of consciousness as if it can remember what it's talking about for a while. Because it's got this very large bit of text from which it's predicting. but it does use of the fact that it's got vast numbers of sentences stored which it can use. And so some of the mystery is that with machine learning in general, intuitively what happens if you've got billions of examples is something which is really counterintuitive. So if you train on these number of examples people never looked at before, then the phenomena are almost different and kind.

25:08It's okay being impressed. I mean, it's impressive. Certainly the smoothness of the sentence is amazing, but I don't think one should assume that these things are providing you more than what it's doing, predicting the next syllable. So certainly you shouldn't take its advice, for example, for some important decision you have to make. But if you were to take a Turing test orientation and now, okay, you're going to have to correct me if I get this wrong, but the Turing test is if you're interacting with this thing and you don't know if it's a human or a machine, but you can't tell the difference between a human and a machine, isn't it a human?

25:45Yeah, okay. So Turing wrote this paper in 1950, where he discussed the word thinking and also intelligence and basically said that, yes, if you can't tell the difference between a human and a machine, then you can't say that the human is thinking and the machine isn't thinking. So this is like almost a definition of thinking. So in some sense, it's the opposite of what I'm trying to do. I'm saying that you should define what thinking is and what intelligence is. So Turing didn't do this. He said, it is what it is. If it looks like thinking, it's thinking. So Turing tested whether the machine can effectively impersonate a person.

26:21But again, I don't know how far that takes us. So for example, if you look at these large language models, In some sense, it looks like a human. So the Turing thing is that you do some 20 questions with it. You ask questions and see what it answers. And, for example, you'll find that large language models don't know about yesterday's news because it was trained a long time ago. Okay, so clearly it doesn't pass the Turing test technically. Okay. But the point is that the Turing test philosophically, of course, was very important. It intrigued people for decades. But by itself, it doesn't tell you what to do to make an intelligent machine.

27:02It just says that you shouldn't quibble philosophically. So the fact that chat GPT is not educable, does that make you more comfortable with everybody jumping on large language models or does it scare you more? Is it a good thing or a bad thing that it isn't educable at this point? One way of saying it is that with a group at all levels, there are some choices to be made and it's not clear what the best choice is. With just learning from examples, as we all know now, we always did, the training set is all important. For example, these large language models can have a political orientation depending on what text it's been trained on.

27:48They could have a bias, it just depends on what it's trained on. Even at this level, it's not clear what you should be doing, right? I mean, depending on your political orientation, you may want to train it a different way. Already we're disagreeing on what the ideal large language model is. So if you make things educable, then the problems pile up. Because if you just tell your system beliefs, then it's pretty important to question what beliefs you give it. So maybe telling it evil beliefs is just such a good idea. So although we're wonderful, we think, and educable, there are all kinds of problems with using our educability.

28:24No one knows what the best method of education is. No one knows what the best beliefs are to instill in people. No one knows what right beliefs are and wrong beliefs are. So if we made machines with the same capabilities as humans, it wouldn't solve anything because we don't quite know what the best thing would be to educate humans. Okay, so if you have an educable machine, you have to educate them. and many choices to be made in education.