#178 Terry Sejnowski on Integrating Human Development Principles into AI Models

0:00I think where the future is going to be is trying to integrate more of the parts of the human brain that are important during development to help humans integrate into the culture. that they've been born into, right? That doesn't happen immediately. That requires, you know, long childhood and then adolescence. None of that is incorporated right now into these large language models because they're just built from scratch and then put out into the world. They don't have an opportunity to live through a period where they're given a lot of feedback from society about, you know, what's good and what's bad and what's expected and that sort of thing.

0:40And this is, you know, how would you expect a model that was only exposed to a lot of words out there to have absorbed any of the guidelines of how human beings behave and how we interact with each other. It's amazing that they are able to do as much as they have done, but there's just a lot more that needs to be put into those models. Hi, my name's Craig Smith, and this is Eye on AI. In today's episode, we explore computational neuroscience and artificial intelligence with Terry Sanowski, president of the NeurIPS Foundation and a distinguished member of the Salk Institute for Biological Studies.

1:23We delve into the evolution of neural networks, the monumental impact of models like GPT-4, and the nuanced interplay between AI technologies and societal structures. Terry offers profound insights into the trajectory of AI development, the ethical considerations that come with it, and the collective pursuit of lining AI with human-centric values. I hope you find the conversation as enlightening as I did. AI might be the most important new computer technology ever. It's storming every industry, and literally billions of dollars are being invested. So buckle up. The problem is that AI needs a lot of speed and processing power.

2:09So how do you compete without costs spiraling out of control? It's time to upgrade to the next generation of the cloud. Oracle Cloud Infrastructure, or OCI. OCI is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds, offers one consistent price instead of variable regional pricing. And of course, nobody does data better than Oracle. So now you can train your AI models at twice the speed and less than half the cost of other clouds. If you want to do more and spend less, like Uber, 8x8, and Databricks Mosaic, Take a free test drive of OCI at oracle.com slash IonAI.

3:05That's E-Y-E-O-N-A-I, all run together. Oracle.com slash IonAI. That's oracle.com slash IonAI. Just give a very quick introduction. I'm here at the Neural Information Processing System's 37th annual conference, and I'm the president of the foundation that organizes it. So that's, you know, the most immediate job that I have. But I'm on the faculty of the Salk Institute for Biological Studies and also on the faculty at the University of California, San Diego, and read across the street from each other. And my area is bestraddles computational neuroscience, understanding how the brain computes, and artificial intelligence, or what we used to call neural network models.

3:58The N in NeurIPS stands for neural, right? So it's always been, at the very beginning, an important inspiration for people who are trying to develop new approaches, computational approaches to solving very difficult real-world problems. And as it turns out, of all the machine learning algorithms that have been developed over the last 37 years, the ones that began back in the 80s, the ones that were developed based on learning algorithms and neural network models have been the most successful. And part of the reason has to do with scaling. A lot of algorithms, they blow up as the difficulty of the problem, the number of parameters grows, they become intractable.

4:47But it turns out that the ones that we developed, which are based on a large number of simple units that are highly interconnected, and then learning the strengths of the connections between them, it turns out that those algorithms have scaled beautifully. Now, you know, the recent large levels that have been developed, like GPT-4, have trillions of parameters, trillions, and that couldn't have been done with a lot of other approaches that were developed, like Bayesian approaches or graphical models. There's a subset who pay a lot of attention to the principles of how biology has constructed our brain.

5:39And it's not like you have to become a neuroscientist. you can extract some very, very important general principles just by looking at the different structures of the brain. And one of the talks that I heard was really quite interesting, pointing out that the models that we have today couldn't survive on their own without humans at every single stage in terms of collecting the data, curating the data, training, testing, and then putting in all the engineering guideway, guardrails and so forth. And, you know, it's kind of a very, very labor intensive process to actually engineer one of these systems.

6:26It takes hundreds of engineers and then to put it into the real world. And this is what happened over the last year with chat GPT, that requires real focus and it requires paying attention to things that are very, very, you know, important, for example, for privacy and safety, you know, reduce the bias, you know, align the models so that they are compatible with humans. And those are all difficult things to do. And those are engineering problems. But there is, like I say, a small group. And interestingly, a lot of them are a deep mind. And they have focused on reinforcement learning, which is a very important part of our brains.

7:22Reinforcement learning is essential for aligning human beings, for example, it's a reward circuitry. and that's I think where the future is going to be is trying to integrate more of the parts of the human brain that are important during development to help humans integrate into the culture that they've been born into right that doesn't happen immediately that requires you know, long childhood and then adolescence. And that none of that is incorporated right now into these large language models, because they're just built from scratch and then put out into the world, they don't have an opportunity to, to live through a period where they're given a lot of feedback from society about, you know, what's good and what's bad.

8:17And what's, what's the what's, what's expected and and that sort of thing and this is uh you know how did how would you expect a model that was uh only exposed to a lot of words out there a lot of them uh to to uh to have absorbed any of the the guidelines of how human beings behave and how we interact with each other it it's amazing that they are able to do as much as they have done but there's just a lot more that that needs to be put into those models. Yeah, and I follow fairly closely the world model development research. Is that something that you're involved in, which is taking data more directly?

9:08There are already a couple of really good companies out there that have robotic devices that are collecting vision and all sorts of sensory data, but also incorporating that into action. And already we have with GPT-4, you can put in a video in the form of images. And I saw a really nice demo which had to do with replicating the demo that Google introduced a few days ago when they were introducing Gemini. It turns out it was very heavily edited, embarrassingly. And he just decided, well, I do GPT-4. I'll just go through the same demo in real time. Just do it. He did it, right? So in any case, yeah, there's a lot of, I think, excitement right now because of the fact that it looks as if as the models have gotten bigger, they've been able to solve more difficult problems.

10:24And that's true, for example, in vision. But now it's happening in natural language processing. And there's, you know, we reached the point now where we have models that can do many things. That's what a foundation model is. It doesn't just, it wasn't trained to do one thing, but it can actually very rapidly one-shot learning, solve a lot of natural language processing problems. Just give it an example and ask it what you want it to do. but it might have lots of other capabilities that we just haven't asked it yet, right? So who knows? You know, this is terra incognita. We don't know yet what the full range of capabilities is, or on the other flip side of it, some of the faults, some of the failure modes.

11:15You know, we do the best we can in terms of the ones that are obvious, but there are going to be unintended consequences that nobody can expect. And we'll just have to wait and see. One of the trends already that is apparent is that theory is coming online where people now are beginning to analyze these large networks and try to understand a little bit about what happens during learning and also to try to understand the internal representations. And that's exciting because it might give us some insights, new insights, that may help us understand not just how these transformers work, for example, but also how the human brain works.

12:03And we desperately need help with that because the brain is a very highly evolved device that has orders of complexity beyond what we can right now create. But the other thing that is apparent, and this has been going on now for quite a while, but when NeurIPS first began 37 years ago, the internet was just about to, you know, become public and be out there. And, you know, 37 years later, it's changed every part of my life. Everything that we do now is oriented toward information flow through the internet. And I don't think anybody really saw that coming. And I remember back then when people talked about internet time.

13:01So that's the speed with which things were changing. And if you think about it, these are massive changes in society, in the way that companies sell products, Amazon, the way that video is streamed. Music is completely disrupted the music industry right now. they're getting more money from streaming than they are from physical, you know, I'm not sure what they sell now, actually. But, you know, the world is changing now on an even faster pace. AI time is not measured in years or decades. It's measured in months. In fact, it was really funny, this brouhaha at OpenAI. That was all over in a week.

13:51In fact, I was joking with some of the people in those companies and saying that, you know, Sam Altman came back from the dead in three days, but he didn't hold the record. Jesus got back in two days. We're going through a period of, like I say, accelerating technology And, you know, it doesn't give a lot of time to explore some of the issues that having to do with, you know, now that the public is interacting with AI. And that's a cause for concern. However, in some ways, the government really is missing, I think, its role. It hasn't played the role in AI that, for example, it did play in physics and some of the other, like nuclear energy and so forth, because things are moving so quickly.

15:05But right now, all of the major models actually have been coming out of these big high-tech companies because only they can afford to run 20 ,000 GPUs for two months, which is what – and$100 million of computing that went into GPT-4, right? So there you go. Academics are sidelined. And it's partly because the government just hasn't, the funding agencies haven't really been able to shift their funding patterns. I mean, every funding agency has its own clientele that they're trying to, faculty and so forth, that they're trying to help with funding. And when something new and this big happens, they're not as nimble as these other companies that can hire some of the best new PhDs coming out.

16:17In fact, I heard a figure which kind of shocked me. So do you know what the starting salary for a PhD right now in computer science who is joining OpenAI? A million dollars. Well, it's way above all the other companies, actually. And in fact, they can't compete. But be that as it may, that's what it takes to get some technology. And by the way, a lot of the people they're hiring are like Ilya Suskevier, who was the chief scientist there. It was into alignment. That was his primary goal was alignment. And that's probably what got him into trouble. But all of the companies are worried about that.

17:10So this is good because, you know, academics have been worried about it for a long time. There wasn't a lot we could do about it. But now that's all changing. Government has been very slow to move in terms of responding to the very, very rapid changes that have occurred in NAIA over the last couple of years. And they're now beginning to try to create guidelines in the U.S. And actually, the new law that was just passed by the EU is really funny. Someone said, the U.S. innovates and the EU regulates.

17:47They've had a history of that. But some of the laws, some of their laws are, I shouldn't say laws, but some of the ways that they've couched the rules. Like, for example, this is 105 pages of details like this, that thou shalt not use AI to screen job applications. I mean, you know, but it's wrongheaded in the sense that it shouldn't be micromanaging. It should be looking at the broader picture of excesses. Where are the excesses? And come up with general principles rather than try to say this particular application is bad, right? The applications should be kind of, in fact, they're going to be the rules for each domain, education, financial, entertainment, everything, the Hollywood strike.

19:04each industry is going to have a different set of criteria that is you know this is that for them the things that are important for them like writers and so forth is going to be different from a doctor who is concerned with other issues and and so there there has to each one of these domains has to be looked at separately and and a lot of the decisions have to be made locally They can't be made by the government. It's got to be made by, you know, the companies that are involved in making movies, the companies that are, you know, the medical community that is involved in helping patients and so forth.

19:43All of those have to learn how to adapt to the new technology, and that's going to take time. So it will unfold. And in terms of compute, the compute being controlled by these big, well-funded corporations, the U.S., there have been proposals to create a national resource that maybe wouldn't actually own compute but would provide credits for research institutions. Well, in fact, Fay Lee at Stanford has been advocating this now for several years. And there is a bill winding its way through Congress. I think it's in the Senate now, which would provide that for academics, which provide the sort of compute.

20:38What's happened, though, over the last couple of years is that the amount of computing that is needed has risen. So it's not clear how they're going to do that with credits. It's just not enough computing available in the world. But, for example, just to give you one amazing thing is that NVIDIA makes these boards, you know, the H100, A100, H100. These are GPUs that are essential for training these networks. And they just, you know, you can't get them. I mean, you know, everybody wants to get their hands on them. And, of course, now it's a trillion-dollar corporation, right? It actually, in just one year, went up by a factor of three, the cap, market cap.

21:32And, you know, it's hard to grow that fast in terms of, you know, the assembly lines or whatever it takes to actually build these boards. but the chips and so forth, all of that is a big pipeline that takes months and years in order to be able to ramp up. And so that's what's happening right now is that the world is unprepared for either the scaling up, ramping up that needs to be done, and on the industrial side, see, almost all of the computing that's going on is being done on digital processing chips. of various sorts, which are very, very energy inefficient. And ultimately, that is not going to be feasible.

22:21You can't just scale that up because, you know, you use up all the energy in the world, right? If you think that it's going to be used for so many different applications, right? It looks like it's heading in that direction. So what you need to do is develop a whole new technology for computing, which is specifically designed for low energy and for this particular architecture that they've designed and built. And we actually have that. It's called the brain. Nature was in that business long ago and has developed super low energy technology. I mean, your brain runs on 20 watts right i mean that's a very dim bulb but it can it can run rings around all of the computing that we now have because it's it's so efficient and and so there is a whole field now neuromorphic engineering which is was founded by carver mead about 30 years ago analog digital processing and and that uh is uh really very low power because it's running silicon near threshold where you're talking about microwatts rather than watts.

23:41And it shares some of the advantages, actually, of the computing. We know, for example, you don't need 64-bit precision when you are doing calculations that in the brain, you know, eight bits, five bits is sufficient. And also the other thing is that it's very fault tolerant in the sense that, you know, if you take out a few logical circuits in a computer, it'll crash, right? Because, you know, it's all deterministic. But if you take out a few units in a neural network, you know, it degrades it a little bit, but it's not going to crash. And so, again, you can take advantage of that when you're designing the hardware so that it doesn't have to be 100 % exact and deterministic.

24:32In fact, the principles of computing in neural networks is probabilistic. So from the very beginning, you take into account that it's not going to be perfect. Training is different from what's called inference. In other words, using it on a particular task to give you an answer. And what we've learned, actually, this is a very interesting theoretical result, is that you need a much larger network when you're training it than when you are using it for inference. And so this process of distillation is how you transfer the big network and download it into a smaller network. Yeah, and that's a very interesting theoretical, why should that be, right?

25:19In other words, it has something to do with the amount of exploration that you can do in a much larger network that allows you to come up with solutions that you couldn't if you started with a small network. But once you have the solution, it can be replicated in a smaller network and it can be distributed to the users and edge devices like your smartphone. So your smartphone will probably within the next five years be talking to you already. In fact, you can talk to your phone and it'll translate for you, but it will be doing a lot, lot more because it can be your assistant. And it will not just answering questions, but it will also be able to remember all of your needs and be able to help you navigate all of the complicated things that are going on in your life and coming to a meeting like this and helping you sort out what you need to do next.

26:21And this is something that will help everybody. Agency, that's a big hot, that's a hot topic right now. Right. In some ways, they already are grounded because of the fact that they know a lot about the world. In fact, there's a lot of questions they can answer that they couldn't have if they didn't know a little physics, right? Yeah. And so in that trillions of words of text that is out there, there's a lot of physics stuff. There's a lot of descriptions of how things work in the world and so forth. And so, and including how human beings work, social stuff. I mean, in fact, one of the strengths is ability to understand human intentions of all things and empathy.

27:05In fact, you know, they could do a lot better than doctors when it comes to patient care. How could that be? So here's a speculation. It all comes from training set in which the goal is to predict the next word in a sentence. Now, if you think about that for a while, right? And by the way, they also now have what's called a context length of tens of thousands of words, like 30 ,000 words. That means they could take in huge amounts of text and take that into account when they are responding to your question, right? That's like a book. In order to get to the next word, they have to take into account all those other words, right?

27:47They can somehow form associations, and this is self-attention, between all of the words and what the next one should be. And in order to be able to get better and better at it, it helps to have created an internal model of the meaning of all those words. And, in addition, an internal model of how the complexity of the world is expressed in those words. In other words, these are very sophisticated internal models. And, of course, that's what humans need to have, too, when they're navigating the complexities of the world and first going to school and dealing with social interactions and ultimately working in companies.

28:35All of that, all of that has to be internalized. You have to have an internal model of what's expected of you, of what you can say and what you can't say. I mean, all those things, you know, that we're taught, we call alignment, all that has to be internalized. And so that's where I think the future is going to be in helping, you know, create better internal models. And how do you do that? Well, a lot of it would be through physics about the properties of the world. And then a lot of it is going to be about reinforcement. Little kids have to be taught what's good and what's bad, right? Well, they have to be given examples and they have to be reprimanded.

29:10And that's something that has to go on during training. You can't wait till the end because you'll end up with an adult brain that is completely unable to understand in some sense or to respond in a way that we expect other humans to respond. That's a problem. That's a solvable problem. I don't think that that's going to take that long to solve. I think it's just a matter of adding in a few more parts of the brain into the model. And actually creating a much more sophisticated scheme for training that includes this developmental sequential schooling, if you will, of the interactions that it has to learn along the way, dealing with the world and with humans.

30:02You can't wait to the end to add that. The guardrails are too late if you have a mature system that is already trained. Yeah. So the pure scaling of existing architectures is given the compute constraints. Do you think that's going to continue in the next year or will attention focus to some of these other? No, it's all going to go on in parallel. It's all going on right now. And here at the meeting, there's every single one of these issues is being debated and progress being made in some cases more than others. But it's, you know, I think we have as a community been aware of all these issues for a long, long time.

30:50academics uh you know but it's no longer an academic issue right because a lot of people in the public are using this these these uh devices now which uh have been completely aligned and but they're but but what's interesting though is humans are incredibly adaptable you know humans for example adapted to keyboards yeah you know and and it was always the humans had to adapt to the machine. Well, now the machine has to adapt to the humans, right? It's the other way around. And, you know, that may take a while. I don't think it'll take as long as it took for humans to adapt to keyboards and for other mechanical devices that have to be, you have to, well, actually another good example that is music, right?

31:40Being able to learn how to play violin or a piano takes years and years and years incredible amount of practice practice practice and then you know to be able to compose requires yet still another complex process in which you now are creating not just reproducing uh or trend you know translating and that all of that you know practice is is part of the part of brain called the basal ganglia and right now deep learning on its own is like the cortex, but without a basal ganglia, you couldn't do all these things, right? You couldn't learn how to play piano. You couldn't learn how to do mathematics.

32:20You couldn't learn anything that requires a sequence of actions to achieve a goal. So now, where are we in terms of the next step? Well, we should be incorporating a basal ganglia into these models. And actually, that's already been done in AlphaGo. Yeah. So AlphaGo learned how to play Go by playing itself. And it did that using two different learning systems. One of them was deep learning, which was good at developing models of the board, the positions on the board. But then it had to learn with the goals of how do you win the game and how do you make moves. And that is where reinforcement learning came in.

33:02And in reinforcement learning, the only reinforcement you get is at the end whether you went or lost. Yeah. But somehow that was enough to help it figure out through a value function, that internal value function about the strength of game positions. But it then used that value function in order to be able to learn new moves, new positions that no human ever thought of, which are actually better ones than humans were previously aware of. So that's fantastic. Here we have an AI that used a very sophisticated visual representation together with a very, very ancient, you know, reinforcement learning is every species in the world has reinforcement learning.

33:45It was thought to be very primitive because of that. Well, it was primitive in the sense that it was something that was evolved at the very beginning, but it doesn't mean it's primitive and it's not powerful. Obviously, all these species have to survive, and that's not easy. but reinforcement learning got them there, right? So here we have reinforcement learning, which helped the powerful cognitive processor be able to solve these complex problems. It's already happening in science. There's been a tremendous, tremendous, just within the last 10 years, advances occurring throughout science. At the level of molecular biology, we can now predict the three-dimensional structure of proteins from their amino acid sequences.