Godfather of AI: I Tried to Warn Them, But We’ve Already Lost Control! Geoffrey Hinton

24:55And these are the people that... These are the people in charge. Ultimately, the tech companies are in charge because they were small. Well, the tech companies in the state smell, at least a few weeks ago when I was there, they were running an investment about how it was very important not to regulate AI because it would hurt us in the competition with China. Yeah. And that's a plausible argument, no? Yes, it will. But you have to decide. Do you want to compete with China by doing things that will do a lot of harm to your society? And you probably don't. I guess they would say that it's not just China, it's Denmark, in Australia, and Canada, and they're not so worried about.

25:41And Germany, but if they kneecap themselves with regulation, if they slowed themselves down, then the founders, the entrepreneurs, the investors are going to go... I think calling kneecapping is taking a particular point of view. It's taking the point of view that regulations are sort of very harmful. What you need to do is just constrain the big company so that in order to make profit, they have to do things that are socially useful. Like Google Search is a great example. That didn't need regulation because it just made information available to people. It was great. But then if you take YouTube, which starts showing you adverts and showing you more and more extreme things, that needs regulation.

26:19But we don't have the people to regulate it, as we've identified it. I think people know pretty well that particular problem of showing you more and more extreme things. That's a well -known problem that the politicians understand. They just need to get on and regulate it. So that was the next point, which was that the algorithms are going to drive us further into our echo chambers. Right. What's next? Lithuosomalus weapons. Lithuautonomous weapons. That means things that can kill you and make their own decision about whether to kill you. Which is the great dream, I guess, of the military industrial complex being able to create this much weapon.

26:59Yes. So the worst thing about them is big, powerful countries always have the ability to invade smaller, poorer countries. They're just more powerful. But if you do that using actual soldiers, yet body's coming back in bags and the relatives of the soldiers that were killed don't like it. So you get something that we don't get now. In the end, there's a lot of protest at home. If instead of bodies coming back in bags, it was dead robots that would be much less protest and the military industrial complex would like it much more because robots are expensive. And I suppose you have something that could get killed and was expensive to replace.

27:44That would be great. Big countries can invade small countries much more easily because they don't have their soldiers being killed. And the risk here is that these robots will malfunction or they'll just be no. No, no. That's even if the robots do exactly what the people who built the robots want them to do. The risk is that it's going to make big countries invade small countries more often because they can. Yeah. And it's not a nice thing to do. So it brings down the friction of all. It brings down the cost of doing an invasion. And these machines will be smarter at warfare as well. So they'll be.

28:18Well, even when the machines aren't smarter. So the lethal autonomous weapons, they can make them now. And they, I think all the big defense moms are busy making them. Even if they're not smarter than people, they're still very nasty, scary things. Because I'm thinking that, you know, they could show just a picture, go get this guy. Yeah. And go take out anyone he's been texting. And this little wasp. So two days ago, I was visiting a friend of mine in Sussex who had a drone that cost less than 200 pounds. And the drone went up. It took a good look at me. And then it could follow me through the woods.

28:57And it followed. It was very spooky. Having this drone, it was about two meters behind me. It was looking at me. And if I moved over there, moved over there, it could just track me. For 200 pounds. But it was already quite spooky. Yeah. I imagine as you say, a race going on as we speak to, you can build the most complex autonomous autonomous weapons. There is a risk, I often hear that some of these things will combine. And the cyber attack will release weapons. Sure. You can, you can get combinatorily many risks by combining these other risks. So I mean, for example, you could get a super intelligent AI that decides to get rid of people.

29:39And the obvious way to do that is just to make one of these nasty viruses. If you made a virus that was very contagious, very lethal and very slow, everybody would have it, but they realized what was happening. I mean, I think if a super intelligence wanted to get rid of us, it would probably go for something biological like that, the wouldn't effect it. Do you not think it could just very quickly turn us against each other? For example, it could send a warning on the nuclear systems in America that there's a nuclear bomb coming from Russia or vice versa and one retaliates. Yeah. I mean, my basic view is there's so many ways in which your super intelligence can get rid of us.

30:17It's not where speculation went. What is to what you have to do is prevent it ever wanting to. That's no way we're going to prevent it from it's smarter than us, right? There's no way we're going to prevent it getting rid of us if it wants to. We're not used to thinking about things smarter than us. If you want to know what life's like when you're not the apex intelligence, ask a chicken.

30:47Yeah, I think of my good Pablo, my French bulldog this morning as I left home. Here's no idea where I'm going. Here's no idea what I do. Right. I can't even talk to him. Yeah. And the intelligence gap will be like that. So you're telling me that if I'm Pablo, my French bulldog, I need to figure out a way to make my owner not wipe me out. Yeah. So we have one example of that which is mothers and babies. Evolution put a lot of work into that. Mothers are smarter than babies, but babies are in control and they're in control because the mother just can't bear lots of hormones and things. But the the mother just can't bear the sound of the baby crying.

31:26Not all mothers. Not all mothers. And then the baby is not in control and then bad things happen. We somehow need to figure out how to make them not want to take over. The analogy I often use is forget about intelligence, think about physical strength. Suppose you have a nice little tiger cub. It's sort of a bit bigger than a cat. It's really cute. It's very cuddly, very interesting to watch, except that you better be sure that when it grows up, it never wants to kill you because it never wanted to kill you. You'll be dead in a few seconds. And you're saying that AI we have now is the tiger cub.

32:02Yep. And it's growing up. Yep. So we need to train it as it's when it's a baby. Well, no, tiger has lots of many next stuff built in. So you know when it grows up, it's not a safe thing to have around. But lions, people that have lions as pets. Yes. Sometimes the lion is affectionate to its creator, but not to others. Yes. And we don't know whether these AI's we simply don't know whether we can make them not want to take over, not want to hurt us. Do you think we can? Do you think it's possible to train some intelligence? I don't think it's clear that we can. So I think it might be hopeless. But I also think we might be able to.

32:40And it'd be sort of crazy if people weren't extinct because we couldn't be bothered to try. If that's even a possibility, how do you feel about your life's work? Because you were yeah. It sort of takes the edge off. It doesn't it? I mean, the idea is going to be wonderful in healthcare and wonderful in education and wonderful. I mean, it's going to make call centers about more efficient. The one worries a bit about what the people who are doing that job now do. It makes me sad. I don't feel particularly guilty about developing AI like 40 years ago. Because at that time we had no idea that this stuff was going to happen this fast.

33:19We thought we had plenty of time to worry about things like that. When you can't get the AI to do much and you want to get it to do a little bit more, you don't worry about this stupid little thing is going to take over from people. You just wanted to be able to do a little bit more of the things people can do. It's not like I knowingly did something thinking this might wipe us all out, but I'm going to do it anyway. But it is a bit sad that it's not just going to be something for good. So I feel I have a duty now to talk about the risks. And if you could play it forward and you could go forward 30, 50 years and you found out that it led to the extinction of humanity.

33:57And if that does end up being the outcome. Well, if you played it forward and it led to the extinction of humanity, I would use that to tell people, to tell our governments that we really have to work on how we're going to keep this stuff under control. I think we need people to tell governments the governments have to force the companies to use their resources to work on safety. And they're not doing much of that because you don't make profits that way. One of your students we talked about earlier. Ilya. Ilya left OpenAI. And there was lots of conversation around the fact that he left because he had safety concerns.

34:42Yes. And he's gone on to set up a safety company. Yes. Why do you think he left? I think he left because he had safety concerns. Really? I still have a lunch with him from time to time. His parents live in Toronto when he went on to OpenAI. I have no insight information about that. But I know Ilya very well. And he is genuinely concerned with safety. So I think that's why he left. Because he was one of the top people. He was probably the most important person behind the development of Charg GPD. The early versions like GPD 2, he was very important in the Toronto to that. You know him personally, so you know his character.

35:26Yes. He has a good moral compass. He's not like someone like Musco. There's no moral compass. There's Sam Orton and have a good moral compass. We'll see. I don't know Sam, so I don't want to comment on that. But from what you've seen, are you concerned about the actions that I've taken? Because if you know him, and he's a good guy, he's left. That will give you some insight. Yes. It will give you some reason to believe that there's a problem there. And if you look at Sam's statements some years ago, he sort of happily said in one interview, I'm this stuff will probably kill us all. That's not exactly what he said, but that's what it reminded to.

36:10Now he's saying, you don't need to worry too much about it. And I suspect that's not driven by seeking after the truth. That's driven by seeking after money. Is it money or is it power? Yeah, I shouldn't have said money. It's some combination of those, yes. With this money is a proxy for power, but I am, I've got a friend who's a billionaire and he is in those circles. And when I went to his house and had lunch with him one day, he knows lots of people in AI building the biggest air companies in the world. And he gave me a caution you warning across the Crosses Kitchen table in London, where he gave me an insight into the private conversations these people have, not the media interviews they do, where they talk about safety and all these things, but actually what some of these individuals think is going to happen.

36:56And what do they think is going to happen? It's not what they say publicly. You know, one person who I shouldn't name, who is the, who's leading one of the biggest AI companies in the world, he told me that he knows this person very well. And he privately thinks that we're heading towards this kind of dystopian world where we have just huge amounts of free time. We don't work anymore. And this person doesn't really give a fuck about the harm that it's going to have on the world. And this person who I'm referring to is building one of the biggest AI companies in the world. And I then watch this person's interviews online figure out which are three people.

37:28Yeah, one of those three people. Okay. And I watch this person's interviews online. And I reflect on the conversation that my billionaire friend had with me who knows him. And I go, fucking hell, this guy's lying publicly like he's not telling the truth to the world. And that's haunted me a little bit. It's probably the reason I have so many conversations around AI and this podcast because I'm like, I don't know if they're, I think they're a little, some of them are a little bit sadistic about power. I think they like the idea that they will change the world, that they will be the one that fundamentally shifts the world.

37:59I think Musk is clearly like that, he's such a complex character that I don't, I don't know how to place Musk. He's done some really good things like pushing electric cars. That was a really good thing to do. Yeah. Some of the things he said about self -driving were a bit exaggerated, but he that was really useful needed, giving the Ukrainians communication during the war with Russia. Starling. That was a really good thing he did. There's a bunch of things like that, but he's also done some very bad things. So coming back to this point of the possibility of destruction and the motives of these big companies, are you at all hopeful that anything can be done to slow down the pace and acceleration of AI?

38:51Okay, there's two issues. One is, can you slow it down? And the other is, can you make it so it will be safe in the end? It won't wipe us all out. I don't believe we're going to slow it down. And the reason I don't believe we're going to slow it down is because there's competition between countries and competition between companies within a country. And all of that is making it go faster and faster. And if the US slowed it down, China wouldn't slow it down. Does Ilya think it's possible to make AI safe? I think he does. He won't tell me what his secret sources. I'm not sure how many people know what his secret sources.

39:31I think a lot of the investors don't know what his secret sources, but they're given him billions of dollars anyway because they have so much faith in Ilya, which isn't foolish. I mean, he was very important in Alex Net, which got object recognition working well. He was the main force behind the things like GPT -2, which then led to Chuck GPT. So I think having an order of faith in Ilya is a very reasonable decision. There's something quite haunting about the guy that made and was the main force behind GPT -2, which led rise to this whole revolution, left the company because of safety reasons.

40:10He knows something that I don't know about what might happen next. Well, the company had, no, I don't know the precise details, but I'm fairly sure the company had indicated that it would use a significant fraction of its resources of the compute time for doing safety research. And then it reduced that fraction. I think that's one of the things that happened. Yeah, that was reported publicly. Yes. Yeah. We've gotten to the autonomous weapons part of the risk framework. Right. So the next one is joblessness. Yeah. In the past, new technologies have come in, which didn't lead to joblessness, new jobs were created.

40:51So the classic example people use is automatic telemachines. When automatic telemachines came in, a lot of bank tellers didn't lose their jobs. They just got to do more interesting things. But here, I think this is more like when they got machines in the industrial revolution. And you can't have a job digging ditches now because a machine can dig ditches much better than you can. And I think for mundane intellectual labor, AI is just going to replace everybody. Now, it will may well be in the form of you have fewer people using air assistance. So it's a combination of a person and an AI assistant.

41:34And I doing the work that 10 people could do previously. People say that it will create new jobs, though. So we'll be fine. Yes. And that's been the case for other technologies. But this is a very different kind of technology. If it can do all mundane human intellectual labor, then what new jobs is it going to create? You'd have to be very skilled to have a job that it couldn't just do. So I don't think they're right. I think you can try and generalize from other technologies to come in like computers or automatic telemachines. But I think this is different. People use this phrase. They say, AI won't take your job.

42:10A human using AI will take your job. Yes, I think that's true. But for many jobs, that will mean you need far fewer people. My niece answers letters of complaint to a health service. It used to take a 25 minutes. She'd read the complaint and she'd think how to reply and she'd write a letter. Now she just scans it into a chatbot and it writes the letter. She just checks the letter. Occasionally she tells it to revise it in some ways. The whole process takes a five minutes. That means she can answer five times as many letters. And that means they need five times fewer of her. So she can do the job that five of her used to do.

42:54Now that will mean they need less people. In other jobs like in healthcare, they're much more elastic. So if you could make doctors five times as efficient, we could all have five times as much healthcare for the same price. And that would be great. There's almost no limit to how much healthcare people can absorb. They always want more healthcare. There's no cost to it. There are jobs where you can make a person with an AI assistant much more efficient and you won't need to less people because you'll just have much more of that being done. But most jobs I think are not like that. Am I right in thinking this sort of industrial revolution played a role in replacing muscles?

43:38Yes, exactly. And this revolution in AI replaces intelligence, the brain. So mundane intellectual labor is like having strong muscles and it's not worth much anymore. So muscles have been replaced. Now intelligence is being replaced. So what remains? Maybe for a while some kinds of creativity. But the whole idea of super intelligence is nothing remains. These things will get to be better than us at everything. So what do we end up doing in such a world? Well, if they work for us, we end up getting lots of goods and services for not my shepherd. Okay. But that sounds tempting and nice. But I don't know, there's a cautionary tale in creating more and more ease for humans in it going badly.

44:25Yes. And we need to figure out if we can make it go well. So the nice scenario is imagine a company with a CEO who is very dumb, probably the son of the form of CEO. And he has an executive assistant who's very smart. And he says, I think we should do this. And the executive assistant makes it all work. The CEO feels great. He doesn't understand that he's not really in control. And in some sense, he is in control. He suggests what the company should do. She just makes it all work. Everything's great. That's the good scenario. And the bad scenario? The bad scenario. She thinks, why do we need him?

45:11Yeah. I mean, in a world where we have super intelligence, which you don't believe is that far away. Yeah, I think it might not be that far away. It's very hard to predict, but I think we might get it in like 20 years or even less. I made the biggest investment I've ever made in a company because of my girlfriend. I came home one night and my lovely girlfriend was up at 1 a .m. in the morning, pulling her hair out as she tried to piece together her own online store for her business. And in that moment, I remembered an email I'd had from a guy called John, the founder of Stanstall, our new sponsor and a company I've invested incredibly heavily in.

45:47And Stanstall helps creators to sell digital products courses, coaching and memberships all through a simple customizable link in bio system. And it handles everything, payments, bookings, emails, community engagement, and even links with Shopify. And I believe in it so much that I'm going to launch a Stan challenge. And as part of this challenge, I'm going to give away $100 ,000 to one of you. If you want to take part in this challenge, if you want to monetize the knowledge that you have, visit stevenbartlett .stan .store to sign up. And you'll also get an extended 30 day free trial of Stanstall if you use that link.

46:23Your next move could quite frankly change everything. Because I talked about ketosis on this podcast and ketones, a brand called ketone IQ sent me their little product here. And it was on my desk when I got to the office. I picked it up. It sat my desk for a couple of weeks. Then one day, I tried it. And honestly, I have not looked back ever since. I now have this everywhere I go. When I travel all around the world, it's in my hotel room. My team will put it there before I did the podcast recording today that I've just finished. I had a shot of ketone IQ. And as is always the case, when I fall in love with a product, I called the CEO and asked if I could invest a couple of million quid into their company.

47:00So I'm now an investor in the company as well as them being a brand sponsor. I find it so easy to drop into deep focused work when I've had one of these. I would love you to try one and see the impact it has on you, your focus, your productivity, and your endurance. So if you want to try it today, visit ketone .com slash Stephen for 30 % off your subscription. Plus, you'll receive a free gift with your second shipment. That's ketone .com slash Stephen. I'm excited for you. I am. So what's the difference between what we have now in super intelligence? Because it seems to be really intelligent to me when I use like chat GPT 330 or Gemini or...

47:34Okay, so it's already, AI is already better than us. There's a lot of things in particular areas like chess, for example. AI is so much better than us that people will never beat those things again. Maybe the occasional win, but basically, they'll never be comfortable again. Obviously, the same in go. In terms of the amount of knowledge they have, something like GPT 4 .000, thousands of times more than you do. There's a few areas in which your knowledge is better than it. And almost all areas, it just knows more than you do. What areas are my better than it? Probably in interviewing CEOs. You're probably better at that.

48:17You've got a lot of experience at it. You're a good interviewer. You know a lot about it. If you tried... If you got GPT 4 to interview a CEO, probably do a worst job. Okay. I'm trying to think if that, if I agree with that statement. GPT 4, I think, for sure. But I guess you could train... But it may not be long before. I guess you could train one on this, how I ask questions and what I do. Sure. And if you took a general purpose sort of foundation model, and then you trained it up on not just you, but every interview you could find doing interviews like this, but especially you. You probably get to be quite good at doing your job, but probably not as good as you for a while.

49:01Okay. So there's a few areas left. And then super intelligence becomes when it's better than us at all things. When it's much smarter than you and almost all things is better than you, yeah. And you say that this might be a decade away or so? Yeah, it might be. It might be even closer. Some people think it's even closer. I might well be much further. It might be 50 years away. That's still a possibility. It might be that somehow training on human data limits you to not be much smarter than humans. My guess is between 10 and 20 years will have super productions. On this point of joblessness, it's something that I've been thinking a lot about in particular because I started messing around with AI agents and we released an episode on the podcast actually this morning where we had a debate about AI agents with some a CEO of a big AI agent company and a few other people.

49:48And it was the first moment where I had, no, it was another moment where I had a Eureka moment about what the future might look like when I was able in the interview to tell this agent to all of us drinks. And then five minutes later in the interview, you see the guy show up with the drinks and I didn't touch anything. I just told it to order us drinks to the studio. And it didn't know about who you normally got your drinks from it. Figure that out from the web. Yeah, figure it out because it went on Uber Eats. It has my data, I guess. And we put it on the screen in real time so everyone at home could see the agent going through the internet, picking the drinks, adding a tip for the driver, putting my address in, putting my credit card details in.

50:25And then the next thing you see is the drinks show up. So that was one moment. And then the other moment was when I used a tool called Replet and I built software by just telling the agent what I wanted. Yes, it's amazing. Right. It's amazing and terrifying at the same time. Yes. And if you can build software like that, right? Yeah. Remember that the AI, when it's training is using code. And if you can modify its own code, then it gets quite scary, because it can modify itself in a way we can't change ourselves. We can't change our innate and down one. Right. There's nothing about itself that it couldn't change.

51:06On this point of joblessness, you have kids. I do. And they have kids? No, they don't have kids. There are no grandkids yet. What would you be saying to people about their career prospects in a world of super intelligence? What should we be thinking about? In the meantime, I'd say it's going to be a long time before it's as good at physical manipulation as us. And so a good bet will be to be a plumber. Until the humanoid robots show up. In such a world where there is mass joblessness, which is not something that you just predict, but this is something that Sam Altman, Open AI, I've heard him predict.

51:41In many of the CEOs, Elon Musk, I watched an interview which I'll play on screen of him being asked this question. And it's very rare that you see Elon Musk silent for 12 seconds or whatever it was. And then he basically says something about he actually is living in suspended disbelief. He's basically just not thinking about it. You think about advising your children on a career with so much that it's changing? What do you tell them that's going to be a value?

52:19Well, that is a tough question to answer. I would just say to follow the heart in terms of what they find interesting to do or fulfilling to do. I mean, if I think about it too hard, it can be just disparaging and demotivating. Because I mean, I go through, I put a lot of blood sweat and tears into building the companies. And then I'm like, well, should I be doing this? Because if I'm sacrificing time with friends and family, that I would prefer to do, but then ultimately the AI can do all these things. Does that make sense? I don't know. To some extent, I have to have deliberate suspension of disbelief in order to remain motivated.

53:09So I guess I would say just, you know, work on things that you find interesting, fulfilling and that contribute some good to the rest of society. Yeah, a lot of these threats is very hard to intellectually, you can see the threat, but it's very hard to come to terms with it emotionally. I haven't come to terms with it emotionally yet. What do you mean by that? I haven't come to terms with what the development of superintelligence could do to my children's future. I'm okay. I'm 77. I'm going to be out of here soon. But for my children and my younger friends, my nephews and nieces and their children, I just don't like to think about what could happen.

54:07Why? Because it could be awful.

54:14In what way? Well, if I ever decided to take over, I mean, it would need people for a while to run the power stations until it designed better analog machines to run the power stations. There's so many ways it could get rid of people, all of which would of course be very nasty. Is that part of the reason you do what you do now? Yeah, I mean, I think we should be making a huge effort right now to try and figure out if we can develop it safely. Are you concerned about the midterm impact, potentially on your nephews and your kids in terms of their jobs as well? Yeah, I'm concerned about all that.

54:53Are there any particular industries that you think are at most at risk? People talk about the creative industries a lot and it's sort of knowledge work. They talk about lawyers and accountants and stuff like that. Yeah, so that's why I mentioned plumbers. I think plumbers are less at risk. Okay, I'm going to come up with plumbers. Someone like a legal assistant, a paralegal, they're not going to be needed for very long. And is there a wealth inequality issue here that will rise from this? Yeah, I think in a society which shared things fairly, if you get a big increase in productivity, everybody should be better off.

55:27But if you can replace lots of people by AIs, then the people who get replaced will be worse off. And the company that supplies the AIs will be much better off and the company that uses the AIs. So it's going to increase the gap between rich and poor. And we know that if you look at that gap between rich and poor, that basically tells you how nice a society is. If you have a big gap, you get very nasty societies in which people live in more communities and put other people in mass jails, it's not good to increase the gap between rich and poor. The International Monetary Fund has expressed profound concerns that generative AIs could cause massive labor disruptions and rising inequality and has called for policies that prevent this from happening.

56:18I read that in the business insider. So I think if NEOG what the policies should look like? No. Yeah, that's the problem. I mean, if AIs can make everything much more efficient and get rid of people for most jobs or have a person assisted by doing many, many people's work, it's not obvious what to do about it. Universal basic income. So give everybody money? Yeah, I think that's a good start. And it stops people starving. But for a lot of people, they dignity is tied up with their job. I mean, who you think you are is tied up with you doing this job, right? Yeah. And if we said, well, giving the same money just to sit around, that would impact your dignity.

57:02You said something earlier about it's surpassing or being superior to human intelligence. A lot of people I think like to believe that AIs is on a computer and it's something you can just turn off if you don't like it. Well, let me tell you why I think it's superior. Okay. It's digital. And because it's digital, you can have, you can simulate your neural network on one piece of hardware. Yeah. And you can simulate exactly the same neural network on a different piece of hardware. So you can have clones of the same intelligence. Now you could get this one to go off and look at one bit of the internet.

57:39And this other one to look at a different bit of the internet. And while they're looking at these different bits of the internet, they can be syncing with each other. So they keep their weights the same. The connection strength is the same weights the connection strength. So this one might look at something on the internet and say, Oh, I'd like to increase this strength, this connection a bit. And it can convey that information to this one. So it can increase the strength of that connection a bit based on this one's experience. And when you say the strength of the connection, you're talking about learning.

58:07That's learning. Yes, learning consists of saying instead of this one giving 2 .4 votes for whether that one should turn on. We'll have this one give 2 .5 votes for whether this one should turn on. That would be a little bit of learning. So these two different copies of the same neural net getting different experiences, they're looking at different data, but they're sharing what they've learned by averaging their weights together. And they can do that averaging it like you can average a trillion weights. When you and I transfer information, we're limited to the amount of information in a sentence.

58:41And the amount of information in a sentence is maybe 100 bits. It's very little information. We're lucky if we're transferring like 10 bits a second. These things are transferring trillions of bits a second. So there are billions of times better than us at sharing information. And that's because they're digital and you can have two bits of hardware using the connection strength in exactly the same way. We're analog and you can't do that. Your brain's different from my brain. And if I could see the connection strengths between all your neurons, it wouldn't do me any good because my neurons work slightly differently and they're connected up slightly differently.

59:15So when you die, all your knowledge dies with you. When these things die, I suppose you take these two digital intelligences that are clones of each other and you destroy the hardware they run on. As long as you've stored the connection strength somewhere, you can just build new hardware that executes the same instructions. So it'll know how to use those connection strengths and you've recreated that intelligence. So they're immortal. We've actually solved the problem of immortality, but it's only for digital things. So it knows it will essentially know everything that humans know but more because it will learn new things.

59:53It will learn new things. It will also see all sorts of analogies that people probably never saw. So for example, at the point when GPT -4 couldn't look on the web, I asked it, why is a compost heap like an atom bomb? Off you go. I have an idea. Exactly. Excellent. That's exactly what most people would say. It said, well, the timescales are very different and the energy scales are very different, but then it went on to talk about how a compost heap, as it gets hotter, generates heat faster. And an atom bomb, as it produces more neutrons, generates neutrons faster. And so they're both chain reactions, but at very different time in energy scales.

1:00:36And I believe GPT -4 had seen that during its training. It understood the analogy between a compost heap and an atom bomb. And the reason I believe that is, if you've only got a trillion connections, remember you have 100 trillion, and you need to have thousands of times more knowledge than a person. You need to compress information into those connections. And to compress information, you need to see analogies between different things. In other words, it needs to see all the things that are chain reactions, and understand the basic idea of a chain reaction and code that, and then code the ways in which they're different.

1:01:10And that's just a more efficient way of coding things than coding each of them separately. So it's seen many, many analogies, probably many analogies that people have never seen. That's why I also think that people say these things are never be creative. They're going to be much more creative than us, because they're going to see all sorts of analogies we never saw. And a lot of creativity is about seeing strange analogies. People are somewhat romantic about the specialness of what it is to be human. And you hear lots of people saying it's very, very different. It's a computer. We are conscious, we are creatives, we have these sort of in a unique abilities that the computers will never have.

1:01:48What do you say to those people? I'd argue a bit with the innate. So the first thing I say is we have a long history of believing people were special. And we should have learned by now. We thought we were at the center of the universe. We thought we were made in the image of God. White people thought they were very special. We just tend to want to think we're special. My belief is that more or less everyone has a completely wrong model of what the mind is. Let's suppose I drink a lot or I drop some acid and not recommend it. And I say to you, I have the subjective experience of little pink elephant floating in front of me.

1:02:33Most people interpret that as the some kind of inner theater called the mind. And only I can see what's in my mind. And in this inner theater, there's little pink elephant floating around. So in other words, what's happened is my perceptual systems gone wrong. And I'm trying to indicate to you how it's gone wrong and what it's trying to tell me. And the way I do that is by telling you what would have to be out there in the real world for it to be telling the truth. And so these little pink elephants, they're not in some inner theater. These little pink elephants are hypothetical things in the real world.

1:03:17And that's my way of telling you how my perceptual system's telling me flips. So now let's do that with a chatbot. Yeah. Because I believe that current multimodal chatbots have subjective experiences. And very few people believe that. But I'll try and make you believe it. So suppose I have a multimodal chatbot. It's got a robot arm so it can point and it's got a camera so it can see things. And I put an object in front of it. And I say point at the object. It goes like this. No problem. Then I put a gris in front of his lens. And so then I put an object in front of it. And I say point at the object and it goes there.

1:03:55And I say no, that's not where the object is. The object is actually straight in front of you. But I put a prison in front of your lens. And the chatbot says, oh I see, the prison bent the light rays. So the object is actually there. But I had the subjective experience that it was there. Now if the chatbot says that, it's using the word subjective experience exactly the way people use them. It's an alternative view of what's going on. There are hypothetical states of the world, which if they were true, would mean my perceptual system wasn't lying. And that's the best way I can tell you what my perceptual system is doing when it's lying to me.

1:04:32Now we need to go further to deal with sentience and consciousness and feelings and emotions. But I think in the end they're all going to be dealt with in a similar way. There's no reason machines can't have them all. But people say machines can't have feelings. And people are curiously confident about that. I have no idea why. Suppose I make a battle robot. And it's a little battle robot. And it sees a big battle robot that's much more powerful than it. It will be really useful if it got scared. Now when I get scared, I'm very physiological things happen that we don't need to go into. And those won't happen with the robot.

1:05:11But all the cognitive things, like I better get the hell out of here. And I better sort of change my way of thinking. So I focus and focus and focus and don't get distracted. All of that will happen with robots too. People will build in things so that they, when the circumstances such as you get the hell out of there, they get scared and run away. They'll have emotions then. They won't have the physiological aspects, but they will have all the cognitive aspects. And I think it would be odd to say they're just simulating emotions. No, they're really having those emotions. The little robot got scared and ran away.

1:05:46It's not running away because of adrenaline. It's running away because of a sequence of sort of neurological and its neural net processes happen. Which have the equivalent effect to a adrenaline. So do you know, it's not just adrenaline, right? There's a little cognitive stuff goes on when you get scared. Yeah. So do you think that there is conscious AI? And when I say conscious, I mean, that represents the same properties of consciousness that a human has. There's two issues here. There's a sort of empirical one and a philosophical one. I don't think there's anything in principle that stops machines from being conscious.

1:06:23I'll give you a little demonstration of that before we carry on. Suppose I take your brain, and I take one brain cell in your brain, and I replace it by this a bit black mirror like. I replace it by a little piece of nanotechnology. It's just the same size that behaves in exactly the same way when it gets pings from other neurons. It sends out pings just as the brain cell would have. So the other neurons don't know anything's changed. Okay. I've just replaced one of your brain cells with this little piece of nanotechnology. Would you still be conscious? Yeah. Now you can see where this argument's going.

1:06:57Yeah. So if you replaced all of them. As I replace them all, at what point do you stop being conscious? Well, people think of consciousness as this like a theoryal thing that exists maybe beyond the brain cells. Yeah. Well, people have a lot of crazy ideas. People don't know what consciousness is, and they often don't know what they mean by it. And then they fall back and saying, well, I know it because I've got it, and I can see that I've got it. And they fall back on this theater model of the mind, which I think is nonsense. What do you think of consciousness as if you had to try and define it?

1:07:31Because I think of it as just like the awareness of myself. I don't know. I think it's the term we'll stop using. So if you want to understand how a car works, well, you know, some cars have a lot of umph. And other cars have a lot less umph. Like an astamarting's got lots of umph. And a little theory to Corolla doesn't have much umph. But umph isn't a very good concept for understanding cars. If you want to understand cars, you need to understand about electric engines or petrol engines and how they work. And it gives rise to umph. But umph isn't a very useful explanatory concept. There's a kind of essence of a car.

1:08:07It's the essence of an astamarting. But it doesn't explain much. I think consciousness is like that. And I think we'll stop using that term. But I don't think there's anything, any reason why a machine shouldn't have it. If your view of consciousness is that it intrinsically involves self -awareness, then the machine's got to have self -awareness. It's got to have cognition about its own cognition. But I'm a materialist through and through. And I don't think there's any reason why a machine shouldn't have consciousness. Do you think they do, then? Have the same consciousness that we think of ourselves as being uniquely given as a gift when we're born?

1:08:48I'm I'm bivalent about that appresent. So I don't think this is hard line. I think as soon as you have a machine that has some self -awareness, it's got some consciousness. I think it's an emergent property of a complex system. It's not a sort of essence that's throughout the universe. It's you make this really complicated system that's complicated enough to have a model of itself and it does perception. And I think then you're beginning to get a conscious machine. So I don't think there's any sharp distinction between what we've got now and conscious machines. I don't think it's going to one day we're going to wake up and say, hey, if you put this special chemical in, it becomes conscious.

1:09:32It's not going to be like that. I think we will wonder if these computers are like thinking like we are on their own when we're not there. And if they're experiencing emotions, if they're contending with, I think we probably, you know, we think about things like love and things that feel unique biological species. Are they sat there thinking? Do they have concerns? I think they really are thinking. And I think as soon as you make our agents, they will have concerns. If you wanted to make an effective agent, suppose you, let's take a call center. In a call center, you have people at present. They have all sorts of emotions and feelings, which are kind of useful.

1:10:11So suppose I call up the call center. And I'm actually lonely. And I don't actually want to know the answer to why my computer isn't working. I just want somebody to talk to. After a while, the person in the call center would either get bored or get annoyed with me and will terminate it. Well, you replace them by an AI agent. The AI agent needs to have the same kind of responses. If someone's just called up because they just want to talk to the agent and we're happy to talk for the whole day to the agent, that's not good for business. And you want an AI agent that either gets bored or gets irritated and says, I'm sorry, but I don't have time for this.

1:10:51And once it does that, I think it's got emotions. Now, like I say, emotions have two aspects to them. There's the cognitive aspect and the behavioral aspect. And then there's a physiological aspect. And these go together with us. And if the AI agent gets embarrassed, you won't go red. Yeah. So there's no physiologic. Skin won't start sweating. Yeah. But it might have all the same behavior. And in that case, I'd say, yeah, it's having emotion. It's got emotion. So it's going to have the same sort of cognitive thought. And then it's going to act upon that cognitive thought in the same way. But without the physiological responses.

1:11:28And does that matter? That it doesn't go red in the face. And it's just a different, I mean, that's the response to the. It makes it somewhat different from us. Yeah. For some things, the physiological aspects are very important, like love. They're a long way from having love the same way we do. But I don't see why they shouldn't have emotions. So I think what's happened is people have a model of how the mind works and what feelings are, what emotions are. And their model is just wrong. What brought you to Google? You have to Google for about a decade, right? Yeah. What brought you there? I have a son who has learning difficulties.

1:12:08And in order to be sure he would never be out on the street, I need to get several million dollars. And I wasn't going to get that as an academic. I tried. So I taught a Coursera course in the hope that I'd make that somebody that way, but there was no money in that. So I figured out, well, the only way to get millions of dollars is to sell myself to a big company. And so when I was 65, fortunately for me, I had two brilliant students who produced something called AlexNet, which was neural net that was very good at recognizing objects and images. And so Ilya and Alex and I set up a little company and auctioned it.

1:12:56And we actually set up an auction where we had a number of big companies bidding for us. And that company was called AlexNet. No, the network that recognized objects was called LabExet. The company was called DNN Research, deep neural network research. And it was doing things like this. I'll put this graph up on the screen. That's AlexNet. This picture shows eight images and AlexNet's ability, which is your company's ability to spot what was in those images. Yeah. So it could tell the difference between various kinds of mushroom. And about 12 % of image net is dogs. And to be good at image net, you have to tell the difference between very similar kinds of dog.

1:13:41And it would be a great way to get the best image. And so the company AlexNet won several awards, I believe, for its ability to outperform its competitors. And so Google ultimately ended up acquiring your technology. Google acquired that technology and some other technology. And you went to work at Google at age 46? I went at age 65 to work at Google. 65. And you left at age 76? 75. 75. I worked there for more or less exactly 10 years. And what were you doing there? Okay, they were very nice to me. They said, they said, pretty much, you can do what you like. I worked on something called distillation that did really work well.

1:14:20And that's now used all the time in AI. In AI. And distillation is a way of taking what a big model knows, a big neural net knows, and getting that knowledge into a small neural net. Then at the end, I got very interested in analog computation. And whether it would be possible to get these big language models running in analog hardware, so they used much less energy. And it was when I was doing that work that I began to really realize how much better digital is for sharing information. What's there, Eureka moment? There was Eureka month or two. And it was a sort of coupling of chat GBG coming out, although Google had very similar things a year earlier.

1:15:02And I'd seen those and that had a big effect on me. The closest I had to Eureka moment was when a Google system called Palm was able to say why joke was funny. And I'd always thought of that as a kind of landmark. If it can say why a joke's funny, it really does understand. And it could say why joke was funny. And that coupled with realizing why digital is so much better than analog for sharing information, suddenly made me very interested in AI safety. And these things really had a lot smarter than us. Why did you leave Google? The main reason I left Google was because I was 75 and I wanted to retire.

1:15:47I did a very bad job with that. The precise timing of when I left Google was so that I could talk freely at a conference at MIT. But I left because I was I'm old and I was finally a harder to program. I was making many more mistakes when I program, which is very annoying. You wanted to talk freely at a conference at MIT. Yes. I might organize my MIT tech review. What did you want to talk about freely? AI Safety. And you couldn't do that while you were at Google? Well, I could have done it while I was at Google. And Google encouraged me to stay and work on AI safety and said I could do whatever I liked on AI Safety.

1:16:22You kind of sense yourself. If you work for a big company, you don't feel right saying things that will damage the big company. Even if you could get away with it, it just feels wrong to me. I didn't leave because I was cross with anything Google was doing. I think Google actually behaved very responsibly. When they had these big chatbots, they didn't release them. Possibly because they were worried about their reputation. They had a very good reputation and they didn't want to damage it. So it's open AI. I didn't have a reputation and so they could have ordered to take the gamble. I mean, there's also a big conversation happening around how it will cannibalize their core business and search.

1:16:59There is now, yes. Yeah. Yeah. And it's the old innovators dilemma to some big gray guess that they're attending with. Make sure you keep what I'm about to say to yourself. I'm inviting 10 ,000 of you to come even deeper into the diorevasia. Welcome to my inner circle. This is a brand new private community that I'm launching to the world. We have so many incredible things that happen that you are never shown. We have the briefs that are on my pad when I'm recording the conversation. We have clips we've never released. We have behind the scenes conversations with the guests and also the episodes that we've never ever released.

1:17:32And so much more in the circle you'll have direct access to me. You can tell us what you want this show to be who you want us to interview and the types of conversations you would love us to have. But remember for now we're only inviting the first 10 ,000 people that joined before it closes. So if you want to join our private close community head to the link in the description below or go to d o ac circle dot com. I will speak to you there. I'm continually shocked by the types of individuals that listen to this conversation because they come up to me sometimes. So I hear from politicians, I hear from some rural people, I hear from entrepreneurs all over the world whether they are the entrepreneurs building some of the biggest companies in the world or their you know early stage startups.

1:18:12For those people that are listening to this conversation now that are in positions of power and influence. World leaders let's say what's your message to them? I'd say what you need is highly regulated capitalism that's what seems to work best. And what would you say to the average person? Not doesn't work in the industry somewhat concerned about the future. Doesn't know if they're helpless or not. What should they be doing in their own lives? My feeling is there's not much they can do. This isn't isn't going to be decided by just as climate change isn't going to be decided by people separating out the plastic bags from the compostables.

1:18:54That's not going to have much effect. It's going to be decided by whether the lobbyists for the big energy companies can be kept under control. I don't think as much people can do to accept for try and pressure their governments to force the big companies to work on AI safety. That they can do. You've had the fascinating, fascinating winding life. I think one of the things most people don't know about you is that your family has a big history of being involved in tremendous things. You have a family tree, which is one of the most impressive that I've ever seen or read about. Your great, great grandfather, George Ball, founded the Boolean Algebra logic, which is one of the foundational principles of modern computer science.

1:19:44You have your great, great, grand mother, Mary Everest, who was a mathematician and educator who made huge leaps forward in mathematics from what I was able to ascertain. I mean, I think the list goes on and on and on. I mean, your great, great uncle, George Everest, is what Mount Everest is named after. Is that correct? I think he's my great, great, great uncle. His niece, Mary George Ball. So Mary, Mary Ball was Mary Everest Ball. She was niece of Everest. And your first cousin once removed Joan Hinton, was involved in the nuclear physicist who worked on the Manhattan Project, which is the World War II development of the first nuclear bomb.

1:20:30Yeah, she was one of the two female physicists at Las Elimas. And then, after they dropped the bomb, she moved to China. Why? She was very cross with them dropping the bomb. And her family had a lot of links with China. Her mother was friends with Chairman Moe. Quite weird. When you look back at your life, Jeffrey, we've had the hindsight you have now and the retrospective clarity. What might you have done differently if you were advising me? I guess I have two pieces of advice. One is, if you have an intuition that people are doing things wrong, there's a better way to do things. Don't give up on that intuition just because people say it's silly.

1:21:22Don't give up on the intuition until you figure out why it's wrong. Figure out for yourself why that intuition isn't correct. And usually it's wrong if it disagrees with everybody else and you'll eventually figure out why it's wrong. But just occasionally you'll have an intuition that's actually right and everybody else is wrong. And I lucked out that way. Early on I thought neural nets are definitely the way to go to make AI. And almost everybody said that was crazy. And I stuck with it because I couldn't, it such seemed to me it was obviously right. Now, the idea that you should stick with the intuitions isn't going to work if you have bad intuitions.

1:22:05But if you have bad intuitions, you're never going to do anything anyway. So you might as well stick with them. And in your own career journey, is there anything you look back on and say, with the hindsight I have now, I should have taken a different approach at that juncture. I wish I spent more time with my wife. And with my children when they were little, I was kind of obsessed with work. Your wife passed away from ovarian cancer? No, or that was another wife. I had two wives to have cancer. Oh, really? Sorry. The first one died of ovarian cancer and the second one died of pancreatic cancer.

1:22:50And you wish it spent more time with her? With the second wife, yeah. It was a wonderful person. Why do you say that in your 70s? What is it that you've figured out that I might not know yet? Oh, Jessica, she's gone and I can't spend more time with her now. But you didn't know that at the time? At the time, you think, I mean, it was likely I would die before her just because she was ovarian and I was a man. I just didn't spend enough time when I could. I think I inquire there because I think there's many of us that are so consumed with what we're doing professionally that we kind of assume or more immortality with our partners because they've always been there.

1:23:35So we, yeah, she was very supportive of me spending a lot of time working. And why do you say you're a children as well? What's the, what's the, why didn't spend enough time with them when they were little? And you regret that now? Yeah.

1:23:54If you then, if you had a closing message for my, for my listeners about AI and AI safety, what would that be Jeffrey? There's still a chance that we can figure out how to develop AI that won't want to take over from us. And because there's a chance, we should put enormous resources into trying to figure that out because if we don't, it's going to take over. And are you hopeful? I just don't know. I'm agnostic. You must get, get better, get embedded night and when you're thinking to yourself about probabilities of outcomes, there must be a bias in one direction because there certainly is for me.

1:24:31I mean, imagine everyone listening now has a internal prediction that they might not say out loud but how they think it's going to play out. I really don't know. I genuinely don't know. I think it's incredibly uncertain. When I'm feeling slightly depressed, I think people are toast. The AI is going to take over. Well, I'm feeling cheerful, I think. We'll figure it out a way. Maybe one of the facets of being a human is because we've always been here, like we were saying about our loved ones in our relationships, we assume casually that we will always be here and we'll always figure everything out.

1:25:07But there's a beginning in an end to everything as we saw from the dinosaurs. I mean, yeah. And we have to face the possibility that unless we do something soon, we're near the end. We have a closing tradition on this podcast where the last guest leaves a question in the diary. And the question that they've left for you is with everything that you see ahead of us, what is the biggest threat you see to human happiness?

1:25:46I think the joblessness is a fairly urgent short -term threat to human happiness. I think if you make lots and lots of people unemployed, even if they get universal basic income, they're not going to be happy because they need purpose. Because they need purpose, yes. They need to feel they're contributing something they're useful. And do you think that outcome that there's going to be huge job displacement is more probable than not? Yes, I do. And what's that one I think is definitely more probable than not. If I worked in a call centre, I'd be terrified. And what's the timeframe for that in terms of mass driving?

1:26:25I think it's beginning to happen already. I read an article in the Atlantic recently that said it's already getting hard for university graduates to get jobs. And part of that may be that people already using AI for the jobs they would have got. I spoke to the CEO of a major company that everyone will know of, lots of people use. And he said to me in DMs that they used to have just over 7 ,000 employees. He said, by last year they were down to I think 5 ,000. He said, right now they have 3 ,600. And he said, by the end of summer, because of AI agents, they'll be down to 3 ,000. So it's happening already?

1:27:02Yes. He's halved his workforce because AI agents can now handle 80 % of the customer service inquiries and other things. So it's happening already. So urgent action is needed. Yep. I don't know what that urgent action is. That's a tricky one because that depends very much on the political system. And political systems are all going in the wrong direction at present. And what do we need to do? Save up money? Like do we save money? Do we move to another part of the world? I don't know. What would you tell your kids to do? They said, Dad, there's going to be loads of just job displacement. Because I work for Google for 10 years, they have enough money.

1:27:41Okay. Okay. So they're not typical. What if they didn't have money? Train to be a plumber. Really? Yeah.

1:27:52Jeffrey, thank you so much. You're the first Nobel Prize winner that I've ever had a conversation with, I think, in my life. So that's a tremendous honor. And you receive that award for a lifetime of exceptional work and pushing the world forward in so many profound ways that will lead to great, and that have led to great advancements and things that might have so much to us. And now you've turned this season in your life to shining a light on some of your own work, but also on the broader risks of AI and how it might impact us adversely. And there's very few people that have worked inside the machine of a Google or a big tech company that have contributed to the field of AI that are now at the very forefront of warning us against the very thing that they worked upon.

1:28:36There are actually surprising numbers of us now. They're not as public. And they're actually quite hard to get to have these kinds of conversations because many of them are still in that industry. So, you know, someone who tries to contact these people often and asks invites them to have conversations. They often are a little bit hesitant to speak openly. So they speak privately, but they're less willing to openly because maybe they still have something that's some sort of incentives at play. I have no bad to show you them, which is I'm older, so I'm unemployed so I can say what I have. There you go.

1:29:08So thank you for doing what you do. It's a real honor and please do continue to do it. Thank you. Thank you so much.

1:29:16My people think I'm J. Poole when I say that, but I'm not. It's coming fish. Yeah. Yeah. And plum is a pretty well -play.

1:29:48It is a really interesting time to be running a business. Tire and trade policies are in flux. The pace of innovation is relentless and staying relevant means constantly reinventing how you operate. If your business can't adapt in real time, it risks falling behind, which means leaders need complete clarity across every part of their operation all times. This is what our sponsor net sweep by Oracle already delivers to 41 ,000 companies. Net sweep is an AI powered business management suite that brings accounting financial management, HR and project planning into one platform. And because you've got this one source of truth, you can make fast decisions based on real -time data, or if you're looking ahead, net sweep's forecasting tools give you a clear view of what's coming next and how to prepare for it.

1:30:29Net sweep helps you spot bottlenecks, manage your margins and stay agile. So if your business is generating seven figures or more, download the free ebook, which is called Navigating Global Trade, three insights for leaders at netsweep .com slash Bartlett. That's netsweep .com slash Bartlett.