AI Will End Humanity. No One Knows How To Stop It - Dr Roman Yampolskiy

25:30Let's do more of that. Let's identify specific issues and have tools where a human decides to deploy that tool to solve that problem, not create a general replacement for all of human labor and humanity as a whole. So why aren't we doing more of that? And why are we doing more general? Is it because there's more money in general? Is it a power thing? What's going on? I suspect it's both. So there is definitely a lot more money if you make free labor, cognitive and physical. You're talking$10 trillion. What is it annually? So that's a lot of money. You can invest in it and still have very good return no matter how expensive the current valuations are.

26:09So that justifies the current valuations. People don't fully understand. They only make$15 billion. Why are we investing trillions into them? Well, because they're saying in two years, you'll get free labor. And power is another thing. If they believe that someone's going to create it no matter what, maybe if I'm the guy who created God, I'll get something out of it. And do you think when you look at the big figures in this world, you know, like the Sam Altmans, And look, how much do you think they are motivated by money and status and power? And how much of it do you see as them wanting to be seen as, you know, the people who created something transformative?

26:50So in one of the blog posts, I think he talks about controlling the light cone of the universe. That's the level of power seeping there. Problem is, if I'm right and it kills everyone, you're not going to even be part of history as a bad guy. There's not going to be history books. So they have more to lose than an average person. And what do you think would be Sam Altman's steel man argument to what you were saying? What would Sam Altman, if we were engaging in a debate, what would he say? We'll figure it out. We have AI helping us do research now. Once we build it, we'll get there. We'll manage.

27:26But that doesn't sound like they have any clear ideas. That's official statements they are usually giving. We will have AI help us solve a problem, or maybe it will turn out to be easier than we think it is. Those are the actual arguments we heard so far. Because the concern is, when I hear about this, we had Jimmy Carr, the comedian, on a few months ago. And he made the point that the barrier to entry with AI when it comes to totalitarianism and mass surveillance is suddenly decreasing rapidly. If you think about East Germany, you had to have the Stasi on every corner. you have to pay informants.

28:05Well, all of a sudden, you don't have to have any of that. It definitely has potential to lock in dictatorships. But again, as long as it's a human dictator, we can look forward to them dying of natural or unnatural causes. If it's AI dictatorship, they're immortal. Once we lock in on a set of values, that's what you're going to have forever. That's assuming you're still around.

28:29Francis Foster:Yeah, I mean, all of these other concerns seem rather trivial in comparison to the thing that you're describing. Let's pause, though, and just set that to one side for the moment and talk about the replacement of humans in the labor market, the impact in the interim period. Let's accept that, you know, within 10 years, superintelligence kills us all. Let's not accept it. Agreed. Agreed. I meant for the sake of argument, of course. But in fact, for the sake of argument, let's say that you turn out, thank God, to be wrong about that. Doesn't happen. In the interim, though, we already see, people like to argue about this, but to me, it's just undeniable.

29:12Francis Foster:I know lots of business owners who say, Constance, no, no, no, we're not laying people off. We're just not hiring anyone. And we probably won't need to, unless literally the people we currently employ die. And even at that point, we may not replace 10 people with 10 people. We may replace 10 people with five people. What will be the impact of this in the next few years on the labor market, on jobs, on the way economy is structured, etc.? So it's all about this paradigm shift from narrow tools to more general tools to a complete general intelligence. We can define AGI as basically having a drop-in employee.

29:51I can take someone add them to the slack and then within weeks they're starting to help except they cost me nothing they work 24 7 no sexual harassment closets like it's just a pure win why would i have a higher now a human so all jobs which are done on a computer cognitive labor where you're a symbol manipulator that can be automated the moment we have that now physical labor may take little longer you need robots you need bodies you need to figure out how that works so another three years but we'll get there as well some jobs will be around because people prefer a human doing them oldest profession is a great example sometimes you want to hear i don't know about that i don't know you'll try a robot but you want a human i don't know do you know i've been thinking

30:42Francis Foster:about this a lot okay everybody laughs obviously yeah yeah but he's been thinking about sex robots a lot tell us why yeah it's because my life is going really well anyway i'm single and uh so anyway because if you think about it like this roman you know dating is hard relationships are hard a lot of relationships fail a lot of marriages fail why would you invest that time that money put your heart on the line, all of that suffering. Let's say we get to this point where you can order a robot and you can design her specifically to how you want every, and let's not get into the details, but every single part of her.

31:26You can also look at, you can design her personality, the spice level. You like it a little bit, you know, spicy or you like it however you like it.

31:35Francis Foster:You want her to shout at you twice a month, not once a month. Exactly. Why would you put up with a human being who is erratic, emotional, is sometimes unfair, when you can literally have perfection as you demand it? So there is a lot of weird human fetishes. Pretty much anything you can think of, there is a website for that somewhere on the internet. And I guarantee you, no matter how well sex robot market will be doing, there will be natural human females market. Fair. But it might be a lot smaller, I think, is what Francis is saying. It might shrink by 90%. But when we talk about predicting unemployment, I basically say that almost everything will be 100 % gone.

32:20But few things will remain. And this is one of the last resorts we have as humans. This is the career aspirations we'll have. I mean, that's a... Because one of the things that we talk about a lot on this show is the crisis of meaning in our society, where people struggle with what does it mean now to be a man, what does it mean to be alive, all of these things where once we had religion. But this will introduce a crisis of meaning the likes of which we've never experienced before. I agree with that. We call it Ikigai risks. So, Ikigai is a Japanese concept where you find happiness by doing something you like, something useful to society, and something you're good at.

33:06So you'll get paid for doing what you like, maybe you're a podcaster. But if that is gone, if there are no opportunities like that, then that takes a lot of meaning jobs. So, some jobs are just terrible. Nobody should be doing them, they're boring, stupid, we're happy to automate them. Our jobs give people satisfaction. They want to do more of them. But they also would be automatable. So this is exactly what we're facing. People make a counter argument. Well, if I don't have to go to work, I'll go fishing. There's 8 billion people fishing in that lake right now. You're not going to fish. And also we say, just to take your argument, there are some jobs that are stupid or boring or whatever else.

33:50but when I was teaching there were I remember I had a child and he was being he was very low ability he struggled at school and he found his lessons very very difficult and he would become frustrated and he would lash out and I knew why he was doing that but nevertheless I had to introduce some form of punishment to show him that his behavior wasn't acceptable and one day I kept him in during a lunch break and I said to him Marcus what you're going to do is you're going sharpen all these pencils so he went and sharpened all these pencils and when he came back to me at the end of lunch break i thought he'd be upset frustrated angry and he had a look of real pride on his face he went to me mr foster look at my pencils look at all the pencils and they were all done beautifully and i realized at that point the reason he was proud and it was my fault as well as everybody else's is for the one of the first times in his life he had been given a task that he could succeed at and he could do and he could have pride in.

34:47I really worry, Roman, that when you take that away from people, we are all going to end up like Marcus, lashing out angry and frustrated because we're not so different. We still have the child within us. Short-term good news, that is rentahuman.com where you can get a job doing things for bots. So maybe we'll hire you to sharpen pencils.

35:10Francis Foster:There you go, mate. Career sorted. Right. Let's try counter argument. What if super intelligence creates endless abundance, right? There are some problems with abundance, the sorts of things that France is talking about, but you know, park that to the side for one minute. And humans are totally satiated by the productivity of our eye, it produces everything we could possibly want. You know, we've got wonderful lives. no one has to work, blah, blah, blah, blah, blah. And therefore, humanity becomes sort of like, you know, a nice pet for the AI to maintain, to look after. You know, it's not quite ideal, but like you're a pet squirrel, the AI looks after you, it feeds you at the right time, puts water in your bowl, and it has no reason to not look after you because you're like its beloved pet.

36:07it could happen again we cannot predict what specifically would happen problem is you are not in control sometimes owners decide to put you to sleep or neuter you or do other things to pets you are not in charge so those decisions will no longer be with us we have eight billion people who are not consenting to this experiment they cannot consent because they don't know what's going to happen maybe you have your pet maybe you're abused pet we don't know.

36:39Francis Foster:I'm struggling for counter arguments here. I mean, this doesn't sound good. This is one of the better outcomes. The safety angle where you're a pet, you're protected, you're not in control. But this is one of the better outcomes. This is what people hope for as a good outcome. Yeah. This is what people hope for. Well, the other things are much worse. Existential risk, suffering risk, all that is way worse. But if you're a pet, you literally have no agency. Some people are very happy with that right now with the government. That's fair. No, I think Roman's point is that it's not that people think this is the good option.

37:16Francis Foster:They think it's the least worst option of the ones available. Normally, my job on the show is to interrogate the arguments that people put forward and try and find gaps. But I've been thinking about the same thing without having your knowledge of expertise in it. And it does seem, I mean, the very simple fact when you put survival instinct plus superior intelligence together, that seems to me inevitably to lead to the things you're talking about, or at least to the very serious risk of the things you're talking about. Okay.

37:50Francis Foster:And then I guess part of the reason it's not getting solved is the collective action problem, right? That's why it's not being... What is good for community is not what is good for individuals. As an individual, you want to have the most progress on your model, have the most advanced model. And then if government comes in and says we need to stop research, you forever are locked in as the dominant corporation in that space. I resisted creatine for years because I assumed it was for people who spent three hours a day in the gym and referred to themselves in the third person. Turns out Francis Fosser was wrong on this one.

38:28The research on creatine has moved way beyond the gym. For years, it was a preserve of bodybuilders and sprinters. But it turns out creatine is something your body makes naturally and uses as fuel. Not just for your muscles, but for your brain, your energy levels, your mood, your memory. The problem is that from your 40s onwards, your body produces less and less of it. And you feel it. The slower recovery, the afternoon fog, the sense that you're running on slightly less than you used to. Talk to anyone who actually knows this stuff and they'll tell you the same thing. Not all creatine is equal.

39:02And most of it isn't doing what you think it is. The formula matters. Specifically where it actually gets into your cells and activates once it's there. That's exactly what Qualia Creatine Plus is built around. Two clinically studied forms of creatine combined with electrolytes and sea salt designed to solve the whole problem, not just half of it. I've just started taking it. And once you understand how the formulation works, you wonder why nobody built it this way sooner. Go to qualiolife.com slash trig for 50 % off and use the code trig for an extra 15 % on top of that. That's Q-U-A-L-I-A-L-I-E.com slash trig, code trig.

39:52Thanks to Qualia for sponsoring the show.

39:55Francis Foster:Thinking about the idea of China and the US in particular working together to stop the creation of superintelligence, I guess the reason that that is less likely than we would want, I think, is the same as it would be with nuclear weapons. You have countries that say they don't have nuclear weapons and won't pursue them. But actually, because of the prisoner's dilemma situation where it's to the benefit of each of them to screw the other, to lie and then to develop the thing, you almost, some people would argue you can't take that risk. But then we're back where we started. So there is a fundamental difference.

40:40We talk about nuclear weapons as weapons of mutually assured destruction. But with AI, with superintelligence, it's literally that whoever creates it and controls superintelligence kills everyone. So it's not the same as with nuclear weapons, I have to decide to deploy them. It's a tool. I have an agent making this decision. The counterparty decides to retaliate. We all die. Here, just the fact that you created it is enough. There is no additional steps you have to take. Yeah.

41:13Francis Foster:And you've been raising concerns about this for a long time. What has been the response from the leaders in the field of AI? So leaders of the labs are all on record as recognizing AI safety as a big problem. Before we became CEOs, they wrote blog posts, talking about it, estimating probabilities of doom is very high and so they are kind of on board like you can see example with elon who was saying we are summoning demon funding ai safety research so doing all the right things until somewhat recently and why do you think he's changed his mind he realized that he's failing to stop it and that others may be less capable people will be creating super intelligence and And at this point, it might as well be his project which succeeds.

42:15Roman, when I read about AI, one of the areas that concerns me the most is when people who program or started AI, and look, push back if I get this wrong, I'm obviously not an expert, but it seems to me that when you program something, you install your own biases within it, even though you may not be aware of having biases. Is there potentially an issue where people who program a certain AI might make it more politically inclined one way or the other? Is that a real concern? So you may program an AI which eventually bends more to an authoritarian angle or maybe more hyper-conservative, and therefore it sees these people as being wrong and evil for a particular reason?

43:06Or is that a misfounded fear? So, A, we are not programming those systems. We are trained on data. The data has certain bias built in. It's human-generated data on the Internet. You know what bias Internet has. So that's what we're training on to begin with. Now, the after-the-fact filtering is where you instill your corporate values. And yeah, they can be more vogue or more conservative. You decided in China, model would not talk about Tiananmen Square, in the US it would not talk about you know what. So everywhere they have their own limits. Elon, I think, is trying to say, let's build kind of truthful AI and avoid those biases, but you still have the same training data.

43:48You don't have your own clean internet with clean data, so you still get a lot of human historical biases into that. You can't remove all bias. Bias is what learning is. When you learn something, you learn to bias data. You're not randomly making decisions. You have some information. A society will say, oh, this is not good information or it applies to groups, not individuals or whatever. you decide. But that's exactly what we train those systems to do. And the concern for me is, let's say you have a concern about the environment, and the AI alights on that. And it says, well, you know, the world is being damaged, climate change, pollution, all of these types of things.

44:42These are bad things. Let's look at who causes the majority of the pollution in the world. Human beings. Who cuts down the rainforest? Human beings. Therefore, if you apply logic to this problem, how do we solve this? Well, we get rid of human beings. Is that something that it could be arrived at very easily? It's a good example. I have a different one where we create AI to reduce suffering. Conscious life forms suffer. So how would you reduce suffering in the universe? reduce life. If there is no living beings, there is no suffering. There is a branch of philosophy, negative utilitarians who value suffering so much as a negative state, anything should be done to remove it at any cost.

45:28So not procreating, for example, is one solution, naturally dying out. But AI can certainly decide that it's more important to end suffering immediately. and also as well you you see it more and more that people come to ai as a de facto counselor or therapist presenting it with moral problems and this is becoming more and more accepted and it's seems bizarre to me that you would outsource very human problems to something that is not human it seems to me that that is profoundly worrying, isn't it? So we are kind of running experiment on ourselves. We don't know what it does long term. There's some evidence that maybe it will take people who are borderline insane or depressed and amplify those tendencies.

46:21But we don't know. We need to do science and we don't have time to do science properly because by the time you start working with this model, 20 new models have been released and this one is no longer cutting edge.

46:33Francis Foster:One of the things that always bothered me about it, and it was clear in terms of the bias that you talked about, because there was a moment when you might say, well, like most of social media was woke, right? And then now, you know, some social media is not woke, quite the opposite, right? And the one thing that I think all of us know who live in the real world is the internet is not real right but to ai that's all it has to go on right that's all the data that it's taken in it's taking in this digital data which is not necessarily reflective of human experience like if you were an alien coming down from space and someone said to you the conversation happening on twitter or on threads is how humans think we humans would laugh at that but ai doesn't know that, does it?

47:26Right, but it's not limited to internet data. To be fair, it has all the books, all the papers, all the movies, all the TV shows. There is some representation of real human interaction.

47:36Francis Foster:Yes, but we are sitting here in Los Angeles, for example, if you watch Hollywood and live outside of America, your impression of America is not remotely accurate because these films and series and movies are made by people who live in a very specific subculture in Hollywood. My point being that the human experience is a lot richer than what you can gather from books and TV shows and the internet. And AI, I think, is almost inevitably going to miss that, which would be another concern, wouldn't it? So this is where we can run experiments and go, okay, you have a psychiatrist who is a model and a psychiatrist who is a human.

48:19Who does better with clients who do clients like more apparently you don't have to have a physical body or be a human to be very good at that job well being liked and

48:29Francis Foster:being effective at different things right the whole field is not effective how do you mean psychiatry psychiatry probably not yeah yeah but there are types of therapy that are very effective

48:48Early studies show that those systems can do really well in many human domains. So comments from nurses, things like that, they are competitive.

48:58Francis Foster:Yeah, actually, I mean, I tested it out. I was having something I couldn't work out what to do. And it was very useful. It was like, oh, yeah, you should do this. And the thing about it I found interesting is it works best if you tell it not to you. if you say to it like cut the bullshit just tell me straight it will do it whereas before you're like you know it is true you get what you're prompted for yeah yeah oh so i one of the ways that ai is going to change the world is in the field of war and so talk to us a little bit how AI will impact warfare. I think we're already seeing it at the start.

49:41And what could be the future that we're heading towards? So right now it looks like it's more physical, mechanical. So you have drones blowing up things. I think long term it's more about cyber security, hacking, infrastructure. So US has everything basically controlled digitally, right? Power plants, internet banking. So if you had a super capable hacker, that would be very impactful if somebody wanted to attack us this way. So just I think yesterday, we learned that Antropic has a more advanced model, which is amazingly good at hacking. And they haven't released it yet. And they are scared of how well it will do.

50:20And so we slowly, slowly trying to release it to cyber defense community to figure out if they can do something with it. Because, correct me if I'm wrong, but if let's say you have a super hacker that is, whatever, 50 or 100 times more intelligent than even the best hacker, human hacker, it could render internet banking entirely obsolete. I mean, what's the point of having internet banking if it's not secure and it can get hacked at random? It could bring about the end of multiple businesses the way we see the internet surely right so you can obviously just hack things directly find zero-day exploits what is also very concerning is social engineering attacks if you can generate believable deep fakes video audio video from your boss from your family telling you i need a password for this or click that everyone clicks even cyber security experts click and things like that so you don't have to hack the actual account you just have to get access to the person and how far are we do you think are we away from that particular reality where you can get a call and it could be sound like my dad who's an older man from a particular part of the uk and it's exactly like his voice yeah so the technology exists and we've seen examples where the company got video from the CEO saying, transfer the funds, I need them to close the deal, and they transferred the funds.

51:49It already happened. Now, it's not as common and easy for every person to do it on scale, but technology exists. I can clone your voice. I can definitely animate a video of you. But it's not quite trustworthy just yet. So some people are not very good at telling deep fake videos from real videos. Long term, it will become impossible. The quality will be exactly 50-50 because that's how they are generated. You have a system generating fakes and a system saying authentic or not, and they meet in the middle. That's how we generate it using different models. So long term, there is no way to know. Short term, you can count fingers and sometimes it has an extra thumb or something, but most Most people don't pay attention to that.

52:39But so could we not design a narrow form of AI to actually combat that and that is highly trained and very specific at those skills and we'll be able to go, that's real, that's fake. So the moment you tell me how you know it's fake, I'll use that information to make it better quality fake. And then we're done with this process of back and forth. You can't tell anymore. So if you're telling me you just count fingers and there is too many fingers, my new model will make sure there is five fingers. So you lost that piece of evidence.

53:13Francis Foster:So it's a constant, what do you call it? It's like an evolution, the war between the predator and the prey. That's exactly what it is. It's an arms race. Yes, it's an arms race. An arms race. Okay. One more thing, because the concern, look, there's many concerns, Of course there is with that. But the real concern is we're distorting reality. Pretty soon we're not going to know what's real and what isn't. How can I know if my dad calls me up, he's like, I've had a fall. I'm going to need some money. We're going to need to go to a private hospital. I'm going to need however much to have a hip replacement.

53:53I'm in the States. He's at home. I'm going to write transfer bank. In my family, we have private passwords. So the kids would know if I'm talking to them or deepfake.

54:07It's terrifying because we're not going to know what's real and what isn't. And that has a dementing effect on ourselves. Because isn't that one of the signs that you're going insane, that you can no longer trust what you see or what you think or what you feel? It's all a simulation anyways. What do you mean? You haven't read my paper and we all live in a simulation?

54:28Francis Foster:I have... We had Scott Adams on the show before he passed, and he talked about this, but not to us, I think. Is this why you're so serene about this, that you don't think this is real? So when you take this technology to its logical conclusion, you will have software which is intelligent agents. You have virtual worlds they can reside in, simulations of this planet, like Google Earth kind of deal. Put those two together, you are now creating virtual worlds populated by intelligent beings. Let's say all the kids are playing video games. So there is 4 billion virtual environments, and only one real one.

55:08Statistically, you are more likely to be an agent in a virtual environment. And I like this a lot because it kind of puts some doubt into the mind of AI about, am I being tested am I still in the simulation or is it time to kill humans so I always try to promote that idea as well but humans are kind of badly

55:30Francis Foster:designed like if you were to design intelligence from scratch you wouldn't make it like need a shit every three hours you know I mean maybe you would why fertilizer the easy ways to make fertilizer not that easy but again you You cannot criticize design if you don't know what the goals are. You cannot criticize simulation if you're not externally understanding the purpose of a simulation. Look at our designs, right? I'm flying in an airplane, some of them still have ashtrays. Why do they have ashtrays? Well, airplanes evolved from previous version of it. No, no, it's not a poor design. Some decision was made for some reason.

56:14You just don't know all the history, all the reason behind it. Starting a business can seem like a daunting task, unless you have a partner like Shopify. They have the tools you need to start and grow your business. From designing a website, to marketing, to selling, and beyond, Shopify can help with everything you need. There's a reason millions of companies like Mattel, Heinz, and Allbirds continue to trust and use them. With Shopify on your side, turn your big business idea into... Sign up for your$1 per month trial at shopify.com slash special offer.

56:50This has not been the most enjoyable episode we've ever done, but very important.

56:56Francis Foster:I'm not trying. I'm really glad for you. Now I know what it feels like to be a woman. All right.

57:08Francis Foster:I'll buy you dinner afterwards. Yeah, to make it up to us. I prefer before, but afterwards it will be fine as well. During. This show exists because people want to hear ideas laid out properly. Not shouted, not spun, not dressed in outrage. Thought through and delivered with clarity. That is a skill. And like any skill, it can be learned and improved. Which is why I recommend Hillsdale College's new free online course, Classical Logic and Rhetoric. In this course, a Hillsdale College professor teaches you the tools to construct a sound argument. You'll learn how to think more clearly, how to structure your reasoning so it holds up under pressure and how to communicate your ideas in a way that people can understand and respond to.

57:52The course is part of a much wider library. Hillsdale offers more than 40 other free online courses covering everything from the book of Genesis and how the Allies won the Second World War to the rise and fall of the Roman Republic and the American Constitution, all of it free. What draws me to this subject is the gap between how most people think they can argue and how sound argumentation actually works. Rhetoric is not manipulation. Logic is not pedantry. Together, they are the tools that allow you to think and speak at your best. This course makes that accessible to anyone. To enroll, go to hillsdale.edu slash trigger.

58:31There's no cost and it's easy to get started. That's hillsdale.edu slash trigger.

58:38Francis Foster:So, I guess the obvious question is, what do you advocate that we now do? So, it's easy. We don't do. Not doing is very easy. Don't build general superintelligence. Don't train models and all the data, multimodal data, to solve every problem. Concentrate on specific problems, train only on relevant data. So, you're talking about breast cancer detection. Okay, great. training that data. You'll have a super intelligent tool for doctors to do early detection. You'll save lives. It's wonderful. I think you can get most benefits from the economy with narrow tools. And how do you achieve that not being done politically and geopolitically?

59:26Francis Foster:What would it take for the US government or for the leaders of these companies to adopt that view? Personal self-interest. If you tell the President of the United States, the moment this technology comes around, you lose all power. That's a compelling argument. I don't think you would like that. If this is the consensus of scientists in that field, then maybe we should not be building it. And is it the consensus? So, if you look at the top three, I believe, computer scientists by number of citations in the field, they are in agreement. This is really dangerous. It's not something we should be doing.

1:00:04So we're talking about Hinton, who has Nobel Prize Turing Award, Banjo Turing Award. I think we got maybe 100 ,000 people sending a letter saying, don't build superintelligence. Many top scientists. Are there outliers? Yes. Do they usually have a company where they get billions of dollars to build AI? Also, yes.

1:00:27Francis Foster:Well, I was going to ask you about that because ultimately, you know the thing that maybe I'm like this conversation is so wild to me that maybe my brain is like opened up to levels of imagination that are not real but I'm just saying out loud what I'm thinking in the moment which is two or three years from now the AI companies will be so powerful and I don't mean powerful in the sense of money I mean powerful in the sense of powerful like the ability to kinetically get what they want. I'm not sure two or three years from now, or five years from now, whenever, the President of the United States will be able to tell them, stop doing this, and unless they actually agree to get them to stop.

1:01:14So maybe nationalizing that technology will actually be something we see happen.

1:01:19Francis Foster:Yeah, but what I'm saying is, like, there comes a point where you actually physically will not be able to nationalize them because they will be more powerful than the US government. So again, it's all about this paradigm shift. Before they hit superhuman levels, it's tools. You can come in, shut it down, change software, all that is possible. The moment you're now dealing with superintelligence, it becomes a lot harder. Yeah. Go ahead. There's going to be a lot of people, regular people, doing regular jobs with regular lives. And they're going to listen to this. Not for very long, based on this conversation.

1:01:54Francis Foster:Well, look. It's an unkind joke, but it's sort of like, I mean, that's what follows. Yeah. Anyway. And they're going to think to themselves, look, if this is true and there's no reason to believe it isn't, this is all coming down the metaphorical pipeline. What can I do to insulate myself as much as possible from this technology and my families? Not much. So you can vote for people who are more aware. Some politicians are now starting to kind of wake up a little and suggest we don't build maybe as much compute for those companies or provide some sort of regulation. But it's sort of like the whole concept of aging and dying.

1:02:41It's always been the case. We all were going to die. Your kids, your friends, your family. What What did you as an average person do about it? Well, nothing. The government didn't allocate funds towards that problem. It seems important. In the same world, you'd have like 90 % of our budget going to fight aging. We're all dying. So it's exactly the same scenario. We just have a different reason we're going to die and maybe a different timeline. It depends on your age. If you're like 95, it's the same. Yeah. Yeah. Absolutely. Absolutely. I mean, here's a question. Do you think it could solve the issue of aging and mortality?

1:03:21Could it?

1:03:22Francis Foster:Yeah, in a negative way. I think it's actually the marrow problem we should worry about. I think somewhere in your DNA, there is a number of factors which allow you to rejuvenate yourself a certain number of times. And if we can reset that number, you'd live a lot longer, much healthier life. Most diseases are a byproduct of aging. And I think we can do it with a narrow superintelligence, not with a general one. Because it seems to me that we are at a fork in the road now, Ron, where we can go down one way or we can go down another way. And the worry is, is that we're heading down one way where it's going to lead to our destruction.

1:04:08that and i i just find it baffling in a way that the people in charge of this technology don't understand that or i'm willing to see that they don't feel that they can say no they cannot say no to investors because they'll be replaced and someone else will say yes the options are amazing the stock options they get. So they don't have an option to not do it. The hope is, again, that there is external pressure for all the companies to stop at the same time, and then they find they have an excuse to investors. Investors bought in at very high valuation. They needed to go another 100x. So they needed to continue growing hyper-exponential towards superintelligence.

1:04:55They cannot just say, let's have normal profits. So the financial pressures are what drives them? Incentives are completely misaligned. We have no incentives which are pro-humanity. All the incentives are to develop this. Do you think part of the problem is as well, Roman, that the politicians don't understand the technology or the long-term effects of this technology? So many don't, especially in the US. Many are so old they don't use computers or internet or anything. Maybe they quit. I don't know. But we have some politicians who are on record as saying this is very bad, dangerous, we need to do something, regulation.

1:05:37Problem is, you can't regulate this away. You can't just say it's illegal to kill humanity. It doesn't work. You need to have specific bans on this particular deployment. And I don't think they're willing to

1:05:50Francis Foster:do that and you need to orchestrate some kind of agreement with china as well i think that would be actually easier i think that would not be the most difficult part because again china doesn't have control mechanism you think communist party wants to lose control they're very good at staying in control and if they see this as potentially threatening their long-term survival they'll be very happy not to do that that's an interesting point you mentioned you have kids i do as well Well, what do you, I mean, is there any point training your kids to be able to do a job at this point? Well, again, it really depends on what type of job.

1:06:27I wouldn't train them to do something boring just to make money. That's going to be automated anyways. If there is something they find personally fulfilling to do. So there is lots of things. We talked about one, only human occupation, but you can be a - I don't imagine - You can do all sorts of training. you're a sensei, you're a guide, you're a tutor. Just human interaction. You take people on hikes, you meditate, you do sort of things where I don't want a robot doing it for me. Yeah. So it seems that we're going to prize human interaction above all else, really. Well, I don't know if that's true right now.

1:07:05We're not valuing that much. We sit at home and scroll. So maybe we don't need it as much as... In terms of jobs, I'm saying that certain jobs we will prefer to be done by humans. Like podcasting. Which ones is not obvious. Podcasting, I think if you are famous and you have people who really like you, you. But I think I would be better at asking questions, better at generating video content. So if you kind of grandfathered yourself in like you are Joe Rogan or something, you'll be okay. I wish you could have just said trigonometry. I mean, who's that? But I think for a new person to start something like that successfully in a world with super intelligent editing and questions, because they watched every interview I ever did.

1:07:52They know every question, they write every paper. How many of my papers have you read?

1:07:57Francis Foster:Not many. None, right. It's a good point. I was thinking about this in terms of the kind of the political element of it. And I can really see, Roman 10 20 years down the line we get a kind of neo-luddite movement which is anti-technology anti-ai and pushes back against that and it wouldn't surprise me if we get also a terrorist element of this you know we will start for instance like I don't think it will be long until you see people when Waymo start taking people's jobs I don't think it'll be very long until you walk past, and by the way, I don't agree with this, I want to make this clear, you'll see a Waymo with a smashed windscreen.

1:08:42We just had the biggest ever protest to stop AI, I think in San Francisco, like 100 to 200 people showed up, which is not a lot, but it's a good starting point. If you're interested in this social unrest, civil war, Hugo de Garis has a beautiful book, Artilect War. He wrote it like 20 years ago, completely predicting all these elements. This is the most important issue of our time. There will be people who want to create godlike machines and go to cosmos and people who are Terrans. They want everything to be local and not to build those machines. And that's the decisive issue of our time. Because we talk about mass surveillance states, the government would say, well, look, you know, more and more particularly young men are unemployed.

1:09:31they can't get a job because the jobs that they used to be able to get like driving jobs, manufacturing, they've all gone. They've all gone. So we've got this large group of unemployed young men. And if they don't have a job, what tends to happen is they get angry, they get more violent. And the government will then come in and go, well, look, we've had all of these civil unrests and uprisings and riots. We can't have this. Therefore, it's very important that we bring in mass surveillance to keep you safe. I mean, that's a real possibility, isn't it? It is possible without concern about super intelligence.

1:10:07We just have governments deploying latest technology to spy on us. We've seen it with Snowden, we've seen it with others revealing what's really happening, right? Yeah. And it's also the concern as well that we're going to live in a world which is far more unstable because we have these large groups and men who don't have access to a job? So the economic part of not having a job is easy to solve. You can tax big AI, you can tax robots and distribute that. That's not the difficult part. Meaning is difficult. Control is difficult.

1:10:41Francis Foster:Yes. Roman, well, thank you, I guess, for coming on the show. We're very grateful for your time. I'm just, I think, unfortunately, you've confirmed a lot of the things that, and I was wondering about this, you know, you are, I think, from the former Soviet Union, I am Francis, you know, he has some family ancestry from countries that have had difficult existences. I always worried that it was my kind of temperamental Russian background that makes me worry about this stuff. but as always when I don't see a logical counter argument that's when I go well I until I hear one I will think this is likely and I just I don't see the counter argument to the very basic point you're making which is if you're a squirrel you cannot keep humans under control and anything that has a survival instinct that you don't control that's more intelligent than you will eventually take over.

1:11:43Francis Foster:Best case scenario. Best case scenario. So the reason we are kind of uncomfortable is that like this is this is kind of become real for us in this conversation. So thank you for coming on. I hope more people hear your message and humanity begins to take this seriously. I hope so. So usually in science, when you publish a paper or book and you are wrong, There is no shortage of people jumping in and publishing rebuttals, corrections, solutions. We have many papers, many books, all arguing the same thing. There is no rebuttals. There is no patents. There is no peer-reviewed papers in nature saying, this is how we control advanced AI.

1:12:25It scales to any level. Don't worry about it. So it's not just that we had this conversation and so far nobody jumped in. They had a decade.

1:12:35Francis Foster:Thank you so much for coming on. And before our audience asks you their questions, the last question we ask all of our guests is, what's the one thing we're not talking about that we should be? Before Roman answers the final question at the end of the interview, make sure to head over to our substack. The link is in the description where you'll be able to see this. Is there an argument that humans are now in service of a new form of organism without realizing it? Do you think there is a risk that AI leads the human race becoming complacent, complacent, not bothering to study, research, and advance ourselves.

1:13:07Francis Foster:What's the one thing we're not talking about that we should be? Suffering risks. Suffering risks. Tell us more. So things could be so bad you wish you were dead. Why? Digital hell. You can create environment where you are tortured, but you are immortal. Or maybe you are uploaded to a virtual environment. What for? you're asking too many questions super intelligence can decide to do all sorts of things maybe it's dealing with some malevolent payload maybe it's running experiments you can ask why this world this simulation has suffering in it right that's what every religion deals with why did all good god create a world with pain and suffering but there are some answers to those questions and It's not ruled out by what we coded into those systems.

1:14:07Nice to end on a positive. All right. At least we didn't talk about it. Say again? At least we didn't talk about it. That's the question.

1:14:15Francis Foster:Head on over to triggerpod.co.uk where Roman is going to answer your questions. In the best case scenario where AI doesn't erase us, is it plausible that humanity avoids becoming merged with technology directly?

1:14:45The right window treatments change everything. Your sleep, your privacy, the way every room looks and feels. At Blinds.com, we've spent 30 years making it surprisingly simple to get exactly what your home needs. We've covered over 25 million windows and have 50 ,000 five-star reviews to prove we deliver. Whether you DIY it or want a pro to handle everything from measure to install, we have you covered. Real design professionals. Free samples. Zero pressure. Right now, get up to 40 % off site-wide. Plus, get a free professional measure at Blinds.com. Rules and restrictions apply.