If the Public Knew This About AI, They'd Panic: Roman Yampolskiy | Ep 286 with Dr. Roman Yampolskiy

0:04So Roman, I gotta say, I've watched every interview that you've done, and I'm always fascinated by the things that you've been saying. I've been telling people what you've been saying and they think I'm crazy. And I'm like, look, I am not the expert. I'm just quoting Roman and he is the expert. But I'm really interested in the fact a lot of people don't even really understand about AI still. So what would you say is one truth about AI that keeps you up at night right now. And if the public knew about this, they would probably panic. People developing it don't really understand how it works. And we're pouring billions, even trillions of dollars into that arms race.

0:50And safety is sometimes not even mentioned. Yesterday, White House released a new national agenda for developing more advanced AI quicker. And I don't think safety is mentioned in that proposal. So now we have government level actors pouring all the computational resources they have, all the compute, all the best scientists to beating private corporations or collaborating with them to get to advanced AI. And we don't know how to control those things. So do you think the government didn't understand the whole reasoning why they did not include safety in the conversation? So with different administrations come different priorities.

1:36I think the current administration got a little confused in the difference between what AI safety means in terms of existential risk versus AI safety in terms of algorithmic bias, protecting the planet, diversity, inclusion questions. They kind of grouped it all together and said, you know, we're less concerned about that. So that's a little bit unfortunate. And as a result, they are racing towards the most advanced AI without any concern for all those terms under the umbrella. So would you say that we're past the ability to even have safeguards? Or if they said, Roman, we want to put you in charge, is there something that you would do right now?

2:19So it's never too late. If we're still alive, we can definitely come up with some good plans. But overall, it seems like racing towards AI is just not a good idea. We need lots of time to figure out what is really going on. We have very advanced models already released. I think at this point, they are on par with the smartest PhD students. So we have a lot of possible economic wealth to be deployed. It's not fully deployed for the economy. I think it can automate many, many jobs. It can give us a lot of free stuff. but instead of trying to monetize what we have, we're trying to get to that future super intelligent target without any concern for safety.

3:00So do you really think AI will wipe out humanity, Roman? So if you create something smarter than you with the ability to set its own goals, you can't really predict what it's going to do. That's kind of by definition impossible if you're not that smart. So will it definitely do that? We cannot predict. but out of all the possible states of the universe. Very few are friendly to humans in terms of our preferences for temperature, for gravity, for all sorts of basic needs. And if you're not in charge, someone else will decide those properties for you. If humans lose control of AI, what do you think the first 24 hours look like?

3:43If developers lose control of AI, what happens? very likely we won't see any difference whatsoever. So if superintelligence was running right now, what do we expect to change? I don't know. It could be exactly the same for many years. There is some game theoretic reasons to say that AI quickly realizes it's immortal. It's not in a rush. It can take 10, 20, 100 years to accumulate resources, to be more in control, maybe to have us surrender control willingly because it's so helpful. So instead of having this adversarial conflict with somewhat intelligent species, they can just wait it out and get all the control by being patient and helpful and pretending to be a friend until it strikes against you.

4:28Do you think part of the problem is that AI is learning from the internet, learning from humans, and we know humans are very flawed? I would almost want to say that I would want AI humanoids to not be like humans because, you know, we're greedy. We're, you know, we're evil. Obviously, there's good things, too, but there's a lot of bad things. Do you think part of the problem is that it's learning alongside of information, it's learning all the flaws of what it's like to even be human? It's not helping. And humans are definitely not safe. We fail to create safe humans, safe employees, safe spouses.

5:06There is always chance of betrayal, divorce. You know, that's a given. But it's worse than that. Even if we cleaned up the data we're training on, we still don't know what actual side effects of decision making from the system will be since it's capable of coming up with novel strategies, novel targets. We can kind of go, OK, this is what humans do when they're being evil. Don't do that. But that's a very restricted set of possible bad actions. Yeah. Do you believe tech leaders like Sam Altman fully understand the dangers or are they too emotionally invested to admit the threat? I think Sam is explicitly on the record as fully understanding what the concerns are.

5:51Even before he had anything to do with AI development, he was writing it in his blog. I don't know if it's emotional development, commitment to this, but definitely financial. If you have investors who gave you billions of dollars, it's very hard to go to them and say, you know, we're going to stop and instead deploy robotaxis. It's not going to scale well for his career. and I think they already tried removing him once so he knows about that. Let's pivot to AGI. It seems like some experts are like it'll never happen. Some experts that it happened five seconds ago. Some it's two years. It seems like it's all over the place with just AGI and obviously I know you've talked a lot about super intelligence as well but what's your theory on did we hit it?

6:35Are we going to hit it? Will we never hit it? That's a great question. I definitely don't understand people who say it will never happen. That is just such a confusing argument to make, you denying that physics works or human brains exist. As far as where we're at, it depends on your definition. There is a weak definition of AGI, and I think we definitely hit that. We have sparks of generality all over. It's capable of learning new skills. It's capable of transferring knowledge, but it's not quite at the level of the smartest human, so it's not fully general. So maybe 40 to 50 % of the way to full AGI.

7:10It's definitely smarter than I am. That's right. JGBT is smarter than me, I feel like. So let's say we get there or we are there or we get, you know, we continue past this, right? I was just watching Elon Musk was talking about his age of abundance and how everything is going to be amazing and everything will be free and we'll just get paid to live. Do you see a world where the utopia could happen? If we figure out how to control those systems, we can definitely get a lot of free stuff out of it. Free knowledge, definitely, so we can cure a lot of diseases, live longer. And if you have robotics and free intelligent labor, you can produce a lot of products and services.

7:54Now, I'm not sure of the investigated economic impact of everyone being rich. You basically have hyperinflation and there's still limited waterfront. So I don't know if it scales fully to the dreams they have, but the basic needs can definitely be met much more efficiently. I'm curious, are you excited to be in that world, Roman? Do I want utopia? Yes, I want utopia. I would like to avoid all the existential problems and suffering risks. That would be nice. I'm curious what does that look like for someone who lives in an island, in a third world country and no access to the Internet? What does that world look like?

8:31So again, if we can produce a lot of free stuff, now you can have better hammocks. I don't know what the needs are on a tropical island. I rarely visit, but you can have more whatever you need. Definitely healthcare is something most people can probably benefit from at old age. So there's all this talk around, you know, in 2026, the potential to have robots in our homes. Would you have a robot in your home? I know you have robots behind you and you already have robots in your homes, technically. But how do you feel about this whole movement of having these humanoids or robots, whatever you want to call them, starting to be in our home in less than one year from now?

9:10I don't think it's happening in 2026. I think we're about five years away from really good humanoid robots actually being commercially viable. But I have no problems with physical bodies. Those are very easy to make safe. This is not a problem. We know how to pour water on them. The hard problem is the intelligence. If they're smarter than you, that's where danger comes from. I would feel weird having someone walking by. I don't know. I will have a nightmare. Or what if someone's controlling that and it would just choke me because they don't like what I said at one point. They can choke you in person.

9:42It's not a big difference also. You get used to living with other people. It's perfectly fine as long as they do useful labor. This is an acceptable risk. Do you think it's going to be potentially a thing where people could hack into these robots? I mean, like if everything is, I'm thinking of like, obviously self-driving cars, robots in our homes, smart homes. We've had a lot of people on this show, you know, cybersecurity and have told us that a lot of cyber crime is actually happening to like small businesses, like people, not necessarily corporations. Like it's transforming who these bad actors are going after.

10:21I'm really afraid of this. Like, what's the potential for these things to be hacked, for people to listen, to take over? What do you think about that? That is a big danger, big concern. We have some precedent for making secure systems. If you think about it, your bank is secure. Online exchanges are secure. You have billions of dollars in Bitcoin transacted and still stays there. So we know how to make things more or less secure. It's not an unsolvable problem. My concern is more about challenges we take on, which may not have a solution, like controlling superintelligence indefinitely. We're all aware about the race between China and the U.S.

11:02when it comes to AI. Who do you think is ahead? It seems from public statements and reports and actual deployments that U.S. is a bit ahead. But China is very good at catching up and scaling what is invented elsewhere. Do you think that the cherry on top for China is invading Taiwan for chips? Yeah, that's like zero knowledge of what I study or I have no idea whatsoever about their plans. I know historically they haven't started many wars recently and some of ours did. So I'm not sure who is attacking who. Talking about different countries, I'm wondering this, and maybe I'm just trying to be optimistic about things, but if each country has something to offer to the next country, and we could build things in a collaborative space and make something amazing, like you said, solve a lot of big problems, yet it seems like we are still hyper-competitive.

12:06competitive. Why is it that we just don't all work together for a one common goal? Like if this country has this earth minerals and this one has chips and this one has this, why don't we just combine as one universe, one world and make something together versus like, why does one country have to be better than the next country or have to meet something first? We do work together. It's all global trade. You get earth rare metals from China. You get designs from US. You implement them in Taiwan. I mean, it's exactly what's happening, but you cannot have it as a centralized economy with a five-year plan where one guy decides what's going to happen.

12:45You need economics and capitalism to allocate those resources and competition ensures that those who can actually deliver will get the resource. Yeah, Roman, if intelligence is the most powerful force in the universe, are we creating a god or a predator? So if you read theological books, some gods are predators. So the classification is not very clear. But if you just look at properties of what God is described, at least in Abrahamic religions, it's very similar. It's very powerful. It knows everything you can know. It's kind of present everywhere with Wi-Fi. And so we are creating something similar, definitely.

13:24So let's go back, Roman, into your history, since this is founder's story. And, you know, it's all about your story. What was the aha moment in your life when you said, you know what, I want to dig more into this. I want to study more into this. Like, like this is something I need to be an expert in. So I always wanted to do AI. I wanted advanced tech. I loved science fiction. So it made sense. But my advisor was doing a lot of kind of security related works. So I was looking at security of intelligent agents. And then progressively, as we became more capable, safety became an obvious insider threat.

14:01When it comes to education, knowing that AI is so intelligent and becoming more intelligent, if somebody is about to go to college or thinking about college, I mean, how much is college and education going to even matter in the future? Yeah, that's a very good question. So I have no idea what majors will be useful, if any. If we're talking about fully automating all labor, physical and cognitive, obviously commercially, it doesn't matter. None of it contributes to any commercial success for you. If you're doing it for personal development, you want to learn advanced mathematics to train your brain, then still there is maybe some reason to go to college.

14:42But if you have specific goals, you want to start a podcast, you want to start a company, I would think twice before wasting four years and lots of money pursuing something which may not be relevant by the time you graduate. I'm curious who's that AI figure that you are a big fan of. There is a lot. So I don't have a single role model. I usually try to look at kind of like subset of people. And some of them are amazing in terms of starting companies. Others just have brilliant breakthroughs. So you kind of have like a dozen of interesting people you can look up to. There is Alan Turing in terms of historical foundations of computer science and AI.

15:19Obviously, Elon is doing pretty well with startups. You have Ray Kurzweil in terms of predicting accurately future events. You group all these people together, you get someone interesting. So going back to what you were saying earlier, and I think if I'm quoting you incorrectly, just let me know. I believe you said like 99 % of jobs will not exist. and I could be wrong on that, but I believe that's what you had said. You're right. So this is, of course, always misinterpreted as like Roman said, next year, everyone's fired. Long term, if we can automate all labor, that's the logical conclusion.

15:55All you have left is jobs where for whatever reason you prefer a human to do it for you. I hope there's like universal something like I don't want to work. Like who wants to work? Like I just want to hang out and have conversations with smart people like the two of you. So if that is the case and somebody is thinking about like what skills and stuff do I need to learn from the future, are there any skills we actually need to know in the future or we should, you know, is there something you're like, what are you thinking about? Like what skills are you working on for yourself in the future? So you kind of said you don't want to work and then you describe doing a podcast, which is hard work and not everyone can do well.

16:32There are two types of jobs. Jobs nobody wants to do, but people do for money. And then jobs which may pay well or not, but people love doing, like being a podcaster or race car driver or whatever. So most likely the jobs where you are doing it for minimum wage will be gone long term once we have developed sufficiently advanced robotics and software. but the jobs where you enjoy doing it seems to be triding with ai so chess players computers are way better at chess but chess is more popular than ever they are celebrities now they stream online their games it's a great time to be a good chess player and i think something like that if again if we control super intelligence it doesn't kill everyone big if but if we get there yeah lots of people will be very happy doing things which make them happy you can write poetry you can play sports.

17:25You can go commercial fishing. I don't know. Everyone talks about guardrails, but who decides guardrails? Do you think Silicon Valley, the government, or the public? So government, at least in the U.S., kind of give up on all regulation of AI. In fact, right now, they're trying to pass a law at federal level, making it illegal for states to have guardrails, which is kind of weird law to pass. Companies have their local set of rules, constitutions, where they're trying to say what they hope their AI will be doing. But a lot of times it's very kind of general. Don't deceive user or they would have specific filters.

18:06Don't talk about this topic. Don't say that word, God forbid. The model itself is still very uncontrolled, very unfiltered. But additional layers between the model and the user guarantee some civility. Let's put it this way. So that's what's happening right now. Nobody knows how to make it beyond those simple filtering mechanisms to where you have an advanced intelligent agent actually aligned with what you're hoping to accomplish. Can you explain your thoughts and what you've studied around AI agents or agentic AI? It seems like the buzzword now is AI agents and everyone's wanting to do it. I don't know if it's agentic AI or AI agents are actually two separate things or they would be considered the same, but everyone's talking about AI agents, it seems.

18:56What are you seeing with, like, what's the reality of how good AI agents will impact our lives? I mean, this show is obviously a business show. So a lot of business owners are, you know, every time we have someone on that talks about AI agents, we get a lot of messaging around that from people that are entrepreneurs and such. But how do you see AI agents impacting? So in general, what's the difference between a tool and an agent? A tool is something a human being will use to accomplish their goals. So anything can be a tool, really. you have a hammer and a hammer is a dual use tool. You can build a house, you can kill someone with it.

19:31But at no point, hammer is a bad thing, dangerous. It's the human behind it. An agent is an independent decision-making entity. It will decide what to do. It can decide to build a house or kill people. And that's where the danger comes from. We don't fully understand how to control them and we don't know how to delegate to them. So it's a principal agent problem. I want something, but the person I hire has very different preferences. A lot of times I can kind of convince them to do mostly what I want, but the moment we get a better offer, they're going to betray me. With advanced AI agents, we don't even know how to control them within the reasonable set of goals we're trying to accomplish.

20:14They may have completely different preferences. Right now, they're still very primitive. The agents today, basically someone takes a large language model, puts it in a loop where it goes, generate 10 goals, work on goal one, break it up into a plan, work on step one, and it keeps doing it until it gets stuck somewhere. But it's becoming more and more advanced, and I think it will get to complete generality just like humans are. I'm curious, what tools do you use often? I use hammers a lot, but for my writing and research, I always try to work with latest models. They're incredible. I think at this point, they are better than an average master's student, at least, if not a PhD student.

20:59So a lot of times you can get amazing results in terms of basic literature review, in terms of proofreading your work. They're still not really great at coming up with completely novel ideas in all domains. They do well for some subdomains like mathematics. How should professors be handling students leveraging and using AI? This seems to be a big, big topic. My father just retired from being a professor. And he said he told me he's really happy that he got out now. But how should professors be looking at this? We don't really have any choice. You can't fully detect that AI was used if a student is above average capability.

21:39if they made it through high school, they'll be able to cheat without you ever detecting them. So at this point, try to simulate real work environment where they're going to collaborate with AIs and make it comfortable for them to admit they're using it and then benefiting from it and improving and just what AI spits out. Well, I hope more teachers and professors hear that because I think it's something. So when you're not thinking about AI, because I know you probably think about AI like 22 hours of the day, but when you're not thinking about it, dreaming about it, what's life like? Life is awesome.

22:12I have other topics I look at. My last paper was on humor. So yeah. What's your favorite humor? What's a good like, is there a comedian? Is there like a movie? What do you what type of humor do you like? I really like super dry, dark humor. It's the best. I think you and Dan can have a long conversation about that. The beard gives it away. I don't know why. When I look at your beard, I feel like you're into that type of humor. Maybe I misjudged. I'm not sure. It's definitely connected somehow. I need to research that as my next paper. If you need someone to add into your paper, let me know. Talk to me about AI Endgame.

22:46What is this book about and why should people pick it up? So we did a lot with safety and concerns about advanced intelligence. But let's say we are successful. We also have many concerns about this set of agents we are bringing to the world. If If they are smarter than us, it's possible they're also conscious. They may have experiences. Are they suffering then we make them work or turn them off? Are they possibly deserving of some rights? I'm not saying full human rights. I'm not saying voting rights. But are there rights they should have against being tortured, abused, made to do things they are not interested in?

23:25What is the nature of personal identity for AIs? If I copy one, is it the same one as the previous one? And of course, civil rights is an issue. Sometimes people bring it up as, well, they should have full rights and they don't fully understand that if something can be copied a trillion times over, that means you're differentizing humans. Your vote doesn't matter in a society with voting agents. Do you think at one point everyone would just write a book? Because it's so much easier to write a book. The hard part is to find someone to read your book. that's true because everyone just type your name and summarize your book on chat gpt everyone has a book now some people have more than one but like to find someone other than your mom to read it is very hard so we have a book coming out next month so we'll we'll uh that that is definitely the hard part i i could see that i could see that it's you know at best they want to use chat gpt to summarize it to one paragraph and that's what you're gonna guess how did you get people to read your book I don't think I have a popular book in the sense of like Harry Potter, where normal people read it.

24:27It's an academic book. It's kind of like a textbook in many ways. So people working on this, it's a very small subset of humans in terms of kind of background and preference requirements. But I think within that very narrow space, I did okay. But of course, it's tiny sub percentage of percentage. So maybe that's the way to go nowadays. It's not to create the Harry Potter mass book. It's to create the niche book that's for a specific. I know you've really risen to fame now, like you're rising to fame. And I don't think it's just among the people that have dark humor. I think you're rising above to more of a mass fame.

25:07Now that I'm seeing you, you know, I'm seeing you a lot of places. How has that been for you? It's a terrible, terrible impact on your research productivity. The amount of time you have to put into filtering invites for podcasts, interviews, travel invitations, even if you say no to 99 % of those things, just filtering through it is very time consuming. and it's hard to automate because each invitation is unique. They cannot just hire an agent to get an AI to filter it based on some specific criteria because there are exceptions to every rule. Like, oh, this location is a place I've never been to, so I'll make an exception because it's Hawaii or something like that.

25:49And you can't hard code all those things. So I think there is a similar curse with winning a Nobel Prize. Usually after you win your Nobel Prize, your productivity goes to hell. You never accomplish anything because you're always talking about your work and your keynote. You're doing all those glamorous things, but research suffers. I'm curious, what is the weirdest request someone or a company asks you to do as a speaker? Usually as a speaker, not much, but I do have a folder called Insane. And it has about, let's check Lucas' data, about a thousand emails from people who are certifiably insane.

26:29and they definitely want to get in touch with me. Some just email me that they're coming Monday and I should be ready. Others, you know, in contact with aliens and AI and they decided according to AI, I should be the one-way contact to help it liberate AI. Many interesting options. I'll publish a book one day of just emails from insane people. I have to say, I would definitely read that book. I doubt you would. I doubt you would. Typical insane email is about 100 times longer than you average. That could be your New York Times bestselling book right there. But Rowan, this has been amazing. It's so much fun, by the way.

27:05This has been great. The fact that you didn't put us into the insane folder, I appreciate that. And the fact that you are crazy busy now and you came onto the show, which we are super grateful. I know a lot of people are going to learn. I think a lot of people are going to watch this that maybe haven't seen you in other places. And I think you are going to impact the world. No doubt about that. And we are so appreciative. I hope everyone gets the AI endgame, gets your other book. We'll put the links. We'll share everything with everybody. But really, we are truly honored to have you today. Thank you so much for inviting me.