972: Mustafa Suleyman | The Coming Wave of Artificial Intelligence

24:15It's kind and supportive. I think that that's the first reason is that you can actually see the power of these models in practice. And then the second thing is just the rate of improvement is kind of incredible. And what's driving this rate of improvement is training these large scale models. And what we've seen over the last 10 orders of magnitude of computation, so 10 times, 10 times, 10, 10 times in a row of adding more computers to train these large models, is that with each order of magnitude, you get better results, right? The image quality is better. The speech recognition is better. The language translation is better.

24:53The transcription is better. The language generation is better. You can clearly see that this curve has been very predictable. And over the next sort of five to 10 years, many labs are going to add orders of magnitude, 10x, 10x, 10x per year. And so I think it's quite reasonable to predict that there's going to be a new set of capabilities beyond just understanding images and video and text. AIs are going to be able to take actions. They're going to be able to use APIs. They're going to be able to predict and plan over extended time sequences. And so I think that's why we're all predicting that this time is different.

25:30This is the Jordan Harbinger Show with our guest Mustafa Suleiman. We'll be right back. If you're wondering how I managed to book all these great authors, thinkers, creators, and just straight up geniuses every single week, it is because of my network. That is the circle of people that I know, like, and trust, and that know, like, and trust me. That's perhaps more important. And now I'm teaching you how to do the same thing, build the same thing for yourself in our six-minute networking course over at sixminutenetworking.com. I know that networking is something most of you don't like to do. I get it.

25:58You don't want to be a used car salesman. This course is non-cringy. It's down to earth. It's going to make you a better colleague and a better friend, not an annoying guy or gal in the office or in your career or in your space. It just takes a few minutes a day and many of the guests on the show subscribe and contribute to the course. So come on and join us. You'll be in smart company where you belong. You can find the course at sixminutenetworking.com. Now back to Mustafa Suleiman. It really is amazing to think that, yes, you're right. The creativity thing blew me away. Looking at some of these image generators, I couldn't believe that somebody posted something.

26:34This is literally maybe a year or two ago at most. Look at this AI created image. And I thought, well, okay, but how does it create the image? Surely it just had an image and then changed some of the things in the image and then redrew it. And it's like, no, someone asked this to draw, I don't know, Jordan Harbinger in front of a communist flag standing on a mountain. And it's like, there it is in a few seconds. That was really mind blowing, this kind of thing. Because if we can do that with still images, and you mentioned now with real time video, that just eliminates a ton of work. But also eventually, you're not gonna have to ask it to do anything.

27:10It's just going to start creating things. I mean, you could easily, I'm sure already craft an AI that would just start making things according to your own preferences and then continue to do that. Mark Andreessen also gave the example of, instead of watching something on Netflix and they hope to get it right, you just tell Netflix what you like, or it already knows because you've already watched 10 ,000 things on Netflix over the past 30 years by that point. It just says, we've made a show for you. It's kind of like Game of Thrones, except it's got that futuristic dystopian stuff. All the dragons are robots and it takes place in space because you like Star Wars and you're just like, I'll watch that.

27:43Right, and then after the first episode, it's like, hey, your eyes were more engaged when the dragons were fighting. So the next episode is going to have way more of that kind of conflict. Oh, you don't like the space stuff and zero gravity. All right, fine. We're going to bring it back down to earth in the next episode because you're more and it's just going to be able to do that kind of thing. And people will of course say, well, how is it going to know what you really like? To your point, I think when people have said, hey, these are they're not humans. They can't read emotions. I think now computers are better at reading emotions than humans are in tests like a robotic doctor could actually have a better bedside manner than a human doctor who's actually really good at their job.

28:18Yeah, you're totally right. And that kind of personalized content generation is definitely coming. I mean, it's actually what we're trying to do with text and image and articles with Pi, right? So Pi actually generates you every morning a news briefing now that's personalized to you. Five stories in spoken text with a nice image to go with it, summarizing what's happened in the news. And then you can actually talk about the news with Pi and based on how you react to the different stories, you may say, oh, I'm really not interested in that kind of sport or I'm sick of hearing about this war that's going on or I'm really into bicycles.

28:57And the next day, Pi's going to produce something that is closer to what you like or you're interested in. And that is in a way where we're already at, right? So let's not get too carried away here. I mean, that's what a podcaster does. That's what a content creator does on TikTok, right? They're constantly trying to produce things which are more interesting and surprising and educational to people. And so we're now just kind of automating and speeding up that process. But you're right. I think the thing that we have to think about as a society is where are the boundaries and where are the limits?

29:26How do you contain this? What is off limits? There have to be some limits, right? What subject matter? What style of persuasion? Is it okay if just I get to control? Do I get to consume whatever information I want just as an individual? Should it be entirely free and decentralized? advice. Clearly, we don't want it to be top down and run by a tiny number of companies. Right. We also don't want it to be run by a tiny number of governments that can say, you know, censor this, that, the other. I mean, we can see what's happening in China as a example of a way that we don't want to live. Right. So no one has the answer.

30:00So if anyone comes to you and is lecturing you about, well, it should be this, this is the problem, that's the criminal. The truth is none of us fully know exactly what the right step to take is next. But the more we sort of talk about the risks and the more we proactively lean into those conversations and not, like you said earlier, put your head in the sand. In the book, I tried to frame it around this pessimism aversion trap. I think it's particularly an issue in the US where there's such a desire to believe that the future is going to be better and the kind of bias towards optimism that I think it leads people to just be afraid of potentially talking about dark outcomes.

30:39Like we have to talk about the potential ways in which things can go wrong so that we can proactively manage them. And so we can actually start putting in place checks and balances and limits and not just have a bias towards optimism that leads to us missing the boat when it comes to the consequences that affect everybody. This is wise because, look, when I was a kid, I had an Apple IIc. It was the kind of computer we had at school. I had one at home because my mom was a teacher. So we got one. It had 64 kilobytes of RAM. Now, I think I've got 64 gigabytes of RAM in my gaming laptop over here, which is, for people who don't know, it's a hell of a lot more.

31:17Within memory, I think I drove to a computer store and I bought a 420 megabyte hard drive. And I remember getting home and going, I'm never going to fill this thing up. And now, if I go and I download a game, the update to the game that has, like, bonus graphics on it or something is way more than 420 megabytes. It's probably 42 gigabytes or something like that, right? That hard drive wouldn't even scratch it, but it would also take me three hours to write to that hard drive or more. So this is in part due to something called Moore's Law when it comes to processors, right? So Moore's Law, can you first of all tell us what Moore's Law is?

31:52And then naturally my follow-up is, is there a Moore's Law for AI? Yeah, great question. So Moore's Law was predicted by this computer engineer, Gordon Moore, who was the founder of Intel that manufactured computer chips back in the late 50s, he predicted that transistors or computer chips were going to get radically cheaper, halving cost every year for the next 60, 70 years. And the crazy thing is that is exactly what's happened. And so we've been able to cram more transistors onto the same square inch for the same price. And so we've just seen this reduction year after year after year in the cost and increasing the density of transistors, which basically is what you're describing is your hard disk is still the same size.

32:44In fact, in many cases, it's actually got smaller, right? You now have a thumb drive, which is the size of your SSD back in the day, right? Your 420 megapy SSD. So that has been the main thing that has been powering this massive revolution, because for the same price, we can store more, process more, etc etc which means we can have photorealistic graphics which means we can have these ai models now that have access to like all the information on the web and super amounts of knowledge so in the context of ai there is a more extreme trend than that right which is that as i mentioned earlier this 10x increase in the amount of compute used to train the cutting edge ai models per year so instead of doubling per year which is the moore's law trend we're increasing the amount of compute by 10 times per year, because in this case, we don't need the compute to be smaller.

33:36We can just daisy chain more computers together. So our server farmer inflection, for example, is the size of four football pitches. It's absolutely astronomical, uses like 50 megawatt of power. And so you look at it, it's like absolutely mind blowing. It roars like an engine. And all of that is, is really just graphics cards. Just like you have in your, if you have a desktop gaming machine, you might have a GPU graphics card. We just daisy chain tens of thousands of these up together so that they can do parallel processing on trillions of words from the open web. Every time Pi produces one word when you're in conversation with it, it does a lookup of 700 million other words.

34:23That's bananas. I mean, it kind of lights up or activates or kind of pays subtle attention to 700 million words every time it produces a new word, obviously. So when it's producing paragraphs and paragraphs of text, there's a huge amount of computation. That is the trend that is accelerating much, much faster than Moore's Law and is going to continue for many years to come. I would assume at some point, the AI itself will figure out how to make that process more efficient because it's learning everything that there is to know from at least that there is on the internet, which is pretty close to everything.

34:59It just seems... Yeah. I mean, we have that today. So we have that server farm that I described to you. We train one giant AI out of that server farm, and we actually use it to teach and talk to smaller AIs, which are cheaper for us to run in production when you get to chat to it. Because it's more efficient for us to have, rather than paying tens of thousands of humans to talk to our small AIs to teach them, which we do do as well. We have 25 ,000 AI teachers from all walks of life and all backgrounds and all kinds of expertise. And they talk to the AI all the time and they're paid to give it instructions, say, this is factually incorrect.

35:38This isn't very kind. This is what funny looks like, et cetera, et cetera. Now we're actually getting the AI so good that it can do the job of the AI teacher better than the human AI and teach these smaller models to behave well. So what you described in a way is kind of already happening. That reminds me of the way podcasts work in brief. People think, oh, I download this from your server, but not really. So I upload this to a server, which is probably on one of Amazon's data centers. But if somebody in Japan downloads this episode of the podcast, there's a copy of that file cached somewhere on servers that are probably, I don't know, outside of Tokyo somewhere.

36:21And then if somebody else in Japan downloads it, they don't connect to my server in the United States. They connect to that server in Japan. Japan server says, hey, is this file the same one that you're still putting out over there in America? And our network says, yeah, but we want to put this ad in there that's in Japanese because the other one that ran was in English. Just switch that out. And the server essentially says, okay, cool, and gives them the exact same file. It sounds like that's a little bit of how Pi works, right? It's almost like, oh, other people have looked up recipes in the United States.

36:48We don't have to ping the main guy right over there in that giant multi-football pitch data center. We understand how to tell them how to cook this soup. This has been done. Here it is. I know I'm oversimplifying it, but it sounds similar. That's actually a great metaphor. I mean, another way of putting it is that you don't need to ask the professor of computational neuroscience how to make the recipe for spaghetti bolognese. You can go to an expert in that kind of area that doesn't require 20 years of training in neuroscience to become that expert. So that's exactly the concept. Just like we deliver content to different parts of the web, we have different specialist AIs that are really small and efficient at answering different types of questions.

37:34Tell me about the video game playing AI machine, for lack of a better word, that you designed back in the day. because this was kind of, it sounds like one of your first experiences seeing AI do something that was, it's funny, I'm putting this in air quotes, truly amazing because it's something you do when you're nine years old and you're playing the same video game. But still at that point, right, it was totally mind blowing. Yeah. I mean, that was more than 10 years ago now, 2013. And we trained an AI to play the old school Atari games. so things like breakout and pong for example where you have two paddles and you bat them back and forth or breakout where you have to bounce a ball up and down with a paddle at the bomb that you get to control left and right to knock down the bricks or space invaders where you you know shoot the enemy ships and the crazy thing about this is that instead of writing a rule that said you know if you're in this position and the ball is coming at these degrees then move the paddle left one degree blah, blah, blah.

38:33You basically allow the AI to just watch the screen and randomly move the paddle back and forth, left and right, until it accidentally stumbles across an increase in score. And then it's like, oh, that's pretty cool. I managed to increase the score. How did that happen? I'll try and do that next time I'm in that position. And so it's through random self-play that millions of times playing against itself, because it sees all the screens, 24 frames a second frame by frame, all the pixels, it's able to learn a pretty good strategy of playing the game. And then one day we saw that it had actually learned a strategy called tunneling, where it would ping the ball up one side as often as possible and try and aim it up in the same place.

39:18And then that would force the ball to bounce behind the bricks back and forth, up and down, and get kind of maximum score with minimum effort. And that was not a strategy that most human players knew about, right? Like most of us didn't really discover that. Some of us did, but I certainly didn't. And that was pretty mind-blowing. I was like, wow, these things can not only learn to do it well, but it can actually learn new knowledge or new strategies or discover techniques and tricks which could actually be useful to us. And that was, after all, why we started building AI. I mean, that's what we want from AI.

39:55We want AIs to be able to solve our big problems in the world. We want it to help us tackle climate change and improve drugs and improve healthcare and give us self-driving cars. And we want to solve these massive problems that we have in the world of trying to feed 8 billion people and growing and so on. So to me, that's always been my main motivation. And when I first saw that, that was like the first sign that we were onto something back 10 years ago. The reason that's so amazing, and I think it's easy to gloss over this and go, so what? It learned from the best players and it copied the strategy, but that's not what happened, right?

40:30It didn't see somebody who was really good at Brick Breaker and they go, oh, okay, what he does, he breaks the side and then he gets the ball stuck in there and then it does the rest of the work on its own and you can't really lose. It figured that out through trial and error, which is really incredible because you might have to play Brick Breaker for a few weeks, months, or even years before you come across that strategy by accident and then go, oh, I need to replicate that. So this can figure it out in seconds, potentially, something like that. And then we also now, we want AI to figure out the equivalent of tunneling for, I don't know, cancer research or something in quantum physics, right, that we would never figure out because humans haven't been there yet.

41:09And the AI goes, huh, if I want this particle to last longer than a few milliseconds in a controlled environment, I need to do all these other things and bada boom, bada bing, now I can make elements that don't exist that can be used to create power, for example, generate power. Yeah, I think that's totally right. That's the ambition. And I think it's a very noble one because the world today, we've got a lot of challenges, whether it's food or climate or healthcare. I mean, the prize is big and we need assistance in trying to invent our way out of these challenges. And all that we've got so far is our human intelligence.

41:44And everything that is of value in the world today is a product of us being smart at predicting things. And that's basically what these AIs do. They absorb tons of data and information and they make great predictions. So the thesis is, well, we could maybe scale up this prediction engine for the next couple decades and really have some massive impact. Are you able to explain just briefly how AI works? Because you mentioned before it searches 700 million words. and in the Marc Andreessen episode, he was like, ah, it's like a really fancy or smart autocomplete, which I understand what autocomplete is because I use Google and my phone tries to guess what I'm going to say and it's often right, but that doesn't really scratch the itch for me because then it's just reliant on looking at what humans have done, which is not really what we're saying AI does, right?

42:33So I think that's right. Look, it is very difficult to describe because it's hard for us to really intuit it and deeply understand very large numbers and very large information spaces. So I think the first thing to try to wrap your head around is that one of these large language models reads many, many times everything that has been digitized on the open web. And so this is trillions and trillions of words, books and blog posts and podcasts and YouTube downloads and everything that where there's text, it's consumed it. And what it's learning to do is it covers up the future words. And given the past words, it predicts which word is likely to come next.

43:20So it's almost like it memorizes the whole thing. And then you test it. And you say, given this phrase, the cat sat on the, what is the probability that the next word is head, chair, car, plane, road, banana, continent, right? And so there's going to be some probability assigned to every single one of those words, even the really, really weird words that have never appeared after that sentence. And of course, the most likely one is mat, right? But that's a very simplistic description because not only is it good at predicting or auto-completing which word is going to come next. It's able to do that with reference to a stylistic direction.

44:07So just as you say to an image generator, produce me a banana in the shape of an owl in the style of Cezanne, right? Now you might be able to imagine that kind of weird combination in your head. What the AI is able to do is to take those three concepts and not just the concept, the plain word banana, but actually its entire experience of banana, every single setting in which banana has arisen, right? All the different kinds of combinations and shapes and styles, and it has this very multidimensional hazy representation of banana. And then it's able to interpolate, which is predict the distance between banana and owl.

44:52So that's a very powerful thing because it's a stylistic, it's a position on the curve. It could be very, very like owl. It could be very, very like banana. Now imagine that you added all the other words. Imagine the owl is flying. Imagine it's big and red. Imagine it's a banana that's going off. Imagine it's a banana that's been thrown off the edge of a building. Now we're honing in and we're reducing the size of the search base. It's almost like adding filters to reduce the size of all possible things. That's just a very difficult thing to grasp when it's massively multidimensional. I've only described it in the context of two or three concepts, but now imagine that it's like hundreds of concepts or thousands of concepts of stylistic control.

45:34And as the models have got larger and they get more access to more compute, you can have more fine-grained control. They become more, and that's why they're more accurate, right? They're more useful because they're able to attend to multiple sort of stylistic directions simultaneously.

46:19sleep. Yes, it is that important. Thank you for supporting those who support the show. Now, for the rest of my conversation with Mustafa Suleiman.

46:50goes, well, there's going to be an occasion where I really need a brown suede jacket. And this one has fine details and I like it. And it's, it's going to come in useful and I don't really care. I just really wanted the jacket. Right. And I can do that. And I'm, I'm quite self-aware that I bought a jacket that I didn't freaking need. And now I'm really trying to rationalize that purchase because it was expensive. This is the best my brain can do. And I'm like a reasonably qualified human. I'm leading a mostly successful life. Right. What happens when we're looking at a brain, an AI that is so much more sophisticated than our own, but it's being terrible in some way.

47:26Are we going to be able to get in there and diagnose that? Or is it going to be just too complicated of a black box? Well, that's a cool question. And I think you kind of nailed the answer in your question, which is that humans hallucinate all the time. Yeah. Our main mode of communication is to retrospectively invent some narrative that seems to fit the bill. We're constantly being creative and making stuff up. In fact, when you remember something, you don't really remember. What did you have for breakfast this morning? You have a very vague, loose memory. Maybe you can get it. What did you do two weekends ago?

48:03You're going to be creating all kinds of stuff that is plausible and vaguely within. We make things up all the time and that's what creativity is. That's what a hallucination actually is. We don't have very good ways of inspecting inside a human brain. You can whack somebody in an fMRI scanner, but it's pretty crude and it's not reliable. So the way that we trust one another is that we observe what you say and what you do. And if what you say and what you do is consistent with what you have said you're going to say and said you're going to do, then over time we build up trust because we have that continuity between intent and outcome.

48:43And that's the behaviorist model of psychology, right? We observe the output and we focus less on the introspection and the inner analysis. And I think that practically speaking, that's going to be the standard to which we hold a lot of these AIs for the foreseeable future. Now you could ask, which I think is what you're getting at, which is in the long term, well, isn't this thing going to be really good at deceiving us because it's just going to get smarter and smarter and smarter. And I think maybe the good news is we can actually interrogate these models better than we can interrogate humans.

49:17So it's not perfect. We're certainly developing methods of identifying when an AI has been deceiving, when it's misrepresented something, where in the model different types of ideas or concepts sit, and what the causal relationship was that led up to a particular output. The challenge is that's very early and early research, but the good news is it's software. And so we have a better time of investigating and interrogating software than we do sort of the biology of the human mind. That does make a lot of sense, right? Because if I go back and ask myself why I got that jacket, I have to really, even if I'm really trying to be honest with myself, I'm still going to sugarcoat the answer so I don't feel like a dumbass, right?

50:00But if you ask the AI why it said something prejudiced or racist sounding, it might actually just go, oh, because this training data set over here says that this kind of person often does this kind of thing. And you're like, okay, okay. We got to take that out of the soup. That's not the kind of data that we want floating around in here. It's not accurate. We don't want that affecting your decisions in the future. And the AI goes, okay, it's as good as gone, right? It can ignore that. I can't do that in my brain. I can't stop buying jackets. Yeah, exactly. And that's actually one of the weaknesses of being human in a way that we have emotional drives.

50:37And at the moment, AIs don't have emotional drives, and it's unclear whether they need them. So going back to my list of capabilities that should be off limits because they potentially cause more risk, I listed autonomy. I listed recursive self-improvement, whether AI can get better over time. On its own. On its own, right? And I listed that it had its own goals. It could set its own goals. And I would add to that, has emotional drives. I mean, it's not clear that we want AIs that have intrinsic motivation, ego, impulse, desire to do things or go places. Really, these should be treated as tools that work for us.

51:17They can still be very, very capable. but adding drives, I'm not clear that I see the justification that that would be a massive benefit to society so far. So these are the kinds of tricky conversations we have to have. Like, what is the benefit there? It's not clear. I think we can have an amazing scientist, an amazing teacher. I think we can have amazing knowledge workers that can be useful to businesses and be creative and so on without actually having emotional drives. The idea of a computer or, well, an AI having emotional drives is something straight out of Star Trek or something like that.

51:49I mean, just thinking about a computer that has an ego. I mean, I say computer, I'm oversimplifying it because, you know, a lot of laymen are listening to this, like myself. And the idea that the computer would go, but I have to be right about this one thing, or you're making me feel bad. I'm going to destroy your whole civilization. Like, yeah, we don't want that. We don't want that. That sounds quite terrible. Doesn't take a genius to figure out that that would be a bad outcome and that we might want and say that that's off limits. Yeah, especially as we create this amazing tool or set of tools, if we can even call it that, that's so much smarter than us.

52:21You mentioned in the book something called the gorilla problem. And by the way, folks, if you buy the book, please use our links in the show notes. It helps support the show. The gorillas that we see, of course, are bigger and stronger than us, but it's them who live in zoos. We're smarter, so we can sort of trick them into getting into a cage, and then we put them in the zoo, and they can't get out. We're currently masters of the ocean, in the land, the air, increasingly even of space. So what happens though, when we create something that's 10 ,000 times smarter than us in pretty much every measurable area, you know, the idea is that maybe it'll put us in a zoo.

52:55And I just hope the zoo looks a lot like where we are right now. Although now, now it sounds like I'm talking about simulation theory. Maybe we're already in the zoo. I don't know. Look, I think the good news is 10 ,000 times smarter than us is a long way off. And so we've got time to figure out that problem. I believe so. I mean, some people think that it's closer to 20 years. I think it's hard to say that it's like maybe more 40 or more, but beyond that time horizon, it gets very hazy and it's hard to judge. It doesn't feel like we're on the cusp of that anytime soon. And I know that's like not the most scientific analysis, but just instinctively, that's where I think me and most of the field are at the moment.

53:34I think the point to say is we wouldn't be able to prove that a system that is that powerful could be contained and would be safe. And therefore, until we can prove unequivocally that it is, we shouldn't be inventing it. That, I think, is a pretty straightforward, common sense reality. right? We can still get tons and tons of benefit from building these narrow, practical, applied AI systems. They'll still talk to us. We'll have personal assistants. They will automate a bunch of work that we don't want to do. They will create vast amounts of value. We'll have to figure out how we redistribute that value so that everybody ultimately will end up with an income.

54:18But that does not mean that we have to create a super intelligence. It just means that we will have created a huge amount of value in the world. And the current structure of society and the politics and governance around that is going to look very different to what it is today. I can get behind that. I think a lot of people can get behind that. I think the only place where people might take a little bit of issue is, okay, we should probably not build that. And then, you know, China's going, okay, fine, don't build it. We're probably going to try to build it though. And then I think you'd said something along the lines of, if one side is not in an arms race, but the other side thinks that they're in an arms race, then there's an arms race.

54:55I did say that. I think that is true, which is a trap that we have to unpick ourselves from because the other side doesn't want to self-destruct either, right? They're not crazies. Thankfully, even Putin doesn't want to commit suicide, right? Everybody has a survival instinct And that is what has led us to create relative global peace in the post-World War era with nuclear weapons. This idea of mutually assured destruction has actually been an incredible doctrine, right? Even though there have obviously been a huge amount of suffering and war over the last 70 years, we haven't been at World War.

55:37And that's great news because it shows that everybody will ultimately act like a rational actor if their future life is truly threatened. So I think that the argument that I've often made and others in the field is that a system that powerful is unlikely to follow your instruction to obey you as a ruler as much as it is you as an enemy, right? Because at that point, it's not going to care whether you're China or India or Russia or the UK or you're a government of how you actually constrain something that powerful, regardless of where you're from. I think that that's an initial starting point for thinking about how, you know, we all like add some serious caution here.

56:23If and when we get to that moment in decades to come, just to be clear, we're nowhere near that right now. But, you know, it's a question that we have to start thinking about. Yeah, it's scary to see the prediction that AI could then self-improve, right? Because it seems like as soon as it gets to that point, that curve could go so fast that we just wake up one day and it surprises everybody? Or is that sort of a sci-fi concern that I don't really need to have? I think it's a sci-fi concern. We haven't seen that kind of, it's called an intelligence explosion. There's just no evidence that we've seen that kind of thing before.

56:58However, the more we deliberately design these AIs to be recursively self-improving, like to close the loop and they update their own code and they interact with the world and then update their own code. And then if you just give a system like that infinite compute, right? Because ultimately, the good news is they run on physical things. They feel like information space bits, but they're actually grounded in atoms. Those atoms live in servers. Those servers live on land, which is regulated by governments. And so there is a choke point around which governments, the democratic process, people in general can hold these things accountable and can rate limit progress.

57:38And that's obviously good news. So I don't see this happening in a garage, you know, lab anytime soon. Right. That does make sense. It's kind of like if a car was sentient, it still needs gasoline and that gasoline still has to come from refined petroleum, which has to still have to dig for the petroleum and have the oil well. So it's like the AI might be able to make itself smarter, but it knows, OK, I need six more of these football field size data centers to do that. It can't just sneakily get those overnight. It has to somehow trick a nation state right into creating that for it or something and then do something nefarious with it, which gives us, in theory, a lot of opportunity to go, do we want to do this?

58:18Is this a good idea? Maybe we shouldn't do this. Maybe we need a safeguard. Maybe we need an off switch that's physical where somebody can go rip this plug out of the wall if this thing starts going cuckoo on us. And just to be clear, that is what's already happening, right? So this company NVIDIA that makes the AI chips, the GPUs I described earlier, in the last year, the share price has, I don't know, gone up three times or something crazy. It's one of the big trillion dollar companies now. And those chips were regulated by US government last year, right? So that they couldn't be exported to China, the very cutting edge chips.

58:54I think there's already a pretty good understanding of the potential for this to be used for military purposes as well. And the government has moved fast on this, proactively intervening to protect national security. And now as a catch up, starting to think about how it actually affects us domestically as well. So I agree with you. I think that there's going to be friction here naturally in the system that gives us time to see, look, are we just fooling ourselves and being doomers and exaggerating this kind of nonsense? Is it actually just nuts? Or is it actually real, right? And is it actually happening?

59:32And do we need to take some other interventional measures to make sure it turns out the right way? If we do end up in an AI arms race and we get to AGI first, and I think maybe we should explain what AGI kind of is as separate from AI. But let's say we get there first. So the pinnacle of AI development, right? We breathe a sigh of relief. But then what? Do we use our AI to prevent other nations from developing AGI? because it seems like whoever gets there first, right? They want supremacy in this area and that requires somehow preventing anyone else from also getting it. And I wonder what that looks like.

1:00:06Yeah, thank you for making that point because it seems so obvious. And I feel like I've been saying it to military hawks, you know, forever, you know, this new arms race with China. Well, what is the end point in this arms race? And let's say that we're winning. Let's say that we win it, whatever that means, that we cross the finish line first. What on earth do we do? We just go and whack them with it or something and prevent them from getting out. This is just like such basic thinking. And the way that we think about an AGI is this 10 ,000 times more smart and capable than a regular human. And that's the thing that I think we have to be cautious of.

1:00:42Regular AI, which is just an assistant in your phone that is going to help you be more productive and efficient and so on. That's really not within the scope of this arms race thing. I think what people are really worried about is this kind of like nation state level, massive effort to build this super intelligence. And I think it is very unclear to me what we would do with that. I'm not even sure we want to finish first in something like that because, you know, like you said earlier, it's just very unclear that we could make that provably safe. There's going to end up being some sort of US AGI.

1:01:13I mean, that's my term, Pentagon, feel free to use it. They have to develop some sort of strategy to develop and implement this because containing your own AI really hard. How do we contain someone else's? We can't get there. Even if we blow up their data center, obviously they've thought of that. The internet was invented to withstand strikes like that in the first place. So man, that's a really thorny product. I've heard people say, you know, the only defense against an AGI is to have your own AGI, right? So we've heard that one before, right? So, you know, it's, look, it's not that we shouldn't try and build these things.

1:01:46There's going to be huge amount of benefit in the next few decades. It's just that there are some big unknowns. We just have to start talking about those. AI is obviously going to be one of the greatest accelerants of wealth and prosperity in human history. And it's going to be the Industrial Revolution plus electricity plus whatever, plus internet. But do you think it'll spread the wealth more equally than, say, the Industrial Revolution, where a few folks own the railroads and the factories? Or is this going to end up being pure chaos? Without question. I mean, we're already seeing it, right?

1:02:17The rate of proliferation of this technology is unlike any other technology in history. And that is mostly because all of the infrastructure for making this technology available has already been built. Everyone's already got a smartphone. We've already got a laptop. We've already got a browser. We've already got cloud. We've already got the internet. And so this is just a tiny add-on to that apparatus for accessing information and talking to your computer. And that's why it's spread so quick. I mean, there's already billions of people talking to AIs like Pi and ChatGPT every day. And I expect that to continue.

1:02:52The models are getting smaller, they're getting cheaper, they'll be available in the developing world on SMS, on WhatsApp, very, very quickly, giving you access to a super expert in every possible field. And likewise, the ability to actually make AIs that are applicable to your own small business or relevant to your cultural context or using your personal data or understanding of your local business, et cetera, that's going to get much, much easier as well because the barrier to entry has been lower than it's ever been. In terms of manufacturing, the really large ones, like we said earlier, it still requires gasoline.

1:03:29You need these chips, you need these data centers. There's only a handful of groups in the world that can do that. And so that's going to end up being quite concentrated. That's for sure. Yeah, these supply chain choke points are kind of what we're relying on right now. I think you'd mentioned that we can't, or NVIDIA can't send certain chips to China. And I know they're sort of figuring out, oh, we can make a dumbed down version of the chip that still is around sanctions. But there's these lithography machines, and I've talked about this on the episode we did about semiconductors, that are as big as the, or probably much bigger than the room I'm in now that cost, I don't know,$180 million.

1:04:05And you can't buy one, even if you have that much money, because the Netherlands, the company that makes them is not just going to sell them to anyone. And they probably like constructing an aircraft carrier in terms of complexity, right? It's not something you can just have shipped. And so we've got these supply chain choke points. And that's kind of heartening, because you know that like a drug cartel is not just buying a bunch of AI stuff to figure out how to evade the war on drugs, which is not going so well anyway, maybe they don't even need it. But the idea that a disinformation farm in North Korea could buy their own semiconductor factory and then start making these is sort of not possible right now.

1:04:42But that's kind of one of the only things we have, right? Because otherwise we have treaties and things like that that might be international in nature. But man, regulation when it comes to an arms race is tough. We kind of did it with nukes, but we just had more visibility into the problem and it was slower. Yeah. Yeah. I mean, in a way, I think that nukes are going to help keep the peace this time around as well. Yeah, you might be right. Because look, nobody wants nukes to get into the hands of small non-state actors. And I don't think this technology is rightly compared against nukes, but it certainly would not be desirable to have very, very powerful AIs that can take massive actions in our world, that can campaign and persuade and do stuff just like a really smart group of humans.

1:05:30And to have that available to absolutely anybody is going to create a lot of instability. So I think there's going to be a collective desire among all nation states to try to keep the peace, just like we have today with all of the new technologies that arrive in our world. We have sensible regulations and they broadly work, right? We generally don't have fake drugs that kill a bunch of people. Generally speaking, aircrafts are pretty safe. Nuclear power is pretty safe. Like we haven't always got it right. Probably tobacco should have been banned a little bit earlier. We certainly haven't got it right always in social media.

1:06:07It's been pretty chaotic, right? So we just have to learn the lessons and keep moving forward. So I've asked other experts on this and I asked Mark Andreessen about this as well. He seemed to think, OK, AI can't be good. It can't be evil per se. And I suppose that makes it. He's like, hey, it's a machine, man. But if AI is absorbing human bias from training data, can't it absorb other undesirable human traits like malice or recklessness? Or am I just not understanding the categories of what it can do? Oh, yeah, definitely. I mean, it generally will reproduce the data distribution that it's been trained on.

1:06:40So if it's never seen a black face, it's not going to produce a black face when it's asked to generate an image. It doesn't know what it doesn't know. and by implication, it therefore knows mostly what it has seen. The training data does matter. Having said that, it also has stylistic control, which is very accurate. You can frame its policy of behavior in one direction or another. You can say, adhere to this set of values or a different set of values. And it generally does it quite well. It still makes mistakes, but I think those mistakes are going down and down and down over the years. And so I expect it to get pretty much perfect in the next few years at imitating the style that you want it to present with.

1:07:24I know predicting the future is really tough. And the further out we go, the less accurate everything gets. You mentioned earlier, hey, when you go so far out, everything's hazy. You've coined this term ACI as separate from AGI. Tell me a little bit more about that and what maybe the, I don't know if this is the right term, roadmap for that is and what this looks like over the next five years or so. Well, back in the 50s, Alan Turing, the computer scientist, came up with a test that tried to evaluate whether an AI or a computer system was intelligent. And he basically said, if it could speak behind a screen and deceive a human into thinking that it was actually a human and not a machine, then that would be intelligent.

1:08:04And today, it's pretty clear that these models, these chatbots like Pi and others, ChatGPT and stuff, are pretty good at conversation. And sometimes it's hard to tell whether it's an AI or whether as a human. So I think we've probably passed that test, the Turing test, or some lighter version of it perhaps. So I think a better measure of progress is to focus on what the AI can do. And capabilities are really what matters, not so much this abstract idea of what intelligence is. Can it write emails? Can it book flights? Can it come up with a new product design? Can it negotiate a contract? Can it market and sell and persuade?

1:08:47Can it do all of those things in concert, in sync with one another, in order to make a bunch of money? That would be a good test because money captures a lot of complexity to reduce everything to profit. It's quite simplistic, but it definitely is a pretty good test. I proposed that a measure of an ACI, an artificial capable intelligence would be one that could go off and make a million dollars with a hundred thousand dollar investment by creating a new product, promoting it online, creating a website for it, marketing it, getting it manufactured, getting it drop shipped, et cetera, et cetera.

1:09:26That to me seems very doable in the next three to five years. And I think that would be a much more sort of profound test, a modern Turing test, if you like, that would actually tell us something material about what it means for labor and the economy. Stuff is so interesting. Man, this really flew by. I want to thank you for doing the show. I've got so many more notes, man. We'll have to do another round at some point. And I just want to say, I appreciate your time and expertise. Really interesting. Thank you very much. It was a huge amount of fun. Yeah. See you next time. You're about to hear a preview of the Jordan Harbinger show about deep fakes with Nina Schick.

1:10:00We're no longer going to know what's authentic and what's synthetic. And not only that, it's going to become accessible to everyone. Porn is the beginning. The creations were just unlike anything anyone had seen before. This is a real live video where the celebrity is moving her face. She's got different expressions. I can make a nude image of your sister, your wife, your mom from a single photo, for example, from Facebook. Minors are being targeted as well. young girls. We are living at a time where there's going to be more disruption and flux than potentially has ever been in the history of humanity.

1:10:38And the reason for that is because of the exponential technological change that's coming our way. What is this information ecosystem that's basically come into existence in the past 30 years? It's going to take some time for society to catch up. To learn more about how we can avoid being duped by deep fakes, check out episode 486 of the Jordan Harbinger Show. Mustafa was really clear on this. Some of this might have gotten caught or been a little bit too in the weeds, but China is raising the United States for AI. Whether we realize it or not, whether we feel like we're in the game or not, AlphaGo, which was a game of Go, which is an ancient Chinese game, and AI played the world's best player in this.

1:11:16And the AI made what looked like a mistake, but was actually just rewriting strategy of the game from first principles that no human had done in thousands of years of this game existing. That was sort of this Sputnik moment for China where China was like, oh, this thing can beat us at our own game and do so in a way where we don't even necessarily see it happening. We think we're ahead and then suddenly we realize we've lost. And that's untenable for national security, China, of course, being no exception here. So they are really in it. And we have to pay attention because it looks like they're ahead right now.

1:11:50There are so many more uses for AI and jobs. In the future, most computer programs will actually be written by AI with humans in a supervisory role. At best, we're not going to need banks of coders. We'll be able to do all the stuff that they do in a day using AI. Even now, most coders use a tool that writes basic stuff so coders can focus on more knotty problems that are difficult to solve that might require a little bit of creativity that the AI just doesn't have yet. It's only a matter of time. AI, of course, will be one of the greatest accelerants of wealth and prosperity in human history.

1:12:23I'm paraphrasing here, this is from the book, but I wonder, will this prosperity spread more equally than wealth did in, say, the Industrial Revolution where a few folks owned the factories? Like, is it just gonna be Sam Altman and Mustafa Suleiman who are trillionaires and the investors are billionaires and then the rest of us don't get a piece? Or is this gonna be something where like everyone is using this and it's working so well that the results, the prosperity, the wealth is actually shared more equitably. I'm not always super positive about that outcome. Might make my job easier, but is it gonna spread the wealth per se?

1:13:01One thing we could say for sure is the people in the bottom are really gonna lose. I was rewriting something. I took a chat transcript and I was like, I wanna turn this into an article that I can send to this place that's not just me talking. And normally I'd pay somebody on Fiverr like$50 bucks to do this. Well, I put it through chat GPT and in seconds without negotiating with some dude who's going to make a bunch of typos. I had this thing rewritten from a conversation into an article in seconds and I just proofread the thing and I sent it over because they were going to turn it into something else.

1:13:30That means that those people who were previously doing that as their career on Fiverr, they're out of a job permanently and it's never going to come back. So the people at sort of the bottom of the socioeconomic scale of knowledge workers, I should say, those people are in deep, deep trouble. Ironically, and I think this is very telling, if you are an HVAC technician or a plumber, you don't have to worry about this stuff. Worst case, it's going to make your job easier. There's no robot coming anytime soon that's going to be able to route pipes and wires and HVAC ducts. So y 'all are set. I also worry that the distribution of the power that AI provides is going to be very chaotic.

1:14:03I'm thinking drug cartels are going to be using this. North Korea is going to be using this. Iran is going to be use in this, the regime that is, I just think that's quite dangerous. And in the book, he talks about how AI is going to enable cyber attacks, which is a unique angle that I had not thought of. Also in the book, how AI will erode the power of the state to protect us and for us to comply. There's too much there to really summarize in a show close. By the way, we mentioned AI and deep fakes. I did a whole episode on deep fakes with Nina Schick. That's episode 486. I think you're going to be interested in that if you're interested in deep fakes and information warfare.

1:14:36Really, at the end of the day, AI is going to be enabling things like terrorism, drone swarms that shoot only non-white faces or something, or only white faces. AI is really in many ways an extension of our best and worst selves. Just as it'll be available to the Mayo Clinic for cancer treatment, it's also gonna be available to ISIS. And containment of AI is something that really keeps me up at night. We know, right? We can't contain diseases. Look at the lab leak episode that I did, or if you believe that COVID came from a lab leak, which I'm still on the fence and leaning towards that's the case.

1:15:08We know we can't contain diseases, even when we try. I really think that it's very clear, even the top labs in the top most developed countries with this technology, they have leaks pretty regularly. You heard that with my Allison Young episode on lab leaks. How many leaks of AI technology do we need before the cat is out of the bag? AI is smarter than us, and it's smarter than a virus, and it actually, quote unquote, wants to get out just like a virus does and wants to spread. this is something that really should sort of keep all of us up at night. And I think it's inevitable that it does get out.

1:15:39So I don't know. It was nice knowing y 'all, I guess, at the end of the day. As always, thank you so much for listening. All things Mustafa Suleiman will be in the show notes at jordanharbinger.com. You can ask the AI chatbot on the website, but then it might follow you home. I don't know. Transcripts are on the show notes, advertisers, deals, discount codes, and ways to support the show all at jordanharbinger.com slash deals. Please consider supporting those who support this show. We've also got our newsletter Every week, the team and I dig into an older episode of the show. We dissect the lessons from it.

1:16:06I would love if you would take part and reply and join the conversation. JordanHarbinger.com slash news is where you can find it. Don't forget about 6 Minute Networking as well over at 6MinuteNetworking.com. I'm at JordanHarbinger on Twitter and Instagram. You're also welcome to connect with me on LinkedIn. This show is created in association with Podcast One. My team is Jen Harbinger, Jace Sanderson, Robert Fogarty, Millie Ocampo, Ian Baird, and Gabriel Mizrahi. Remember, we rise by lifting others. The fee for this show is you share it with friends and you find something useful or interesting.

1:16:36The greatest compliment you can give us is to share the show with those you care about. If you know somebody who's interested in AI, how this might affect their job, national security, etc., definitely share this episode with them. In the meantime, I hope you apply what you hear on the show so you can live what you learn. And we'll see you next time. This episode is sponsored in part by Conspirituality Podcast. You know how I'm always talking about critical thinking and spotting manipulation? Well, there's a podcast that's all about dismantling New Age cults, wellness grifters, and conspiracy-mad yogis, basically the wild overlap of spirituality and misinformation.

1:17:07It's called the Conspirituality Podcast. The hosts, a journalist, cult researcher, and a philosophical skeptic, dive deep into how this stuff spreads, from Project 2025 and the Heritage Foundation's dystopian vision of the future to how former leftists get pulled into far-right conspiracies. An interesting episode to check out is called Speaking Truth to Goop, where Jen Gunter breaks down the pseudoscience behind the wellness industry in a way that is super entertaining and eye-opening. It's sharp, funny, and makes you a lot harder to fool, which if you listen to this show, you know I'm all about that.

1:17:36From exploring cults to analyzing our cultural and political landscape, the Conspirituality Podcast will help you stay informed against misinformation and resist fear tactics. Find Conspirituality on Apple Podcasts, Spotify, and wherever you get your podcasts. This episode is sponsored in part by the Something You Should Know Podcast. Finding a great new podcast can feel like searching for a needle in a haystack. Let me save you some hassle. Check out Something You Should Know with my friend Mike Carruthers. Every episode is packed with fascinating insights that you can actually use. Mike interviews top experts on topics that shape your life, like how to optimize your brain chemistry, the best and worst ways to apologize, probably could have used that recently, why couples hate to talk about money.

1:18:13No matter the topic, Mike asks the questions you'd want to ask, getting straight to the good stuff, and the conversations are always engaging, always insightful, guaranteed to leave you a little smarter than before. They've got thousands of five-star reviews, but do you even need that? I'm recommending it. You don't need other people's opinion, right? At this point, I would hope. Anyway, give it a listen. Just search for Something You Should Know wherever you get your podcasts. Look for the bright yellow light bulb and start listening to Something You Should Know.