EP 4: Speedrun

27:08It can be intuitive. It can sort of like master this thing that I always thought was a human task. The more I see this move, I feel something changed. Maybe he just can show humans something we never discovered. Maybe it's beautiful. So the AI played a move that no human had ever made in any at least recorded Go game. And that means it discovered its own original strategy? Some people would go as far as to say that it had something like an original thought, an original idea. And what do we know about how it did that? How did Demis Sabas and his team at DeepMind explain Move 37? Well, they really wanted to know.

27:50And so they spent time digging through the code and looking inside the guts of the system. They wrote a paper about it. And while they were able to, you know, glean some information, because it's one of those connectionist neural net AI toddler styles of AI. Remember the trade-off that Yoshua Bengio was talking to us about. To get this kind of impressive performance, this level of intelligence, you just have to accept that you're not going to get satisfying answers to these kinds of questions. You have to accept some level of mystery. This is the black box.

28:32And when I asked Yoshua Bengio about this AlphaGo moment, With AlphaGo, I thought, ooh, now we're getting close to something important. He said this is when he realized that AI was now entering a whole new era. And he wasn't the only one. All across Silicon Valley, across the world of technology, people were singing Demis' praises. People were abuzz about AlphaGo. And of course, yet again, Elon Musk doesn't like this one bit. DeepMind's show of force. in AlphaGo freaked Elon out a lot and made this sort of ambling approach that OpenAI had at the beginning of, let a thousand flowers bloom, all the researchers can kind of pursue their own different areas.

29:19It made him have little tolerance for that. We now know in some detail what happened inside of OpenAI during this time because a number of their internal emails were revealed as a part of a lawsuit. And so you can see in these emails, Elon Musk telling Altman and the leaders at OpenAI just how frustrated he is that they're losing to Dimas still. In one of them, he says, OpenAI is on a path of certain failure relative to Google. There obviously needs to be immediate and dramatic action or else everyone except for Google will be consigned to irrelevance. What does immediate and dramatic action mean?

30:00Well, according to Keech Hagee, This is when Musk was saying, where is our game player? Elon wanted to drive like fighting tit for tat with DeepMind. And he really wanted to respond to it by showing an even cooler and harder game that AI could beat. We need to challenge DeepMind in some kind of public display of our dominance in a game with an AI game player. and Altman and the team at OpenAI, they were saying to Musk, well, we do have a way that we think we could beat DeepMind. We've got this strategy, but for us to implement that strategy, we're going to need way more compute power and we're going to need more money.

30:47So they pitch him on this idea that the nonprofit OpenAI could have a for-profit arm and then that way, Altman could go out and do the thing that he's best at, right? He could go get investor money that they can use to up their compute power. They start discussing, how are we going to convert the nonprofit into a for-profit? And all of a sudden, Musk and Altman start butting heads because when they go to form the for-profit, the question becomes, who will be the official CEO of the for-profit? And both of them want to be the CEO, and they cannot agree. And Musk comes back and basically says, no way, I gave you all this money to start a nonprofit.

31:28If you're going to turn that nonprofit into a for-profit, then I should be the head of that. He even thought about folding it into Tesla and just making it an arm of a for-profit company he was already running. He wanted to be CEO and have controlling voting power. So Elon says if it's going to be for-profit, then he's going to be in charge. Yes, or he's going to walk. And so now OpenAI has a decision to make. lose Elon Musk and his celebrity and his money and his tech prowess or change the structure of their company that they designed specifically not to have one person be the ultimate controller of this technology that they think is going to be so powerful that no one man should wield it.

32:11Right? Right. One OpenAI co-founder, Ilya Seskover, he wrote to Elon in this email and he says, quote, the goal of OpenAI is to make the future good and avoid an AGI dictatorship. You are concerned that Demis could create an AGI dictatorship. So are we. So it is a bad idea to create a structure where you could become a dictator if you choose to. And so what does Elon do? Well, he responds to this email, guys, I've had enough. this is the final straw. And then Musk decides in a huff if this is not going to stay a non-profit and it's converting to a for-profit where I am not in total control, I am leaving.

33:02And not long after, he quits OpenAI. But within a few months, he starts going around on a very different kind of campaign, essentially telling people that he has also quit trying to sound the alarm about AGI and what it's going to give rise to. Four, three, two, one, boom. Thank you. Thanks for doing this, man. Really appreciate it. You're welcome. Very good to meet you. Nice to meet you too. And thanks for not lighting this place on fire. You're welcome. And it's at this time that Elon goes and makes his first appearance on the Joe Rogan experience. This is the infamous episode where Musk smoked pot on camera.

33:45Yes. I mean, it's legal, right? Totally legal. Okay. There's tobacco and marijuana in there. How does that work? Do people get upset at you if you do certain things? And as you remember, this became like a whole big thing. The stock value of electric car manufacturer Tesla tumbled 9 % Friday morning. Billionaire Tesla head Elon Musk. What is he up to? Shares in Tesla took a hit today shortly after video was posted of CEO Elon Musk apparently smoking pot. But of all the spectacle in this moment, there was this other part of the podcast that even I didn't really notice until I went back recently and listened to it.

34:23You scared the shit out of me when you talk about AI. Between you and Sam Harris, I realized like, oh, well, this is a genie that once it's out of the bottle, you're never getting it back in. That's true. Are you honestly legitimately concerned about this? Is like AI one of your main worries in regards to the future? It's less of a worry than it used to be, mostly due to taking more of a fatalistic attitude. Musk basically tells Rogan, hey, I did my best to warn people. I tried to convince people to slow down, slow down AI, to regulate AI. This was futile. I tried for years. This seems like a scene in a movie where the robots are going to fucking take over and you're freaking me out.

35:03Nobody listened. Nobody listened. No one. You met with Obama and just for one reason. Just talk about AI. Yes. I met with Congress. I was at a meeting of all 50 governors and talked about just AI danger. And I talked to everyone I could. No one seemed to realize where this was going.

35:27After a short break, OpenAI, now without Musk, stumbles into a breakthrough that will transform the industry and, yet again, make the people most concerned about building AI safely decide that they need to build this even faster. Stay with us.

35:56Hello, everyone. This is Matt, co-founder here at Longview, where we report stories that are grounded in curiosity and context. not political bias. As we say, it's not the left view, not the right view, but the long view. One way we sustain this business is by advertising, but we are also listener supported. And if you would like to go ad free and support us at any dollar amount that you'd like, you can do that by clicking on the link in our show notes or by going to longviewinvestigations.com. Until then, here is a brief message from our sponsors.

36:37The Last Invention is brought to you by Quince. Every spring, I feel the same urge to clean things out. Closets, habits, routines. And with clothes, I keep coming back to the same idea. Fewer of them, but better quality. That's what I like about Quince. The materials feel elevated, and the cuts are clean, and the prices, they aren't insane. They make everyday staples using premium fabrics like 100 % European linen and this incredibly soft flow knit fabric for their activewear. The kind of gear you end up wearing way beyond the gym. What surprised me most is the pricing. It's about 50 to 60 % less than what you'd expect from comparable brands.

37:15And that's because quince goes straight to the source, working directly with ethical factories and skipping the middlemen. So you're actually paying for the quality itself instead of the markup. My favorite piece these days is one of their blue chore jackets. It fits great. It's really comfortable. and I know it'll hold up even though I wear it all the time. Refresh your wardrobe with Quince. Go to quince.com slash last invention for free shipping and 365 day returns. Now available in Canada too. Again, go to quince.com slash last invention.

38:00skills, certifications, and more. Spend less time searching and more time actually interviewing candidates who check all your boxes. Listeners of this show will get a$75 sponsored job credit at Indeed.com slash podcast. That's Indeed.com slash podcast. Terms and conditions apply. Need a hiring hero? This is a job for Indeed sponsored jobs. Okay, so once Elon Musk walks out, what happens at OpenAI? Well, it turns out that even though this was a nightmare for everyone at OpenAI and they were worried that this might spell the end of the company, it kind of turned out to be a blessing in disguise.

38:36I talked to Andre Karpathy, who worked for both Elon at one point and OpenAI, and he said, you know, in the beginning, OpenAI was trying to copy-paste DeepMind. And in the end, it turned out that DeepMind had to copy-paste OpenAI.

38:55As Keech Hagee was saying it to me, pretty much the whole time that Elon was at OpenAI, he was pushing them towards this AI game player strategy. And after Elon left in 2018, over in the corner, a completely different researcher had a breakthrough with a completely different technology, a language model. It was only after he was gone that instead they focused eventually on language. And that is how by 2022, they flip everything and have Demis and Google chasing after them instead of the other way around. The next generation of artificial intelligence is here. It's called ChatGPT. But before that would happen, there was another split inside this company.

39:45And it's a split that some people in Silicon Valley think may end up being far more consequential even than Elon Musk's. And this is the paradox of Dario Amadei. Who is Dario Amadei? Why does he end up at OpenAI? And what is it that he really contributes to the team there? Dario is, like Demis actually, has a neuroscience background, which is one interesting thing. He is also very interested in the brain, which sort of ends up informing a lot of his theories for AI systems and how they should work. What he was really known for at OpenAI was his emphasis on safety. Jasmine Sun and Kevin Roos, they're both working on a book right now about AI, and Dario is one of its central characters.

40:30He is, by his own admission, kind of a nervous person. And he was really a pioneer not just in developing AI systems, but in worrying about them and how they might go wrong. He does end up being extremely concerned about AI risk and the potential for systems much smarter than us to develop their own goals, become unaligned with, opposed to, or just sort of not caring about human goals, and then to sort of end up taking over, screwing humans up. So Dario was one of the people who came to open AI as someone whose motivation was more about stopping a dangerous AI. Yeah, he's very worried about superintelligence.

41:12He's associated with this group called the Effect of Altruists. And he says that he comes to open AI in large part because of this altruistic, safety-focused mission that they have. And how do you sum up what it means for someone like Dario to study AI safety? Like, what exactly is AI safety? So AI safety is a big field. It contains a bunch of different subfields. One of them that Dario and his colleagues have been very instrumental in is called mechanistic interpretability. That's a very long name. I've told them they should rebrand it to something people can actually pronounce if they listen to me.

41:51But basically, mechanistic interpretability is the science of figuring out how AI models make decisions, why they behave like they do, what is going on inside the guts of the system. So he's studying the mysterious AI black box. Yes, making that interpretable to humans. Okay, so he's trying to probe the AI to figure out why it's doing what it's doing, and is it going to do anything we don't want it to do? Yes, that's one part of AI safety. There's other parts of it too. The other thing that Dario is really into is size. Early on in his career, Dario Amadei had worked on a project at Baidu, the Chinese internet conglomerate, that dealt with these so-called scaling laws.

42:40And this was a theory at that time. It was an unproven theory that basically the key to making an AI system more intelligent was just making it bigger and training it on more data. This was sort of countercultural in AI research at the time. Lots of people were theorizing that you needed some clever new algorithm or some very different architecture to make these models smarter. But Dario and his colleagues sort of had this idea that you could actually just make them bigger and the systems would get smarter. And so the idea here is just that if we take an already promising neural network AI system and we just make it bigger, that maybe like the human brain, which is bigger than the bird brain or the cat brain, is smarter.

43:31This thing will also get smarter and smarter and maybe one day will even become a general intelligence. Yes. And essentially, this is sort of his best guess at how companies like OpenAI are going to get more intelligent systems. It's not by training them on, you know, more specialized data. It's not by coming up with clever efficiency hacks. They are just going to make the models bigger, and that is going to take care of a lot of the problems. So his theory is that if you just take a promising AI model for OpenAI that became their language model, that is a neural net that looks for patterns in text and language, then what you just need is a massive amount of text and data to pump into it, as well as a massive amount of GPU computer chips, which are, of course, very expensive.

44:30And this is why they're pretty sad to lose Elon Musk and his money. Yes, this is one of the reasons that it was sad to see Elon walk out the door. But not long after he does, Sam Altman goes out. He does the thing that he's so good at. He knows that they need lots of money, lots of GPUs. So he goes out and strikes up a partnership with one of the biggest companies of all time, Microsoft. So Microsoft ends up fulfilling both of these things. They become the largest investor in OpenAI, and they partner to build the supercomputers that OpenAI needs. Greg, this is a detail of the story that's especially wild to me.

45:08When Dario and his team at OpenAI, when they get access to these Microsoft supercomputers, they decide to take their scaling theory as far as they possibly can. Dario Amadei was really pushing for the idea of, no, we really go big or go home. So, for example, at DeepMind, when they did that Atari demo, they were using just one GPU. For the richest universities in the world, like MIT, Stanford, it would be a big deal to have a few dozen chips. And in places like India, you would have grad students, multiple grad students sharing one computer chip. So they're trying to do their research on fractional amounts of GPUs.

45:53But with this Microsoft partnership, OpenAI now has access to these supercomputers with thousands and thousands of GPUs. And so as the story goes, Dario approaches the leadership team at OpenAI and says, guys, what if next time, what if we take this model and we crank it up to 10 ,000 GPUs? So 10 ,000 computer chips. I mean, no one had ever thought about that before. Like, that's bananas. us. Actually, like within OpenAI, this was a contentious decision because some people were like, is that even possible? That just seems improbable. Other people were like, no one has ever done this before. And if we think that AI could go badly, maybe we should more gradually scale it, not just do this dramatic step change.

46:51But his philosophy was we need to accelerate the development of this technology so that we can then retain hold of it and figure out how to perfect it in the lead time that we have over other potentially bad actors getting a hold of it. And Sam Altman also really liked the idea because his entire career has been adding zeros to things. So he was like, let's do it. And Ilya Sutskever also philosophically was always more in the camp of scaling will potentially bring wondrous and potentially terrifying things, but we should not be afraid to go in that direction. And so the main people that were running OpenAI all converged on, yeah, let's give it a go.

47:37However, if you are going to massively scale up your GPUs, your compute, you also have to massively scale up the data that it is searching for patterns inside of. Because just think of it like, no matter how smart and powerful it is, if it only has access to a limited amount of data, it's never going to truly become like an artificial general intelligence. Or it's almost like if Einstein, as smart as he was, if he'd only ever read one book. He would know that book really well. Right. But he wouldn't be that smart. Yes. And so they're going to need a massive amount of data to match the massive amount of compute.

48:16But here's the problem. There's only so many open source free databases on the internet. And so here's where OpenAI does something that right now has a lot of people, a lot of different corporations suing them, including the New York Times, because it appears, and some former OpenAI employees have leaked some details about this, that they just started dumping big chunks of the internet into their AI. I believe this is going to be the pinnacle scene in the inevitable Hollywood depiction of this story one day where they're just ramping up all the stuff that they're throwing into their system. Like, oh, here's a free database.

49:01Let's put that in. Oh, look, let's put some Reddit in there. Yeah. And then, hey, how about Wikipedia? And then, hey, man, these researchers over at the University of Toronto just scraped all these books off the internet. I'm sure they wouldn't care if we just took all them copyrighted books and fed that to the LLM. No problem, right? What else we got? Yeah, that's pretty much what happened. Allegedly, they started throwing in scientific journals, news articles, blogs, transcripts from YouTube videos. They just kept going and going.

49:31And how far does this go? Do they feed it the whole internet? Like I said, there is a lawsuit that's happening right now. So we're going to learn more details, I think, as information comes out from those suits. But it's been reported that basically if a website or if some text online didn't explicitly have a label on it saying, do not use this to train your AI, they adopted a stance of better to ask for forgiveness instead of permission. So all we know is that they just dumped a lot of the internet into the system. We don't know how much or what exactly they put in there. Yes. And we know that this is eventually the strategy that would give birth to what we now call chat GPT.

50:19And so Dario is both the guy who is saying, let's scale this thing up further and faster than anyone has before. Let's crank this up to 10 ,000 GPUs. But he's also the safety guy. isn't there a tension between those two between let's crank the knob up to 11 and oh we really need to make sure this is safe yeah this is sort of the classic paradox of Dario Amadei and of AI safety in general is that they on one hand fear the effects and implications of these very large very powerful models and they're trying to build them and stay on the cutting edge of AI capabilities. And I've asked Dario about this before.

51:05And he says, in order to be able to study the safety challenges of very powerful AI systems, you have to have very powerful AI systems to use as your testing grounds. You can't sort of learn about safety on a Formula One car by practicing on like a jalopy of a 10-year-old Honda Civic. It just won't teach you that much about what kinds of risks are going to take place when AI is very powerful. And so the argument here is that to make a powerful AI that is safe, that is good for humanity, you are going to need to learn about powerful AI systems and test powerful AI systems. And so therefore, you're going to have to make one.

51:49Yes. If you want to do cutting-edge AI safety research on very powerful systems, you need to actually build those powerful systems. I think the other thing that Dario Amadei would say is like whichever system is the best, it's going to be embedded in every part of society. Like we're going to use it to make decisions about who to give a loan to. We're going to use it to plan our cities. We're going to use the super intelligence to maybe even like figure out our military strategy. And so like the only way to have the impact that we want to have in the world to ensure that we have super intelligence and that the super intelligence does things like curing cancer instead of like screwing us all over and self-sabotaging ourselves is by having both the safest and the best model.

52:35Because if ours is safest, but it's not actually a very good model, then the unsafe model is going to be the one that's going to be widely deployed, and that's a much worse world. And the last one I've heard him make is the only way to stop a bad guy with a powerful AI is a good guy with a powerful AI. Essentially, this is the argument that this technology, it's so powerful and so lucrative that someone is going to build it. And in Dario's mind, that someone could be an authoritarian government. It could be a rival AI company that doesn't care as much about safety. It could be a terrorist group.

53:14And so the ethical thing to do in his mind, if you are concerned about the power of these AI systems, is for you to be the one who builds it and keeps it safe and kind of sets the high bar of safety that the rest of the industry will have to follow. And it was this mindset, this idea that to make AGI safe, you need to make it fast. This is part of what drew Dario to open AI in the first place. This is a part of the mission that he loved. However, one day, late in 2020, he just up and quit, along with several members of the AI safety team. And they went out and pretty much immediately started a rival AI company called Anthropic.

54:00All right, so Kevin Dario Amadei has yet to accept my interview request, although the people that he works with are very nice, and I had a nice meeting with him, and maybe one day he will come on the show. But in the meantime, I know you've spoken to him. What do you understand is the reason that he leaves OpenAI? What does he see there? What is it he doesn't like? So the official story of why Dario and his colleagues left OpenAI is that they had philosophical differences about AI safety approaches and priorities that Dario and his team wanted the company to put more emphasis on safety and that others at the company were less interested in that.

54:45I think the real story is more complicated and involves a lot of not only philosophical differences, but also real personal differences and beefs. Lots of disagreements I've heard about in reporting about specific decisions they were making, whether they were taking safety seriously enough, whether they were becoming too commercial. You have to remember that when Dario joined OpenAI, it was a research nonprofit. It was specifically set up not to be and act like a normal AI company. And by the time he left, it had started this for-profit subsidiary. It had struck this deal with Microsoft. It was starting to look more and more like a kind of normal tech startup.

55:30And I think that made him and his colleagues very uncomfortable. And there are some other juicier stories that I'm going to save for my book. so we don't know what dario saw that scared him no we don't know i mean maybe kevin knows something and we'll just have to wait for his book all we know is that he quits he says that it's connected in some way with ai safety and that he opens a competitor claiming that now he's going to be the one to make AI truly safe. And so now we have more competitors in the race. Yes. And this pushes everyone to work even faster. And really where you see that most dramatically is around chat GPT, because OpenAI had already decided that eventually they wanted to release a version of chat GPT to the public.

56:26But they didn't think it was quite ready. It had gone from GPT-3 to GPT-3.5, but it was still buggy. It still regularly had these hallucinations that they didn't understand. So they were trying to hit their benchmark of GPT-4 before going public with their chatbot. But suddenly this rumor starts to spread within the company that Anthropic also has a chatbot and they might release it soon.

56:57Allegedly, this rumor starts to go around OpenAI that Dario, Amadei, and Anthropic are planning to release their own chatbot and to do it before OpenAI can. And so OpenAI executives make a decision. We are not going to wait for the GPT-4 launch because the model's just not ready. But we have the chat interface and we have GPT 3.5. And so they're nervous that if Anthropic beats them to market with their chatbot, then OpenAI is going to seem like they're behind the ball. They're going to come off like a copycat. They're operating under, you know, a very Silicon Valley belief of winner takes most.

57:44So you need to be the number one. You need to be the one that has the name recognition, the one who invented this kind of chatbot. And so they just decide, you know, let's do a low-key, you know, no press release, no advertising, no social media blitz release of ChatGPT 3.5. And Keech Hagee and Karen Howe, they were telling me that supposedly back at OpenAI, the team did not think that this was going to be a very big deal outside of Silicon Valley. Like they didn't think it was going to make a very big public splash. And so why release it if they didn't think it was going to be a hit? Well, in some ways it was like insider signaling to just say to the world of technology, we were here first.

58:30It doesn't matter if the public uses it or not. It mattered that the world of technology doesn't think that they are just copying off of their rival's anthropic. Inside the company, it was a low-key research preview is how they described it. let's just release this model with this new interface that is just like a chatbot and see what people think. The night before, they were like making bets on how many people would actually start using the model. And I think the first weekend and the highest bet was 100 ,000. So that's how many users they provisioned their servers for. And so on November 30th, Sam Altman goes on to Twitter and he just writes, Today we launched ChatGPT.

59:17Try talking with it here. And he pastes a link. The next generation of artificial intelligence is here. The future is now. The internet's going crazy over new artificial intelligence called ChatGPT. A new artificial intelligence chatbot. ChatGPT is like a Google you can ask to do things. They can answer essay questions, write songs. Who knows what companies and ethical issues. It already has more than a million users. Very creepy. A new artificial intelligence has gone viral. Probably the area of... And I caught the eye of Wall Street this week. Or questioning whether chat GPT is viral. ...be the threat to some established big companies.

1:00:15Next time on The Last Invention.

1:00:43born out of you guys' determination to chase after this idea, despite all that they're saying, how did that feel? Like, I imagine it felt really good. Oh, yeah. I mean, it was great. It was... Let me share something emotional. So, shortly after AlphaGo, I don't know, maybe 2018 or something. Oh, I guess that's when I got the Turing Award with Jeff and Young. I thought, I've achieved the greatest prize that a computer scientist can expect in their life. And I've accomplished so much. And, you know, my career has been so rewarding and successful. What else is there to do? I felt like if I die tomorrow, I'll go with, you know, serenity.

1:01:37You did it. But, wait, but there's a but. November 22. Chat GPT. It dawned on me, yes, but like, look, this has been a really big step. How far are we from human level? Maybe just a few years, maybe a decade, maybe two. And then what? Like, what's going to happen? With this kind of technology, aren't we going to build machines that we don't control and could potentially destroy us? How do we make sure this doesn't happen? And I didn't have an answer.

1:02:33The Last Invention is produced by Longview, home to the curious and open-minded. We are an independent outlet focused on giving people the backstory to the debates shaping our future. To support our work, click on the link in our show notes or visit us at longviewinvestigations.com and become a subscriber. And as always, it really helps us if you leave a rating and a review on Apple or Spotify or wherever you listen to your podcasts. One last thing to mention, audio from the documentary AlphaGo was used in this episode. Thank you for listening, and we'll see you soon.

1:03:35gives restaurants the tools and tech they need to help them perform under pressure. Sounds pretty good, right? Toast, built for busy. Uncovered windows can make your home feel up to 20 degrees hotter. Stay cool and save up to 50 % off custom window treatments during the 4th of July mega sale at blinds.com. From outdoor shades to room darkening blinds, finding the perfect fit is easy. Get free samples, expert design help, and professional measure and install services, or DIY with confidence and support every step of the way. Shop up to 50 % off site-wide, plus huge savings on doorbusters, right now during the 4th of July mega sale at blinds.com.