2 Ex-AI CEOs Debate the Future of AI w/ Emad Mostaque & Nat Friedman | EP #98

0:00I'm NFL Lineback at Tijayawatt and this is my personal best. YPB by Abercrombie is the active wearer I'm always wearing. That's why I reached out to co -design their latest drop. I work with designers to create high -performance active wear that holds up to my toughest workouts. Shop YPB by Abercrombie in -store online and in the app. Because your personal best is greater than anything.

0:31When did making plans get this complicated? It's time to streamline with WhatsApp. The secure messaging app that brings the whole group together, use polls to settle dinner plans. Send event invites and pin messages so no one forgets mom's 60th and never miss a meme or milestone. All protected with end -to -end encryption. It's time for WhatsApp. Message privately with everyone. Learn more at WhatsApp .com. What I think happened in the last year really is the starting gun was fired. We spent a trillion dollars on 5G. Is AI more impactful than 5G, of course it is. Do we actually know what's going on inside the models?

1:13We can measure everything and still we don't really quite understand mechanically what's happening inside them. They're not engineered and designed. They're kind of grown. AI is clearly more capable than humans now. We know this is going to get better. When does it slow down? Right now it's a few companies doing this, but we need standards around it, and we need the expansion. And again, every country will invest in this, every company will invest in this. We're here to talk about what happened. I'm always to say WTF just happened in the past year, because a lot has happened. You were on stage with me a year ago, and things were amazing then.

1:51But a lot has happened, so I want to split this into a couple of parts. It's like what just happened the last year and then I want to talk about how far forward can you see what you expect is going to happen? You know, I'd love to talk about the companies, the models, the dramas, what did Ilya see? I think that's going to come of me. Matt, do you want to kick us off? What's the big things that happened in the last calendar year in your mind? Well, yeah, it's been kind of an amazing year. I do think for people who just started paying attention to this sort of new AI and deep learning revolution in November of 22 when ChatGPT came out, things probably seemed very fast because we had stable diffusion come out over the summer.

2:43Previously in 22, we had Chatch -TPT in November and then just a few months later, GPT -4 came out sort of demonstrated a new level of capabilities that were shockingly improved over what was there previously. And so I do think there's a set of people who sort of extrapolated from those three data points in terms of the progress that was going to happen from there. And it's incredible actually how quickly people can adapt to these new things. And there was a point at which kind of chat GPPT blew everyone's minds. And then a few months later, it was sort of just something we accepted as part of the world, as part of reality.

3:22And we can very quickly enter this kind of slump, where's the new model, GPPT3 is boring now. When do we get GPPT5? Come on, Sam. Put it out already. What are you waiting for? This kind of thing. So I think we had a little bit of that this year. And then over the course of the year, though, what we started to see, and I'd been waiting this for a while, for this for a while, but we started to see real adoption taking off. And so, you know, back in 2020, GPT -3 had come out in 21 when I was running GitHub. We put out GitHub co -pilot. And it was, I guess, one of the first LOM commercial products.

4:00And I thought that immediately after that there would be this rapid wave of commercialization of large language models because developers would have seen how capable they were, what they could do, and start building products with them. But it really took this kind of chat GPT moment before entrepreneurs, developers, and product people started to do this. And so what I think happened in the last year really is the starting gun was fired on the actual exploitation of these raw capabilities, the building of products, the working of them into companies and organizations. And now that's what we see, the adoption's been incredible.

4:35Chat Gbt is rumored to have gone from basically zero to $2 billion in revenue in just about a year. That's amazing. There's organizations like Stability that put out models that have hundreds of millions of users. You have mid -journey, you have now Google in the game. Finally, with Gemini, I thought it was exciting to see them not only to release Gemini, very quickly follow up with Gemini 15. And so there's a way in which Google has clearly internalized the piece here and the need to iterate and ship. And so that... I don't think a lot of people realize how far ahead Google was for a decade. Yeah.

5:15So Google had really developed a lot of the early tech here and said it's not ready for release. right, they were being responsible to a great degree and then how do you not release after Chacha Petigos gets released? They're shipping tiled now, they're going to ship. They're going to ship, yes. Imaud, the open model story that you told us last year has really blossomed and in fact it's become sort of an ethos in the organization, right? It's like, Elon is like really twisting the knife in with Sam. And then says, but, you know, Grak's going to be open now. Can you talk about what's happened in the last year in open?

5:58Will open win? Do you think? And I'm curious for both of you guys. And describe what open is versus closed for the audience here. Yeah. So proprietary AI is, you don't see the code, the way it's anything, and it's provided it as a service, whereas open models and code are ones that you can adapt to yourself, you can take, bring to your own data and you own. And that's important because these models are something a bit different, they're like extensions of our mind. And so the best analogy I've said is that these are actually like graduates. So they can code, they can write, they can sing, they sometimes go a bit crazy when they try a bit too hard, you know, or give them better education.

6:31And so open models like graduates that you hire and then proprietary models like consultants you bring in. Interesting. And in the early stages, we all needed consultants. Now people are saying, well, I want this for my own day, so I want this for that. And Open allows for innovation to happen. So we've had now 330 million downloads of our models on Hugging Face. Wow. So we just don't know. And what is Hugging Face for folks? It's GitHub for AI models. So it's where you go as a developer to download the models. And so millions and millions of developers are using this technology. But then you look at the language models side, people are taking it, adapting it, and optimizing it to have innovations that are now catching up with even through a prior tree guys.

7:09But they're complimentary. You will have your own team and then you'll bring in the specialists. And I think that's the best way to think about this. Because you can't outsource your internal intelligence from a personal company with a country level to models that you don't know what the provenants are. And this is one of the debates that we saw over the last year. The first step was like, oh my god, exponential extrapolation. You know, as Nat said kind of actually the foundations for this were dug maybe a decade ago. And now we've been filling in the cement and now we're all building houses and we're like, well, the houses have become skyscrapers.

7:40So it was like, well, they could kill us all and those are valid things to discuss. Then it became about sovereignty and, well, GPT -4 is amazing, but I want my own version as well for my own private data. And so I think as we advance, proprietary models will always beat open models. They can always be open models. Open models. I said, because open models like generalized graduates, as it were. But both of them need to exist. I think we'll see both of them continue to take off. Do we actually know what's going on inside the models? I was listening to a conversation with a chief scientist at Anthropic who will be on stage with us next year.

8:21And he's saying, we actually don't understand really how these models work. Is that a fair statement? Yeah, we have. I mean, just so you understand, all right, it's like they're amazing, but we actually don't understand how the weights and connections and all are really working. I mean, we understand it at this very micro level. You know, we can see the multiplications happening and we can see the signal moving to the next layer in the neural network. But it's actually sort of similar, although not the same mechanisms, to the way we don't have a perfect understanding, the way thinking happens in our brains.

8:58We don't, even though in the case of our brains, when we try to do the neuroscience and understand what's happening at the neuron level and the organelle level, we're limited by our measurement, right? We can't measure the state of every single neuron in your brain while you're thinking. That's not something we have the sensing technology to do now. That's not a problem we have with these AI minds that we're building. We can measure everything. And still, we don't really quite understand mechanically what's happening inside them. They're not engineered and designed. They're kind of grown. You know, they're the products of us actually because we have the internet which digitized the world and put the sort of all the data we've produced online.

9:40And there's a way in which that process of building the internet was like a bootstrapping process for building AI because it was a precondition to making AI. We digitized the world, we put all the contents online and then we could use all that to sort of grow and train these AI models, but we don't quite know how they work. Now there's a new field, new, not in time, but relatively small field called interpretability, which is about trying to understand what's going on in these things. What are they doing? How do they make the decisions that they make? Obviously, there's benefits if you can do that to make them better in all sorts of ways.

10:18You can make them smarter, maybe, and make better decisions. And you can also hope to make sure they do the thing you want them to do, and not something else. But it's new. Elon put a tweet out, which I read during my opening remarks that said, by 20, yeah, we're going to have human -level AI next year, and by 2029, AI will be equivalent to all 8 billion people. Do you agree with that, E -Mod? You think it's going to move that fast? I think we had a big discussion about generalized AI that can do everything, and that was the focus of deep -mind and open AI. But for specific tasks, AI is clearly more capable than humans now.

10:56So, for example, Google's Gemini Ultra Model has a million, 10 million token context window. What does that mean? It means that someone can upload themselves debugging a piece of code and the code base and it will correct it. No human could ever do that. You know, Ram isn't big enough. On image now, we can generate images faster than anything, songs and seconds and other things. Let's talk about data second. Sora was pretty like holy shit moment, right? Someone is doing it right at OpenAI with chat GPT and then and Sora. But I remember last year, you told us it used to take like 30 seconds to generate on stability a single image, and then it was down to a second.

11:42And you've advanced the technology orders and magnitude since then. Yeah, so I think if we can put the thing up. Put up the slide, stable image. Yeah. And then if we actually get to the next slide, talk about speed here. Oh, should I click this? Let's see. Oh, we can get this to give an idea of where it's gone. So image was kind of one of the things that kicked this off in 2022. All of these images Generated on a MacBook on a MacBook just from description and now we've perfected text and other things But the next step after you can generate all these beautiful images is that you want to move to control So the models are just the first step you have to have chat gpt and other things to make them useful But then you want to be able to take that guy and say upscale and that's all we said and upscale him You want to replace the line with a cat and we can now do that real time You know or tie it with a unicorn if I was back around with a forest I love you forest so all of this you can do pretty much real time because if you look at the next slide from this Okay, oh, I said it was 20 seconds to one second.

12:44This is live real time You can just type and it automatically adjust the cat it gives them a hat But then if you look at the optimization of that with the next slide Back when we missed it. Oh, okay, well, we missed it. We'll get back on slide So with this process here, we're just releasing a new distillation model today. We've got it to 200 cats with hats per second 200 cats with hats per second. So that's your speed of image generation That's the new unit of image generation 50 milliseconds Yeah, exactly it's because there's not enough cats on the internet, right? So we're gonna add more cats to the internet.

13:18Oh five milliseconds five milliseconds Yeah, but I think with the new chips that are coming and videos gonna announce another one will get up towards the thousand images per second. So that's a real live video. It's times 30. Yeah, and so OpenAI's innovation, there were two major things. There was a transformer architecture, the language models, and diffusion that drive the media models. They combine the two together. So our new image model, stable diffusion 3, which the best performing image model combines those together as well. So if you go forward a few slides from here, it's OK. As skip, another one.

13:51Another one. That's the upscaling. So you're basically said upscale this image on the left and you get image on the right. We'll have that real time in a couple of years. So your boxy video games will look a lot more realistic. So just literally as you can take an old game and play it in surreal life. Yes, but if you go to the next slide, these videos with our video model all generated on a consumer graphics card with five -year -gabytes of VRAM. So I mean the point is it's this is in everybody's hands. It's in everyone's hands and again we haven't even optimized the data so in an hour Next slide We're releasing the world's most advanced 3d model so all of these are just generated just from descriptions and So the fastest version of this does in one second so you can generate that dragon But then the next step is you'll be able to control every element make its ones bigger make these adjustments So how long would that take for a normal creative person in kind of industry, a huge amount of time, but now it works on the edge, and it works even faster in the cloud.

14:51So a future here where you can be describing the video game you want created, and it's generating all the characters and generating the play. Yeah, and so if you go back a few slides to that thing with all the nodes, I think this is an important thing. Sorry, I didn't do the slides properly. I think one of the things that we're having right now is this is a system we built called comfy UI that's used by just about everyone now. So you take the face, the pose, the dress, and then you have that output. But if I share that image with you, it reconstructs the entire flow. Because last year was the year of creation, models that create, then chat GPT comfy UI allowed you to control and compose them.

15:31And the next bit is bringing that all together because chat GPT, all the knowledge that you build from writing your speech, you You don't have files anymore. You have flows with these models and assets there. I think that's the next step, because when you go beyond just spitting out ideas to be able to control them like that, that's a huge deal. Amazing. And you're announcing this in an hour. The 3D model's releasing in an hour. Open source to everyone. Give it up for E -mod here.

16:01You really have, I don't know if you're to say this, I mean, you're driving revenue and soon that profitability, what's your financial? I can't say that publicly. You can't. Okay. It's going well. We're ahead of full costs, as it will say. All right. He told me backstage it was good. Yeah. Okay. I didn't know it. Yeah. Everybody wants to take a short break from our episode to talk about a company that's very important to me and could actually save your life or the life of someone that you love. Company is called Fountain Life and it's company I started years ago with Tony Robbins and a group of very talented physicians.

16:39You know most of us don't actually know what's going on inside our body. We're all optimists until that day where you have a pain in your side you go to the physician and they burn into your room and they say listen I'm sorry to tell you this but you have this stage three or four going on and you know it didn't start that morning. It probably was a problem that's been going on for some time. But because we never look, we don't find out. So what we built at Fountain Life was the world's most advanced diagnostic centers. We have four across the US today and we're building 20 around the world. These centers give you a full -body MRI, a brain, a brain vasculature, an AI -nabled coronary CT looking for soft plaque, dexascan, a grail blood cancer test, a full executive blood workup, it's the most advanced workup you'll ever receive.

17:32150 gigabytes of data that then go to our AIs and our physicians to find any disease at the very beginning when it's solvable. You're gonna find out eventually. Mice will find out when you can take action. Found life also has an entire side of therapeutics. We look around the world for the most advanced therapeutics that can add 10, 20 healthy years to your life and we provide them to you at our centers. So if this is of interest to you, please go and check it out. Go to FountainLife .com -peter. When Tony and I wrote our New York Times bestseller Life Force, we had 30 ,000 people who reached out to us for Fountain Life memberships.

18:15If you go to FountainLife .com -peter will put you to the top of the list. It really is something that is For me, one of the most important things I offer my entire family, the CEOs of my companies, my friends, it's a chance to really add decades onto our healthy life spans. Go to felonlife .com, backslash, Peter. It's one of the most important things I can offer to you as one of my listeners. Alright, let's go back to our episode. I am curious about the idea of how far are we from having an AI that I can have a conversation with and say I'd really like to create a new business that does this and this and just to have a brainstorm partner AI and it will do the incorporation, it will write the code, it will generate the marketing materials and it will be able to, you know, we're all entrepreneurs here that's all we, you know, find a problem, build a business, find a problem, build a business.

19:18How far are we from that reality of an AI created? Well, I'll come back the second part in a minute, but an AI that can be your thought partner and really step up a company. I think it's already happening gradually, and then maybe suddenly. So, we have, I think companies will, these models are neural, they're neural networks. And so I think the right way to think about it is that companies in the future will just be increasingly and there will be more and more of the company and what used to be the departments of the company that are single models or swarms of models that are doing work. I think it will, you know, at some point probably very soon we'll look back on 2024 and say, God, do you remember when companies have like hundreds of people in the finance department doing accounts payable and just like transforming information from one form of text to another, essentially in doing this coordination And so I think you will have some form of both existing companies that just adopt more and more AI because it gives them advantages, it makes them more efficient.

20:22Maybe it gives them better customer service, people enjoy the responsiveness, intelligence, politeness, clarity, etc. of the AI counterparty. And you'll also have new companies. They'll be AI native, started from scratch, neural from the beginning. So, some people may remember when Instagram was acquired by Facebook for a billion dollars, everyone was just marveling at the fact that a small company... 13 employees. 13 people could be worth a billion dollars. How is that possible? And the joke was, you know, gosh, could you ever get down to a one person company that's worth a billion dollars? And while like this is clearly going to happen, maybe eventually a zero person company.

21:03So, we talked about this getting ready. And I said, when are we going to have a scenario where a government, and we have some governance in the room here, says, we're going to create a new regulation that allows an AI to incorporate on its own in our jurisdiction. I think it's a winning scenario because all of a sudden, the AI incorporates, you get the tax base, and every AI -incorporate company will move there. I think Wyoming's done that. What's that? Why I'm being done that they have a new structure called a duna for a decentralized autonomous organization. I'm sure who has why I'm Wyoming White old places.

21:41Yeah, we have some Wyoming fans here in the audience. Yeah, so technically that could happen today

21:49Amazing, I mean I do think that that is a so the speed of wealth creation How should so let's talk about what are the wow moments that might be possible in the next year? What are some crazy well -mongering? You might see. I think one of the most important things is the accuracy and then these long context windows. So explain with the long context windows again. So you're absorbing information right now and it's quite high definition because you can see everything here and other stuff. You're still writing it down. And the reason our organizations get big is because text is a lossy transmission format.

22:26We lose so much context. The final PDF loses all of that stuff. Now with the new Google models, that has some amazing companies in this space as well. You can upload hundreds of thousands of words, thousands of documents, and the AI can interpret them all at once. There doesn't need to be trained on it. So you can upload all of your ideas and say, build a business based on this, and it will do that. Or you can upload like a whole bunch of movies and then tell it to write a script that encrypt all of that, and it will do that in front of time. But again, there's something superhuman. But we all have these massive repositories of all these ideas we've had.

23:03Being able to dump that now, and then the AI, without having to be trained, spit back, answers ideas, and things like that, I think is a really huge step along with that composition step that kind of discussed before. I think, yeah, I totally agree with that. I think there will be a couple things coming probably pretty soon. It's hard to predict exact timelines on these things. sometimes things happen faster than you expect and sometimes a little slower. But one of the clearly amazing, clearly possible now products to build is a voice to voice model that's indistinguishable from talking to a human, maybe for a conversation of up to a couple minutes.

23:39What happens after a couple of minutes? Well, maybe you can kind of just tell somehow that it's not quite human after a couple of minutes. I'm setting a milestone that I think is achievable this year. maybe if you can do two minutes, you can do 12 minutes, I don't know. But yeah, you know, it's actually about all the technologies there, it all just has to be integrated. And so you need the sort of the ability to recognize speech is there, the ability to interpret it with language model and generate responses is there, and then the ability to turn that text into incredibly realistic voices there, and kind of putting that all together into a package that has very low latency, see that's talking the way we talk, where you can kind of interrupt me, and maybe there's an avatar that's giving you this human, like, I mean, Aristotle was very impressive, but I knew that that was not a real person.

24:27And so I think we could, yeah, we could - You was a real person. Yeah, that's right. And then I think the other thing that's a very big deal is this idea of autonomy and agents. There's been a lot of talk of it with AI over the last year. Today these things are not agents. There are tools. They're calling response. You go to chat, you type something, you hit enter, you watch the response kind of stream back. And I think what people don't necessarily understand is that when these language models are responding to you, it's almost like a rap battle. They have a fixed amount of time to generate each word.

25:00And so they can't sort of sit there and ponder for a minute, you know, what they're going to say. They have to talk to a metronome. And so that's why when you see the words that they're writing out, that's not like they've thought about it a lot and then you see the words, it's actually the thinking is happening during the output. And if you say that's how a human does it too? I often do. I mean a lot of time. I often do. Yeah. Yeah. Really pissed off at what came out of my mouth because I didn't think about it in the back. Yeah, I think so. I mean, sometimes you have a conversation with a smart friend and you just forming the words to respond, you have a better idea.

25:34And so I think we definitely do that too. But the agent, the autonomy is about kind of increasing the unit of work, you can trust the AI to do without making a mistake. So right now you can ask it one question, get a response, you interpret it, you figure out the next thing. But what if it could go do 10 or 100 steps you're talking about an AI business, do you trust it to come up with the title of the blog posts that you're gonna post? Or do you trust it to actually come up with the whole idea of the marketing campaign, right? Come up with a strategy for how to execute it. All of the content it's gonna generate, the partners it will reach out to and negotiate advertising or whatever it's going to do.

26:13Have those conversations to multi -step successfully all the way to like measuring its results without supervision. And I think that agency thing, we're starting to see it in programming. There was a very impressive demo a week or two ago of a company called Cognition that I think it was maybe the arguably the first really impressive demo of working agents in AI and they did it with GPT -4. So not with a new brand new model, they did it by being very clever about the way they squeeze and distill the intelligence out of GPT -4 by repeatedly calling it and analyzing and evaluating its results and choosing the best ones.