Top Minds in AI Explain What’s Coming After GPT-4o | EP #130

0:00Welcome to another conversation on AI. I don't think we get enough of this conversation going. I do want to thank Richard Atias and the FII team for really increasing the conversation this year on AI because I think there is no greater topic of import on the financial side, on the leadership side, education side, medical side, it's transforming everything. We have three incredible CEOs here who are representing a variety of different parts of the AI emergence. I'm going to start by asking each of them to take just one minute, introduce themselves and what they're doing. And they're going to jump into where is this going, how fast is it going, how big is it going to get?

0:50We'll ask the question, what is after chat GPT? Prem, just be going yourself. Awesome. Thank you. Thank you, Peter. I'm Prey Makaraju. I'm the CEO of Stability AI. We are one of the leading open source image, video, and 3D models in the world and past GPT. Pictures are worth a thousand words and we're making quite a few of them. And in fact, 80 % of all the images that were generated by AI last year in 2023 were driven by our models, stable diffusion. Amazing. Richard, hi everyone. Really excited to be here. My name is Richard Socher and the CON Founder of U .com, yoU .com. It's a productivity engine, which is the next generation after a search and an answer engine.

1:38So we really make people more productive across a whole host of different kinds of organizations from hedge funds to universities to companies, insurance companies, and so on. Publishers, news agencies, and almost everyone else in between us, sales, service, marketing, research, analysis, and so on. I also run an adventure fund called AIX Ventures that invests in early stage preseed seed AI companies and startups. I've been very fortunate that when I was a professor at Stanford, I had two students who created a cute company called Hugging Face, and that's a net of five million valuation worth one and a half billion now.

2:13So, fun to do. That's bragging. That's just great. I wish I could brag like that. Dr. Kaipu Lee. I've been working on AI for about 43 years. I was two at a... no, in college when I started AI. And I think that may have started before my colleagues were born. But I actually worked on machine learning. And I have a PhD, Carnegie Mellon. And I have worked at Apple Microsoft Google. Some of you may know me as with my books, AI Superpowers in AI2041. My part Our time job is I run innovation ventures, which invests globally. And then my full time job is I run 01 .AI. It's a generative AI company. We build a large language model.

3:06We're currently ranked as the third company with the highest performance only next to the best models from OpenAI and Google. And you can find it online. We're also building consumer and enterprise products. where based in China, but our products are accessible globally, and also we extensively do open source as well. So incredible, and first of all, Kaifu is a legend and one of the greatest leaders globally in this field, so very honored to have him on here. Prem, let's start with you. You very famously were able to recruit James Cameron onto your board. And since stability is creating video and creating sort of the future of Hollywood, I am curious about two things.

3:58One, did Jim get it right with the Terminator? And secondly, there's been a lot of conversation about the disruption of Hollywood that we're going to have AI's creating the future of all movies, all content and so forth. So you said beyond GPT models, images worth a thousand words, talk to us about what this future is what is kind of happened in sort of the visualization world of TV and Hollywood? Love it. So, did Jim get it right with Terminator? Let's hope not, I guess. But what a great movie it was. And I love when he actually jokes about it, he says, I told you guys, like, you know, this is coming.

4:47And now it absolutely is here. And why would someone like him get involved in stability? Yeah, great question. So, I had the great fortune of working on Avatar with him when I was the CEO of what a digital before I joined his CEO of stability and that movie took over four years to make and that's because there was fully rendered and I think if you fast forward to five to ten years from now the vast majority of film and television and visual media as we know it today is not going to be rendered it's going to be generated and in fact in Avatar there were certain shots that took 6 ,000, 7 ,000 hours of compute time to render one single frame.

5:32Thousands about that literally could be reduced down to minutes now. So I think Jim just wants a whole lot of life back. And when you think about like the creative process we all watch films, we watch movies, we love them. From the time we've born to our last memory, it's a commodity we never get sick of. We never, we never not want to watch it. And so there's this insatiable appetite out there in the world to consume stories and to create stories. And I think that we should just accelerate that. The problem with the film production process is time and money. So what he really wanted to do is rip those things out so we can move from a render to a generated model.

6:14We're going to see a situation where we're ever going to have AI is generating entire movies because it knows my preferences, what I love, and it's like the perfect movie for me. You know, personally, I kind of hope not. I don't think that actually the creative process, I think, needs to start with a human, and I think that human needs to dictate these tools in separate agents to actually make that story. And so I'm hoping that you'll probably want to hear stories that other people want to tell you. All right, well, let's take a different direction then. Sure. Am I going to see Marilyn Monroe and all stars the past coming back?

6:55These are a need for human actors if you can generate absolutely life -like actors and actresses perfectly. I mean, I can't see a situation where they're still around. Yeah. I think that it's actually quite, it's faster when they're talking about the film production bosses. It's actually easier to just shoot plates on an actor and just shoot real photography and get their performance. I think that's the visible layer of production. People gravitate toward it a lot. I think that AI will enhance those performances. I think the physicality of a director with a camera and an actor in front of it is a very important part of the creative process.

7:33And I don't think that that's going to go away too soon. And in fact, I think about the things that aren't going to change just as much as I think is going to change. But I do think after they take one take the director is going to say I got it because they're going to be able to do what you're talking about Which is manipulate that performance? I asked one more question. Do you for a move on what is the most dramatic change we're going to see in film and TV? Ten years from now as we see digital super intelligence. What's what's the craziest vision of what we're going to see in entertainment? I think we're going to see on the magnitude of five to ten to twenty X more content being created I think we're going to see a variation of time, where it's going to be a two minute.

8:13Like you said, you may want to have 20 minutes before you go to bed. You want to see a movie, but that's you'll have different type of time signatures. And I think that you're going to have an explosion of content creation, an explosion of number of artists in the world. I'm going to come back in 10 years and see if you're right about that. OK.

8:33Richard, a lot of your work was instrumental in the early days of bringing neural nets to natural language processing. So, what do you see as the next frontier beyond NLP? So, explain if you would what LLP is and where is it going next. Yeah, natural language processing, NLP, used to be a sub -area of AI and it has, I think, influence pretty much every other area of AI and there are lots of different algorithms you could train, and 2010 I had this crazy idea to train a single neural network for all of NLP, and 2018 we finally really built the first model that invented prompt engineering where you can just ask one model all the different questions you have.

9:17And over time, of course, you can ask questions not just over text, but also over images. And so I think next one of the answers to the panel's main topic of what's after ChatGbT is that we have many more multimodal models, You'll be able to have conversations over images. You have seamless inputs and outputs in not just the modality of text, but also programming, which is a huge unlock, visual, videos, images, voice, sound. But one really interesting modality that not many people have quite realized yet is that of proteins. Proteins are essentially the basic Lego blocks of all of biology. Everything in our bodies governed by proteins, and you can create a protein just like you can ask a large language model to write a sonnet for you or a poem for your wife.

10:04You can ask an LM to create a specific kind of protein. It will only bind to SARS -CoV -2 or only bind to a specific type of cancer in your brain. And what that means is that it will unlock a lot of different aspects and medicines. So I'm extremely excited about the future of LM's going into different modalities. And we're seeing that with deep minds products in alpha -proteo and such. So we had a conversation in back, but I didn't hear the answer. And the question is basically, is there an upper limit to intelligence? And we've talked about, and we just did a conclave on digital superintelligence and how fast we're going to get there and what does it mean.

10:51as we think about AI becoming more and more intelligent, yes, now I'm speaking to Elon, he said, okay, 2029, 2030, equal to intelligence to the entire human race. Is it just, you know, a million times more, and then a billion times more, and then a trillion times more, is there an upper limit to intelligence? Yeah, so really interesting question. So just to talk about alpha fold and Google for a second, as you mentioned it, like that was really interesting to understand how proteans fold, because that will help you understand how they are likely to function into acting your body. What we did in 2020 is actually create the first LM to generate a completely new kind of protein.

11:29And it was 40 % different to any naturally occurring protein. And it actually re -synthesized it in the wet lab. This was at Salesforce Research Institute. What did it do? Scientists there. And it was an antibacterial lysosine type of protein that is basically as antibacterial properties. And just to put that into perspective. 2020 was really close to COVID -19. So I'll make sure you're word. I gotta be careful what you say online sometimes. But what was interesting is that multiple startups have now started from this line of research. And I think it's hard for people to fathom like how much that can change medicine.

12:02In terms of upper bounds of intelligence, it's a really interesting question. Can it just keep going and going and going? I think you have to basically look at the different dimensions of intelligence, right? There's language intelligence, visual perception intelligence, reasoning, knowledge, extraction, and a few others, physical manipulation. And just, I'll show you just one example so I don't want to talk, could talk about this for hours, but visual intelligence, right? There are, you know, for a long time, people have looked at just the electromagnetic frequency spectrum of human vision. And there, you know, classifying every object on the planet is actually not that hard.

12:39and the upper limit is classifying all the objects on the planet. And we're probably going to reach that and we're not too far away from it. But that's just human vision. AI could eventually see all the way down to gamma frequencies and see and try to perceive atoms, right? And there you actually start to hit limits of physics, like quantum limits of like what can actually be observable. and you can go all the way into, like, seeing massively larger scale things at the universe level and how many different sensors do you have in the end? You can process all of that information. And AI could have billions of sensors that go out and then you get into really interesting limits of, like, the speed of light cone of, like, light cone.

13:25So I could talk about it for hours. It's a really tough subject, but in some cases, we are astronomically far away from those upper bounds and in some cases we already get pretty close. Fascinating. You talk about work productivity as U .com's objective. What does that mean? And I guess the question is the same. Is there any limitation on work productivity that we're going to see given the fact that I can command AI agents and robots to just do anything and everything and just and self improve along the way? It seems like we're going to hit sort of an infinite GDP at some point. Yeah, there are some areas of AI where AI can actually get into a self -training loop if there's a simulation of something that, and anything that can be simulated, AI can solve everything in that area.

14:16For instance, chess, the game of Go. You can perfectly simulate it. Hence, the AI can train and play with itself, of billions and billions of times, create almost infinite amounts of training data and hence solve every problem in that domain. What are other domains that we can perfectly simulate is programming. If you can programming languages can be run and then you can simulate the outputs, obviously in the computer, and then the AI can get better and better and eventually get superhuman in terms of programming. But where I can't simulate things infinitely many times is in like customer service, right?

14:49You can't have billions and billions of customers kind of ask about all the different things that can go wrong with the product that you're sending. And so in those kinds of areas, the limits are going to be on data collection. Can you actually fully digitize the process? I often joke like plumbers are probably the safest from AI disruption because no one's even collecting data on how to do plumbing. And it likes crawl somewhere, get different pipes. No one's having GoPro and 3D sensors and robotic arms and so on collecting data for that. So that will take much, much longer. I think in terms of work productivity, a lot of us are going to become managers.

15:25A lot of current employees that are individual contributors are going to have to learn to manage any eye to do the kinds of work that they do. And it turns out managing is also a skill. Not everyone is a good manager from day one. You have to really explain to the AI how you do a certain kind of job. And what we've seen, for instance, a really large cyber security company called MIMECAS, they've had 200 seed licenses using their product and then we did a workshop with them and actually explained to all the different groups, like this is what you can do. And some of them marketing can say, well, I usually get this long product description and then I have to describe it for these different industries and the email campaign and I have to write three tweets and three LinkedIn messages, all this stuff.

16:08And we're like, well, just say that to this agent and And then the IA agent does it for them. They're like, wow, now it's like six to 20 hours of work every other week, just got automated by describing this workflow that I used to do manually to an IA agent. And I think that will change pretty much all work. And thank you very much every industry. Kai Fu, I can go in a thousand different directions here. First of all, your venture fund innovations, which is how many billions of capital AUM? We managed about $3 billion. about $3 billion and you've been one of the most prolific AI investors I've had the pleasure to visit you multiple times in China and thank you for your amazing hospitality You've now become an entrepreneur and you're running both a company in China and a company in the United States Why did you do that?

17:00Well because this this time is for real right imagine you know this was my dream practice before What this was my dream when I went to college that AI was nothing. No one knew what it was, but I felt this was the thing I needed to do. And then we went through multiple winters of AI where there's disillusionment and I had to do other things. And about seven, eight years ago, we saw with deep learning. It was became clear. It would create a lot of value. But at the time, I didn't really see it becoming AGI. So I was an investor. We actually created 12 AI unicorns in sign -of -ation ventures. But this time with generative AI, the speed at which it's growing is just phenomenal.

17:47So you could help yourself. Yeah, I felt if I just invested, I'd be missing out. I would be in the backseat. I wanted to be in the driver's seat. By the way, everybody, I hope you feel the same. Right? I'm very clear about saying there are two kinds of companies at the end of this decade. Companies that are full utilizing AI, and everyone else is out of business. And I fundamentally believe that is true. You've written a number of books, AI superpowers, I commend to all of you. So since that was published, what's the biggest changes in the global AI race? And it is an AI arms race going on. Well, it isn't, isn't, because the companies in China are largely competing against each other for the China market.

18:31And they're generally not - I don't mean the national, but it is between companies around the world. Yeah, so you mean Chinese companies? What are their characteristics? So in my book, I superpowers I described, the American companies are generally speaking more breakthrough -innovated, that come up with new things. And then the Chinese companies are better at engineering, execution, attention to detail, doing the grant work. Use your interfaces. Use the interfaces, building apps. So in the case of mobile or deep learning, we saw that Americans invented pretty much everything, but China created a lot of value arguably more given technologies that were largely invented in the US.

19:17So now we're in this generative AI, again, invented by Americans, and we're in a unique position where the technology is disrupting itself very quickly in the US and elsewhere. So it arguably is still the age of discovery in US ought to win. But then the Chinese companies are able to watch the innovations, make some themselves, and then do better engineering and deliver solutions. So the company I'm building, zero one .AI, is doing exactly that. We don't claim to have invented everything or even most things. We learn a lot from the giants in Silicon Valley, open AI and others. But we think we build more solidly faster execute better.

20:02So an example was I talked about how 0 1 .a n now has is the third best model in company in the world ranking number six in models measured by LMS and UC Berkeley. But the most amazing thing I think, the thing that shocks my friends in the Silicon Valley, is not just our performance, but that we train the model with only $3 million. And GPT -4 was trained by AB -200 million. And GPT -5 is rumored to be trained by about $1 billion. So it is not the case. We believe in scaling law, but when you do excellent detailed engineering, it is not the case you have to spend $1 billion to train the brain.

20:44So this is really important for the audience here, because there's a lot of parts the world that don't have access to, you know, 100 ,000 H, you know, H 100 clusters. Right. And the question is, oh my God, can I really build a business or a product in picker favorite country with a small number of GPUs? And I think the constraint on GPUs forced you to innovate. Right. Can you speak to that? I think it's really important. We talked about that on our last podcast together. Yeah. I think as a company in China, first we have limited access to GPUs due to the US regulations. And secondly, the Chinese companies are not valued what American companies are.

21:28I mean, we're valued at the fraction of the equivalent American company. So when we have less money and difficulty to get GPUs, I truly believe that necessity is the mother in innovation. So when we only have 2000 GPUs, well, the team has to figure out how to use it, IS the CEO have to figure out how to prioritize it, and then not only do we have to make training fast, we have to make inference fast. So our inference is designed by figuring out the bottlenecks in the entire process by trying to turn a computational problem to a memory problem, by building a multilayer cache, by building a specific inference engine, and so on.

22:11But the bottom line is our inference cost is 10 cents per million tokens and that's a 130th of what the typical comparable model charges going Where's the 10 cents going? Yeah It's while the 10 cents would lead to building apps for much lower cost So if you wanted to build a you .com or perplexity or some other app You can either pay open AI $4 and 40 cents per million tokens or if you have our models it cost you just 10 cents. And if you buy our API, it just costs you 14 cents. We're very transparent with our pricing. Yes, Richard. There's a really interesting paradox called Jeven's paradox from the previous industrial revolution.

22:54A lot of smart people back then were working on making more efficient steam engines and using that use less coal. They thought, oh, if we make the steam engines more efficient, we're going to need less coal. But instead, we needed more steam engines everywhere. I think that's exactly what's going to happen. and recurrently in the Jevons paradox of intelligence. We're just gonna use intelligence in many more places. Everyone is gonna have their own assistant, their own medical team that understands everything about them, instead of being restricted by intelligence being very, very expensive. Yeah, I totally agree.

23:25I want to clarify, I'm not saying there's a fixed workload. We're making it cheaper. Right, right. I'm saying we're enabling a work really, much, much larger. I want to ask one closing question to all of you. We have people here who have daughters and sons or nephews or brothers and sisters, what's your advice to someone who is 20 years old, listening to this or through this? What's your advice to someone at the beginning of their sort of academic and professional career, given what you know is going on in AI right now, Prem? I think it's, don't waste your time learning how to code. Is I think the new language that's just been, I think the new code language is going to be English.

24:08And I think that absolutely learn as fast as you humanly possibly can on all AI modalities. And I think if you, and then when you find your passion, I think you're going to then find a very narrow AI to empower you to do what you're, what you're really set out to do. Thank you, Prom. Richard. I will disagree. I think you should still learn how to program. I think that is how you get to really understand how this technology works at the foundational level and how it becomes less magic and more something that you can yourself modify and construct with. But you need to combine computer science and programming with another passion that you can actually apply all of that intelligence to.

24:51And ideally the younger you are, the more you learn the foundation's math physics, the sciences. I think I'm gonna cut you off because I'm being I want to have Kai Poo's final word here. Okay, I actually agree in this with both of you I think people should follow their hearts, right? If you dream of becoming a fantastic program and you can do it You should do what Richard says if you think programming is the way to make the most money No, then you should follow our process Ladies and gentlemen, please give it up to these three amazing CEOs Thank you Thank you