Ex-Google China President on How China Is Shaping the Future of AI w/ Kai-Fu Lee | EP #134

33:39You don't have a lot of compute. And when you make it, by the way, if you train it very efficiently, we can also, can we also have an inference that's very efficient, and costing only a few percent to run it in apps. So, talk to me about your product, Beagle. By the way, I asked you earlier how to pronounce it where the name came from. And I think it's worth repeating so people remember it better. Beagle and Goathing Retriever, right? It's a dog game. Yeah. Yeah, yeah. I mean, well, I'm not with a name Beagle, B -A -G -L -E, it's a cute, good dog, and Golden Retriever, but when the two of them make a little puppy, that puppy is called BeGOLD, B -A -G -O.

34:26And, BeGOLD, BeGOLD, B -A -G -O -E is very good at hunting, and Golden Retriever is good at retrieving. So it's an apt name for an AI search engine. But I should also point out that BeGOLD is not 0 .1 .ai products. It is a product that I did venture build and it's actually an American company and it uses a model very similar to the model that zero one that AI has built. Super fast, super cheap. Thereby thinking that AI search could be reinvented. So do you want to talk about zero one or you want to jump into a little bit about BGO, which you prefer? Well, the alley me start with zero one then we go into beagle.

35:11So yeah, zero one. I think you know This may we came up with a very good model called e large and e large was a bit behind GPD40 which came out one day later and We we at the time were ranked number seven and which is great number seven model number four company Just behind opening i google and entropic something to be really let's put the chart on One second. Yeah. Go ahead. So that's the May chart, but I want to talk about what just happened in October because over between May and October, lots of models emerged. E large was no longer competitive. And but we had been working based on what I described as working super hard, building to match GPD40 and maybe even be faster.

36:03And that was accomplished in October that we kind of took revenge on the GPT -40 May version which you can now see on this chart as just below us as number seven in the world. We just beat them by a little bit. So this is a case in point where we saw GPT -40, we saw what it could do and we knew it could be done and we said let's go do it. And basically with no hint on how it's done, we figured out how to do it ourselves. I'm sure the methods are different, but we did match their performance in just five months. Of course, in these past five months, other great models came out, including new version of GPT -40, and GROC and others.

36:49So we came out in October with a tiny model called E -Lightning, because we wanted to be lightning fast. This is a much smaller, much faster model, but it became number six model in the world in number three company. And we also surpassed Anthropic this time. How big was your team building this? The pre -training team is basically three or four people. The post -training team was maybe ten people. The infrastructure team may be another ten people. So it's a 20 to 30 person project. It's pretty small. And the thing we're most excited about and proud and unique about is that we trained this model, the pre -trained only cost a little over $3 million.

37:37And this is 3 % of what GBD4 cost to train. And we actually beat GBD4 in performance. And the inference costs is very, very low. It's around $0 .00 per million tokens. Let's go to that chart. There's a chart here that looks at inference cost over time, which is super impressive as well. So explain this chart here, please. Yeah. So back in June, we were $1 .00 for the per million tokens costs, which was a lot lower than GPT -40 at the time, which was, I think, about $10 price. And then, by September, we came up with a number of breakthroughs, including new ways of doing mixture of experts and better inference and ideas of PV cash management, et cetera.

38:32So we had really a big breakthrough not in launching the E Lightning, because E Lightning was 114th the cost of the previous E -large model and at 10 cents per million tokens. And GPD40 have also come down in price, but it was $4 .40. So the developer... Yeah, just to give folks who are just listening and watching this back in June, E -Lightning was a buck 40 per million tokens. Today it's at 10 cents per million tokens and next June it's expected to be three cents per million tokens. It's a 50 -fold decrease and comparing to GPT 40, which was at four and a half bucks per million tokens. So, I mean, we are seeing this precipitous efficiency gains over time?

39:31Yes. So another way to look at it is GPT -40 dropped 10X in one year. We actually succeeded in dropping 50X in the past year. So we feel we now have the most competitive lowest price, lightning engine. Our cost is 10 cents per million tokens. our price is only 14 cents per million tokens. So we're also not taking a big margin. So if you look at the performance of GPD40 and feed Lightning, their new version is a little better, not a lot only a little better, but they're 440 and we're 14 cents. Incredible. Is it all algorithmic gains?

40:19There are a number of differences. I think we actually, I'm sure we use different algorithms because we don't know what they use. We came up with our own improvements. The improvements you're making over time. Are they out? I'll agree with the games there. Yeah, our performance gains going from E -Large to E -Lightning are using a new mixture of experts model and new ways of modeling and also getting more high quality diverse data and also having super fast infrastructure so we can train multiple times to learn more and to do research. The efficiency gains were also mostly by the super fast mixture of expert model, but also by some inference advancements in terms of KV -cash memory management.

41:14As an example, the way we do our next -gen model design is not go and invent a bunch of new things and go make them fast, but it's from the get -go. They have to be fast. So we would first ask the question, where do we project in four months, which is our product cycle? the best chips might be. And how do we get those chips to inference really fast? And oh, these chips with a lot of HBM, which is high bandwidth memories coming out. And can we turn the inference problem from a compute problem to more of a memory bound problem? Then should we rewrite our inference engine? Then how much RAM can we put out?

41:58As a second layer memory, how much SSD can we put out? can we construct a computer four months from now that is super fast. I mean, it's not an, it's made out of standard parts, but still it has a lot of memory. Then we put the memory bound inference engine on top. Then we asked the modeling team in four months, what model can you build that fits perfectly into this box? Not too large, not too small, use up all the memory, but don't go too far and use power of two. So a lot of constraints for the researchers, which some companies might face reluctant researchers, but in our case, we are all building a product.

42:39We're one team in one direction. So we all took each team took the order and then marched ahead and out came a very accurate and a super fast model thanks to this vertical integration from model to inference engine down to the hardware and memory. I love the old saying, you know, innovation comes from thinking in a smaller and smaller box when you put constraints on yourself. Everybody, I want to take a short break from our episode to talk about a company that's very important to me and could actually save your life or the life of someone that you love. Companies called Fountain Life. It's a company I started years ago with Tony Robbins and a group of very talented physicians.

43:22You know, most of us don't actually know what's going on inside our body. We're all optimists. Until that day, when you have a pain in your side, you go to the physician and they burn into your room and they say, listen, I'm sorry to tell you this, but you have this stage three or four going on. And, you know, it didn't start that morning. It probably was a problem that's been going on for some time, but because we never look, we don't find out. So what we built at Fountain Life was the world's most advanced diagnostic centers. We have four across the US today and we're building 20 around the world.

43:58These centers give you a full -body MRI, a brain, a brain vascular, and AI -nabled coronary CT looking for soft plaque, dexascan, a grail blood cancer test, a full executive blood workup. It's the most advanced workup you'll ever receive. 150 gigabytes of data that then go to our AIs and our physicians to find any disease at the very beginning when it's solvable. You're gonna find out eventually. Might as well find out when you can take action. Fountain Life also has an entire side of therapeutics. We look around the world for the most advanced therapeutics that can add 10, 20 healthy years to your life.

44:37And we provide them to you at our centers. So if this is of interest to you, please go and check it out. Go to FountainLife .com, Backslash Peter. When Tony and I wrote our New York Times bestseller, Life Force, we had 30 ,000 people who reached out to us for Fountain Life memberships. If you go to FountainLife .com, Backslash Peter will put you to the top of the list. Really it's something that is, for me, one of the most important things I offer my entire family, the CEOs of my companies, my friends, it's a chance to really add decades onto our healthy life spans. Go to fountainlife .com, backslash, Peter.

45:20It's one of the most important things I can offer to you as one of my listeners. All right, let's go back to our episode. One of the conversations over the last year is we're running out of data to really improve the models. What do you think about that? Do you believe that to be the case? I believe it has a bit of a dampening effect on how much we can expect scaling law to continue. But I do think we have ways of getting more data just not as easily as it used to be. Because the fact is that humans were smart to create language as something that could be passed on over millennia. And we have, so once we start doing Gen AI, we took all the language data and put it on.

46:07Now, every year we're generating more language data, but clearly way less than the total collection. So that is incrementally much, much slower. But on the other hand, we have video data, we have audio data. And also we're going to have Embodied AI gathering spatial data. So those, and also we have ways of creating synthetic data, Which is not as good but better than not using it. So these are the ways I think we're trying to Compensate for the fact that most actual data has been used in the outcome is I think we will still get more data benefit Just not as much as we used to get. Yeah Let's jump over to to be go.

46:52So it's a US company and And why was it started in the US? Is it something that was funded out of CenoVations? What's its mission talk to us about it? Right as I was building up 0 1 .ai. I ran into a lot of brilliant American engineers and researchers They want to stay in America, but they liked my vision So I said why don't I help you guys venture build a company? so they build a company called rhymes technologies and and they build an excellent model, very similar in approach to the model in 0 .1 .ai. And on that model, they added a lot of their unique multimodal and launched RIA, which is an open source multimodal engine, which is one of the best in the world, but only 3 .5 feet.

47:44So continuing the tradition that companies that I help build are very committed to open source. And on an advanced version of that ARIA, they built an AI search engine. So I was pleasantly surprised when they showed it to me. In fact, I was blown away by how good it was already and also how really, really fast it was. What's your hope with Bigo to come in and through an app become the dominant search player? Yeah, I always remember when I joined Google, Larry Page came and talked to us and he said Google in this current form is not the ultimate form. The ultimate search engine should be one where you ask a question and Get a single correct answer that always kind of stuck with me and when I venture build the The rhymes and beagle team in the US we talked about it and we feel that the time has come And in building such an engine, we also consider, well, first on the mobile phone, is a very small screen.

48:52So you can't have all the tabs. So doing a research -oriented multilingual search exploration is very, very awkward. And a single answer just makes so much sense. But of course, the first issue with a single answer is whether it is not correct, whether there is hallucination or some errors. So we work very hard to maximize factuality and Beagle is actually better in factuality than a lot of the other AI engines measured by Objective third party queries, so I think those really bring us one step closer to Larry pages stream I think right now the team just wants more people to try it and they want you know really knowledgeable, caring, smart people to try it first and give them their most feedback.

49:42And it's great that that can be your abundance program because those are the types of users your readers are. Yeah. And how do you possibly compete against, you know, companies who've got billions of dollars in this field? Is it just that much better in implementation, that much better and alternate? Yeah, it's a tough challenge. That's why very few companies go after the space. The fact that the complexity gains somewhere is an indication. People want something refreshing. And also, I think we're confident about Beagle's factuality and engagement. It also has pictures inside the search making more engaging and entertaining.

50:33But also, I think just the search players, particularly Google, and to some extent being, will be hesitant to replace their search engine with a one -answer engine because with one answer, people don't look at ads and the ad revenue will come in. Yeah. They will be. I have to ask the question that probably a lot of people are thinking is this another TikTok where it's a Chinese owned company and it's a way for, you know, what people's imaginations that it's just a way to get US data into this is a US based company, a US own company, yes. It's actually both US based and US own. So, it's not, it's employees are Americans, Singaporeans, Taiwanese, I myself, I'm Taiwanese.

51:23So it's not a Chinese home. It's quite different from TikTok. Yeah, I get that. I've been a fan of your work, Kaifu, for a decade now and I've had at the pleasure of calling you a friend. I had Elon Musk and Jeffrey Hinton and Ray Kurzweil you know them all on my stage at abundance 360 last year and it was a fascinating question they came up. The probability that AI will be the greatest invention versus the probability that it will destroy humanity to put it very bluntly. And I think, Hinton congrats to him on his Nobel. And Elon said, yeah, 80%, it's good, 20 % were screwed. Where do you come out on that?

52:17Do you have an opinion on this? And then how do we protect the downside in your mind? Yeah, so if we assume it's like 10 year horizon, is that reasonable? Yeah, I think all of it's going to play out in the next five to ten years. I think if we get to the next two, my belief, I don't know if you agree with me, if we get to the next ten years, we're fine. I totally agree. That's why I ask the question. Okay, so in the ten -year horizon, I would say we have a 5 % chance of a disaster caused by AI, And of 35 % chance of a disaster caused by humans using AI and 60 % were good. Okay, so now you've written an entire book on this, but I'm going to ask you to provide some some summarization.

53:10What do we do? How do we, you know, we protect our downside, the upside is fantastic. Do you have any advice for parents, entrepreneurs, leaders here? How should they think about protecting our downside? If you're head of the world here, what do you do? What do you think? Well, I think a lot of technological risks are best addressed by technologies. Like when electricity went out, the invention of circuit breakers, when the internet went out, the antivirus. So technologies are the best likely savior to technological problems. So I would encourage more computer scientists, AI people, to not just work on the biggest, next big model, or AI applications, or AI inference, or whatever, but some percentage of the ones who feel a responsibility and their conscience asking them questions, then they should jump into AI safety.

54:15To find the various types of safeguards and guide rails that will protect us. I think to me, that's the most important thing. Regulation comes second. I would actually feel general AI only regulation to be in uncharted territory and potentially not constructive. I think would be better to take existing laws, let's say laws about fraud, laws about other blackmail, and then apply the use of AI to achieve those things, laws about slander, laws about theft. So we have lots of those laws, those laws are effective, understood. So apply them to people who use AI to break those laws, make sure the punishments are equally if not more severe, that would create some deterrence to start to regulate AI before it's mature and while it's changing, by governments that are slow moving, seems like a few times exercise.

55:19Yeah, governments are linear or sub -linear at best. You're going to be joining me on stage in one of my panels in Saudi Arabia in just 10 days or so. Excited to see you there at the Yeah, FI will summit. Yeah. Yeah, for the great one. One of the conversations we're gonna have is around the potential dangers of ASI, artificial superintelligence. But before I go there, I would argue that we passed the Turing test many years ago and no one really noticed. It just, just, you know, it's coming on. Will we know when we get to AGI, I don't know that there's a good definition of AGI. And I don't even know if there's a good definition of digital superintelligence.

56:14I mean, these are challenges when we talk about these words. Do you agree with that? Yeah, I think AGI was created to me that AI could do absolutely everything humans do. And that may not be the right definition because we can't yet project when AI will have love or even when AI will be viewed as having love. Those are still some distance away. But I think thinking generally that AGI just means something overwhelms us that does almost everything we do so much better, even the most challenging intellectual tasks, like inventing a new theorem or something in physics or chemistry. So if we kind of extrapolate that to be the ASI or AGI, then I think it's highly likely that it will arrive in the next five to ten years and we do need to put in the safeguards.

57:10I imagine we just saw a number of Nobel Prizes related to AI. I have to imagine that in the very near future, every single breakthrough in physics and math in chemistry is going to be enabled or driven or connected to some AI models doing the work. Yes, absolutely. Yeah, I met an economic professor on my recent trip to the Bay Area and he said he already treats GPT01 as a graduate student. one that's able to challenge him and find mistakes and he will teach and guide the student and the student learns and the two of them are great partners in inventing new things. So it's already happening and so I would add also economics to be part of another area to what you listed.

58:09Do you say please and thank you to your AI when you're speaking to it? I do say please, I don't say thank you. I'm not sure why. It's interesting, right? I find myself saying, please, sometimes thank you for it. But it is interesting to get how it's just a very small step away for being a part of every aspect of our lives. People are worried deeply about jobs. You've made some prediction about loss of white collar jobs. And of course, all the multimodal AI systems that you're speaking about are being embodied in robots. There is a number of fantastic humanoid robot companies coming out of China.

58:56China needs robotics for its aging population. The one child policy has significant implications for an aging population. So talk to me about your prediction about jobs, your advice. I know you wrote about this as well in your previous bestselling books. But if you just have a few, how should people think about jobs, what are color jobs, and then labor with human art robots coming? and what should, what do they tell their kids, what do they do for themselves? How do you think about that? Well, I think the fact is that the white color jobs are gonna be the first set most challenged by AI, because just software can replace a lot of the routine and even non -routine work.

59:56And they will do so very rapidly in the next five years. That's, I think, now universally believed when I wrote about it in my earlier books, it was met with a mixed reaction. So, the people who surprised everybody expected it was going to be blue collar work leaving first? Yes, yes, right. Because it seems, you know, having intelligence in a white collar job is harder to replace. But it turns out dexterity is harder to replace because that's not necessarily solved by the Gen AI technologies. So blue color work, I think, going from factory to the type of caring work you talked about for elderly, is going to happen as the next wave.

1:00:44I'm among the more conservative on how fast that will happen because I think these technologies are very expensive. Not only do you need the LLM expense, but also these robots that Elon Musk has shown are way out of any consumer's price range. So they're kind of going to be a while because before the the Kings are worked out, before people accept them into their families and lives and offices and the costs have come to have to come down. So you say that. So you say that, but you know, I wouldn't, I'm an investor and figure AI, Brett at Cox company and figure and Tesla both are projecting around a $30 ,000 price tag.

1:01:29Let's say it's $40 ,000 price tag. If you could lease that, right? And you lease it at $300 or $400 per month, having a $24, say, seven employee for $400 a month is pretty affordable at least here. I can see your point. I can see your point. I'm still a little more cautious because especially used around the home, You know, the, just clean my room is one thing that I would predict in three years this robot cannot even begin to do because every room is different. Every definition of clean is different and every family home is different. But there are many other things, you know, like talking to the kids or doing more household repetitive work that can be done.

1:02:2330 ,000 I think is probably a reasonable price point for middle class America, but for China, India, other countries is still way too expensive. Real quick, I've been getting the most unusual compliments lately on my skin. Truth is, I use a lotion every morning and every night religiously called one skin. It was developed by four PhD women who determined a 10 amino acid sequence that is a cynolytic that kills senile cells in your skin. And this literally reverses the age of your skin. And I think it's one of the most incredible products. I use it all the time. If you're interested, check out the show notes.

1:03:04I've asked my team to link to it below. All right, let's get back to the episode. Kai Fu looking forward into 2025, 2026. Would you mind sharing some predictions of what you might see in the AI world coming that would be surprising? Sure. I think from the product side, we will see basically every category of mobile app be incorporated AI with many of them showing AI first disruptive types of changes. So, in other words, in about two years, every app we use will be replaced by another app or super upgraded by the same app. I think agents will be a major technology where we delegate what we want rather than just get an answer.

1:04:00I think multimodal will not only be dramatically better because a lot of the super smart people are working on it and we're seeing text to video something that would be fairytale in my youth is now starting to happen. And I would also project that these great technology attachments will find real uses in applications. So it's not just demonstrating, hey, look what AI drew for me in a video today. But while I created this marketing video for $10, those are the kinds of advances I would definitely anticipate to see from a business product and some of the known technology side. You know, we just, I don't know if you saw this, we just mapped the connectome of the Drosophila.

1:04:46Did you see that? There was a, they were able to map 50 million synaptic connections. And it's a step away from a mammal. Let's see the mouse. I think we're going to probably see the connectome of a mouse done in the next year or two. where do you come out on the whole, you know, brain computer interface world? Hi, I'm still a little bit more, I would say I'm a bit more cautious about it. I think this is one of the areas where there's major disagreement on how fast this is moving and what dangers it might provide. And I think we need to be cautious because it is intrusive to our bodies and it's a kind of a potential, a potential slippery slope, right?

1:05:48I think people can all get on board with treatments using interfaces and get on board with non -intrusive kinds of BCI. But as we go deeper and deeper into reading our mind, creating smart issues, And I think we just have to make sure people who are being experimented with our aware of what kind of risks they have and that the downside doesn't outweigh the upside.

1:06:23Let's wrap up with a quick look at something I've heard you speak about, which is you were there at the PC revolution, the mobile phone revolution, and the AI revolution. And you've seen those progressions. And I think you've modeled what the progression will be for AI. Can you give me that summary? Because I think it's super useful for entrepreneurs listening. If you're looking at starting a company in the AI world, there's a lot of lessons learned from the PC in the mobile phone world, yes? Yeah, I think applications always follow a reasonable pattern of being replaced. Because when a new technology, a new form of content comes out, you, as users, have to first browse them, then you make the content, then you search and organize the content, then the content gets richer into multimedia, multimodal, then you can transact on the content, whether it's by payment advertising e -commerce or online to offline.

1:07:33Because these are the fundamental needs of people and the progression of apps that I talked about go from fewer users to more users, smaller money usage to more usage, simple will usage the complex usage. So it's a really exciting iteration of better technology enables the next step on the protrojectory. More you people use it, more money's made, more entrepreneurs, more funding, more GPUs, more products, more models. So the virtuous cycle goes on. And the most most exciting thing is it took PC ecosystem easily 30 years to play out. It took mobile maybe 15. But we're going to see AI play out in the next three years or so.

1:08:17So if you jump in to start the company, this is the biggest roller coaster ride you can ever imagine. What's your advice to the entrepreneur jumping in to start a company in the AI space? What should they do? What should they not do? Right. Yeah, I use the roller coaster as a metaphor because I don't see it as a rocket ship purely upside. There are a lot of challenges and traps. I would be cautious to probably look at an app company because that's the biggest space with the most entrepreneurs and the inference costs are coming down. But when you think about starting an AI app company, be cautious first about can you handle the inference costs because those are too expensive.

1:09:01Don't run out of money because inference cost is coming down. So time your launches, time your product design, according to the technology you need, and when that technology will be low enough in inference costs. Secondly, be careful of the modeling companies, because we've seen companies like Jasper, who build great apps, but then the model sucked all their know -how because they saw all the data. So find, ensure that you don't do that. But the last advice is all the models are getting better. One day they'll be close to AGI. Does that mean my app will be eventually limited to a veneer and with very limited value.

1:09:41I don't necessarily think so because historically we've seen great platforms emerge but other apps can often build the modes. The mode that TikTok, Instagram and others have their value were not taken away by the lower level transaction layers or operating system layers or browser layers. So the key is when you build an app and gain some edge and don't sit on your laurels, think about how to build a mode. That mode could be your brand, your user loyalty, user data, or social graph, things that we have seen, or maybe new things in the AI era. Yeah, I like to think about it and I'm curious if you agree.

1:10:26When I'm evaluating an AI company to invest in it, you know, they're, I'm looking at what unique data do they have and what customers do they have a very close relationship with. And everything else in the middle will get demonetized and replaced over time. Yeah. Yeah, I think your advice is great for B2B. I was thinking what be to see the two are definitely in concerts. Yeah. What should people know about, you know, let's turn to the last question, which is We live on one planet and we've got sort of this bipolar Element of China versus the US in AI and we have this split universe It would always be better to have alignment and everybody working well together.

1:11:20But what advice do you have there? How should people think about this? I mean, because it's a complicated way above my pay grid. And I don't want to put you in a situation where you're talking about anything that, you know, feel comfortable about. But I can't not have the conversation of, you know, I see a lot of people feeling like China is the enemy there or US is being nopplistic. How do we navigate the next 5 -10 years, which are the most critical? Yeah, there are some things that we're just not able to change. They are what they are. But I think each of us can make our own judgments and decide where we can reduce the impact of this unfortunate geopolitical situation.

1:12:12For example, in the open source is one area where all the countries collaborate equally and generously. Academic collaborations continue on. Areas of collaborations not involved in the sensitive model or a semiconductor can still go on. And I think connectivity in the world, working people to people business to business needs to go on. It has to be good. Globalism has to be right. Differences between governments is kind of like, you know, when our parents have fights with another parent, we kids can still get along and do something interesting and fun, right? Agreed. I like to say we all have the same biology, so a breakthrough in medicine in China is equivalent to a breakthrough in the Bronx.

1:13:03Absolutely. We all share 24 hours in a day and seven days in a week that's a it's something every single human has and So anything again just time efficiency in one place gains time efficiency in another Well, we share the same planet Yeah, we Yes, thank you for sharing time Super excited about their performance. I've seen in Bego and look forward to playing with it. And thank you for joining me on moon shots to talk about your passion, your vision and congrats on going from the guy behind the curtain to the guy in front of the company. Thank you. Thank you so much. See you soon, my friend. Bye.