Mark Zuckerberg Wants to Make Open Source AGI

0:00Today on the AI Breakdown, we're looking at Mark Zuckerberg's new ambition to build open source AGI. Before that on the brief, ASU and OpenAI signed its first university partnership. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube, our newsletter, and our Discord.

0:24Welcome back to the AI Breakdown Brief, all the AI headline news you need in around five minutes. The relationship between AI and education has been interesting. On the one hand, it is contentious in the sense that many professors and faculty members and teachers have been concerned that the rise of things like ChatGPT just means endless cheating. On the other hand, others have seen it as an incredibly powerful tool for changing the way that we learn. What if every student had an incredibly sophisticated tutor for any subject they needed? Throughout that process, there's been a lot of evolution.

0:57For example, New York public schools initially banned ChatGPT, but then went the other direction after they considered it a little bit more. Well, now today, OpenAI has announced its first partnership with a university, that university being Arizona State. So the three areas that ASU is saying that they want to focus on for their use of ChatGPT enterprise are 1. Enhancing student success, 2. Forging new avenues for innovative research, and 3. Streamlining organizational processes. So one of the things that I think is really interesting about this is that it reflects a phenomenon that I think we are going to see more across not just the educational world, but the corporate world as well.

1:32Take this quote from Chief Information Officer Kyle Bowen. Our faculty and staff were already using ChatGPT, and after the launch of ChatGPT Enterprise, which for us addressed a lot of the security concerns we had, we believed it made sense to connect with OpenAI. So in other words, this was a bottom-up movement. People just naturally adopted a tool which they found really interesting and useful, and then that ratcheted up at some point to the organizational level. Now, the organization had some concerns, but those were addressed by a specific product offering, leading to the opportunity for this larger partnership.

2:02I think in general, this is what we're going to see in businesses as well. We're going to see employees make the decision themselves to adopt some of these tools, be it ChatGPT or something else. They're going to be productive and then show their friends and colleagues how they're using those tools, who are going to then adopt them as well. And at some point, managers and bosses will decide whether it makes sense to just go big across that tool set. Now, one of the things that's valuable about when an organization does decide to invest is that all of a sudden there's more space for cross-pollination and for best practices from one section of the organization to get to others.

2:34For example, at ASU, the university is going to begin taking submissions from faculty and students on how and where to use ChatGPT. Now, it sort of makes sense that this is coming from ASU, as last year around all of this excitement around ChatGPT, they had already launched an AI accelerator program, and they had even begun to offer classes focused on things like prompt engineering. Now, this is a story I would have covered either way because I think it's really interesting and reflective of a trend, but I will note that I got this news from so many people at ASU yesterday, and one, the fact that they're listening to the AI breakdown suggests to me that they have great taste and are very smart, but two, the fact that it wasn't just coming from PR people, but from professors and faculty members and lots of folks, suggests that this is something that that university is really excited about, so I think it is worth watching.

3:20Over in the K-12 world, Axios Columbus shows how Ohio is preparing to launch an AI policy for their K-12 schools. Per a new report, they are one of 11 states that are preparing policy guidance for K-12 schools on generative AI. Here, once again, you have a situation where lots and lots of teachers are already using AI in their classrooms in various ways. The sort of guidance that comes from the district or state level gives them more confidence that they're not going to be punished or reprimanded for doing so and help shape the bounds of what they can do, which of course will help expand what they can do.

3:52Now, moving away from education into the business world for a moment, the Financial Times reports that Cohere is in talks to raise as much as$1 billion. The company's last fundraise was in June 2023 when it raised$270 million at a valuation of$2.2 billion. That round included not only venture capitalists like Index, but also strategics like NVIDIA and Oracle. Cohere is led by influential ex-Googlers like Aiden Gomez and is trying to go after the enterprise LLM market as opposed to the consumer market like ChatGPT. Still, the capital that they are looking to raise, somewhere between a half million and a billion dollars, shows that even when focused on something of a niche, the costs to compete in this space are just absolutely enormous.

4:33As if to make a point on that, the information has shared a special report around how Google has developed a special class of stock as a way to fight the AI talent wars. The information writes, Even as Google makes a major round of staff cuts, it is tapping a special pool of stock compensation to retain its top artificial intelligence researchers, as rival OpenAI lures them with multi-million dollar pay packages. Select researchers at Google's DeepMind unit have received large grants of restricted stock worth up to millions of dollars per person. Leaders of the unit have been increasingly using a little-known advanced compensation program to compete with OpenAI's eye-popping offers.

5:07Now, basically, these are accelerated vesting programs. So instead of the standard four-year vesting period for grants, some of these vest as fast as a single year. To me, there is nothing particularly surprising about this, it just shows the intensity with which these organizations are competing. And it's clearly a two-part competition with compute resources on the one hand and talent on the other. Meanwhile, over at Microsoft, the company has just announced that its AI-powered reading coach is now available at no cost to anyone who has a Microsoft account. They wrote in a blog post, It's well known that reading is foundational to a student's academic success.

5:39Studies show that fluent readers are four times more likely to graduate high school and get better jobs. With the latest AI technology, we have an opportunity to provide learners with personalized, engaging, and transformative reading experiences. So once again, here we are back at the education theme. And part of the reason for that is I just think AI is going to open up so many new avenues for more personalized learning, which has been a drum that education people have been beating for years, but now we actually have the technology to make it so. Lastly today, a little update from the stock market.

6:07Nvidia and AMD both reached new all-time highs on Thursday. If you thought that 2024 was going to slow down the chip wars, think again, because despite 127.6 % gains for AMD in 2023 and 238.8 % gains for Nvidia, the slowdown doesn't appear in sight. Anyways friends, that is going to do it for today's AI Breakdown Brief. I'll be back soon with the main AI breakdown. Welcome back to the breakdown. I think that everyone believed that 2024 was going to be another big year in the AI arms race. And starting off right fresh, we have some big, bold statements from Mark Zuckerberg over at Meta that certainly reinforce that point.

6:47Comically summing up the situation is James Mishra who tweets, Zuckerberg in 2004. I want to look at pics of Harvard chicks. Zuckerberg in 2024. We will build AGI. We will storm into heaven and execute God for crimes against humanity. We will build a vision of the human race that encompasses a thousand suns. So what is James referring to here? Well, it is this reel that came out on Instagram yesterday, where Zuckerberg writes some updates on our AI efforts. Our long-term vision is to build general intelligence, open source it responsibly, and make it widely available so everyone can benefit. Now, I think this is a really significant announcement, and so let's actually watch it in its entirety.

7:25I am bringing Meta's two AI research efforts closer together to support our long-term goals of building general intelligence, open sourcing it responsibly, and making it available and useful to everyone in all of our daily lives. It's become clearer that the next generation of services requires building full general intelligence, building the best AI assistants, AIs for creators, AIs for businesses, and more. That needs advances in every area of AI, from reasoning to planning to coding to memory and other cognitive abilities. This technology is so important, and the opportunities are so great that we should open source and make it as widely available as we responsibly can so that everyone can benefit.

8:10We are building an absolutely massive amount of infrastructure to support this. By the end of this year, we're going to have around 350 ,000 NVIDIA H100s or around 600 ,000 H100 equivalents of compute if you include other GPUs. We're currently trading Llama 3 and we've got an exciting for a map of future models that we're going to keep training responsibly and safely too. People are also going to need new devices for AI. And this brings together AI and the metaverse. Because over time, I think a lot of us are going to talk to AIs frequently throughout the day. And I think a lot of us are going to do that using glasses.

8:48Because glasses are the ideal factor for letting an AI see what you see and hear what you hear. So it's always available to help bow. Ray-Ban Meta Glasses with Meta AI are already off to a very strong start, and overall across all this stuff, we are just getting started. All right, so tons to dig into here. Let's kind of take things one by one. First of all, right leading the pack, easily the thing that is being most discussed, is the fact that Zuckerberg has identified creating artificial general intelligence as the company's new clear goal. Now, on top of that, he says that to the extent that they can and reasonably and safely, they want to open-source it.

9:28These are two very big things. Other companies like OpenAI have, of course, long had AGI as their core mission. This is a new declaration from Zuckerberg, and is likely to shape the way that they design their efforts going forward. Now, when it comes to this idea of open-sourcing it, that's also a fairly radical and important position relative to the rest of the field. This really cements meta on one very clear end of the spectrum of the AI safety discussion that is frankly diametrically opposed to many others on the opposite end of that spectrum. Now, alongside this reel that was released on Instagram, Zuckerberg did some interviews with The Verge and others.

10:02They tried to get a little bit more information about what his definition of AGI was, and he said, I don't have a one-sentence pithy definition. You can quibble about if general intelligence is akin to human-level intelligence, or if it is like human plus, or if it is some far-future superintelligence. But to me, the important part is actually the breadth of it, which is that intelligence has all these different capabilities where you have to be able to reason and have intuition. Zuckerberg also thinks that we will not have some singular moment, but that AGI will arrive as a gradual process. He said, I'm not actually sure that some specific threshold will feel that profound.

10:33The Verge also asked Zuckerberg more about his position on openness. He said, I tend to think that one of the bigger challenges here will be that if you build something that's really valuable, then it ends up getting very concentrated. Whereas if you make it more open, then that addresses a large class of issues that might come about from unequal access to opportunity and value. So that's a big part of the whole open source vision. He also very clearly tried to contrast Meta's approach with that of OpenAI, although he didn't say the words OpenAI. Quote, There were all these companies that used to be open, used to publish all their work, and used to talk about how they were going to open source all their work.

11:04I think you see the dynamic of people just realizing, hey, this is going to be a really valuable thing, let's not share it. Zuckerberg also alleged basically some amount of regulatory capture or safety capture, saying, The biggest companies that started off with the biggest leads are also, in a lot of cases, the ones calling the most for saying you need to put in place all these guardrails on how everyone else builds AI. I'm sure some of them are legitimately concerned about safety, but it's a hell of a thing how much it lines up with the strategy. Now, of course, one could easily be cynical about Zuckerberg's open source turn and believe that he just shifted into this lane because this was the lane available for him.

11:37The question is practically whether the motivation actually matters, or whether what matters is that the most singularly powerful person in one of the world's biggest companies has decided to walk down this path, whatever the reason. Now, one thing that he would not commit to is for sure saying that if they reach AGI, they will open source it fully. He's hedging, right? He says, for as long as it makes sense and is safe and responsible thing to do, then I think we will generally want to lean towards open source. Obviously, you don't want to be locked into doing something because you said you would.

12:04So, like I said, this is by far the most significant part of this announcement, the fact that Meta is going after AGI and they want to do it in an open source way. But there is a lot more in here as well. One is the sheer scale of computing resources that Zuckerberg and Meta are going to bring to this goal. He told The Verge that by the end of the year, Meta will own more than 340 ,000 of NVIDIA's H100 GPUs. Now, external research has suggested that Meta's H100 shipments for last year was at 150 ,000. The only other company with shipments of that size is Microsoft, and it's at least three times larger than everyone else.

12:37The Verge writes, when its NVIDIA A100s and other AI chips are accounted for, Meta will have a stockpile of almost 600 ,000 GPUs by the end of 2024. That's around$20 billion worth of compute. That's the scale that we're talking about when it comes to these large models. And that gets to another part that's important from this announcement, that Meta is training Llama 3. Now, in some ways, this isn't surprising at all. Of course, Meta is training Llama 3. We've heard things from behind the scenes that suggest that as soon as Llama 2 came out, they were working on it. However, there seems to be a different emphasis now.

13:09Said Zuckerberg, Llama 2 wasn't an industry leading model, but it was the best open source model. With Llama 3 and beyond, our ambition is to build things that are at the state of the art, and eventually the leading models in the industry. Now, one of the things that I wonder in terms of that switch is how much it has to do with the rise of Mistral. For basically all of 2023, until Mistral launched, Meta's Llama was the default and leading open source model that people were building around. Its ecosystem was benefiting the most from the entire open-source AI movement. When Mistral came on the scene, they started to have some amount of competition.

13:43And Mistral was not content to just have the best open-source model, they were obviously going after a GPT-4 level performance. Mistral has stated very clearly their goal to have a GPT-4 level model open-source this year. So perhaps in some ways, this big announcement from Meta reflects that type of competitive pressure. Now the last part of his announcement has to do with the connection between AI and the metaverse. There are, of course, a million pundits and critics who have for as long as Zuckerberg has been talking about AI, saying that it was just some big pivot away from the metaverse after crypto crashed over the last couple years.

14:12And so to some extent, Zuckerberg is probably trying to put a pin in that narrative. But at the same time, I don't at all think that Zuckerberg is just blowing a bunch of smoke up our butts when he talks about his vision of humans interacting with AIs. I think that he and the team at Meta genuinely believe that a big part of human social interaction in the future is not going to be with just other humans, but with synthetic humans in the form of artificial intelligence. There's certainly some early evidence that that's the case given how much particularly young people are using things like character AI as well as the traction of virtual boyfriends and girlfriends.

14:41And so on the one hand, Zuckerberg is shoring up a narrative, but on the other hand, I do think he's revealing part of the company's vision for the future. Now, the company reorganization bringing the AI research team fair to sit inside the product organization and put it all under a shared leadership sort of mirrors what Google went through last year bringing together their disparate AI teams as well. It also is representative of the incredible talent wars that are going on as companies compete for a very limited set of engineering resources. Chief Product Officer Chris Cox wrote in an internal note to employees, with this change, we elevate the importance of AI research as an essential ingredient to the long-term success of the company and our products.

15:18Alongside the major infrastructure investments Mark mentioned today, moving FAIR and Gen.AI closer together will mean a more coherent AI research portfolio and roadmap, with Llama becoming the primary launch vehicle for progress towards AGI, plus a streamlined development process for new technologies or novel AI research that navigate the legal policy and brand landscape consistently in an increasingly scrutinized space. TLDR on this reorg then is not that just two teams are coming together, but that Meta is putting AI at the very center of the product. That, I think, is the big shift. Now when it comes to takes on this, there are some folks who are absolutely incredulous.

15:51Jeffrey Ladish wrote, Mark Zuckerberg just announced that Meta is going full speed developing open source AGI. This is insane. You can reasonably debate how far you can push open weight models before the risks outweigh the benefits. I don't know where that line is, but AGI is clearly too far. One reason AGI is too far is that many AI researchers think we won't be able to maintain control of AI with human-level scientific R &D abilities. But even if you don't buy this, open-source AGI will present enormous weaponization potential, e.g. everyone could make pandemic viruses. We've made very little progress on making models resistant to malicious fine-tuning.

16:20For the foreseeable future, if you release model weights, you give people access to the full weaponization potential of that model. Maybe let's solve that problem before trying to build open AGI. Jeffrey Miller writes something similar. This is insane, he says. Open source AGI available to everyone? Hundreds of millions of Islamists want deaths to the West, a global caliphate, and Sharia law. Why would we want them to have all the access to human-level AI to help them plan terrorism? Others are very excited. Renji, the synthetic data maximalist, writes, It's so, so cool that Zuck wants to open source AGI, one of the coolest developments in the AI space since ChatGPT.

16:52Aravind Srinivas, the CEO of Perplexity, said, Open source AGI is an amazing vision. You are building a very powerful technology and actually aligning to what makes sense for the world. More people have a say in what makes sense and doesn't. Zuck and Jan LeCun are leading the revolution, and it's going to have an incredible short-term impact for the AI ecosystem as a whole. Meta gets a lot of s***, but there's literally no other company doing this today. Bindu Reddy writes, Open Source may win the AGI race. Just a year ago, a statement like this would have sounded ridiculous. In fact, folks like Ilya from OpenAI have said that Open Source winning the AGI race would be akin to someone in a small garage winning the space race.

17:24Now, a year later, Open Source is in the second position, and it may take the lead by the end of the year. Here's how we get there. Open source techniques can improve a base LLM by 5-10 % today. Every day we are moving that needle, so by the end of the year we could move that up to 15-20%. Meta just announced they are going to open source LLM3. Mistral has announced they are going to open source a GPT-4 class model. So assuming both Meta and Mistral open source their models and the rest of us have invented new methods to mix, match, and improve them, we may have a shot at catching up to GPT-5.

17:49The race will get intense post-GPT-5 release, and open source has a very strong chance of winning. There are thousands of talented researchers and startups who are working on this 24-7. I don't know if OpenAI will be able to keep up. Jan Peleg put an image of the Mark Zuckerberg reel next to Times Magazine cover, The End of Humanity. How real is the risk? He wrote, These images are six months apart. Imagine saying open-source AGI six months ago. Nine months ago, Yudkowsky calling for bombardment of data centers. Six months ago, the end of humanity on the cover of Time Magazine. Today, Zuckerberg saying in public, open-source AGI.

18:22Public opinion shifts fast. Now, I'm not so sure that public opinion has really shifted. In fact, what I think is that as public opinion has gotten louder, the folks on the open-source, techno-optimist, accelerationist side of this equation have made the determination that they also need to get louder. Ultimately, I think who wins that debate will have less to do with who can shout the loudest and more to do with how benefits show up in real people's lives. But that is a topic for another show. For now, that's going to do it for today's AI Breakdown. Appreciate you listening or watching as always.

18:53And until next time, peace.