OpenAI's AGI Committee

0:00Today on the AI Breakdown, we're discussing OpenAI's AGI committee. Before that on the brief, how the AI startup funding landscape is changing. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our Discord, our YouTube channel, and our newsletter.

0:24Welcome back to the AI Breakdown Brief, all the AI headline news you need in around five minutes. Today, many of our news stories have to do with the startup landscape in AI. And we're going to start on more of a meta note. Now, in case it wasn't clear that artificial intelligence and generative AI in particular represent something different than just another sort of startup movement, the way in which the startup financing space for AI startups has evolved over the last year has demonstrated that they are just their own different beast. You might have heard me discuss a post from Sam Hogan from back in July, where he wrote, Six months ago, it looked like AI and LLMs were going to bring a much-needed revival to the venture startup ecosystem after a tough few years.

1:05With companies like Jasper starting to slow down, it's looking like this may not be the case. Basically, what Sam was observing was a couple things. First, that AI startups, especially post-ChatGPT, had been somewhat more impervious to the shifts that had been happening ever since the Fed started hiking rates and the broader economy moved out of the zero interest rate era. Specifically, generative AI startups were still able to raise big amounts of capital at very high valuations, almost entirely based on potential and team and opportunity rather than actual demonstrated growth. Now specifically, Sam was identifying that the wrapper startups, companies that were sort of just building on other people's APIs, were having a really hard time, and he was writing that before the big labs like OpenAI really started to compete with many of those companies.

1:45He also noted that enterprise companies that might have in the past simply partnered with startups were actually building their own AI solutions, customizing open source models, and generally being more proactive than they had been with previous tech changes in the past. Today, there was a somewhat related post on Twitter from Gokul Rajaram from DoorDash that he called investors of last resort. And in it, he discusses how important the role of big tech companies has been and seems poised to continue to be when it comes to startup funding and taking roles that previously had been just for venture capitalists.

2:15Gokul writes, The revelation that Character AI is speaking with Google about their new financing round is not surprising in the least. Despite an incredibly engaging product, i.e. 4 million daily active users with the average user spending 2 hours per day on the product, the company has extremely high burn since they are building their own models, a competitive advantage in the long run, monetization is non-existent today, although obviously lots of promising future avenues, and competition is on the horizon as modes switch from text to images and video, and Meta readies its own avatars. It's impossible for VCs to underwrite a$5 billion valuation for the company.

2:47The only viable investors at this stage for character are Alphabet, Amazon, Microsoft, NVIDIA, and a handful of other large corporates. They are price insensitive and are driven by completely different considerations, such as GPU cloud enterprise customer Lock-In, then venture capitalists are. As we've seen with OpenAI and Microsoft, Anthropic, Google and Amazon, Stability AI, Intel, and Cohear, NVIDIA and Salesforce and other cases, any company building their own foundation models will ultimately need to go this route. It's just far too expensive to train these models, and the ROI and revenue from inference is too far out for the venture model to support.

3:18The OpenAI-Microsoft partnership seems to have paid dividends for both parties. With the proliferation of this model, curious to see the implication of this on exits, future financing rounds, strategic flexibility, and of course, long-term commercial success. So one thing that he doesn't get into in that is why the economics don't make as much sense for venture capitalists. But I think that there's probably a lot to explore there. Some of that has to do with the prospects for companies to go exit into the public markets, Some of that has to do, I think, with the overall availability of capital or lack of available capital to the venture capital space as a whole.

3:48In other words, there is nothing theoretically that would make it, quote unquote, impossible for VCs to underwrite a$5 billion valuation for character. It's just that in his estimation, their incentives now point them in a different direction, as opposed to the incentives of the big companies like Google and Amazon. Now, this is a phenomenon that I've been watching more broadly and have been talking about on this show as well, and that I think is going to have a major impact in how startups are built in the AI space. I don't know yet exactly all of what the implications will be, but if you come back for the main episode or keep listening in the case of the podcast, we'll talk about why, even though the Microsoft OpenAI partnership has set the template, it's certainly far from uncomplicated.

4:24Moving on, however, to some more startup news, the information reports that YouTube co-founder Chad Hurley is back 18 years after he started building YouTube with a new startup, ITEL, which is using AI in the short-form video space. The company is apparently 5 months old and has raised a seed round from investors including Ron Conway and Kevin Hart's Astar Capital. From the information, ITEL's product is intended to speed up the video creation process by producing scripts based on text prompts, as well as providing inspiration for content creators. Using ITEL, users will also be able to record their characters' AI-generated lines, and ITEL will generate AI video content based on those recordings.

4:59Now, according to the information sources, ITEL is not building their own AI from scratch, but is working with existing AI models. And at least to the information, the story here is less about ITEL specifically and more about it being an example of another quote-unquote seasoned tech executive jumping into generative AI. Meanwhile, on the other end of the startup lifecycle, Airbnb has acquired an AI startup called Game Planner AI that basically no one knows what they do. Airbnb said on Tuesday that it had acquired the company, which was a 12-person startup for an undisclosed sum, although other reports have put the number at around$200 million.

5:32Notably to many was that Game Planner AI was co-founded by Adam Chire, who was also previously one of the founders of Apple voice assistant Siri. Said Airbnb CEO Brian Chesky in a statement, AI will rapidly alter our world more than any other technology in our lifetime. So again, while we don't know the details here, it seems likely that Airbnb views the future of travel and experiences as involving artificial intelligence in some meaningful way. Speaking of big companies that are all in on AI, popular team and workspace tool Notion has been for some time a leader in the AI space and has just introduced something they're calling Q &A.

6:07Q &A is basically an AI-powered search experience that allows members of teams to theoretically sift through a ton of information across all of the wikis, projects, documents, etc. that are contained in Notion to get answers quickly. Now, I should caveat that Notion is a current sponsor of the AI breakdown. You probably heard their ad yesterday, and they have a few more ads coming up. But I will also un-caveat and say that as I've discussed frequently, I think the big theme of this fall going into next year? Is AI actually integrating into the workflows in the software that we're using? And Notion is, for me and many others, a primary tool that is doing exactly that sort of integration.

6:41Plus, I'm far from the only person who's excited about this release. Balaji Srinivasan writes, Notion Q &A came out. It's already insanely good. Yeah, it's quote-unquote just a better search box, but this is what we need for email and PDFs too, understanding not just indexing. Now, of course, Balaji being Balaji, he also asked a thought-provoking question, writing, I wonder if there's a way to do AEO, like SEO to write documents or entire wiki such that AI can understand them better. Should be lightweight and not interfere with authoring flow, but if there are things one can do along the way, perhaps it'd help.

7:11Now, going back to the biggies and what they are building, Google's deep mind has gotten a ton of press today randomly with an AI that is focused on more accurate weather predictions. Basically, this tool called GraphCast uses decades of historical weather conditions to produce 10-day forecasts, and according to a study published in Science on Tuesday, that model outperformed the quote gold standard forecasting model from the European Center for Medium-Range Weather Forecasts on roughly 90 % of metrics tested. Now of course, this is not just some novelty feature for getting press, but something that has really important implications, especially in a world where weather events have gotten more unpredictable, more devastating, and thus more costly.

7:45If we're able to predict, for example, the path of cyclones, or anticipate severe temperatures better and farther in advance, we can take better preparatory steps. Over in the world of geopolitics, U.S. chip exports continue to have their desired impact, as well as the impact that chip makers were concerned about, as Reuters reports that Chinese giant Tencent is now seeking to find AI chip supplies rather than relying on companies like NVIDIA. Said President Martin Law on a call with analysts, quote, we will have to figure out ways to make the usage of our AI chips more efficient, and we will also try to look for domestic sources for these training chips.

8:17We got similar news from Tencent rival Baidu last week. Lastly, an interesting story out of Argentina, where the presidential election has come down to two final candidates. A lot of ink has been spilled trying to understand what the potential implications for AI in the US presidential elections next year might be, but we're kind of getting a preview down in Argentina right now. Both candidates have used AI to create imagery and posters, and the campaign of Sergio Massa has also gone even farther when it comes to deepfaking his opponent, Javier Mille. As the New York Times notes, the Masa campaign has also used the system to depict his opponent, Mr.

8:50Mille, a far-right libertarian economist and television personality known for outbursts, as unstable, putting him in films like Clockwork Orange and Fear and Loathing in Las Vegas. Continuing, they write, Much of the content has clearly been fake, but a few creations have towed the line of disinformation. The Masa campaign produced one deepfake video in which Mr. Mille explains how a market for human organs would work, something he has said philosophically fits in with his libertarian views. Now, when the New York Times showed Sergio Massa this particular deepfake campaign, they say that he appeared disturbed and said, quote, I don't agree with that use.

9:20Anyway, super interesting stuff out of Argentina. And again, if you're watching to see what will happen with the U.S. presidential elections in AI, that is a good situation to pay attention to. However, that's going to do it for this AI Breakdown Brief. Next up, the main AI breakdown. And now, a word from today's sponsor. Are you interested in how two top-of-mind trends, AI and crypto, can work together? If so, I have the perfect podcast recommendation for you. Web3 with A16Z Crypto, the chart-topping show brought to you by venture firm Andreessen Horowitz. Web3 with A16Z Crypto is your definitive resource for the future of the internet, whether you're already building in these spaces or simply curious about what's next.

10:00If you need a place to start, they recently released an excellent episode with Stanford cryptography professor Dan Bonet, and former Google Xer Ali Yahya, in conversation with host Sonal Choksi about the intersection of AI and crypto. From fighting deepfakes and proving humanity to large language models like ChatGPT, they cover it all. I highly recommend checking it out, especially if you'd like to learn more about how AI and crypto will impact our everyday lives. Beyond Crypto and AI, this show is for creators seeking more ways to truly own their work, for business leaders trying to prepare for the future today, and for innovators exploring trending tech topics.

10:34So go ahead, listen to Web3 with A16Z Crypto wherever you get your podcasts. Hello friends, quick note before we get to the main episode today. You may have heard me talk about this in October, but once again, I have opened up a very small number of micro consulting slots for the rest of November. These are basically one-on-one sessions with me that are designed to dig into your big questions around generative AI and how it can impact your career, your professional goals, or if you're an organization, how your company functions. It's designed to be short and super high impact, and these sessions are paid.

11:07I've got about three or four sessions left for November, and if you are interested, email me at nlw at breakdown.network for more information. Excited to help you bring AI to your big goals for 2024, but now let's get back to the show. Welcome back to the AI Breakdown. One of the things that we have been tracking here on the show is the increasing war of words between the AI safety or responsible AI community, however you want to define that, on the one hand, and the accelerationist technology-based community on the other. Now, the rhetoric in this battle has increased in severity, commensurate with the fact that AI is now firmly on the policy agenda.

11:46In other words, the stakes have been raised on the AI safety conversation because it could turn into specific policies that, again, those accelerationists or people in the technology community would view as fundamentally limiting to what they can build or what can be built in general. Today, we're going to talk a lot about an example of how that war of words is playing out, but I want to start with something that was almost throwaway, but which I think is a little bit more loaded than it at first seems. So Sharon Goldman, who writes about AI for VentureBeat, tweets, OpenAI says its six-member board will decide when we've attained AGI, a threshold which excludes Microsoft.

12:21What does this timing, context, and process of that possible future decision mean for all of us? She continues, The information was highlighted by OpenAI's Logan Kilpatrick, who was responding to critics calling OpenAI an arm of Microsoft. Kilpatrick said Microsoft does, get access to our models to use in their products, but they have no control over our strategy as a company. But the information also comes right after a new interview with Sam Altman in the Financial Times. Asked if Microsoft would keep investing further, he said, I'd hope so. There's a long way to go and a lot of compute to build out between here and AGI.

12:51Training expenses are just huge. Okay, so let's get into this a little bit more and why this whole Microsoft side of the conversation actually matters. First, what's important to understand about OpenAI's structure is that it's a little bit different. Basically, there is a non-profit that owns all of the for-profit IP, and that in the case of this designation of attaining AGI, it has specific implications. As VentureBeat writes, Thanks to a for-profit arm that is legally bound to pursue the non-profit's mission, once the six members of its non-profit board of directors decides AGI or artificial general intelligence has been reached, such a system will be, quote, excluded from IP licenses and other commercial terms with Microsoft, which only apply to pre-AGI technology.

13:32Now, as Goldman and VentureP point out, the definition of AGI is not something that there's a shared consensus around. Twitter user NLASOL says, So basically, if you hear anyone from OpenAI announce that they've attained AGI in the near future. It doesn't mean a program that can solve problems as well as a human. It means a program we don't want to give to Microsoft. Nice goalpost shift, y 'all. Now, of course, this relationship between OpenAI and Microsoft has had a ton of scrutiny. In a lot of ways, it's become a Rorschach test for how you feel about big tech and artificial intelligence in general.

14:03At the same time, it's undeniable that it has been influential in how the field has developed. We've subsequently seen big startup labs like Anthropic tie up with Google and Amazon, and if you listened to the brief this morning, you'll have heard why it seems increasingly like the big capital that's needed to train discrete models isn't really within the purview of venture capital and might have to increasingly come from these big existing corporate players. In that way then, the OpenAI-Microsoft relationship is not just about OpenAI and Microsoft, but also about a proxy for the rest of the field.

14:33I often think when I look at commentary around this partnership, particularly commentary that suggests that something is going wrong or that there's been a shift or that one of the two partners is walking away from the other, and I tend to think that it usually reflects not so much that anything has changed in the relationship, but more just the fact that they've chosen an inherently messy relationship in the first place. These companies are, by virtue of this partnership, deeply tied together in ways that make the ways in which they compete much more uncomfortable. But at the same time, at no point have they called it a proto-acquisition or a semi-acquisition.

15:07To hear everyone talk, the companies are partners but still discrete companies. In other words, I don't necessarily think there's something mutually exclusive about the fact that on Dev Day, Sam Altman told Microsoft CEO Satya Nadella that he thought that their partnership was the strongest in tech, but at the same time, both OpenAI and Microsoft in their own right are doing things that wouldn't be in the best interest of the other partner. Basically, I think this is just a part of the messiness, and they're just going to have to sort it out as it comes. Now, speaking of messiness, let's get to the Responsible AI Commitments Declaration that was released yesterday, which has generated an absolute boatload of chatter.

15:41Hemant Tanesha, the CEO of General Catalyst, tweets, Today, 35-plus VC firms and another 15-plus companies, representing hundreds of billions in capital, have signed the Voluntary Responsible AI commitments from Responsible Labs, RIL, the non-profit I co-founded. As chairman of RIL, I'm proud to unveil this today with tech leaders and Commerce Secretary Gina Raimondo in San Francisco. These commitments include, one, a general commitment to responsible AI, including internal governance, 2. Appropriate transparency and documentation 3. Risk and benefit forecasting 4. Auditing and testing 5. Feedback cycles and ongoing improvements In addition to the commitments, we are publishing a 15-page responsible AI protocol, a practical how-to playbook to help investors and startups alike fulfill the commitments.

16:23Since April, our steering group of cross-societal AI experts has worked hard to understand the unique considerations and relevant and realistic standards for technologists at any stage. We strongly believe in the power of AI to transform our world for the better. Our role as investors is to advocate for our startups and the innovation economy from day one. Everybody saw the executive order last month. The reaction in the Valley has generally been to denounce it. The reality is that right now it's largely just reporting requirements. However, there is a risk that devolves into regulation that slows innovation down and makes America and its businesses uncompetitive.

16:53But the right path forward is not to be antagonistic towards DC. We in the Valley need to learn that this is not about regulation versus innovation, but about innovation at the intersection of technology, policy, and capital. We have to embrace collaboration with our elected leaders. And as investors, we must hold ourselves accountable for what we fund and found. We are facing what is likely to be the quickest adoption of any new technology ever in AI. It's difficult to think about the regulatory framework that protects the pace of innovation, as well as protects against any nefarious use cases of technology.

17:20While we all learn how this role of technology evolves, it's important for us to be intentional about the mindset and mechanisms used in building early-stage companies as both investors and founders. We can't wait or kick the can down the road. Now, this also came with an announcement document, Responsible AI Commitments for Startups and Their Investors. The Responsible AI Commitments include securing organizational buy-in, i.e. implementing internal governance, fostering trust through transparency, i.e. documenting decisions about how and why AI systems are built or adopted, forecasting AI risks and benefits, taking reasonable steps to identify the foreseeable risks and benefits of technologies, audit and test to ensure product safety.

17:54Under that, they write, Based on forecasted risks and benefits, we will undertake regular testing to ensure our systems are aligned with responsible AI principles. Testing will include auditing of data and our systems' outputs, including harmful bias and discrimination, and documenting those results. We will use adversarial testing such as red teaming to identify potential risks. Finally, make regular and ongoing improvements. We will implement appropriate mitigations while monitoring their efficacy, as well as adopting feedback mechanisms to maintain a responsible AI strategy that keeps pace with evolving norms, technologies, and business objectives.

18:24Now, there were a few spattered quote tweets and posts of support, including this one from Brandon Goldman, who is a partner of AI safety at Lionheart VC and writes, This is an excellent step forward. I've been one in collaboration of safety commitments between investors and companies for Safer AI for a long time. I'm excited to see what happens here. However, by far, when it comes to Silicon Valley's builder community, the reactions were extremely negative. I'm John Massad, the CEO at Replit, said, this is at best busy work, and founders should be very suspicious of VCs who ask them to do busy work to support their virtue signaling.

18:57And this was a really common theme, that this was either A, proto-regulation, in which case it should be rejected, or B, toothless virtue signaling, in which case it should be rejected. A16Z's Martin Casado says, will never sign. Preemptively regulating innovations in computer science in ways that can hurt open source innovation and competition is wrong. PhD Casey Hanmer says, charitably, this is an attempt at forming an

19:27And indeed, uncharitably, Adapai writes, Always follow the money. Here's the shortlist of VCs struggling to raise their next fund, and hoping signing this and saying they're into AI gets at least one LP over the line. Now for entrepreneurs, this had maybe the exact opposite of the intended effect. Martin Skreli writes, New Do Not Call list just dropped. Investors of Last Resort would be another name. Indeed, then Martin continued, here is a much, much better list of investors with much more capital that aren't brainwashed. Some other investors appealed to history. Steven Sanofsky said, Everyone reading this should be thankful this mindset did not prevail at the advent of the microprocessor, the database, the PC, the internet, mobile phones, and so on.

20:03Each step, there were those that championed caution over innovation. And it wasn't just A16Z, who was using this as a wedge to say we're different and you should work with us instead. Other investors also signaled their pledge not to sign these commitments. Julie Fredrickson wrote, I've signed nothing. I make no pledges to any ethical standards set by anyone else but my own conscience and my faith in other men and women of conscience. Julie also added, Remember when Google's public relations campaign was Don't Be Evil? How'd that turn out? Maybe we can learn some lessons and apply to concerns about artificial intelligence.

20:32Now, outside of just the foundational principle kind of arguments, some folks tried to focus on specific problems. Fast.ai founder Jeremy Howard wrote, The big problem is point four, which is audit and test to ensure product safety regularly. Jeremy continues, It is literally impossible to ensure safety of a general-purpose model, and attempts to do so are likely to reduce safety. Jeremy actually also wrote a long piece about this back in July called AI Safety and the Age of Disenlightenment. Model licensing and surveillance will be counterproductive by concentrating power in unsustainable ways.

21:03Now, Balaji weighed in on this as well, writing, Free internet means free AI. I like Hemant and many of the people on this proposal, but fundamentally disagree with the philosophy of capitulation therein. We will fight government control over compute with everything we have. Because we have just been treated to the mother of all bait and switches, where AGI was used to somehow justify a new TSA. The true purpose of this executive order has nothing to do with avoiding existential risk, everything to do with gaining existential control. Remember, this government is the generator of existential risks, not the regulator thereof.

21:33As there is still no accountability for funding the development of COVID-19 in a Wuhan biolab. Exactly how can it regulate anything when it can't regulate itself? Rather than immediate capitulation, this coalition would be much better served if it was A. Suing the federal government over the illegitimacy of an order that essentially regulates speech and logic itself, and B. Working with smart governors in pro-tech states abroad to set up intelligent pro-AI environments. The fight for free, decentralized AI will be like the fight for encryption and a free internet. And we will win, because 640k of compute isn't enough for everyone.