OpenAI's Potential $5B Hole

0:00Today on the AI Daily Brief, another example of why OpenAI is perhaps the most capital-intensive startup in Silicon Valley history. Before that in the headlines, Mistral launches another open competitor to Llama 3.1405b. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes.

0:23Welcome back to the AI Daily Brief Headlines Edition. All the daily AI news you need in around five minutes. Just one day after Meta released its Llama 3.1 family of models, Mistral has hit back with another competitive open model that they're calling Mistral Large 2. Devandra Chaplow from Mistral writes, Super excited to announce Mistral Large 2. 123 billion parameters fits on a single H100 node. Natively multilingual, strong code and reasoning, state-of-the-art function calling, and open weights for non-commercial usage. The blog post that they announced this with was called Large Enough. And the company writes, compared to its predecessor, Mistral Large 2 is significantly more capable in code generation, mathematics, and reasoning.

1:05And indeed, the blog really focuses on the code and reasoning dimensions of this. They write, we trained Mistral Large 2 on a very large proportion of code. Mistral Large 2 vastly outperforms the previous Mistral Large and performs on par with leading models such as GPT-40, Claude III Opus, and Llama 3-405b. They also write that a quote significant effort was devoted to enhancing the model's reasoning capabilities. One of the key focus areas during training was to minimize the model's tendency to hallucinate or generate plausible sounding but factually incorrect or irrelevant information. This was achieved by fine-tuning the model to be more cautious and discerning in its responses, ensuring that it provides reliable and accurate outputs.

1:41Now, anytime Mistral launches a model, the developer community gets very excited. Indeed, in many ways, the battle for the standard bearer flag for open-source AI has been a competition between Meta and Mistral. Alex Banks tweets, The pace of AI innovation is relentless. Just 24 hours after LAMA 3.1 405b, Mistral announced their latest model. This is a much smaller model, one-third the parameters of LAMA 3.1 405b, yet large 2 performs equal or even superior to both LAMA 3.1 and GPT-40 across leading benchmarks. After sharing comparison benchmarks for the MMLU, Human Eval, and GSMA-K, Alex continues, this is incredible performance given the model's size.

2:19Foundation model competition has never been higher. If this isn't the catalyst for Sam Altman and OpenAI to release GPT-5, I don't know what is. In other parts of the open source community, people are still wrapping their heads around the new offerings from Llama. Vindu Reddy of Abacus writes, Mistral-Large 2 is good, but Llama 3.170b is insane. Vindu continues, We've updated LiveBench AI to include Llama 370b in Mistral 2. Interestingly, Llama 3.170b is the best model given its size. It's even better than Mistral 2. Another small update, we fixed a couple of bugs and now Llama 405b beats GPT-4-0, making it the second best model in the world.

2:53A fantastic week for open source AI. If you've been following along, you'll know that the early leaked benchmark suggested that Llama 405b was right up there in that GPT-40 Claude 3.5 Sonic class, and maybe even exceeding them, and that seems to be being confirmed by these independent tests. However, the Mistral announcement wasn't without its detractors entirely. Artificial Guy writes, This isn't a fight, this isn't shade. But honestly, Mistral Large doesn't make much sense right now. It's a model that most people can't run locally, non-commercial, only at Mistral API, and with a price of$9 per 1 million token output, it doesn't make sense compared to Llama 405b at$3 for 1 million token output.

3:30And the non-commercial license was a really big sticking point for many people. Andrei Burkoff writes, on July 23rd, Meta released a 405 billion parameter 128k token context multilingual model for commercial use. On July 24th, Mistral released a 123 billion parameter model restricted to research use only. I'm not sure what the goal of this release was. I will not even link the model because it's pointless. Has Mistral lost the way? Jeff Flaherty ironically points out that most of the people who have those H100s that are required for running Mistral Large 2 aren't necessarily non-commercial. He writes, Mistral responds to Meta's Llama 3.1 with a much smaller model that claims to perform slightly better.

4:06The license, however, doesn't allow commercial usage. Wonder what kind of research this will enable for all those non-commercial researchers with all those H100s. For those looking for signs, though, that this is all forcing the space to move towards the open, some have pointed out to OpenAI announcing that they were offering free fine toning for GPT-40 mini for the next two months. Many people took this as a signal of the pressure coming from these open source models. Overall, there are definitely some interesting vibes. Leaker extraordinaire Jimmy Apples writes, There's something in the air, a schizo vibe of hope.

4:35Let's get mathy. Andrew Curran points out that we're due for a big announcement from Google, and that there had recently been reports that Google's deep mind had made a big leap in math reasoning. All in all, a good week for model competition and a portent of exciting things to come. Speaking of models, Kling, the Chinese AI video generation model that took the internet by storm about a month ago, is now widely available. Initially, Kling was only available in China and required a Chinese phone number, but on Wednesday, July 24th, Kling tweeted, the moment we've all been waiting for is here, introducing the official global launch of Kling's AI International version 1.0.

5:08Any email address gets you in, no mobile number required. Once again, this has got to put pressure on OpenAI to release something when it comes to Sora. Over in the search engine wars, Bing has gotten an AI redesign, and like what we've seen recently from Google's AI overviews, of course in many ways feeling to imitate the example of perplexity, Bing now puts the AI-generated answers right at the middle as the main part of the experience, while moving traditional search results over to the side. As The Verge points out, Bing's new layout goes beyond general summaries. It's basically much more equivalent to a customized Wikipedia page that's created on the fly, and at least that particular reviewer is worried that it will be overwhelming.

5:47That said, I'm certainly willing to give it a try, and I think from Microsoft's perspective, Bing has to make some bold moves to try to carve out space in a search market that is still obviously dominated by Google. Lastly today, former NFL quarterback Colin Kaepernick has launched a new AI startup called Lumi Story AI. It's backed by Reddit co-founder Alexis Ohanian and, quote, plans to use AI's capabilities to give aspiring creators tools they might otherwise not have access to. Says Kaepernick, it allows us to help fill in the skill gaps of creators. We're now building in that direction to try and open that up and democratize storytelling.

6:19In his post announcing their investment, Alexis wrote, Lumi is much more than a cool AI demo. It's a place where anyone, no matter their skill set, is empowered to build, publish, and actually monetize their stories. Too many new AI tools are forgetting about that last part. How do they do that? we're talking end-to-end story creation, simple AI tools, physical and digital publishing, and built-in merchandising. It's everything a creator needs to make and scale their work, even without a network in Hollywood or the dollars to pay a big team. That's how we level the playing field. That's how we get more people in the room telling more diverse stories.

6:48Lumi is about to turn every storyteller into their own personal Disney. And so once again, it appears that we have here an example of the trend away from just general AI interfaces towards actual full product suites that have a vision of how to integrate AI to make things happen. That, however, is going to do it for the AI Daily Brief Headlines edition. Next up, the main episode. Today's episode is brought to you by Venice. The leading AI companies store your entire conversation history and attach it to your identity forever. That's every question you ask, every answer you receive, every image you generate, every thought you share with the machine, it's all being spied on.

7:23If you trust all the companies, hackers, and NSA board members that will ever have access to your AI conversations, then rejoice, for you are well-served. For the rest of us, Venice is an alternative. Venice is a powerful AI app for text, image, and code generation that respects you as a sovereign individual and believes privacy and free speech are not only human rights, but necessary for civilizational advancement. Private, permissionless, and uncensored, you can try it for free without an account. AI Daily Brief listeners receive a 20 % discount on Venice Pro. Visit venice.ai slash nlw and enter the discount code nlwdailybrief.

7:54That's nlwdailybrief. All one word. Today's episode is brought to you by Superintelligent. As you guys know, Superintelligent is a platform we are building to help everyone, individuals and teams maximize their use of AI. We help you figure out how to use AI tools, as well as what to use AI for. And this is really important. The whole goal of Superintelligent is not just to give you tutorials and lessons, but to show you how other people like you are actually getting value from AI right now. For those of you who are still out there working, learning and grinding deep in the summer, I'm excited to share our best offer ever.

8:31If you sign up with code YEAR50 right now, you will get 50 % off the already reduced annual price. That means you'll get access to Superintelligent for a full year for less than$100. Again, that code is YEAR50 for 50 % off the already reduced annual fee. This particular code is going to expire in just a week or two. So head on over to besuper.ai and check it out. Welcome back to the AI Daily Brief. Today, nominally, we are talking about a new report from the information that suggests that OpenAI could be on track to spend$5 billion this year. We'll get into their arguments and what sourcing they have for this, but we're going to situate it in the broader context of whether Wall Street is starting to turn and view AI as being in a bubble.

9:16Let's get into the reporting from the information first, though. If you listen to this show regularly, you probably know that the information is one of the best sourced, if not just straight up the best source publication when it comes to insider information about the AI industry. Their analysis is based, they say, on undisclosed internal financial data, as well as people involved in the business. They write, our conclusion pinpoints why so many investors worry about the profit prospects of conversational artificial intelligence. Our results also underline the question of whether those companies will eventually need to charge higher prices for their technology if they can't find a way to reduce the cost of developing and running AI.

9:49So, with that in mind, let's get into the details. The information argues that as of March, OpenAI was on track to spend around$4 billion this year on basically renting servers. That comes from a person with direct knowledge of the spending. And that is, of course, just for running ChatGPT. That's not about training costs. Those training costs, including their new deals where they pay for data, could be as much as$3 billion this year. A person with direct knowledge of the decision said that last year, OpenAI ramped up the training of new AI faster than it had originally planned. The company had earlier planned to spend around$800 million on such costs, but ended up spending considerably more, and so the information is estimating that those costs will double this year.

10:27Next, they estimate OpenAI's 1 ,500 strong workforce to cost them around$1.5 billion, but this seems to be one of the areas where they are least confident, calling it a guesstimate to be sure. OpenAI had previously projected workforce costs of$500 million for 2023, while doubling headcount from$400 to$800 over the course of that year. Given that it's nearly double that workforce again and is likely to add even more people in the second half of this year, that's where that$1.5 billion estimate comes from. So that puts OpenAI's operating costs this year at around$8.5 billion, or at least as high as$8.5 billion.

11:00The revenue story is one we've heard before. ChatGPT recently was on pace to generate around$2 billion annually. Although as the information flags, the issue that OpenAI faces of people using a free version of ChatGPT raising computing costs without generating revenue could be exacerbated this year when and Apple begins rolling out ChatGPT on the iPhone. As of March, the information writes, OpenAI's API business was generating around$80 million per month, and so all in all, they estimate that its full-year revenue could be between$3.5 and$4.5 billion, depending on sales in the second half of this year.

11:30From there, it's just a matter of simple math. Potential costs of up to$8.5 billion from revenue of up to$4.5 billion, and you get losses of between$4 and$5 billion. Now, it's not like OpenAI isn't clear-eyed about this. Sam Altman has previously described the company as the, quote, most capital-intensive startup in Silicon Valley history. But as the information points out, quote, it means OpenAI will need to raise money soon. Where does this put them relative to competitors though? Well, the piece argues that although OpenAI may be burning a lot of cash, it's better off than some of its rivals.

11:59They point to a discount in what it pays Microsoft for renting its servers and its better revenue profile than its competitors like Anthropic. The piece estimates Anthropic's revenue to be between a fifth and a tenth of OpenAI's, if their burn may be around 50 % of OpenAI's. According to a person who saw the figures, earlier this year, Anthropic had projected spending$2.5 billion on computing costs alone. Anthropic projects that it'll reach around$800 million in annualized revenue this year, but shares some of that with Amazon, meaning that its net could be between$400 and$600 million, leading to the information's conclusion that, quote, although Anthropic is growing faster than OpenAI, it is nowhere near as efficient.

12:34So what comes next? Well, in classic business fashion, OpenAI is looking at ways of reducing costs and generating more revenue. The reports are that OpenAI is planning to launch a search engine as well as a computer using Agent, both of which would handle different types of multi-step tasks. They also anticipate GPT-5, whatever it ends up being called, due out before the end of the year, which could be another boon to growth. So that is the OpenAI story. Nothing in there I think is particularly surprising, but it is expensive, and it of course then plays into the larger debate around the ROI from AI that we've been discussing over the past few months.

13:08The Wall Street Journal yesterday published a piece called Google Fails to Wow as AI Bills Mount. Advertising business faces tough growth comparisons while AI spending continues to surge. I mentioned that the interpretations of Google's recent financial results were sort of a Rorschach test, and the Wall Street Journal here is kind of trying to play both sides. They start the piece, it's good to be Googled these days, but it isn't easy and it will keep getting harder. And effectively, the story that they're telling is revenue growth that is right around expectations, although a little bit better, but increased capital expenditure, particularly on AI infrastructure, that just seems to continue to be going up.

13:41Don't expect to see a shift in Google strategy anytime soon, however. CEO Sundar Pichai said during the earnings call, look, obviously we're at the early stage of what I view as a very transformative area. The risk of underinvesting is dramatically greater than the risk of overinvesting for us here. Mark Zuckerberg of Meta has made a very similar point. And an additional point I'll make, which I do think is germane to the whole bubble conversation, is that these are not levered companies getting into some risky area that's more smoke and mirrors than real opportunity. Alphabet is sitting on$98 billion in cash.

14:13That gives it a lot more room to run when it comes to these sorts of big long-term bets. But the bubble talk keeps increasing. The Washington Post writes, Big Tech says AI is booming. Wall Street is starting to see a bubble. The industry has rushed headlong into AI and stock market investors are following them, but a growing number of analysts are skeptical. Then again, the quote-unquote growing number of analysts actually are pretty much the same voices that keep saying the same thing over and over. Jim Covello, who is the main skeptic in that Goldman Sachs piece that we did a deep breakdown on recently, is the lead quoted analyst here.

14:44What's undeniable is that this week has been rocky when it comes to public markets. The BBC writes, shares drop in US and Asia as AI stocks slide. On Wednesday, the S &P 500 lost 2.3%, and NASDAQ fell 3.6%, which is its biggest one-day fall since 2022. In the same way that for the last couple of years, all the gains have been driven by big tech, and particularly big tech that's touching AI, the losses this time were also driven by those firms like NVIDIA, Alphabet, Microsoft, Apple, and Tesla. Said Jun Bai Lu, a portfolio manager at Tribeca Investment Partners, investors are now becoming more concerned about all this expenditure with AI without the revenue benefit.

15:18I don't think this will mark the start of the disbelief in AI. It just simply means investors will focus more on returns in the space than just buying the whole sector. And that, my friends, would be a completely reasonable thing. One of the weirdnesses of AI is the fact that Wall Street is involved so early because of the presence of big tech at such an early stage in a new technology's life. Usually there's a decade or more where something like generative AI would be incubated in the private market bastion of Silicon Valley before it came to public markets. But because of the particular dynamics of AI, that's just not the case this time around.

15:48And Wall Street isn't necessarily having the easiest time figuring out how to price things. I think ultimately we need to break apart the conversation around the market bubble and valuation bubble from AI value and utility. For now, though, that is going to do it for today's AI Daily Brief. Until next time, peace.