In short
Robinhood CFO Shiv Verma discusses Robinhood’s second publicly traded venture fund on the NYSE, shifting from late-stage to early-stage exposure via YC incubator partnerships, including a 20% performance fee on realized gains and how NAV is set using SAFE caps, valuation committees, and third-party services. Analyst Brent Phil reviews why CoreWeave and Nebius surged after earnings, arguing AI demand still outstrips supply, with CoreWeave seeing sold-out capacity, shorter contract durations, higher prices, rising margins, and new on-demand inference demand; he compares CoreWeave vs Nebius and claims enterprise AI deployment is still early. CEO May Habib of Writer explains Palmyra X6, a cheaper enterprise AI agent model (52% lower cost, faster task completion, agents running 8+ hours) plus a “harness” for QA/guardrails and watermarking stance. Reporter Rocket Drew covers the compute crunch for “Neolabs” training their own models: spot/on-demand capacity dried up, contract terms lengthened to 1–3+ years, prices up ~50%, and examples include a robotics foundation-model team waiting months and a smaller research org needing ~60 chips.
Guests (backgrounds)
Shiv Verma, CFO of Robinhood; Brent Phil, Senior Analyst at Jefferies; May Habib, CEO of Writer (Ryder); Rocket Drew, AI and robotics reporter at The Information.
Written by AI. May contain mistakes. Listen to the episode to check what was said.
Chapters
Tap a time to open that second in VODiscussion with Shiv Verma on Robinhood's New Venture Fund
0:56 to 4:50
Shiv Verma discusses the new venture fund's structure and focus on early-stage companies.
“I want to bring on CFO Shiv Verma to talk about the calculus behind this fund and also the company's recent results.”
Brent Phil Analyzes CoreWeave and Nebius Growth
4:50 to 14:00
Brent Phil evaluates the booming businesses of CoreWeave and Nebius amidst high demand for AI.
“And I think what really stood out is they're shortening their contract durations.”
Introduction of Brent Phil
14:00 to 14:22
Brent Phil, a senior analyst at Jeffries, shares insights on current market views.
Introduction to Writer's New Model
14:23 to 14:54
Introduction to Writer's new flagship AI model, Palmyra X6, and its advantages.
“a new flagship model today entitled Palmyra X6.”
Cost Efficiency and Sovereignty in Enterprise AI
14:55 to 17:08
May Habib discusses the importance of cost efficiency and sovereignty in AI adoption.
“Anybody who's spending time with the enterprise these days knows that cost efficiency, along with sovereignty, is just the number one issue for folks now.”
Training Models and Market Perceptions
17:09 to 18:50
Discussion on the training of models and the changing perceptions of enterprises toward open weights.
“You know, a harness is really, you know, all of those things and more.”
The Race in Open Weight Models
18:51 to 20:26
May explores the competitive landscape of open weight models and U.S. companies' chances against China.
“And so now just shifting to the race in open weight models broadly, I mean, we had NVIDIA come out earlier this week with their latest open weight model or a version of it, I should say.”
Watermarking in AI and Transparency
20:27 to 22:57
Discussion on watermarking in AI and the need for transparency in AI-generated content.
“NVIDIA is certainly a leader in, you know, what they're trying to do.”
Control Issues in AI Labs vs. Enterprises
22:58 to 24:16
Exploration of the control enterprises desire versus what AI labs offer, and implications for the future.
“I mean, this is a watermark that travels, you know, with the text.”
Writer's Business Update
24:17 to 26:28
May Habib provides an update on Writer's growth and customer commitments over the past year.
“I mean, how do you think this all plays out ultimately?”
Show all 18 chapters
Palmyra Model's Inspiration
26:29 to 26:52
Discussion on the inspiration behind the name Palmyra for Writer's new AI model.
“Oh, well, before you go to the name, Palmyra, What's the inspiration behind the name of the model?”
Introduction of Rocket
26:53 to 27:12
Rocket, the AI and robotics reporter, joins the show to discuss Neolab's compute challenges.
Neolab's Compute Challenges
27:13 to 28:03
Discussion on the specific challenges Neolab faces in securing compute resources as a startup.
“So that introduces a lot of additional complications.”
Challenges of Compute for Startups
28:03 to 29:25
Learn about the difficulties startups face in securing compute resources for AI models.
“So your future compute needs, say, three years down the line, depend a lot on how your early experiments go.”
Market Changes in Compute Needs
29:25 to 30:59
Explore how the market for compute resources has shifted significantly for startups.
“These days, there are providers that are pushing for one to three years or three to five years.”
Neolabs and Their Compute Strategies
30:59 to 33:09
Understand how Neolabs navigate the complexities of acquiring computing power.
“reference class because they're working with large models, they're doing training, and they're also like doing pre-training experiments, which is a really compute intensive part of the training process.”
Neoclouds and Their Business Decisions
33:09 to 34:13
Analyze how neocloud companies are adjusting their strategies amidst the compute crunch.
“Are the neoclouds, are they addressing this issue at all?”
Future of Compute Availability
34:13 to 35:38
Speculate on when the current compute crunch may ease for startups and Neolabs.
“So Rocket, what's your sense on when this crunch starts to ease for the smaller startups and the Neo Labs?”
Transcript
Automatic transcript. May contain errors.0:13Welcome, everyone, to The Information's TITV. My name is Akash Pasricha. It is Thursday, August 13th. Today on the show, Robinhood's CFO joins us from the New York Stock Exchange to discuss their new publicly traded venture fund vehicle. We'll then dive into why NeoCloud's CoreWeave and Nebius are surging following their latest earning results and what the numbers signal for AI infrastructure. We'll also take a look at how a new AI model from Ryder is designed to cut the cost of running agents. And to close out the show, our AI and robotics reporter Rocket Drew will share a bit about how NeoLabs are dealing with the compute crunch.
0:52It's going to be a great show, so let's get right on into it. Robinhood is debuting its second publicly traded venture fund vehicle today on the New York Stock Exchange. I want to bring on CFO Shiv Verma to talk about the calculus behind this fund and also the company's recent results. Shiv, welcome back to the show. It's great to have you here. Thanks for having me. Very excited to be back. So let's talk about this second venture fund here you guys are debuting. The first one had some big names in it, Databricks, OpenAI, Mercor, and Ramp. What is in this second fund now? Yeah, absolutely. So the first fund was focused on late-stage growth companies.
1:32Many of them you mentioned. What we heard from customers is they actually want exposure to early-stage companies. So what this fund does is it focuses on early-stage in partnership with YCC, one of the biggest incubators in the world. And so we're giving access to a new asset class using some of the same vehicles as the first fund. Now, I want to get into the structure of the fund because these are smaller companies. One thing that stood out to me looks like there's a performance fee on there, 20%, which it didn't have in the first fund. Walk me through the decision to add that. Yeah, great question.
2:09So traditional venture has a management fee and a performance fee. Given this fund is much more similar to traditional venture, we thought it'd be important to have it as well. Now, it's really important to remember the performance fee only is charged on realized gains. So if we have the next Airbnb, Stripe, DoorDash, etc., and that comes into this fund and has a liquidity event, then we will charge the fee. And so we took the venture model and now we have daily liquidity. You don't need to be accredited. And so we think this is a better fund than traditional venture. How do you go about calculating the net asset value for this fund, given that the companies are so much smaller?
2:52I mean, YC companies, you know, this is the startup world we're talking about here. So it's hard to put a value on them sometimes. So how do you mechanically do that? Yeah, great question. So every fund that we invest in is typically done through a safe. So safe are done for early stage companies. This means the valuation is not set until the company raises at a future round. We also put a cap on it so that if they raise at a large valuation, our investors get better. We have an independent valuation committee that looks at every single company. And then we have a third-party valuation service that helps value them.
3:26And so we do our best to make sure that customers are getting actual NAVs. We then report every quarter. It's SEC registered. And so even though these are early-stage companies, there's a very deliberate process to try to value them. And how much ownership do you end up taking in these small companies then? Yeah, it's a great question. So typically we might have half a percent to one percent ownership. And so for an early stage company, that's actually quite large. If you think about companies that have come out of Y Combinator, such as DoorDash or Stripe or Airbnb, they can go public in the billions or tens of billions of value.
3:59So getting a ownership stake very early on is really powerful. Is it all AI companies or give us some of the names here?
4:11So the company is a mix of different portfolios. So it is some AI companies. We put out some information or perspectives. It's about 50 % are focused on AI infrastructure. There's also companies doing real hard tech, so building physical things. There's fintech companies. There's healthcare companies. So it kind of spreads the wide gamut. And for us, we're just trying to look for the best companies in each individual batch. Great. Well, Shiv, it's an exciting fund, and I want to congratulate you on the debut of it. We'll have you back on when you guys are announcing the third fund, whenever that's coming.
4:44That is Shiv Varma, the CFO of Robinhood, here on TI TV. core weave and nebius's businesses are both booming core weave revenue more than doubled in the quarter from last year nebius revenue is up more than fivefold for more on those results i want to bring on brent phil senior analyst at jeffrey's brent welcome back to the show it's great to have you here thanks for having me okay so core weave let's start there what stood out to about their results corley continues to see uh massive demand uh the demand is outstripping supply uh like microsoft amazon google uh most of the hyperscalers have been saying that customers are screening for more demand so this concept that uh that we're done with ai is kind of uh silly in terms of what wall street's been saying uh the the demand environment remains incredible core We've said they're basically sold out.
5:41And I think what really stood out is they're shortening their contract durations. They're seeing better unit economics on these contracts because they're able to raise price. And as they raise price, customers aren't pushing back. They're saying, we'll take more. So margins are going higher. And I think there was a fear that Wall Street had that they were not going to be able to make money, that this would just be a top-line revenue acceleration with no bottom-line profit. And what CoreWeave is telling you is that's exactly the opposite. You're seeing accelerating revenue. You're seeing good margins.
6:19And we saw that across all the other hyperscalers. Microsoft, Amazon, and Google saw accelerating growth and higher operating margins. So I think what we're seeing in this age of AI, that the fear of, hey, it's really expensive, you'll never make money, I think that the big four just told you that it's not true, that you can have growth and profits together. I think the other thing that's interesting for CoreWeave is that they're seeing broadening demand. It's not just coming from the hyperscalers. Obviously, they built for Meta. They built for Microsoft. But they're actually starting to see other customers like Caterpillar and other smaller companies come in and build.
7:01They built an on-demand inference business where you can just basically turn up inference through them on demand. That business is new. It's booming. They're seeing great demand on that side. So overall, really good numbers, good expectations. Wall Street's view was really negative going in. I know I don't think you cover Nebius, so I won't ask you specifically about those results. But how do you sort of compare and contrast these two companies? I mean, Nebius, their results revenue was up 450%, I think it was, compared to last quarter. They are a bit earlier, it looks like, in their build in terms of their profitability metrics, stuff like that.
7:47How do you compare and contrast these two companies? Yeah, I mean, I think right now what Corey Lee will say is that they're building for the best companies on the planet. and nothing against the other competitors, but their view is that we're building for the biggest of the big. And so the analogy would be from their perspective is they built the John Deere tractor and others are building weed whippers and lawnmowers for the mass market. So nothing wrong with that, just a different approach. Remember, CoreWeave serves and powers some of the most sophisticated enterprises in the world. meta microsoft google go through the list there's a handful of others that are on their their web page that of companies that they they power so um there have been tied uh to a smaller number of very very large customers and so i think you know there's different strategies and obviously what we're seeing is there's room for multiple vendors right you've seen oracle get into this market and helped open AI.
8:54You've seen many others take different approaches. I think what we're seeing is most of these approaches are working because the demand environment is so good, and there is no supply. So let's get to the big question then. I mean, the demand is great right now. Supply is low. It's all great news for the NeoCloud, certainly in terms of pricing. How sustainable do you think this growth will be for their companies? right now we're seeing no slow down capex goes higher for all the vendors across the board we're seeing consistent patterns of of the pickup and demand when you look at enterprise ai deployments uh it's tiny like we're talking like single digit agent deployment at large enterprises right the tech companies you know all laugh they're like what do you mean we ai is everywhere well like you go into a big bank an insurance company a big airline i mean like they've scratched surface in AI deployment.
9:47And I think what we've said repeatedly is that having followed tech for 25 years, there's no boards that ever talked about Web3, Bitcoin, the move to cloud, that none of them talked about it. They're all talking about AI, from financial services to healthcare, you go across the board. So I think we're in early stage of deployment. Remember, this is glacial, doesn't happen overnight, big enterprises that are regulated, can't just jump in and cannibal into the AI pool, they do it very slowly. And because the demand we're seeing today, basically on putting one toe into the water, this gives you a signal that it's going to be sustainable for multiple years.
10:32Now, are we in a bubble? Are we going to overbuild three, four, five years out? I think that's the debate. No one can see that yet because right now, everything sells out. Prices are going higher. Customers are saying there's an ROI. We're seeing backlog and record highs far exceeding the CapEx dollars that are going in. So when you have revenue that's exceeding the CapEx dollars, how do you say it's a bubble? And how do you say it's a bubble when you have the CEO of Medtronic saying doctor's eyes get tired in the afternoon, never get a checkup. AI never gets tired. It's going to save lives. Aviation companies saving lives in the sky.
11:07Financial services firms making their financial agents more productive. Right. So if there is overcapacity that is built and these neocloud companies would be at the center of it, I mean, what numbers do you look at in terms of looking for signs of concern? I mean, is this operating costs starting to rise faster than, well, I mean, I'm pausing here because right now it's, you know, profitability is not the priority here. But what metrics specifically do you look at to say, hey, overcapacity is starting to be an issue? I think when you look at this and you say pricing goes lower, all year we've heard pricing from Q1, Q2 into Q3, and now pricing is going higher, right?
11:55So is pricing falling out of the sky? No, it's rising. How's the demand picture? It's rising. What are you seeing in terms of revenue growth acceleration? Not just for the NeoClouds, but for Amazon, Google, Microsoft, Oracle. They're all accelerating. You look at the actual split between revenue intake and dollar spent. We've said this repeatedly. There's a huge, you can drive a semi sideways through this. It's not close. So we think right now, those are the signals, which are if your revenue is exceeding that of the cost to build, as a financial analyst, what we're seeing is this is a good return.
12:36And every single company has showed you improving operating margin. when that starts to stall when the demand picture stalls when you get into the enterprise and you start rolling on enterprise ai and everyone's like this is this isn't working or this is a joke and this is hocus pocus and these agents are off you know playing golf when they should be at work again we're going to have rogue agents then we start to concern get concerned we don't have any signals of that right last question for you brent uh any sign that core review is going to diversify away from NVIDIA in the near future? Any expectation from you?
13:11I don't think that they have any incentive to disconnect their rowboat from their massive luxury yacht. Zero reason to disconnect at this point. But, I mean, that being said, I mean, there's other boats on the horizon too, right? I mean, no? I think what Corey said is they're agnostic. They put a press list out about Kimi before anyone was talking about Kimi. Like they embrace all models. They'll embrace all semi-infrastructure, all hardware, all liquid cooling, wherever you want to build. They're open to their own data center. They're going to part with other people that have their own data centers.
13:49They're open. And I think they're taking an agnostic view. And this is what Nadella did at Microsoft, you know, when they started Azure. Have an agnostic view. Work with everyone. Don't just say we're only working with one nail to build a house. You want to have multiple nails. um so no i i don't i don't see a real change i think their view is the right view which is embrace everyone but like do you want to be tied to jensen's yacht i would like no reason to disconnect right right great well brent i want to thank you for coming on that is brent phil senior analyst at jeffrey's here on ti tv writer an enterprise focused ai marketing and sales company released a new flagship model today entitled Palmyra X6.
14:34It also made upgrades to its software. It says we'll make the platform cheaper for its customers. To break that all down, I want to bring on May Habib, CEO of Rider. May, welcome back to the show. It's great to have you here.
14:46May Habib:Hey, Akash. Thanks for having me. So Palmyra X6, I'll let you explain the name in a minute here, but tell us about the rationale here to release this new model and what's different about it. Yeah. Anybody who's spending time with the enterprise these days knows that cost efficiency, along with sovereignty, is just the number one issue for folks now. I think the adoption of and maybe lack thereof of fable in the enterprise is really showing the market that we've got a new upper bound for how much businesses are willing to spend on AI. And folks just do not want to have spend explode, right, as adoption does.
15:28May Habib:And they're really tired of chasing these frontier benchmarks, frontier prices. They want models that can do the work that they care about reliably at scale. And that's really what motivated X6, how we built it, and the harness that we've built around it. And I think we're really seeing purpose built, giving that performance advantage and that cost advantage that enterprises are looking for. And it's trained on 5.2 GLM, is that right? Yeah. We are the first of the applied AI companies that were really training our models from scratch. We spent years doing that. We're still doing that. But it also means we know that pre-training isn't the only place where value is created in applied AI.
16:14May Habib:And, you know, the post-training that we've done on task-specific environments alongside, you know, the investment in the harness really built for relationship management use cases, marketing use cases. it's really just delivered this trifecta that folks want, right? Cheaper, faster, better, 52 % lower cost, tasks completing in seconds. We've got agents that can run for eight hours or longer. And how do you get those results with the model? Walk me through the mechanics here. Yeah. So, you know, the ability for us to really think about, you know, what goes into a harness, right? How does the model pull context?
17:00May Habib:How does it use company data? How does it use its context window? How does it manage handoffs, right, among tools? How does it apply guardrails? You know, a harness is really, you know, all of those things and more. How it does QA, right? How it spawns sub-agents. How it checks those sub-agents work. And when you, like us, really sit on top of years of understanding how customers want to use agentic AI in the domain that we focus on, right, sales and marketing, it just gives you a tremendous leg up in being able to really build that model, build that harness. The harness itself is actually model agnostic.
17:42May Habib:We have built it to be model agnostic, and our platform is model agnostic, but of course works best with our model, and the model and the harness together work best with the writer platform. Now, so GLM 5.2, that's one of the models that comes out of China, and there's been a whole discussion around whether or not people have hesitation using models from overseas, whether they want to stay here in the U.S. Did you have any of that hesitation yourself? I think the market has really shifted on this, and enterprises' understanding of this has really shifted, right? This is a MIT-licensed open weights model.
18:21May Habib:It's the strongest available open weight model um and the the weights the environment the fight the post uh training the harness all of that built by a u.s company on u.s soil on u.s infrastructure uh it's running on u.s infrastructure there's literally no connection to the original developers i think you know the market has really turned um uh ship has sailed on this i mean there isn't a single enterprise that's not using, you know, open weight models in this way. Right. And so now just shifting to the race in open weight models broadly, I mean, we had NVIDIA come out earlier this week with their latest open weight model or a version of it, I should say.
19:04Meta is also re-entering the fray, I should say, with their open weight models. Who do you think, which U.S. company do you think has the best shot of challenging the open weight leaders in China? Because this is the race that everyone's curious to see how it plays out, right?
19:23May Habib:Yeah. I mean, look, I think not a single applied AI company is not absconding from its ambition to be in this race as well. I think what we are likely to see over the next few months is a real reopening of the enterprise race. Like, it is by no means over. I think the enterprise, when you really get a CIO one-on-one, right, they're not happy with what Anthropik's trying to do. They're not happy with what OpenAI is bringing to the enterprise. I think ultimately, OpenAI is a consumer AI company. Anthropik is a coding model company and the enterprise is actually really, really wide open and incredibly diverse in terms of the winners.
20:12May Habib:NVIDIA is going to be a winner. We're going to be a winner. The applied AI, native AI companies are going to win. And it's because the enterprise wants control, right? And the labs are building. But who's your money on, May? I mean, there's so many exciting, who would you put your money on to really challenge the open weight leaders in China? I think it's the startup, the U.S. startup ecosystem. NVIDIA is certainly a leader in, you know, what they're trying to do. So reflection, maybe. Maybe reflection. Yeah, I think it's still a really wide open race here. And, you know, I think, like, you know, the provenance of the weights matter a lot less, you know, whether they come from China, whether they come from the U.S., to folks who are really in the weeds here and, you know, enterprises who are making these purchasing decisions.
21:01May Habib:decisions. Let me ask you a question about, so you talked about OpenAI and Anthropic, and you had an interesting post yesterday on LinkedIn about watermarking. And I wondered if you could explain to us, what was the message you were trying to send with that post with respect to how some of these players are addressing watermarking and then what that really says overall about their strategy around content and ownership and stuff like that. Yeah, the context here is, you know, in the EU, the EU AI Act has gone into effect, and, you know, watermarking is one of the things that they ask companies to do.
21:42May Habib:And it's really, you know, on the part of the customer to disclose, you know, when they've put out content that has been generated with AI. But this gets back to, you know, know, the comment on one size fits all AI, you know, vast majority of enterprises using AI to develop content, you know, they're doing a ton more than pressing a button and publishing what Claude, you know, spits out. And so, yes, we are pro-transparency on watermarking content. Obviously, you know, we're going to be complying with the law, but labeling, you know, every AI-assisted output as generated by X model gives the model way too much credit.
22:25May Habib:And we're pursuing a solution that allows companies to really identify the role, right, that AI applied. And then that provenance is going to be really under, you know, their own brand and commensurate with the role that AI played. So, you know, anyone who actually listens to the enterprise would have come up with a solution like this. And I think, you know, they're just continuing to show this really lack of understanding of what enterprises want, the level of control that they want. And so, sorry, have the big AI labs, have they taken a different stance on this issue? Oh, yeah. I mean, this is a watermark that travels, you know, with the text.
23:06May Habib:So they're applying it to text, not just to video. And, you know, really very little transparency on, you know, how these watermarks are being developed, how little or how much human intervention retains the watermark or not, our approach is radically different than that because it's been co-developed with enterprises themselves. So I'm putting all this together, May. I mean, we're sort of here in this moment here where there's been a couple months of significant pushback against the big AI labs insofar as how they've addressed some of their strategies here with their customers, with their content, with their products.
23:48I mean, training is a big concern. You know, it sounds like you are sort of in the camp here that we want to provide an alternative to what they are offering businesses and customers. I mean, do you think that this lasts? I mean, you've been around tech long enough here. We've probably seen versions of this rivalry before. Or do you think they ultimately come around to customer considerations around this? Do you think that they bend a knee? Do you think they just start buying businesses up? I mean, how do you think this all plays out ultimately?
24:22May Habib:I think the level of control enterprises want is fundamentally at odds with the breadth of what the labs are trying to do, right? It's just not befitting a business model like theirs to essentially allow customers to fine tune every level of this stack that they want. You know, the smallest example, right, a fabled data retention policy of 30 days when our customers don't want us to retain data for 30 seconds. Right. I think it's just like, you know, they'd be doing it if they could. And I don't know that that changes after they go public. Right. Just give us the update on the business. We had you on about a year ago.
25:15How many people now does Ryder have? How much revenue generating? Just give us the update.
25:19May Habib:Yeah, just incredible growth. Certainly since last year, we just closed our second quarter, record quarter on every dimension. I think what we're very excited about is just how much customers are now really willing to invest and commit. A year ago, you know, we'd celebrate if a customer signed a two-year deal. Now the majority of our new deals and renewals are multi-year. You know, majority of the renewals are two and three years. So, you know, very, very excited that so much of the enterprise has really come up the curve on understanding just how important it is to have purpose-built platforms under the hood, especially for, you know, those functions that drive revenue.
26:06And how much revenue is the company doing now?
Read the full transcript
26:11May Habib:Yeah, I mean, we haven't disclosed that. It's a big round number that most companies kind of issue press releases about, but we are on to the next milestone. A hundred. Maybe that means a hundred. More than that. Maybe you're fundraising. Come back on and tell us when you do. May, I want to thank you for coming. Oh, well, before you go to the name, Palmyra, What's the inspiration behind the name of the model? Yeah, it's an ancient city in Syria. Both me and my co-founder have Levantine roots. They had a female leader for a very long time. So the research team went with Palmyra. Great. Well, I want to thank you for coming on, May.
26:52That is May Habib, the CEO of Writer here on TI TV. AI startups that train their own models are struggling to find cloud deals they can afford let's bring in rocket true ai and robotics reporter for the information with more on that rocket welcome back to the show it's great to have you here hey kosh great to be here so it's hard for anyone to find compute in this day and age uh as per your column today you are contending that it is extra hard for the neo labs we were talking about neo clouds before we're talking about neo labs here for them to find compute what is the story here yeah i think it is extra hard For a Neolab, you're working with all of the challenges that come with being a startup, but you're targeting the ambitions of being a large AI company.
27:41So in practice, you need a lot of chips, but compared to other startups that are also looking for chips to run existing AI models, you're looking for a large number of chips that are all located in one place that you can use for long periods of time to train your own AI models. So that introduces a lot of additional complications. And then you're doing it on the budget of a startup, and often you're getting locked into long-term contracts, which is difficult for a startup that barely has a product, hasn't trained a model yet, is planning to do a lot of fundamental research. So your future compute needs, say, three years down the line, depend a lot on how your early experiments go.
28:24But you can't run the early experiments until you get the compute. So there's just a lot of extra challenges that get introduced when you're training your own models and you're trying to do that kind of fundamental research. One of the most interesting hurdles that I thought was very practical was also the length of the contracts you pointed out in the piece. Yeah, and that has really shifted in recent months. It was not too long ago that startups could get compute basically right away. They could get compute on an as-needed basis on what were called the on-demand or spot markets for compute. And then a lot of that capacity dried up.
29:02These days, people will sometimes wait in a queue for hours for their job to get processed by a cloud provider. Sometimes people will come back again and again for, I've heard, up to a week or more before they finally get access to just one or two chips. So the sort of on-demand and spot access has dried up significantly. And then it wasn't too long ago either that if you were going to sign a contract, you could get a contract that was as short as, say, a year, which was considered a reasonable amount of time to sort of train your first model and do your first big experiments. These days, there are providers that are pushing for one to three years or three to five years.
29:44And that's a much bigger challenge for startups. It gets much more expensive and it's a bigger commitment. Now, I want to make this a little more tactical for people. What types of Neo Labs are we talking to? Did you end up, did you talk to people from any of these companies specifically? Are we talking about? I've talked to a few. So one of the companies that I talked to in the story is a robotics company. They developed foundation models, so AI models to power robots called Generalist. They were about six months into a one-year contract that they had signed, and it was time to go out into the market and find more compute.
30:19And they realized, whoa, the market has changed a lot since the last time we were out here looking for compute. In terms of the term length, like we talked about, the duration of the contract, but also the availability, the lead time before those contracts could go into effect, and prices were up significantly. Depending on some measures of prices, prices are up 50 % over the last six months or so. So that's been a pretty dramatic change. I also talked to a smaller research organization, which was interesting because they're maybe not a Neolab properly in the sense that they're doing research rather than commercializing models, but they're in a similar reference class because they're working with large models, they're doing training, and they're also like doing pre-training experiments, which is a really compute intensive part of the training process.
31:10So they're facing similar hurdles to the other Neolabs, but they're doing it on a research organization's budget. So even though they're only looking for, say, in the ballpark of 60 chips relative to Generalist, which was looking for the high hundreds to low thousands number of chips, they still faced a lot of challenges going out and it represented a large portion of their budget to spend that money. So much so that when they finally closed the deal, one of the employees at this organization bought a cake to celebrate and to give it to the person who had done the hard work of going out into the market and talking to all of these providers.
31:46I think that just underscores a little bit what a big deal this is for these organizations. You talked about price going up. You talked about the length of the contract. You talked about their own means to afford the compute as well, given that they're startups. What about the consideration from the sellers end? So I'm looking at the hyperscalers and we'll talk about the neoclouds here in a second. Do they actually want or are they incentivized to give compute to these neolabs, these smaller companies? Is it more of a risk, I guess, for Microsoft to get business from 100 different startups as opposed to one large customer.
32:28I imagine that the incentives here must be not really in favor for the startups either. Yeah. You know, I think there are challenges that come up there as well. On the other hand, I imagine, you know, hyperscalers don't want to have all of their revenue completely concentrated on one customer that could fall through or go under. But yeah, you're definitely taking a bet on Neolabs. And that puts cloud companies in general in kind of an interesting position where they are playing favorites a little bit, right? Like one thing they could do is just auction off their compute to the highest bidder. But in practice, they can make judgment calls about which companies they believe in the most.
33:04And that puts the cloud companies almost in the position of playing VC and choosing which companies to bet on. Right. What about the neoclouds? Are the neoclouds, are they addressing this issue at all? And of course, we're, you know, I'm not just talking about the nebbiest and the core weaves of the world. I mean, there are even smaller neocloud companies that are popping up. Yeah, yeah, they are. I think there the situation is that they're pushing for these longer contract terms to lock in revenue at the really high prices that we're seeing right now. Because if they can get customers to sign three to five year deals, that's a lot of revenue that they have guaranteed in the years going ahead.
33:40Now, not all of them. Every once in a while, you maybe still come across a one year contract, but you're really going to pay for that kind of a short duration. And sometimes you can even find capacity that's still in the sort of spot market or on demand. But likewise, you're going to have to pay a lot for it. So the tradeoff from the Neocloud's perspective is, do we want longer term revenue locked in at the high prices we see right now? Or do we want really high margins on the compute that we're selling in the short term? So there are business decisions that can be made on that tradeoff. So Rocket, what's your sense on when this crunch starts to ease for the smaller startups and the Neo Labs?
34:21I mean, are we going to have to wait years for these data centers all to get constructed for this to get easier for the smaller companies? Or is this something that could get better in a matter of months? What are you thinking here? I mean, that is the big question, Akash. And I don't think anyone really knows. Some people are tentatively optimistic that it could ease up by the end of this year or within a year as new compute comes online. As you said, new data centers are getting constructed and more capacity is getting made available all the time. On the other hand, the pace of demand has just exceeded the pace of new production of chips.
35:00And you could imagine that going on for a long time. If AI is as useful as a lot of people are betting that it's going to be, the marginal value of a single chip will just keep going up and up. Like the value of the work that an AI can do running on that chip will just keep going up as those AIs can do more actually valuable work for enterprises, for example. In that scenario, maybe it doesn't turn around. Maybe the compute crunch doesn't go away for a very long time. It's hard to say. Right. Because as things get cheaper too, then it's the whole Tvon's Paradox thing. It's like, well, it's cheaper, but that means more people can afford it.
35:36And then you've got to build more data centers. So it's certainly an interesting story to watch Rocket. I want to thank you for coming on. That is Rocket Drew, our AI and robotics reporter here at The Information. Thanks, Akash. That does it for today's show. A reminder, we are on this stream Monday through Friday at 10 a.m. Pacific, 1 p.m. Eastern. If you can't make it then, episodes are available on theinformation.com, on our YouTube channel, or wherever you get your podcasts. make sure to follow us on social media on x on instagram on tiktok and on linkedin i'm already excited for our next show tomorrow have a great rest of your thursday bye-bye for now
From the publisher
Robinhood CFO Shiv Verma talks with TITV Host Akash Pasricha about Robinhood’s new publicly traded venture fund vehicle. We also talk with Jefferies Senior Analyst Brent Thill about CoreWeave and Nebius’s booming earnings, WRITER CEO May Habib about enterprise AI economics and their new model, and we get into the startup compute crunch with our reporter Rocket Drew.
Articles discussed on this episode:
Subscribe:
The Information: https://www.theinformation.com/subscribe_h
Sign up for the AI Agenda newsletter: https://www.theinformation.com/features/ai-agenda
TITV airs weekdays on YouTube, X and LinkedIn at 10AM PT / 1PM ET. Or check us out wherever you get your podcasts.
Follow us:
X: https://x.com/theinformation
IG: https://www.instagram.com/theinformation/
TikTok: https://www.tiktok.com/@titv.theinformation
LinkedIn: https://www.linkedin.com/company/theinformation/
Chapters:
00:00 - Introduction
01:13 - Robinhood Debuts Second Public Venture Fund
06:06 - CoreWeave & Nebius Cash In on Soaring AI Compute Prices
15:45 - WRITER Releases Flagship AI Model 'Palmyra X6'
27:59 - Why the AI Compute Crunch is Hitting Neolabs Hard
