How CoreWeave Sees the Market for Compute Right Now

25:11Tracy Alloway:Obviously, just to push back on this a little bit, and I'm not really in any position to push back. I can only relay what past guests have said in my own reading. So what one of our guests said is that absolutely, NVIDIA has the lock on model training, that if you want to train a model, that yes, NVIDIA chips are the only game in town. But that for inference, It's the really his view. This is Ian Dunning again. His view is there really were options. And then, of course, we had someone who was much more biased. We interviewed the CEO of Cerebros, the company that makes the gigantic plate and or sorry, the gigantic gigantic plate.

25:50Tracy Alloway:And of course he did. But I mean, of course, he was going to say, yeah, the the CUDA moat is vastly overrated for inference. It barely exists. Now, of course, of course, he's going to say that. So, like, you know, he's in a competitor. But we've also heard it from a user of inference. And intuitively, it makes sense like training is very complicated and all that stuff. But what you're saying is that from the customer standpoint, you see the demand for NVIDIA on both the training and the inference as being steady and that you perceive that advantage to be consistent through both aspects? So I believe in our last quarterly report, our CEO might qualify that inference workloads represent well in excess of 50 % of infrastructure utilization on our platform.

26:39Okay. It's the exact same infrastructure that we use for training as well. Going back to my comment of like, it's very fungible between those different types of workloads. Those customers are choosing NVIDIA to work with on inference. I think what you're going to see is people will want to try at small scale other types of silicon. But the reliable, proven and remains from our perspective, most efficient infrastructure to use is NVIDIA today. Does that change over time? Who really knows? But I think we've seen NVIDIA battling this concept for years and every year they show up and like they remain the de facto choice for AI infrastructure.

27:27I think we're going to be one of the first people in the market to see it because that will be a tone shift change from our clients asking us to run something else. That hasn't happened. OK, so have the constraints on your business changed at all? So three years ago, we were talking about GPUs and how hard they were to actually get. I imagine GPU, securing GPUs is still competitive, to say the least. But are you seeing other constraints emerge, like Joe mentioned in the intro, just land usage, just places to actually build data centers? Land usage, specifically, I wouldn't say is as much of a concern.

28:07Having a powered shell is the bottleneck today. and let me qualify powered shell. Powered shell is effectively an empty data center that is energized, right? It has all the power and associated components. I can come into it and deliver electrons into a rack. Has the cooling system built within it. Like it has the whole thing, right? Powered shell is the industry term for it. That is the bottleneck because of all of the supply chains that come into that, right? Like not only do you have electricity, do you have the land, et cetera, but you have the backup battery supplies, you have the transformers, you have personnel, right?

28:50Let's just think about the electricians for these sites and getting the accreditation on the electrician side to be able to participate in these bills. I mean, I think it's a five-year plus apprenticeship to be able to go through that program, right? We can't just make new electricians leveraging a supply chain, right? That's a trade that you can't really scale efficiently. So that is absolutely the bottleneck for us. And I think our peer set that's out there right now, access to chips. I think we have a phenomenal relationship with NVIDIA where we've just proven to be the best operator of this infrastructure on the planet.

29:30You know, a bottleneck that existed for us previously, I think, was access to financing. Yeah. Right. We all know. Doesn't seem to be an issue anymore. I would agree with that broadly, but that's years of work and execution that has delivered that ability for us. I mean, year to date, we've raised over$21 billion of financing for our business. You don't get to do that and just go from zero to 20 out of nowhere. And I think that's largely driven by our track record of execution, right? Our investors, our creditors can see this deep set of experience over the years of consistently delivering on these builds.

30:18I mean, we have over a gigawatt in active power at this point, right? Like a gigawatt at the data center level with GPUs delivered into clients. And I think that there has been kind of a misunderstanding in the market where people are conflating the concept that something on paper is the equivalent to being physically done and delivered. And all I can say is there's an enormous gap between signing for power delivery in 2030 versus actually delivering that into billable GPU hours. And that gap of execution is what has driven down our cost of capital so aggressively. That gap is where our business sits and why it's been so successful.

31:08I mean, that's the secret sauce is our ability to take these data center deployments and these customer relationships and deliver billable GPU hours into them.

31:20Tracy Alloway:You know, speaking of financing, I just want to say, you know, during last year, like maybe six months ago, that might have been the sort of near peak of the Michael Burry inspired. These chips are like in the last two years stuff. And one of the viral charts that you would see on Twitter was the CoreWeave CDS chart. Those have come way in. So it is it is, you know, I haven't seen those charts in a while. Yeah, that's right. That's the thing about CDS. No one ever posts charts of credit default swaps when they're coming. People love to post them when they're blowing out. They have come in. So, you know, that does speak to some of this point about these anxieties having been alleviated at least somewhat since the start of the year.

32:05Tracy Alloway:You know, it occurred to me like we're talking about credit default swaps or talking about financing. I'm sort of gearing up to write a big thing maybe, but I'm writing it in my head currently that there really are a lot of analogies between the business of data centers and the business of banking. And one of the things in banking, as we all learned from SVB, was the risk of industry and depositor concentration, that if you have all your depositors are either in like one depositor gets too big or all your depositors are in the same industry, then you have this risk of like correlated withdrawals.

32:41Tracy Alloway:And that's what obviously did in SVB. When you think about planning and you think about, OK, here's an investment, et cetera, how much does this come up sort of like thinking about, I guess, tenant diversification as something that you think about in your multi-year planning? It's a critical aspect of it, right? As I said earlier too, like this was a key criticism of us coming into our IPO last year, right? Where we had that customer concentration in our revenue and we have made enormous progress there. And I think the best way to think about it is we could take all of our unallocated capacity.

33:21And I say that very specifically, it's not unsold capacity, implying that there's no demand for it. It's unallocated. There's intense demand for it. We're figuring out where it should go. And that customer piece of it, I think, honestly, like we could allocate all that capacity to like single name clients, right? Like there is a pretty significant number of single name clients we can go allocated out into. But I don't think that is the business we are supposed to be building here. I think the business we are supposed to be building is a diversified cloud that is supporting the leading AI consumers and producers on the planet.

34:03I don't think we're supposed to be sporting just one or two companies.

34:22Missatisfying breads and pastas and want to add protein without going overboard on calories? stacked sandwiches, fully loaded bagels, noodles built for serious sauce. Hero Bread delivers up to 19 grams of protein. Think bagels and elbow noodles with nearly twice the protein of national bestsellers, but less than half the calories. Plus, you can get up to 32 grams of fiber per serving, a bonus when you're trying to stay full and fueled. So whether you're grilling burgers, building a serious sandwich, or digging into pasta, you're getting real flavor with a smarter protein-to-calorie balance. Hero makes loaves, buns, tortillas, bagels, and noodles with 5 to 19 grams of protein per serving that all go the distance.

35:01Shop now at Hero.co. Use code IHART for 10 % off. That's H-E-R-O dot C-O. All figures per serving. See nutrition info on Hero.co. 39 % and 61 % fewer calories than regular plain bagels and noodles, respectively. Calorie content has been reduced from 270 to 130 and 200 to 80 calories per serving for plain bagels and noodles, respectively. Data accurate as of 2020-26. We believe in starting with your financial goals, not a formula. At Oppenheimer, we put the full strength of our longstanding expertise to work, understanding your life and your ambitions, and designing the precise strategies that build and protect your wealth with confidence across this generation and the next.

35:42Put the power of Oppenheimer thinking to work for you. Wealth management, capital markets, investment banking. This is Jacob Goldstein from What's Your Problem. Business software is expensive, and when you buy software from lots of different companies, it's not only expensive, it gets confusing, slow to use, hard to integrate. Odoo solves that because all Odoo software is connected on a single affordable platform. Save money without missing out on the features you need. Odoo has no hidden costs and no limit on features or data. Odoo has over 60 apps available for any needs your business might have, all at no additional charge.

36:23Everything from websites to sales to inventory to accounting, all linked and talking to each other. Check out Odoo at O-D-O-O dot com. That's O-D-O-O dot com. When it comes to financing, can you say a little bit more about what changed to make the market more comfortable with this? Because, like, this is the big story in markets. It's just how much AI is now being issued through the corporate bond market. The equity market, as we know, is basically all big tech at the moment. What changed on the part of investors? Was it just pure return and performance? Or were there, I guess, efforts to make the contracts more robust or increase visibility into demand and that sort of thing?

37:09Seeing the inference aspect of it really emboldened investors. But that was really just January or maybe late Q4, where you started seeing this just massive inflection of demand driven by inference. For us, right, it's tough for me to speak about other companies, but for us, like why have we been underwritten at such scale and at a decreasing cost of capital? I think it goes back to that track record of execution, right? is just the market has watched us execute and watch us deliver on these contracts. And the way, and tell me if I'm going into too much detail here, but the way that we finance our business, you kind of break it into two broad buckets, right?

37:55You have parent co-financing and asset co-financing. And asset co-financing is where all of the GPUs get financed, right? It's where all of our client contracts sit. And we can take these financings and put them into SPVs, or we'll just call it a box, so to say. And you... Sorry, I'm sorry.

38:20Tracy Alloway:Keep going. Mentioning boxes is dangerous. Boxes on lots of connotations, but keep going. We put them into SPVs. And these SPVs, they have the infrastructure, they have the data center costs, and they have the debt agreements within them. And so you're able to pair this like five year take or pay contract to an amortization schedule on the debt. And you have the revenue come into the box, pay down the amortization schedule, pay down the operating costs of the data center. And it still contributes a it has a 25 percent contribution margin of profit up to the parent code. Right. Like these are highly profitable agreements down in the SBB stack.

39:06And so you take that SPV out to the credit market and say, look at this instrument. It's a discrete set of contracts with counterparties, like entities who want to consume GPU compute. You have the data centers within it, et cetera. And, you know, one of the latest ones we did was, as we call it, DDTL4. This was a investment grade rated, first of its class. No one had done this before for GPU financing, non-recourse HPC infrastructure financing. It got done at SOFR plus$225. That is a phenomenal cost of capital for us. And importantly, we were able to bring in the insurance charge of capital, which is a massive charge of capital out there that is looking to do allocations into the space.

39:55So we're kind of continuously making progress through these different stacks of capital and locking access to more and more types of investors. It's why you've seen us moving to the convertible note market and to the unsecured market as well, along with taking direct strategic equity investments. But for us, it's really important for the entire investor space to understand this business because this business largely didn't exist before. right like people weren't making loans into the hyperscalers to go credit these build-outs right it's on core weave honestly to be building this path into how do you finance the ai hyperscaler effectively and i think um we've just done a terrific job of it over the past few years

40:44Tracy Alloway:you used to be in a prior lifetime a trader right yes i was a commodity trader so i'm curious like There's a lot of interest in, and I don't know if it's going to materialize, in GPU capacity trading. And there's going to be a new contract. We recently interviewed the CEO of Compute Exchange, and they're very close to having something listed on the CME. From your perspective, because I don't have a view on this yet. You see, okay, a big AI company does a five-year contract. As you say, the duration is lengthening. We're going to lock this in. I don't know what the need is for tradable compute in that environment, etc.

41:24Tracy Alloway:What's your guess? Do you anticipate that there will be a sufficient ecology of hedgers and speculators such that there will be a liquid market for tradable compute? I think it's very much a timeline question that's out there. Short term, no. Let me offer why no short term and then I'd say maybe in the long term. Okay. And it all comes back to fungibility, right? If you think about gold, gold is defined by its chemical composition, right? And there's no question of what is gold and not gold, et cetera. Compute really isn't, right? And especially GPU compute. GPU compute today is not fungible. And I think that this is well understood by our client base, by our suppliers, by third-party consultants like Sydney Analysis.

42:21And it's this idea that an H100 deployed in one cloud doesn't have the same performance of an H100 deployed in another cloud. And the metrics that people use are things like good put or model flop utilization, MFUs. And there are these measurements of like how much more performant is one, the exact same GPU, by the way, versus another GPU deployed in another facility. And so in order for something to be commoditized, it has to be fungible, right? Otherwise, there's just too much, you know, murkiness and there isn't like an exact data point in there.

42:59Tracy Alloway:Can I push on that a little bit further? So, I mean, I think that seems like a reasonable view. Is the non-fungibility related to configuration of like how they literally like the configuration of the GPUs within physically? Like, what is it? Is it about power? I mean, I think they all like, you know, there are plenty of places that will say, you know, we have nine nines or however many nines you need in your industry or whatever. What is it, in your view, that would cause significant changes in the performance of an H100 in one cloud versus another? It could be in some part, configuration, right?

43:38We build everything to DGX reference spec, which is the most, outlined by NVIDIA, it's the most performant way to build, operate, and deliver GPUs. But the rest of it, honestly, is just how you operate the GPUs. And that is the core weave software stack. That is how do you keep these GPUs online, right? Like what happens if a GPU fails? Can you predict if a GPU is about to fail and swap in other infrastructure so that the client doesn't have downtime on that component? And there's an immense suite of software solutions and infrastructure management solutions that we have built to have the best good put, to have the best MFUs in the industry.

44:23And none of that is off the shelf, right? And so I wouldn't say it comes down to the strict components. That's kind of like a bare minimum starting point, right? Like you have to start in DJX reference spec. But where does differentiation come from there? I mean, that's the core lead product. you're describing right there.

44:42Tracy Alloway:By the way, Tracy, I'm just looking up. Terms of art, good put measures the fraction of peak hardware performance that the training job can extract. This is according to Google. And MFUs, model flops utilization, hardware metric for evaluating real-world efficiency of LLM's training. So two new terms. I actually hadn't heard of MFUs or good put before this. So I just learned two new terms today. We got to create a glossary. AI glossary. Yeah, we do. Brandon, when Joe asked you that question about compute markets earlier, you said it was a timeline question, which in my mind implies that it's inevitable.

45:18Like, it's just a question of how long it takes. But then when you describe the fungibility problem, it seems like this is an actual issue that will be very difficult to solve. Yes. I think that characterization is absolutely correct, right? Like, if you just take general commodity theory and i traded natural gas electricity agriculture products for over a decade like it suggests that it should become that at some point but what is the reality today the reality is this stuff isn't getting easier to operate right we've moved from these kind of relatively simple 42u air cold racks of hopper to these immensely complex blackwell deployments moving into Vera Rubin following that, like it's not getting easier to build, operate, provision, deliver these GPUs.

46:10It's getting more difficult. And I think until it starts becoming easier, you don't really have a path to commoditization. You will have to continue to prioritize working with the world-class and world-leading operators of infrastructure. That's where we sit.

46:29Tracy Alloway:First of all, this is helpful. And I like that we're getting multiple perspectives because I do think this is going to be like one of the big questions for financial markets. Because let's say if they took off, then you could imagine that might even improve financing conditions because then the lender can hedge against the price. Yeah. So like there would probably be some good things for the industry if this took off. So I appreciate it. It's good to have your perspective on this. Why is it – you know, I'm an inference – I am an inference user, by the way. So I made a little machine learning model in one of my hobby projects, and I provide inference over at Havelock.ai, or I'm a user of inference or whatever.

47:09Tracy Alloway:I have a model, whatever. Why is it that I— It would be impressive if you were providing inference. I'm trying to—I guess I'm a consumer of inference. I use a—anyway, why is it that I'm actually very easily able to get—now, not a huge allocation of, like, GPU access? So I was like, how do I train this model? It's a model called BERT that Google released in 2018 or 2019. I fine-tuned it for my purposes. And then literally using CloudCode, I was able to, in 10 minutes, sign up. I started using this company called Modal, and I was able to start training a model. I was surprised that there was like, and it didn't cost me very much and I have like no volume, but nonetheless, evidently there was a little GPU capacity out there that I could get.

47:56Tracy Alloway:And it cost me like$5 or something for the whole thing. Given what you always hear about, like a utilization is slammed. Why is it actually not that hard to find GPU capacity for someone like myself? You know, I think it's the scale difference right there. Finding ones or tens of GPUs, I think that's way more accessible out there. Okay. Our clients are focused on the hundreds of thousands of GPUs. I'm not there yet, but I'm not there yet. Not yet. I'm sure you'll get there. Yes. And that's where it kind of decommoditizes itself with scale as well, right? Like as you're in the hundreds of thousands component, there's just not that many deployments, right?

48:39It's handfuls of deployments at that size. But getting access to ones of GPUs, I think that there is a lot more ability to go secure that sizing in the market. So Joe and I are heading to Hong Kong very soon, and I expect that AI in China is going to be a big topic of conversation. How would you characterize, I guess, the difference between the U.S. and the Chinese market at the moment? I'm sure this is something you think about, even though you don't participate in the Chinese market directly.

49:09Tracy Alloway:yeah that that's tracy's asking for questions yeah that's basically like it's like questions that we can ask people when we're over there yeah that that's likely going to be my response tracy is like we just do not participate in that market um i think that there's opportunity for us to be expanding as you guys know we we operate in canada europe um i think moving uh further east makes a lot of sense for us but we're trying to be very methodical in the way that we expand so i unfortunately I'm not going to be able to help you with specific questions in that market, but I would imagine you're going to encounter a lot of the same things that you're seeing in the U S which is just insatiable, unrelenting demand for AI.

49:51And like, you know, we just kind of keep coming back to this is like, there is no solution in sight for being able to satiate demand, right? There's just too many supply chain. there's no path to solving demand in the near term or even the medium term, frankly.

50:09Tracy Alloway:You mentioned, so Tracy asked you about land use. You said that really was an issue. But like the first time we talked to you in 2023 or whenever that was, there was not a major growing movement of people who are just like anti-data centers in America. Maybe there were a few fringe people, but it was not something that was on the minds of politicians and activists and so forth. And you do see these headlines, you know, about some projects really having been shelved. There was like a big one. Northern Virginia is a huge hotspot for it. And there was a big project that was, they pulled the plug on due to some, they couldn't get an agreement with the local government.

50:46Tracy Alloway:That must affect you. What are you seeing in terms of like your capacity to build? How has it changed specifically in light of, or have you seen a change, would you be able to build faster in a world where this had never become a political hot button issue? I believe it has become that hot button issue. It's something that we're quite proactive about in market. And I think you just kind of go through the checks on the diligence process to make sure you're going through it correctly. I think that there's misconceptions out there, like water usage. Yeah, setting aside the misconception, like setting aside, I know, setting aside the whole debate about, but just in terms of like operationally, what's it changed for you in terms of your plan?

51:32No, I would say our greatest challenge is still just getting that delivery of our, like the construction and all the components and getting everything in there. Like that is truly more of the bottleneck that's in the market today.

51:46Tracy Alloway:Brandon, thank you so much for coming back on OddLots. We'll have you back next month for another market. No, or at least, or maybe in three years. Not three years. Yeah, not three years, but really. That's an eternity. Yeah, I know. Thank you so much. Thanks, guys. Appreciate it.

52:15Tracy Alloway:I'm very excited about whether compute features will take off. I think this is an exciting story. It's not the biggest story in the world, But it is actually a very exciting story. I've said this before. Even if you're not that interested in AI, this is a really interesting market structure story, right? It's basically the creation of a brand new market and poses all these interesting philosophical questions about how you do that. And I thought Brandon's point about fungibility, I mean, that is a real issue. And it does seem like it's a challenging one to fix at the moment. I don't know if it's inevitable in the future, but who knows?

52:54Tracy Alloway:No, no. I mean, it makes a lot of sense. This was also Lewis Hart's point that it's like, you know, it's in the word commodity, right? If it's not a commodity, you're not going to get a commodity market for it. And of course, a number of entities are betting that it will be commoditized. But if it's true that, like, you know, they're getting more difficult to work, that the technical demands on the inference provider, on the data center company are getting greater in order to get the maximum, you know, juice, then maybe it doesn't become commoditized. But I think that's like a fascinating question.

53:31But at the same time, like if you do see those efficiency improvements and new designs and things like that, you could imagine that like the demand is there for a standardized GPU as well. So I don't know. Like I'm really torn. It feels like it could go either way.

53:44Tracy Alloway:Well, and even in his answer, he talked about how they configure their own GPUs to a spec largely that NVIDIA itself has come up with. So in theory, like there is a spec that everyone can match to. So that's like a really interesting, that's a really interesting question. I also really want to do more on all of these. So Google has TPUs, Amazon has Tranium, Microsoft has its own hardware, maybe even Jane Streets and the Hudson River Tradings will have their own hardware. If they're not like I want to understand better why. Yeah. Right. Because like they presumably have some reason and they at least like the Microsoft will say, well, this will run better on our customized hardware.

54:32Tracy Alloway:I want to understand why that would be how much difference in performance is there and then the degree to which demand materializes from users for non NVIDIA silicon is like a really big question. Yeah. Why custom chips? Yeah. And what can you get out of that if you align model and chip to optimally work together? I have no idea, but I feel like it's an episode I would like to do. Yeah, we should. All right. Shall we leave it there in the meantime? Let's leave it there. All right. This has been another episode of the All Thoughts Podcast. I'm Tracy Allaway. You can follow me at Tracy Allaway. And I'm Joe Weisenthal.

55:07Tracy Alloway:You can follow me at The Stalwart. Follow our guest, Brandon McBee, at Brandon McBee. Follow our producers, Carmen Rodriguez at CarmenArmandDash, Elbenet at Dashbot, KaleBrooks at KaleBrooks, and Kevin Lozano at KevinLloydLazano. And for more OddLots content, go to Bloomberg.com slash OddLots or the daily newsletter in all of our episodes. And you can chat about all these topics 24-7 in our Discord, discord.gg slash OddLots. And if you enjoy OddLots, if you want us to do an episode on custom chips, then please leave us a positive review on your favorite podcast platform. And remember, if you are a Bloomberg subscriber, or you can listen to all of our episodes absolutely ad-free.

55:44All you need to do is find the Bloomberg channel on Apple Podcasts and follow the instructions there. Thanks for listening.

56:21This is Jacob Goldstein from What's Your Problem? Business software is expensive. And when you buy software from lots of different companies, it's not only expensive, it gets confusing, slow to use, hard to integrate. Odoo solves that because all Odoo software is connected on a single affordable platform. Save money without missing out on the features you need. Odoo has no hidden costs and no limit on features or data. Odoo has over 60 apps available for any needs your business might have, all at no additional charge. Everything from websites to sales to inventory to accounting, all linked and talking to each other.

56:59Check out Odoo at odoo.com. That's odoo.com. Picture this. You're with your spouse at a seaside botanical garden. The moss hangs low, but your spirits are high. And you've never felt more in love. This is when it hits you. That Myrtle moment. It's the second everything clicks. When you realize Myrtle Beach is more than a getaway. It's right where you belong. Don't just daydream about it. Find your own Myrtle moment at visitmyrtlebeach.com.

57:38Paramount Plus is now the home of all your BET favorites. What? Yes! With all new episodes of Tyler Perry's Divorce Sisters. You've always liked a little drama. Plus a whole new world of movies like Gladiator 2. Now I will control an empire. Original series like The Chi. Just make sure we protect each other. And live sports like UFC. Welcome to the history books! New home, same family. Your BET favorites are now on Paramount Plus. Subscribe now. Thank you.