CoreWeave's CSO on the Business of Building AI Datacenters

21 Jun 2024 · 55 min

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

Odd Lots Podcast - Episode Summary: CoreWeave's CSO on the Business of Building AI Datacenters

Podcast Information

  • Title: Odd Lots
  • Hosts: Joe Weisenthal & Tracy Alloway
  • Description: Exploring interesting topics in finance, markets, and economics.

Episode Overview

  • Episode Title: CoreWeave's CSO on the Business of Building AI Datacenters
  • Episode Description: This episode features Brian Venturo, Chief Strategy Officer of CoreWeave, discussing the intricacies of building AI data centers, including energy sourcing, operational logistics, financing, and the nuances of AI-focused cloud computing.

---

Key Themes and Discussions

  1. Overview of CoreWeave's Operations
  2. CoreWeave is a specialized cloud service provider focused on AI data centers.
  3. They leverage NVIDIA chips and have seen massive growth due to the AI boom.
  4. The company blends advanced technology services with physical infrastructure to support complex AI workloads.
  1. Components of Building an AI Data Center
  2. Technology Services: CoreWeave must ensure software resilience and performance, managing complex interconnections among thousands of GPU nodes to prevent job disruptions.
  3. Physical Infrastructure: Challenges in building and operating data centers include logistical challenges and maintaining optimal environment conditions for hardware.
  4. Financing: Capital-intensive business model requires structured financial instruments; CoreWeave successfully utilizes GPU-backed loans, which are underpinned by commercial contracts.
  1. Market Dynamics and Customer Relationships
  2. High demand for compute power leads to selective customer engagement.
  3. CoreWeave prioritizes long-term relationships to ensure capacity can meet customer needs.
  4. The transition from co-location to building proprietary data centers enhances service delivery.
  1. Technology and Structural Customization
  2. Customization of server clusters based on customer requirements is common, with extensive collaboration on design.
  3. Building from the ground up allows CoreWeave to optimize performance for unique customer needs.
  1. Challenges in the AI Data Center Sector
  2. Electricity Supply: Growing concerns about electricity demand and sustainability; CoreWeave strategically selects sites based on stable electricity availability.
  3. Labor and Component Shortages: Increasing difficulties in securing electrical components and skilled labor for data center construction, exacerbated by supply chain issues.
  1. Insights on NVIDIA's Dominance
  2. NVIDIA's chips are preferred for their engineering excellence and proven performance in AI workloads.
  3. Competitors face challenges breaking into a market where NVIDIA has established strong support and community engagement.
  1. Future of AI and Data Centers
  2. Anticipation of new use cases as technology evolves, particularly with the introduction of new NVIDIA chip architectures that promise enhanced performance for AI and scientific computing applications.
  1. Financial Implications for the Sector
  2. The shift towards debt financing in the tech sector is influenced by the high capital requirement for building AI infrastructure.
  3. CoreWeave's financing structures, such as GPU-backed loans, are evolving and represent a new avenue in securing capital for tech firms.

---

Key Takeaways

  • AI Data Center Demand: A significant urgency exists for scalable, efficient computing solutions driven by the AI boom.
  • Strategic Location Selection: The careful choice of data center locations in areas with ample electricity and minimal community backlash is crucial.
  • Financing Evolution: The emergence of specialized financial products, like GPU-backed loans, reflects the unique capital needs of AI infrastructure providers.
  • NVIDIA's Market Position: Their dominance continues to shape the market landscape, although emerging competitors may seek to disrupt this dynamic.

---

Conclusion This episode provides valuable insights into the operational complexities, market dynamics, and financial strategies involved in building AI datacenters. Brian Venturo's expertise sheds light on how companies like CoreWeave are navigating this rapidly evolving landscape and positioning themselves for future growth amidst challenges.

For further exploration of these topics, tune in to the Odd Lots Podcast and stay updated on the latest discussions in finance and technology.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:00You're being sold an AI future where you're obsolete or irrelevant. That vision is wrong. At Palantir, they're building AI that helps workers and unlocks their full potential. American workers are our nation's greatest strength. AI shouldn't eliminate them. It should elevate them. Palantir is here to tell their stories. From factories to hospitals, AI is freeing people from drudgery, letting them do what humans do best. Create. Solve. Build. Palantir, making Americans irreplaceable.

1:01mobile devices, message and data rates may apply. JPMorgan Chase Bank N.A. Member FDIC. Copyright 2025, JPMorgan Chase and Company.

1:32Hello, and welcome to another episode of the Odd Lots podcast. I'm Joe Weisenthal. And I'm Tracy Allaway. Tracy, you know, we've done tons, of course, on like electricity and AI and data centers and all that stuff. But we've never actually done like a, well, we've never talked to someone who is building data centers. Putting it all together, you mean? Yeah, putting it all together. Like what, you know, just a bunch of, you know, I've had consultants So we talked to energy people. But like, how does this business of essentially, I guess, building a building, putting a bunch of chips in there, getting the electricity and then in theory, selling all of that at a markup?

2:12Like, how does it actually work? You know what I was reading recently? This is kind of a tangent, but not really, because we're talking about the physical and financial process of building these things. But I saw this is online. There is a guide to the like physical planning around an IBM system 360 from like 1963 or something. And it's 213 pages long. Have you read it yet? I did flip through it. There's guidance on minimizing vibrations, obviously, like temperature and humidity and stuff like that. I did not read the full 200 pages. But I'm kind of thinking, if this is all the thinking that had to go into one computer, albeit a supercomputer in the 1960s, but a pretty basic machine when we look back on it now, how much planning and thinking has to go into building these huge cloud servers?

3:10and all their associated infrastructure, both physical and software as well. No, totally. And you know, one of the ways that we've touched on this subject a little bit is in our conversations with Steve Eisman, who's been investing, at least as far as we know, in a lot of these like industrial HVAC companies and electricity gear companies and stuff like that. So like companies that have actually been around for a really long time, sort of standard cyclical businesses and then they've like caught the secular tailwind because with this boom in ai data center construction suddenly there's this sort of continuous bid for all their gear and services i'm gonna start an anti-vibration floor maker or something do you think that's a viable business does anyone care about vibrations anymore i am certain that in various high-tech environments you do not want to have vibrations you know you have like valuable chips you don't want them to be like degrading.

4:06Because people are walking around? Yeah, or just, you know, what all the machine and all your air conditioners and cooling equipment and all that stuff, you can't be having that stuff degrade. Well, the other interesting thing that's happening in the space now, so in addition to, you know, the physical challenge of building a bunch of this stuff, there's also the financial aspect of it. And I guess as AI becomes more and more of a thing, and clearly, as you laid out, there's a lot of enthusiasm around the space at the moment, you are seeing a bunch of financial entities get interested as well.

4:39So obviously venture capital has been pouring money into the space, but we're starting to see some new types of financial investments in AI. And I'm thinking about one thing in particular, and it is the recent GPU or chip-backed loan that was reported by the Wall Street Journal. And I think we should talk about that aspect of it too. Totally. Because one of the things that's happening in tech is this big sort of shift from like, okay, we're all of your costs in the past, or a lot of them are sort of OPEX, the cost of engineers, etc. And now suddenly tech companies have to think about CapEx for the first time, these big upfront costs that are in theory going to pay off for a long time, which in theory then changes how you should think about the financing model.

5:23Absolutely. Well, I am excited to say, because we literally do have the perfect guest, we are going to be speaking with Brian Ventura. He is the chief strategy officer at CoreWeave. CoreWeave, for those who don't know, it's probably the company right now that people most associate with being at the heart of the AI data center boom. They have a bunch of NVIDIA chips. They have investments from NVIDIA right here in the sweet spot. But as you mentioned, one of the interesting things that's going on is they not long ago announced a debt financing facility, sit back basically by the GPUs that they would acquire.

6:04So literally the perfect person to understand like the business of these AI cloud data center operations. So Brian, thank you so much for coming in. Thanks for having me. It's the second time I've been on the podcast. That's right. We talked to Brian years ago. It's interesting to think about at that time, because I think that may have been like 2020 or 21. And the excitement then was that these chips could be used for crypto mining and other things like sort of distributed video editing and stuff like that. And then Ethereum stopped using mining, but it was sort of fortuitous timing because right around then, AI went crazy.

6:40And that's probably, I don't know, in my view, maybe a higher use of these chips. Before we get to that, do you worry about vibration in your data centers? So everywhere that's close to a fault line is designed around that and is part of code. So the engineering firms that help us build these data centers have taken all of that into account. And all of our racks are seismically tuned to make sure that we can withstand the normal vibration from the earth. So, yeah, it's been something that's been in those manuals for a long time. some of our hardware manufacturers actually have vibration testing labs where they put the racks on top of a big kind of platform that shakes and it's pretty dangerous and uncontrollable and hard to watch.

7:24But, you know, there's people out there that have been solving this problem for decades now. I missed the boat on that business, Joe. It sounds like it's been dealt with decades ago. Okay. Well, actually, why don't I start with a very simple question, which is when you're looking at the business of CoreWeave, so a specialized cloud service provider, let's put it that way, what are the different components that you have to think about? You know, Joe kind of alluded to all these different ingredients that go into the business, but walk us through what those actually are. Sure. So there's three pieces that, as a management team, we think are incredibly critical to the business.

8:03The first is, you know, our technology services that we provide on top of the hardware, right? And this is everything from the software layer through the support organization to, you know, how we work with our customers. This isn't the type of thing that you just go plug in and it works. In these large supercomputer clusters, there may be 200 ,000 InfiniBand connections that connect all the GPUs together. And if one of those connections fails for whatever reason, the job will completely stop and have to restart from its previous checkpoint. So, you know, everything that we do on the software side and engineering side is to make sure these clusters are as resilient and performant as they possibly can be to ensure our customers can run their jobs, increase efficiency, and get all of the kind of monetary value they can out of the chips.

8:48So technology piece is really hard. It's something that I think is very overlooked by the market, but it's just as hard as the two other kind of pieces that this business stands on. The second is the physical nature of the business in that you have to actually build and run these data centers and those hundreds of thousands of connections inside the supercomputers, like somebody has to go put those together and make sure they're clean and make sure they're labeled correctly to be able to remediate failures. And when you're building a 32 ,000 GPU supercomputer, that is one of the fastest three computers on the planet, you know, you're running thousands of miles of cable inside a very dense space, right?

9:27These data centers are built very tiny to make sure that you can connect everything together. and that becomes a huge logistical challenge. So, you know, the data center piece, which we're going to talk more about today, is very challenging to design for the use case. And then the third piece is, how the hell do you finance the whole thing, right? And, you know, we've been very successful in the financing aspect of this, but, you know, whether you're financing technology operations or the physical build of these things, it is an incredibly capital intensive business and constructing those financial instruments to back our business is very hard And we have to be very, very thoughtful around who the counterparties are.

10:04How do we think about credit risk? How do our investors think about that credit risk? How do we deal with contingencies inside the contracts to make sure that they are financeable on the scale that we've done over the last 18 months? Talk to us a little bit more. We could probably talk about data center financing credit and have that be a whole episode. But when you think about, you have to think about your counterparties' credit risk. Talk to us a little bit about who those are, what the type of entity is. Sure. So I'll get myself in trouble if I just start naming them off. Yeah, I know. Some of them are more public than others.

10:39I'm going to refer to them as hyperscale customers. We have AI lab customers. We have large enterprise customers. We've really constructed our portfolio of business around the idea that if we're going to build$10 billion of infrastructure for somebody, we have to know there's a balance sheet we can lean into behind it. Right. And we're the pace at which we've grown, you know, our customers are demanding scale so quickly that the credit of the counterparty is incredibly important to find the low cost of capital we have with these credit facilities we've announced. Right. So, you know, when people talk about how this is a credit facility backed by GPUs, it's not really backed by GPUs.

11:19It's backed by, you know, commercial contracts with large international enterprises that may have triple A credit. Right. So, you know, it's it's the framing of that. Trade receivables finance, basically? Yeah, it's closer to trade receivables financing than it is, hey, we're going to go leverage up a bunch of GPUs and see what happens. Okay, well, walk us through, I guess, the sequence in some of these financing agreements. So, you know, if a customer comes to you and they say, we want a certain amount of compute, can you do this for us? And you start going down the process of like, okay, what do we need to make this happen?

11:55what do those like financial agreements actually look like and who's bearing the initial risk is it the customer is it you good question so when we're approached by a customer right you know the ask is typically going to be pretty pretty general and they're going to say hey we're looking for capacity in q1 of next year what's the largest thing you can do and uh you know we take that effectively as a mandate of okay hey you know this customer we've done business with before you know we're really comfortable with them. We know that we're going to get a contract done. We'll go out and we'll try to secure an asset to, you know, to go build it.

12:28And we may have it in our portfolio already. We may be, it may have been a strategic investment that we made. But once we find the data center asset, that's when we go back to the customer and say, okay, like we can commit to doing this. This is the timeline. We'll structure a contract around it. Depending upon who the customer is, there may or may not be some credit support associated with it around the scaling of the, you know, that asset. And then we'll get a commercial contract in place and we will initially fund a large portion of that project off of our own balance sheet, right? It's why you also see us raising equity, right?

12:59As we have to have the capital to accelerate the business. And then once we have that and we're making progress, you know, think about it as you're building real estate, right? You have a construction loan and then you have a stabilized asset loan. And we basically fund the construction loan piece off of our balance sheet. When we get to a more stabilized asset, that's when we go out and kind of do that trade financing or trade receivables financing with our, with our partner lenders, you know, they worked with us before. They know that these things are going to stand out. They know how they perform.

13:24And at that point in time, it's pretty easy for them to underwrite that risk.

13:43Silicon Valley is selling you a future where you're obsolete or worse, identical. At Palantir, they're witnessing something different and revolutionary. From re-industrializing the nation's defense base to shipyard workers building faster and frontline workers boosting productivity, AI is transforming work across the nation. AI is not replacing American workers or flattening them into conformity. It's unleashing what makes each one irreplaceable. Their judgment, their craft, their creativity. When American workers become more powerfully themselves, they own the future. Palantir, making Americans irreplaceable.

14:24Support for the show comes from Public.com. You're thoughtful about where your money goes. You've got your core holdings, some recurring crypto buys, maybe even a few strategic option plays on the side. The point is, you're engaged with your investments, and Public gets that. That's why they built an investing platform for those who take it seriously. On Public, you can put together a multi-asset portfolio for the long haul. Stocks, bonds, options, crypto, it's all there. Plus an industry-leading 3.6 % APY, high-yield cash account. Switch to the platform built for those who take investing seriously.

14:57Go to public.com slash market and earn an uncapped 1 % bonus when you transfer your portfolio. That's public.com slash market. Paid for by Public Investing. All investing involves the risk of loss, including loss of principal. Brokerage services for U.S.-listed registered securities, options, and bonds in a self-directed account are offered by Public Investing, Inc., member FINRA, and SIPC. Crypto trading provided by ZeroHash. Complete disclosures available at public.com slash disclosures. It's funny. Tracy and I had coffee with someone yesterday who is sort of in the space. I'm going to dox him here.

15:32And I was like, what should we ask Brian? And he's like, ask him why he won't let my company, why I'm still on the waiting list or something, or why he hasn't approved my company to use CoreWeave. But what are some of the bars or the thresholds? So, you know, apparently there's a lot of demand for compute these days. What does it take to get in the door and get access to some of your chips and electricity? So it's a great question. It's a question that we get all the time from our sales teams, right? is we're faced a lot with a sales team that is incredible at delivering product to customer and we don't have anything to sell.

16:07And it's kind of my job as the strategy organization at Core, we're responsible for two things. It's product and infrastructure capacity. And I spend most of my time going out and finding those data centers and being able to support those deals. And the growth that we had over the past 12 months, the company was pretty flat out, right? In building and delivering this infrastructure, you know publicly on our documentation page it says that we have three regions we'll have 28 regions online by the end of the year i think we delivered 11 of them in q1 alone right so we're building at a scale um you know i'd say that almost larger than some of the three big hyperscalers but in terms of how do you become a customer of core weave it's really relationship driven right is we want to make sure that we're going to be able to be successful with our customers and have an engineering relationship and we're aligned in what they need and we can deliver what they need.

16:58The last thing that we want is for somebody to walk in the door and say, hey, I need this for three weeks and two weeks into it, they're unhappy and we can't give them what they need to be successful. Our customers are making such large investments in this infrastructure that we have to have a lot of conviction that we will be successful with them and provide a good experience. So it's not that we're trying to keep people out, it's we're trying to ensure positive experiences for people that we do bring on board. Do you build complete housed facilities or is it all you're going to bring your chips and expertise into an existing tier one data center and essentially rent floor space from them?

17:37Yeah. So a year ago, it was we were effectively just a co-location tenant. And now we've gone a lot more vertical for some strategic builds where we're either a partner in the project where we own equity in the development company or building the project ourselves. ourselves, we've been scaling that team up over the past six months. And we had to, at our scale, to be able to guarantee outcomes, right? Is we were in a position where we had data centers getting delayed with things that weren't communicated to us. And, you know, we had to go build the capability to handle that situation and, you know, make sure we can still deliver for our customers.

18:10One of the differentiators that you and some of your colleagues have emphasized previously is this idea that you're designing the server clusters kind of from the ground up, whereas like other hyperscalers maybe are doing it on a sort of different mass scale. But can you walk us through like what is the benefit of doing it that way? And then secondly, does that end up being an impediment to, I guess, efficiencies or economics of scale? And how customized like do you really get here? So from a customization perspective, it's aggressive, right? And I say that because, you know, our customers are involved in the design of, you know, our network topology of the east-west fabrics for the GPU to GPU communication for things like cooling.

18:56You know, I have customers that tour the data centers under a construction process with me like once a week. And it's to the point that they're impacting how we build the base level networking products to ensure they have enough throughput to, you know, meet their use case needs. Whereas in what we call the legacy hyperscaler installations, it may be they have a couple thousand GPUs that are in a data center that was really built for CPU computation or provide services to 10 ,000 customers that is really with a much lower base expectation of what they're going to be doing. right so it's things around connectivity for storage it's things around power and cooling it's things around how they want to be able to optimize their workloads inside of the GPU to GPU communication you know we have some customers that even customize their InfiniBand fabrics and the size of those fabrics and how they connect together so you know we work with them to really understand what their use case is where they're worried currently and in the future and the design around that so it's a pretty comprehensive program when we're building something from the ground up And how much complexity does that introduce into the business?

20:05And does it end up being a limiting factor on your growth? Or is demand just so strong at the moment that it's not really an issue? The customization that we do is typically going to be above what our base level offering is, meaning the environment will be more performant because the customer required it. So it's typically not going to be limiting to us from a future revenue or resale perspective. It's going to make the asset more valuable. but we're designing our reference builds for 99 % of use cases and we're trying to price it efficiently. And then when a customer wants something above and beyond, it impacts price, but for these installations, it's probably de minimis, right?

20:42So it doesn't really add a lot of complexity for us from a business perspective. So we're happy to do it. You mentioned that some of the hyperscalers, yes, they have GPUs, but they were like built in an environment for like legacy CPUs. Can you talk a little bit about, A, just the difference between the legacy architectures and the new one? And then in the design, like what kind of bottlenecks do you run into? Is there issues with labor, like the types of people who know how to string these things together well? Or are there different cooling requirements for this type of compute environment that did not exist?

21:20Like what are the challenges in building out these sort of like fundamentally different environments? Yeah, so that's changed also in the last 12 months in that you used to be able to take what was an enterprise data center and creatively retrofit it to be capable of supporting the AI workloads to a certain density level. Instead of filling up a cabinet, you could put two servers in a cabinet and you could meet the power and cooling requirements of the installation. You used a lot more floor space, but it was doable. One of the incredible things about NVIDIA is that they're always pushing the boundary on the engineering side.

21:55and their next generation of chips is largely dependent upon much more aggressive heat transfer and they've introduced liquid cooling to the reference architectures. So as liquid cooling comes in, it changes what type of data center is capable of doing this and it truly requires that ground up redesign and almost greenfield only build to support it. You've gone from an environment where you could take an enterprise data center and deploy less servers per cabinet and get away with it to, hey, nobody's ever built this before. It's at an incredible scale and it has to happen on a yearly cadence now.

22:29So the data center industry is in a full sprint to figure out, OK, how do we do this? How do we do it quickly? How do we operationalize it? Right. And, you know, that's kind of where I've been spending all of my time over the past six months. Can I ask a really basic question? And we've done episodes on this, but I would be very interested in your opinion. But why does it feel like customers and AI customers in particular are so, I don't know if addicted is the right word, but like so devoted to NVIDIA chips? Like what is it about them specifically that is so attractive? How much of it is due to like the technology versus say maybe the interoperability?

23:10So you have to understand that when you're an AI lab that has just started and it's an arms race in the industry to deliver product and models as fast as possible, that it's an existential risk to you that you don't have your infrastructure be like your Achilles heel. Right. And NVIDIA has proven to be a number of things. One is they're the engineers of the best products. Right. they are an engineering organization first in that they identify and solve problems they push the limits you know they're willing to listen to customers and help you solve problems and design things around new use cases but it's not just creating good hardware it's creating good hardware that scales and they can support at scale and when you're building these installations that are hundreds of thousands of components on the accelerator side and the infiniband link side It all has to work together well.

24:06And when you go to somebody like NVIDIA that has done this for so long at scale with such engineering expertise, they eliminate so much of that existential risk for these startups. So when I look at it, I see some of these smaller startups saying, we're going to go a different route. I'm like, what are you doing? You're taking so much risk for no reason here. This is a proven solution. It's the best solution, and it has the most community support. Go the easy path because the venture you're embarking on is hard enough. Is it like the old – what was that old adage? Like no one ever got fired for buying Microsoft?

24:39Or IBM, yeah, yeah. Yeah, or IBM, something like that. But the thing here is that it's not even nobody's getting fired for buying the tried and true and slow or moving thing. It's nobody's getting fired for buying the tried and true and best performing and bleeding edge thing. So I look at the folks that are buying other products and investing in other products almost as like they're trying – they almost have a chip on their shoulder and they're going against the mold just to do it. There are competitors to NVIDIA that they claim cheaper or more application-specific chips. I think Intel came out with something like that.

25:18First of all, from the CoreWeave perspective, are you all in on NVIDIA hardware? We are. Could that change? The party line is that we're always going to be driven by customers, right? And we're going to be driven by customers to the chip that is most performant, provides the best TCO, is best supported. And right now, and in what I think is the foreseeable future, I believe that is strongly NVIDIA. Thinking about, okay, maybe one day you guys IPO and I'm looking through the risk factors. And one of the risk factors, we have a heavy reliance on NVIDIA chips. There is a risk that a competitor thinks.

25:55What would it take for one of these competitors that does ostensibly offer cheaper hardware or perhaps lower electricity consumption, in your view, to make one of those risk factors real? I think that they'd have to be willing to, quote unquote, buy the market. And when I say that, I mean they'd have to subsidize their hardware to get a material market share. And from what I've seen, there's no one else that's really been willing to do that so far. and what about meta with pi torch and all their chips uh so they're in-house chips um i think they have those for very very specific production applications um but they're not really general purpose chips okay right and i think that when you're building something for general purpose and there has to be flexibility in the use case um while you can go build a custom asic to solve very specific problems yeah i don't think it makes sense to invest in those to go to be a five-year asset if you don't necessarily know what you're going to do with it.

Read the full transcript

26:51So you talked about the advantages of NVIDIA hardware, like the chips themselves. But one of the things you sometimes hear is that those same chips might perform differently in different clouds. So what is it that you can do to sort of boost the performance of the same chip in your structure or ecosystem versus, say, an AWS or someone like that? Sure. Great question. We do a lot of work around this internally. And it's a big part of our technical differentiation. And what we call it internally is mission control. And mission control is effectively a portfolio of different services that we run on our infrastructure to make sure that these incredibly complex supercomputers are healthy and performant and are optimized, where we take a lot of that responsibility off of our customer engineering teams.

27:41And it sounds like that might be an easy lift, but when you're running supercomputer scale, you know, you need a team of 50 to do that, right? So we provide a ton of software automation around that, providing that health checking and observability to our customers. But it's also the engineering engagement, right? Is, you know, working with our customers to understand, okay, what are you doing? What's the best way to optimize this? How do we, you know, how did we design the data center to be more performant, to make sure your storage solution was correct, your networking solution was correct. So it's not just a, hey, CoreWeave provides like this one little thing that makes it better.

28:14It's the comprehensive solution starting from the data center design through the software automation and health checking and monitoring via mission control, via the engineering relationships that really add that value. Let's talk about electricity because this has become this huge talking point that this is the major constraint. And now that you're becoming more vertically integrated and having to stand up more of your operations, we talked to one guy formerly at Microsoft who said, you know, One of the issues is that there may be a backlash in some communities who don't want, you know, their scarce electricity to go to data centers when they could go to household air conditioning.

28:48What are you running into right now or what are you seeing? So we've been very, very selective on where we put data centers. We don't have anything in Ashburn, Virginia, right? And the northern Virginia market, I think, is incredibly saturated. There's a lot of growing backlash in that market around power usage. And, you know, just thinking about how do you get enough diesel trucks in there to refill generators that they have a prolonged outage, right? So I think that there's some markets where it's just like, okay, like stay away from that. And when the grids have issues and that market hasn't really had an issue yet, it becomes an acute problem immediately.

29:21Like just think about the Texas power market crisis back in, I think it was 2021, 2020, where the grid wasn't really set up to be able to handle the frigid temperatures. and they had natural gas valves that were freezing off at the natural gas generation plants that didn't allow them to actually come online and produce electricity, no matter how high the price was, right? So there's going to be these acute issues that, you know, people are going to learn from and the regulators are going to learn from to make sure they don't happen again. And we're kind of citing our plants and markets where our data centers and markets where we think the grid infrastructure is capable of handling it, right?

29:56And it's not just, is there enough power? It's also on things, you know, AI workloads are pretty volatile in how much power they use. And they're volatile because, you know, every 15 minutes or every 30 minutes, you effectively stop the job to save the progress you've made, right? And it's so expensive to run these clusters that you don't want to lose hundreds of thousands of dollars of progress. So they take a minute, they do what's called checkpointing, where they write the current state of the job back to storage. And that checkpointing time, your power usage basically goes from 100 % to like 10%.

30:27And then it goes right back up again when it's done saving it. So that load volatility on a local market will create either voltage spikes or voltage sags. And a voltage sag is what you see is what causes a brownout that we used to see a lot of times when people turn their air conditioners on. And it's thinking through, OK, how do I ensure that, you know, my AI installation doesn't cause a brownout when people are turning their, you know, during checkpointing, when people are turning their air conditioners on? Like, that's the type of stuff that we're thoughtful around. Like, how do we make sure we don't do this?

30:57Right. And, you know, talking to engineers and NVIDIA's engineering expertise, like they're working on this problem as well. And they're they've solved this for the next generation. So it's everything from is there enough power there? What's the source of that power? You know, how clean is it? How do we make sure that we're investing in solar and stuff in the area to make sure that we're we're not just taking power from the grid to also when we're using that power? How is it going to impact the consumers around us? I want to ask you more about what NVIDIA is doing. But just on that note, what's the most important metric for evaluating a data center's quality or performance?

31:32Is it like days without brownouts or uninterrupted power supply? Or is it measures of efficiency, like power usage effectiveness or something like that? If I'm surveying a bunch of data centers, I want to pick a good one. What should I be looking for? So right now the market's pretty thin. So right now - I won't have a lot of options. Okay. Imagine I'm like the biggest customer on earth and I can get in anywhere. What should I be looking for? So the first thing goes back to the electricity piece, right? Is the grid stable? Is there enough power supply? Is there excess renewable generation in the area that doesn't have the ability to make it to downstream consumers, right?

32:12A lot of the renewables that we have in the U.S. are built in places that don't necessarily have the consumers. So you're citing these data centers in places where you have this excess supply. So that's the first piece, right, is how good is the electricity supply and how angry are the people around me going to be if I take it? Now, you go from there into everything else is kind of solvable, right, in the way that you design it. And if you're building a greenfield, it's, okay, you know, what type of UPS systems am I putting in? Are they capable of handling that load volatility? You know, how am I thinking about my cooling solutions?

32:46There's been a big shift to liquid cooling, right? And liquid cooling from a PUE perspective isn't a 30 % to 40 % decrease in electricity utilization like people think. It's more like 60 % to 70%, right? And the reason for that is it's not just the efficiency of the data center plant. it's also that now if you're not cooling things with air you don't have to run the fans inside the servers as well and for these ai installations because they're so dense the fans consume a lot of energy right so everything that we're building now is a combination of liquid and air cooling right and the liquid cooling piece has solved the pue issue right and we're everything we're doing is trying to say okay how much power can we use only for running our critical like it operations versus cooling the environment, making sure the environment's running correctly from a resiliency perspective.

33:39And there's been big strides made there over the last 12 months.

33:58Support for the show comes from Public.com. You're thoughtful about where your money goes. You've got your core holdings, some recurring crypto buys, maybe even a few strategic option plays on the side. The point is you're engaged with your investments and public gets that. That's why they built an investing platform for those who take it seriously. On public, you can put together a multi-asset portfolio for the long haul. Stocks, bonds, options, crypto, it's all there. Plus an industry leading 3.6 % APY, high yield cash account. Switch to the platform built for those who take investing seriously.

34:31Go to public.com slash market and earn an uncapped 1 % bonus when you transfer your portfolio. That's public.com slash market. Paid for by Public Investing. All investing involves the risk of loss, including loss of principal. Brokered services for U.S.-listed registered securities, options, and bonds in a self-directed account are offered by Public Investing, Inc., member FINRA, and SIPC. Crypto trading provided by XeroHash. Complete disclosures available at public.com slash disclosures. Introducing the all-new Adobe Acrobat Studio, now with AI-powered PDF spaces. Do more with PDFs than you ever thought possible.

35:05Need AI to turn 100 pages of market research into five insights with a click? Do that with Acrobat. Need templates for a sales proposal that'll close that deal? Do that with Acrobat. Need an AI specialist to tailor the tone of your market report to sound real smart in real time? Do that with the all-new Adobe Acrobat Studio. Learn more at adobe.com slash do that with Acrobat. Does co-location trump grid reliability? Like if I'm Elon Musk building some sort of new AI thing, as I think he's doing, in Texas, say, like, am I just going to have to find a data center in Texas? Or how much flexibility do I have to use one further away?

35:49So great question. It's a different answer for different use cases at different times. And right now, we're in the middle of this rush to train, whether they're open source or proprietary foundation models, the largest, most valuable companies in the world. And they're mostly worried about access to contiguous compute capacity. How much compute can I get in one location all connected together so I can go faster than the next guy? But when the models are trained, they want that compute to then be local to their customer base, right? How do they take it from the middle of nowhere and then go serve it in the metropolitan markets?

36:25And as the use cases are more distilled and they get more real time, think like the type ahead suggestions that you get in your Gmail account, right? As you're typing something and it's getting better and better, it's, you know, that's an AI model somewhere predicting what you would want to say next. and they want to make sure that's delivered at human speed. So that human speed is a latency consideration, right? As you're citing those GPUs and you're citing that compute to be local to the people that are using it. So that move started probably four months ago where we saw customers finally becoming concerned around latency for their serving use cases.

37:03So initially training, people don't really care where it is, cheap power, reliable grid. They just need it all contiguous and they need it fast. And then down the road, as their applications find success, they're more worried about where the compute is for their customers. What are some of the areas that are going to be the next northern Virginia when it comes to data center clusters? So I think we're seeing it in Atlanta already, where Georgia has paused or has attempted to pause some of their tax incentives around it because they want to make sure they do grid studies. I think that we're probably going to see it in some of the other hotspots.

37:39You see AWS up in Oregon who is trying to find creative alternative ways to power their data centers from non-grid generation to alleviate some concerns there. But I think that the market has to solve this problem. And you're starting to see some of the startups around nuclear generation and the small reactors at the data center level as people are being thoughtful for five to 10 years from now. Do you have any influence on the type of power being built in certain areas? Could you say to a utility company of some sort, we're here, we need access to energy, but we want it to come in a particular form?

38:20So you can, but you have to understand that the investment cycles and the physical build cycles for those are so much longer than how quickly our customers need infrastructure. So you may go to a market and say, hey, we're going to be here over the next 10 years. we'd like you to install XYZ, you know, renewable. And they're happy to do it. It's just that you have to find a medium term solution while that's being built. I'm going to ask a question. So there was a news story, and maybe you won't comment on the news story specifically about CoreWeave having made a$1 billion offer for a Bitcoin miner called Core Scientific.

38:55Apparently it was rejected according to things I've read in the news. Setting aside this deal, there used to be a lot of crypto mining and then Ethereum went from proof of work to proof of stake and that all basically disappeared overnight. There are still Bitcoin miners. I never get the impression it's that great a business, but whatever. Are there Bitcoin miners that have latent value in the fact that they... I mean, I know the Bitcoin mining chips, the actual ASICs don't work for AI because all they are is Bitcoin mining chips. But are there, by dint of their access to electricity, space, et cetera, is there a fair amount of latent value in the general physical structures that they've built for the mining?

39:39So I'm just not going to answer your question at all. I'm going to go on a tangent. Okay, that's fine. So I think that when I think about CoreWeave and what our mission is, it's to find creative solutions to problems in various markets. And those various markets can be blocking for us and our customers to achieve our goals. so if power is a concern for us and power availability and substations and substation transformers bitcoin miners definitely have access to power uh that that is true i'm just stating facts here you can keep doing it uh so you know as we go and we try to solve these problems you know we're going to go to places that others may not have thought of and we're gonna go do due diligence and i'm gonna personally go and walk the sites and i'm gonna to look through and see, okay, can we pull this off?

40:31And we're going to get our engineering partners in to help us design retrofits. And we're going to do deals with the companies that we believe have the ability to provide us value. Since we're doing stuff in the news, this has been in the news for a while, so it doesn't really count, but the new NVIDIA chips, the GB200s, what will those do for CoreWeave? And when would you expect to get them? what will they do for us? It's more about what they're going to do for our customers. Right. And I think that they are, this is a great question. Um, they are going to open up a lot of both training and inference use cases in the AI side that I think our customers have been blocked by, uh, with the existing generation in that, um, you're now able to link 72 of these GPUs together to work almost as one unit.

41:27And previously that was limited to eight. They have a much larger, what's called a frame buffer, which is how much memory that's usable for their matrix operations. So, you know, I think that we're going to see a lot of new use cases show up for this stuff, but I think it extends well beyond AI as well. And it's going to be a lot more useful for things like scientific computing. One of the things that has me really excited is the computational fluid dynamics. And I'm specifically thinking about the uses for that in F1 under the new regulation in 2026. I'm excited for the new platform. I think in a year and a half, people are going to be using it for things that are different than anybody expects today.

42:08And that's to me the pace at which this is changing is the piece that's really cool. Wait, I'm sorry. I hate sports. What's the F1 2026 thing? Yeah, I was going to ask you, and explain how the NVIDIA is. Yeah, so the F1 platform, they have very tight restrictions around what type of compute and how much compute you can use to do aerodynamic testing in your cars. And you can either do real-life testing in a wind tunnel, or you can do it through CFD analysis. And one of the great uses for the Grace Blackwell and the Grace Hopper architectures in pairing that Grace super chip with the GPU is they're great for CFD workloads, right?

42:45And the - CFD stands for, oh, computational fluid dynamics. Yep. Yep. And the regulations around the existing program in F1 are they're only able to use CPUs. They have very specific limitations around it. But there's been a lot of talk of that changing for the 2026 car models. And for me, that's pretty cool. And I'm gung ho excited about possibly supporting that. That does sound very fun. I want to get back to actually the financing a little bit because I guess two questions. So the logic of why you would borrow money both, I guess, for the acquisition of chips and the chips are sort of collateral, but I understand they're not really chip backed loans per se.

43:30A, do you see your clients getting more into debt financing rather than equity financing? I mean, there's a whole generation of software companies from the Zerp era that was just, you know, all equity and never had any debt at all. And they never really had to think about like their compute costs. They did, but not as much. Do you think that will rise their own use of debt instead of equity in terms of their own financing? And another topic we talk about a lot on the show, private credit, like, is there an emergence of an ecosystem of lenders for whom this is going to become a specialty of some sort.

44:09To the first piece of the question, I don't believe that the venture-backed kind of AI lab startups will ever take on debt in this type of environment, largely because they don't have the collateral to back it if they're buying cloud services to run their infrastructure. And you may see some that start to buy their own infrastructure and to do that themselves, but it is a Herculean task to do this at scale, right? There's a reason why clouds exist is that there's a lot of complexity that they abstract away. On the second question around, is there a private credit sector that's going to be built to do this?

44:40I think that it's more, you're seeing public lenders that are extending into the private credit space because the opportunities are there. And I'm going to give you the party line answer that my CEO gives all the time is that, you know, as we're thinking about financing our business, the biggest thing for us is our cost of capital. and we're always going to do the things that provide us the lowest cost of capital. And the lenders that we work with, including Blackstone, that have been so wonderful for us, them extending on the private credit side as we go to the public markets, because we're dragged there by cost of capital concerns, I would expect them to be involved as well.

45:17So I think it's a continuation of the business they've been doing in the public markets, just kind of extending into this capital intensive business. Wait, what was, I guess you can't get into specific details, But my impression was for these types of loans that the interest rate is usually higher than like a basic bank loan or say issuing a corporate bond. I would definitely say our cost of capital is lower than some of the corporate issuances out there. OK. But, you know, it's definitely higher than if our cost of capital today is definitely higher than if we were a public entity. But specifically on the GPU-backed loans, and I know you keep saying it's not really a GPU-backed loan, but that's sort of an uphill battle to call it trade receivables financing instead.

46:01It sounds so much better that way. I know, I know. But on that in particular, okay, there's collateral, so maybe that brings the overall borrowing rate down. But on the other hand, it's kind of a new thing, new structure. How does that compare with more traditional types of financing? Yeah. So, you know, every credit facility that we do, the cost of capital declines and it's declining because it's the execution risk and the ongoing concern risk are reduced. Right. And, you know, when we first did this, people like you guys are crazy. You have no history of execution. And as we've gone through and we've done it, like now there's a path that everybody that's underwriting these loans now understands, OK, this is what happens.

46:39This is how it performs. This is what we should expect from the customers. This is what we should expect from receivables. They get more comfortable. They're willing to do it at more aggressive rates. So the risk premium associated with it has just decreased over time. Got it. I just have one last question. I sort of touched on it earlier, but okay, we know that power is scarce. We know that there's not an infinite number of NVIDIA chips, et cetera. Those are quite scarce. For the other stuff, we've done episodes in the past talking about just generic electrical gear components. And we've certainly done a lot on labor shortages.

47:13What are you seeing on that front, sort of like simple gear and the sort of basic building blocks of a new construction and how difficult that is to acquire versus, say, if you were doing this? You know, you started in 2017. I imagine a lot of things were more plentiful back then. Yeah. So it's not even that they're less plentiful today than they were. You know, the lead times were always the lead times for this electrical gear. It's that there was capacity to go buy off the shelf. Okay. There was inventory in the data center market and the inventory is basically gone. And I see deals today that get brought to me and there's seven people bidding on the same deal and they're all trying to sell it to similar customers.

47:52So the market has gotten pretty thin. So now you're looking at it going, okay, my only option here is for new build. And you're looking at lead times that haven't really shifted that much on things inside of the data center. The substation transformers are multiple years out. and part of that reason is that it takes a year for them to cure after they're manufactured like there's no getting around that there's no speeding that piece up what does that mean it takes them a year when the transformer is built that's taking on so much power that whatever the process is it has to sit for a year and harden before it's able to take on that electrical load so even if you went and said hey i'm going to build 10 more of these this year it's still a year away before you can use them huh right and those are the types of things from a manufacturing perspective you just can't get around and it takes time for the supply chain to catch up.

48:40But you know, the problems that I'm solving on a day-to-day basis in these builds isn't even around the substation transformers. It's around like small components that somebody missed when they ordered the gear 16 weeks ago. And now you have to go scramble and call in favors across the country of, Hey, who has this part? I need it by tomorrow because I have 50 ,000 GPUs that are blocked by this one little thing, right? So it's a lot of it is logistical and human coordination and solving dumb problems in real time. Brian Ventura, thank you so much for coming on AdLoss. That was fantastic. Thanks for having me.

49:23Tracy, I'm really glad we did that conversation because there are a number of these sort of like big picture ideas in there that we've sort of hit on, of course, about data centers and AI and electricity consumption. And it was really interesting to hear some of them. So like, for example, just this idea of like Northern Virginia is out and like needing this sort of hunt to find these spots in the country where there is ample electricity and basically nobody local is going to get upset at you for using it. Yeah. No one will come out with pitchforks. The thing that stood out to me from a bunch of these conversations at this point is the arms race aspect of it and how urgent building out AI is for a lot of these companies.

50:07And then there seems to be this mismatch between the immediate need for scale and compute and energy now versus these really long timelines of actually building the stuff out. And Brian mentioning the substation transformers taking a year to cure. I had no idea about that. I didn't know that either, but that's a really good example. That's super interesting. And of course, now we have to do a, how do you build a substation transformer? How do you cure a substation transformer? Totally. I mean, maybe this is probably something that electrical engineer is not interesting to them at all. But for me, I did not realize that there was this one year long, one year long curing process.

50:50You know, I think there are like a couple other things that now I like want to talk more about. So I'm interested. I mean, like CoreWeave is an NVIDIA company. It's not owned by NVIDIA, but, you know, it's joined at the hip in many respects. So how difficult is it going to be either for some other maker of chips, whether it's an Intel or some other maker of software environment, whether it's Meta and PyTorch going against CUDA or whatever? Like, that's a really interesting question to me. We have to do more essentially on how much of a lock NVIDIA really has on this industry. Yeah, this seems to be the really big question.

51:32And then the other thing I was thinking about, and I know Brian emphasized this and other CoreWeave executives have emphasized this before, but this idea that hyperscalers maybe are starting from a point of being disadvantaged because they have to retrofit all this old infrastructure for this new AI technology. Totally. And like, I can see that. But on the other hand, these are insanely impressive companies who are explicitly trying to compete against CoreWeave in this business. And they're not going to stand still. And so I guess there's an open question over how much progress they're making or how fast that progress is actually happening.

52:12Right. Right. Large incumbent companies always are going to have some challenges when there's like a new model or something. But these companies have all the money in the entire world. Right. And they also have all the, you know, one of the things that Brian said is like they if they were if one of them are going to do it, they would have to go out and buy a big chunk of the market, which, again, they have all the money in the entire world. So theoretically, whether it's the big companies and retrofitting the clouds or building new clouds or, you know, a lot of them like a Google, even if they're for now using their TPUs internally, primarily like it does seem like in theory the opportunities out there, particularly with the the sky high amount, you know, valuation that a company like NVIDIA is getting.

52:57Oh, yeah. You mentioned the sky high valuation. That was something that also stood out to me just on the financing side. So this idea of, you know, the debt financing deal that they did. And I'm not going to call it trade receivables because no one. GPU backed loan. Yeah, no one will be interested when we start talking about trade receivables. But the GPU-backed loan, this idea that, okay, it's a new structure, but the more you do it, the more the cost of that particular capital starts to fall, the more the market gets comfortable with it. I mean, we can talk about whether or not it's priced correctly for a new type of unfamiliar risk, but it does seem like that might be a new avenue for the vast amounts of capital that are needed for this business.

53:40So one, it's interesting to think about the idea that like, you know, I don't think it's like totally true, you know, that if you need compute at scale for AI, that you don't just get to call up CoreWeave and get it. And you actually have to prove that you're going to be a good customer. And so like have something that is probably going to be sustainable, have the balance sheet capacity. So this, even if the sort of software, the end users aren't themselves raising debt, it does sound like they have to have a lot of equity up front just so that they're perceived as a sustainable, viable customer for a company like CoreWeave.

54:19I also thought on the electricity front, like obviously we talk all the time about just sort of the raw demand for electricity. But this idea, what he said, and I hadn't heard anyone say it, that the runs, the modeling runs stop every, what do you say, 30 minutes and have to be saved. Oh, yeah. And so you have this big variability at times and that creates its own specific issue because it's not just steady state flow of electricity and solving for that. That's probably another area in which the legacy data centers or cloud companies, perhaps my guess would be that they're just sort of the demand is more constant and therefore something that would be a novelty for them.

54:59Just thinking about the financing more, I do kind of wonder how much of this is like AI built on top of AI on top of AI, like to the point where if the bubble were to burst or if funding was suddenly pulled from a bunch of these startups, like what would that mean for CoreWeave's financing? And what would that mean for BlackRock, which lent money based on the GPUs that the clients are taking on who might not be there anymore? I don't know. By the way, have you ever looked at a chart of Riot blockchain? Oh! No, it's interesting. Not for a while. Yeah, well, I mean, they're still there as a miner.

55:37But, like, here we are in the midst of this pretty big crypto bull run. I mean, I guess it's cooled a little bit. But, and that stock has done terribly. Yeah. So it's interesting to wonder, and apparently it doesn't seem like anyone's made a bid for them. But it is interesting to wonder, like, OK, those chips are useless for for AI because they don't work for that. But, you know, they do have capacity and they do have electricity agreements already in place. So it does make you wonder whether like some of the Bitcoin mining companies, which aren't really getting a very the market is not excited about them, clearly, even in the midst of this crypto bull run.

56:16Maybe they should go back to being a diagnostics company. That's what they were before, right? Is it? I think so. I think they're one of the ones that changed their name and then like something including blockchain. And then their shares went up enormously and now they're back down. Well, they have been. Riot Platforms has been around. OK, now I'm curious. Yeah, so it's a Bitcoin mining company, but it's been the stock has been around since 2003. So pretty clearly they were in some other business. I don't know what I mean. Yeah, I'm looking on the terminal. It says Riot Blockchain, formerly Bioptics, has ditched the drug diagnostic machinery business for the digital currency trade.

56:57Well, there you go. So if you have some sort of computing power or something, I don't know what they were doing before, but maybe it is interesting to think about maybe some of the option value for some of these miners. is in all the infrastructure other than the Bitcoin mining operation. Maybe we should put in a bid. Let's do it. We can crowdfund and start our own business. OK, maybe we should leave it there. Let's leave it there. This has been another episode of the All Thoughts podcast. I'm Tracy Allaway. You can follow me at Tracy Allaway. And I'm Joe Weisenthal. You can follow me at The Stalwart.

57:30Follow our guest, Brian Venturo. He's at Brian Venturo. Follow our producers, Carmen Rodriguez at Carmen Armin. Dashiell Bennett at Dashbot and Kale Brooks at Kale Brooks. Thank you to our producer, Moses Andam. For more OddLots content, go to Bloomberg.com slash OddLots, where you have transcripts, a blog, and a newsletter. And you can chat about all of these topics, including AI, including semiconductors, including energy in our Discord, discord.gg slash OddLots. And if you enjoy OddBots, if you like it when we talk about AI and chips and energy and all that stuff, then please leave us a positive review on your favorite podcast platform.

58:07And remember, if you are a Bloomberg subscriber, you can listen to all of our episodes absolutely ad-free. All you need to do is connect your Bloomberg account with Apple Podcasts. In order to do that, just find the Bloomberg channel on Apple Podcasts and follow the instructions there. Thanks for listening.

58:57This is Doug Gottlieb for the Doug Gottlieb Show. Reminding you, Toyotathon is on. So stop by your local Toyota dealer for the incredible year-end deals on cars, trucks, and SUVs known for their legendary reliability. Like the stylish and efficient Camry, the ready-for-anything RAV4, or check out the rough and rugged Tacoma or Tundra. All with great financing and lease options available to qualified customers. Visit buyatoyota.com to find out more. Toyota, let's go places. Residents at Brightview Senior Living Communities enjoy enhanced possibilities, independence, and choice. Brightview Dulles Corner and Great Falls offer vibrant senior independent living, assisted living, and memory care services through various daily programs, chef-prepared meals, safety and security, transportation, resort-style amenities, and high-quality care.

59:46Take financial possession of your apartment by December 31, 2025 and save. Discover more at brightviewseniorliving.com. Equal housing opportunity.

From the publisher

Everyone knows that the AI boom is built upon the voracious consumption of chips (largely sold by Nvidia) and electricity. And while the legacy cloud operators, like Amazon or Microsoft, are in this space, the nature of the computing shift is opening up new space for new players in the market. One of the hottest companies is CoreWeave, a company backed in part by Nvidia, which has grown its datacenter business massively. So how does their business actually work? How do they get energy? Where do they locate operations? How are they financed? What's the difference between a cloud AI and a legacy cloud? On this episode, we speak with CoreWeave's Chief Strategy Officer Brian Venturo about what it takes to build out operations at this scale.

See omnystudio.com/listener for privacy information.

More from Odd Lots

All 683 episodes
CoreWeave's CSO on the Business of Building AI DatacentersOdd Lots · 55 min
Listen in VO