A new market for AI compute: What GPU futures could mean for energy

0:00One of the major inputs to GPU is energy consumption. That is one of the largest variables in the pricing. So you will have the opportunity to hedge not just your GPU, but tie it to other commodities that are also actively traded.

0:14Carmen Li:So traditionally, the divergence market is 10 to 15x of the spot market size. If you look at spot today, last year, the number is$300 billion annual spend. And this year is$2 trillion. You 10x that, if we froze today, let's say to next year, the same amount of spend this year, we're looking at$30 trillion easily for that particular market. Yeah, in a normal market, though, that would take 10 years to 15 years. This is a three-year compressed time frame. So everything is very compressed in this sort of new world where AI is moving faster. This market needs to develop a lot faster because the investment cycle here is very compressed.

1:00As AI and data center expansion turn computing capacity into a critical business expense, managing GPU costs is becoming vital. That's where CME Group, in partnership with Silicon Data, is introducing Compute Futures, a new way for AI builders, hyperscalers and investors to manage price volatility. Learn more and sign up for updates at cmegroup.com slash compute.

1:28Hello and welcome to The Energy Gang, a discussion show from Wood Mackenzie about the fast-changing world of energy. I'm Ed Crooks. And on this special episode, we're going to be talking about an innovation in financial markets and what it means for technology and for energy. It actually, I think, takes quite a bit of explaining. So you're going to have to bear with us a little bit as we tell the story. But I will say you should trust me on this. It is super interesting. I was absolutely fascinated when I heard about it for the first time. And it does have some very, very important implications for the energy industry.

2:00So I certainly think you should stick with us as we tell this story. To talk about it, it's a pleasure to welcome to the show Pete Keevey. Pete is the Global Head of Energy and Environmental Products at the CME Group, which is the derivatives exchange company. Hello, Pete. Welcome to the show. Thank you, Ed. Happy to be here. Appreciate the time. And it's also a pleasure to welcome Carmen Lee, who is the founder and CEO of a company called Silicon Data. Hello, Carmen. Thank you for joining us.

2:24Carmen Li:Thank you. Glad to be here. Yeah, great to have you on the show. So before we get into the real subject of the show, I think it would be helpful for listeners to get a sense from you about what it is that you do and how you interact with the world of energy. So perhaps, Pete, to start with you, your role is head of energy and environmental products. What does that mean? What do you do in your job? That's a good question, Ed, and I've heard it frequently. What do you do at the exchange? So my job at the exchange, which everyone knows is a very large futures and options exchange, the largest globally.

3:00I oversee the business development strategy and growth of our suite of energy products and environmental products. The most well known is WTI and Henry Hub Natural Gas. I've been in these markets for 36 years. I was a trader for many years of natural gas. And I've joined the exchange to try to build our business, serve customers, find out what their needs are, build products that serve their risk management needs, and make sure they're distributed and available to all of our customer base. So, you know, in short, that's what I coordinate at the exchange. Right. Got it. And so something like what we're talking about today, which is innovation, the introduction of new products, that's kind of right in your wheelhouse.

3:42Then that's typical kind of thing that you do. Yes, we have to have one eye on the future at all times. So what's changing in the market? As you know, energy markets are always growing, always evolving. And now with GPU being so closely tied to energy markets, particularly power, it's a natural extension of our customer interest and our customer base. So we have to make sure we're looking forward, innovating on risk management needs in that space particularly. So that kind of leads us to where we are today. Absolutely. Fantastic. And so, Carmen, what about you? What's your story? Can you tell us a little bit about Silicon Data?

4:20Carmen Li:Yeah, of course. So Silicon Data, we're independent indices, benchmark providers for compute AI economy. So we have indices, physical verification layer, which is a benchmark, as well as alternative data sets for anything related to compute. So think about GPO pricing, secondary transactions, token pricing, RAM indices for suite of products for this particular underlying assets. I started my career off in energy as well. I was an option market maker for a few energy products back in Chicago due to financial crisis errors at DRW Trading. And I actually got a CME scholarship when I was in college.

5:03Carmen Li:I don't think Pete knows that, but it's a full circle for me. I didn't. Fantastic. Great, great way to get started, I'm sure. So as you've been mentioning then, so you have this data on GPU usage, a lot of issues related to compute. And what we're talking about today then is this question of having an exchange traded product for trading GPU compute. I mean, that's the basic way you'd put it, right? Yeah. So we can talk about what makes up an exchange-traded product first. Why would something trade on an exchange? And then maybe we can get into further detail on what this specific product is. Yeah, there's a lot to unpack here.

5:53It's good to break it down. There's so much ground to cover. We'll try to keep it for the audience just to say, how do these things develop? So when we look at a market, we see a spot market. So that is simply transactions where buyers and sellers are exchanging a product, a commodity, in this case, GPU. And those happen in open markets at prices, right? So why build a futures market? Why build an exchange? So there's three key functions that an exchange serves in this market. Number one, a central clearinghouse that eliminates counterparty risk. And counterparty risk is always an enormous part of all financial markets.

6:37And managing that takes up an enormous amount of time and resources. Secondly, liquidity. So how does a market develop liquidity? A centralized order book is the key to developing liquidity. So that means all bids and offers and desires to buy and sell markets get routed, whether through technology or through other means, into a centralized order book where they can be matched. So you simply put buyers and sellers together. That is the second key function of this market is to match them. And it has always traditionally for hundreds, thousands of years worked to concentrate liquidity and to bring buyers and sellers together in standardized fashion.

7:23The last thing, and this is in these days almost the most important, is price transparency and benchmark pricing. So that means when you trade on an exchange, the price that you have transacted out is available for the entire market to observe. And what that does is an exchange also adds term structure, which means you get forward pricing and liquid. So transparency, liquidity and forward pricing. And that benchmark doesn't just serve a function for traders. So buyers and sellers. While the main function of a futures contract is to hedge risk, a secondary function is to provide a benchmark for economic decision making and make sure that that pricing is well known and observed and accepted as standard throughout the industry.

8:17And when we looked at GPU in particular, this is the area where we really started getting excited about this market. Right. That's fantastic. Excellent introduction then to, as you say, an exchange of what it does and why it's important. I think the thing that's really important to unpack then is the question of what it means to be trading GPU. Because I think that's a concept a lot of people will have difficulty getting their heads around. I know I certainly did. I don't know, Carmen, do you want to talk about this a little bit? As I say, when we talk about trading GPU, what do we actually mean by that?

8:49Carmen Li:This is a really good question. In the spot market, if you need GPU resources, there's two ways you can get GPU resources. Number one, you can reserve them on a per hour, hourly basis, just like reserve sort of a traditional CPU resources that hyperscalers, for example, right? Or you can buy the GPU, right, and put it on your own co-location and use them. That's yours, for example, right? The predominant way today's environment is people rent them in a longer term contract, either with hyperscalers or neoclouts. So the upside for doing so is you're now locked in to spend millions of dollars buying those GPU servers.

9:30Carmen Li:And you can be flexible how many hours you want to use it for, for a given term or given chip types. So reserve contracts as per hour basis is a dominant way for people to get a compute. So that comes down to, okay, if I'm buying compute, I'm actually just reserving GPU resources on hourly basis. I can tell someone, hey, I need GPU for, you know, next two years, you know, for this particular specification. And we'll rent them from that particular provider for the next two years. So I'll lock in prices for X dollar per GPU per hour as a unit price. Or you can say, hey, I just want it for the next hour.

10:09Carmen Li:That's kind of on-demand prices. So you're really paying for the hour. But again, if you want to use for the next hour, you're at the next hour supply-demand curve. Right, got it. And so the sellers of this then would be essentially data center owners, operators. And they say, we have this GPU processing time to offer to people. And then the users would be people like AI companies, presumably anyone that needs cloud services, anyone that needs compute time. That's right. So the provider can be hyperscalers, can be NeoCloud providers, can be data center who can really do bad metal, right? Rental, those kind of services.

10:51Carmen Li:So it can be anyone. It can be someone who has 509s in their basement, right? If you need 509, which is, you know, very consumer-grade GPUs. So it can be people owning those assets. And the people using them can be AI companies, right? The AI model labs, but can be token factories. They need GPU to produce two tokens, right? Or they can be enterprises, right? Enterprises just want to, you know, reserve a bunch of GPUs and produce intelligence on top of that. Right. And so how does that market work at the moment then? I mean, do you have to, do you kind of pick up the phone and say, Amazon, I need, as you say, I need an hour in your data center in three days time.

11:30Can you give me that? How much will it cost?

11:32Carmen Li:It is really fascinating. And so I've been with this market for the last three years. And I literally saw the sort of proliferation from the provider side, right? So when I started, there's really three, four new hyperscalers. And to your point, it depends on the size of your trade. You can go online and just say, hey, on demand, two clusters, USC, Storm, you're down. Or you can call them and say, hey, I need 20 ,000 GPUs. And that's definitely worth a call, right? Because there's a lot of resources you're asking for. And the price will be very different. And there's probably a few new cloud providers back then, but now we have over 200 new cloud providers globally.

12:11Carmen Li:So all depends. And just side note, aside from silicon data, I also run a company called ComputeChange, which precisely add to your point, we facilitate through a platform our fuel port process for spot transactions. So people actually procure GPU resources through the platform. Right. And so to Pete's point then about the advantages of being on an exchange, if you're not on an exchange, you have these issues about lack of price transparency, presumably. No one knows exactly how much anyone is paying for how much GPU time. And also potential lack of liquidity. We don't know where the liquidity is.

12:49Bars and sellers are not always being matched together. So presumably there's a lot of kind of strains in that trading system as it works today.

12:55Carmen Li:100%. So everything in the spot exchange, compute exchange, right, is the counterparty risks, right? So for example, if you sign a contract with any new cloud providers or platform, you and that new cloud provider, that's it between you guys, right? And obviously, the contract usually lasts longer than a month, can be two years, three years. And what happened to each one of you guys, then no one knows, right? So that's kind of the really raised premium. So you're looking at a spot market per se. Right. And you talk about the market having evolved a lot just in the past two or three years. Is this very much being driven by AI?

13:31Is that really what's creating a lot of the demand here? And also, I guess, a lot of the trading activity because we're getting so much more data center capacity being built out.

13:38Carmen Li:It is. I mean, everyone can sort of predict the supply side. You can calculate how much TMC can remanufacture every single day. You can figure out data center, you know, the really constraint a lot of times energy, right? People can talk about that. From the demand side, at least I don't know, right? I think, you know, I expect one thing to shoot up when new model release happens. And people think, oh, this is great. I can use that for this and that, right? So then you see different demand shifting from one workflow to the other. So, yeah, we definitely see tremendous growth in terms of the market adoption of AI globally.

14:14Right. And so then what you're talking about now then is taking that over-the-counter market, essentially, and putting that on an exchange, right? putting it on CME. So what does that mean?

14:25Carmen Li:Yeah, the oil counter market today is spot, right? So meaning it's a reserve contract starting today. It's not the future, right? So we actually leveraged the transaction data we generated from the spot markets together with other providers of the ecosystem and generating the daily set-up of prices, right? So then the indices is used to leverage by CME and P's team to become in the future. to settlement prices, right? Down the line. Right. And so a typical contract, I mean, have you actually specified what the contract's going to look like? Then give me an example of what the contract might say.

15:03So good question. So let me just talk about first where the market structure sort of dictates what type of contract makes sense for the marketplace, Right. And so where do we see market structure going? The first thing is, you know, and Carmen just alluded to it. Much of the capacity is today sold through RFQs, through long term leases. And that's changing over time. Right. So you get so you have an increased user base, constantly expanding user base. But their demand profiles are highly unique and different for each one. and the market's highly fragmented. So one of the challenges for an exchange is to make sure that a product that we list, okay?

15:49And this product is just a financially settled reference to Carmen's index. So part of it's the index, part of it is who is involved in the exchange, right? So we have to, you have to make a decision with a highly fragmented market, with a highly fragmented customer base, a product has to serve the proper economic function for as many users as possible without over correcting into one or another. Right. And and markets get disintermediated over time. So these large selling structures tend to get fragmented because the market is getting too big. Right. So so that brings us to the point of what are we doing?

16:27So we if the common tradable function today is a rental hour, we have to come up with what we think is what matches the economic function and the economic or transactional nature of the underlying market. So customers today rent space by the month for a contiguous 24-hour period, right? So one day, one hour, 24-hour period, they rent use on the space. So we came up with the concept of what's the average monthly consumption and create a monthly contract, right? So how many hours are in a month? There's an average of about 730, which is the normal standard function. So you have a 730 hours in a month, and then it's how much per hour and how much per hour per chip.

17:19And that develops the product. So, Carmen's index is on the H100 NVIDIA chip, which is a highly installed, observable, and common chip to form many functions in the market. And that forms the basis of the index and the basis of the first futures contract. So, as you said, it's going to be a futures contract. How far out into the future is it going to go? Good question. If you look at oil, for instance, oil is listed out 15 years in some cases, 10 years. When we look at GPU, in order to get a viable economic signal to make decisions, you need to be out 36 months, which is where we'll start. So that now not all of those will be hugely active.

18:07As you said, a lot of the volume is concentrated in the first few months because it's building from an active spot market, which is literally traded by the hour. and then it will move into multi-month periods as customers lock in longer periods of time and they get better observations and better economic certainty. So all futures contracts start with a liquid front month and a first few months and then liquidity builds as the market gets more confidence and starts indexing longer and longer term deals to that futures market. And we see it kind of building very similarly to the other futures markets.

18:45Right. So going back to your point that you opened with about the three crucial advantages of having an exchange. How is that going to apply then, do you think, to GPU trading? What are the benefits going to be of having this product on an exchange? So, you know, the benefits, obviously, in risk intermediation, counterparty risk intermediation are quite high. But there's the secondary benefit. One of the greatest benefits is that one of the major inputs to GPU is energy consumption. And that is one of the largest variables in the pricing. So you will have the opportunity to hedge not just your GPU, but tie it to other commodities that are also actively traded.

19:30So many of the variables associated with GPU economics are fixed. You know, the cost of a data center build, you know, the delays connection to a grid, And, you know, but a lot of those can be managed. A data center may build behind the meter gas fire power generation to serve the data center. But then they're exposed to the price of gas. And that may need to be hedged as well. So you put together the total economic exposure. And that is why the customer base interest is not just from people who create GPU or directly consume GPU. But there's also lenders who are lending enormous sums of money in very short periods of data center construction or to projects to support the build out.

20:16They are all effectively exposed in one way or another to the cost and also to the production value of GPU. And they don't actually today have a viable way to look into the future. So, you know, they can hedge as many of the inputs as possible in their economic chain. And that's the goal. That's the benefit of being on an exchange. Right. That's really fascinating. So thinking about it from an energy industry perspective, as you say, a lot of people involved in the energy business, the actual companies themselves, lenders, investors, worry about future energy demand. We're moving into this period of apparently rapid growth in demand for power, particularly in the United States, but other countries around the world.

21:02that has very significant implications for the price of power for the price of gas as you were saying but there's a whole lot of uncertainty about it no one knows exactly how it's going to play out there's clearly a lot of risk and uncertainty in that market and so what you're saying is this is creating a tool to help people manage that uncertainty that could actually be useful to people in the energy industry as well as people directly involved in tech in compute as data center operators or AI companies or whatever. Yeah. I mean, and we see it playing out today. For instance, the need for generation at PJM is one example where one of the main data center demand areas is in Virginia and economic decisions have to be made about where to build the baseload generation, where to go behind the grid.

21:52All of those decisions depend on knowing the revenue that will be coming in on your GPU production. And so the more information they have, the better the decisions will be made. And also where to locate. So for instance, Texas and Virginia are the two biggest build areas right now. One of them in Texas shares a very cheap power grid or at least an economic power grid. So the companies are already trying to solve or many of them are solving for the energy cost issue and making assumptions based on that. And I think if you can start pricing the output, then you can start solving in a better way for some of those assumptions.

23:12Right, so just to walk through then exactly how this might work. So the kind of, you know, the natural buyers in this market then are going to be companies that think they're going to need GPU compute services in the future. And there's various companies, companies using cloud services, AI companies, whoever it might be, on the buy side. And on the selling side, it could be the data center operators themselves, or it could also be energy companies. Is that right? So as I say, when I think about those energy companies hedging risk in the compute services market, this is a tool they could use to do that.

23:53Is that right? If you are a producer of energy and your investment case depends on selling to data centers, as an example, then you should have a strong interest. You may not trade GPU, but you have a strong interest in understanding whether the demand for that power will continue in a certain region or just generally in the market. Where do I locate? Where do I invest? So, again, that is the second order of huge benefit of a futures market is you can make that decision. So, again, you may not you're selling your second order influence that may have a second order influence on your economic decision, but it's a vital one.

24:33So do we keep building here? Do we keep building there? Do we think that the GPU demand curve is going to support all of this investment? Should we have a 10-year plan to continue to build here? Or should we see what happens? So today, the demand curve is exponentially up, right? But when will that level off? When will it shift? And also how? So today we have a huge build in, you know, call it like baseload production of GPU. But there's other parts to GPU like inference. We don't know where that's going to get set up. And so the GPU market itself is evolving and will continue to evolve. And so will the products.