SF Compute: Commoditizing Compute

24:45Honestly, I think we're the only thing that actually is like a real market, that there's like bids and asks, and there's like a trading engine that combines everything and so on. I think we're the only place where you can do things that a market should be able to do. Like you can go on SF Compute today and you can get thousands of H100s for an hour if you want. And that's because there is a price for thousands of GPUs for an hour. That is like not a thing you can reasonably do on kind of any other cloud provider because nobody should realistically sell you thousands of GPUs for an hour. they should sell it to you for a year or so on.

25:17But one of the nice things about a market is that you can buy the year on SF Compute, but then if you need to sell back, you can sell back as well. And that opens up all these little pockets of liquidity where somebody who's just trying to buy for a little bit of time, some burst capacity, so people don't normally buy for an hour. That's not like actually a realistic thing, but it's like the range. Somebody who wants, who was like us, who needed to buy for a month, can actually buy for a month. They can like place the order and there is actually a price for that. And it typically comes from somebody else who's selling back.

25:42Somebody who bought a longer term contract and they bought for some period of time, their code doesn't work, and now they need to sell off a little bit. What are the utilization rates at which a market like this works? What do you see the usual GPU utilization rate and at what point does the market get saturated? Assuming there are not hardware problems or software problems, the utilization rate is near 100 % because the price dips until the utilization is 100%. So the price actually has to dip quite a lot in order for the utilization not to be. That's not always the case because you just have logistical problems.

26:17Like you get a cluster and parts of the InfiniBand fabric are broken and there's like some issue with some switch somewhere and so you have to take some portion of the cluster offline or, you know, stuff like this. Like there's just underlying physical realities of the clusters. But nominally we have better utilization than basically anybody because, but that's on utilization of the cluster. Like that doesn't necessarily translate into, I actually do think we have much better overall money made for our underlying vendors than kind of anybody else. We work with the other GPU clouds. And the basic pitch to the other GPU clouds is, one, we're still your broker.

26:52So we can find you the long term contracts that are at the prices that you want. But meanwhile, your cluster is idle. And for that, we can increase your utilization and get you more money, because we can sell that idle cluster for you. And then the moment we find the bigger customer and they come on, you can kick off those people and then go to the other ones. You get kind of the mix of like, sell your cluster at whatever price you can get on the market, and then sell your cluster at the big price that you want to do for a long-term contract, which is your ideal business model. And then the benefit of the whole thing being on the market is you can pitch your customer that they can cancel their long-term contract, which is not a thing that you can reasonably do if you are just the DBU cloud.

27:29If you're just the DBU cloud, you can never cancel your contract because that introduces so much risk that you would otherwise like not get your cheap cost capital or whatever. But if you're selling it through the market or you're selling it with us, then you can say, Hey, look, you can cancel for a fee. And that fee is the difference between the price of the market and then the price that they paid at, which means that they canceled and you have the ability to offer that flexibility, but you don't have to take the risk of it. The money's already there and like you got paid, but it's just being sold to somebody else.

27:58One of our top pieces from last year was talking about the H100 glut from all the long-term contracts that were not being fully utilized and being put under the market. You have on here a dollar per hour contracts as well as it goes up to two. Actually, I think you were involved. You were obliquely quoted in that article. I think you remember. I remember this. Because this was hidden. Well, we hid your name, but then you were like, yeah, it's us. Yeah. Could you talk about the supply and demand of H100s? Was that just a normal cycle? Was that like a super cycle because of all the VC funding that went in?

28:33in 2003. What was that? Like GPU prices have come down. Yeah, GPU prices have come down. Some part has normal depreciation cycle. Some part of that is just, there were a lot of startups that bought GPUs and never used them and now they're lending it out and therefore you exist. There's a lot of like various theories as to why this happened. I dislike all of them because they're all kind of like, they're often said with really high confidence and I think just the market is much more complicated than that. And so everything thing I'm going to say is like very hedged. But there was a series of like places where a bunch of the orders were placed, and people were pitching to their customers and their investors and just the broader market that they would arrive on time.

29:18And that is not how the world works. And because there was such a really quick build out of things, you would end up with bottlenecks in the supply chain somewhere that has nothing to do with necessarily the chip. It's like the infiniband cables or the mix or like whatever, or you need a bunch of like generators or you're, you don't have data center space or like, but like there's always some bottleneck somewhere else. And so a lot of the clusters didn't come online within the period of time, but then all the bottlenecks got sorted out and then they all came online all at the same time. So I think you saw a shortage because supply chain hard, and then you saw a increase or like a glut because supply chain eventually figure itself out.

30:00And specifically, people over-ordered in order to get the allocations that they wanted. Then they got the allocations, and then they went under, yeah, whatever, right? There was just a lot of shenanigans. A caveat of this is every time you say somebody over-ordered, there is this assumption that the problem was the demand went down. And I don't think that's the case at all, and so I want to clarify that. It definitely seems like there's more demand for GBUs than there ever was. It's just that there was also more supply. So at the moment, I think there is still functionally a glut. But the difference that I think is happening is mostly the test time inference stuff, that you just need way more chips for that than you did before.

30:36And so whenever you make a statement about the current market, people sort of take your words and then they assume that you're making a statement about the future market. And so if you say there's a glut now, people will continue to think there's a glut. But I think what is happening at the moment, my general prediction is that by the winter, we will be back towards shortage. But then also, this very much depends on the rollout of future chips. And that comes with its own. I think I'm trying to give you like a good, here's Evan's forecast. But I don't know if my forecast is very... Nobody's going to hold you to it.

31:10But I think people want to know what's true and what's not. And there's a lot of vague speculations from people who are not that close to the market, actually. And you are. I think I'm a close to the market, but also a vague speculator. I think there are a lot of really highly confident speculators, and I am indeed a vague speculator. I think I have more information than a lot of other people, and this makes me more vague of a spectator, because I feel less certain or less confident than I think a lot of other people do. The thing I do feel reasonably confident about saying is that the test time inference is probably going to quite significantly expand the amount of compute that was used for inference.

31:46So a caveat of this is pretty much all the inference demand is in a few companies. A good example is lots of bio and pharma was using H100s, training sort of the bio models of sorts. And they would come along and they would buy thousands of H100s for training. And then just like not a lot of stuff for inference, not relative to like an opening eye or an anthropic or something, because they don't have a consumer product. Their inference event, if they can do it, right? There's really like only one inference event that matters. And obviously I think they're going to run in batch and they're not going to literally just run one inference event.

32:20But like the one that produces the drug is the important one, right? And I'm dumb and I don't know anything about biology, so I could be completely wrong here. But my understanding is that's kind of the gist. I can check that for you. You can check that for me. Check that for me. But my understanding is like the one that produces the sequence that is the drug that cures cancer or whatever, that's the important deal. But a lot of models look like this, where they're sort of more enterprise-y use cases or they're... So prior to something that looks like test time inference, you got lots and lots of demand for training and then pretty much entirely fell off.

32:53for inference. And I think like we looked at like open router, for example, the entirety of open router that was not anthropic or Google, uh, like, or Gemini or opening eye or something. It was like 10 H100 nodes or something like that. It's just like not that much. Um, it's like not that many GPUs actually to service that entire demand, but that's like a really sizable portion of the sort of open source market. But the actual amount of compute needed for it was not that much. But if you imagine what an open AI needs for GPT-4, it's tremendously big. But that's because it's a consumer product that has almost all the inference demand.

33:30Yeah, that's a message we've had. Roughly open source AI compared to closed AI is like 5%. Yeah, it's super small. It's super small. But test time inference changes that quite significantly. so I will expect that to increase our overall demand but my question on whether or not that actually affects your compute price is entirely based on how quickly do we roll out the next chips the way that you burst is different for test time. Any thoughts on the third part of the market which is the more peer-to-peer distributed somewhere like crypto enable like hyperbolic prime intellect and all of that where do those fit?

34:11Do you see a lot of people will want to participate in a peer-to-peer market or just because of the capital requirements at the end of the day it doesn't really matter I'm like wildly skeptical of these to be frankly the dream is like steady at home right I got this 1590 nobody has 1590 4090 sitting at home I can rent it out yeah like I just don't really think this is going to ever be more efficient than a fully interconnected cluster with InfiniBand or you know whatever the next spec might be like I could be completely wrong but Speedolite is really hard to beat and regardless of whatever you're using you just like can't get around that physical limitation and so you could like imagine a decentralized market that still has a lot of places where there's like co-location but then you would get something that looks like SF Compute and so that's what we do that's why we take our general take is like on SF Compute you're not buying from like random people you're buying from the other GPU clouds you're buying from data centers that are the same genre of people that you would work with already and you can specify, oh, I want all these nodes to be co-located.

35:15And I don't think you're really going to get around that. And I think I buy crypto for the purposes of transferring money. The financial system is quite painful and so on. I can understand the uses of it to sort of incentivize an initial market or try to get around the cold start problem. We've been able to get around the cold start problem just fine. So I didn't actually need that at all. what I do think is totally possible is you could launch a token and then you could subsidize the compute prices for a bit. But maybe that will help you. I think that's what Nuse is doing. Yeah, I think there's lots of people who are trying to do things like this, but at some point that runs out.

35:54So I would, I think, generally agree. I think the only thread in that model is very fine-grained mixture of experts. Algorithms can shift to adapt to hardware realities. The hardware reality is like, okay, it's annoying to do large collocated clusters, then we'll just redesign attention or whatever in our architecture to distribute it more. Yeah. There was a little bit of a block attention last year that Strong Compute made a big push on. But I think in a world where we have 200 experts in the MOE model, it starts to be a little bit better. I don't disagree with this. I can imagine the world in which you've redesigned it to be more parallelizable across space.

36:43But assuming without that, your hardware limitation is your speed of light limitation. And that's a very hard one to get around. Any customers or stories that you want to shout out of maybe things that wouldn't have been economically viable like others? I know there's some sensitivity on that. My favorites are grad students, are folks who are trying to do things that would normally otherwise require the scale of a big lab. And the grad students are like the worst possible customer for the traditional GPU clouds, because they will immediately turn if you sell them a thing, because they're going to graduate and they're not going to go anywhere.

37:22That project isn't continuing to spend lots of money. Sometimes it does, but not if you're working with the university or you're working with a lab of some sort. But a lot of times it's just like the ability for us to offer like big burst capacity, I think is lovely and wonderful. And it's like one of my favorite things to do because all those folks look like we did. And I have a special place in my heart for young hackers and young grad students and researchers who are trying to do the same genre of thing that we are doing. For the same reason, I have a special place in my heart for the startups, the people who are just actively trying to compete on the same scale, but can't afford it time-wise, but can't afford it spike-wise.

38:03Yeah, I liked your example of I have a grant of 100K and it's expiring. I got to spend it on that. That's really beautiful. Has there been interesting work coming out of that? Anything you want to mention? Yeah, so from a startup perspective, like Standard Intelligence and Find, P-H-I-N-D. We've had them on the pod, yeah. Yeah. That was great. And then from grad students' perspective, we worked a lot with the Schmidt Futures grantees of various sorts. My fear is if I talk about their research, I will be completely wrong to a sort of almost insulting degree because I am very dumb. But yeah. I think one thing that's maybe also relevant startups and GPUs-wise is there was a brief moment where it kind of made sense that VCs provided GPU clusters.

38:48And obviously you worked at AI grants, which set up Andromeda, which is supposedly a$100 million cluster. Yeah, I can explain why that's the case or why anybody would think that would be smart. Because I remember before any of that happened, we were asking for it to happen. Yeah. And the general reason is credit risk. Again, it's a bank. I have lower risk than you. I do the credit transformation. I take your risk onto my balance sheet. Correct, exactly. If you wanted to go for a while, if you wanted to go set up a GPU cluster, you had to be the one that actually bought the hardware and racked it and stacked it, like co-located it somewhere with someone.

39:23Functionally was like on your balance sheet, which means you had to get a loan. And you cannot get a loan for like$50 million as a startup, like not really. You can get like venture debt and stuff, but like it's like very, very difficult to get a loan of any serious price for that. But it's like not that difficult to get a loan for$50 million. if you already have a fund or you already have like a billion dollars under your assets somewhere. Or like you personally can like do a personal guarantee for it or something. Like if you have a lot of money, it is way easier for you to get a loan than if you don't have a lot of money.

39:55And so the hack of a VC or some capital partner offering equity for compute is always some arbitrage on the credit risk. That's amazing. Yeah. That's a hack. You should do that. I don't think people should do it right now. I think the market has like, I think it made sense at the time and it was helpful and useful for the people who did it at the time. But I think it was a one-time arbitrage because now there are lots of other sources that can do it. And also I think like, it made sense when no one else was doing it and you were the only person who's doing it. But now it's like, it's an arbitrage that gets competed down.

40:27Sure. So, I don't know, it's like super effective. I wouldn't totally recommend it. Like, it's great that Andromeda did it, but the marginal increase of somebody else doing it is like not super helpful. I don't think that many people have followed in their footsteps. I think maybe Andreessen did it. Yeah. That's it. I think just because pretty much all the value flows to Andromeda. That cannot be true. How many companies are in the AirGrants? Like 50. My understanding of Andromeda is it works with all the NFGG companies, or like several of the NFGG companies. But I might be wrong about that. Again, you know, something, something, Nat, don't kill me.

41:05I could be completely wrong. But the, but you know, I think Andromeda was like an excellent idea to do at the right time in which it occurred. His timing is impeccable. Timing. Yeah. Nat and Daniel are like, I mean, there's lots of people who are like, yeah, Seer, like S-E-E-R. Oh, Seers. Like Seers. Oh, Seers, the mall. Yeah. They, for years and years before any of the like ChatGPT moment or anything, they had fully understood what was going to happen. Like way, way before. AI Grant is like five years old, six years old or something like that, seven years old when it first launched. It depends where you start.

41:44The non-profit version. Yeah, the non-profit version was happening for a while, I think. It's going on for quite a bit of time. And then Nat and Daniel are the early investors in a lot of the early AI labs of various sorts. They've been doing this for a bit. I was looking at your pricing yesterday. We were kind of talking about it before. and there's this weird thing where one week is more expensive of both one day and one month. What are some of the market pricing dynamics? What are things that, to somebody that is not in the business, this looks really weird. But I'm curious if you have an explanation for it that looks normal to you.

42:18Yeah, so the simple answer is preemptible pricing is cheaper than non-preemptible pricing. And the same economic principle is the reason why that's the case right now. That's not entirely true on SF Compute. SF Compute doesn't really have the concept of preemptible. Instead, what it has is very short reservations. So you go to a traditional cloud provider and you can say, hey, I want a reserve contract for a year. We will let you do a reserve contract for one hour, which is the part of SFC. But what you can do is you can just buy every single hour continuously and you're reserving just for that hour.

42:48And then the next hour you reserve just for that next hour. And this is obviously like a built-in, this is like an automation that you can use. But what you're seeing when you see the cheap price is you're seeing somebody who's buying the next hour, but maybe not necessarily buying the hour after that. So if the price goes up too much, they might not get that next hour. And the underlying part of this, of where that's coming from in the market, is you can imagine like day-old milk, or like milk that's about to be old, it might drop its price until it's expired. Because nobody wants to buy the milk that's in the past, or maybe you can't legally sell it.

43:18Compute is the same way. You can't sell a block of compute that is in the past. And so what you should do in the market, and what people do do, is they take a block of compute and then they drop it and drop it and drop it and drop it into a floor price right before it's about to expire. And they keep dropping it until it clears. And so anything that is idle drops until some point. So if you go on the website and you set that chart to like a week from now, what you'll see is much more normal looking sort of curves. But if you say, oh, I want to start right now, that immediate instant, here's the compute that I want right now is functionally the preemptible price.

43:59It's where most people are getting the best compute or like the best compute prices from. The caveat of that is you can do really fun stuff on SFC if you want. So, because it's not actually preemptible. It's reserved, but only reserved for an hour, which means that the optimal way to use SFCompute is to just buy on the market price, but set a limit price that is much higher. So you can set a limit price for like$4 and say, oh, if the market ever happens to spike up to$4, then don't buy. I don't want to buy at that price for that hour. But otherwise, just buy at the cheapest price. And if you're comfortable with that, of the volatility of it, you're actually going to get really good prices, like close to $1 an hour or so on, sometimes down to like 80 cents or whatever.

44:40You said four, though. Yeah. So that's the thing. You want to lower the limit? So four is your max price. Four is where you basically want to pull the plug and say, don't do it. Because the actual average price is not, or the preemptible price doesn't actually look like that. So what you're doing when you're taking four is always, always, always give me this compute, like continue to buy every hour. Don't preempt me. Don't kick me off. And I want this compute. Um, and just buy at the preemptible price, but never kick me off. The only times in which you get kicked off is if there is a big price spike and, you know, let's say one day out of the year, there's like a$4 an hour price because of some weird fluke or something.

45:13If there are other periods of time, you're actually getting a much lower price than you. It makes sense. Your average cost that you're actually paying is way better. And your trade-off here is you don't literally know what price you're going to get. So it's volatile. But your actual average historically has been, like everyone who's done this, has gotten wildly better prices. And this is like one of the clever things you can do with the market. If you're willing to make those trade-offs, you can get a lot of really good prices. You can also do other things like you can only buy at night, for example.

45:44So the price goes down at night. And so you can say, oh, I want to only buy if the price is lower than$0.90. And so if you have some long-running job, you can make it only run on$0.90, then you can cover back and so on. So what you can kind of create as a spot inst is what the CPU world has. Yes. But you've created a system where you can kind of manufacture the exact profile that you want. Exactly. That is not just whatever the hyperscale is offering you, which is usually just one thing. Correct. SF Compute is like the power tool of GP financing. The underlying primitives of hourly compute is there.

46:21Correct. Yeah, it's pretty interesting. I've often asked OpenAI, so all these guys, Claude as well, they do batch APIs. So it's half off of whatever your thing is. And the only contract will return in 24 hours. Sure. And I was like, 24 hours is good, but sometimes I want one hour, I want four hours, I want something. And so based off of SF Compute's system, you can actually kind of create that kind of guarantee that would be like, you know, not 24, but within eight hours, within four hours, like the work, half of a workday, I can return your results to you. And if your latency requirements are like that low, actually it's fine.

46:59Yes, correct. You can carve out that. You can financially engineer that on SFC. Yeah. I mean, I think to me that unlocks a lot of agent use cases that I want, which is like, yeah, I worked in a background, but I don't want you to take a day. Yeah, correct. Take a couple hours or something. Yeah. This touches a lot of my background because I used to be a derivatives trader. And this is a forward market. A futures forward market, whatever you call it. Not a future. Not yet a futures. Yes. We can talk about that one. But I don't know if you have any other points to talk about. So you recognize that you are a marketplace and you've hired, I met Alex Epstein at your launch event.

47:36And you're building out the financialization of GPUs. Yeah. So part of that's legal. Part of that is listing on an exchange. Maybe you're the exchange. I don't know how that works. But just talk to me about that. From the legal, the standardization, where is this all headed? Is this a full listed on the Chicago Mercantile Exchange or whatever? What we're trying to do is create an underlying spot market. That gives you an index price that you can use. And then with that index price, you can create a cash-tuddled future. And with a cash-tuddled future, you can go back to the data centers And you can say, lock in your price now and de-risk your entire position, which lets you get cheaper cost of capital and so on.

48:16And that, we think, will improve the entire industry because the marginal cost of compute is the risk as shown by that graph and basically every part of this conversation. It's risk that causes the price to be all sorts of funky. And we think a future is the correct solution to this. So that's the eventual goal. Right now, you have to make the underlying smart market in order to make this occur. And then to make the smart market work, you actually have to solve a lot of technology problems. You really cannot make a smart market work if you don't run the clusters, if you don't have control over them, if you don't know how to audit them, because these are supercomputers, not soybeans.

48:52They have to work in a way that, like, it's just a lot simpler to deliver a soybean than it is to deliver it. I don't know, talk to the soybean guys. Sure, yeah. But you have to have a delivery mechanism. Your delivery mechanism, like somebody somewhere, has to actually get the compute at some point, and it actually has to work, and it is really complicated. And so that is the other part of our business, that we go and we build a bare metal infrastructure stack that goes, and then also we do auditing of all the clusters. You sort of de-risk the technical perspective, and that allows you to eventually de-risk the financial perspective.

49:23And that is kind of the pitch of SF Compute. Yeah. I'll double-click on the auditing on the clusters. Yeah. This is something I've had conversations with Vite on. He started Rika, and I think he had a blog post which kind of shone the light a little bit on how unreliable some clusters are versus others. Correct, yeah. And sometimes you kind of have to season them and age them a little bit to find the bad cards. Correct, you have to burn them in, yeah. So what do you do to audit them? There's a burn-in process, a suite of tests, and then active checking and passive checking. Burn-in process is where you typically run LINPACK.

49:54LINPACK is this thing that, like a bunch of linear algebra equations that you're stress testing. This is a proprietary thing that you wrote. No, no, no, LINPACK. LINPACK is like the most common form of burn-in. If you just type in burn-in, typically when people say Burnin, they literally just mean Limpak. It's like an NVIDIA reference version of this. Again, NVIDIA could run this before they ship, but now the customers have to do it. It's annoying. You're not just checking for the GPU itself. You're checking the whole component, all the hardware. It's an integration test. It's an integration test.

50:22So what you're doing when you're running Limpak or Burnin in general is you're stress testing the GPUs for some period of time, 48 hours, for example, maybe seven days or so on, and you're just trying to kill all the dead GPUs. or any components in the system that are broken. And we've had experiences where we ran Linpack on a cluster, and it rounds out. Like, you know, sort of comes offline when you run Linpack. This is a pretty good sign that maybe there is a problem with this cluster. And so Linpack is like the most common sort of standard test. But then beyond that, what you do is we have like a series of performance tests that replicate a much more realistic environment as well that we run just assuming if Linpack works at all, then you run the next set of tests.

51:07And then while the GPUs are in operation, you're also going through and you're doing active tests and passive tests. Passive tests are things that are running in the background while somebody else is running, while some other workload is running. And active tests are during idle periods, you're running some sort of check that would otherwise interrupt something. And then the active tests will take something offline, basically. Or a passive check might mark it to get taken offline later, and so on. And then the thing that we are working on that we have working partially but not entirely is automated refunds which is basically like yeah is the case that the hardware breaks so much and there's only so much that we can do and it is the effect of pretty much the entire industry so a pretty common thing that i think happens to kind of everybody in the space is a customer comes online they experience your cluster and your cluster has the same problem that like any cluster has or it's I mean a different problem every time but they experience one of the problems of HPC and then their experience is bad and you have to like negotiate a refund or some other thing like this.

52:12It's always case by case and like yeah a lot of people just eat the cost. Correct so one of the nice things about a market that we can do as we get bigger and have been doing as we can bigger is we can immediately give you something else and then also we can automatically refund you and you're still going to experience it like the hardware problems aren't going away until the underlying vendors fix things. But honestly, I don't think that's likely because you're always pushing the limits at HPC. This is the case of trying to build a supercomputer. But that's one of the nice things that we can do is we can switch you out for somebody else somewhere and then automatically refund you or prorate or whatever the correct move is.

52:46One of the things that you take in this conversation with me was like, you know a provider is good when they guarantee automatic refunds. Which doesn't happen, but... Yeah, that's in our contact with all the underlying cloud providers. You built it in already. Yeah, so we have a quite strict SLA that we pass on to you. The reason why I'm hedging on this is because we have some amount of active checks, we have some amount of passive checks. There are always new genres of bullshit. And the new genres of bullshit might cause a customer to have a bad experience, and the active or passive checks didn't catch it.

53:17And so then it's a manual process after that. then we have a literal thing in our website that you can just say, hey, some hardware problem, please tell us. And then we will go and resolve it for you. I mean, cards don't change generation to generation. What is a new genre of bullshit? If every component piece in the cluster has maybe a 1 in 100 chance of failing or maybe a 1 in 1 ,000 chance of failing or maybe a 1 in 10 ,000 chance of failing, you discover them. So there's ones that maybe nobody saw, maybe you didn't see, or maybe only matters for this one cluster with this motherboard in this particular data center or something.

53:55There's new interactions that otherwise don't happen. Most problems are really common and you can adapt to them. Like a GPU falls off a bus is one of the most common things that can happen. So it's not SF Compute's job to go fix those things. No, it totally is to some extent. Totally is to some extent. So we operate the cluster. So unlike a reseller, which is what we were doing before, in almost all cases, we have BMC access. So if on your laptop, there's like the button in the top right hand corner that you can hold down to like re-image the machine, there's a similar thing in like ServerX that you is like this other box that kind of plugs in.

54:28And it basically lets you reset the machine from outside. And it's like remote. It's a remote hand sort of thing. So we ask for this and we get this from a lot of our vendors, which means we have quite a lot of ability to solve problems for customers in a way that you might not actually get from a reseller. Oftentimes, we are the person who's debugging your cluster. For most customers that we work with, we have Slack channel. Our entire engineering team gets put in the Slack channel. If there's a problem at 2 a.m., we are the ones who are debugging your problem at 2 a.m. Not always the case because we don't physically run the hardware cluster or the data center itself.

55:01But most problems are solvable through this. So that's the auditing side. The other side is, I think of a standardization or whatever you call it. beyond auditing, the other part of the work is kind of standardizing the commodity contracts. Yeah, so there's two ways that we do that. One is that you set like a this or better list. So you set like a spec list and you say, oh, you're going to get like a common variability is the amount of storage on the cluster. And so you'll say like, oh, you're going to get X or better. And there's some guarantee minimum and sometimes you might get more. And then we're working on a persistent storage layer that might sort of abstract a lot of this way.

55:37But mostly it's that. And then there's a whitelist of motherboards and various genres of things. But the other part is we run the clusters from bare metal up. And so we make a thing that's this like, it's a UEFI shim. And if you're not familiar with what UEFI is, UEFI is like the sort of modern version of BIOS. Modern meaning it's been around for like forever. But, you know, BIOS is like really old. It's like this whole IBM thing. And you can write code that exists at the UEFI layer. And again, when you hear UEFI, you should think BIOS. and it does the same sort of thing as a Pixie boot, but in environments in which Pixie boot doesn't necessarily always work for us.

56:15So it basically sits at your BIOS, downloads an image, boots into an image that's custom for the user, and then on top of that image, we can throw Kubernetes on it, we can throw VMs on it, or whatever you want. And at some point, we'll probably do more stuff with that, but that's functionally what we can do. The nice thing, though, is that because you control from that layer, you can easily image an entire cluster, you can make it all the same, You can run your performance tests, it's all automated. So much nicer than what we used to do. Yeah. I mean, that is a very important work. I think for me, as a trader, I need standard contracts.

56:50And so there basically needs to be the safe of a GPU. Yes. What we functionally do is we have a market under the hood that is focused on the buyer and the seller, and it's optimized for them. And then beyond that, for a trader, you can standardize around a certain segment of it and you can trade on that contract. That's the goal that we're trying to get to. But you start by making something that works really well for buyers and really well for sellers. For those who are not familiar with derivatives markets, I can go ahead and say this because the point of being cash settled, which is something that you mentioned, which I think people might miss, is that you don't have to take physical delivery of the GPUs.

57:28So it's a pure financial instrument, which actually does mean that almost for certain, there will be more volume on SFC's marketplace than actually change hands in GPU terms. To be super clear, we are not a derivatives market. We may in the future work to create a cash-level future. We are not currently a derivatives market. We are an online spot market. I just think people, normies, get really upset when they learn things like, oh, derivatives on mortgages are 12 times larger than the mortgages themselves. Yes. Yeah, no, a common thing that people have talked to us about, or like a fear or concern I think people have, is like, oh, you're financializing compute and this will like, cause various problems of sorts.

58:12A subprime crisis. Yeah. And I think, so first, I think part of this is just because crypto caused a lot of people to think about finance in the like very degen way, for the right word. And then before that, the sort of 2008-2009 crisis caused people to think about it also in sort of like a degeny way. And this is very much not our mindset. The reason to create a derivative at all, or the reason to create a future at all, is a risk reduction thing. That's what futures do. The reason why a farmer wants a future is because they have no idea what the weather is going to do. And they don't want to be on the hook for, like, they have small margins.

58:48And if things go wrong, they really, really want to have a locked-in price. So that way they can continue to exist for the next year. Data centers are the same way. The way that they solve it today is you go out and you sign long-term contracts with your customers. What that does for you is it means your business is de-risked. You don't have to worry about the revenue for the next year. But that means that the customer now has to worry about what they're going to do with all this compute and if they don't optimally use it and so on and so on. And that just pushes everything onto the startups, who then in turn push it on to VCs.

59:20And so what the VCs are forced to do in order to invest in AI is they have to go and write big, giant valuations like pre-revenue at ridiculous multiples. So what you've done by not having a future is you've inflated the venture capital market. And that is a bubble that's totally going to pop. at some point. Like a lot of the companies are not going to work and the valuations are not going to work. And what's going to happen is a lot of these funds aren't going to return back to their LPs. And that affects the broader market. The way that you solve that, the way that you add security to the entire economic system in this chain is you add a future.

59:56That's how we did it in lots of other markets. It doesn't have to be this like, oh my gosh, we're going to like speculate on GB prices and like whatever. No, the whole point of SF Compute is to reduce the risk, reduce the technical risk, reduce the financial risk. Let's just chill out a little bit. There's so much other random shit. It's supercomputers, it's AGI, whatever. No, let's just chill the fuck out. I mean, also Dan is raising at a$30 billion valuation for Ilya. If everybody else in all of AI is pushing the hype and the extreme, everything we've been trying to do is go the other way. Whole website is just a fucking single page.

1:00:34like the entire brand is just like, what if we were like calm in nature? And then everything that we do as the product is just calm. What if we were the opposite force of the big hypey extreme thing? What if we just like chilled things out? And part of that was because we, in the beginning, were at the whim of the hypey nature. Like our entire origin is every 30 days and we don't sell out, we're going to go crazy and just completely bankrupt the company. And so everybody in the company is just like, what if we just chilled out? What if we stopped? This is the first time I've ever heard derivatives are the way to chill out.

1:01:10Yes! No, futures are the way to chill out. Futures are the way to chill out the entire industry. And we wouldn't be doing this if it wasn't that case. I like that. You have a very nice brand with a, you know, Claire Sky. We have to ask about the website. Yeah. What was the inspiration behind it? Why did you not go to black neon more cool thing and go the more nature yeah i don't think i really am a black neon sort of person i say that's wearing black pants and i thought i was wearing a black shirt but apparently i'm not so um the actual the actual thing was a lot of companies do this thing where they their website you go to there and it's like a magical experience and like everything is extreme and amazing incredible and you go to the product and it's like some sass app or something um and it's not actually that exciting.

1:01:57And that expectation of being like, really, really good. And then the fall off the drop of not being really, really good was something that from a product perspective, I never wanted to happen, especially because in the beginning, like our product was really bad. And so I don't want to set the expectation that it's going to be like an amazing experience. I want to set the expectation that it's going to be like a good price for short term bursts. And so what we did instead is we set the thing to be really low, you set your expectations really low and then you get a supercomputer um for like millions of dollars cheaper than you would have otherwise gotten your supercomputer and so you have the opposite expectation you have like really low expectations that uh are like mile are met higher and i think that's like the correct way to do things but also i think we were just like so sick of hype and excitement and um i just like really want to like not do that it's weird like by being anti-hype you have created hype like i would say like the the vibes are immaculate you know like you just you go to at the Caltrain, you just put up a banner that just says SF Compute.

1:02:59True. That banner was created about five minutes before we had to actually put something up, before the deadline was there. You opened up Microsoft Word and you did some serif. What is the font? Exactly. I don't know. Indeed. I think every time we try to do... The only caveat to this, the only caveat that we ever violate this rule with is when we're pitching San Francisco. I think San Francisco is amazing. So sometimes you will see these like advertisements from the city. Yeah, the city. So if there's a part of San Francisco computes brand, which are these beautiful like images of SF or various SF things.

1:03:34And I am the complete opposite about this. I am such a San Francisco promoter that anytime we talk about the city, I want to show the city from the like eyes that we have, which is mostly just gorgeous, beautiful area with nature. Like a lot of people think about San Francisco and they think about like the tech industry or they think, yeah, or the tenderloin or something like grind culture or something. And no, like I think about like the fog and just like the gorgeous view over the bridge and just the fact that there is this like massive amount of optimism in the city. And it's the backdrop of that optimism is the most beautiful countryside in all of the world.

1:04:13And so anytime we talk about SF, you will see like, or like we have a billboard somewhere that's just like local friendly supercomputer or whatever. and then the backdrop is like beautiful and amazing and that's because to some extent we're pitching the city and the people here and i think the people in the city here are actually really amazing and so you get to earn the brand um because the expectations are met whereas i think on our own product i'm typically want it to be better and so i set the brand a lot lower um and then the expectations are higher um and you still meet the expectations but you you set them a little lower i know are you the designer i know you have an artistic side um so i was in the beginning so I'm like a figurative artist, so I draw people.

1:04:49But we've worked with a design firm. Al4O was really excellent with us. And then nowadays, though, Jon Pham had a design firm. Yeah, Jon is unbelievably amazing. I think the amount of care and craft and attention to detail that he puts into just everything is so cool. If you go on our buy page right now, you go to sfcompute.com slash buy, there's an Easter egg there that you should find. I almost don't want to spoil it, but you should go find that Easter egg. If you just hover the mouse around the thing in the top right-hand corner, you'll find it. Tweet it, Evan, if you find it. And then the other person is Ethan Anderson, our COO, who has this RISD design background.

1:05:31And so he used to be sort of industrial designer-y. I'm probably going to say that wrong. He's probably not an actual industrial designer, but design background, same. So I think between me and John and Ethan, I think we... The source of the vibes. The source of the vibes. I had to ask. Yeah. Okay. So we're going to zoom out a little bit. One of the last things I wanted to ask you was actually, like, I remember, I think the first time I met you was in like kind of Cello and you were working on your email. Oh, yeah, yeah. And I have a favorite pet topic of mine. We were here with Dharmesh yesterday talking about someone, build an agent that reads my emails.

1:06:05Yeah. And you did. And I think I actually paid for the first one. You were so excited in the early GPT three days. I was like, you were like, I'm building the most expensive startup ever. It's so expensive. Anyway, so the point being, what I'm trying to get to is you are a very smart guy. You built email. You didn't like it. You pivoted away. I've seen other, like every year there's someone who is like, I will crack email and then they give up. What is so hard about email? I didn't pivot away because the product or the idea was bad. I pivoted away because I was super burnt out. I did a startup for like four years and the first thing didn't work out Is this Room Service?

1:06:41Yeah, this is Room Service So my startup before this originally started as Quirk which was like a mental health app but then Quirk had the same problems that basically every mental health app has which is like your retention goes to zero if you work it in any capacity and so switched and then said okay, well, I will do something that's closer to my actual background was like a distributed systems company called Room Service Room Service went for about nine months and then sort of had the same problem that I think every other competitor at Room Service has which is mostly people building a house.

1:07:07And so then I went back to our investors at the time, which was Nat and Daniel, and specifically Daniel, told me that I should go stare at the ocean. And, you know, I will find something else to do and just throw shit at the wall. And then I think it was Gustav at YC? Maybe, it was probably actually Dalton Caldwell. Dalton Caldwell, like, just said, don't die. Like, you can just keep doing things and don't die. And so I think I just got it in my head that you should just, like, keep trying things and not die. and I really, really, really did not want to die and didn't really know what to do. And so I just threw out like 40 products with the assumption that if you just keep trying things, you won't die.

1:07:44This is actually not the most ideal thing to do. You actually should totally just pick a thing and go with it. But my brain wasn't set on like, oh, I should do this particular thing. It was set on not die. And so I just kept going for a very long time, for like four years. And by the end of it, I think I was just super burnt out. and I was going to do the email thing with one co-founder and then they quit. And then I was going to do an email thing with another co-founder and then they fell in love and decided to go get married. Okay, so it wasn't that email is intractable. I'm just trying to figure out, look, is there something bad?

1:08:14Is this the graveyard of ideas, right? Everyone wants to do email and then nobody does because something. I think it's just hard to make an email client. I think it's hard to make an email client that is a competitive space in which there are lots of things. I do think that the better version of that is something that looks closer to what Intercom is doing. And Intercom obviously existed beforehand. So you can think about any product, should you be doing it? Or should somebody else in the industry who already has the existing customer set do it? And I think Intercom has pretty much very successfully done.

1:08:47They already had the position to do it. What do you actually need the AI to write your emails for? Most people don't need this. but who does need this is support use cases is pretty much there. And the people who are best able to execute on this is totally intercom. So props to Owen. I think that was completely the correct move. Closing thoughts, call to actions. You're hiring. Oh yeah, we are. We are hiring for two roles as of this recording. I don't know, maybe this will change and we'll be hiring for different roles. So go to the website or whatever. But the first role is for traditional systems engineering.

1:09:20This is like low-level systems or low-level Linux-y people. Rust. Yeah, so all Rust, most all of our code bases in Rust, but we're not necessarily just looking for Rust engineers. We're specifically looking for Linux-y people. The sort of pitch is you get to work on supercomputers. You get to work on one of the few places in supercomputers that I think has a pretty good business model and is a working thing. And people generally seem to think that our vibe at SF Compute is very nice. we have just an unbelievably excellent team I think nowadays our CTO is Eric Park he's the co-founder of Voltage Park which is one of the other GPU clouds and he is quite possibly the sweetest man I've ever met he is extremely chill and also just extremely earnest and kind and the rest of the team kind of feels that energy very strongly and then the other role that we're hiring for is financial systems engineering which I really should learn it's not systems engineering but we should really find a better name for this role.

1:10:20It's basically a fintech engineer. That it's, we have the same problems as traditional fintech does. And that's like, we have a ledger, we have recording requirements and all that stuff. This role is responsible for the not lose all the money goal. Like we've got a whole bunch of money flowing through us. There is a bunch of stuff that you need to do in order to not lose all that money. And then the actual outcome of that work, besides not just losing all the money, which is very important, is that you end up with better prices for the vendors and better prices for the buyers. And this means that your grad student who is making the cancer cure or whatever and needs to be able to buy like 100K of compute to like scale up really big, actually can do so.

1:10:57And that's, I think the, like this is part of the reason to work at SFC is that the things you do actually matter in a way that doesn't necessarily always at all the companies. Functionally, we run supercomputers, like not soybeans or, I don't know. It's a very cool place to work because your outcomes of what you do have real deal impact in a way that you don't always get when you're doing SaaS. Excellent pitch. I bet you've done that a lot, but it's nice to hear for the first time. I was going to say, have you looked into Tiger Beetle, the dual entry accounting database? We have. That seems to be the thing if you want to make systems that don't lose money.

1:11:35Yes. Systems that don't lose money, there are lots of other things you have to do. You have to make things in a format that your accountants can read and then get audited and so on. it's not purely just the yeah it's not purely just the tech awesome thank you so much of course thank you so much for having me