Uncapped #57 | Andrew Feldman from Cerebras

30:26You go to Cleveland, and then there's a decision whether it's best to take the next hop. And that's sort of the way we worked, where we knew there was a pot of gold out there. but the path to it we knew was unknowable. And so each time you get over a new mountain, you look around and you re-decide. So it's this sort of hop-to-hop thinking. So can we build it? Yes. Can we make it work in routing? Yes. Is routing now where everybody's focusing? That's where they're focusing. Can we be a fast router? Yes. But now we're seeing the lay of the land. We're seeing the sort of unfolding of AI, the rapid growth of intelligence.

31:12Well, who's going to use it? We use AI through inference. So we're in a position. We're engaged in conversations. We had product being used where it allowed us to see something new. And that new thing was, holy cow, the trajectory of AI will make it smart enough that everybody will want to use it. If everybody wants to use it, inference is going to crush the compute infrastructure. We better be there. And so each time we achieved something, it sort of moved us up a mountain or a hill, gave us a new view of the landscape. We could make some new decisions. And we were always sort of moving in the same direction, but we didn't know how to get there.

31:54And so decision, hop, you get to a point of view, think carefully, earn the next viewpoint. Yeah. Earn the next. Well, it's interesting because with software, you obviously can both think nimbly and change everything the next day. Right. It's hard where you can't do it like that. That's right. Ours, we have bigger discrete decisions, right? And that's why the choice in your chip of specialization versus flexibility is so important. We made a couple really good decisions in our first architecture. where we decided not to sort of embed technology that would accelerate convolutional networks. Instead, we said, we don't know how long those will last if we work underneath that and accelerate the algebra that underpins all AI we knew about.

32:56That was a really good decision because when transformers came out, we were the fastest at those two, even though we'd never seen them and never heard of them and they hadn't been invented when we set the architecture. And so it helps to make a few good decisions. Yeah. It's interesting, like, on the backs of, you know, companies like SpaceX, Andro, Cerebris, you know, Palantir, maybe others. But obviously, like, hardware is now hotter than hot. And everyone wants to fund hardware. But it does, just talking this through it, it's crazy hard. It's really hard. Yeah. It's hard. And it requires enormous internal fortitude.

33:35and it requires success has historically been predicted by some experience in the field. Yeah, different than like AI where you see the advantage to it. I think in AI and social networking, a lot of the founders and the leaders were building tools for themselves and their classmates and their friends. And there they had unique insight. I mean, if you look at cognition, if you look at cursor, are some of the best engineers, software engineers in the world. They're building tools for themselves, right? That's true. That's different from... I also wonder if this last topic probably plays a big role in it, which is when you can change your opinion the next day and it's fine that you were wrong yesterday, speed and decision-making speed trumps versus if you do something and then you have to live with that for a year.

34:27Years. Years. years you gotta you you can't just be wrong no we fix it tomorrow i think in in chips we we we measure three times and cut once right we it is really unforgiving yeah to big mistakes and so you have to be very very sure that that you get the big items right yeah one of the things that i think is interesting about how you built the company which is you know you and you mentioned a lot of the early team had have done multiple companies together like a lot of you guys had had worked together for a long time um you're older and and more experienced i think you're not that old but somebody called me a boomer yeah i'm just like my dad's a boomer right boomer you know at some point you're looking up and that's just it's all the same when you're that low you're looking up He's really old, like 27.

35:24But you know what's interesting actually is if I look at the product leaders in your organization, if I look at some of the go-to-market leadership, they're actually very young. They are. And I'm just interested in like how – was there intentionality around that? Not necessarily ageist, obviously, but just like how you combined the perspective of, you know, young at the cutting edge product leaders with seasoned hardware engineers. I think we tried to think really hard about where experience mattered, where blistering intelligence matters. You know, if you look at our product organization, unbelievably smart.

36:08I mean, some of the best product people I've ever seen. all young, all promoted from within. We don't have big company rules. You've got to be in a job for this amount of time before you can get promoted. If you're extraordinary, we're going to give you more and more responsibility and more and more, and if you do a great job with it, there's no end to where we will take you. My co-founder, Sean, one of the five of us, in my last company, we hired him as an individual contributor and when we were acquired four years later, the guy was 25 or 26 when we hired him in 2007, right, when we were acquired by AMD, we made him a corporate fellow, right?

36:54There was just no end. I mean, and now, he's a founder and he's CTO of a public company. And we do that sort of ruthlessly. And there are some areas where to be exceptional requires a tremendous amount of experience. There are some areas where it doesn't require any experience. We don't have a long history of understanding what customers want in AI. So their methodology, smarts, insight, trump experience. In other areas, in the making of chips, it's been my experience that some odd years of previously building chips is the best predictor of whether you're going to deliver exceptional chips. And that's also from the mechanical side on the system side.

37:44There's just not a lot of chance in college or in graduate school to actually build silicon, to actually build a machine. They're so expensive. They're so hard to build. They take so long that even in a doctoral program, you don't get a chance to tape out a chip, to deliver it, to bring it up, to put it on a board, to power it, to write the software for it. And so it takes some time. Are the relationships something that are critical, or can those be earned quickly by young people? I think the following. First, five is too many founders, without question, except that we'd worked together before.

38:19And so everybody knew what the other folks were good at. And so there wasn't a lot of head-butting at all. I mean, we'd all worked together previously. We all had tremendous respect for what the others could do. and some humility about what we couldn't do. And so we were able to do that. Your specific question is, does it take a lot of time to build trust? I think within the six or eight weeks of working with someone, you can tell if they're extraordinary. Extraordinary people, in the first email they send you, you go, whoa, that's exactly what I needed. Every list is in descending order of importance.

38:58There's not a lot of fluff. There's high signal. So there's, you go, whoa. And then you see that again. And then you watch the way they run a meeting. And you go, whoa. And then you watch them deliver something. They motivate a bunch of people around them. They can work across the organization. You say, well, that's a person I need on the next important project. And then they just, they chew through work. I always thought with recruiting, it's like, if I came away learning real things and I wanted to have another interview or meeting just because I was like, I'm going to learn more stuff, I was like, that's my best.

39:28The best. Yeah. The best. It's so good. The best. What about the external relationships? You've got to work with a lot of people outside your company. You've got to be in Taiwan. You've got to work with... It helps to bring those with you a little bit, some of them. We've been building chips with TSMC for decades. We've been working with our contract manufacturers for decades. Especially in a time of contention, that those relationships had been in place for years, that you'd been good to your word, not once, not twice, not just in good times, but you'd been good to your word over good and bad times.

40:06That was really, really important. I think you earn relationships with new partners in exactly the same way. You're good to your word. You get them information early. You write. I mean, it's, you know, write a thank you note. No, really. I mean, do what your mother said, right? Be a good person. Write a thank you note. Do what you say you're going to do. This, to me, was one of the big learnings, particularly during the COVID era and everything. If I think of the software companies, even software infrastructure companies we work on, it's really like their vendor that matter is AWS. And maybe now their vendor that matters is AWS and a foundational model company or something like that.

40:55There's like two vendors that matter. this is like i don't know what the vendor list is but it it's it's insane it's dozens right it's it's it is the all of these components that go into the system all of these specialty manufacturers that build the cooling plate and part of the water system and like and so you have all of these things that have to kind of come together and and we just aren't used to that like that those meant that many dependencies in order to deliver our product or at least yeah i wasn't That was like a big thing, and you have to kind of work with them on. Software guy coming to grips with a supply chain.

41:35They call it a chain for a reason. And it really is. And you're like, wait a minute. Holy cow. This is all of this stuff. And when you're growing exponentially, like then everything becomes even more complicated, right? Because it's like you're putting in. I had never thought about this. You know, it's like AWS. We want more capacity. You go online and you add capacity. You want more capacity in terms of number of wafers? That's a 15-month lead time kind of decision and a huge amount of capital. And your vendor is also making a huge allocation decision. And so they have to buy into it too. And so that was a big learning for me in just terms of how much complexity there is in that.

42:23One of the things we've been talking about a lot internally is that it feels like no matter how big you think it is, it is hard to wrap your head around the size of the AI build-out that's happening and the CapEx going in. I actually saw a chart this morning that was like 1 % of GDP was spent per year on highway and telecom. And then it was 2 % for railroads. And then this AI build-out is like 3.5 % of GDP. it's like only one person got it right which was your brother no that's true yeah i mean the only person and everybody thought he was out of his mind i mean what what what what sam's really good at and i think is so hard is he saw an exponential and wasn't afraid right you take that exponential out two or three or four or five years you go holy crap right he wasn't afraid everybody else was afraid.

43:18Now it's going to slow down. He was like, no. I remember the Stargate, like$7 trillion or whatever. Right. I mean, now the initial Stargate is sadly small. Yeah. Right? And it was mind-boggling. And people were laughing. They were laughing. And so all of us got it wrong except him. I think we got it wrong. I think TSMC got it wrong. I think NVIDIA got it wrong. Everybody, the memory guys got it wrong. We all got it wrong except Sam. Yeah. So you've got this insane build out going. And the demand for inference seems like it's going even faster than no matter what we're going to be able to build.

43:56And so then there's a lot of questions about what will that mean for the price of compute and what will it mean in terms of how much will be available and token spend and all of this. but I'm curious, day-to-day, what you're seeing on the data center side and your experience now, what is that like for you? The data center is one of the links in the supply chain to deliver compute by the cloud. And the truth is, it's whether you deliver it by a cloud or you deliver it what we call on-prem, it impacts both cases. In one case, you rent the capacity and you put your equipment in it. In the other case, your customer rents the data center.

44:37And so it is a limitation in the market right now. It exposes a whole bunch of weaknesses in the U.S. Our grid is pathetic and sort of built on 1940s or 50s technology. We stopped doing work in very interesting technologies that turn out to be extremely clean like nuclear. I mean, wouldn't it be ironic if it took AI to bring us back to doing nuclear? Right. I mean, that's right. It showed that in these data centers, we use generators as backups. And these are either diesel or LNG or one form of gas, right? And these come from GE Vanova or they come from Caterpillar. There'd been no innovation in generator and diesel gensets for decades.

45:33Now suddenly there's innovation. The guys at Boom want to use what they designed for jets to power data centers. We're seeing all sorts of interesting things in battery backup. Like from Bloom Energy, you're seeing interesting fuel cell. There's just this enormous innovation because there's necessity driving it. We don't have enough data centers. They're coming on too slowly. The grid is old. And so it's an area of tremendous innovation. I think the industry did itself no favors by doing some dumb stuff at the beginning, tried to pawn off some costs on local communities, and tried to take advantage of local municipalities.

46:19There is no reason a data center shouldn't pay its way. There's no reason why it should use very much water. We use closed loop systems, right? All the data centers in the US use less than the California almond growers. Not by 1x or 2x or 4x, but between four and seven times the almond growers use more. So, I mean - The water thing is just not a thing. The water thing is not just a thing. But as a community, we didn't do a good job of communicating with local communities, getting their buy-off, showing them that we're going to bring thousands, and in some cases, 10 ,000s of high-paying construction jobs.

46:52We're going to pay ongoing jobs, and their tax base ought to go down over time. And we didn't do a good job of that, and now we're paying the price. I mean, it kind of goes with the whole theme of tech doing a terrible job communicating about AI in general. Horrible job. We're doing a horrible job. Yeah. I think the data center thing, the other part of the data center thing that's interesting to me is like if you if i asked you i don't know 2018 2020 whatever what's the probability that data centers were going to be a critical factor get another thing i'd have gotten wrong right yeah we'd all got it wrong we got it wrong and it just and that was just like a it's a new thing it's like okay now you have to build it all the way through and deliver it but how long does it take to go from shovel i know you joked about shovel i actually want to ask you about that but how long does it take to go from start to finish on a data center shovel ready is an expression you hear in the data center world, it means we haven't done shit.

47:45But we're ready. We want to sell you a pile of dirt ready for your shovels to start doing something. It does sound better. It does sound better. I've got raw land or I've got dirt in Oklahoma. Doesn't it mean you have permits and other things that are important? Sometimes, sometimes not.

48:06We would never have expected to be where we are today. If you have a good builder, it takes probably from raw land and permits to stand up 50 megawatts, which is a reasonable block. And often even the big sites unfold in 50 megawatt blocks. 50 megawatt blocks is 18 months. Now, if there's an existing building there and there's grid power already, we call those brownfield sites. So a lot of what's going on right now is people are going to the Rust Belt and they're buying old factories, paper mills, because they had a lot of power, right? They had the permits for power already and retrofitting them.

48:50So how much of their kind of supply chain bottleneck is energy versus chips versus construction versus permitting versus whatever else? Each chunk of the supply chain has its own supply chain. So data centers need, there's plenty of concrete. In most places, there's labor. Although in Wyoming, where there was a battle for all these data center sites, they were bringing in electricians from as far away as Denver.

49:21Generators and electrical transmission switches are long lead time items right now. And those are hard to come by. And so those are the long poles usually. Usually you can get up what's called a cold shell. You can build a concrete tilt-up building or a metal building fairly quickly. But then you have to fit it out and turn it into a high-powered hotel for compute. Yeah. Right? And that takes electricians, takes cooling, chillers, all these other things. Those all are long lead time right now. I was really, I happened to fly over leaving Memphis last night. I happened to fly over macro hard and macro harder.

50:06It was astonishing. Like the size. Nobody can build like you are. It's unbelievable. The size of buildings and construction nearby and everything. I was like, whoa. That is tremendous. Which leads to an interesting question now, which is like, I could imagine a world where either it'll just never be enough data centers, and for 10 more years, it's always going to not be enough. or because of the dynamics of the lead time and what's going on right now, you could imagine like an overbuilt and betting on which of those worlds you think is going to come out is there's probably some alpha there for whoever calls it.

50:41We won't have enough data centers. We won't certainly, I mean, seeing out 10 years in our space right now is really hard, right? 10 years ago, transformers weren't running. People were running tiny little models and trying to identify faces or cats and chairs or whatever. I mean, 2016 was not a big AI year, right? But five years, we will still be chasing data centers, and we'll still be chasing chips. And for those who use HBM, they'll still be chasing HBM. Yeah. It'll be interesting also how politics plays in all this, because now you've got probably more government interest and involvement in tech and AI specifically than we've ever had, obviously.

51:24What we need is more people who don't understand making decisions. that's clearly what we need i mean i think more people with absolutely no clue um making making important decisions you know i i think this is an area where uh they're really thoughtful discussions to be had about what's good for the u.s yeah um and good for different municipalities different rural areas um and those aren't being had by the politicians yeah right they're knee jerk their sort of... Let me ask you a question on this. Don't move away from politics for a second. Let's say we do agree on this compute thing. The compute is constrained in some form or another, whether it's memory or chips or data center space, whatever.

52:08There's some constraint there. And the demand is overwhelming. Doesn't that argue then the only thing that matters is tokens per watt? Basically, we're going to have a limited number of denominator, and so however much, whether it's tokens per watt or tokens per facility or tokens per chip or something like that, but it feels like we need the most of the numerator for the limited denominator. Or task per watt, maybe. And so if that's the case, then Cerebris has a speed advantage, a huge speed advantage. But how do you think about that dynamic if we fast forward in terms of what the constraint actually is?

52:51I think there are a couple things. First, speed, which is what we've chosen to focus on, has historically created markets. If you think about it, there's no market for slow search. There's no market for dial-up. Your kids are, what, 14 and 12? If you want to punish them, don't take away their phone. Ratchet it back to dial-up speed. right this is a real punishment let him use it for a week at dialogue speed slowly right we laugh and then we say it's okay for AI to be slow think about it when the internet was slow Netflix delivered DVDs and envelopes and when the internet got fast they became a movie studio that's not they didn't get better at their other thing they became something entirely new And I think what we're seeing with the launch of GPT-56 Sol, right, it's put out in limited availability last week, that people are thinking of whole new applications.

54:01You've got frontier intelligence instantly. And that opens up all sorts of new opportunities. So that's one thing you do with speed. The other thing you do is you try and think about how you can drive up throughput, how you can make more tokens. And one of the ways we're doing that is by partnering in something called disaggregation. And we're doing it with AMD, we're doing it with AWS, where you think about in the work of inference, is there a part that can be done by somebody else? And is there a part that can be done by you such that the result is higher throughput? And with AMD, we're seeing 5x additional throughput, I mean, five times as much throughput, while keeping the speed the same.

54:47And we're seeing similar numbers with AWS. We have an opportunity, because of our architecture, to do that with the entire GPU landscape. We could do it across the board. And so there are four major chip makers right now in our category. Obviously, NVIDIA. We'd love to partner with them. There's AMD. There's the Google TPU. And there's AWS with their training parts. And we're already working with two. So that's something we've thought a great deal about. And as a vector, we are extremely interested in chasing down. What has it been about NVIDIA that has made them have the just crazy run that they've had?

55:27I think most people are wrong about what makes NVIDIA great. First, NVIDIA is, you know, in the first quarter of the century, they're the great company without any question. And I think people look to a bunch of things. They look to CUDA. I don't think it's CUDA. They look to their chip architecture. I don't think it's their chip architecture.

55:55it's this unbelievable grit and intensity that was born of a decade of not having success as a public company I think if you look at their stock chart between about 2000 to what 2003 and 2013 2004 2014 for a decade right they traded horribly and you're a public company and you're fighting tooth and nail and no one's listening to you and you can't sell very much. And the relentlessness and the grit that that takes is awesome. And to come out of that as the most valuable company a decade later, right, the most valuable company in the world, that sort of intensity and grit for a company of their size, in my view, makes them one of the great companies in history.

56:40It's not these other things. These are just things people say. But that sort of a decade of being a public company and fighting and fighting and fighting, that gets in your DNA. And that's awesome. Yeah. I mean, you see Jensen still at events. Fighting tooth and nail. Yeah. Right? I mean, could you have imagined - Five trillion dollar companies. Could you imagine old tech leaders before Jensen doing that? No. Right. Right. that kind of fight? I mean, that in my view is what's awesome. Yeah. That level of fight. I think the other stuff, cool, good, but when I look at what I can do better, when I look at sort of what I ought to be thinking about as being a CEO, those are the things I look to.

57:29Do you think, did your near-death experiences give your company some of that DNA? You know, we are, and I think this is what's interesting about Jensen too, is he sees himself still as the underdog. Now he's the big dog. But this is my fifth startup. We sold three and took one public previously, and we've now taken this one public. I'm a professional David in the battle with Goliath, and I wake up every day with that mentality. And that we're now bigger. We have sort of bigger competitors, right? We have bigger challenges. we have more people throwing stones at us and so I hope we take that I personally wake up every day with that passion and that drive every day I think to myself when we started they said it would never work you can't do way for scale, we made way for scale and then they said alright you did way for scale but you can't yield it in volume and then we yielded it in volume they said ok you can yield it in volume but you can't package it in volume production and then we packaged it in volume production They said, okay, now you've packaged it in volume production.

58:40You only have a government customer. And we said, okay. Then we won a sovereign cloud. And then they said, you don't have a frontier lab. And then we won OpenAI. And then they said, okay, you've got government. You've got sovereign clouds. You've got a frontier lab. You don't have a hyperscaler. Then we won a hyperscaler, right? Each time, then they said, you've got all these cool customers, but you couldn't do big models. Now we're serving GPT-5. What are they saying? What's that now? The CEO's a boomer.

59:17That's good. But each time... I became a millennial and I showed them.

59:25Right? And so I think when you're sort of a professional, David, when you are an entrepreneur at heart, each one of those fires you up we're only interested in solving problems that other people can't solve we're only interested in doing things that other people can't do that's why we get up every morning it's not the money it's not notoriety it's because we love building cool things and we really like building cool things that are so hard that other people can't build them Eric told me you had a pretty cool and unique childhood I don't know if that fed into this at all but how did you grow up?

1:00:01I grew up on the Stanford campus. My parents were faculty. And there's a little neighborhood where all your neighbors are other professors. My neighbor was William Shockley. That's crazy. Transistor. So crazy. So all we knew about him, we're 10 or 8, is that his wife gave out full-size candy bars at Halloween. The inventory was at Bell Labs. he invented he brought silicon valley right he but the movement of shockley to the west coast created the foundation for silicon valley and we're thinking he gives big three musketeer bars right um but i i think there were a couple things that were glorious first the only currency was intellectual horsepower right that that was nobody cared if you're rich nobody cared if you'd started a company that this was the 70s.

1:00:58What they cared about was, oh, that dude's really smart, he does good work. My other neighbor was Amos Tversky. And for his work with David Kahneman, they got a Nobel Prize in economics. My dad's tennis match. There were six or eight guys in rotation. They played doubles on Saturday and Sunday. Somewhere in my mid-20s, I realized three had Nobel Prizes and one had a Fields Medal. Right? He sucks at doubles. That's right. No, no. Let me tell you, it was some old man tennis. I mean, their serves were grim. Their physics was good. It's a crazy way to grow up. It's a crazy way to grow up. And, you know, the neighborhood was safe.

1:01:40We'd get on our bikes and we'd just go all summer and come back when it was dark. And so did you think you'd be an academic? I did. Actually, I was working on a PhD. I got a little bored. I went to the business school at Stanford while I completed my qualifying exams for my PhD. And I sort of got sucked into this by mistake. And my dad still asks me, he's like, you're going to finish your PhD, Andrew. I'm like, dad, all my professors are dead. Nobody left. Yeah, amazing. All right, well, Andrew, this was a blast. Thanks a ton for doing this with us. And obviously, you've built something extremely special.

1:02:18and it's been cool to just watch and learn vicariously through Eric. So thanks for everything. It's a pleasure to be here and chat with you guys. Benchmark was an extraordinary partner. I mean, I think if you do hardware, you're going to be in bed with your backers for a decade and pick good ones. And I'm proud we did. Thank you.