How Diamond Cooling Could Power the Future of AI, with Akash Systems

0:05Welcome to KnowPriors. Today I'm chatting with Felix Ajekom and Ty Mitchell, the founders of Akash Systems, which makes diamond-based cooling technology for computing platforms, from space satellites to AI data centers. Their innovation uses highly conductive diamond to help computers run cooler and faster while using less energy. Felix Sentai, welcome to KnowPriors. Good to be here. Thank you, Sarah. I think we should start with just a quick introduction as to what Akash is. Sure. Again, very good to be here with you, Sarah. At Cop Systems, we are a venture-backed company based in the Bay Area that is starting from the ground up at the material science level.

0:47And we are making, using proprietary materials, materials specifically of diamond that we grow in the lab, using it to make electronic systems that are disruptive in the world by an order of magnitude. It's in contrast. Oftentimes when we start companies, even in the hardware space, we tend to start injecting ourselves in the middle of a supply chain at a cosh as material scientists. We come in at the periodic table level and we start there to build up chips, boards, systems that ultimately change the lives of our society, whether you're in business or as a consumer. And we do that in several ways.

1:31We change the structure of a basic material. The systems that we've chosen to affect, we started off in the space world, where we make some of the fastest satellite radios ever made by humans. And then we go over, as we are doing now, to AI, where we are able to cause compute a GPU to go faster than has ever been done before since the beginning of this new space, or reducing energy consumption in the data center by a significant amount, all because of innovative material science that we've pioneered at the ground floor. So maybe that's a good segue into how you got started working on this, because you've had this idea for a long time.

2:19As you said, you started on space applications earlier. Can you talk a little bit about your background and the original scientific idea and how you thought it would be applied? Sure. So my background is in material science and electrical engineering. I obtained a Ph.D. in electrical engineering with a minor in material science and device physics from Cornell. and in my PhD, I focused on bringing together very dissimilar materials in such a way that one plus one equals 10. Okay. And, you know, for example, silicon, very well-known ubiquitous material that's ushered in the current modern era that we have today.

3:02But then there are other materials, plastics, other types of semiconductors that don't actually do as well as silicon, but they have their own strengths. And so for my PhD, I looked at ways of trying to bring together, say, the optics world with electronic silicon and merging them together such that the overall system is incredibly powerful. That philosophy I've brought to Akash when I started Akash with Ty in 2017 to try to do the same thing. I've often found personally, Actually, I think it's a very good metaphor for humans, how we interact together. When you bring different people that have different strengths, the combination can be incredibly powerful in ways that excel and exceed the simple summation of the parts.

3:55And that's exactly what we do at Akash, where we bring artificial diamonds, well known as the most thermally conductive material ever grown in nature or ever to occur in nature. And then silicon or even gallium nitrite and quite frankly, any other semiconductor, when brought together, amazing things happen. Very happy and excited about doing that in the world of AI. We did that in space where we have now made and launched the fastest radios ever made by man. And now with AI, we're able to achieve performance levels, whether energy efficiency levels or compute speeds ever obtained by simply using these artificial materials that we've created in the lab.

4:45Ty, are you the silicon or are you the diamond here? I'm actually a little bit of both. I'm the silicon carbide guy. My PhD was on silicon carbide. What that taught me when I went into the business world, working for a company, Cree Woolspeed, is that Cree Woolspeed developed very good silicon carbide materials level technology. and applying this technology to any sorts of systems like radar systems or a power electronic system for EVs or light emitting diodes. One of the things you learn is that when you have a materials level advancement, as the person who has that material level advancement, you really have to make the system to convince people that you have the solution.

5:28If you just go to someone who is making, let's say a car and you say, Hey, I've got this great silicon carbide diode, a shocky diode or a MOSFET, they'll say, all right, great. You know, I already using a, uh, a silicon IGBT. Um, if you meet their price, I'll put you in. And you say, look, if you, if you put my part in, I think I can increase your range 200 miles. I can increase it 40%. They'll say, okay, yeah, sure. However, if you make the car, okay, you find a partner to get your part into the car, uh, or you make the box, you actually make the MOSFET, um, you make the module, the power module, The farther you go in the system, the better chance you have of convincing the customer that you have the solution.

6:10So that's the approach that we took at Akash was even though we had this materials level technology, we would make the system and then go directly to the consumer or to the customer and be able to prove our technology out that way. And with AI, why did we go into AI? We were in space solving problems that were very difficult, actually more difficult than the AI problem we face today. Because in space, you have a limited area. You don't have any fluids that you can use to cool because there's no airflow in space. And you have a bunch of other reliability and survivability requirements that are much more difficult to meet than in AI.

6:55So we thought, all right, AI is a very difficult problem they have. Heat is a very difficult problem they have to solve. It's growing. And nobody really has the right approach to solve the problem. And we think we can help. So that's what caused us to dive in. And just to give a couple of numbers to what Ty just said, comparing space to AI, the power densities that we cool in space are at the level of 10 to the 3 watts per square centimeter, so 4 ,000 to 5 ,000 watts per square centimeter. The chips that we cool in on grounds, in AI, in a typical server, is a full order of magnitude less than that, a couple hundred watts per square centimeter.

7:39So that's what gave us the confidence that if we could address the problem in space, that we absolutely, using the same technology, even backed off a little bit so we can rapidly ramp. If applied to a server, it would be a home run. Yeah, I think you guys had a demo to just sort of explain the advantage in connectivity that diamonds have, or that specifically. We do. Thank you for making that very nice segue. What I'm going to show you is how diamonds can very effectively cool down or rather melt ice. So this is an ice cube that you're looking at here in my little video. This is diamond, a little piece of diamond.

8:23This is a diamond wafer that we grow in the lab. You can see the Akash name. And what I'm going to do is show you that heat from the ambient, and more specifically my fingers, my body temperature, will flow through this diamond rapidly into the ice cube and melt it as rapidly as I touch it. It'll be just like butter. I wish you could touch it yourself. It'll feel cold to the fingers. So I'm just going to wedge it here, and you will see. And there you have it going in. I'm feeling very cold right now. I don't know if you can see it, but there you go. Very cool. You can see it wedged in. You can cut ice.

9:05Yeah, you can cut ice with your fingers.

9:10And I'm going to rotate it so you can see. And that's the feature. That's the property that we bring to bear with our chips, with the GPU. We're looking at reducing temperatures. Initially, we're starting off with 10 degrees, which is already worth millions for any data center who has a small number of servers. But we're looking at further reductions 20, 30, 40 degrees down the line over the next 12 months. In space, we already reduced temperatures there 80 to 90 degrees. So it is quite significant, the effects of this, and the economic impact is far reaching. Yeah, maybe this would be a good time to actually just talk a little bit about why heat dissipation is a problem at all for AI chips in AI GPU servers and data centers, right?

10:01So, you know, if you imagine these chips in servers and racks of servers and big rows of them in the data center, like we have cooled large data centers for a long time with fans. Like how does this fit into the, I guess, alternative set of fans and liquid cooling? And like how how is large scale AI training changed the game at all? So this technology that we have, this materials level technology using synthetic diamond, it actually fits with any other cooling technology that's used today. Today, the cooling techniques that are used are really at the data center level where they do airflow containment and air containment, you know, keeping the air from mixing.

10:43It's at the rack level where you are using liquid cooling, you know, CDUs and manifold and pumping liquids right to the devices to keep them cool or using fans. And you have it at the chip and package level where people are using techniques to either speed up the chips, right? Make more transactions on the chip so that you get more efficiency transactions per watt. Or in packaging, doing things like fan out packaging where you're spreading out the heat as much as possible for any chip or group of chips. Also, things like HVM memory, you can stack up right in the same package with the chip and give it more efficiency and essentially also more transacted robots.

11:34So these are all techniques that are being used today. And the beauty of our approach is that it works with all of these. You can use diamond cooling by itself or you can match it with anything else that you're doing and give yourself additional operating margin, give yourself additional performance margin. that you can then use to drop your temperature, run your system hotter, and give yourself the opportunity to perform more transactions. And that's something that's critically important because you see just in the last year when NVIDIA has introduced a couple of new chips. First, the Samsung HVM, there were heating issues back then.

12:15Then with Blackwell, again, heating issues came up where that delay, that rollout was delayed due to due to heating issues. And this is now starting to for the first time. I think if you listen to the last conference call with NVIDIA, heating issues were mentioned. This is something that's going to increasingly be on the radar for companies, for investors. I think that Jensen was able to not directly answer the question. People are very skillful in these conference calls, but more and more of these questions are going to be asked and people are going to need approaches to address them. We have what we think is the most effective approach because it goes right to the heart of the device.

13:02Yeah, it's interesting because intuitively not being from the data center management space, like I get nervous when anything has a mechanical component, right? But I do have friends running large-scale data centers of this type. And I think even though there have been announcements of, for example, like the liquid-cooled GB200 and VL72 system, and some interest in adoption of liquid cooling, fans and liquid cooling come with their own reliability issues, of course. It's just complex to go implement that and keep that from, I don't know, leaking and breaking and things that movement requires. Yes, Sarah.

13:42Actually, the servers that we ship today, the H200s from NVIDIA, for example, is both liquid-cooled and diamond-cooled. So just to illustrate Tai's point that our diamond technology layers on top of whatever technology that you use, whether it's liquid, fan, or both. But to push on that point further, we believe that a cautiousness that if a material science and more specifically a physics or chemistry approach to solving the heat problem is not used today, that the needs of AI data centers around the world based on projections today will crash the grid as we know it. And if it doesn't crash the grid, we believe that the cost of electricity will be exorbitant.

14:30OK, it's not sustainable at the path that we're taking today. And that's part of our inspiration for attacking these problems, starting with physics and chemistry. So taking an example is your laptop. OK, your laptop has a whole bunch of chips inside of it. You put it on your lap and you feel the warmth of it. If you bring an ice pack and put your laptop on top of an ice pack, nothing will change. It will not speed up your CPU. You're not going to change the heat extraction of your CPU because there's just such a great distance, a thermal barrier between your lap and the GPU or the CPU. Going straight to the heart of the heat, the heat source, the chip, the material with physical chemical solutions allows you to make a difference.

15:17And that's what we're doing. And we don't see a lot of approaches like that out there. We think that this is going to be really key to curb the consumption and actually allow the AI vision, I think that we all hope, love to see, actually come to fruition. If I think about the complaints that people have running large data centers as blockers or issues to deal with, it is chip supply, GPU reliability, power supply, heat. Like how does heat relate to all of these other issues? Everything you mentioned is heat. I mean, we see the same thing, by the way, in space. Every problem that one addresses in space, it's almost like a whack-a-mole.

15:59Unless you go to the source of the problem, which is the heat producer, you're really just playing whack-a-mole. You knock it down here, it'll show up elsewhere. Everything you've just described is a heat problem. And by the way, this is right. There are billions of these chips. One simple server, the ones that we ship today, have eight GPUs in each single blade and then and then scores more of other chips equally heat producing. And so we at Akash are actually just scratching the surface of this problem. It's a pervasive problem up and down the supply chain. You just mentioned the way that it shows up in the world, the fact that we're trying to do more with what we have.

16:44All it's doing, it's breaking the bank. It's costing a tremendous amount of money. It's leading to reliability issues. Servers, oftentimes it's not uncommon for them to suddenly have an infant mortality. It shows up and it has problems right away. When it gets up to that 80, 90 degrees C temperature, it starts to curb back performance. Thermal throttling. This is something a lot of your listeners are going to be familiar with, the fact that the operating system has to back off the workloads on the GPU, which means slowing it down so that they can do the inferences that the customers are asking of it.

17:20So it is a significant problem. And I think that just attacking it at the network level or sort of putting Band-Aids at the system level will just kick the can or add cost to the overall ecosystem. You have to attack it at the material science, at the level of physics and chemistry. And Sarah, you used an interesting word with a blocker. And we are getting to the point where people are starting to get stuck with this issue. We mentioned the issues with the device rollouts and the delays that this has caused. You're going to see this happen more and more going forward, where the drive to increased performance, you know, whatever, two to three times every two, three, four years.

18:14And we're not going to be able to do that just because drawing power and water to the site and then getting the parts to run with this heat buildup and you are stuck inside the server. Anything you want to do inside the server, because right now you've got layers, you've got, you know, your chassis, which is probably aluminum. You've got like a copper heat sink. Then you have some sort of epoxy bonding material. Then you've got your chip material, which is silicon. Then you've got some sort of solder, gold tin. Then you've got FR4 or some other polymer board. Then you've got more gold bumps. Then you've got another board.

18:58So you've got this sandwich, and I'm just listing off some of the layers. And every time you have an interface between those sandwiches, that's a thermal barrier. Okay, so what are you going to do about that? because they're really, you have to attack that. And right now that's not really being attacked and this is going to have to be attacked. Otherwise, it's not going to block creation of data centers. But you look at the multiples in the market, you look at earnings, this is something that probably the people at AMD and NVIDIA are thinking of the results are growing now, right? But look what happened at Intel, okay?

19:39you miss two, three quarters and you go from being at the top of the mountain to, you know, listening to those bells tolling. And I think this is something that's going to be very, very, this is going to be top of mind for any executive at any of these companies. Yes, I understand. You got to cool the sandwich down from the inside as the sandwich gets bigger and hotter. I think one of the things that really resonates with me, at least hearing from friends in the industry, is the thermal throttling that you described, the fact that you see and have to manage erratic behavior when these GPUs are at higher temperatures.

20:16The vast majority of people working in machine learning right now, it's a very abstract software field, right? And so the idea that you have these challenged, non-deterministic behaviors based on how the materials themselves are interacting and you have to account for that. And there's a real like a, you know, burden of that is it's just a new domain to think about. Maybe just because it is your your area of expertise, how do you fit into the sort of partner ecosystem of like the NVIDIAs of the world, the super micros of the world, other SIs, et cetera? So just to be clear, so we are buying chips.

20:51We're not making GPUs. We're taking the hottest chips in the world and we're cooling them down so that you can We can open the envelope of performance for the system architect. Okay. And so we fit in, we're coming into the world as a server maker that is opening performance envelopes for folks in inference work, folks training models, data center operators, cloud service providers. Okay. That's our entry into the world. it makes sense that we would go to the most challenged parts of the market, the folks that are struggling most at that performance edge. So I'm going to go out on a limb here and say that we think that with our diamond technology, that we will be able to hyper-accelerate Moore's Law so that in two years, we will be achieving what previously folks had to wait six, seven years in the past to get to in terms of performance.

21:52Because remember, Moore's law is about squeezing transistors closer and closer and closer together. But you can only go so close before you have thermal crosstalk between these devices. And so right now, the limits we see in AI, the pace at which we can do inference work, is limited by that thermal crosstalk. If we open that thermal crosstalk and we're able to allow greater densities, then all of a sudden, you know, we can create a feature-length film in seconds rather than the timescales of, you know, if you're doing it offline, you know, months, years, if you're doing it right now with the thermal limitations that there are in AI, probably days.

22:34But, you know, I think we'd like to see seconds in production time to do a full 90-minute feature-length film. And that'll happen because of the unblocking of the thermal limitations inside the GPU. I'm going to ask a silly question, but you've mentioned it several times. You're growing diamonds. How does that process work for the form factor that you want? Assuming the vast majority of our audience has only ever heard of the concept of growing diamonds in the realm of jewelry. It's really no different than growing other semiconductor materials. if you're growing silicon or silicon carbide or gallium arsenide or indium phosphide, any of these electronic substrates, you start with a seed crystal and then you use a typically, you know, some sort of process, chemical vapor deposition to grow out from that crystal, you know, perfect single crystal material out from that crystal.

23:40And that's the same way, that's the same way you grow diamond. Diamond is just carbon, right? So you take a seed crystal of perfect carbon, of diamond, and you use a plasma to grow the diamond in a reactor. You know, it tastes very high temperatures, you know, very high pressures to do this, but it's essentially a similar process to growing silicon or silicon carbide wafers. and sarah um our specialty our secret sauce uh lies in how not only we grow the diamond but also how we intimately couple that diamond with the semiconductor using physics and chemistry okay so it's not a it's not a trivial process it does take some work you were asking about why does it take so long um it does take time to do material science uh and uh but that that That intimate coupling of the atoms of diamond with the atoms of a semiconductor is what we understand.

24:43And that's what we bring to bear in both space and in AI. And that's what makes this not so easy. But we're very excited about it. We're deploying it in the servers that we ship. And very strong market pull that we see right now, today, given what's going on in the world. You guys announced an exciting customer just this past week, NextGen Data Center and Cloud Technologies. Can you talk about why they're a good early customer? And they're also like a sovereign cloud player. So I want to talk about that as well. Sure. So NextGen is the largest sovereign cloud service provider in India. they handle the country's data in a very careful way, making sure that it stays within the sovereign borders of the country.

25:39We see that requirement coming from countries all over the world. Nobody wants their people's data leaving the boundaries of their country. We see that, by the way, in space. When data is coming down from a satellite or being pulled away, that satellite has an obligation to keep the data within the boundaries of that country. So that's an opportunity for us. It means that we're going to be able to address this issue with every country individually. So that's number one. Number two, they are the leader in that region, in India. And so we thought that that would be a very good test case to show the world what is possible, the fact that we as a small growing company can scale to the kinds of volumes that they need.

26:29And we can scale rapidly. We've got to ship all of this stuff within the next quarter. And a lot of small companies trip at that. NextGen selected us, believing that we have that ability to scale. Thirdly is the fundamentals of the technology. I think they saw very quickly, and they're led by some very innovative leaders, that this is a problem that will stay with us for a very long time unless we get to the very heart of it, the material science nature of a solution. You're just going to be tippetoying around the big elephant in the room. And so we were very excited when they saw that opportunity, the fact that, okay, this is a company that's coming at this problem from the material science.

27:16We jumped at it when they saw that we could scale. We were very excited about that. We jumped at that. And then, you know, I think NextGen is positioning themselves and using our technology to not only scale within India, but potentially scaling around the world. Okay. Again, respecting the sovereignty of country data within that country. So we think that this is a very, very nice match. They opened with this size order. We're excited about the things that are even coming down the pike with them in just 2025. This is just the beginning. And these problems are also faced by U.S. companies as well.

27:57You know, India is not the only country that's dealing with this. And we're talking to them as well. This is definitely a very important topic that you brought up, Sarah. Yeah. What do you think is the importance of American manufacturing of AI chips and data center capacity? This is especially relevant given you're one of the only small companies that is a CHIPS Act recipient. And, you know, Gelsinger just stepped down. What is your current view of American capacity and your outlook for it? My view is that the U.S. is not doing enough and needs to do a lot more. This is a technology that the U.S.

28:37needs to be the leader in. And it needs to lead all the way up and down the value chain. We can't just rely on what NVIDIA or AMD has done to date. We have to continue to invest in not only the larger companies, but also the smaller companies like us, like others, who are working on some of the very critical problems. Because, as you know, AI is not only a critical technology for business, it's also a critical technology for national security. and these are things that the U.S. cannot rely on other countries to develop for it. And so we have to really drive technology development. We have to drive manufacturing all the way up and down the supply chain and put a lot of investment into this technology.

29:30It's going to be very important for the future of this country. And we're there to support that and that's what we're focused on. Let me add to that by saying that our receiving the CHIPS Act is a testament to our support of USA, USA, USA. We're all about doing the things that Ty mentioned, strengthening our supply chain. That's one of the key tenets of the CHIPS Act. Supporting national security we supply to defense. It's public that we work with Raytheon, iconic American defense company. This technology allows Raytheon and U.S. defense to maintain defense military supremacy around the world in a way that has never been done ever in the history of mankind.