Redefining Chip Architecture with Arm CEO Rene Haas

0:00There's no computing problem that's ever been invented that doesn't utilize and can't utilize the microprocessor. It is the heart of everything. All roads lead through it, around it, past it. Something has to do the orchestration, arbitration, decision around where those tokens go. That's what CPUs do. Chip design can take anywhere from 24 to 36 months depending on the complexity of the chip, etc. The actual design is not the largest amount of time. The largest amount of time is in the verification, the validation, the debug. AI is really good at that. And if we were to shut it off, it's like being in the 1990s, you've got internet and you're now saying, you know, only internet between the hours of two and four.

0:36After that, go to the library that we have down the hall. It'd be anarchy. The genie's out of the bottle and there's no stopping that.

0:48Hi, listeners. Welcome back to No Priors. Today, Alad and I are here with Renee Haas, the CEO of Arm and SoftBank Group International. We talk about the position of Arm within the chip industry, the resurgence of interest in chip innovation, the challenges of the supply chain, the future of robotics, energy, his place in the SoftBank group, and how he sees workloads changing in the future and for Arm. Rene, thanks so much for doing this with us. Pleasure. Congratulations on the chip presentation, Hot Chips, and all of the progress that Arm has made. I think there's an enormous amount of interest from the technology industry and the software industry in sort of better understanding the chip supply chain recently.

1:30For anybody who's not super familiar, can you explain Arm's position in it? And then we'll get into sort of more recent topics. So we have two positions in the chip supply chain. Our primary business is licensing IP, the CPU core that finds its way into smartphones, data centers, automobiles, you name it. But our customers are the ones who either build the chips themselves, a Samsung who's got their own fab, or the vast majority of companies that take their chip designs and go to TSMC and get them taped out. So in that world, and this is the cool thing about Arm, because we're so broad in terms of the markets that we serve, we kind of see everything.

2:09We have a very good sense of what's going on in automotive, data center, smartphones. So we see the supply chain situation from all angles. We also introduced our first product last March, the one you just mentioned at Hot Chips, the Arm AGI CPU. So now we're in that soup ourselves from the standpoint of we're also having to figure out how to buy substrates and buy wafers and buy memory, et cetera, et cetera. So we're up to our waste in everything on the supply. Why do you make the move now? So for, you know, Arm, I believe, existed for a few decades now. Now, the focus was always on IP, which is effectively like designing the way that different chip components are put together.

2:47And then you license that out to other people to actually manufacture and incorporate into their designs. Why did you decide to start making some of your own CPUs? Yeah, it was an evolution from the early days of where we just supplied simply the IP components, the pieces, the CPU IP, the GPU IP, the system IP, et cetera, et cetera. A few years ago, what we were starting to see was that product cycle times aren't slowing down. Chip manufacturing times are extending. The ability to get solutions out faster was becoming more and more important. So we moved from these individual components into what we called compute subsystems.

3:24I used, when we did the roadshow a few years ago, I used the Lego analogy where essentially we're providing the blueprint on here's how you stitch it all together. Demand for that was insane. And what we were finding was we were, and we initially, people thought, well, people aren't going to want these subsystems because that's what a chip designer does. Why are you providing that piece? But it saved time to market. And it saved a whole lot of things in terms of cost, speed, et cetera, et cetera. The physical product was sort of the next, the next leap, if you will. And there are certain sets of customers that will license IP to, and they've got all the capability in the world to build chips based on ARM.

4:02There's a lot of companies who want to have product based on ARM. Not all of our customers build products that serve those markets. So Meta was that first example. They wanted a general purpose agentic CPU. There wasn't anybody out who could give it to them. They came to us and said, hey, why don't we do this together? And that's how we got into it. How has that landed with the rest of your customer base? So one of the things that we were very careful about was getting, making sure the ecosystem was on board with this. because we do CPU IP, which is really only as good as the ecosystem, the ecosystem of chip people and the ecosystem of software folks and people who build around that.

4:42So we talked to just about everybody who were customers and said, how do you feel about this as the direction we're going? And surprisingly, we got a lot less pushback than I thought. And the reason for that was the more software that's available in the wild, whether it's proprietary and or open source, benefits the broader ecosystem and the customers themselves. So whether it was NVIDIA, Amazon, Microsoft, Google, all people who build ARM-based server chips, they were all on board. And I think the ultimate proof point was when we announced the product last March, we had Jensen, we had Ronnie Boker, we had Amin, we had James Hamilton, you know, all the folks from those customers I mentioned, all saying, congratulations, it's a great thing.

5:24So it's been okay. What is the, you know, where are you in the learning cycle as a business now selling physical chips? That feels like a lot of new capabilities. Yeah, so we obviously to deliver a product and we're a fabulous semi-company, right? We don't have a fab and we have no intention to build a fab, but we fit in that ecosystem. But that means you need supply chain operations people. You need to work with, as I said, the TSMCs and Samsung's of the world. You need to work with the Samsungs and the Microns, the SK Hynix to get memory allocation. And then on the engineering side, you need a lot more different capabilities.

6:02You need back-end people, layout people, implementation people, bring up labs, physical stuff, right? We didn't have a lot of physical stuff, which was kind of the beauty of the business, the original business. I remember discovering that ARM had a 98.5 % gross margin. Yeah, kind of beautiful. I don't think I've seen that otherwise. I came from NVIDIA before I came over here. Most of my career was in the chip world. And I remember coming to ARM in 2013 and thinking, no inventory, no RMA, no scrap. What's not to like? So we had to add a lot of those capabilities. We have a lot of people on the leadership team who've come from that world.

6:43I've got execs from Broadcom, Qualcomm, NVIDIA. I work for NVIDIA. And so we have the leadership that's done this before in other companies. So we've been able to build up that muscle pretty quick. How have you approached AI adoption? So, you know, we were speaking earlier that there's news from Open Editor today about Jalapeno, any chip that they designed. Their claim is it was a very fast time to market. And part of that was using AI tooling to sort of design chips faster. How much adoption have you seen there? I know other companies have also talked about things like adopting formal verification at Amazon or other places where they're trading chips.

7:15So I think the chip world is starting to evolve in terms of AI usage. And I'm just curious about how you've done that at all. Personally, I'm a huge believer in AI as a utility that's going to help productivity for every single industry. It is going to be the great leveler in terms of companies that get started super quickly. And for industries, whether it's healthcare, infrastructure, robotics, every industry is going to use artificial intelligence as a utility and stop. So since I'm such a believer in this, of course we use it very heavily inside of ARM. On the non-engineering side, we're using it all over the place.

7:55But on the engineering side, we've seen huge, huge benefit. You mentioned verification. Chip design can take anywhere from 24 to 36 months, depending on the complexity of the chip, et cetera, et cetera. The actual design of the architecture, the RTL generation, if you will, the mapping of the architecture, is not the largest amount of time. The largest amount of time is in the verification, the validation, the debug, the documentation, et cetera, et cetera. AI is really good at that. And I would say we probably have 80 to 90 % of engineers today in SiteArm who use it on a daily basis. And if we were to shut it off, my analogy I give to people, it's like being in the 1990s, you've got internet and you're now saying, you know, only internet between the hours of two and four.

8:42After that, go to the library that we have down the hall that's got all the books that you can go up and look all this information up. People are to be anarchy. So the genie's out of the bottle, right? And there's no stopping that. Now, there's certain things that the tools are still not that mature of. One of them is really around RTL generation and then physical design and implementation in best of class. And that's simply because the models, you know, they're trained on what's available publicly. and a lot of the information is quite proprietary. That being said, there's massive opportunity between the ecosystems and everyone in the industry to make that better.

9:20It's only going to get better. Have you been fine tuning models to try and address that gap given the proprietary information that you want? We've been working with model makers around that, absolutely. And I think that's a big, big opportunity. And one of the things that I'm proud of at Arm is given our business, our core IP business, we probably have the richest IP portfolio both in terms of not only the IP, and this is the killer, the documentation, the test benches, you know, how you build the IP. You know, I've worked for chip companies in the past that have said, hey, why don't we license this IP that we've got because it's really, really valuable.

9:54And then you get into, oh, wait a minute, there's no documentation. There's no explanation on how you - No one's ever going to be able to use this. It's unusable. It's untestable, right? And if it's unusable and untestable, it's actually untrainable. And if it's untrainable, it's not usable for AI. I think we have some built-in advantages based on our business model that'll allow us to really be able to take advantage of the tools as they get better. It's really exciting. How much do you think, if you were to extrapolate out, and this is a little bit of an uncertain question, but if you extrapolate out two, three years and all the tooling is likely to come in AI and the ability to fine-tune models against some aspects of the design that you mentioned, do you think that 24 to 36-month cycle shrinks to a year to six months?

10:33Do you think it stays roughly where it's at? I'm a little bit curious about how does that really impact these cycles and time to market? Because that has pretty dramatic ramifications in terms of the clock speed of the entire industry. I don't know. I don't know if it's in two to three years away, but five plus years, can you go from idea to a GDS2 file? A GDS2 file being the file that you actually send to the fab to go get built for certain designs? Quite possible. So it takes that whole design piece out of the way It takes that whole piece out of the way relative to the verification. So I think for the more straightforward designs, quite possible.

11:12Now, if you go into the tool and say, design me something that's 10 % faster than Vera Rubin, 20 % cheaper, and 30 % more efficient on this model, you're not going to press a button and have it happen right away. But I think in five to 10 years, you know, our industry as well, we're going to see some amazing differences relative to how chips are designed. How does it change? I'm sure you had some prediction of this, but how does it change the way you look at the business, given there's just a big diversity of large players and new players that all, you know, want to have their own chip designs now?

11:50And, you know, the Veros and the Gravitrons of the world, they all use ARM. It's a big step up for them, but it's a big diversification of the customer base, right? That can be only good. Oh, absolutely. I think what's going to matter, back to the earlier discussion we had on supply chain, it's understanding the supply chain impacts, how all of it gets built and put into ultimate end products. I think that's going to become a much more important muscle as we go forward, because it's one thing to say another way. There's a lot of really great young companies today doing AI chips, well-known companies getting tons of funding, innovative designs, et cetera, et cetera.

12:30selling into an industry where the capital requirements are just massive. And the relationships with memory vendors is incredibly critical or the relationship with substrate vendors. So companies are going to have to be much more. Or access to a three nanometer line, a six nanometer line, advanced packaging line. All of it. Yeah, all of that. And I think that is, that's not going to stop in 12 months. It's not going to stop in 24 months. I think we're going to be in this constrained environment for three to five years at least. So long as the transformer is the unit of energy relative to how you generate AI training and AI inference by design, it's very compute intensive, it's very memory intensive.

13:11So if you think about that, that's going to drive a lot of demand on having supply chain acumen, which then goes back to people who've got great ideas on ship design. They're going to have to have need a lot of other things just to be able to get access to capital, wafers, everything you just talked about. We just had a cascading series of things that have been the bottleneck to more compute for the AI industry. So, you know, two years ago or so, I think it was like packaging and packaging related items. And then eventually now people want to talk about how it's memory and things like that, that are in some sense limiting to certain systems being built at sufficient scale.

13:44Do you have a view of what is the next sort of bottleneck that's coming? I think building out the data centers is going to be a bottleneck. And when I say building out, if you look at all the projects that are being done today, not a lot of them are ahead of schedule and needing less labor than they thought. And then when you when you layer on top of that, a lot of buzz that's coming from different parts of the country in the United States here relative to slowing down data center development or putting restrictions around it. I think that infrastructure buildout could be a headwind just relative to everything going on, which may be, you know, end quote, OK, because if it wasn't if infrastructure buildout was not a headwind, I think capacity for wafers, capacity for memory, that probably would be would be a headwind.

14:34So I think you're going to see a number of different governors, if you will, not governors of states, but different things are going to throttle the growth of this, which just expounding for a second. I've been on a bunch of panels and I get a lot of questions about AI bubble and when's it going to stop? And there's setting aside the valuation bubbles, which is a stock market index component. The bubble in terms of are we over oversupply to demand? Not even close. And I think, again, that's because the demand is insatious, just given the way these models work. And infrastructure build out, access to wafers, access to memory, all of that's combining.

15:13You mentioned that, and I think a lot of companies are learning today, that strategic use of the cap table, access to capital in an era where you either need to consume a lot of compute or you need to put a lot of CapEx into the ground. or you're just doing big technical projects like coming up with CPU, IP. You run SoftBank Group International. You have this one dominant shareholder. Arm itself as a business is just like a beautiful cashflow machine from the outside, right? I'm sure you think a lot about like the leverage of SoftBank and how to use that well. Like what advice do you have for entrepreneurs navigating these capex intensive industries from where you said?

15:55Yeah, so one of the benefits we have at Arm, publicly traded, yes, but a very, very large single shareholder. So I have lots of informal investor meetings with my chief shareholder, you know, all the time about this. We have a big advantage in that there's a lot of things symbiotically we can do together that can help Arm advance its initiatives by having SoftBank as our largest shareholder that we look to be very, very innovative around. To your point in terms of, you know, young companies, I would say strategic partnerships incredibly early, whether it's with people inside the supply chain, people in private equity, the banks, the banks themselves.

16:40It's a different game, you know, now on one hand, semis are kind of back because you now have a wave of semiconductor startups. There was a long time where that was just not happening, investment in the industry. Now we've got a lot, but access to capital is going to be the gate for them in terms of how they get through that. So I think getting much more creative in terms of how they work with the ecosystem is going to be super, super key. And we at SoftBank, that's one of the things we look at very strategically. You know, companies that we can bring in into the portfolio that we can help, that we can provide a combination of either the backstop, you know, and or if you think about SoftBank, we just announced, we being SoftBank, like SoftBank Neo, which is our intent to become a Neo cloud.

17:28And in that world, we could become a home for these young companies who have chip technology that in other worlds, they'd have to go up and figure out how to get a design win at Microsoft or Google. We can provide a lot of interesting avenues for that. Can you talk a little bit more about the portfolio, things that fall under your purview at SoftBank? I know as mentioned, there's ARM and then there's this sort of broader suite of things. So we'd love to hear more about what else you're responsible for. And we had some specific questions for some of those. Yeah. So the way I think about it is SoftBank Group, which is headed in Japan, and that is Masa, have a lot of different operating companies underneath them.

18:08One of the largest ones is SoftBank KK, which is essentially SoftBank Mobile. Inside the U.S., there's a lot of investment activity that's going on with SoftBank Group International. There's SoftBank Vision Fund. But increasingly, a lot of the strategies that we're trying to do around SoftBank is helping the strategies that Masa talked about publicly at his shareholder meeting in Japan, which is around robotics, open AI, infrastructure, and ARM. So I've probably got my eyeballs on a lot of stuff, to be honest with you, in terms of helping Masa really realize the execution of that vision. So, yes, I'm leading the direction of Ampere and Graphcore and another company called Stack AV that's doing things around autonomous.

18:56But maybe a better way to think about it, Elad, is that I'm kind of in the room on a lot of discussions that Moss is having and helping them sort of formulate that strategy and more importantly, help execute it. Mm-hmm. How has being part of the SoftBank group or working with all these different companies or even SP Energy and the broader ecosystem changed your point of view on what you can do with Arm? Well, one thing it does, it gives us a huge bird's eye view relative to where the broader industry is going, whether it's around infrastructure, whether it's around capital, whether it's around energy.

19:32But also you can imagine it could provide a home for our products. Right. So it doesn't need to be the home, but it certainly can be a home, which is also a big, you know, a big, big help. When we think about the verticals that soft banks involved with robotics, energy, data center infrastructure. And then you look at the products that Arm has. The only one we've announced so far is the Arm AGI CPU. You can start to connect the dots and say, gosh, there could be some very interesting opportunities that that could be an opportunity for Arm, which necessarily doesn't mean that we're getting into the broad merchant ship business.

20:08We could be just doing products simply back for SoftBank. We're a couple years into, you know, serious efforts in more generalized robotics at this point, right? If you compare it to like about a decade for LLMs. There's increasingly interesting demo results from companies on generalization of tasks and environment, more robustness, maybe even in context learning, but not like wide scale deployment quite yet. First, would you agree with that characterization? 100%, yeah. What predictions do you have about the robotics market and any opportunity for ARM there? Oh, well, broadly speaking, I think whether it's humanoids or dedicated machines to do certain level of tasks that can be retrained is going to be enormous, right?

21:00The robotics 1.0, which is a purpose-built industry, you had a piece of mechanics designed to do a certain task and the software that was optimized for that task. If you had to, a brand new automobile line came up or some different piece of equipment, if the robots weren't well suited for that, rip up the line, et cetera, et cetera. So as you can imagine, then the barrier was pretty high. getting to a world where the robots can learn just based upon either being trained or what they see. And then when you then combine that with, can you design something mechanically general purpose enough that can take advantage of being reprogrammed?

21:44And then when you layer on top of that, the costs coming down, you look at it and say, oh my gosh, what will it not be able to do? So it's almost like something out of the Jetsons, right? Where a lot of things will ultimately be done by robots. construction, infrastructure, service, security. You know, right now you see a lot of stuff on Instagram or TikTok of Olympic races with robots, et cetera, et cetera. I don't think anyone's gonna have any interest in watching a sports league of robots. There may be an enthusiast class who might be interested in that, but the broader utility is going to be around a lot of human labor tasks that can ultimately easily be replaced by robots.

22:23That's no question. All the hypotheses people have about the form factor of robotics tends to split into two or three camps. One of the camps is that they're going to be humanoid or roughly sort of the human footprint because so much of the physical world is already designed that way and the tooling is designed that way. And so you can just slot robots right in. Others view it as there's going to be much more sort of specialized task-specific form factors. Do you have a hypothesis on? I think it's both. Yeah, I think it's both. There's a lot, there are a lot of jobs and work tasks that are optimized around a person being six feet tall and having arms of a certain length, et cetera, et cetera.

22:54But I think it'll be both. and I think the fact that they're going to be smart and can learn, and to answer your earlier question, ARM is going to be everywhere. We have a tremendous amount of technology from a real-time sensing standpoint around microprocessors that will be out at the fingers. They can do perception and sensing. That's all going to be ARM-based. Today, whether it's NVIDIA or some of the work that Qualcomm does, most of the brains, the brains that you see in the humanoids, those are all running on ARM today. So I think for us going forward, the robotic industry will be powered by ARM.

23:30Are you seeing any early indications? I mean, you have this great seat to your point where given the ubiquity of ARM and a lot of these different types of devices, you can kind of see the future before others in terms of where adoption is happening or where shifts are happening from a technology perspective. Are there specific pockets that you think will be most likely the early adopters of robotics that you're starting to see some signal from? I think it's still a little bit early because the business models have not been actually figured out. The cost of robots are so high, right? Because the cost of robots are so high, people buying the robots themselves, that's a tough model to sort of get people's heads around.

24:04Does it actually replace? So I think costs need to come down and the business model need to be ultimately vetted. Because another robotic footprints tend to be things like automotive or certain surgical robots or distribution centers, right? There's a few very sort of bespoke applications that I think are most of the robotic sales today. And so that's why I was a little bit curious. Distribution centers for sure. I mean, that can ultimately go completely automated relative to, and even to the ultimate to the delivery. Right. And you can question, to me, loosely speaking, a truck that has an autonomous, is a robot of sorts.

24:41So around factory automation and delivery and distribution, that will be one of the very first to be automated. No doubt. There is increasing, you know, debate and very quickly like policy or EOs around supply chain controls and usage controls around both robotics and chips and data centers. Right. So I'm going to throw export controls in there. So four types of controls. all of these controls are relevant for you. And now either from your end customer perspective or as a relatively new entrant to, we're gonna own the end product and have a supply chain organization of your own. Like what's your stance on how protectionists, I realize it's not an American company, but you do a lot of business here.

25:33How protectionist the US or the West should be about manufacturing of chips, creation of data centers, robotics? Like what are your overall stances here? So putting my American citizen hat on for a moment, and Arm, as you said, is not an American company. It is a UK based, correct. Our HQ is in the UK, but we have a lot of employees. I wouldn't say half our employees, maybe 30 % are in the US. I think 40 % are in the UK and maybe 30 % Asia. So we're a global company, but with a huge U.S. footprint. But as an American citizen and someone who grew up in semiconductors, and I remember in the 1980s when the U.S.