Memory chip mania isn’t going away, with Dan Kim

0:00Chips are one of those products that everyone cares about. Consumers care about them because they're in every electronic device we own. Tech companies care about them because they are crucial to the AI build-out. Politicians and economists care about them because they are central to geoeconomic tensions between the US and China. In the past, drama in chips has been focused on logic chips. Those are the ones that process data. But recently, another, less glamorous corner of the chip market has been going crazy. Memory chips are red hot right now. AI data centers are hoovering them up, pushing up prices and crowding out customers elsewhere.

0:43It is a wild time to be in the chips business, and today I'm going to try to make sense of it. What are the causes and consequences of this mayhem? And could China play a role in bringing this madcap market into balance?

1:04This is The Economics Show with Samaya Keynes. I'm joined today by Dan Kim. Dan is the Chief Strategy Officer at semiconductor analysis firm Tech Insights and was previously the Chief Economist at the CHIPS Program Office at the US Department of Commerce. Dan, welcome to the show. Thank you for having me. Okay, so first question, on a scale of 1 to 10, how crazy is the memory chip market right now? Right now, it's an 8. Next quarter, it'll be a 9. And the quarter after that, it'll be a 10. It'll stay at a 10 until the end of 2027, at least. And then we'll see how long it stays a 10. But it'll be a 10 until at least next year.

1:47Okay, that is remarkably precise, which I appreciate. Could you put some slightly more serious numbers to that craziness? What are prices doing right now? Well, on a year-to-year basis, the prices for DRAM, which is one specific kind of memory that is being used for data centers, mobile phones, pretty much everywhere there's electronics. From last year to this year, we're anticipating a over 200 % price and increase of DRAM. That is historically unprecedented. You do see swings in price cycles in the DRAM industry and the NAND industry, which is the other part of memory. But this amount of price swing is historic.

2:28And we actually don't quite know when this upcycle will end because there is no new supply coming online in significant volume until at least 2028. And because of AI demand, demand is increasing, but supply is not increasing. And so the only thing that could adjust right now is pricing. And that's what's happening. Okay, so pricing is high. And presumably, there are customers elsewhere who are just not able to get the chips that they want? Yeah, so, you know, interesting thing about the memory market is that because the goods are standardized, it acts like a commodity market. There's actually a spot price on a daily basis for DRAM, for example.

3:08Now, most of the chips are bought and sold on a contract basis, but there is a spot market as well. So you can track what the prices are doing. So in a market like this, you would presume that if you're willing to pay the price, you could get the good. But we're heading into a territory in which the price signals are actually not working as well, meaning the price signals are so high that even when you're willing to pay the price, you might actually get a reply back from the manufacturer saying, we could only give you X percent of what you're looking for, even if you're willing to pay the price we're quoting you.

3:42And so we've been hearing that from not only the high end of the market, the data center AI companies, but we've been hearing that more and more from the lower end of the market, meaning the legacy demanders, automotives, medical device manufacturers, and others, that this is an issue across the board where the prices are going up. But because there's no new supply coming online and demand is increasing, there is now an increasing allocation happening in the market. Do you have a rough sense of what share of these chips that are actually being supplied are already being sewn up in contracts and what share are being traded on the spot market?

4:23Yeah. And so effectively, there is no supply out there that hasn't already been spoken for. So practically, most of the supply has been sold out. This has been the case for high bandwidth memory, which is the specific type of DRAM that is paired with AI accelerators to make AI training chips. But it is also the case for most of the memory products out there for this year, if not for next year. And so if you're looking around, hoping that there is some memory products out there that are on the shelf that has not been spoken for, it's going to be very difficult to find right now. To what extent are companies trying to lock in supplies, contracts, for these things even more than they need because they're worried about future shortages?

5:08Is there that kind of hoarding dynamic? There is some stockpiling. There is some overbuying. That is partly what explains the memory cycle. In the previous cycles, hyperscalers, for example, have overbought. And then the year after that, it turns out they don't need as much. And so the orders crash. And so the price crashes. This is the usual cycle. This time, we're not seeing too much of that. We are seeing effectively, if you have memory, you're using it. There's not some stockpile out there. We are in a situation in which if memory is being offered, companies would buy it, even if they don't think they would need it.

5:44And so there is some of that going on, I am sure, but it is an extraordinary tight market at the moment. Do you have a rough sense of how important the cost of memory chips is for some of those, you know, high profile electronic goods, like, say, the iPhone? Well, memory is already among the highest part of in a bill of materials for any electronic good. So, for example, if you open up an iPhone or if you open up any smartphone, the three highest value components in there would be the application processor. This is a leading edge logic chip that is usually designed by the firm and then produced at TSMC or Samsung.

6:20And then the two other components that are highest value is the DRAM, which is the memory, and then the NAN, the storage. So those are the three highest values. Now, the two of those chips, prices are going up significantly. On an average phone, the memory component could be somewhere between 15 and 25%. Now it has a potential to be somewhere between 30 and 40 and even 50%, depending on the pricing. Okay, so you said that basically there was no new supply coming online. why? So we actually have to go back a little bit to explain why we're in the situation that we're in. It's not long ago, actually in 2023, that we had a down cycle in memory where the prices were depressed and the memory companies were in negative margin territories because there was too much supply.

7:11Coincidentally, this was the same year that ChatGPT was really taking off, but there was too much memory in the market. And so there was a decision, a rational decision by memory makers to cut their production and to delay their build out. At the same time, the demand was skyrocketing for AI. And there was some uncertainty about how much demand that would be. And so it was treated like business as usual, but then the demand tsunami came. And so because of the delays that were happening, the supply is not being introduced today. And so at the moment, the memory companies are all of them, all memory companies throughout the world, the Koreans, the Japanese, the Americans, and the Chinese are racing as quickly as humanly possible to build new supply, but they're not coming online until late next year or early 2028.

8:02So how long does it take between executives saying, okay, we need some more, and okay, there are chips being sent to customers? Well, if you're starting from scratch, you have to find a site, you have to develop the site, You have to build infrastructure around it. Then you build the shell of the fab. Then you equip it. And then you qualify it. All of that could take somewhere between two to six years, depending on the site readiness and depending on how fast you could build and depending on whether you can have the equipment ready to go. And this is why you usually have some plans to build in case of new demand coming online.

8:38And so for pretty much every memory company, they have sites where they could build another fab on top of it in case needed. But the demand is extraordinary. And so the industry really was just cut off guard. So there is a site in South Korea, for example, two sites in South Korea, for example, in which both Samsung and SK Hynix are building just as quickly as possible. There's a site in Idaho that Micron is building that is trying to build as quickly as possible. The same thing is happening in China by Chinese companies. And so the supply is coming and they're coming in large numbers. They're just not coming yet.

9:16Could you talk a bit about the structure of this market? Because it's very highly concentrated, right? There are only a few firms and entry. Isn't that easy? That's correct. In fact, it used to be that there were many memory firms and then there was a wave of consolidation that happened in the memory market over the last two decades. And the reason is because the cycle on the market is really brutal. And so unless you actually have the right volume and the right strategy to ride out those waves, it's not a business you can really survive in. In terms of DRAM, there's only three companies left for mainstream DRAM production.

9:54It's Samsung, SK Hynix, and Micron. So it has become a three-form market, at least for DRAM. There's a couple more in NAND. And then there are the entrants in terms of the Chinese companies, YMTC for NAND and CXMT for DRAM. You asked about whether it is possible for new companies to enter the market. The answer to that is it is close to impossible for a new company to come and become competitive in the memory industry for two reasons. One is the simple scale and volume you need to compete. This is a scale and cost business because it is standardized, because it is commoditized. Unless you have the economies of scale to actually survive in the business, it's almost impossible to compete.

10:37So that's economics is the biggest mode. The other mode, of course, which is considerable is technology. It is very difficult to make these now. And so there are two types of firms in the semiconductor industry that actually uses EUV lithography. One is Advanced Logic, TSMC and Intel and Samsung, Boundary. The other is DRAM. And so you are really at the highest end of sophistication when it comes to manufacturing. And so you have the economics mode and you have the technology mode, which makes it almost impossible for a new entrant to come in and become competitive. The only exception to that are Chinese firms.

11:14And that's because there is almost no end to the state support that they're receiving to survive in the business. So the Chinese state is clearly interested in Chinese companies becoming more dominant in this memory market. What is the current trajectory of the firms that it is supporting? So I'll take it one by one because the two markets are quite different within memory. So I'll take NAND first and then I'll go to DRAM. On the NAND, and NAND is the storage part of your memory. This is the memory that stores your pictures, your videos. And there is a Chinese firm called YMTC that is actually quite competitive in the market today.

11:49So depending on how you count, it's either the second or third largest provider of NAND into the market today. They're not as sophisticated as their Western counterparts are. So their ability to stack the 3D NAND chips is not as good. And so they are still challenged. And so they wave back and forth between being profitable and not. Right now, they're very profitable, as all memory companies are. But they are a global provider of NAND at the moment. NAND from YMTC is primarily found today in Chinese products that are being sold into the Chinese market. And so very rarely would consumers in America or Europe actually find and use YMTC-enabled devices in their businesses or in their homes.

12:36So YMTC is already there. This is already an entity listed company, meaning that no Western equipment can be sold to them. But despite that, they're functioning quite well and making progress. And then to DRAM, there's a company called CXMT that is more behind relative to YMTC to the Western competitors. But they're starting to introduce more and more of their DRAM devices into the market. we here at Tech Insights are finding more CXMT memory products within flagship phones like Huawei and other Chinese handsets. So they're more behind, but they are challenged by the fact that they do not have access to EUV lithography, which is absolutely necessary now to make advanced DRAM products.

13:20So they are trying to find ways to get around that and also introduce large volumes of DRAM into the market. Do you have a sense of what their market share could be like in the next few years? I mean, obviously, Chinese buyers are a pretty big chunk of the global market. Yeah, that's a good question. So already in the NAND space, YMTC is producing something between 10 and 15 % of the bits into the market. And so their market share is growing. But if you sort of take a step back and look at the historical trends of market shares in the semiconductor industry, the market shares among the companies stay remarkably stable.

13:59As I explained before, it is very difficult to gain entry. It's also very difficult to gain market share. So you really have to offer a differentiated technology or something else has to come along to change market share significantly. We've seen some of that through, for example, high bandwidth memory that SK Hynix has really been the leader of in the past couple of years. But of course, the other firms are catching up and providing competing products of their own. So it is an interesting question to ask, how much can the Chinese actually provide into the market? Yes, they have a little bit of a captive market that they can sell into in China, and China does consume significant amounts of memory.

14:38But the U.S. is actually the largest memory market in the world, period. And Chinese are a little bit far behind. So we are estimating that in the next few years, the DRAM market, the U.S. will be somewhere above 60 % of the DRAM market. And that is primarily because there's so much AI buildout happening in the U.S. And because the DRAM for hyperscalers and DRAM for AI, the value of that is so much higher that U.S. has become by far the most important market. So there is a little bit to the limit of how much Chinese memory producers can gain market share as long as they don't have access to the United States.

15:16But to your point, they can capture more of the Chinese market, and I'm sure they would have more favorable access to it. So they can definitely gain more market share over time. And I think the test will be when the next down cycle happens, because it is a cyclical market. We're definitely in a prolonged up cycle. When the next down cycle happens, how much countercyclical investments can the Chinese make at that point to capture more market share? That would really be the telling sign. We're a few years off from that, but I see the Chinese getting ready for that. And that is a very predictable playbook in the memory market because it is effectively how the Japanese memory makers planned to disrupt the American memory producers back in the 80s.

16:00And the same thing happened with the Koreans and the Japanese. And so the Chinese, I think, are hoping that they're next. Oh, that's interesting. So today you hear concerns about potential future overcapacity in Chinese chip making, but maybe they were just copying a playbook that was played out previously. So there is a significant difference between how the Japanese and then respectively Korean companies have competed in the memory market to disrupt the incumbents than what the Chinese companies are doing today. The level of state support in China is simply unprecedented. There has been trade disputes before, famously in the 1990s, where trade remedy cases were brought against the Korean companies.

16:42But if you were to ask memory companies today how to compare the disputes then to the amount of state support that Chinese companies are receiving today, they would all tell you it's night and day, not even comparable. So just thinking about what the Chinese state is doing here. I mean, for a long time, they have spoken about wanting to reduce their reliance on other countries. And so you could see this domestic build out of chips as part of that. To what extent, though, is this supply crunch reinforcing the idea that these memory chips are a sort of strategic asset, right? And, you know, maybe actually the South Koreans or the Japanese have power because they could restrict access to those chips.

17:24Or I suppose we've already seen the U.S. restricting access to particular chips. Well, in this case, for memory chips and other advanced chips, the idea that it is a strategic good to have state support around it in some way, shape or form, China is not the only one. They've just been earlier, in a sense. If you go to the U.S., I think they would see now that the memory is a very strategic asset that they need to develop. And there is state support going into that. If you go to Korea, they would say absolutely, it's a matter of critical importance to the country to have memory competitiveness.

18:00I think if you go to Japan, they still have memory production and it is a priority for them. And the same could be said with China. It's just that they were in our current era earlier to recognizing that memory is something they wanted to develop and support and have been at it longer. I mean, you worked on implementing the CHIPS Act. Was memory a big part of your job then, or has it really become more important over the last few years? So when we wrote the strategy for implementing CHIPS, we did say that memory was important and that we were in particular looking to support DRAM manufacturing in the United States.

18:40Part of that was because there was already a project or two that had the capacity to make DRAM at the time. But this was 2023 in which there was a glut of memory into the market. And there was an overarching concern about the concentration of advanced semiconductors of logic at foundries coming from Taiwan. And so the feeling at the time was that we needed to hedge more against the logic. But we also recognized that we were playing the long game here and that we needed a little bit of everything. And that memory was an important thing to focus on. What I think, to be fair, we did not anticipate is how dramatically the economics of memory would shift due to AI.

19:23I'm not sure if anyone really fully appreciated that. Of course, if we did, there would be more supply today. But we missed that. And I think the entire market missed it. But memory was not a forgotten topic. It just was a little bit less emphasized. Does this memory production represent a choke point in the same way that high-end logic chips do, right? I mean, is the concern that China might get huge market share and then cut off other countries? Or is it something a bit different? Like, what's the thing that everyone is worried about happening if you weren't to intervene? Well, right now, the concern is there's not enough supply.

20:01Right. But that's not really a geoeconomic thing, right? Right. That's just like we need chips. So let me try to connect the dots here. Right now, the concern is there's simply not enough memory chips to go around. So the temptation is for non-Chinese companies to be sourcing from Chinese companies today, which opens the door for them to grab more market share down the line. So if you're a careful observer of this market, that is a concern if you believe that more market share concession is coming down the road on the next down cycle. But I'm not sure if anyone's thinking that much far ahead. It is difficult to gain market share in the memory market because the Korean, American, and Japanese producers have been so good and so disciplined and efficient at making them.

20:50The Chinese plan in about 10 years ago was actually to acquire technologies and failed at that, meaning they actually tried to take over certain companies back in 2015, which did not receive regulatory approval. I think they recognize that it's just a very difficult business to get into with nothing, that they need to acquire technologies and the know-how and the capacity to do that. But that road was blocked to them. And I think you have to give a lot of credit to the Korean, American and Japanese producers of keeping the market so well disciplined over time to be able to keep the Chinese out.

21:28But it is a very brutal, competitive business with very difficult time to entry. But I think you're starting to get a little bit more sense now that this is a focus for China and that they're not going to give up. And they are looming in the background, even as the market has never been more successful than before. OK, so the Chinese government is clearly giving a lot of support to its actors. They want to get a bigger market share. How are governments elsewhere responding? I mean, you said that the Chinese were just early. Is it the case that, you know, the U.S. government, the Koreans, the Japanese, are they now increasing their support to increase investments?

22:09Or actually is the market signal so powerful that they don't need to? I haven't really seen a memory strategy from either the U.S. or any of the partner countries of the United States that is very deliberate, except for South Korea. They have seen and they recognize that memory is one of the most important geopolitical assets they have. and the South Korean president, along with industry leaders, announced a development of a new production cluster in the southwest region of the country. I think that is a signal that says, okay, there's a lot more build out of memory here that's going to happen around the world.

22:45We want to make sure that we're part of that build out as well to keep our competitiveness. And so in some ways, the Chinese have been recognizing the importance of memory for a long time. The Koreans are clearly recognizing that. And then now I think it's the Americans and others because of the shortages that will soon wake up and to think about, you know, what they want to do. Okay, well, I want to go to a break now. And when we get back, I want to talk about the wider chip markets and not just memory, but logic.

23:26credit scores were generated with just a snapshot, a snapshot that left millions of people out of the picture. So VantageScore is modernizing the credit score. VantageScore uses 400 % more credit report and alternative data than legacy models, bringing more people into the credit economy, like 5 million more mortgage-eligible people. Banks win and francs win. Lenders win and so do lenders. When you build credit scores the right way, everybody wins. Get started at VantageScore.com.

23:58We are back from the break. So as you said, the Chinese are massively expanding in memory chips. What other types of chip have the Chinese been showing success in? Well, let's be clear. They're showing progress in a lot of different places in the semiconductor market. So two ones that I would highlight. One is just legacy foundry capacity. and foundries can make logic, can make analog, can make different types of chips. And so they are really copying Taiwan's foundry model in which they have fabulous or design-only companies in China utilizing domestic foundry capabilities. So there's a big build out on that.

24:40In fact, most of the new capacity that we see from say 2022 to projecting out to 2027, most of the new foundry capacity in those older nodes are mainly in China, not so much elsewhere. So they're becoming more self-sufficient there. The other place I would note is that they are making great strides in what we call compound semiconductors. This is semiconductors that are not based solely on silicon, but on compounded materials like silicon carbide, gallium nitride, and other more exotic materials that are much better at power semiconductors, so they can handle higher voltages, for example. So one example of that is a silicon carbide-based power inverter for electric vehicles.

25:29So how do you enable high voltage charging, which enables faster charging in the electric vehicles market? So this is a very difficult material science challenge for the industry. And China is making great strides in that field as well. Are there any areas where the Chinese have really been trying to expand their own chip capacity, but they haven't managed it? It is actually memory. When I started covering the semiconductor industry for the US government more than 10 years ago, it was widely assumed that that was the most obvious place where they would go and take over first. Standardized good, scale economics, it sort of really fell into Chinese wheelhouse.

26:14But they had a really difficult time breaking into the market for a long time, particularly because the Korean manufacturers and American and Japanese manufacturers really kept them at bay. It is very difficult to catch up with the scale and the pace in which the memory market moves. It's only recently that they're making really great strides now that they are posing more of a threat. What about the advanced logic, right? So these are the teeny tiny two nanometer, three nanometer chips that TSMC excels in where production has mostly been in Taiwan. There were some US efforts to try and pull some of that production away to the US.

26:50What's been happening there? So there is two challenges when it comes to high-end logic. One is economics and one is technical. So on the economics, if you do not have access to the most advanced lithography tools, then you're bound to have more defects on the wafer. And so that poses a technical challenge as well as an economic one. So for example, the latest node that we see from Chinese foundry SMIC is what they would call an N plus three node, which is a six nanometer node, which has already been introduced by TSMC and others several years ago. The technical challenge actually poses an economic challenges here as well, which is if you have too many defects, in a wafer, then you can't actually make larger size chips, which actually limits your market opportunity to some degree.

27:41AI chips require larger size chips, larger dyes. And if you can't yield those, then you don't really have market access to the degree that you need. And so they are challenged because they don't have access to these tools. Okay, so the Chinese are struggling because they don't have access to the highest end lithography machines. There was that story about how they may have managed to cobble together an EUV machine using old parts. The US administration certainly seems concerned that they've managed to do that. What did you make of that story? Well, we haven't seen evidence yet that they have a domestic EUV capability.

28:16But I think what we are seeing is that because they are challenged in this particular technology path, they are looking for other technology path to make up for it in ways that you would do not need to do outside of China. So for example, if you only have access to smaller, less performing chips, then you have to figure out a way to package them more efficiently in an advanced way to cobble together the performance. And so we do see more advancements in advanced packaging in China. And we also see that systems level integration advancements are also present in China. So how to string together chips, how to string together systems together, and then sheer volume of power flowing through the system to get the compute performance out.

29:01So what they lack for in efficiency and economics because of the expert controls, they are trying to make up for it in different technical solutions, but also just sheer volume and sheer systems level integration and power to make up for the compute power lack. What do you say to the claims that because China can wriggle around the controls, because it can adjust along these other margins, actually, there's no point in applying the export restrictions. The US should just let China have what it wants. So let me try not to make a policy point on export controls, because that's really not my place to make them.

29:33But let me make a market observation. If you were to allow every type of equipment that is currently under restriction to be sold into China today, they would purchase a lot more. And they would advance more quickly. There's no question about that. So I think you have differences of opinion as to the effectiveness of expert controls. Some prominent government officials in the U.S. have called them a speed bump, which is not a permanent solution, and the need to run faster. But these are very difficult challenges for policymakers to grapple with, for sure. What's the latest on U.S. progress in trying to move production of the highest-end logic chips away from Taiwan?

30:15This is obviously an objective as part of the CHIPS Act, score your own progress. Yeah. And to be clear, the objective of the CHIPS Act wasn't just to hedge Taiwan risks. It was to build more semiconductors in the United States because it was a strategic choice that the Congress has made. So there is significant progress. The Arizona facilities at TSMC is yielding quite well in Fab 1, and they are now at Fab 2. Intel is producing 18A chips that has already been introduced to the market, and Samsung is continuing its build in Texas. And so there are several facilities that are going up and expanding for advanced logic in a way that hasn't.

30:57But it is also the case that capacity for advanced logic is being built faster outside of the U.S. than inside the U.S. But compared to where we were a few years ago, when we pretty much had no advanced logic capabilities for foundries, it is significant progress. Thinking about what the Trump administration is doing right now, they've clearly been applying, considering trade restrictions. If you were back in government, what would be your advice to senior decision makers? What would be your priority? What I would say is businesses need predictability in the policy environment that they live in.

31:36And so providing as clear guidance and consistent guidance as possible is necessary for very large investments to happen and to execute, which is what the semiconductor industry needs. I would also note that there is enormous amounts of human capital involved, particularly because foreign companies are such an important part of the equation here for them to make foreign investments into the United States. Having a predictable policy environment on both labor and capital and trade are so important to have. Dan, thank you so much for joining me. and also thank you for all your help when I was writing my book, How to Win a Trade War.

32:16I'm so glad that I got you to share your insights on the podcast as well. Thank you.

32:27That is all for this week. You have been listening to The Economics Show with Samaya Keynes. If you enjoyed the show, then I would be eternally grateful if you could rate and review us wherever you listen. This episode was produced by Misha Frankel Duval with original music and sound design from Breen Turner. Our executive producer is Manuela Saragossa. I'm Smeya Keynes. Thanks for listening.

32:53The new episode of The Next Five is about industrial decarbonisation. Dino Otranto, CEO at Fortescue Metals, joins me. The single-handed reliance on oil and gas is over. As does former World Bank CFO Bertrand Padre. There is more money going into the world than into decarbonizing cement or steel. And you refer to Director General Axel Eggert. The steel industry needs to be decarbonized by 90 % by 2035. Listen to The Next Five wherever you get your podcasts. Enjoy. For years, credit scores were generated with just a snapshot. A snapshot that left millions of people out of the picture. So Vantage Score is modernizing the credit score.

33:33VantageScore uses 400 % more credit report and alternative data than legacy models, bringing more people into the credit economy, like 5 million more mortgage-eligible people. Banks win and francs win. Lenders win and so do Lindas. When you build credit scores the right way, everybody wins. Get started at VantageScore.com.