IMEC Says Today’s AI Will Look Ancient in 10 Years

0:00Steven Latré:Welcome to Sorcery.

0:10Adam Chambers:The potential of photonics. The technology to interconnect these GPUs. Global payroll and HR platform.

0:16Steven Latré:8 billion operational data points. Scale-ups over startups. You'd rather have a million of something than one of anything. Building the intelligence layer for defense. There isn't enough power, there's not enough memory chips.

0:27Adam Chambers:We're at the end of physics of what we can still do.

0:40Steven Latré:Okay, amazing. So we're on our final talk for today. Today we have Steven and we have Adam. Steven, you come from iMac. Adam, you're from Strike. So we're going to talk a little bit about what comes after the GPU. So let's start with you. Could you explain what iMac is?

0:57Adam Chambers:Sure, I think we're a little bit the most unknown hidden gem in semiconductors, at least unknown for the average Joe, definitely not unknown for the semiconductor world itself. We're the worldwide leader in chips, but in actually the early stages of building a chip. So hardware is not like software, it's not that you prompt it and suddenly 30 seconds later something comes out of it. No, it's a process of five to ten years and we're at a very early stage of that process where we have a unique clean room facility that nobody else in the world has and so all the biggest chip providers in the world actually first come to us first in Belgium to kind of design that next generation chip that you'll see maybe in seven eight to ten years from from now and going through a whole phase of designing that chip we hand it over then to to them to further commercialize it but in that sense there's almost no chip in the world today that has not been touched by IMEC in some way.

1:54Adam Chambers:We already exist for 40 years and we've been driven more or less that chip roadmap for the last 40 years.

2:00Steven Latré:So I should have asked you the secret question because you know all the secrets on chips.

2:04Adam Chambers:And some of them I'm allowed to answer.

2:06Steven Latré:What can you answer?

2:08Adam Chambers:Well, I think we're definitely going to talk more about that, but indeed that we're actually seeing a huge amount of disruptions going on. I think we're now kind of plateauing in what we call Moore's Law. What does that mean? chips have been becoming bigger and bigger or faster and faster over the last years. We always did that through scaling. So what does that mean? It means that we made the transistor smaller and smaller and we're at the level that a transistor has about the depth of one nanometer. What is one nanometer? It's about 100 ,000 times smaller than a human hair. That means that we're at the end of physics of what we can still do and I think we're gonna come into an era where where what we do in hardware needs to be way more inventive than what we've been doing in the last 10 to 20 years to keep on scaling that roadmap of more to make sure that chips become faster and faster, as they already did the last four decades, basically.

3:04Steven Latré:Adam, as you look into the chip industry, how do you make sense of it? How do you map it out? Yeah, I think right now, GPUs are very good at the training part. and we're now looking at the bottleneck that sits in the inference layer and the bottlenecks that I look at the most and try and look at companies that attack those bottlenecks sit in power and energy, memory, cooling and data transfer. So when we look at the inference side of things, memory, the memory shortage in general and increasing the bandwidth and lowering the latency of getting the useful memory to a GPU as quick as possible is a really interesting and exciting place that needs to be builded within.

3:49Steven Latré:Cooling as well and where that's going, I think bringing the liquid that takes heat away from the chip closer and closer to the GPU itself is how it's going to progress over the next five to ten years and I think a lot of the value will go to the manufacturing side of that. And yeah, I mean, power, everyone knows how much of a constraint and a bottleneck that is. And increasing the capacity there is huge. I recently interviewed Tony Kim of BlackRock at the Raze AI Summit. It's actually in Paris. And we're talking about hardware. It's like the hardware era, renaissance, everything's shifting to hardware.

4:29Software is getting, it's taking a bit of a beating,

4:34Steven Latré:But the margins, people don't understand the margins for chips are fantastic and for hardware versus software, which is getting eaten by AI. And again, like the token spend and that kind of thing. So how do you make sense of the shift in value and how like the pyramid kind of flipped? Well, I think the demand is just so high for the thing that is less fruitful out there. Right. So there aren't there isn't enough power. There's not enough memory chips. The shortage in memory, I mean, if we look at, there's only three companies in the world that produce memory, which is 95 % of the market, Micron, SK Hynix, and Samsung.

5:17Steven Latré:Two of them are in Korea. And a lot of their shift has been filled out by hyperscalers taking over their whole inventory. So you've seen prices rise by 700 % this year just for the memory part of the chip. and that will happen across the layer because it's the thing that's in the most demand. The training side and the software side of things is continually getting better, but the physical side is the thing that's harder to get better overnight.

5:50Adam Chambers:Yeah, to be honest, I don't really think it's necessarily flipped. I think it's more the immaturity of the software market right now. Might seem weird that I say immaturity of the software market, but I think what we've been doing at AI the last four or five years has of course in software been tremendous but if you look at it from okay how how complex it is to actually in terms of computations do whatever I mean large language models can do it's still very much a brute force approach and as a result the value that you can get out of it is pretty limited I think if you're gonna get further we're gonna see new types of software evolutions as well which are tied way more closer to to hardware and because of that I think the value will increase way bigger.

6:31Adam Chambers:And I think so we're kind of filling the base of that pyramid already in hardware of creating a lot of value there. I think software will follow in that sense.

6:40Steven Latré:We're seeing more and more chip companies founded every year, probably than ever before. And venture dollars are definitely helping fund that. Is there a capacity to how many chips can exist in the market?

6:54Adam Chambers:I think right now there's kind of endless demand for that. and I think it's still going to be the question of how that can continue. I think it's rather the bottleneck is manufacturing, manufacturing the right type of components in different regions. And I think, for example, what Temma Foundries is doing there is super important. But having the right type of components for that is, I think, the real bottleneck, not necessarily the number of companies that do this, because every company can come with a completely different, unique design and in that sense can also kind of revolutionize what we're going to have as well.

7:32Adam Chambers:And probably in the software world, we're going to see way more different types of algorithms. And as a result, we will also need more diversification in the type of hardware. So more than just GPUs as well.

7:44Steven Latré:As models get bigger and bigger, what breaks first on the chip layer?

7:49Adam Chambers:Well, yeah, I think for sure right now, memory. Memory is absolutely the number one component that is already broken to today. But I don't really think that models will become necessarily bigger and bigger. If you just think about it, the latest open source models like Kimi K3, they already have more parameters than what the human brain has. But on the other hand, that K3 chip or the chip that we need to fuel that K3 model is about a million times more or less energy efficient than what the human brain is. And so I think we're going to see a different type of paradigm. We kind of went into an era the last five years of building bigger and bigger and bigger models.

8:37Adam Chambers:We went from 175 billion parameters to kind of 10 trillion parameters right now. That's going to stop. And there's a lot of companies right now taking that shift as we speak, abandoning what we call the scaling hypothesis of building bigger and bigger models and therefore trying to solve those those memory bottlenecks as well.

8:58Steven Latré:Copper has kind of met its days. I'm curious how do you think about photonics and how that changes the layer?

9:05Adam Chambers:Yeah I mean feel free to.

9:08Steven Latré:Yeah I mean I think there are so many benefits to using photonics and instead of copper and copper is easy because it's all in electrical, it's all digital. And the bottleneck with photonics is changing between optical and the electrical layer. But transferring data more quickly and more efficiently and not having that latent heat aspect that electrons and copper wires do will only continue to scale the efficiency of data transfer. And then I think we'll see the architecture continually shift for where photonics will be within data centers. At the moment, it's between the clusters and we're using linear pluggable optics.

9:53Steven Latré:But I think in the next five years, we will see these optical components get closer and closer to the actual GPU. And we will go towards a state of co-packaged optics.

10:03Adam Chambers:So for me, I mean, every day what I do actually at IMEC is thinking about what could be AI in the future. So how is AI going to evolve in like five to 10 years from now? and I think we're seeing kind of from a software perspective that very much diversifying path of a lot of different approaches whether it's world models or more reinforcement learning based approaches but what is 100 clear is that whatever scenario in the end will become the dominating model of the of the future moving data as fast as possible in the quickest way with the huge amounts of data that we need to shift is always going to be what we need because again towards the analogy of the human brain that's what our human brain is also unbelievably good at we have an unbelievable 3d structure that is able to to do this and so photonics will play i think for me uh especially in the the next years it's going to be duck killer technology to to to enable

11:03Steven Latré:the chips of the future i think also on photonics it's attacking many parts of the bottlenecks that I touched on at the start. It's attacking energy because it can transfer data quicker. It's attacking cooling because you don't need as much cooling around these GPUs if you're transferring through light. And then it's attacking data transfer at the core. So I think having spanning most of the bottlenecks that we're seeing in the AI Infra build out right now, it's like the key technology that is likely to win. As we close out, and this is the last talk of the day, so thank you all very much. What is your hottest take right now?

11:43Adam Chambers:I think we're going to, five to ten years from now, we're going to laugh at how old-fashioned AI was today. That's a bit my hottest take in the sense that, of course, we went through this explosion of what AI was bringing us, but I fundamentally don't think that today's technology, that this is what we're going to be talking about in ten years from now. I think an actual, also in software, revolution is coming. and for those who are old enough we still might remember the the internet days of a dial-up connection where we had all these bleeping sounds to make connections to the internet and we kind of laugh at how that's how we did it in the back of those days i think in 10 years from now we're going to talk about large language models which are very much a brute force approach

12:27Steven Latré:in exactly the same way i might be a little bit biased on this one but i would say it's a very good idea to recruit or invest in people that have had a academic and technical background in Europe and had the wherewithal and business sense in the US and combining those two mindsets. A lot of the best founders that I've backed and I've seen have that dichotomy between the two. So I'd say both recruiting and investing those two traits. Amazing. Well, Stephen, Adam, thank you very much. I think Louis is going to come up next and say something.

13:16Steven Latré:Hey, it's Molly. If you enjoy our interviews, check out our newsletter, sorcery.vc, where we deliver a once a week top deals and tech headlines email, and also go deeper on our podcast interviews. Subscribe to Sorcery today. And don't forget to subscribe to the podcast on YouTube, Spotify, Apple, or wherever you listen. Link in description to sign up.