Chip Design in the AI Era with Thomas Andersen

0:00Synopsys is a leading electronic design automation company specializing in silicon design and verification, as well as software integrity and security. Their tools are foundational to the creation of modern chips and embedded software, powering everything from smartphones to cars. Chip design is a deeply complex process, often taking months or years and requiring the coordination of thousands of engineers. Now, advances in AI are beginning to transform the field by reducing manual effort, accelerating timelines, and unlocking new design possibilities. Thomas Anderson is the Vice President of AI and Machine Learning at Synopsys, where he has spent over 15 years.

0:41He joins the show to talk with Kevin Ball about the evolving role of AI in hardware design, the challenges of training models on tacit, undocumented chip engineering knowledge, the emergence of domain-specific LLMs, and where this fast-moving field is going next. Kevin Ball, or KBall, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action discussion group through Latent Space. Check out the show notes to follow KBall on Twitter or LinkedIn, or visit his website, kball.llc.

1:33Thomas, welcome to the show. Hey, very nice to be here, Kevin. It's a pleasure. I'm excited to get to know a little bit more about you and what you're doing at Synopsys. But let's maybe start with you. So can you give us just a quick background of who you are and what brings you here today? Yeah, I'm Thomas Anderson. I have been a long time, we call them synopsoid. So Synopsys employee, I've been there for more than 15 years. And I'm running essentially the AI and machine learning team in Synopsys, developing automation with AI for our software products. I love this topic. I'm completely geeking out on all of machine learning and AI these last couple of years.

2:14But before we dive into that, let's maybe do a quick overview of EDA and what Synopsys does in general, since we have folks across a wide range of software engineering backgrounds. Yeah, totally. Because I think obviously not everybody's familiar with what Synopsys does. Even though we are one of the 20 largest software companies, we're obviously a B2B business, and therefore not everybody is as familiar with us. We essentially provide solutions, what we call silicon the software. So we provide offerings to help our customers build and design their chips, as well as implement and test software offerings.

2:53So essentially from the entire spectrum, from what's called the electronic design automation software. So anything that's required to build the little chips in your phone and in your TV and in your car, all the way to IP that goes under those chips. And also from the software side, obviously, it's not just hardware. You have a lot of software that runs on top of this whole stack. We provide solutions for application security testing and software integrity. So that's our main offering. Essentially, if you think about it, next time you pick up your iPhone or your Google Pixel or whatever is your preferred choice, think about it.

3:31That little chip in there, as well as part of the software that runs on it, was essentially created and designed with Synopsys software. Yeah. And it was fun to me doing the research preparing for this because I have lived most of my life in sort of the consumer software space that starts at an abstraction layer where I just assume all of your stuff is already working. Well, that's sort of where the magic happens, right? See, a lot of the people, they understand software. But I think not many people are familiar with hardware or how hardware is designed. So the assumption is, yeah, that thing just works somehow.

4:05But if you think about like building a chip has so much complexity to it. It's not just the logic side that you have to do, like the equivalent of software, so that something runs on the hardware that computes something. There's also all the physical aspects, right? They need to be manufactured in fabs with very, very small nodes. And that's super, super challenging, pretty much across the spectrum. All right. So let's now talk a little bit more specifically about what you're doing. So how are you applying machine learning and AI to this very deep software stack that Synopsys offers? Well, essentially, if you think about it, so the software we're building, our users are expert designers or engineers, right?

4:48It's not software like Microsoft Word, where everybody essentially is familiar with it and just types something in it. It's very, very complex software. And designing a chip from start to finish, like our customers, it can take 12 to 18 months to build a chip from start to finish. That sounds like an exorbitant amount of time as you think about it compared to software design. Software design people, I mean, of course, I don't want to simplify it, but you write something, you compile it, and you can get something out relatively quickly. Chip design, I remember talking to an executive at Google, actually, and they were fascinated by this.

5:27We were working essentially on some reinforcement learning together, and I remember he was telling me, he said, it's just fascinating how complicated it is to get those chips out the door. So it's a very, very long process. The tool chain, essentially the EDA, electronic design automation software that we provide is also highly complex. And the manuals are complex. The way you optimize your design requires a lot of human interaction. And there is a very, very high desire, of course, to reduce the human manpower that is required to get a chip out the door. Today, I would say chips, you can take thousands of people.

6:04Really, it's no underestimation at all. It can take thousands of people to get this thing done. If you want to do it faster, if you want to do it with less people, well, we need automation. And of course, AI is the path to it. But because it's complex, it's not as easy, right? It's not as easy as, say, self-driving cars. I love you to use that as the baseline. Not as easy as self-driving cars, which they've been working on for, what, 20 or 30 years? And we keep saying, oh, it's going to be here. It's going to be here. That's exactly why I said that, because it's a funny joke. Elon Musk has been saying this for how many years?

6:41They're going to have it next year, next year, next year. I think he stopped talking about when exactly it's going to happen. But in fact, other companies, of course, I would argue have better systems, right? But regardless, we're still not there, right? I think Mercedes has like a level three certification. And to get to level five, I'm sure will take a long time. And now think about it. Driving a car, you can argue is a very simple task. As long as I can hear, I can see, and I can move my hands and my legs, I can operate a car, right? Relatively simple rules. And at least in theory, it is something that I would argue a machine can do better than a human simply because, I mean, the interesting thing there, in my opinion, is that the simple things that we as a human just know to teach that to a machine actually takes time.

7:34But then when it gets to more complex things, like let's say you're driving on the road and there is a situation where you have to do some emergency handling to avoid some object or some kind of scenario. There's no doubt in my mind that the machine can do this better because if I train it with all these possible scenarios, it can calculate exactly what action needs to take away. I'm just in panic mode. I don't know what to do. I've never done it before. So I'm only familiar with what I've done before, but not new things that I've never experienced. So that's interesting. But regardless, I would still say that compared to operating complex software, of course, driving a car is a much easier task, right?

8:12So that kind of tells you a little bit how complicated it is what we're trying to achieve. Yes. Well, and I think one of the things that is often underappreciated outside of hardware is all those different layers that you have to go through the different layers of abstraction. You talked about starting at the electronic layer, right? You're simulating physics down there to understand how is this going to work and function. And then you have to go multiple layers up before you even get to the logical layer, which is where many software engineers start their thinking. So as you started looking at applying machine learning to this, where did you start?

8:46Yeah, that's a very good point. So where we started is essentially, I would say, we were looking at, let's say, tedious or like high toil human tasks, things that humans have to do, but it's sort of repetitive. It's sort of like not super creative. And it's something that I think if I automate this, everybody would be happy not having to deal with it. So a few examples, and that maybe brings us in our first application that we developed, which is DSO.AI stands for Design Space Optimization. Like as an example, like let's say when you optimize your chip and you implement it, one of the important things that you have to do is you have to tune your software to meet your performance requirements, right?

9:28So every chip that gets out the door has different requirements. Some are tuned for highest performance. Some are for lowest power. Some are for a trade-off in between. So our tools, of course, optimize for all these things, But typically, users have to spend a lot of time in running our tools over and over again to tune for certain performance parameters and tradeoffs. So that's one of the earliest things that we thought about we could automate. And that's where this DSO.AI technology came out of. Essentially, rather than a human running lots of experiments and trying all kinds of options, these could be technology options, tool options, flow choices, and so on.

10:08And instead of a human doing this and just running lots of experiments and seeing what gives you good results, you can have a machine do it. And a machine is much better at it. And that's like a typical application of what I would call is reinforcement learning based AI optimization. So in this case, I have a complex workflow. I sort of intuitively know based on past experience what I should tune and how I should tune, but I don't know precisely. And plus, every chip is a little different. But if I have a machine, I can run lots of experiments. I can take sample points. I can make decisions, just like, for example, what was done for AlphaGo.

10:46You can think of it as a very large decision tree where I make choices. Do I go this way or this way? And what I want to do is I want to have a technology where I can prune this decision tree and get to a good result. I don't want to say the best result, but to a very good result relatively quickly. So that's what we're doing with DSO.AI. And that has been, I would say, a runaway success for us because it has really helped this process of tuning, performance tuning of the implementation of the chips. So to make sure I understand then, what was happening before is you had kind of a workflow that a human could go through and they could say, OK, I have all of these different knobs and levers I can turn.

11:25Let me try turning this one, run the simulation, wait for however long it takes, which is not fast. These are complex simulations. I don't know what it is exactly for you all, but I'm guessing it's hours or days. Yes, it can be weeks even. It can be weeks. Turn the knobs, run the test. Oh, that didn't get quite what I want, or that got me partway. Turn another knob, run a test. And what you've done here is you've trained a model that kind of tries to reproduce some of those same intuitions of which knobs are going to be valuable. So it can pick what to turn and run those in a loop on its own. Exactly.

11:57And on top of it, you can sort of learn from chip to chip because a lot of the chip designs out there are sort of evolutionary. They may be redesigned or maybe 30 % maybe of the logic is new on the functionality on the chip. But there's a lot of similarity. So you can then essentially build also a behavioral database where you sort of know this type of chip. I kind of know how it will respond to my knobs. and so the second time I do it when I do a next version of this chip I sort of have a starting point already which is similar to what I said a human has some intuition right but here the beauty is I essentially have a centralized brain that has all this information it's not distributed in people's heads and people may leave the company they may move to a different project and then you lose this knowledge if you have a system that captures it it becomes way more powerful.

12:50Can we dive one step deeper and talk about building this first version, dso.ai, like what types of models were you using under the covers? What does that tool chain end up looking like? So I don't know if that goes too deep for all the listeners experience, but I'll try to keep it high level. So we started focusing on what's called synthesis in place and route flow. So that is a very complex part. Essentially, that's the part where you have a description in a language, like similar to say a C language, but it's called RTL or Verilock. So you have a description of what the circuit should do. You translate this into like Boolean gates, and then you implement those Boolean gates essentially towards the transistor level on a physical layout, right?

13:34On how they're interconnected, how they talk to each other, and how the electrons move between the individual gates. That part of the process is one of the most complex ones with very long run times, And that's where you have a lot of knobs to tune. So I think because I have a lot of background in this space and I've been in this space for quite some time, I started thinking this could be a very good example of automation. So we tried various different algorithms. We settled essentially on some form of reinforcement learning, ultimately. And like I said before, we got some inspiration from AlphaGo because that was a prime example of how essentially they were able to solve a very, very complex problem without having the computational expense of it, right?

14:20I don't know if you remember the story even from the 1990s when IBM built this deep blue supercomputer to beat, and they did beat Kosporov at the time, the world champion of chess, but the system that they had would not have been scalable, right? Like it was essentially sort of not completely, but it was to a large extent brute force and computing all the possible choices. And if you think about it, the way a human operates, a human doesn't play chess by calculating all possible combinations ahead of time. A human has like cognitive abilities and he sort of knows, ah, I know this scenario and I know what to do here.

14:59Anyway, so this was sort of our inspiration. And we used this technology to optimize the workflow. And I still remember in the beginning, it was primarily about productivity, meaning I can get to the same performance that a human designer was able to do by tuning knobs. But the story was we can do the same performance, but we can do it faster. And obviously with less people with automation. The interesting thing that we found is that we were actually pretty much always also able to get better performance. And our customers, they got excited not just about the productivity, but also about the aspect of getting higher frequency on their chips or lower power.

15:41So that's sort of how we started this. And this really was an incubation within the company at a time where I would say 2017, roughly, where ML and AI was already pretty hyped up. but nobody really had any idea how to make it work in our domain so that's why i think this was a good first application for us to develop yeah well and i just to summarize a little bit of what you're saying there because i think it is a helpful mental model the process sounds a lot like a recurring theme for me like a compiler where you're you're starting from this high-level language verilog you're compiling it down gradually lowering it into logical computations then into gates then to a physical layout and this first implementation it sounds like was kind of looking at that physical layout step where you have a lot of different knobs and whistles and applying a reinforcement learning approach to kind of pattern match and say, oh, for this, I want to tune this way, that I want to tune that way.

16:34And you were able to achieve not only the speed up, but the performance. Yeah. At a high level, that is a good understanding. That's correct. Yes. So let's maybe then look a little bit at what came next. And I immediately, if I'm thinking about that layers of compiler step, I'm wondering, oh, do you look at some of those other stages as you're lowering things down? Or where did you go? Yeah, exactly. So that's sort of what we did as a next step. So the reason I would say why we developed DSO is because myself and other members of that initial team knew this space really well. But there's many other areas where you can apply it.

17:08We have verification tools where essentially you run tests and try to achieve certain coverage of your design with those tests. You have test equipment where ultimately you need test patterns to test the accuracy of your chip, make sure it actually is functionally correct. That is done on test equipment, and there's software that needs to essentially come up with the fewest amount of test pattern necessary to reach a certain coverage, because the more test patterns you have, the more time you have to spend each and every chip to test them on the test equipment. So this is what's called the test pattern generation.

17:45Then there's many other areas. there's even on the analog design side, there is this whole transistor optimization. When I move either from one node, I move a design from one node to another, and I need to come up essentially with a new circuit optimization, or if I start a new design. So these are all very good applications also for this space optimization, because they're essentially complex workflows. When I look at it as a user, to me, it's sort of a black box. Of course, like I said, you have some intuition, So it's a little bit more than a black box. But I have to tune knobs to get to some certain results.

18:20All these types of applications are really ripe for like a system that essentially sits around and then optimizes the workflow. And that's what we did. So when you hear the word synopsis.ai, that's essentially the expansion of the DSO idea to other domains. So we have now verification space optimization for verification. We have test space optimization for the test pattern space. We have analog space optimization for analog space. Now, one of the more recent things is 3D space optimization, because when you design three-dimensional chips, so traditionally chips are two-dimensional. When you do three-dimensional chips, you have a lot of design choices to make on how you design the interconnect.

19:05You have to watch out for thermal between the different dies that sit on top of each other. There's lots of, again, design choices that you have to make where you want to have a system that explores that for you rather than manually. So a few different questions this sparks for me. One is, to what extent are you able to kind of do this in isolated slices? Like are all of these different places where you're inserting an optimization automation completely decoupled from the other in the sense that you can put this in here and not worry about its impacts upstream or downstream? Yeah, that's a good question.

19:39I think the examples that I brought, like verification, digital implementation, tests, and so on, we wrap around essentially slices in there. That is correct. You can theoretically wrap it around a bigger space. The issue, I think, was that, at least at this point in time, and I think this will change, especially once we go to Gen AI. But the issue right now, the reason why this is broken up into these steps is because there is still a human that owns that piece. there's a human that says i'm the verification guy and there's another human that says i do the synthesis like the logic synthesis sort of the compile step then there's a person that does the physical implementation then there's a person that does the sign of testing these are all different engineers and they hand pieces over to each other so it's sort of like a little bit the victim of an established workflow or tool chain that was done simply because of the complexity of these steps that had to be broken up into like, hey, there's multiple people doing these pieces, kind of like an assembly line where there's people sitting there.

20:42Ultimately, I think as we reduce the complexity of each of these steps further, I can totally envision that maybe you have an optimization that goes around bigger pieces rather than those individual slices. And as you were applying this to these different slices, were you able to more or less reuse the same approaches and tool chain? Or were there places where you had to innovate a little bit further? Very good question. Yeah, there's reuse, but there's also new technology required, because not all the problem spaces are exactly the same. As an example, like in the analog design space, where I'm operating at a transistor level, the response function of the system is quite different.

21:25The nice thing there is actually that I can take more sample points, because is like usually you optimize relatively small pieces of the circuit and I can run lots of simulations really fast. So I can take a lot of sample points that sort of requires slightly different algorithms, or at least you can benefit from the fact that I can take more sample points. So yeah, there's, it's not all the same. There is reuse, but it's not a hundred percent identical that what we're using across all these different applications. Another question related to this, One of the big areas of challenge, I think, and people are running into this as AI leaks this way into more and more things is these move us from a very deterministic world of software to something that is much more statistical distributions.

22:11And so the question of how do you evaluate your models? How do you validate? How do you make sure that if you're changing the model that you're using or you're training it further, you're not breaking old use cases? What do you all do for that? That's a very good question. The interesting thing, though, is that the AI technology that we have are still companions to the underlying, like the traditional EDA software. So therefore, at the end of the day, what I'm replacing is what a human operator does. And a human operator is not without error either. So there's always checks and balances, because let's say you implement a design and let's say I relied on some ML model to tell me which knobs to tune.

22:50But in the end, there's always like a verification step that is sort of independent, where I can make sure that the result that comes out indeed is accurate. Having said that, it is true that I think it becomes a little bit maybe more difficult to debug. If there's a problem that happens, where does it come from? Is it because my statistical model and the ML model wasn't correct? It's just a different way, I would say, to debug a problem. And it's not that in a non-MLAI way, it is all that easier. because there, like I said, I have to rely on humans and humans make mistakes, possibly more mistakes than a good trained model.

23:29Yeah, I love a thing that you said there, which is I think many of the most fruitful applications of AI to date are those in which the generated output has a non-AI validator in some form or other. Yeah, exactly. That was your space. You were validating all these things. So it's a natural fit. So that makes a ton of sense. is actually one of my when people ask me what is AI good at or so I always say well in my mind the intention is to essentially automate or at least assist with the human efforts and like to give you an example let's say in our space when I go from the logic gates to the physical design one of the common steps there is it's called physical placement so let's say you have one million of logic gates that I need to place on my die.

24:20Actually, today, it's way more. It's probably 10 million. Now, for these type of things, I think it's pretty obvious that a human will never be better than some algorithm, right? And therefore, to me, this is not really an application of AI. It's a different problem. It's an optimization problem. If I want to optimize this placement, which is similar to, like, say, graph optimization. So I want to, for example, optimize the total wire length between all these cells that exist on my chip. There's no doubt that a traditional optimization algorithm is better than a human. It's like saying, I'm trying to compete with a calculator.

24:56There's no way I can ever do it. A human isn't really meant for that. A human is meant for cognitive tasks, creative things. And that's what I'm trying to automate with AI, which is why, like I said, the AI to us is a companion, but it has not replaced, And I don't really see it replacing the traditional computational algorithms, so to speak. That makes sense. You mentioned a little bit that this was not Gen AI. This was some of these more traditional models. Are you using Gen AI in any domain? Yeah, of course. I mean, who isn't? Everybody's working on Gen AI. The thing is, of course, Gen AI is, I think, for different applications.

25:35Like Gen.ai is very good at like summarizing lots of content or like taking lots of documents and you can ask questions about certain things. It's good at generating content also. So it's a different application that is really complementary to the other AI optimization that I just talked about. But it's not replacing. It's not necessarily better. It's just different. But good examples, and I think every software company or actually not just software, any company is doing that is like essentially building some type of chat bot that answers questions about our tools. We have endless documents. And because our software is highly complex, it is a lot more than user manuals.

26:14We have user manuals. We have lots of other documentation that give you guidelines on how you can use the tool to better design your chips. So building a system there that answers these questions, initially just in a static fashion, meaning like chat GPT, I ask a question, gives me an answer. I think that's an obvious thing, and that's what we have already developed. We call this the synopsis.ai copilot, but obviously it goes much beyond that. Ultimately, what we're working on is technology where I can ask questions in the context of my actual design. I don't just ask static questions. How do I run this tool?

26:53What does this command do? How do I set this up? Instead, I want to be able to have a window where I say, what's happening in my design? There's a problem. How do I fix it? Right? These are the things we're working on. And of course, agentic workflows. I mean, that's the big topic, right? Automating some of these more complex tasks where a human is involved today with reasoning ability, analyzing a particular pattern and saying, ah, I know this. I've seen this before. I'm going to do this as an action, or at least try multiple choices until it is resolved. These are all the kind of things Gen.E.I.