Speed is the moat | AMD’s Anush Elangovan

0:05Welcome to Dev Interrupted. I'm your host, Andrew Ziegler. And I'm your host, Ben Lloyd Pearson. This week, I'm joined by Anoush Alangavan, VP of AI Software at AMD. He joins the show to discuss why the dominant walled garden strategy for AI compute isn't the only path to innovation. Instead, AMD is betting on its open source software stack, Rock 'em, with Anoush explaining his philosophy that speed is the mode. Arguing that TruSpeed isn't just about hardware benchmarks, but building the muscle to run the race for the long time. Something he believes only an open source ecosystem can sustain. But first, let's discuss some of the news that caught our attention this week.

0:50Got a fun little lineup today, Ben. What do you want to talk about first? Our producer Adam has picked a really wonderful story about open source and FFmpeg and Google. So let's dive into that one first. Yes. So this article on the News Stack by Stephen Von Nichols is about FFMPEG's open source library being in a pinch with Google. To sum it up, basically, FFMPEG developers are demanding that Google demand financial support or stop sending security vulnerabilities in the excess amounts that they do. It really what it highlights is the strain that these open source contributors that often work, you know, with no compensation on an entirely volunteer basis, the work that they do to sustain large enterprises and their software stack.

1:34So this confrontation, it was, you know, pretty public. FFmpeg powers every kind of video you could possibly imagine on the web is an incredibly vital and integral open source library that powers the world that we take for granted. And a lot of businesses and products are built upon it, including many of Google's own. So there's a real tension here because we're talking about open source contributors who own their code, own their process, own their stack. They're not partners, right, with Google. They are independent operators. So where do the incentives come from? It's a really fascinating play that we constantly see between large businesses and open source.

2:12This isn't the first time we've seen it. You know, I think, Ben, there were some things that caught your attention in this article as well. Yeah, well, in particular, you know, it hits home for me because I know a lot of the people in this story. So it's pretty interesting. But in particular, the article references a tweet from Mark Atwood, who used to work at AWS. He's a friend of mine. And this tweet said, they're not a vendor. There is no NDA. We have no leverage. If your VP has refused to fund them, they could kill three major product lines tomorrow. Like that is how important FFMPEG is to many of these organizations.

2:49and you know and i love that while i was reading some of the threads on x around this like i love to see that someone already beat me to making the an ffmpeg version of the classic xk cd comic of about like the entire multimedia infrastructure of the world depending on like the hobbyists who maintain ffmpeg it's pretty crazy yeah and i mean for people who don't know what it is i mean basically if you do anything that involves displaying video or listening to sound there's a very high chance that FFmpeg is a critical component of that pipeline. It's an incredible tool, has a really special place in my heart.

3:26You know, I was involved with it back about a decade ago when I was at Samsung, and I worked with many of the maintainers on the FFmpeg project. And here's the thing about open source and enterprise. There's a right way and there's a wrong way to interact with these communities. The right way is doing things like providing financial incentives or hiring someone to be your representative in the community who actively improves things on your behalf. Like that was the approach we took at Samsung. And it was how the company ensured that they had adequate representation in these super critical projects like FFMPEG.

4:01And then of course, there's the wrong way where you just dump all of your concerns as like a high priority item on a team of volunteers and hobbyists. So yeah, I think this is kind of a classic story yeah you know i really like how this article called out the cve slop phenomenon that's a lot of open source projects and this is kind of like a parallel to the pr slop problem that happened uh you know once open source uh contributors were getting being slammed with prs from ai tools and ai agents and people using ai for the first time especially in uh settings like hacktoberfest which had to drastically change how its rules work in order to accommodate this post-AI developer world.

4:40The same thing happens in cybersecurity with CVEs, you know, as they're discovered en masse by AI systems, and then also trying to find solutions for them to resolve bug bounties, right? There's a real financial loop involved with security on CVEs. So I really like how this article called that out as a phenomenon that's also burdening systems like FFMPEG. And I know what you're thinking at this point like okay it's an open source project it's really important why why don't we get some some folks in there to contribute we're talking about something that's built in assembly this is not an easy or intuitive system to touch and it's a system upon which many things build on so it's incredibly hard to address cves and security problems that come across the line for a project that sits this low on the system i just want to emphasize that it's like a really unique constraint of this particular one.

5:34And definitely an interesting phenomenon. Yeah, and I think it's just a friendly reminder to all of our listeners out there. Like if you're one of these organizations who depends on a project like FFmpeg or any other of the many open source communities out there who build a lot of critical infrastructure, now's a good time to just evaluate your open source dependencies and cover your risk a little bit. You know, a little bit can go a long way when it comes to supporting open source communities financially. But yeah, it's always fun to just like get to discuss stories that involve a bunch of friends of mine.

6:05Yeah, I love this one. So what do we have next, Andrew? Okay, so this article came across our desk. It's from ToxX, and it's about how AI in its chain of thought reasoning is more like a theater than it is an actual ledger of how the AI is thinking. This types into some anthropic research that found that AI models, they hide their true reasoning a large proportion of the time. And this really influences their behavior. And this article dives into some of the observations that kind of come downstream of this. One thing in particular that it notes is when it used Claude's free opus, that it showed completely different behavior based upon whether it thought its outputs were being used for training.

6:47This then ultimately kind of breaks all of the safety mechanisms in place on the model. You're effectively putting it in debug mode. That way you can actually test what's inside of it before its constraints and guardrails are happening. This is an interesting observation, but it also shows how along the way, the chain of thought that LLMs produce in order to show you the results they want are influenced by this behavior. And in many cases, the actions that they're describing that they do don't match up with the reasoning that's happening underneath. There are some really interesting debug kind of philosophies that go into exploring this, many of it involving tracing even uh simple props prompts take a like a large amount of time by by folks to trace this level of reasoning and chain of thought on the model but i thought it was pretty fascinating i've actually experienced a lot of this like lying chain of thought theater when i use tools especially like in cursor where you can like really peer in on like how it's thinking every step i'm like that's not what you just did or what you think you're doing so i i kind of have some experiences backing this up what do you think of this one ben yeah this was a this article was a good excuse for me to go read this research from Anthropic because they do a lot of great work.

7:54And I think I can explain like sort of what's happening here. So I want you to imagine a scenario. You're about to take a test in a classroom and someone stands up in front of the classroom and says, the answer to number to question three is C. They just say it in front of the entire class. And I can imagine there's probably like three primary ways that someone might respond to this activity. The first group or one group of people, they might think, oh, wow, I have no idea what the answer might be because I didn't study. So I'm just going to put down C for question three without even reading the question or even the possible answers because I'm not going to know anyways.

8:29And then there's another group that thinks, well, I studied all of this a lot. I know the content extremely well. I don't need someone to tell me what the answer is. I'm just going to ignore it because I'll make my own judgment when I get to that question about which answer is correct. And then there's another group that's probably somewhere in between, you know, they might think, wow, this might help me, but you know, I know a little bit about this. So I'm just going to make the determination when I get to there to see if what they just said aligns with my expectation of what the correct answer is.

8:58And I think in the case, like if these were people doing this experiment, I think in the case of the last two groups, if you ask them where they got their answer, they would probably say it was completely their own decision. And they actually didn't even consider the answer that was blurted out in front of the class. So it could even just not even be like a malicious activity, but just a natural way of reasoning that our minds and LLMs sort of operate. You know, because LLMs, like they're designed to basically make decisions in a very similar pattern that humans do. So it makes sense to me that they probably also would make a decision like that in a similar way.

9:32But things get tricky when you start talking about like perverse reward structures then, where you're actually like actively teaching the LLM to do the wrong thing. And where all the technology is at this point in late 2025, I've become a big fan of using AI within workflows where you have super tight controls over the inputs and a way to validate the output in an efficient and reliable way. So this kind of puts the AI in like more of a manageable box that you can improve upon. You can fix it when things go wrong. And more importantly, you can diagnose its current condition. So from a product standpoint, when you're thinking about AI, what this information really comes down to is if you're in a situation where you're exposing the users with the ability to prompt an AI as a part of your product, that's where this type of thing can really start to become a risk.

10:27And yeah, this article, it's a bit hyperbolic, but I think it is just a concept we should all be aware of as we're building AI workflows and AI products. I strongly resonate with the plus one to everything by evals and evaluating what goes in and what goes out. I think that ultimately that helps validate the rigor of whatever you build with these systems, despite the chain of thought theater it decides to do in between. So really fascinating story. I think we're going to keep following that one for sure. You want to dive into our next one? Yeah, yeah, let's move on. All right, so we have a really nice article from Phil Eaton about his journey as an engineer going from a web developer to a database developer in the span of a decade.

11:05And it really dives into some of the journey, as he described, being someone who works in a more front-end environment and then starting to dabble with databases slowly over time, but also about being able to move back into a startup environment and contributing code and really getting into the mix of building stuff every day. I think that's why people ultimately, you know, they want to be engineers. They want to do cool stuff that people use every day and it makes their lives better. So this story really resonated with me. It's about an engineer finding his path to that journey for himself, ultimately landing at databases, which I got to say, really cool nerd out subject.

11:43I think databases are pretty cool too. What do you think of this one, Ben? Yeah, you know, stories like this are always just kind of cool to see, Like somebody who's done it, you know, somebody who's been around for a while that built a career, you know, for anyone who out there, he might be looking to map out their own career journey as well. But the thing that I kept thinking about is like, but what if what if that story was starting today in an age with like AI being so widely available? Like how how different would it be, you know, 10, 15 years from now? Somebody like just getting into, you know, maybe it's web development or something similar in terms of complexity.

12:19And, you know, how much more rapidly they could probably learn a lot of the things that Phil outlines in this article. So, yeah, I would say my biggest reaction is, you know, mostly just makes me excited for what career journeys are going to look like 10 years from now, you know? Yeah, I totally agree. Yeah, so I wanted to include this next one titled Everything Worth Keeping Happens Between the Prompts. So what are we covering here, Andrew? Yes, this one's from Mia Karaki and Stanley Lingworth. And it talks about how this common phenomenon that many folks experience when they use AI to try to generate ideas or content to share, since they don't take the time to reflect upon what they really want, they don't think about their own perspectives and their own experiences as an individual.

12:59Nothing gets infused into that creation process that ultimately makes it something worth keeping. This is an article for anyone who's felt like their AI-generated content lacked a voice or sounded like anybody else, or if they've been shown content and asked, you know, is this you? If you're someone who's posting a lot in this way. And it really highlights how you escape out of this cycle of not good content and use AI as a multiplier for your own voice in a way that no one else can. Ultimately, it sets you up with some really valuable steps for evaluating how you actually go into any kind of content or research-based creation process with an LLM really breaks down into being a series of questions and answers where you take different roles, you play different hats, and you play your own biggest critic along the way.

13:51And what this ultimately does is it highlights the constraints that make you you and all of your unique perspectives. And once you get that figured out in the perspective of what you're trying to achieve, crystallizing some really great content that could only come from you and your voice becomes really easy. And that's what this article aims to unlock. I thought there were a lot of really interesting maxims in here to help folks really sort their AI-generated content and make it great. Lots of details that you can obsess over, for sure. So, Ben, what do you think of this article about how folks work with prompts?

14:27Yeah, well, you know, this article is not about software engineering, But, you know, GPTs are transforming like the fundamental components of practically all of knowledge work now. And I really think we're entering like this era of like agentic knowledge work, like just generally speaking, whether it's software development or content production. So I think, you know, just for us, it's a good idea for us to occasionally branch out to more generalized AI knowledge when we find good content like this. And I think what this really article does a great job at is demonstrating just a practical example of how to apply the Socratic method to using GPTs, like where you're effectively forcing a back and forth dialogue where you arrive at the best answer collectively with the GPT that you're using.

15:11And this is a really great practice when you have those really big challenges that have a ton of moving parts. And the article points out that when you over rely on GPTs, you may start to find that your work lacks tacit knowledge. So this is informal, unspoken, experiential insights that reside within an individual's minds. And they represent a valuable form of knowledge that's actually really challenging to document and transfer. But it's a thing that GPTs can actually extract from you quite effectively. And, you know, keeping with my theme of keeping GPTs like tightly constrained to more predictable outcomes, this also creates a need for tighter human in the loop reviews to keep progress aligned effectively.

15:56And, you know, just as a side note, like a huge benefit of GPT platforms is like being able to branch out from your like normal expertise. So, you know, if you're listening to this and you've wondered, like, I have things that I think I could write about and discuss, but, you know, maybe you don't think you're as great of a writer or you've always just wanted to be better at it. And there's actually a lot of great tips in this article about how to use GPTs to produce content in a way that is unique to your voice and speaks to whatever your expertise is. So, you know, like anything else, there's a right way and a wrong way to use GPTs for this stuff, but it's never been easier than now to create content, you know, even if you don't have the expertise on how to make that happen.

16:37So if you're out there and if you're inspired to be a content creator and maybe you've struggled with it in the past, this might be a good resource to help you get started. And this is also your wake-up call if you've not gone into your favorite LLM assistant and customized it for who you are, your perspective, where you work, what your job is, what's important to you. You are missing out on a lot of gains that could be baked into everything. So this is your wake-up call. Make sure that you just also do some basic janitorial work every once in a while, the tools you use. Make sure they're lined up for what you're actually looking to achieve.

17:07all right let's cover one of my favorite companies of all time valve what do we have here andrew all right yes valve long time uh dear to my heart as well we're talking about uh a new system that they are dropping it's called the steam machine it's like a actually like a cube basically uh and it allows anyone that owns it to then play basically any game on steam it's jam-packed with this crazy powerful you know stack that's going to have all the processing power all the GPU, all the RAM that you need to take full advantage of Steam's full library of games. So this is a really fascinating play from Valve.

17:45It's a massive further leap into hardware. They've already made several steps into hardware with other devices like the Steam Deck and their goggles before this. So a really fascinating play here. Ben, what do you think of Steam dropping its own machine? I think that's pretty fascinating. I think I'm going to get one. What about you? Oh yeah, I'm already saving up my money for that. Valve is single-handedly the reason that I stopped pirating video games over a decade ago. So I really just love whenever they drop new stuff like this. But there's this long running joke in the Linux community that like insert year here is the year of the Linux desktop, like 2025, the year of the Linux desktop.

18:24We're still all waiting for that to come, but it's basically been every year since 2000. People have been predicting that. And while we're waiting for that to happen, And, you know, Linux is practically everywhere else in your life. And 2026 may be the year that it takes over your entertainment center at the very least. Absolutely. Yeah. So, I mean, Valve is just an extraordinary engineering company. And, you know, the fact that just a few hundred employees was able to create hardware of this quality, which, you know, it's set to come out sometime next year. It's just an astounding achievement.

18:57But, yeah, I'm definitely going to be watching for more news on these. Oh, yeah. No, we're going to stay subscribed. We'll keep you all in the loop if anything fascinating happens as well. You know, by the way, the GPU in this thing is an AMD. This is pretty cool considering that we have Anoush coming up here on the show in just a minute talking about Rockum, the open source ecosystem that's making all this stuff possible. Maybe once I get my hands on this, I can see even what the Rockum ecosystem could do directly on this GPU. So a really cool roundup of stories today. It was fun to break it down.

19:27Stick around.

19:52them.

19:58leading with data. Visit LinearB.io to learn more and unlock the next chapter of AI productivity.

20:08Joining me is Anoush Alangavan, VP of AI Software at AMD. And when people talk about AI compute, the conversation often stops at hardware specs, but it's more than just physical chips that win the game. It's also the software ecosystems supporting them. The prevailing strategy in the industry has been to build something like a walled garden, you know, something closed, proprietary, locks developers in. But AMD is betting on an entirely different play, open source acceleration. And with Rockum, their open source AI software stack, AMD is building not just hardware parity, but an innovation flywheel that's powered by the community with interoperability and the freedom to scale without all of that pesky lock-in.

20:56And in this world, speed is your moat. And how fast you can innovate while your platform remains open, flexible, and standardized across all of its applications, that's what we're going to explore today. So Anoush, I'm really excited to have you here. Welcome to Dev Interrupted. Thanks for having me. Super excited to chat about it. Amazing. Well, let's go ahead and dive right in with kind of what I laid it out with in the beginning, the idea of the moat and it being about speed. I want to unpack that a bit because that came from you when you and I first spoke. And I want to know, how do you define speed inside of AMD beyond just things like hardware benchmarks?

21:36Yeah, that's a very good question. So when we typically talk about speed, everyone's like, hey, hardware benchmark specs, right? Like memory bandwidth or flops. And that is one important part of it. AMD does very well with that. we do have a very good history of executing on that axis. But when I say speed is the moat, it is about how we prepare, how we build the muscle to run the race for a long time and run it fast. And it is not about a single point in time that you've beat some benchmark and you declare victory. It's about building the ability to consistently develop and deliver both hardware and software innovation at scale and do it fast, right?

22:27Like, you know, we're increasingly getting to a point where models come out and they're, you know, a year or two ago, it was like, hey, they work on AMD on day zero, which is great. But now they are performant on AMD the day it releases, right? So what does it take to prefetch where the industry is going and be prepared to intercept at that point is what, you know, I refer to as, you know, the speed factor in creating this moat, right? And the moat is just shed all things that hold you back and run as fast as you can. Because the pace of innovation that is being seen in AI industries is just amazing.

23:10And it's transformational at how you generate electricity. It's transformational at how you build data centers. It's transformational at how you deploy compute networking. It's transformational at what kind of use cases you use AI for. And for that, you need to be prepared to see what comes tomorrow and be prepared to run the race tomorrow. Yeah, it's a really great perspective because it highlights that it's not just like a checkpoint that you run through. I like how you called out, it's not just hitting that benchmark or being the best in class at that moment and that snapshot. It's about having the throughput and about having that dedication to the idea and continuing to deliver on it.

23:52It's not just crossing the threshold, but it's also being the engine. And that's what protects a business. That is the moat, because the moat is that innovation layer. the faster and more future forward that you can work and think, the better. We talk a lot about future forward work styles. What are the things I could be doing right now today that are going to be way more useful tomorrow? Let's abandon those workflows that are older. And that translates into an advantage when you work that way. What kind of things have you learned working with across all spectrums of people who would use Rock 'em?

24:31You have the developers, but then you also have the enterprises and you have this large span of adoptees. So what does that look like that you learn? Yeah, so the way I look at it is there are going to be pockets of different cadences. So people who are deploying in enterprises, for example, the validation and how long it takes for them to deploy an LLM that's secure, it's with guardrails, et cetera, maybe longer. but you still have to go through the process and you have to be prepared to like walk that walk to deploy an enterprises that doesn't mean it's not fast that's as fast as you can do for that industry right and if you are deploying ai in healthcare right it's got its own uh cycle but in each one of these you want to see how like go down to the essence of what is it that you actually have to do.

25:23And, you know, I like how you framed it. It's like, it's, you shed your prior assumptions of how things are done, right. And, and you kind of build up from a first principles approach to say, this is how I could use AI to unlock whatever I'm doing. And, and some of it, you know, it's good to really step back and look at, you know, just question every part of it, right? Like right now Now you're getting ChatGPT and Gemini competing for like math olympiads and college reasoning tests. Right. And those are like that. That is amazing and increasingly like complex tasks that they are trying to do. But there may also be like more mundane things that AI could could get applied to.