Can You Buy Your Way to DevSecOps Success? | Arcjet’s David Mytton

0:08Welcome to Dev Interrupted. I'm your host, Andrew Ziegler. And I'm your host, Ben Lloyd Pearson. In today's news, we're talking about a few topics. Top of mind for me is Google's new AI search mode. Another one is a new buzzword I was reading about called AX. And not like an axe that you hit someone with, but actually agent experience. There was another one. AWS formed an internal agentic AI group. That was pretty cool. And I found some sweet, sweet useless apps online. Ben, what do you want to talk about first? As much as I love useless apps, I think we should dive into the Google search. Okay.

0:53So we've been talking about the search war between AI and Google on this podcast for a few weeks now. Google search has unveiled a new AI mode that lets you ask complex questions with many different parts directly into the search bar. Before, what might have been five or 10 or maybe even many more individual queries by you, as you try to understand a question or seek something out online, you can now explore these search results in a more natural conversation-like way with Google. Yeah, it seems that Google search lives to see another day. You know, it's kind of extraordinary to think about how much our typical research process has changed in such a short amount of time.

1:39Like the way we used to acquire information just feels like a distant dream at this point. But, you know, and I think this is also a great example of how disruption from AI doesn't necessarily mean death. You know, a lot of people are concerned about whose jobs are going to get taken or what sort of industries it's going to wipe out. And I think the reality is more like Like AI is going to disrupt us, but it's probably not going to wipe out like swaths of professional jobs overnight. Like there's a lot of room to just adapt and leverage all the benefits that these tools are bringing us. What I see here is Google taking another bite back from AI and the ongoing search wars, earning back a little bit of that market share.

2:20I think we've all had our mixed experiences going to Google and getting AI results or answers in response to your queries. And so one step further towards a better version of that goal is really great. And like what you were saying, I think that, you know, there's more disruptions always on the horizon. They come from everywhere. But ultimately, what this disruption means is a higher demand for more unique experiences, because now you can go to Google and ask that really in-depth, maybe even personal question and crawl across a lot of real-time resources in a way that maybe before you were using an LLM for anyways.

2:53it's going to create a more personalized experience for everyone online which i think is going to just increase the demand for more jobs and more professionals and more technologists that come in and create these personalized experiences for people with the technology yeah and this this ties nicely into the article that i wanted to bring up today and that's this concept of a agent experience so maybe this is disrupting devx or at least that's the point of this article that i read this week. And the whole point is that we have to start focusing on agent experience, which is defined as the holistic experience that AI agents have as the user of a product or platform.

3:32So think about how everything is sort of built for humans as the end user, like the web, for example. Most websites are built for a human to visually analyze them and extract information from them. And if you're a bot that's just like scraping text off of a website, the web actually looks entirely different to you. And in that same way, AI agents are the same. The patterns that work for humans aren't always going to work for these agents, at least not in an optimal way. And this article highlights two ways that agents are getting integrated into products. The first is vertical. So that's like what we were talking about with the Google search example.

4:11And that's where you have tighter integrations of internal AI into your product. In addition to Google search, adding these new AI capabilities. It's like Google also adds like Gemini buttons to Google Docs, for example. And then the other side of that is horizontal. So making it easier for external agents to interact with your platform. So think about like bring more select external models into a platform to leverage capabilities that it has. Thinking about the agent experience for the first time is very eye-opening. It kind of shows us the way that we're going to build technology to be consumed in the future who are its future consumers.

4:48This kind of news also reminds me of the recent model context protocol and all of the development around that. And if you're not familiar, I talked with a guest this week about MCP. I mean, he told me all about how companies are turning their products and tools for LLMs into things they can use. I'm excited to bring him on the podcast in the future, but it really got me thinking about this entire experience, because this is exactly what he described, creating the scheme of the tooling for you to plug it into cursor, for you to plug it into your at-home chatbots, for you to put it into any kind of in-product or personalized experience.

5:24And so the idea that they have an experience too, and there's things that they need, it flips the entire script on how we might build our actual workflows to be used. And ultimately, I think it's time to think maybe a little bit about that agent experience alongside your developer one when you're creating resources or otherwise developing your technology. Think about who we might be using it. Yeah. So what's the next story you want to talk about, Andrew? Well, we got to talk about the useless apps at this point, right? Oh, yeah. Yeah. We've been talking about a lot of AI stuff. This one's a little more fun.

5:59Have you ever kind of wished that your phone kind of just like wouldn't let you use it at all? Maybe like at all at all. Do you ever pick it up and you're just like, I wish I wasn't picking it up as much? Yeah, at least twice a week. At least twice a week, right? And there's all sorts of ways you can fix that. Now you can put privacy locks on your phone. You can put time locks on your phone. Here's a really fun one. Someone built an app that doesn't let you open any kind of media app on your phone unless you literally verify that you've gone outside and touched grass. So you have to open the app and then walk outside and find a patch of grass and reach out with your camera facing your hand and touch the grass.

6:37You can't actually use anything until you go outside, right? Go touch some grass, like get some perspective, cool off for a second. Yeah, if you've been on Reddit any length of time, somebody has probably told you to go touch grass. i love useless apps this one though for me it's like i immediately thought like oh how could i how could i break this could i use this app and then like could i print out a picture of grass and i was probably a little more sophisticated than that but you know like i actually have a bonsai tree what if i like faked some perspective right and it thought i was outside so that was immediately where my mind went with it this was you're illustrating that for it to work for me it would have to be go touch snow like oh very true my grass is under a foot of snows.

7:20I didn't even think of that. What if you're somewhere where there is no grass? I think maybe there maybe needs to be regional variants of this app. Yeah, but you know, this is such, this is, I think, such an emblematic representation of how AI is going to change software development. Let's set aside all these engineering teams who build these robust platforms and need complex orchestration of AI agents to just think about the everyday person who doesn't know how to do all of that stuff. One of the common refrains about generative AI is like, it's pretty good at building hobby apps, prototypes, proof of concepts, etc.

7:54Whether I wanted to like, create a version of this that works for me where I can go outside and touch snow, or if it's somebody who's never built software before, but has a great idea for a simple app, it's empowering like all different types of people to create new things that they weren't capable of doing just a couple of years ago. I completely agree. We're heading for a time where anyone can spin up very personalized software on a whim that can serve a very specialized or personal niche for them. Maybe it doesn't have to work well, doesn't have to scale literally at all. We're talking about one user, maybe even for a temporary amount of time.

8:31And I think we're going to see a proliferation of these almost disposable like apps and technologies that pop up just to explore an idea or to build it on something, or in this case, maybe even to make a bit of art, because I think that AI will ultimately lower the barrier for people that are not in technology to build and create things. We saw the same exact thing happen with video games, which was a very highly technical field that then slowly became democratized by tooling to become more accessible to artists. And now you see video games as an artistic medium. I think you're going to see the same thing happen with software with applications because when you unleash it on a world of people who don't think like technologists they're going to build things in totally different ways and we're not even ready for it yet yeah i think there's a bold prediction you're hiding there that maybe you should own like will ai normalize software as art i think andrew ziegler says yes i absolutely think so i'm excited to see what kind of things come out of the future Yeah, well, I love how every topic that we talk about, even if it's not about AI, always comes back to AI.

9:35I think we've got one more AI story, Andrew. So why don't you cue that one up for us? Oh, OK, great. One more AI boomerang for you. So this one is, you know, Amazon, AWS, everyone knows it. They formed a group internally that's focused on agentic AI. This is yet another large company we've been talking about on the podcast, taking yet another big move to form their internal teams, to put together their greatest minds, to focus on how AI can transform them internally. And it goes back to Conway's Law, even, a basic principle in software development that you ship your teams, right? Your software products reflect your team structure.

10:14And so if AI is important to you and it needs to be everywhere and it needs to be parts of your product, then it's important that you put together a task force, put resources behind, a diverse group of representations of users in your company to figure out how to make this work and to get it in place. And, you know, we saw this with Goldman Sachs. We saw this with Microsoft. So here is yet another one from AWS. Yeah. Humans managing AI agents will eat the world. You know, we've heard software ate the world. We've heard open source ate software. I think humans managing AI agents really is going to be the future of this all.

10:51And I actually want to make just a completely shameless plug for the engineering intelligence newsletter over on LinkedIn. This is a new newsletter from Linear B that I've been contributing to. And in last week's edition, we shared a whole bunch of practical advice for how you can start adopting agentic AI into your software delivery lifecycle. So we have all these giants, you know, Amazon, Microsoft, Google, Meta, everyone out there is adopting agentic AI. And there's no reason that everyone that's listening to this right now shouldn't also be doing that. So I'm definitely going to recommend everyone go follow Linear B on LinkedIn, subscribe to the Engineering Intelligence newsletter, a ton of great news happening over there.

11:34Yeah. So Andrew, tell us, who do we have on the show this week? Oh, I'm really excited for this guest. So after the break, we're bringing David Mitten onto the pod. He's the CEO of ArcGET, a security SDK for developers, and he's the founder of Console.dev, which is a weekly newsletter where he focuses on a curated selection of interesting developer tools. David has a really discerning eye for what's good and what's not, what you should use and what you should try. He tries everything so you don't have to, and his experiences go really far. He has a finger right on what makes for a delightful developer experience, and I'm really excited to share our conversation with y 'all.

12:12So stick around for the discussion where we talk about the realities of shift to left in security and the status quo of DevSecOps.

12:24Are you struggling to explain developer experience to non-technical leadership? Join Linear B's upcoming workshop and learn how to translate DevX into language the business cares about. We'll show you how to present data on developer productivity, AI performance, and engineering health in ways that drive alignment and investment. Plus, you'll get an early access to our CTO board deck template, making it easy to connect engineering metrics to outcomes like faster time to market and cost savings. The link to sign up is in the show notes. We hope to see you there. Hey, I'm delighted to be joined by David Mitten, CEO at ArcJet and the founder of the console.dev weekly newsletter.

13:04David, thank you for joining us today. Thanks a lot, Andrew. You're a seasoned developer and founder and an angel investor as well. And we brought you on the show to chat about your approach to shift left within DevSecOps, which is something that you describe as largely buzzwords that haven't achieved their intended impact. And I really want to start there. So perhaps you can set the stage for us about why the shift left approach is applied to DevSecOps and what are some of the issues with it? It's really modeled off the very successful DevOps philosophy. and this takes us quite a long way back in history to when development was entirely about just writing code and then you would hand it over to somebody else to deal with in very old days that was physically shipping it and so ops wasn't really a thing but as services became more popular and we needed to access things that were constantly updated then it became well how do we run the software that we write and traditionally you would have sys admins and then later on that brought into different types of operations but essentially it was a separate team that would deal with deploying and updating and making sure the the infrastructure the servers and the network was all running smoothly as we moved to more of a cloud environment then the physical side of it kind of disappeared obviously still there but for most people using the cloud abstracts the compute the storage the data center those kind of things and so as more software became involved in the operation side of things like software defined networking the actual virtualization of the system it became a question of how do we run this entire environment and separating code that really if you write code what's the point if it's never run and whose responsibility is that and then when something breaks, who should fix it?

15:04Operations teams didn't write the code, so why should they fix it? Or even could they fix it? Because they don't know what's going on. And so this idea of DevOps just meant that developers took more responsibility for running the code that they wrote. And that became testing it and setting up CI, CD as you get closer to production, deploying it in the way it's actually running, dealing with monitoring and any instance that happen and that was pretty successful because it's quite easy for developers to understand well why do they need to run their software it's because what's why are they writing in the first place it's to deliver a service or to provide something to users and so it brings them a lot closer to that and you can replicate the production environment very closely on your laptop or in a staging environment so it's a lot easier to understand how to actually set it up and test it and make sure it's all working.

15:56And so DevOps kind of merged into DevSecOps with this idea of, well, if we're getting developers to run the code that they write and take responsibility for it, then why can't they also do the same thing with security and take what the security team had typically done and let developers take responsibility for it? That was kind of the idea. Does that make sense? It does. It does make sense. And so based upon the story you've told, it's really a growing amount of abstraction and complexity away from being maybe a more physical-based computing solution into something that's abstracted in the cloud.

16:33And each step on that process, the developers get closer and closer to where things are released and things are secured and tested. Is that kind of what you're describing? Yeah, that's right. It's giving developers more responsibility so that they can understand everything that goes into actually running the code that they write. And so this then led into the specialism of site reliability engineering, which was popularized by Google and a few other, the larger tech companies. Because if developers are running their software, what else is there to do? Well, there's quite a lot to do. And there's a lot of specialist knowledge involved with large scale systems, making sure they're reliable and that when developers are building, they don't have to understand the full details and can build on top of a platform.

17:19That's kind of what you're doing with it. the cloud where you're building on top of Amazon or Google or Microsoft's platform. They abstract out all the data center. And as a user, you just have to think about the particular primitive that you're using, the compute or the database or whatever service it is. But underlying that, there's still a lot of operations work. It's just you're paying someone else to do it. And so the idea with DevOps and SRE was that developers can do a lot of things, but there are still things that are probably not a good use of their time, or there are specialists that can help build our platform for larger teams.

17:52And so SRE became this specialism where they would operate the internals and the underlying systems. And then they would give tools to the developers to allow them to autonomously deploy onto those systems. And again, that's worked really well. SRE come in and maybe advise on the design of a system, and then the developers will use the platform and the tools that they've created. and ideally SRE are not involved in particular projects, but they are responsible for the whole system, the system as a whole. The idea with DevSecOps is basically the same thing. Developers should take more responsibility for the security of their software and then when there's a problem, inevitably they're going to have to fix it.

18:34But perhaps you could have a specialist security team who would give them tools to make that security job a lot easier. Unfortunately, it hasn't really worked out that way. And why hasn't it worked out that way in execution? I think it's down to the incentives. So for a software developer, their job is to build things, either because they're building an internal system and they have internal customers who need something, or they're externally providing a service, product, service to users. And so you've actually customers paying you to build things. And so that's the developer's job, build something new, fix bugs, and deliver value to customers, essentially just building.

19:13The incentives for security people are very different. They are either there to break something because they're doing pen testing or looking for vulnerabilities, or they're there to mitigate risk, which generally means stopping things from happening. And that is basically the opposite of what developers are responsible for. So even before we get into any of the details of specific things that might have failed with DevSecOps, the incentives just don't align between the security team and the development team. and they have to work together in many cases, which can be a challenge. There's often a lot of friction there, but this fundamental mismatch in incentives is the big problem because everything comes down to incentives, really.

19:54Right, so the devs are there to build stuff and to make things and security there. And sometimes in the devs' mind is there to stop things or to halt or slow things down. And then the security team only wants things that are secure, obviously, to go through. So you end up in this state of tension where both of their incentives of what they want out of the engineering process are different. Is that because of how DevSecOps has shifted the responsibilities more into developers? Or is that a response to that tension that is felt between security and dev professionals? That was a response because ultimately all vulnerabilities come down to some error or problem in the software that someone has written or the processes around it.

20:37And so you always have to involve developers in the security process. There's no getting away from that. But because the incentives were so different, developers are incentivized to build things. They're on deadlines. They've got to deliver customer value. And security is often a distraction from that. Now, it's a distraction for very good reasons. It's just it is counter to that incentive or the real goals of developers. and I think that's why it's become such a challenge to encourage developers to think about security from the beginning because right if you're in a startup what's the point of securing something if you have no users and as your company gets larger you really only think about security either when you're required to from a regulation perspective or when your customers start asking you for details about your system and that's when the incentives start to align again because there's a customer asking you for something so you should implement it otherwise it's really down to the developer to put the best efforts in or to do things because they think it's the right thing to do and we know from human behavior that probably people try and do that but maybe you don't do that all the time exactly and so if this state of tension is just innate to those roles what can developers do i mean security professionals as well to bridge that gap and maybe foster that more security conscious culture that allows them to exist in more harmony?

22:03I think this is the real challenge and it's developers are not thinking about security problems and this has given them the reputation of not caring about security. And I think that was actually just, it's the outcome. It's what happens as a result of that thinking, but it's not a deliberate thing. In most cases, it's just, in some cases it's a lack of knowledge, but that's not an excuse really companies run education programs and training and it's all really boring and so that's the that's one of the challenges but it's really about developers are trying to solve a problem they're not thinking about security because security generally doesn't help them solve that problem and so this means that they might be dealing with api abuse or sign-up form spam or fraudulent credit card transactions and those are problems that the developer thinks about.

22:55Now from a security mindset, the solution might be, well, we'll buy a bot detection product or we'll build rate limiting or use one of the credit card fraud services. And those are security solutions. But the developer's thinking in a completely different mindset. They're not thinking in terms of a security product they should buy or a particular acronym that they need to understand and implement in their system. They're thinking, loads of people are using my API and it's costing me loads of money, or we're getting loads of signups and they're all spam and they're not helping our service, or we're hurting our reputation with our credit card processor because we're getting loads of chargebacks.

23:34Those are the problems that developers are trying to solve and security needs to provide the security team. The security industry needs to provide solutions to developer problems, not try and sell them another product. Right. So this brings us back to the core of our topic of shifting left and applying that to DevSecOps. And some of what you're touching on now is I think really salient here about part of that is putting the right kind of security tools into developers' hands earlier in the process makes it more part of their development journey as they're creating the actual software. And it allows them to have earlier conversations with security about what they're building as well.

24:11And so what are some emerging trends or tools from your perspective that maybe are delivering on this or making steps to bringing this more into harmony? The buzzword right now is secure by design. And this has been pushed in particular by governments. The US government has a lot of effort going behind that, and the EU has recently passed some laws around defaults. Secure by design is a great idea, and the challenge is the actual implementation. What does it mean? On the consumer side of things, it means things like when you buy a new Wi-Fi router, it doesn't have a default password, and it can be updated for the next five to ten years, so when there are vulnerabilities discovered, they can be updated automatically.

24:58That's quite easy to build in security by design, or at least you know how to do it. You can put on a checklist. It's a lot more challenging to do that with the underlying software engineering approach. And how do we encourage developers to build security into their products when the incentive isn't there? Well, one answer is just change incentives by regulating and force developers to think about this. And there's a question around liability and how the industry works when there's a big vulnerability found whose responsibility is it is there a monetary value attached to that that can be a good way to change behavior but as soon as you get regulation involved then it becomes well who's got the most money to do the lobbying how do you balance the different requirements and then regulation increases barriers to entry which has been the great thing about software like you literally need no qualifications you can just write code and you can put something online and People can start using it, which is great for innovation, perhaps not so good for safety and security.

26:00So I think that's the big challenge around security by design in terms of how you actually implement it and enforce it. And ideally, we don't have the government forcing people to do things. Maybe that's where it ends up if it doesn't change. But I think the fundamental is around behavior and how developers think about this. And it's very difficult to change human behavior. And so you have to change the underlying system, right? So the developers, people, whatever the system is that you're thinking about, the user of that system gets the benefit for free. They don't have to think about it. So by using that particular system, they automatically get the benefits because the operator of the system or the designer of that system has made that change on your behalf.

26:44That is the responsibility then of the developers of infrastructure and the tooling to encourage developers to do this or to use these systems that give them security by default. So if there exists a class of tools that allow developers to implement security practices earlier in their process, I'm curious from your perspective, because you have so much experience reviewing hundreds of tools for your newsletter, console.dev, which we're going to talk about a little bit more in just a moment. And when you approach this idea to evaluating these tools or thinking about how can we put better tools in developers' hands earlier in the process, what are some of the things that you look for as someone who's looked at a lot of tools and it has become very discerning for what we put in developers' hands?

27:33Yeah, so for console.dev, I've reviewed probably 300 or so every year for the last four or five years now. So I've played with a lot of tools. We're putting two or three every week. So those are just the ones that actually get through. and the first thing I think about is would I use this and having been in the industry for always 20 years I might kind of have a it's my opinion on this but then there are some specific things so the most common problem I see with the dev tool is the documentation has some flaws often the quick start guide is fine and it works sometimes it doesn't and then that's essentially it's off the list straight away because if the quick start doesn't work then no one's gonna be able to use it but the common failure scenario is the you do the quick start the tool does what is advertised it's got a load of other features but you have no idea how to use them because none of them are documented that's a real problem developers don't necessarily like writing documentation i spend a lot of time at artjet writing documentation for our products for our security product just because i know that that is the like whenever i'm trying a new dev tool the first thing I'll do is go to the docs and so that needs to be part of the product we consider the docs part of the product the other failure scenario is once you've done that quick start and it's working how easy is it to adapt the product to the complexities of whatever it is you're building this is a failure of breaking out of the happy path and a lack of flexibility in the design of the product or a lack of documentation explaining how to use all of the options and the configurations because most of it like you see all these templates for like a chat app or how to implement authentication really straightforward those are important things and building a chat app is really fun but that's not what most apps are and you've got like all sorts of different requirements that customers are asking for and you're building all these different variants and that's often where dev tools break is because they don't work in these various scenarios That makes sense.

29:38You're touching on something that is very close to my heart, especially when you talk about documentation and evaluating new tools. As a developer advocate, I completely understand the mindset of when you go to a new tool and you're trying it out for the first time. There's a few gut checks that you do. You go to the repo and you see when was the last commit. You go to the documentation and you look at the quick start guide and is it a chat app? And if it is, does it work when you read it from top to bottom? So those types of approaches are, I think it's very delightful to find that they're universal when other people evaluate software as well.

30:11And so when you go down this path of evaluating software and you look at the docs and it's about putting great examples in developers' hands so they can envision the tool. And it's about the tool also being flexible for what they're looking to implement. Do you see these tools oftentimes as starting grounds or do you see them as holistic solutions? Are they there to inspire or maybe guide a path for the developer to start with this tool and then maybe kind of build it into something else? Or do you see it as like a one-stop shop developers need to be looking for like that Swiss army knife? This is often the distinction between a really great open source tool that someone's built has become popular and a product from a company.