The Real Measure of Success in Platform Engineering | MassDriver CEO Cory O'Daniel

0:00If you would have talked to me like two years ago, I was like, I was sunshine, rainbows, kittens, all the good shit on platform engineering. because I saw an opportunity there. I saw a word that was getting popularized, books are getting written, blog posts are getting written. And I was like, this is the moment where we can take this word and we can market it correctly within our orgs to get the buy-in and maybe actually use that to actually get the f***ing work done. But what's happening is we're just, it's rebranding very similarly to ops to DevOps, DevOps to SRE, SRE to platform engineers.

0:30So you meet some teams where it's like, hey, we're doing platform engineering. We manage all the infrastructure in AWS. It's like, oh, how do you do that? And they're like, we're not software developers, so we just click around the AWS console. Oh, wow. So it's really exactly the same as it was 10 years ago. It's just a different word, right? Tell us if this problem sounds familiar to you. You want to use engineering metrics to improve your work. But every time you try, you realize there's a gap between the metrics world and your team's reality. And even with all the frameworks that already exist, is there enough context to make an informed decision about what metrics to focus on?

1:03That's why Linear B is teaming up with Refactoring. Together, we're creating a comprehensive industry report about how engineering teams use data and metrics to improve their practices. But we need your help. This week, we're launching a community survey to collect real-world stories that will help us build a practical playbook for tech leaders everywhere. The best stories will be quoted and published with the final survey results. To share your story and get a chance to help engineering leaders everywhere, visit refactoring.fm or head to the link in the show notes. Hey, everyone. Welcome back to another episode of Dev Interrupted.

1:42I'm your host, Dan Lines, Linear B co-founder and COO. And today I'm joined by Corey O'Daniel, CEO and co-founder at MassDriver. With over 20 years of experience in DevTools, SRE, PlatformEng, super excited to have you on the show today. Corey, welcome. Yeah, thanks for having me. You wrote this, I was reading it before coming out on the pod, you wrote this killer article. I was actually late to the pod because I started to click around to the different links. And of course, you know, we've talked with a lot of different, you know, pros and DevOps and platform eng and all of that. And I know you're probably doing it on purpose.

2:27You wrote an article called DevOps is bullshit. So, you know, that's me. That's you. I'm sure we'll include it. Where has DevOps gone wrong? You know, how are we failing on this or are we failing to meet business needs? Like what is your like high level take on all this? Yeah. So I think one of the key things, and depending on how far you get into the article, I think you realize this, but I'm on your side. If you've got DevOps in your title, I'm on your side. If you know that DevOps shouldn't be a title, I'm also on your side. Yeah. But the way we've gone about it for the past, what is it, 15, 16 years or so since the term was coined?

3:06I think so. We've gone about it in a pretty bullshitty way.

3:15over the past few years, like looking at like stack overflow survey, looking at number of breaches that we have, like the amount of operations experts or expertise that we have as like a community is going down year over year because we're making so many developers, you know, out of bootcamps, out of colleges and whatnot now. And we're not teaching them the cloud and operations and whatnot. Like it's just kind of, hey, you'll get there eventually. You'll learn it in prod at some point in time. Like you don't come into it knowing it like you do. Hey, I came in from a bootcamp. I know Rails. I came in from a bootcamp.

3:44I know React. I came in from a university. I know C. Well, this thing that is extremely important to how we integrate software, work with other people, we're just winging it when we get there, right? And so that's what the article is really about. It's like, we've been a bit bullshitty about the way that we've approached it. And what you see a lot is people are like, well, you're just not culturing it right. And it's just like, well, yeah, most people aren't culturing it right. You're right. But most organizations don't have the resources to culture it right. And so this whole idea of like, hey, everybody should do DevOps.

4:16We're doing DevOps. You're doing it wrong. You're holding it wrong. Like, I think it just kind of undermines all this doing the right work. And I think we need to figure out a way of getting that experience into developers' hands earlier. Yeah, no, actually, that's a good take. You were saying, hey, you know, so first of all, there's more developers in the world. Everyone's a developer now type of thing. There's boot camps, great career, all of that. very difficult to simulate what it's actually like to work at a company, release code to prod, what it actually takes to do so. I don't know if we'll ever be able to do that in a bootcamp.

4:54Maybe. Right. So I think you're right that most people coming into their first few roles, like all this stuff is new. It's not taught. And I also remember when dev, maybe it's even like, Is it like 20 years ago now? Maybe it's like 15. When DevOps, that terminology first came out, I remember some teams just being like, yeah, we're going to call ourselves DevOps now. I guess we were ops before and now we're DevOps. And so we're like cutting edge. Nice little pay raise. Nice little pay. Cool little title. Yeah. Get a couple of thumbs up on LinkedIn for the role change. It's nice. Yeah. Now you just got to do the work.

5:37Now you got to do the hard part. You got to do the hard part. You got to do the hard part. And when we're talking about like DevOps being bullshit, are you more so saying like, hey, we haven't actually like changed? Or is it like maybe we have changed, but we're kind of missing it? Like what are your thoughts there? I think there's two or three things. So I think one is for many organizations, like much hasn't changed. So, you know, in my day job, I interact with some very large organizations. The company's been around for 50 years, 100 years. And you get in there and they're like, I'm definitely an ops person.

6:16I click a bunch of stuff in the browser for AWS to build it for my engineers. And they ask for stuff through ServiceNow. Like they want a database. They ask me through ServiceNow, bug me via Slack for a couple of months and I'll get there. I'm a dev op. And it's just like, you're not. I mean, your work is important, but you guys haven't figured out how to do that. And it's like, it's still some big companies that are still there. Like we haven't moved much in 15 years. But at the same time, while we haven't moved much, the world around us has just absolutely changed, right? So there's a follow-up article to DevOps' bullshit called Elephant in the Cloud, where I go into this a bit more.

6:55But if you look at what we do as software developers, our job hasn't changed much in 50 to 60 years. We write some code, there's some ifs, there's some variables, there's some loops. Sure, the syntax changes from language to language, but writing software is writing software. If I take a Node engineer and I throw them in a Ruby code base, it'll take them a while to realize that it's sane and there's no event loop. but they'll get it, right? Now, when you talk about the cloud, the way, like everything has changed about the way that we operate software in the past 20 years. So like 20 years ago, let's say 24 years ago, we were on Metal.

7:30And then there was like the VMs started coming out. It's like, okay, we're putting things on VMs on Metal. And then there was like the slice hosts. And then there was the passes. And then there was like this like brief, like containerization Mesos moment. And then there was the Kubernetes moment. And then there's Lambda. And now there's serverless containers. and like everything about the way that we've operated software has changed. So that's interesting. That's changing on us constantly. But also the footprint of our applications has changed significantly. People are always talking about, ah, monolith versus microservices.

8:00But I think the bigger pain point in our applications is even if you have a monolith, there's a good portion of it that is secretly microservices because you're using 14, 15 different cloud services to do your thing. I've got a Ruby monolith, but I'm using SQS. Okay, well, that's interesting because now this configuring a cloud thing just became a part of my job. If I'm doing DevOps, I should be able to do that. Yeah. Okay. Now I do parity. Okay. And now I got to do compliance. Okay. And now I got to do the security around it. Okay. Now I got to become an IAM expert. Okay. And now a whole bunch of stuff got stacked on me.

8:32Yeah. And I'm just trying to build a feature. I'm just trying to fucking process a queue, man. I'm trying to get some stuff out and get some stuff done. But now all this other work came up, right? And so when we look at like DevOps 15 years ago, It was, hey, we got to figure out how to like ship stuff more frequently, get things on some of these VMs. And like the big cloud wasn't a thing then, right? And now where we are today is you have a lot of cloud around you, not just underneath your application. It's integrated into your application that you have to figure out how to manage and deal with.

9:03And that's where I think it starts to get real bullshitty and hand wavy. Like that idea of DevOps where you literally did everything or like you figured out a way to like make your team super harmonious, like that's just hard with how things are today. And I think we need to look at it in a different scope. And I think one of the things that we really screwed up with DevOps is we didn't market it. Everybody just took that title. They took that money. They took the thumbs up on LinkedIn and like some companies got in, they did good marketing. They got the stakeholders to understand why it was important.

9:35They got them bought in. They got the culture to happen. And a bunch of other people were just like, my title changed, right? And I think that is kind of the source of the bullshit. People started to get different definitions of what it is. The entire world's changing around us. And just a whole conglomerate of services are kind of getting thrown at us now to like meet global scale that like what even is DevOps? And you could argue on Reddit all day long about what it is, but no one's going to agree. Yeah, yeah. I mean, no, no, I know. I think it's right. And I remember when it was like 15 years ago, I'm looking at your blog here.

10:10I think it's actually the picture where it was like worked fine in dev ops problem now with a house on fire in the background. Like, I think that's what like the original DevOps, at least to me, the meaning was where it was like developers are throwing things over the wall to ops and then there's no good communication. I don't even think that is really what it is now. Now, what caught on when you were talking is it's almost like you're saying I'm trying to get this feature done, trying to get I'm a developer trying to get to the feature done. But now there's all these other things on my head that I'm expected to do.

10:47Are you seeing out there that it is like on top of the developer to actually like coordinate all this stuff, know all the technology, everything that it takes to get off to like a production? or are you seeing it more like, okay, if you're doing DevOps like properly, in your opinion, it looks different than that. Yeah, I mean, I think that's part of the problem is like so many people, like we see so many different teams that like when they're like, hey, we're, yeah, like I'm the DevOps team or we do DevOps. I have no idea what you mean until like 15 minutes into the call. I have to like, I have to see your world.

11:23I'm like, oh, I get, yeah, we can call this DevOps. That's a little DevOps too. I get it. Like, you know, like you're just doing it your way. Right. And like, and so you do see teams where it's like, I'm responsible for everything. We've shifted it all left. And like, that's, that's the rub is like, there's, there's too much to shift left nowadays. Yeah. Right. And like, and that's kind of where, you know, in the post, I moved towards platform engineering and it's like, now there's caveats there too. Like there's some bullshit in platform engineering. There's a lot of bullshit in platform engineering too.

11:55But like what we need to be doing as teams that have the resources to do DevOps or platform engineering effectively and as providers like MassDriver, AWS, et cetera, is starting to build more of this toil and tasks that devs are kind of getting caught up in into their platforms. Whether you're building it or whether you're buying it, like we need to start doing more of that because just shifting everything left onto the developer isn't working. It's not what you pay them to do. You pay them to build features to build, generate revenue. Yeah. pay a developer make value yeah fiddle with checkoff right like it's like go do something makes value right at the same time we have to remain compliant and secure and all this other stuff like whose job is that right yeah and you might say the security team but now you're tapping and you're waiting on somebody right and so like like the whole thing it's just like like we don't have we got to grease the gears of of the way we're doing this right everybody's becoming a software company now there's more stuff moving to the cloud and we're not building out this expertise.

12:51And we're either saying, slow it down and tap that guy, or it's all your problem. And neither of those I think are effective strategies for scaling in the cloud, scaling you or people in the cloud, not necessarily your compute. I would love to pick your brain on platform engineering. I know it's another word, so it could be bullshit as well, but there's like DevOps, there's platform engineering. When you think about what platform engineering is, can you describe to like what comes to mind for you if you think of like some elite platform engineering team like how does that look to you yeah so this is where the bullshit starts to come in because the exact same things happen with platform engineering so if you would have talked to me like if you would have talked to me like two years ago i was like i was sunshine rainbows kittens all the good shit on platform engineering because i saw an opportunity there i saw a word that was getting popularized books are getting written blog posts are getting written and i was like this is the moment where we can take this word and we can market it correctly within our orgs to get the buy-in and maybe actually use that to actually get the fucking work done.

13:57But what's happening is we're just, it's, it's rebranding very similarly to ops to DevOps, DevOps to SRE, SRE to platform engineers. So you meet some teams where it's like, Hey, we're doing platform engineering. We have this, you know, separate part of the business where it's like, we're building this thing as a product and we're giving it to our developer customers to use. Like they're doing like the platform engineering. And you meet a lot of folks where it's like, oh, what do you do as a platform engineer? It's like, ah, yeah, we manage all the infrastructure in AWS. It's like, oh, how do you do that?

14:23And they're like, we're not software developers. So we just click around AWS. Oh, wow. So it's really exactly the same as it was 10 years ago. It's just a different word, right? And so when I think platform engineering, I'm thinking of the company that you work for has decided that they've gotten to a level of maturity in the DevOps maturity model where they can actually do DevOps well, and they're ready to take that to the next level. We need to pull more of this responsibility off the devs. We need to start building something that feels a bit like maybe Heroku, like it feels like a pass. It feels like a platform.

14:59It feels like I can write code and it just goes and runs. I can build it and run it without taking on 400 other jobs. Like that's the goal of platform engineering. I think that's the distinct difference between this and where we were 15 years ago because we didn't have the cloud then. And we see a lot with platform engineering is infrastructure provisioning, being a part of it, talking about the cloud, talking about getting self-service of cloud resources, which wasn't like the terms that we were using 15 years ago. But these terms really matter now. As a developer, I need a queue. Do I need to know how compliance, security, IAM, doing parity with Terraform work?

15:34Do I need to learn HCL? No, I just need a queue. Why isn't it easy? I can do it in the AWS console. So how do I do it at work without tapping you or going through hell and back to like set up a bunch of CI CD pipelines to like do it right. And I think that's, that's what platform engineering is. We're mature enough. We have the resources, we have the buy-in, and now we're building a product to solve the biggest problems of our developers cadence and delivery. I, that really clicked for me the way that you said it. Cause I do think like, uh, if you have a platform engineering team or you have a dev I've seen, like I always, I'm the type of person that has to go back to like what my mission is.

16:12Otherwise I can't figure out like what, what my purpose is on earth. So I got to know my mission. And what you said that I thought was cool is remember how it feels like to use a pass as a developer. Now, of course you're boxed in more and it's not, you know, you might not be able to use everything that you want to use, but the cool thing about it was you could just code and then like the other stuff just works. Like the app just runs, the code makes it to where it needs to be mated to, the customers get it. It's like super simple. That's the coolest thing about it. And I think you said something like, if you're a great platform engineering team, the vibe that you're giving to your developers that you're servicing, that's kind of how they feel for your company.

16:59They can just do their job. Like that totally resonated with me. I feel like you should be able to take that team and be like, man, this platform engineering team has done such a good job. We're spinning them out as their own business. To me, I'm like, man, that team is killing it right there. If that's the way you feel about them. But there's also many of these teams where it's just like, hey, we manage Kubernetes for everybody. And this is a place where I see like, I'm a Kubernetes fanboy. I develop on Kubernetes locally. I like the thing, but there's a lot of people that are just like, oh, we're platform engineers.

17:32Kubernetes is our platform. It's like, no, no, no, no, no. That's not it. Like if you think about like what Kubernetes is and how big it's getting and how its control plane is expanding into controlling the cloud as well, Kubernetes is just kind of another cloud API, right? It's just another cloud. That's like saying, oh, our platform, we have a platform engineering team. They manage AWS. It kind of starts to feel the same, right? And you can't just like throw a Kubernetes at a developer and be like, we platform engineered it for you. Like there is DevX that needs to happen around it. There is some smoothing out.

18:03Like if your developers are thinking about Kubernetes all day, I'd say that you haven't done platform engineering. You're still in that world of like mixed, like dev-y, op-sy bits, right? Yeah. Right. But that's, but that, that is interesting because it also starts to get, it gets interesting, right? So it's like, you see these companies where it's like, they have a team that just manages Kubernetes. It's their job. And that's a hard job. There's a lot of work to do there, right? But like, what is that team? Yeah. Are they DevOps team? Are they ops team or they, you know, the compute management team, right?

18:33Like, but, you know, calling it platform engineering, we're actually doing engineering work to build out this stuff that's going to serve these engineers well. Like, I feel like that's a, just another misuse of the term, but that's where we are. That makes sense. That's fine. Um, so I'll be writing a blog post here in about two years called platform engineering is bullshit. And I'll have you back on the pod. Yeah. Hopefully we'll find another really good word to use that like actually sticks this time, but probably never. Yeah. I think whatever, whatever word it is, it will be weird. That's the thing is like, you know, it's really hard.

19:07It's so easy to like write a blog post or like hop on a podcast and talk about like the way things should be. But the reality is, is like the real world's messy. Production is messy. Organizations are messy and your teams are probably messy and your code's probably messy. Like there's, there's a ton of mess, right? So like having this, like, like this golden idea or a golden path, it's like everything's fit into this perfectly. It's like, it's so unrealistic, right? So when you start looking at orgs, you'll see orgs that like, you'll see the Googles of the world where they went and did their platform engineering and it came out as something called GCP, right?

19:38And then you'll see other companies like we do platform engineering and it's very much like, A, we write Ansible scripts, right? Like the way this is servicing in many companies is, looks very different, very similar to DevOps, but it's because our companies are different, right? Like think about the average, like Series A company, Series A, Series B, you can't give them a platform, right? I can't go to like a Series B company and be like, everyone's using Heroku now. From now on, that's what you're doing. Sure you could. There's some architecture decisions you're gonna have to deal with, but then your team that's working on like Next.js, Jamstack stuff, they're not gonna have like the greatest experience on Heroku, right?

20:16They probably want something more like Vercel. Your AI and ML team's probably not gonna have a great experience on Heroku, right? Like you're not going to find this like one thing that's perfect that works for everybody. And that's the other thing that's kind of problematic about this is like that happens at the team level, it happens at org levels. And I think that's why you're seeing like these very different definitions of what platform engineering is. It's people sitting around thinking, I got to make something happen and this is the way I'm going to do it. It's going to be, it's just Kubernetes or it's this complicated software thing that we built.

20:45And some teams are doing it and they're doing it all in CI. Like you can get pretty far doing platform engineering and infrastructure provisioning and self-service with just GitHub actions. Like you can do it. It's cumbersome to navigate, but you can do it. You can deliver self-service and DevEx that way. But so many questions now. I mean, certainly the takeaway there is like platform engineering is specific to your organization or like the mission of what your engineering team needs to accomplish in order to sell the product that you're probably creating for your business, like that totally makes sense to me.

21:21And then I think I would say like the level of eliteness that you go, you know, I think it's about the experience. Like I keep going back to what you first said, like if I'm like the head of platform engineering, even if I'm like in a startup or a midsize company, like how am I measuring my success? Like how would I rate myself or like what what what is a metric that i would use in order to say like i'm in the right way or am i just doing bullshit stuff yeah i think this one's this one's an interesting one and my answer is going to sound a little bullshitty unfortunately but bear with me you can go and define golden signals and sre stuff you need to like those are those are things that you have to measure when you're running a platform those are things you should probably be measuring when you're running a product for users.

22:11Like if you got users, you should care about the thing being up. I'm going to kick all that shit to the side. I'm sure people are stoked about measuring that. Two things matter. Dev happiness and how unblocked people are. Like that's in the grand scale of things. I think that's what really matters. Are developers happy with the way that the thing works? Is it solving a real problem for them? And is it improving the cadence and delivery of our software? And that second one's a little harder to measure sometimes because Sometimes you'll have teams that like the stuff's real stable, right? Then maybe they're not releasing 14 times a day.

22:45Maybe they're releasing once a week and they're making billions of dollars, right? So like, what does that stability look like? What does that release process look like for those developers? Is it breaking often? And like, we need to make that a bit smoother. But that's really, I think what it comes down to is like that, the qualitative happiness of the developers using the thing. And then that delivery experience, am I shipping software faster? in my shipping software with less like build failures of getting the thing out. Now you can come in and like come up with like plenty of other ones. I think you could, again, like this comes back to the org.

23:19Like it really depends on like what the mission of this platform engineering team is. And since it can pan out so many different ways in orgs, it's hard to like pinpoint it. Like those are the two that I find most important, but you'll see other people where it's like, hey, one of our goals is like minimizing cloud costs. It's a weird one, but like we've seen it, but it's a real problem. Like developers have just been clicking stuff forever, or you have an ops team has been doing it for devs. But since they're kind of a world apart, like there's a bunch of stuff in prod that they don't need anymore.