Finding the Best Gen AI Use Case for Your Dev Team | Sonar’s Peter McKee

0:00We're so early. I don't think the learnings are there yet to be able to give really, really, really good concrete prescriptive advice, right? I think companies are going to have to learn some of that as they go, right? What are the problems? How does it upping the amount of code and the speed that we're trying to move at? How is that affecting things? Is it working? Is it not working? Is there certain types of code? Is it backend that does better with Gen AI right now, as opposed to React or Angular that might be a little framework-based and less language-based, right? Yeah. I don't have anything very prescriptive other than I would think Start as a normal engineer, start small, build upon that, go slow, walk, crawl, run.

0:37And as an engineer, I think you need to fight back that with the business, right? The business, you still need to push. I think you need to push, right? But I think we walk through it slowly. Developer productivity can make or break whether companies deliver value to their customers. But are you tracking the right metrics that truly make a difference? Understanding the right productivity metrics can be the difference between hitting your goals and falling behind. To highlight how you can adjust your approach to both measure what matters and identify the right corrective strategies, Linear B just released the Engineering Leader's Guide to Accelerating Developer Productivity.

1:13Download the guide at the link in the show notes and take the steps you need to improve your organization's developer productivity today. Hey, everyone. What's up? And welcome back to another episode of Dev Interrupted. I'm your host, Dan Lines, Linear B co-founder and COO. And today I'm joined by Peter McKee, Vice President of Developer Relations and Community at Sonar. Welcome to the show, Peter. Yeah, thanks for having me. Glad to be here. Glad to be here. We were joking before. I love that title. That must be a really fun job to have to be a VP of like developer relations. Like how is that for you?

1:53It's great. It's great. I was a software engineer for 20 some years, 25 years or so. You know, really enjoyed it. I thought I was going to be the guy you put in the dark room, slid pizzas to, you know, leave alone. And I was that in the early career. But as you can't tell, I turned out to be the smiley, talkative engineer. And when I was at Docker, they were like, this might, this role might be good for you. And yeah, I love it. I love it. You get to talk about tech, stay involved in it, be hands on. But I don't have any deadlines for production on the dev team. So, you know, there's that. So, yeah, I really, really enjoy it.

2:25Yeah, you thought you were going to live off of pizza, Mountain Dew, maybe some Reese's piece. I remember that's what I used to say, a lot of Reese's pieces when I was a developer. I love Reese's, yeah. But you're the guy that can also talk, smile, be on a pod, relate to people. Who gave you your first shot at it? You said it was Docker? Yep, it was Docker. That's cool. Yeah. Yeah, we had sold off the enterprise business to a company called Mirantis, and we kind of started over and recapitalized. I kind of lucked out, right? Because it was a huge community already. Docker had a massive brand. So it was able to kind of step into that brand.

3:00So yeah, it was a great, great first Devra relationship role. And I loved it. Love it. That's really great. Yeah. Thanks for sharing your background a little bit. I know some of the developers listening and stuff may be like, how do I get a shot at that? What do I need to do to be like Peter? But today we are going to continue. So my co-host Connor and I have been interviewing leaders from across the engineering spectrum. We have this series going on on Gen AI, obviously like the hot topic, variety of viewpoints on different approaches with AI and what does it mean for developers. So it's been really fun so far to get everybody's perspective.

3:41So I guess we'll start out, Peter, like what's your perspective on how Gen AI is impacting software or development processes. You can take it from wherever you want to take it from. Yeah, man, it's a huge question, right? It's super interesting, super interesting. So many ways, it's definitely impacting development. And I think it will just continuously impact us. Impact us might be a strong word, right? Because I also feel that we, let's think five years from now, three years, five years from now, the progression of how these LLMs and the Gen IA is progressing, the speed it's going, um, is crazy.

4:19Right. And I don't think we know exactly what it's going to look like in three to five years. I was thinking about this the other day, I take a step back. Right. And if you look at when I first started developing in the nineties, right, their assembly was still around. C was around, you know, I walked uphill both ways to work, but, um, uh, you know, these terrible days, but yeah. And then C plus plus, right. But as we went, you abstract and you get higher and higher and higher, the barrier to entry becomes lower and lower and lower. Right. And I think that's what Gen.ai, I think if we look at it right now as a tool as such, right, that you're going to start programming at a higher level.

4:55Right. Does English become the new programming language? Right. If you think about it, we're writing English just in specific keywords. Right. To give commands. It's not that big of a leap to go to, you know, general English to then gets transformed into JavaScript like everything else and run in the browser. Who knows, right? But it's very interesting. So it's affecting development. It's affecting the way we think, the way tools are being built. It's going to change not only us, but knowledge workers in general, right? There's a whole other thing too, right? We thought it was going to be the robots, the autonomous vehicles, right?

5:30But no, it's coming for the lawyers and the software engineers and the doctors, right? With triaging. It's pretty interesting. Very, very interesting. So first of all, I do think it's affecting all industries, right? We're going to talk about software development but i think you're right like everything's going to have its own co-pilot i guess right i'm a lawyer i have my co-pilot to help me formulate i don't know what lawyers do but help help me formulate like my disposition or whatever right but you know i i do think everyone's going to get paired up with some type of bot that helps them be more productive and I think it's pretty fitting that it's starting with you know developers the ones that are creating these you know bots anyways do you see anything like difference between I don't know like entry-level developers versus more seat like hey I started developing in the 90s versus I'm coming out of school now or like I you know am in my first few years like are you seeing anything differences with that?

6:32Yeah, definitely. Definitely. Right. Yeah. Someone with 30 years experience, 20 years, 15 years experience. Right. So let me take a step back and I kind of think of things through this kind of a model when I was thinking about this is, so I have seven children, we homeschooled them all. We, my wife really, you know, I acted like the principal, right. But there's this, there's this classical education model called the trivium, right. And you start out with your language and grammar. It's your basics, right? Then you learn logic. And it's traditional logic, but it's a logic in how do solutions work?

7:05How do solutions work logically to solve a problem? And then above that is rhetoric, where you start to argument for different problems to solve, right? And so if you think of that through engineering, entry level, mid-level, junior, even senior folks could stay in that logical phase. principals and staff go up into that rhetoric, right? They're working across the organization. And so if I apply that to general AI, those junior developers, right? They really just know the mechanics, right? Of writing software. They can't argue at the rhetoric level, right? And I think to really use Gen AI where it is now as powerful and as quickly, you have to understand your language really well, your environments really well.

7:45You have to be able to see what genii produces and know how to shape that into what your domain space and the way you construct software junior developers don't do that as easy right um and so have we accelerated the copy and paste from stack overflow yeah with genii kind of just sped it up right and so i think that's the real risk is for you still at this point have to understand what the code is doing how it does it but you can offload things right like i can never remember how to read in a file and parse a csv file, right? Like Stack Overflow is gold for me for that, right? And Gen AI, boom, boom, boom, can do it two seconds and I can read the code very well and make sure it's solid.

8:24So I think senior level folks, and I'm just put into two big buckets, right? They're able to reason quickly about the Gen AI that's the code that's been produced. And it's just experience, right? It's just wisdom, let's say. It's not knowledge, it's wisdom, right? So it's the more wisdom you have, you're able to see the shit code first quicker, right? Than the gold code. So I think those are the two big buckets I kind of think about it and how you should kind of maybe as a dev manager thinking about noise tools. That's interesting. So you gave, I think you gave the example of like parsing a file or something like that.

8:53Now, if you have the wisdom, you're a developer, you've been around a while, you have some wisdom, you would say like, hey, for parsing a file, like that's not where I want to spend my brain power. Let me, you know, I'll review the code. I'll make sure that it's quality, but it's like, that's not what I'm really trying. Usually that's not what I'm really, I'm not like writing this parser. It's like, that's like a means to the end of the actual business value that I'm trying to achieve. Now, maybe there's some danger there then, because if a junior developer is using Copilot, this is just speculation.

9:25And before, at least I had to go to the internet, search like overflow, like do some copy paste, maybe read what people are saying, like, Hey, this is a good solution. Not a good solution. Like I remember doing all this, trying to find. I don't know if it's just like auto-populating for me. Maybe I'm not going to, maybe the quality will go down or maybe there is a little bit more risk. Do you think there is more risk for junior developers? Yeah, a hundred percent, a hundred percent. And that's what we worry about at Sonar, right? We're really worried about quality and we do static code analysis, right?

9:57And for us, Gen AI now is, our big message is, hey, this is even more so while you should be scanning your code and looking for quality, right? Right. There's just tons more code being produced. And what I've learned working at Sonar, right? There's security risks that are very, very hard to spot in your code, right? There's things in greps and in regular expressions and, right? And it looks good, but the machine is way better to do it, right? So yeah, I think there's a big risk. I think there's a big risk. There's always a risk to go faster. Gen AI is producing more code. Hey, let's use that.

10:31It works, right? functionality and quality are two different things. You can have something functioning, but it could be a shit quality. Excuse my language. It could be poor quality. What's the three pigs and the wolf? How they build their houses on sticks and brick. Sure, the dollars all worked. They had windows in all the houses, but the one that was built on a solid foundation last. Sorry, using a kid's story, but it's true. Build on quality and static analyzer code for quality first, especially if you're using Gen.I. You just cannot ensure the quality that's being produced. That's not what it does.

11:06Yeah, I mean, I agree with you there. You know, like my co-founding partner, his name is Ori, and he's our CEO at Linear B. What we're observing is the amount of code that is being developed. Now, it might be developed by a human, but it's actually also developed by some type of AI. The volume of code is increasing. so there's a lot more code now that you know i i know you're you're at sonar so that's gonna put a lot of stress on tools like like sonar you need more usage right i open up a pr what's the quality of that what needs to be reviewed is there a vulnerability here and i also think that there is there is pressure um on engineers to move faster like i'm talking with a lot of vps of engineering they're coming to, to linear B like our, our company, and they want to measure the impact of generative AI.

12:04Right. I'm spending millions of bucks on this. Is it, are, are my juniors faster? Like what, what's the impact of it? And so I do think there is that it's maybe like double dangerous because there's that expectation. Like you got, you got to move, you got to move faster. I just bought this thing for you. Right. Right. Yeah, exactly. Yeah. It just compounds it, right? Most business leaders and probably yourself, right? You think of ROI, right? ROI. They got to. And that's what they're judged on, right? Exactly. And coupled with that, right, is how hard is it to, first of all, quantify how good a dev team is, right?

12:42It's so hard. You can't look at numbers of lines of code. You can't, you know, maybe it's solid features that the business wants delivered, right? Something like that, right? But it's not lines of code. It's not, you know, issues fixed, right? Because what will happen if we go with those, we'll become sales folks, right? Sorry, failed sales folks. And we'll game them, right? Sure. You performance me on a number of defects I fix or a number of issues I, you know, features I develop even. Well, I'm grabbing the smallest features I can, the easiest ones to do, right? Or I'm going to produce a bunch of defects and then I fix them and look like a hero, right?

13:18So even that's an issue, right? And now throwing more code into the mix and saying go faster, faster, faster. Yeah, I mean, so in some sense, we do need these abstractions so we can go quicker and still bring more value to companies. But at what cost? It's usually always like in software engineering, it's like the balance, right? You want to go fast, but you got to have the quality. You want to get out to production as fast as possible. But if there's a bug found in production, it costs you like 10x more. So you don't want that. And I think that's kind of what everyone's dealing with. Then you got the security issues on top of it.

13:52Yeah. Really interesting times. Have you seen anything around coaching developers around using some of these generative tools? Like is any organization trying to say kind of like, okay, like here's how to use them. If you're a junior, even just like what we said in this pod, like, have you seen any coaching stuff going on? I haven't. We do some clean as you code or learn as you code, but it's a real more around vulnerabilities, around issues, why it's an issue and why you shouldn't code like that. And that's really good for junior developers. But no, I've seen other companies and YouTube, you know, coming out for the general public around prompt engineering.

14:28I'm sure you've seen, you know, which is powerful, but nothing specific for developers, right? At least that I've seen. Yeah. No. Like, how do you like truly integrate these in? What does that workflow look like? What does that pipeline look like? Yeah, it's very interesting. I haven't I don't know. Does that plug into some kind of these DevEx platforms that are coming out? It certainly does. Like for us, you know, we'll say, hey, this code had some or this pull request had like Gen AI modified code in it or Sonar. Hey, found something on the PR scan, set up a rule for like multiple reviewers.

15:04Like we're doing that type of stuff with companies. That's kind of like putting policy and I would say like guardrails around it, which I think it is really needed. And then the other thing that, you know, we're kind of like exploring a bit is like, how could we coach engineers like earlier on so you don't have to get to that point that you need, you know, all the rules. But yeah, I haven't seen anyone doing that yet. One thing that popped in my head, I don't know if it's germane to the conversation, but we've been thinking about, you know, just like everything, right? It's junk in, junk out, right?

15:37And so I think part of the problems that I'm not sure, right, is some of these LLMs or GenII, it's matching a pattern that's probably not the best pattern for that solution. And it's a little bit of randomness in there, gets a little bit off. If Sonar is at the front and you're scanning and cleaning the code that then you're feeding into your LLMs, or at least I forget the term, but you have your LLM and then you're putting your data into it also. If that code is clean, I think that's a great start. You should be starting there to then feed into your LLM some good quality. And it could be specific to your organization how you do remote procedure calls.

16:18You can do them a handful of different ways, but we do it here at Visa this way. And maybe LLMs could then be powerful to help that way, to help say, I'm going to generate you code, but I know the pattern that you use. And so you get maybe a higher quality. Still not kind of in the training, but that's one thing that we're kind of thinking about at Sonar that would help with some of this, right, with junior developers, that they can start trusting the answers a little bit better. Yeah, that makes sense. Have you seen anything around Geni affecting quality? Like, is there any data reports or anything that you've seen?

16:52I've written a couple articles pointing to GitHub, Microsoft, and the like that are the amount of code churn that's happening. Increasing. The amount of... Yeah, we see that too. Yeah. The quality is definitely down, right? Developer love, right? Developer joy is dropping, right? Because you are, well, you know what it is like, you know, as an engineer, you want to work on the code that you wrote. That's nice and formatted, nice and neat. You can understand it right away and you can add to it. Yeah, like the art of it. Yeah. But it's rare, right? You get into a code base that's been built 15 years ago.

17:30you know that they're so bound together right and you work a lot on fixing other people's code right and now if code's being generated faster it's gen ai it's not that great of code now you got to fix even more code right so developers getting upset with that you know and they're looking for and it's it's still a little bit of a seller's market right like um we we can uh we can move top engineers still have a lot of leeway to move so if you go somewhere and you're like this code base is terrible i'm not getting the help to fix it i'm gonna leave Right. And so we're seeing developer turn leaving places to find that holy grail of development.

18:06That's one of the good things about being a developer. It's like you can get the job. Usually you can always get a job. Right. So that's like a good thing for like the VPs and the managers listening. I mean, I think most people know it's like that's why developer experience matters so much. That's why, you know, companies like us are measuring it. That's why companies, I think like you are saying, like, let's keep the quality high. Nobody wants like a crappy code base. Like that sucks. I mean, developers can go wherever they want. But the fact is like most of the VPs that I'm talking to are experimenting with Copilot, right?

18:41Like they are rolling out initiatives. They're testing it. They want to measure the impact. Like it's here. It's real. Yeah. I think you have to. Gotta. Yeah. Yeah. You don't want to be left behind. Now, we were talking about the junior developers a little bit, but probably on the flip side of that, maybe if you're like intermediate, I think you were saying like senior developers, the ones before principal, maybe, you know, we have been seeing increased throughput for sure. I mean, I said, so I take notes, right? I imagine every day I have my, my sublime and I do markdown in there. Right. And I have some little goofy markdown for tasks.

19:17And I would go day to day. I'd go across each one of those files and I pull the tasks up and I put them into another file. don't ask why it's just weird process but i'm like well i'm an engineer i should write a script to do this right and so i'm like i'm gonna write it in c why because i haven't written c in a while threw that out very quickly because there's no strings but i went back to javascript okay so i jumped over the chat gpt and um i was using i think three five uh maybe even before this three five right and just started firing up asking questions and i hesitate to say 80 it's probably more like 65, 70-ish percent get me there.

19:55It would start to break down as I started getting into the little bit of refactoring places. It would break down. It would add modules that weren't compatible with other modules. But my point is I'm almost 30 years into software engineering, heavily, heavily in the JavaScript since like 2003. So I know it really, really well, right? Didn't matter to me, right? I was able to take that last 40, 30, 20 % and code it really easily. So it was very powerful for me. It helped tremendously, right? Because I knew I could see what it was doing. I knew what it was doing. I could fix the versions real quick.

20:28I could refactor really quick. Oh, yeah, you shouldn't really use that method, right? This is a cleaner way to do that. But you can clean up. It's like the 80-20 rule or like the 60-40. It's like you could do the 40, 20, 30. At the end of the day, what you're saying is you're moving faster, which means more code is going to get pushed at a faster rate in general, let's say. Do you have any opinions on, like, if we're in agreement that more code in general is going to be pushed faster throughout the planet, right? Do you have any thoughts on processes or, like, infrastructure to accommodate this load, in a sense?

21:09Like, have you thought about that at all? Other than AI monitoring AI? I mean, that is a thing. So we're working with some customers where they are, so let's say Copilot has assisted the developer in creating some of the code. They open up a pull request. And on the linear B side, we can see that it's assisted with Copilot. So then also what they're doing is experimenting with some of the AI review. Right? Right. what they're what we see them doing is then writing some rules some automation to say okay if there is this much code that's like gen ai like orchestrated or if we have like a reviewer that's like an ai bot i do want to still make sure there's a human or two like i see a lot of that that happening again that's kind of like streamlining i would i would call it like end to end more on the on the sdlc because the load is moving to that pr basically Yeah, I think, you know, AI is really good at pattern matching, right?

22:14And predicting. And, you know, we might not use these large LLMs maybe for code reviews, right? Maybe those are specialized tools that are really tweaked and there's thresholds there. But yeah, I think that's where we're going to go. I think to answer what you should do now, and I'm extremely biased, right? So I will talk in terms of the solution and not so much Sonar, but Sonar has a great product, right? But static analyze your code. Should we static analyze your code? AI, I think, will eventually get there. We'll help augment static analysis. But, you know, right now, handwritten parsers, static analyzers, these are written by PhD students, not even students, graduates, doctors, right?

22:54Really, really good. So I think quality first, right? You've got to be checking your quality. And I think that's the, you know, the barrier to entry, right? It's kind of like we need to get there, I think, how we are with unit testing and functional testing, right? Like no good software engineering team, professional software engineering team, DevOps team will not allow their code without any unit tests, right? And like, you just would not do that, right? And we don't do that with quality, right? Why aren't we doing that with quality, right? We should be doing that with quality. I think DevOps personnel should be going, great.

23:28Your unit tests don't turn green. You're not going. If your quality gate isn't passed, it's not going. I think we need to get there. And I think that'll help with all this code produced. And then how do we look at AI just in general? How can it help streamline the processes? How can it help with code reviews? To your point, how can it help take a junior developer to a senior developer faster? right it i think there's a huge power there right and you were hinting at and talking about it earlier i think that's maybe you and i should start a second company and do that but um right but i think the time in half from junior developer the senior developer with our new ai software right but i think it's honestly a compelling thing like again ever like the people that are putting up millions of dollars in budget so it's like your vpes your cto's they report to the CEO, they are on the hook to show this investment moved our projects along faster, gave more ROI to the business.

24:29So it's actually not even that much of a joke. I bet they'd go for it. Yeah. Yeah. A hundred percent. This might be contrary. I know we're talking to the developer tech crowd, maybe. Businesses look at us as a liability. We're a cost center, usually. We're a risk to put security into production. Code is a risk. And the more and more you have, it's harder to maintain and technical debt gets harder, becomes more expensive. Software engineers are expensive, right? So whatever we could do to lower that risk for the business is what I think an engineering and development team should look to do, right?

25:04Because then that brings even more value to the business and they love you and they keep paying you to do what you do. So I think a lot of tools are going to focus on that. How do we, and we can't as engineers get too tight, right? And hold onto our domain too much. Like I have the right code. That's what I do. I do my, and we do, we do so much more than just typing in code. Right. But you got to let some of that go, right. It's just a step up in, in, in the abstractions, right. We're just moving up higher. You know, I would never come to you and Dan and say, Hey, you're going to go, you, the next at linear B, you should absolutely go to assembly language.