AI-assisted coding with GitHub's COO

0:03Welcome to Practical AI, the podcast that makes artificial intelligence practical, productive, productive, and accessible to all. If you like this show, you will love The Change Log. It's news on Mondays, deep technical interviews on Wednesdays, and on Fridays, an awesome talk show for your weekend enjoyment. Find us by searching for The Change Log wherever you get your podcasts. Thanks to our partners at Fly.io. Launch your AI apps in five minutes or less. Learn how at Fly.io.

0:44Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I'm CEO at Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris? Doing great, Daniel. How's it going today? It's going great. I was just commenting before we hopped on about I'm feeling the emotional boost of seeing the sun again. after a long Midwest winter. So feeling good today and excited to chat about all things AI and code assistant and development and all of those things because we have with us Kyle Daigle, who is COO at GitHub.

1:29Welcome, Kyle. Thank you so much. It's so great to be here. Yeah, yeah. It's awesome to have you on. just even in your comments about how you like to think about the practical side of AI. This is your place. So I already feel a kindred spirit. I feel very much at home already. Yeah, yeah. Well, speaking of which, I mean, you're, of course, you know, really kind of at the center of a lot of what's going on in terms of code assistance with GitHub Copilot, of course. But you're also, I'm sure, seeing a ton of things out there. I'm wondering if you could just kind of take a 10 ,000 foot view and kind of for those that maybe aren't following all of the things happening with AI code assistance and development.

2:16What's kind of like as of now sitting in what is it March of 2025 if you're listening to this. What's kind of the state of AI code assistance and how people are kind of generally using those right now. Yeah, I mean, it's so interesting to see how far I feel like we've come in such a short period of time, right? It was only a couple of years ago, when ChatGPT came out, GitHub Copilot came out. And back then, the novelty was sort of like, it wasn't going to disappoint you, right? For GitHub Copilot, you know, you would type some lines and, and it would respond with, you know, a line, two lines, a method, etc.

3:00It was going to complete your code. Very similar to, you know, I'm going to ask instead of Google a question, I'm going to ask ChatGPT and I can keep asking a question. I think what really, you know, locked in this enormous transformation then was finding a user experience that was simple, straightforward and didn't need much explanation. Right. Like I'm a dev, I'm writing code and it's just working there versus, you know, needing to figure out how to use a tool, figure out how it works. in my workflow and kind of go through hours of onboarding. Fast forward a couple of years, right? Not only have the models materially gotten so much better, but we found more and more ways to kind of have that similar, joyful, expected user experience with code assistance.

3:52So it's not just really about writing the code in some ways, right? It's not about that at all right now, I think that's at the bleeding edge of what we're experiencing with code assistants, where it's much, much, much more about sitting down with a couple of dev friends and saying, hey, I have this idea for an app. But instead of pitching it to your friends, now you're pitching it to your IDE. And that's a code assistant is going to jump in and help you get that next step done. So when I look back over this wave and how it went from sort of, you know, cool, but in in retrospect, right, a little bit simplistic behavior of, wow, it really knows what I want to write next into like the next level of what it's always been like to be a developer, which is I have this idea and now I have to explain it to someone else.

4:43We keep finding ways to augment, improve and speed up what a dev does kind of every single day. And we're at a point now where I think we're seriously starting to blur the edges of like, what is a developer? I don't think we're there all the way to be very clear, but I think, you know, a year ago we were talking about that and it was like, sure. And now it's getting closer and closer to say, you know, well, what is that distinct, that distinct need? And that's only really been in a year. And then, you know, about two and a half, three years from the start of the start of this journey. And so I think the code assistant category has always been so interesting to me because it's kind of matching how we work.

5:31It's finding ways to augment and improve how we work, not trying to teach us totally to do something completely different, which I think when we zoom maybe from 10 ,000 feet to 40 ,000 feet and we look at AI, the best tools are the ones that are just helping us do work we're already doing. The tools that aren't the best or having more difficulty finding traction, in my opinion, tend to have to make the human contort to get the most power out of the AI tool. And so because we're devs, we're just kind of iterating in what we know. And that's been the power of, you know, code assistants and the growth of them, you know, over the last year or so, I think.

6:10I'm curious. I mean, that's a it's a great point you're making there. And about the changing developer experience and changing so incredibly rapidly. I mean, you know, month by month, there are changes in what it means to be a developer now. And so I know, you know, I'm sure I'm speaking for a lot of people. I like I keep reinventing kind of parts of my own workflow as I'm doing stuff because new tools become available and what I am doing or not doing is changing constantly. It's kind of a it's it's both amazingly wonderful, you know, given where where we've been over the years. But it's also quite tumultuous.

6:47and when we stop and if I stop and kind of lean back a little bit and have a cup of coffee and think about it I'm kind of going maybe a little bit scary in the future about how good it's getting and where that's going what are your thoughts since you've talked about the developer experience explicitly and the user experience of code assistants and that they are going they are rapidly going so far ahead what kind of thoughts and don't even go out a long way I'm just talking about in the next few months in the short term like where how can we be thinking about adjusting ourselves to an ever-evolving state right now as we're trying to think about that, even before we get into the specifics of the tools themselves?

7:28Yeah, yeah. I mean, you know, I think what we've seen at GitHub by rolling out these tools is like, we'll talk to customers or I'll just talk to devs or open source maintainers, et cetera. And they can kind of fall on this continuum, right? This continuum of, I absolutely love every AI tool. I'm going to use every single one. And I'm going to try every single one. And then you have the folks who are like, I'm never touching these things ever. They're terrible and they're going to destroy software. And then there's all the folks in the middle. And so I think the thing that I tend to tell folks is like, you know, just like in our careers, we've all had a moment where a new piece of technology comes in.

8:05And I feel like for some reason it's in at least 50 % developers minds of like, oh, well, that's just a silly thing or that's just a toy or whatever. So I'll just say for myself personally, Rubyist by nature, JavaScript took over and I'm like, JavaScript, Ruby's, you know, blah, blah, blah. And so like over time you grow and you realize, oh, well, I should really understand that and try it out. And it may not become my new go to tool, but I, it would not help me or honestly, like the industry or my peers for me just to be like, I'm never going to touch JavaScript. So I think that experimentation that you were talking about, Chris, is the important thing.

8:42I see a lot of devs like try out a new tool or try out a new feature or a new library or a new model and then drop back to whatever their floor is, whatever the thing they're most comfortable with, the model they know, et cetera, et cetera. And I think that is the minimum because the change is going to happen just like it's always happened between serverless languages, databases, you pick it. Right. And if you just don't experiment, my fear personally would be that you do kind of start to get left behind because you don't know how to reach out to the new tool that is actually excellent and actually helpful.

9:23And you're kind of stuck behind the eight ball learning something that you could have been learning as you go. I will say like, you know, in the next few months, even not even just kind of like now, I do expect way more kind of AI functionality to come outside the editor. Because if you're if you're developing software as part of a team or as part of a company, not as a solo dev or a smaller startup, but a bigger group, you we all know, like writing code is a important part of the job, but it's not all of your day, right? You're reviewing code, you're building out decision records or architecture diagrams, or you're debating how to roll this out.

10:06You're operating a live site, so on and so forth. I think as AI comes into those spaces to fill in the gaps more and more, again, like you're going to want to have those skills from, you know, figuring out the right way to word things when the AI can't just figure it out or the LLM can't just figure it out on its own. or again, like every developer, know how the system is working inherently so you can best benefit from it. So as long as you're kind of trying these things out, even if you drop back to your baseline, I think you get set up for more productivity and I think just kind of like more joy when the AI can take more of those mundane tasks away from you.

10:46Again, like I think over the next couple of months, not even the next year. What do you think are the, for those devs, some that have jumped right in, they've figured out their workflow. Maybe there's devs out there that are experimenting with the tools. What do you think are those new kind of, everyone's kind of got their muscle memory of how they develop the things that they like to use. What are kind of the new muscles that need to be developed for kind of AI-assisted coding, like the most important ones that you've seen over very many use cases? Yeah, I mean, I think there's kind of two major ones.

11:28Every developer has, like you said, come up with the kind of practices and principles for you personally, right? We've all worked in systems that have linters and CI and everything that stops you from making mistakes. But there's just also a bunch of things that I like to work in this order. It helps my brain process what's going on. You know what I mean? And so I think on a tactical level, stating those rules, those prompt instructions, whatever, depending on which tool you're using for this, there's a different name for it. But I do think that that's something just the act of sitting down and writing out, well, how do I work on this project?

12:08Even if you work as a part of a company, you know, how do I care about it? I always want to define a schema for the backend API before I implement the front end. And then I go back to the backend or whatever the thing is for you. It's writing that down and then letting the tool use that, I think is a dual benefit, which kind of gets me to my second point. The big skill that everyone is, I think, trying to work out is it used to be called like prompt engineering. And I honestly think it's just describing a problem. We use so much shorthand and sort of we skip over the details like the hilarious meme of, you know, what the product manager said, what the engineer did, what the designer did.

12:49But that is exactly what we have to do with these tools every day. We go build an app that X, Y, Z's and suddenly it comes back and it makes no sense. And you go, oh, this stupid thing doesn't work, you know. And yes, sometimes it just doesn't work. But realistically sitting down and saying, well, what are the must do's of this app? You know, how do I want it to work? What do I want the flow to be? Whatever those things are, being able to clearly communicate, particularly in a written form, is like crucial in this new era. And I think it's been a skill that in some ways we've kind of let fall down.

13:25Like, you know, when I think back to the era in which I was a much more active dev, you know, I think there was just so much written communication, whether that be blog posts or GitHub has always been remote. And so for us, it was usually like a GitHub issue or campfire back in the good old days, Slack these days, just just writing down what you mean. That's a skill to bring to saying what I want this app to do. And I think that's why when you're on Twitter or X or wherever and you're looking at, you know, wow, how did this example get one shot? It's like ask for the instruction. That instruction was certainly not build a game, a multiplayer game that allows me to fly airplanes.

14:05Like that was not it. You know, it was much more. But with all the practice that came from describing problems socially, describing problems for your LLM, being able to do that regularly. And I really think it's mainly describing problems as the models have gotten so much better. There's a little less like, how do I make it work for each model than there used to be? That's a skill that's going to serve you both in those tools and with your colleagues, with your manager, with your open source friends and maintainers, just cohesively, if you can do it really well. I'm curious to that. Do you think, just as a two-second follow-up, that for developers, kind of that describing a problem skill that you've been addressing along with kind of the communication skills that support that, should we think of that as developer skills now?

14:55And maybe that is a muscle that we have that we should start exercising as well. Yeah, I think it's something that the best teams, the best companies have considered that. And I think we've kind of let a little bit of the 10x developer meme take over and make communication not be as big of a deal. There's no major application site or app that serves hundreds of millions of people or tens of millions of people where being able to communicate what's happening or what the problems are isn't core to the job of being a developer. And if we just play out over time, you know, if AI and LLMs are going to continue to write more and more and more and more of the code, even if it never hits, you know, all of the code, whatever that ultimately means, all that's left is collaboration.

15:45All that's left is collaborating with your peers, with LLMs, with agents, with designers, with your boss, with your client, whatever that is. And so suddenly the fact that you can write an app incredibly well, succinctly, well factored and tested, whatever, that's great. That's a great skill too. But the human factor will be, I can look at you in the eye, I can read what you're writing, intuit what you're saying, what you're looking for, and describe that in such a way that I can benefit from all of these tools. It's going to be incredibly necessary as those more rote or highly automated tasks can be done by AI tools.

16:41Well, friends, today's ever-changing AI landscape means your data demands more than the narrow applications and single model solutions that most companies offer. Domo's AI and data products platform is a more robust, all-in-one solution for your data. is not just ambitious, it's practical and adaptable. So your business can meet those new challenges with ease. With Domo, you and your team can channel AI and data into innovative uses that deliver measurable impact. And their all-in-one platform brings you trustworthy AI results without having to overhaul your entire data infrastructure, secure AI agents that connect, prepare, and automate your workflows, helping you and your team to gain insights, receive alerts, and act with ease through guided apps tailored to your role and the flexibility to choose which AI models you want to use.

17:38Domo goes beyond productivity. It's designed to transform your processes, helping you make smarter and faster decisions that drive real growth, all powered by Domo's trust, flexibility, and their years of expertise in data and AI innovation. Data is hard. Domo is easy. Make smarter decisions and unlock your data's full potential with Domo. Learn more today at ai.domo.com. Again, that's ai.domo.com.

18:14well Kyle one of the things that that I was thinking about the other day was there's a sort of generation of developers that are growing up sort of not having any other experience than having this sort of AI assisted experience both on the kind of like educational debugging IDE side but also, of course, using interesting tools, whether it be kind of vibe coding tools or other things. I was listening to the A16Z podcast and they did like a, I think it was them, I forget where. It was like somewhere they mentioned a survey of the latest cohort of Y Combinator, that cohort of companies. And they were saying like 95 % of the code is AI generated.

19:03What kind of impacts are on your mind in terms of like this generation of coders that are really like this is what coding is to them what does that mean for kind of both organizations that are hiring kind of developers out of that environment but also you know new opportunities that maybe like people people that wouldn't have maybe broken into developing cool projects or that sort of thing now have have opportunity for yeah i mean you know i look back on how i personally got started coding and it was because i wanted to build a video game and i feel like that's not very unique uh but it's one of those things where like i enjoyed playing video games but it's still cool exactly and i wanted to go build a video game.

19:58So back in the day, I went to probably Barnes and Noble and bought, you know, the red C++ book because you had to learn C++ if you wanted to write a video game. And that thing was, I don't know, 650 pages, probably, you know, that was an enormous book. And so that is a huge immediate barrier to entry to like learning because you're like, the reason I came here was to solve a problem. And if I just do 650 pages of how C++ works, I'll eventually get to build a text-based video game, probably. You know what I mean? And at GitHub with our teams in GitHub Education, I get to work with them and the team there on, well, how do we approach learning in this era in a way where we can bring that problem up front, which is essentially what vibe coding is, right?

20:49I want something in the world. I want to go build it. I think the piece that is necessary to continue to learn is that problem solving piece. And I just want to make it accessible to you so you can bring a problem, something you want to go learn. But in the process of getting you to your destination, we can just expose you to the ideas around why this application works this way or why there's two files, one for the front end and one for the back end or whatever. So you're kind of learning as you go, but still focused on ultimately, you know, solving that problem that you're going after. So I don't think it's a bad thing that, you know, uh, these startups or folks online or even me on the weekend, I'm writing an app that is just for me, it's going to have a user of one in perpetuity.

21:39I just want it to get written. You know, I want it to just work. But if we can help folks learn as they go, I think we'll actually create more, you know, crafts people in a way similar to like, I always describe, you know, changing out a light switch in my house. Like if you own a home, we've all probably replaced a plug or a switch, but there's no way we're going to go into the circuit breakers on our own. We'll probably fry ourselves. So we call in an electrician to come do that. But I'm not an electrician. I just know how to go change the light switches. And that's what I need in order to solve my problems.

22:15That's what I think learning coding in the AI era is going to be, is that you can continue to start from this place of, well, I just want something. And that's fine. I think that's great. And it makes the idea more accessible. I want to be able to get you to that, you know, journey person stage of, oh, okay, I know how this works. I understand variables. New technology came out. Oh, I want to try to play with that, etc. But it's possible that at some scale and speed, we're still going to rely on, you know, professional software developers in perpetuity, running these apps, building these apps, kind of etc.

22:47The real thing that's interesting to me about that stat I was talking to some teammates about is I really think there's a huge opportunity in operating the apps. And I'm a little dumbfounded that that hasn't been something that's been tackled yet. I mean, at GitHub, right, we kind of focus on like you've got to production and like, OK, great. And then you use Century and PlanetScale and whatever, Azure and so on and so forth to run it. but I really think that in all of our probable life experiences as developers the thing that bothers you is you get paged you get an email there's an error thing and you're like crap what is this that is another place that I feel like as vibe coding continues once you run an app and you have thousands or tens of thousands or hundreds of thousands of users I'm not on team like oh well that's when you got to bring in the serious people no rewrite it the right way I really think there's still space to just, okay, well, an error came in, the AI saw what it was, it resolved it, it wrote a test, the test passed, it deployed it to Canary or to a small version, and you just get a text message that's like, we fixed it.

23:56That I feel like is the next step of this, you know, era of writing, learning how to code, writing and deploying these apps versus deploying them and going, uh-oh, now I need a real, you know, a pro to come in and help me out. That makes so much sense and and are you actually seeing anyone out there kind of early people doing some of this uh as is this is this in the wild more than just you know because we tend to think of ai in terms of writing the code operating the app makes perfect sense who's doing it i think the like the issue here is it will require us all to work together so i mean when i joined github uh oh my like nearly 12 years ago now, like I joined to work in the ecosystem on APIs and webhooks and how you connect everything with GitHub.

24:44And I really, that's where my passion lies. It's in, you know, the hub part, you know, of like, how do we get everything connected? And so as I look at, you know, how quickly the industry has gotten so excited about MCP and being able to connect tools together, I'm really hoping this hype wave drives into something valuable, which will be if I can bring the context of my error tracker, my database, my two cloud services, my email provider, etc., etc., all together, then I believe it becomes possible for tools to work together to solve these problems. Unfortunately, right now, each tool is attempting to solve the problem that it can see.

25:31And I do not think that's terribly valuable, right? As an end consumer, I don't want to use three AI tools to solve an error in production. I want one. I want one tool to do that. Or I at least want them in some future magical state where agents all actually work together and blah, blah, blah. Then eventually that could also happen. But I have yet to see a tool that is kind of tackling this, I think, because of the interdependency problem that a tool like that would have in this current very quick moving AI tooling state. Yeah, I think it's somewhat connected to my concern around the ease at which all of this can get built is great.

26:17The burden on the debugging side is potentially growing and you have all this stuff. And then I guess it's more around, yeah, more around decision support in terms of like making good decisions based on like these overwhelming pieces of information because you built just so much stuff and you might not have kind of visibility and intuition around that. What is your thought kind of, because as more code is AI generated, there's potentially not a good intuition even on how things are interconnected or like, oh, this function exists, right? I didn't know that this function existed, right? I've never heard this function name.

27:06I have no context there. So what's needed from a tool standpoint to really get the proper context around that kind of decision support or whatever you want to call it for the developers in the tools that they're working in? I think for most of modern history of software development, I feel like most folks are working in a relatively high-level language. A lot of abstraction, ultimately. Most of us aren't working in C or even lower than that. I think that in order to help us understand our code bases or our multiple code bases and multiple systems, like at GitHub, there's no world in which as a developer who works on uh you know web hooks i'm gonna understand how git uh systems is ultimately gonna work for me um and so for me i think the piece that i'm trying to figure out is how can we get more kind of uh that higher level abstraction of how the code base is working available to me.

28:15And it probably needs to be in a way that as a human, I can like understand how that works more so than, you know, this class, this file, this whatever. I don't really need to understand that. I need to know that the webhook system is having an issue or this other piece isn't working or there's a bug over here where we process images. And then I can kind of click down and dive in and dive in a little bit more because usually when you have a bug even if you do understand the system your goal is to figure out what to ignore you know like you're like okay well it's not any of this stuff it's got to be over here uh and i do think that similar to you know humans being good at describing a problem ultimately to uh the llm i think the llm has to help us abstract up to a level where I would draw on a whiteboard, you know, and then let me double click in and understand more deeply what's ultimately going on.

29:16Yeah. Yeah. That that's a, that's a great point. It reminds me of like the sort of peak microservices days and, you know, everything, everything expanded into it. You know, we're, I was at a small company at the time and I don't know how many microservices we had. And, you know, we had alerting set up, right? But then the alert would go off and, you know, everything was dependent on everything else. So all the alerts would go off. It's either none of the alerts go off or all the alerts go off. And then you're like, well, I give up. Like, where do I even hop in here? Yeah, it seems like a big opportunity.

29:54I guess in terms of the, you know, and I want to talk about Copilot specifically here in a second, but just in terms of the IDE specifically and at a more general level, how do you see kind of the IDE, you know, obviously people are trying various things with both what Copilot's doing and Cursor and Windsurf and all of these things, all hands and all of that. How do you see that interface morphing over time? Do you see that still being recognizable in a year and a half or two years or being something completely foreign maybe to certain people? I'm hoping that you know in the next honestly six months that a startup just because of the nature of how these things move you know can kind of show us a future state that is in some ways backwards compatible.