How Devin replaces your junior engineers with infinite AI interns that never sleep | Scott Wu (Cognition CEO)

0:00Devin is async. Once you kick off the Devin session, Devin's going to start working and looking through the code, but you're not expected to be there with it. It's just as if you gave your intern a project and your intern is going and working on it. Devin's my favorite intern on my team and I have infinite of them. Why don't you pick a task that you might bite off for your product and show us how you would work through that end to end. I'll say, please go research the chat PRD MCP server. This will produce a pull request for us. Often you're running a few of these at once. It's just like a nice way to have multiple tasks going and then check in on each of them.

0:32One of the benefits of this from a How I AI use case is you can multi-thread a lot with tools like this and set two, three, four, five, ten of these going at once on different projects and not feel like you have to sit there and babysit things.

0:49Welcome back to How I AI. I'm Claire Vaux, product leader and AI obsessive here on a mission to help you build better with these new tools. Today is a very special episode for me because we're talking to Scott Wu, CEO and founder of Cognition Labs and the builder of one of my favorite AI products, Devin. We're going to hear about how Scott uses DeepWiki and Devin to kick off well-scoped tasks to get things done, uses Devin as his favorite and most tagged employee inside of Slack, and how he's making it not weird to bring a chat GPT voice into your meetings. Let's get to it. This podcast is supported by Google.

1:29Hey everyone, Sreshta here from Google DeepMind. The Gemini 2.5 family of models is now generally available. 2.5 Pro, our most advanced model, is great for reasoning over complex tasks. 2.5 Flash finds the sweet spot between performance and price. And 2.5 Flash light is ideal for low latency, high volume tasks. Start building in Google AI Studio at AI.dev. Scott, thanks for joining How I AI as Devin's number one reply guy on X. I am really excited about this conversation and for you to show off how your company uses and you use the product that at least makes me very happy and I'm sure makes lots of software engineering teams out there very happy.

2:16So welcome. Thank you so much for having me. Now I'm honored to be here, honestly. I'm a big fan of you guys and all the work you do. Great. Well, we want to get into, we have lots of stuff to talk about, but what we really want to do is get into how you AI, and in particular, how you AI with the products that you've built. And I think what's really fun as somebody who's building AI products, it's something you get to use every day and get really good at, but also probably show some of our listeners and watchers some tips and tricks about using the tools that you've built that they may not have thought about so far.

2:55So we're getting the expert look into how to AI with the Cognition product. So what are you going to show us first? And what are some of your common workflows when you're doing engineering work or trying to move the product forward? Yeah, for sure. No, for us, it's definitely, I mean, as a bunch of programming nerds ourselves, you know, building an AI that can code is, has got to be one of the coolest things that we could probably spend our time on. I wanted to show a couple of flows actually of how we use the Devon stack, because there are a few different pieces involved. There's Flack and Linear.

3:25There's the wiki, obviously. And then there's like AskDebit. And then there's, you know, starting Devon sessions and getting call requests out of it. I think there's some real, I think there's some real nuance and like, what are the right flows of like, how do you work with Devon as an employee? Because I think it really is quite different from a lot of the tools out there, which are much more kind of like an IDE, for example, or like a terminal UI. Like Devin is, I think, first and foremost, almost like an engineer on your team. Yep, totally. So what are some of the things that you reach for with Devin and the capabilities that you think really make a difference for you as a software engineer?

4:03The way that we like to describe it is Devin is a junior engineer. And so Devin is not going to go. And, you know, we're working on getting Devin to senior engineer, obviously, you know, we'll get Devin the promotion and everything. But like, Devin is not going to go and solve some, you know, really hard architectural problem or make some big strategic decision that you know, you're going to make and then kind of like execute on for the next month, like you probably want to be involved in those as well. Well, Devon can help you with the decision, obviously, by kind of like referencing the right things or giving a few things as an input.

4:34But I think where Devon really shines is one way that we say also is kind of like tasks, not problems. And so often when you have a very clear, like, here's exactly what we need to go do, and here's the task, and here's all the details of what we need, Devon is really great at going and executing that for you and makes that much faster. And so naturally, I think the next question that comes to mind then is like, how do you figure out the spec or the, you know, the, or, you know, the, the task exactly that, that you want to do. Um, and so a lot of the other tools like Wiki and search, you know, really, um, are, are here for you to be able to kind of like ask the right questions that you want about, um, Shannon, the code base or, or, or what needs to be done.

5:14Um, and then putting a task together. I think in practice for a lot of the, the use cases that we see all the time are, um, you know, probably number one is just crawling through your issue backlog. You know, whenever you have an issue that comes up, or, you know, we have a lot of Slack channels where we talk about issues, and then on every single one of them, we just tag Devon as the first pass. That's a big one. And so, like, you know, someone says, oh, you know, we need to go fix this thing in the front end, or, you know, maybe we need to go support this other, you know, support this other MCP, for example, which we'll show in a second, things like that.

5:45And then for a lot of the other kind of, like, I'll say, like, engineer and toil use cases, it also does really, really well. And so often that's like, you know, going and doing a version upgrade or added documentation throughout, you know, your repo or adding unit tests for a specific thing that you have up or responding to, you know, a crash report that just came up and trying to diagnose what went wrong. Yep. I love that. What you said about, you know, Devin's a junior engineer. I say Devin's my favorite intern on my team and I have infinite of them. And then I like this idea of scoping task, not problem.

6:22And I do think it's something that people are working with AI and even other AI tools, not in the engineering space, really thinking about task level orientation sets you up for success, or at least a sequence of tasks can be very helpful. And so why don't you pick a task that you might bite off for your product? And show us how you would work through that end to end. Yeah. Yeah, let's do it. So as you might know, I'm a huge fan actually of Chat Priority. And the natural thing that came to mind for me was we need to integrate into Chat Priority's MZP server. And so I was looking into how to do that with Devon.

7:04And so the first thing that I always kind of go to as an initial thing is what we call the dbookie, which basically for any repo, this is true for public or private repos. You can come in and get a full AI-generated documentation of the repo. And so in this case, this is the Devon web app repo, appropriately, they're not too sensitive here. But it's basically, it explains Devon, it's pulling a lot of information from the readme or understanding the system architecture. And I can search this and pull up different things. And so if I understand how the MCP marketplace is set up, it'll point out what particular components there are or what particular files are called here.

7:47And I can read up on this and kind of understand exactly how this is set up. And the natural question here that I might ask is, okay, cool, just show me where the MCP server list is implemented. and to this we'll look through our repo and Devin at this point has done a lot of work in the DemWebApp repo for StandardBird and so that helps a lot which is Devin builds this representation of the code base over time and we can see what's going on here it has all this and so you're getting both sort of a natural language explanation of how this server list is implemented and then you also on the right side of this for folks that aren't watching get the actual code snippets and reference files that you can view and really understand the deep layer of the code.

8:34So you have like sort of a combination of, let me explain how it works. And then this is the nitty gritty. Yep, combination of English and code. I think it's an interesting one where it's like, you know, some day it'll probably all be English, you know, but I think especially now, you know, in this current period, I think we're really in the era where, obviously, you have to pick up. You as the engineer want to be looking at BERT, English, and code. And you can see here, it's giving you kind of the answers of what's going on. And in particular, it'll point out, okay, here's our list of all the different marketplace servers that we have.

9:04And we have an Atlassian, MCP, and there's a HubSpot, MCP, and so on, right? And from here, the natural thing that I'll want to do here, which is what we found to be a big flow for folks, is to use this to produce actually a prompt for depth. And so the whole idea is now that we're in this context, You know, we know what the questions were. We know what part of the code base that we're looking at. It gives a lot for Devin to be able to start from. And if we have an app for PASC in mind, then we can get that going. So I'll say, please go research the chat PRD MCP server and add that to our list.

9:48And so what this will do, I use basically construct a Devin prompt from this. And so this has my prompt here, which I just typed in, which is not super refined, but it also has all the detail about the part of the code that we're in and what components we're looking at and so on. And so then it will generate for me this prompt in Devon that I can just go ahead and use immediately. And you can see here, you want to follow the pattern of existing servers like Atlassian and HubSpot. Here's the exact type dip structure that's being used here. here are the functions that you should be working at. And here's what you should check to make sure that it works.

10:27One of the things that I want to call out for folks in terms of a workflow that they should think about is a lot of people, myself included, sorry, Devin, would have just sent that prompt, which is add, you know, add chat parodies, MCP server to the list. And I do think that one very short but important loop of take this prompt and turn it into an effective prompt, given the context you know, and then sending that into the task to do just saves you a lot of heartache. And it feels like extra friction at the time. But I think pretty soon is one going to be the job to be done of the tool itself.

11:08So does that like loop become invisible either through these reasoning models or some application layer that manages it? And two, it's just worthwhile for people to do. So this, you know, when you're thinking about sending a five word prompt, think instead saying, here's my Fiverr prompt, build me a better prompt and sending that into your system. Yeah, for sure. And I think it's, you know, it's a great call because, you know, as we said, Devin is async, right? And so from this point onward, the nice thing about this is, you know, once you kick off the Devin session, Devin's going to start working and looking through the code and reading online about chat BOD, for example, right?

11:39I'm going to do all this, but you're not expected to be there with it, right? And so, you know, it's going to work on its own. It's just as if you gave your intern a project and your intern is going and working on it. And so, you know, they can ping you on Slack and ask you if there's questions or something, or you can go kind of like, you can go take a quick look and see, you know, how your intern is doing. But you don't have to be sitting there with Devin for every step of the way here. And so one way that we kind of describe it is, you know, for a lot of tasks, there's often this sync component, like a synchronic component, and then ACPRIS component, right?

12:11And a lot of what search and wiki is for, is for doing the synchronous part of the task before you do the async, right? And so like, if you had an intern, for example, would you just send them the five word Slack message and just leave it at that? Maybe sometimes for something that is like, you know, super clear, and then you know exactly, often what you actually would do is you would sit down with them, talk it through for two minutes and be like, okay, yeah, like, you know, you know how we have this MCP marketplace, and then we go and look at it together, you know, we read the particular line of the code, and then you say, okay, yeah, so let's let's add chat PRD to this.

12:42And, you know, just go take a look at how that MCQ server is implemented and make sure we add it to the list. And then you kind of hand off there, right? So you kind of have the first two minutes of going back and forth with Devin, your intern. And then as soon as you hit go on the Devin prompt, you're kind of expecting it to be more of an ASIC or SYN where you don't have to be in the loop. Well, and one of the things I want to call out for people that are building AI products out there, you know, like you, like me, is in these sync products, latency really matters. People get really frustrated with wait times.

13:12But if you set up your product to really be this asynchronous modality, you actually buy yourself a lot of user love on waiting time because there's not that expectation. Just like you would not say, hey, intern, okay, now go research this other MCP and do a PR for me and come back when it's ready. You know, just like that would not be something you expect an intern to come back to you immediately. You also, from a product perspective, don't expect Devin to come back immediately. Now, one of the benefits of this from a how I AI use case is you can multithread a lot with tools like this and set, you know, two, three, four, five, ten of these going at once on different projects and not feel like you have to sit there and babysit things.

14:00And so I'm wondering, you know, while this is running, do you go pop off and go to a meeting or get a coffee? What has this sort of like asynchronous workflow enabled for you? For better or for worse, I'm in meetings for a lot of the day. And it's great to be able to just kind of kick these off or, you know, you have an issue backlog or, hey, there's these three or four things I was hoping to look at today, right? And you click off each one with Devin. And then, you know, these go and work asynchronously. And it'll make the pull request for you in GitHub. And it'll kind of show you the diff and what work it went through.

14:33If it's like a cron change or something like that, it'll send you the screenshots. of what, of the before and after, right? You can see it's going and researching chat PRD. Well, I will say just my, clearly my SEO on the MCP is not good, but Devin did make my MCP homepage. So it's in the top nav. Yeah. That's funny. Which is a hard one for me, so it should know. Yeah. Cool. So this is great. So I think for sure, you know, often you're running like a few of these at once. And like you said, it's just like a nice way to be able to kind of have multiple tasks going and then check in on each of them.

15:09Yep. And so what this would do, and maybe we can come back to it later when it's done thinking, what this is going to go do is it's going to go do research. It's going to find my docs page on the MCP server that Devin did make for us. And then it's going to pull that docs in and then you're going to get actual code out of this. Your goal for this is to get a PR, right? yep so this will produce a pull request for us um and then from there i'll be able to review the pull request and then if that looks good then i'll merge it and then obviously we'll we'll have this out in the next step amazing and then your prompts are going to be so much better and i'm feeling guilty so i am just going to slack you uh the the mcp homepage and you can give that to devon to go Yeah, sure, sure, sure.

15:55You're getting a true live demo here. Yes, yes. This is when your intern comes back to you and says, hey, I was looking this up and I couldn't find it. Can you point me to where? Okay, you have it. Chatprd.ai slash product slash MCP. Okay. Has code snippets and everything. Okay, here we go. Great. So this is a good example of you've done your research. You use that research to create a better prompt. You use that prompt to kick off a task. That task is worked asynchronously as a sort of like more junior engineer would work, including doing research in your code, external to your business. And then it's going to go ahead with the context of your repo and do a PR and ship this feature.

16:43And otherwise, you would have had to like ask somebody to do this. And I think about, for me, I think about the people that you'd have to involve in something like this. Like you'd have to go find the senior engineer that wrote the MCP server code and say like, please explain it to me. You'd have to, you know, get, you take the time to write out that nice spec of this is what you want to do. And then you'd have to task it to somebody to actually implement. And so I think you can press that workflow of a team of like, you know, even three people's time into, you know, about 10 minutes to get something done.

17:21Yeah, yeah. No, and I think often a lot of the folks who we see who really, really love Deader and use it this way, especially are, you know, folks who are like tech reads or product managers or things like that. You know, it's a great kind of intersection of one is, you know, on the one hand, you're already used to the flow of kind of like, you know, figuring out an issue and getting into what is going on there and then handing off something, you know, here's exactly what we need to build, right? And then I think two is naturally like the async workflow for people who are in meetings or have a lot of back to back going on is just a great way to kick off and check in on tasks quickly.

17:56And so from starting things from the web app or serving things from Slack, for example, is a nice lightweight thing. If you're not in your IDE all the time, you can start tasks from the IDE as well, obviously. But we see this kind of flow a lot with leads and PMs, basically, who are going back and forth with a lot of things. Yeah, and one of the things that I've been telling people more and more is as part of your PM onboarding, you should be now giving everybody access to GitHub, which isn't something that typically happens in a lot of product organizations. Giving access to GitHub, giving access to tools like this, because I think it does enable product managers to do a lot more.

18:39So while this is running, what I wanted to talk about is, you know, before we got into the show, you and I are saying you're just a little bit busy, you know, over the last month, just doing a few, few interesting things with the business. In addition to, I'm sure, wanting to build and spending time with the team. And so, you know, this asynchronous nature and this junior engineer on demand, how do you actually use that day-to-day to just keep a float on top of all the stuff that's coming in your team? You know, not, I have a feature I want to build, let's go build it. We just saw that flow, but like the kind of reactive stuff in your company, how are you using AI to stay on top of that and keep the velocity high?

19:19For us, a lot of it is just setting the right workflows in our Slack and in our org and so on. And so, you know, Devon obviously has knowledge, which means it'll learn your code base over time as we keep working with it, or you can kind of give it more details about how certain things work. And a lot of things, it's almost just like institutionalizing Devin as first line of response is how I would describe it. And so I could show a few examples. The big thing is to really get to the point where for a lot of these different things that we file, you know, like Devin is the first person that gets tagged on all of these, right?

19:54And Devin won't be able to do every single thing, on one shot on the first try, but often you're working back and forth with Devin and Devin puts up a PR and there's some slight touch up that you have to do at the end or that you have to build, then you're able to do that. And so we have a ton of channels where we go and talk about issues or various things that we need to build or things like that. We have one for all the crashes that come in. We have one for kind of like core infrastructure things that come up. We have one for, this is the one for our web app, which is hopefully a little bit less sensitive.

20:27And you can see here, basically every single thing that folks talk about and we do, we start in Devon Fetch. And so it's like, hey, can you standardize the font size, spacing and style for these three levels, right? And then we just go and start the Devon Fetch and Devon will make the PR. It'll go through the PR. This one gets merged because there's some back and forth feedback here. And so like Devin goes and edits, he can import this up and see. And so Devin made this BR, there were a couple back and forth edits, and then Dave, our engineer, went and merged this. And this is often how it works.

21:09You know, it's how this is another good example. Hey, Devin, can you make it to that when you command click on the notification, it takes you to that in a new tab, right? Now for feature, probably one of our users requested it. And you just started Devin's session. and Devin will give you this progress update. Here's what I'm doing so far. Here are the files that I'm looking at and here's what I see. In this case, by the way, it's actually confidence medium. And then we'll say, oh, no, no, no. Like, you know, you should take a look at this thing instead. One of the cool things I want to point out too is because of this, Devin is a naturally multiplayer experience.

21:43And so we will often have a few different folks going back and forth. If somebody else is looking at this issue or if somebody else is the expert on this part of the code base, um they'll go and give their own kind of input here and devin will just go back and forth with them as well and so really it is just a thread where a group of you are communicating and figuring out how to how to work on this issue and devin is just one of the players in threat right and so you know even comes into walden's thread here and says hey make sure to use a link element from tan stock router uh um and then gives that feedback right and then devin goes and makes that change in the pull request.

22:19And so you can see Devin had like an initial thing and then had some additional commits and it went and did this link from Tanstack router instead. As an AI founder, you're used to sprinting towards product market fit, your next round, or that first enterprise contract. But speed isn't enough for AI startups. Buyers expect security, compliance, and transparency from day one. That's why serious AI startups use Vanta. With deep integrations and automated workflows built for fast-moving AI teams, Vanta gets you audit-ready fast and keeps you secure with continuous monitoring as your models, infra, and customers evolve.

23:04AI innovators like Langchain, Rider, and Cursor scaled faster and closed bigger deals by getting security right early with Vanta. Listeners can claim a special offer of$1 ,000 off Vanta at vanta.com slash howiai. You know, one of the things that I like about this, and again, kind of a shout out on our use case for folks that are trying to drive more AI adoption in their teams, is doing this as much as possible in public is really helpful from a learning perspective. So one of the experiences I had running the engineering team at LaunchDarkly was when we started putting Devin and Devin-like agents in public channels, we saw a lot more adoption and upskilling of our team on how to actually talk to these agents, how to get the right outcomes.

24:04And so, you know, I we were talking earlier and I was saying I DM Devin all the time because I have no employees, no one to talk to. He's my only buddy. But I DM I DM Devin all the time. And we have these sort of like side conversations. He's sort of my intern on the side. But in larger organizations, I was very much a do it in public channels, do it where people can see it. because not only does the work get done and it's nice kind of muscle memory to tag in these tools immediately, but also just learning how you use them. What is an effective prompt? What are the kind of things that it's good at and not good at is really useful for just overall engagement with these tools.

24:47And so I think hiding your AI use is kind of the worst thing you can do. You can do it in work. So I say do it all in public. Yeah, yeah, yeah, yeah. And I think there's two sides of it that I was going to say, work, work. One is like the kind of like, when we talk about these multiplayer experiences, right? I think there are two benefits, right? One is this kind of like, the knowledge transfer for the agent itself, which I think more and more products are starting to have, which is, you know, one person uses Devin or uses this tool or that tool, right? And that adds to the knowledge of the tool itself.