How Stripe built “minions”—AI coding agents that ship 1,300 PRs weekly from Slack reactions | Steve Kaliski (Stripe engineer)

0:00Steve Kaliski:At Stripe, we're landing about 1 ,300 PRs that have no human assistance besides review per week. A lot of where our work begins is it could be in a Google Doc as we're planning a new feature, or maybe a GR ticket comes in, or we're talking about something in Slack. I can click an emoji, and then the menu will sort of attempt to one-shot resolving that prompt using all the tools that are available at Stripe. When you're in larger organizations, there's so much friction that can come between a good idea and getting it into the world. Not only can I have one of these, but I could have many, many of these running in parallel in isolated environments, making isolated changes all at the same time.

0:33How are you getting all this code review done?

0:35Steve Kaliski:Whether the text has been written by Steve or the text has been written by Steve's robot, you still want that CI environment that's providing confidence that the code that's being changed is safe. And that as it rolls out, we're having blue-green deployments so you can roll back too. All that is super critical, independent of the nature of the authoring of it. No matter how juiced these laptops are, you get three or four work trees in and like it starts to sound like an airplane taking off. It's no good. And so I do think on this multi-threading agentic engineering work, cloud environments and virtual environments are so important to unlock velocity.

1:13Welcome back to How I AI. I'm Clara Vaux, product leader and AI obsessive here on a mission to help you build better with these new tools. Today we have Steve Koleski, a software engineer at Stripe, and he's going to show us how the Stripe team deploys a bunch of minions to do their engineering work. We'll also watch an agent spend a little bit over$5 to plan a birthday party all in Claude Code. Let's get to it. This episode is brought to you by Optimizely. Most marketing teams aren't short on ideas, but what they are short on is time. And that's exactly what Optimizely Opal gives you back. with AI agents that handle real marketing workflows.

1:54You know, like creating content and checking compliance, generating experiment variations, personalizing user experiences, analyzing pages for GEO, even tasks like approvals and reporting. It's your AI agent orchestration platform for marketing and digital teams, plugging seamlessly into the tools you already use, handling the boring busy work, and keeping everything on brand. That leaves marketers with more time to do your actual job. See what Opal can automate for your team by signing up for a free enterprise agentic AI workshop with Optimizely. Find out more at Optimizely.com slash howiai. Attend live and you'll get a free pair of Ray-Ban meta AI glasses.

2:40Steve, I'm so excited to have you on How I AI because I saw the Stripe minions on the timeline. And one, exceptional branding, don't sue us. And two, I just love the idea that you and your colleagues in the team at Stripe have created not just one agent, but minions all across the company that can help with development work. And I'm so excited for you to show us how that helps you in your day-to-day here. So welcome to How I AI.

3:11Steve Kaliski:Thank you for having me. So tell me, what has been the effect that minions have had on you personally at Stripe and at the Stripe team as a whole? Sure. So, you know, for me personally, I think sort of anecdotally, I don't remember the last time I started work in the text editor. Right. So I do end up there often. But, you know, what I found is that, you know, a lot of where our work begins is, you know, it could be in a Google Doc as we're planning a new feature or maybe a GR ticket comes in or we're talking about something in Slack. And those are sort of like the more natural entry points to starting work, right?

3:46Steve Kaliski:And then you end up in a tech center when it's time to actually do the work or make the final tweak. And it just felt very natural. And I think in particular, the sort of like activation energy of starting work feels a lot lower, right? So if you're in a Slack thread and maybe there's a piece of user feedback and it's something simple like we have to update the docs or maybe it's something more consequential and we just want to build a prototype, I can click an emoji and like the work begins. And often the work finishes too. At Strip, we're landing about 1 ,300 PRs that have no human assistance besides review per week.

4:23Steve Kaliski:But at the minimum, the activation energy of starting to write code, seeing tests pass, maybe a test fails, occurs without me even participating. And then I can jump in and I can tweak and I can kind of like have that momentum sort of sort of like generative momentum that I can hop in halfway through. What I think is magical about this, and I won't call Stripe a big company, but you do have a decent amount of employees and very, very large business, is I love that concept of activation energy going lower. because when you're in larger organizations, there's so much friction that can come between a good idea and getting it into the world.

5:03And it's not malintent, right? It's nobody's like, oh man, I really want to slow this process down. It's either functional. I don't have access to a technical area of expertise to actually get from here to there. It's operational. I don't know how to organize people and communicate effectively to get the next step done. Or it's just kind of like people get siloed in their day to day and don't think of new ways to get work done. And one of the things that has been so revelatory about AI for me personally is like, all that just kind of goes to zero because coordination costs can go down, execution costs can go down, communication costs can go down.

5:40You just get closer to the work, which I think is the fun part we all really care about. So show me how you actually activate a Minion? And, you know, we skipped this a little bit. What a Minion is?

5:53Steve Kaliski:The quick spiel of a Minion. When I, as an engineer, sort of in pre-AI time, you know, want to make a modification to Stripe. Well, Stripe is a huge code base with tons of services. It can't run on my computer alone. So Stripe already has a long history of investing in great developer tooling, having hosted development environments that I can spin up, that, you know, have all the code already there and services running, and I can SSH in and make modifications. And we have a ton of great CI tooling around that. So that's the context. We have all that. The idea with the minion is that I can provision one of those environments seeded with a prompt, and then the minion will sort of, you know, attempt to one-shot resolving that prompt using all the tools that are available at Stripe.

6:41Steve Kaliski:So all of our internal documentation, our internal CI, our test data, so on and so forth. And it will loop through that in an attempt to solve that problem. So let's go ahead and jump in and see what sort of a prototypical experience might look like. So I'm in a Slack channel. It's called Steve Klisky Robots-Claire. I actually have a Steve Klisky Robots channel that has 76 humans in it. But I do have every – it sort of is just me and my robots. and now there's some sort of audience observing. But let's imagine that maybe I'm thinking of a new feature idea or I want to improve documentation that we have.

7:20Steve Kaliski:So we have a launch coming up soon, and I want to sort of embellish the documentation. So I'll say I have this cool idea for docs.stripe.com slash payment slash machine. This is our new machine-to-machine payment work, which we'll look at later in our call. and I want to make sure the landing page really sticks and gives a good code example of how to get started quickly. So maybe someone posted a message like that or it came in through a ticket or whatever the origin may be. All I have to do now is add a reaction, which is create minion pay servers. This is a particular repository within Stripe.

8:05Steve Kaliski:we get the one sec cooking from the dev box agent. And then we get a reply in here saying your minion for pay server, it's the repository for a new branch that's created, landing page code example has been created. And it's going to kick off our doc service. So I can eventually preview it. Now I'm going to click follow along. So right now what it's doing is it's provisioning that development environment I was talking about earlier. Right. So this is, this part isn't new. It is excellent, but it's not new. And basically, it's going to spin up an instance in the cloud. It's going to apply all the configuration that's required for both me and the agent to do coding within Stripe.

8:45Steve Kaliski:So this will just take a few seconds. It's going to check out that repository with a new branch, configure the local database, apply my git config. It's going to set up a VS Code server so I could connect to it just through the web or locally. So some extensions. So what's really great about Minions is obviously there's the agent loop that's making the code modifications. But it's built on top of a ton of incredible work that our developer productivity is done around just making it easy to get a perfectly operating Stripe development environment for coding. which means that, you know, not only can I have one of these, but I could have, you know, many, many of these running in parallel in isolated environments, making isolated changes all at the same time.

9:34Steve Kaliski:So, you know, that little one-click emoji, I could have done that with a few messages at the same time, which is really great. Yeah, one thing I want to call it here is we had my friend Zach from LaunchDarkly on, and one thing he said was, look, what's good for the developer is good for the agent. So there's this virtuous loop of if you have or do invest in developer experience for your human engineers, your agents will benefit off of that. And in turn, if you invest in developer experience or agent experience for your agent engineers, your development team benefits from that. And so I always tell people, you know, engineering team, we've always asked, like, can we just give a little bit more time on the roadmap to DX?

10:15Like, pretty please. Can we invest here? And I think if you attach it to an AI initiative, that's like the secret way to get some of that good stuff done.

10:24Steve Kaliski:Yeah, I mean, imagine you're, you know, some code bases are small, but Stripes is huge. You know, imagine you show up day one and there's no documentation and there's no tools and they say, good luck. Like, anyone would have trouble. And even if you threw the agent at it, it's very likely that the context window would be blown by the whole code base. Just scanning through to understand all the intricacies would be impossible or extremely expensive. So if there's a very blessed path for 90 % of the common activities in being an engineer at Stripe, that makes the propensity that the agent succeeds really high too.

11:02Steve Kaliski:So imagine we wanted to make an API change, which we do hundreds or thousands of times a year. We have really good documentation on how to add a new field or a new method or a new resource that the minion would read and would execute against. And then the propensity it would one shot is very high. So good docs for developers are equally important for the agent, to your point. So we've now transitioned from booting up the development environment to now we're in the first agent run. So we have that prompt that I posted in Slack here. And now what it's going to do is boot up an instance of Goose that's basically the harness that's going to run through all this.

11:40We did have an episode with the block team about Goose, this open source agent harness that got set up. And I want to call out one thing for folks that are not watching and are listening, which is I love your system prompt. So sophisticated. it says implement this task completely colon and then just whatever you put yeah no mistakes no mistakes you forgot no mistakes but you know i think people really think they have to over architect their initial prompt and i think if you have a great harness it can go a long way to extracting out um a successful outcome from a pretty loose prompt totally and we a lot of this

12:19Steve Kaliski:is an experiment in some way, right? As new models come out and we build new tools, there is this sort of dynamic nature to it. And we've built a lot of interesting bots that help write the prompt, right? So maybe first it will do the task of searching through the code base or looking at other pull requests or Google Docs or whatever it may be. I think now it's straight. Most things that could have an MCP server have an MCP server. So we're able to interact with a lot of the internal data we have. And then it can make a prompt that I could then paste in here or I get assigned to the agent. So that's sort of, you know, part of why I wanted that public channel we were looking at is like, you know, we're going to see that we don't pair program anymore, but we, you know, pair prompt, right?

13:01Steve Kaliski:And that activity could be with other engineers or other data sources or other agents too, right? To like figure out if we can, you know, properly explain to the agent, you know, how to do it correctly. In any case, you know, what it's doing now is it's taking the link I gave it, which is to public documentation. It's going to search through the code base and use some of our code searching tools to locate where that change in particular should go. It's going to execute a whole sequence of tools. And over time, as it figures out where in the code base it should work, what the modification should be, they'll ultimately commit those and make those available in a pull request that me and my fellow colleagues can review.

13:42Yeah, I have a couple of questions on this because we've seen a few examples of folks building their own cloud agents and kind of, and I'm curious, you know, why Goose, you know, versus doing something on your own or doing sort of a more commercial solution. I'm curious if there was an internal discussion or how this, or did this happen organically because it worked for one engineer? Curious how you kind of seeded the idea of minions on top of your development tools.

14:12Steve Kaliski:Yeah, sure. So we also make Cloud Code and PerseSuer and tools like that widely available to engineers at Stripe. So I think our general sentiment is like we want to accelerate development so we can build new features for our users. And there are going to be new models coming out, new tools, and we want to be able to proliferate those as much as we can. In the particular case of Minion, it's very, I don't want to say very specific, but it's very specific to, like, the Stripe developer experience in the Stripe developer environment. And we had been experimenting with Goose early on. And I think in this particular case, we'd forked it to make some modifications as well.

14:51Steve Kaliski:And really what we were looking is, like, sort of a base harness and loop to apply all of our own tools and software to. So we spent a lot of time on making good tools available and making sure that the routes that the minions go through work closely with the most common Stripe developer workflows. So it's sort of like commercial versus custom things. There are things that are very specific about Stripe's code base and being a developer of Stripe and the way we build things that it was just sort of easier for us to build and deploy that. But the commercial solutions are great, and we use those extensively.

15:28Steve Kaliski:And even later on in this demo, I can sort of show, like, I can, you know, for example, I can pop into VS Code Web, where I could manually edit some of the code that's going on here as well. But I can also boot up Cloud, and I can have sort of the typical Cloud experience with all the Stripe MCP tools, internal Stripe MCP tools available as well. So, you know, there's no singular tool to rule them all. But I think the overall end-to-end development story at Stripe is built on minions. So you can see I'm in that dev box and caught now. Yeah. Cool. I have one other question and then an observation I want to make sure that the listeners don't miss.

16:05So my first question is, you know, Stripe is a very well-resourced, I would say, engineering organization. So I'm presuming you have a team dedicated to working on not just your dev tools, but as well as minions and managing that as an internal product itself. Has that team been sort of built as a standalone team that's focused specifically on internal developer experience? Is that how it works?

16:29Steve Kaliski:Yeah, we've had a developer productivity team for as long as I can remember. I think about six and a half years now. And, you know, that team is focused on all the tools that I engage with and making them more useful. Right. So that's all the way from how we interact with with Git and version management to our tech centers and our configurations there to our development environment and how that whole story pieces together. And, you know, we, you know, just as, you know, as a product engineer in Stripe, I care deeply about our external users and them being successful in Stripe. That team cares equivalently about engineers at Stripe being successful and being able to build things quickly.

17:09Steve Kaliski:And I think that's been even more accelerated by AI in the last couple of years. And then one other observation I want to make, because I think you glossed over it a little bit at the beginning, but it's so important for folks that really want to go ham on coding with AI. Sure. Which is, look, all of us engineers have a MacBook Pro that weighs 8 million pounds. That can do some damage. Mine, for anybody who wants to know, its nickname is Big Boy. So whenever I need my kids to get my coding laptop, I say, can you bring me Big Boy? Because I call it San Francisco rucking when I carry two of them in my backpack.

17:44Steve Kaliski:Oh, my God. But, you know, no matter how juiced these laptops are, you get like three or four work trees in all running. and like it starts to sound like an airplane taking off. It's no good. And so I do think on this sort of like multi-threading agentic engineering work, cloud environments and virtual environments are so important to unlock velocity. And that's one place where I haven't seen enough large engineering teams invest in those environments to really unleash the power of either AI-assisted coding for their software engineers or agents in general. So if there are any CTOs, VPs of engineering listening, if you were to invest in something to really unlock growth in the next year, getting that situation locked up would be really good.

18:35Because again, I hear so many people be like, oh, I can cloud code everything. I can codex anything. I can spin up all these work trees. I'm fine. And I'm like, are you running all these local? What are you doing? And so that's one thing I just want people to not miss is the limitations of your actual machine on how multithreaded you could be, especially in a complex road base like Stripes.

18:58Steve Kaliski:Totally. And, you know, I have Slack on my phone, right? So I can even kick off one of these minions on the way to work, right, as I'm sort of going through Slack on the subway. And then, you know, by the time I'm there, I can jump in halfway through. I think, like, maybe, like, the hyperbolic thing here is, like, imagine if all engineers at a company could only, like, work on, didn't have Git, We all had to like coordinate working on the one code base together. That would be crazy. And, you know, the equivalency here is like imagine if I'm bounded by, you know, my agents are bounded by just what's available and can work on my computer.

19:35Steve Kaliski:The 10x thing to do is, you know, be able to have 10 of them run in parallel, but also not be contingent on my like. It's like everyone's buying a Mac, a Mac mini. Right. So it doesn't fall asleep. Right. Right. It's like there's a whole business around just the computer not falling asleep. I legitimately, first of all, I have like four Mac minis upstairs and one of them is just basically a laptop that doesn't close. Like I use it as a laptop that does not shut. And it's really unlocked my my velocity. So, OK, we thank you for going on this side quest about virtual environments and local host and all those things.

20:11I'm a founder, so I know most people don't start companies because they love running payroll or managing compliance. But somewhere between hiring your first employee and raising your next round, you end up in the weeds with HR, IT, and all that other stuff. That's what Rippling was built to solve. Rippling is a unified platform that lets startups run HR, payroll, IT, and finance in one system from day one. The Rippling startup stack replaces disconnected tools that don't sync with a fully connected platform. Over 15 ,000 startups, including Cursor, Clay, and Sierra, trust Rippling to scale fast without adding additional ops and HR headcount, so founders like you can keep building.

20:53Right now, venture-backed startups can get six months of Rippling startup stack for free. Head to rippling.com slash howiai and sign up today. that's r-i-p-p-l-i-n-g dot com slash howiai to sign up for six months free today focus on what you're building and leave the rest to rippling okay so you are now running this you're going to it's it's you said one shot at the beginning really you're trying to take one one prompt and not a single reply gets you gets you what you want but it goes into the harness it goes through its own lube, hits the tools it needs, and ultimately you as the end user get one response back, which is here's the successful implementation.

21:39Steve Kaliski:Exactly right. So we can already see that it's identifying the relevant files. It's keeping track of its own to-dos. That's something that we've codified in it to focus on. It's making changes. It's preparing the commit and so on and so forth. And ultimately, sort of like taking out of the oven, we'll see a response at the end of just like it finished, you know, you can go ahead and look at the pull request and the sort of normal human review part continues. So let's talk about that really quickly. You said 1300 code or agent initiated PRs per week, something like that. And then humans are involved in code review.

22:16How are you getting all this code review done?

22:18Steve Kaliski:Well, you can make the argument that, you know, if I'm spending less time actively writing code, I can recenter my time on reviewing the code that's being written or working with users and so on and so forth. So I think that's a big part of it. I think the other side of it, it comes back to that CI environment. So having really good test coverage, having synthetics that run to simulate end-to-end interactions with your product, those all help inspire confidence in the code you're reviewing. Right. So absent those, like it'd be really difficult to look at code, especially in a huge code base and have high confidence that it works.

22:55Steve Kaliski:So, you know, again, whether the text has been written by Steve or the text has been written by Steve's robot, you still want that CI environment that's providing confidence that the code that's being changed is safe. in that as it rolls out, you know, you're having sort of blue-green deployments so you can roll back too. Like, all that is super critical, independent of the nature of the authoring of it. I do believe, like, if coding becomes easier and coding historically has been the bottleneck in product development, it's just going to shift to other areas, right? So if, like, coding in effect becomes free, the review is going to be really challenging, right?