Build your own company brain: the enterprise AI playbook from Stripe’s engineering team | Sharadh Krishnamurthy

24:05and the fact that they have a sandbox that anybody, regardless of whether they're an engineer or not, can get the agent to write code for them and do whatever the heck they want with the data, it's really powerful. And I'm pretty sure that, again, as expected, it's gone in, it's all like said, okay, here's the actual data and I want you to go do some summation somewhere to do the other tab and out again. So super interesting. And if I open up the updated dashboard, It's the exact same dashboard. And you should now see this really cool little segment below that. So I could keep yapping about this dashboard.

24:45I love the fact that our marketing team is like 100 % all in.

24:50Sharadh Krishnamurthy:They need it, right? I don't know a single marketing person that doesn't either want some sort of app built or some sort of dashboard. So I think you have product market fit. That leads me to my next question, which is, how do you roll out? I'm just curious, kind of, you know, inside the doors of Stripe, how do you roll something like this out? Is it really organic adoption? How did it get built? How did it get shared to the team? Was this like 20 engineers? Like, how did this come to be? Definitely not 20 engineers. Stripe, we run fairly lean and very nimble and very fast. We built this super quick.

25:31it's a very interesting case because so we had like say me and my colleague we had the idea I was moonlighting as an engineering individual contributor again trying to get this out the door and so it took us like one and a half people over two weeks to get VZ out the door and something we realized was a lot of the questions became the answers became apparent once we could show people something it was very hard to tell people why something like this is required in a world where you had the coding agents around and they could be like super powerful. But the moment we got that V0 out, super inexpensive, one and a half engineers for like two weeks, right?

26:11V0 out. And then we moved into a pilot stage where we started seeing a lot of interest from primarily, we have a great collaborator, Ilia on the GTM team, who builds like AI for GTM, right? And they were like super interested in this because like marketers, our go-to-market function, it's right, like they are extremely, like they're looking for whatever they can do to reach more people, to reach them more in the right manner and so on. So they saw a lot of adoption from them and that's how it kicked off. It went into this pilot stage. We still had about 200 to 300 users. At this point, we had like two and a half, three people working on it and this was the next like month or so.

26:56So things really ramped once we did a company-wide demo saying that, hey, we built this thing. We invited you to use it. And it just clicked for everyone. And people started, that's when you see the really steep ramp up somewhere, I feel. And everyone started using it. Even then, the team itself, I wouldn't say is humongous. Like, yeah, 10 ,000 plus people use it every week. But the core team that manages the experience is still like less than 10 people. And we have a lot of other things we have going on with those 10 people as well. And the things that let us build it, of course, coding agents and the productivity that they've given us and our DevFra team are incredible.

27:39We've spoken about minions on the show before. We have incredible tools that strive to get more from who we have. And we have all this infrastructure that you referenced. And all that has helped us. So I would say it's less than 10 people, but I also want to give credit where it's due. Like there are a lot of people helping those 10 people do what they can do.

27:58Sharadh Krishnamurthy:I love it. This episode is brought to you by HyperAgent, the platform for deploying always-on agents that actually run your business. With HyperAgent, you build agents in the cloud and deploy them where your work already happens, like Slack, Telegram, or email. An agent will scan your inbox and draft replies to vendor follow-ups. Another monitors competitors and spins up rich ad kits and landing pages. A third notices a deal going cold in Salesforce and writes the save email with full account context. These aren't chatbots waiting for a perfect prompt. They're proactive, learning your preferences, retaining your playbooks, and getting better with every run.

28:38Sharadh Krishnamurthy:One user built four agents to run an outbound sales pipeline, prospecting, outreach, follow-ups, CRM updates, all in a single afternoon. No local setup, no VPS bills, no fragile permissions on your laptop, just powerful agents with full control over skills, tools and guardrails. How IAI listeners get$100 in free inference to start building. Claim yours at hyperagent.com slash how IAI. What else, maybe one or two other things that you think are worth pointing out in CHI that you think make it pretty unique or at least, you know, fun and easy to work with? I'm going to do two things. I'm going to show skills and I'm going to show projects, right?

29:21So when I come to say skills, we have this thing, it's great. Everyone loves the dashboard, but I don't want to be creating this dashboard and paying a bunch of tokens and time every time. So the thing that I think I did really well and one reason for its product market fit was I can create a skill that basically takes what I've done this session and packages it up so that it can become a load-bearing, repeatable workflow, right? And that's when the AI goes from, here's something I'm just like iterating with on the site, like a chat interface, to here's something I can trust to run my, to sort of like run my business or run my workflows or help me do that.

30:01And a little bit after, you're going to see this kick off this like skill creator skill. It's a skill that the harness has that's going to go in and create, like take all the things it's learned from the session, from its interaction with me, and package that up nicely into something that I can just like pull up at any time. And we'll talk a little bit more about how we do the skill retrieval. I think that's a really cool part of the system as well. But while that's cooking, right? Let's actually look at this other thing I love about how we built Kai, which is this notion of projects, right? So the thing about projects is there's so much stuff happening at Strap.

30:39We have like 2000 skills, right? Projects are this really nice packaging mechanism where we can draw a boundary around those skills and say, these are what most people who are doing this workflow must be using. We have projects that are created for projects like short-lived things. We have projects created for teams. Like the people team has a super secure version of Kai in a different project that's backed by a totally secure backend and stuff. The thing about this is, again, it lets one person or a few people who are DRIs of the space to figure out how to get the agent to perform well for everyone.

31:15The really interesting thing I have on projects, there's this thing called settings that, like I said, you can use it, you can use a custom agent to power your project. It doesn't have to be Archive, which is pretty good and general purpose, but let's say you have something really bespoke. You can use all the same features we have, but just backed by a different API in the backend and a different harness, right? So really, again, when we talk about AI at the enterprise, there's going to be heterogeneity. It's going to be a lot of different cases and building this in layers so we can give maximum leverage and customizability.

31:49One of the cool things about projects that can be very concrete for people is the idea of tool policies. Now, I mentioned the people team. Let's say you're a person on the HR team who's dealing with a bunch of sensitive information, right? You really don't want the agent to sort of go rogue and put that sensitive data into some public Google document that all Stripes can access. That seems like an accident waiting to happen. We don't like that. But you also don't want to tell them, oh, you can't use any tools because your workloads are too sensitive, right? So what projects let us do is to say, for this workflow, I'm going to set up a tool policy that says, in this case, I said the run Hubble tool so that I don't inadvertently put some confidential information into the demo, right?

32:36but we could test this out with some other tool and it's going to kick off what is a is a human in the loop workflow so create a calendar invite for me and Wong Morrow at 11 a.m. Pacific right and I set this up ahead of time just to show what a human in the loop flow would look like but you can you can see it extends to any other kind of tool what this is going to do is going to tell me hey should be familiar to most people who've used like Cursor or the other big products out there. But I want to do this, right? The interesting part and why this isn't the fun part. The fun part is what I showed before, which is that someone who is the DRI of a space can decide that certain tools are kind of sensitive for the workloads that these people are going to be using.

33:29So we need a human in the loop to confirm if that action can be taken by the agent. Agents are really good. They're very creative. So we've got to put some restrictions on them so they don't go rogue, right? And projects help us decide. I'd also don't want this to be happening for every person at Stripe. That would be kind of frictionful. So projects are, again, drawing a boundary around it.

33:48Sharadh Krishnamurthy:Yeah, I love this because I think a lot of the existing tools let you maybe configure some of this at the individual level, but then it applies to every session. It's not contexted to what you're working on and you can't share that permission set across different users. And so what I think is interesting about Kai is the like permission and context boundaries are very purpose built for how your company works on things. And I think, you know, when people are asking themselves either, should I build something myself and does that make a lot of sense? Or do I need to pluck something off the shelf for my enterprise use case?

34:28Sharadh Krishnamurthy:Again, you need to ask yourself how much appropriate or inappropriate friction will this put in everybody's day-to-day work. Because at the end of the day, what you want to do is make everybody's life easier without causing chaos or trouble. And you want the management requirements to go down really low, right? You don't want everybody to have to think every task like, do I need to turn on this connector, off this connector, connect to this data? And so I do think one of the benefits right now of teams building their own thing is they can really think about bespoke agents for bespoke use cases, but kind of hide all that complexity from the end employee, the end teammate, and just let them get to work.

35:12That's super insightful because as you were speaking about bespoke agents, and I showed a little bit of this earlier, Kai looks like a single product. It really isn't. It's like the icing on top of a multi-layer cake and each of those layers can be like customized to work at the enterprise right so uh 100 agree that the notion of both customization but lowering the cost of management the cost of ownership and just the friction if you put too much friction in front of people they're just going to do unsafe things because that's how humans are right we we don't if we if i showed you this every single session for every single tool eventually you're going to press the wrong button right so really thinking through that is a big part of what we're trying to do here so that's projects and why I love projects as a unit of governance let's hop back really quickly to the skill builder flow that I spoke about again we made that dashboard we want to make this something that I can that I can reuse right and it's not just me it could be my entire team why do we have to like keep things close to ourselves right and so I can now go So Kai has created a skill for me.

36:23It's like a standard open spec skill that you can use on any of your harnesses, right? But for the purpose of this, I'm going to go click this button and Kai is going to say, okay, what do you want me to do? I'm going to go ahead. I could choose to push it to an area and we'll talk a little bit about area skills. But for now, I want to keep it private to myself, right? And of course, this is like nothing special about me here. Any user of Kai can do this and that's why we have a lot of skills now that are making people more effective. So it's filled in the description for me. It's filled in like when Kai could use it.

36:57So this is important. We'll come back to this in like a second, right? And I'm just going to go ahead and export draft. Oh man, I already created one. Again, this is what happens if you prepare too well for demos.

37:14Sharadh Krishnamurthy:This is when you do it live. I mean, we believe you. I think what you're showing here, great. You have kind of a skill creator, skill or tool. You have a specific spec that you're using to ensure that it's both written well, generally for agents, but also written well, very specifically for the Kai harness. And then I love this idea of a draft kind of skill editor that you can test and edit and optimize and manage. It's quite nice. I know people just love futzing around in Markdown and, you know, Python files, but just a little quality of life UI here can go a long way. 100%. We really invested a lot in making this feel like a little bit like an IDE so that everyone can get access to that quality of life improvement.

38:07Going back here for just a second, now that I did this, I can go and this is the magical part. This is the part that I'm really excited about. And the team has like really kicked ass here. So give me the latest by adoption dashboard, right? That's all I'm going to say. And this is the part where the magic of Kai really shines. Now, when you're in a coding agent, you can see what it's picked up. It's picked up the skill that we just like literally just created, right? And the reason why this is interesting is when I said all the way at the top of this, that you can just go in and start using Kai and it knows what to do.

38:45This is how it knows. Looks like my tool policies are too secure, right? So now the thing that they've done is when you're a coding agent, right? You're in a repository, you're in a folder, you have this natural structure to what you're trying to do, right? And so you can pick up the skills in the hierarchy of where you're working and you get the right set of skills required to do your job. When you're in an enterprise and you're starting to work, you don't know, like there is no hierarchy of what you're trying to do. You're frequently trying to fit like five different systems. And so a large part of the investments I've done and what we've managed to give to Stripe is the ability to package skills and retrieve them.

39:30And we do this like really rigorous flow of knowing when the right skills are being invoked so that Kai can perform at a high level, right? And the number of skills that we have, we've got to do some pretty interesting things to ensure that we keep those skills in tip-top shape. We've also built out this interesting thing where it's not enough to enable people to build a bunch of skills. How do you make sure that they actually know what it's doing and how you keep them in top shape? And that's the other thing that we're really investing in. this automatic platform-driven suggestions for how to improve your skills so that especially since we let anybody publish a shared skill, that anybody else can pick up, it's become really important to ensure that we can give people the tools to keep those in tip-top shape.

40:19Again, something that you probably don't worry too much about if you're just using AI for yourself, but the moment you introduce that sharing and the enterprise, Quality, governance, policies all become something really important. And they're usually something pretty specific to the company you're working at. And Stripe is no different.

40:38Sharadh Krishnamurthy:You know, the only other thing that I've seen here that I'm curious, maybe you have, but you haven't shown, is we see a lot of folks that are building these internal harnesses do skill and tool telemetry and observability and see where, like, tool calls are failing a lot so they can auto-eval that. And then they also have a deprecation policy for skills. So if skills have not been invoked for like 30 days, you get a little notice and it's like, hey, you haven't used a skill. If it's dead, maybe we archive it. And if they don't get a response, it goes into like deprecation status and then they delete it two weeks later just to like prune all this stuff that's happening.

41:19Sharadh Krishnamurthy:And so I think this enterprise level maintenance of the skill library is really important, not just from a quality perspective, which is what we see here with the evals, but honestly, from a quantity perspective. Like, just is any of this useful anymore? 100%. And I almost think you can't separate quality and quantity when it comes to these systems because, you know, context is everything. The more underrated context you throw into the AI, the less good your results to come. So quantity is almost a facet of quality. We started off with a bunch of different skills, and we do have telemetry. We have, let's say, 50 skills that are used, hammered every day across the company.

42:10We have this long tail of 100 to 150 other skills that are used by subcategories, parts of the org chain. And we have a bunch of tools that are used by two or three people. We need to respect that all of these exist. There are teams, there are three-member squads doing some really bespoke thing, and they want to share between themselves. But then, again, what is the telemetry we have? And part of the process I'm showing here is, this is the user-facing part of it, but part of the process is telling us as harness owners These are the kind of skills you probably want to promote up into like a general workflow.

42:45And these are the skills that you want to like delegate out and move out of the general workflow because it's just taking up context. Right. So definitely, unfortunately, don't have something cool I can show around that. But it is something that there's an ETL pipeline happening somewhere that's doing this.

43:02Sharadh Krishnamurthy:Amazing. Well, I just want to recap for folks because this has been awesome. Just high level things about Kai, personalized context for people, org chart awareness, you know, tune tools, a sandbox that you can put data in, a sandbox you can pull data out, shared artifacts, projects, which I feel like if you missed that part, rewind, go back to it. Because projects are not just how you organize chats across a team, but how you give a specific space, whether that's a team or an initiative, access to tools, access to data, permissions and controls, including what requires a human in the loop. A skill builder skill, but not just a skill builder skill, a skills platform for the company that allows you to build skills, edit skills, eval skills, share skills.

43:55Sharadh Krishnamurthy:and then, you know, AI that just works for the things that matter, including data analysis, which benefits not just from this tuned harness and skills around it, but investment from infrastructure all the way up the stack on a great agent-ready data layer. And it took one and a half agents or one and a half humans, sorry, probably a million agents. Many more agents. A couple of weeks to get V1 going and now is serving 10 ,000 stripes with less than 10 people, plus a bunch of great infrastructure that you've been investing pre and post AI. That's it. That's all. That's all. Not much to it. But yeah, I think that was a fantastic summary.

44:42A lot of good stuff. I think it's important also, I would be remiss if I didn't say, it sounds like we figured this all out. We absolutely haven't. We are very cognizant that we are in the earliest parts of this journey. And we're hoping that, like you said, the strong foundations we have help us iterate and move forward with the times. But yeah, it's been a fantastic journey so far. And maybe we'll be back in a year showing you something completely different because that's how quickly the space moves.

45:10Sharadh Krishnamurthy:I really hope. I hope sooner than a year. Well, before we get out of here, let's do two lightning round questions. My first one is, let's just put Kai aside for a minute. Let's put aside Stripe Blurple. What are personal AI things that are fun that you're doing or that you're excited about as an engineer? You know, when you shut, you know, as you say, we like shut the work laptop and open the fun laptop on the weekends. What are you excited about? What's cool? I'm super boring, but so I'm going to like say something less boring first, which is it's helped me like not sound super dumb to my six-year-old, who's right at the stage where he's asking me all these complex questions about like exoplanets and galaxies far away.

45:59And I'm like, it's on the side, you know, Gemini on my phone. Like, hey, can you tell me what's happening? And then I act like I know the answer. So it's helped me, you know, keep up to his model of dad knows everything. So that's good. Perfect. But on the work front, honestly, like the stuff that I'm not doing when I'm building Kai, it's like my I have this whole workflow now, which is around using the AI to make sure that I don't miss things. It's so boring, but it's so good. We didn't get to show you Kai's schedules, but I basically use Kai as like my personal assistant. It just tells me things that I'm supposed to be doing.

46:36And so it's so basic. I almost I'm almost embarrassed to say it out loud. But it's been the biggest life hack. Just not having to keep it all in the brain has been awesome.

46:49Sharadh Krishnamurthy:Yeah. What I tell people is we think a lot about how to put agents to work. I want the agents to put me to work. I want them to say, Claire, please fill out this form. Claire, please do this thing. you said you were going to do. And so I think it's a give and get relationship. And I love that. And I also have many children who ask me really existential questions about the universe and about dinosaurs and about history. And I agree, intelligence on demand helps us keep our superiority in that parental child relationship. Yes, yes. For a few more years, at least, until they figure out what they're all doing.

47:28Sharadh Krishnamurthy:So last question. When Kai, well, maybe not Kai, Maybe you're very sweet to Kai, but when AI is not listening, what do you do? How do you prompt? Are you a yeller? I don't know. I feel like I'm, maybe I'm just subconsciously afraid of what it's going to do to me when it figures out, you know, knows where I live or something. But I'm very nice to the AI. I just say, hey, that's not what I wanted. Here, I'm going to say it again. And maybe I'll like all caps it. But I don't know. I don't like shouting at me. It feels wrong. It's almost like I'm shouting out the people who built the AI. So maybe.

48:07But I just insist. I just, I say yell harder to my team, but I actually end up just like saying a lot of, please, if you will, read the thing I said better. But it does mess up and it's very frustrating.

48:20Sharadh Krishnamurthy:Like many of our guests, you gentle parent the AI, which is. This is true. I know you can do better. I believe in you. I'm not mad. I'm disappointed. I'm disappointed. I'm just disappointed, Kai. Like you should have done better. I love this. Well, this has been super helpful and interesting for me. It's given me so many ideas about just my own use of AI and how I talk to people in enterprises about their use of AI. Where can we find you and how can we be helpful to you in the strength team? Well, I'm on LinkedIn and I'm happy to connect with anyone who's like super interested in learning more about what we've built here and how we think about scaling AI for the enterprise.

49:05And Stripe is always hiding. We are always on the lookout for people who want to join this crazy band of people trying to build amazing things for the world. So please look out on the Stripe careers page. But otherwise, my email is sharad at stripe.com. And I'm happy to engage with anybody who has questions about anything we covered today. But yeah, keep those questions coming.

49:32Sharadh Krishnamurthy:Awesome. Well, thank you to you and thank you to the Stripe team for being so generous with all the things that you shared with the audience. We really appreciate it. And thanks for joining How IAI. Thank you for having me. And this was fun. I don't know if I mentioned, but you're a minor celebrity on the team. So I now have some reflected glory and yeah, it has been an amazing time. Thank you so much. Thank you. Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app.

50:09Sharadh Krishnamurthy:Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.