Katelyn Lesse and Angela Jiang: Not the Anthropic you're expecting

18:30Katelyn Lesse:So I think like really transformative technology, it's not really like, it doesn't really express itself as like, you know, a new UI per se. Like, yes, we've all seen like chatbots and agentic interfaces, and we type something and it just, you know, spits something back at you. But at the end of the day, like to me, all those things are just methodologies to get to a place where, you know, ideally, like healthcare truly is much more accessible. Energy truly is more accessible. Education is cheaper and you get access to the smartest people ever who can help you get smarter yourself. And I think those things are honestly the most exciting things you can do with AI.

19:04Katelyn Lesse:So I'd love to actually then kind of

19:07Angela Jiang:pivot a little bit and ask you a bit about like, you've built your careers through the middle of like harnessing all of this new technology and then building it yourself. Imagine you would like be at the beginning of your career right now, maybe with a founder mentality, knowing the capabilities of the models right now? Like, how would you even begin to think about what you would build knowing what you know? Yeah, I'm going to give a little bit of a boring answer and say, I'll be the judge of that. I'm sure it's not boring. I feel pretty strongly, actually, especially for engineers, that we, it will still matter for a long time that you actually just understand like the fundamentals.

19:44Angela Jiang:What's been kind of interesting and almost like a challenge with what's happening lately is you've got engineers and non-engineers who are building systems and they're like, okay, cool agents, please build me whatever we need to build. And you have agents like firing off sessions to work on like really complex and large systems. And you have to actually be really careful about steering those agents to modify these systems in a way that actually adheres to best practices around testing or best practices around infrastructure and scaling and all these sorts of things. Like a really good example of something that's happening now is so much code is getting written.

20:23Angela Jiang:And so CI or continuous integration, which is the thing kind of builds the code and runs the tests, is starting to become the bottleneck. It's getting slowed down because there's so much code getting written. And part of the problem is these, like, unless you're steering your agent correctly, you're ending up with testing that's like either slow or unhelpful, like not the best kind of bang for your buck in terms of preventing problems and also being able to run quickly at that integration stage. And so I think there's a lot that still matters, I guess, in terms of fundamentals of software engineering and how we do these things so that the engineers who are actually steering agents and designing systems and designing how the problems will get solved, even if the agents are writing the code, can make sure that those things happen in the way that they still should.

21:10Angela Jiang:And so I feel pretty strongly that, you know, if someone's just starting out their career right now, and they want to be a builder, they should still learn all of the same fundamentals that you otherwise would learn. And I think just, you know, be someone who's going to like go push the boundaries of what you can actually accomplish with the tools, like get more done with your time, right. And just get more ambitious and creative with what you can do, but spend the time to really learn the fundamentals because it's still going to matter.

21:36Katelyn Lesse:I would really love to, if I were to restart everything, I would love to go really, really deep into a domain where it's something that you want to see end to end and then really try to just rethink it from scratch. I think because we're in this really awesome, transformative state, one of the best things that you can have is actually a little bit of a blank slate. and so you know there's especially if you know you've done something for a very long time in industry yes you have a lot of domain knowledge and that's really really important but at the same time you you also accrue a lot of like you know this is how what we've done things this is how i've always done things and that's that's a bit of a mental barrier i think oftentimes with with like transformative technology and i think if i were to start over from scratch i'd probably go deep and like i actually started off my career originally in investment banking um like that's i don't know that i would do that again frankly um but actually no i definitely wouldn't do that again but um i would probably go like for example like really deeply into like a portion of finance is things are really important to see a problem end to end at the surface might seem simple or maybe even potentially kind of like confusing or or potentially even unnecessary but as you kind of go through you realize like there's there was value that was created here um and i would just do maybe like one or two years just getting deep seeing something like end to end um and then i would just try to say like reimagine because i wouldn't have had like you know maybe all the of the processes already baked into my head.

22:56Katelyn Lesse:And I try to reimagine what that would look like from scratch and see what I could do with that. You know, you mentioned actually earlier,

23:02Angela Jiang:Caitlin, like just like getting better at taking things off people's plate and making their lives easier. And now like agents and the capacity of these models allows, you know, a lot of people to go from like A to F, I think was your kind of version. And you were saying like, actually, it's actually about the level of depth you can take on a given process and the level of workflow integration you can do is really where much more of the value over time will come. But I guess to your point, you need deep domain expertise to understand like, well, what's the point of each of these steps in that process?

23:29Angela Jiang:Is it needed? If it's needed, how do I automate it? If not, like how do I rethink it?

23:34Katelyn Lesse:Yeah. I think you have to just kind of like experience the real world a little bit oftentimes to kind of really figure out like how deep in that integrated flow you can go. And, you know, if you can experience something like end to end, whether it's like a complete supply chain of that kind of like problem, there's like probably so many different points that just added friction or cost or unnecessary complexity that if you were to, again, just at least observe it, then you could probably have a perspective around how you can completely transform that with new technology or a new way of doing things.

24:04Angela Jiang:Yeah, super cool. I'd love to pick up on a thread, Caitlin, that you brought up, which is maybe surprising to people, which is like, developers should still learn to be developers, even in this modern world, even if it's to oversee and better understand the technologies that they're building. um maybe you could share a little bit about you know lots being written about how anthropic structures itself the way i wanted to start actually is like i'm sure everyone thinks hey well like how do they structure themselves maybe i should be structuring myself the same way do you think there's like wisdom in attempting to like replicate the way you work as a team yeah um well maybe i'll start by saying i actually don't think that we're structured that interestingly

24:42Katelyn Lesse:differently than most people are the spicy take is it's not that spicy very spicy take on this And it's actually, it's like, it's like, you know, it's very mild. There's like a little bit of paprika on it. Yeah, exactly.

24:51Angela Jiang:Where's the paprika? Where are you opinionated? Yeah, so I guess I would say the way that my org and my team is structured is not that different from how I would have structured a team a year ago, two years ago, right? I think you still have a group of engineers that own a system. You need to have like the right amount of humans to be able to like all hold that same complexity in their head and then be able to rotate through who's on call and who's dealing with certain problems and these sorts of things. So like the size of an actual team is not that different. The structure of an org is not that different.

25:22Angela Jiang:The one thing that I would say we've changed is the ratio of people who actually spend more time on stuff like systems design and architecture versus people who are implementing. Like previously you would have had like one technical lead who's like, here's how to design the thing. And then like eight people who are like, great, now I'm going to go pick up tickets and write code. And now it's probably like that first person is like, here's how to design the thing. And then like one other person is like, cool. and now I'm going to orchestrate like 20 coding agents to go and get that thing done. And so that has definitely changed.

25:52Angela Jiang:And that's been interesting, because I think on the path to changing, that has been kind of painful, I think, for for some people, like for in particular, those technical leads and the people who are like, here's how I architect stuff, they're suddenly like drowning in a bunch of people who want to work with them on Okay, great, there's all these projects that we can do, like, how should we design it? And they're like, okay, cool, here's a design and I have to coordinate with 10 other teams. But now I'm doing that on behalf of a bunch of projects at once, right. And so we've actually kind of changed in terms of like, how we're organizing ourselves and how we're hiring to make that ratio a little bit more kind of one to one, in terms of someone who's maybe slightly more on the junior side and someone who's slightly more on the senior side.

26:34Angela Jiang:So that's maybe one way that we've changed. But I would say we're not otherwise doing anything insanely dramatic in terms of like, how teams are sized or how organizations are structured and things like that. And Angela, is that the same on the product side? Like, does it feel like the structure of your team is radically different than it was at OpenAI or maybe OpenAI is not the best comparative, like back at Stripe or your prior organizations?

26:59Katelyn Lesse:It hasn't changed much either. I think like the types of work that PMs do has dramatically changed. And so the profiles that we look for are a bit different. And I think, you know, I wouldn't necessarily have done the same thing in other places. and I think they started to fall into kind of like two camps, I think, and one camp for a small set of features, not for everything. For a small set of features, we do have the PMs actually like ship that directly. And so they'll work. Terrifying. Yes. I try to set some guardrails around that. But, you know, for a small set of features, product managers are engaging directly with coding agents and they're able to actually ship.

27:33Katelyn Lesse:And so that's, I think, like, you know, obviously very, very different. historically you would have to you know like have at least one engineer um and have some time and maybe a designer and all these things um but those those pms i think are you know they've kind of like compressed a lot of that which is really interesting um and then the other side which you know happens generally for more like senior uh pms but we're doing it like faster and faster almost um is we we are pushing pms to like own more and more problem spaces and so it's like you know if uh the code gets produced as fast as it gets produced if we are shipping at the speeds that we're shipping, what it means for a product manager, which is always about problem solving, about thinking through, like, is it worth our time to go do?

28:12Katelyn Lesse:How should we do it for the user? Who are the users we actually care about for this particular feature set? Those things become bigger and bigger and bigger. And so oftentimes the amount of scope a product manager has has dramatically increased. And so the kind of people that we need to bring in tend to be a bit senior in their ability to kind of handle a lot of those components. And so we're kind of seeing that sort of bifurcation. Maybe say a little bit more about that.

28:34Angela Jiang:Like what's the part about a senior person? Is it that they just are better at holding multiple projects in their mind and kind of internalizing the tradeoffs between them and how you think through that? Or is it, is there something about the kind of the breadth of customer representation that you find senior people are just like better at holding in their own mind?

28:52Katelyn Lesse:I think it has a lot to do with experience actually. So I think a lot of these things are a little hard to kind of like quantify. like the you know just the act of like shipping a feature in and of itself is is obviously helpful but as a PM if you go through enough iterations of this you'll kind of realize you know like hey you make a lot of mistakes you think that this was super useful for this user you try to do it you launch the thing no one cares you're like okay I always had yeah you know misunderstood that one but then you like learn and then in some areas you know you think you did the right thing again by shipping maybe like just a sliver of it and it turns out like that's not how people want to use the product.

29:27Katelyn Lesse:And actually, you should have thought about the bigger thing and then tried maybe a different slice of it. And so a lot of those kind of like nuances around, you know, it's kind of like the shape of the type of problem. And it is about like balancing kind of, you know, what you're referencing about customers and the market and a lot of these kind of like judgment calls, I would say, which come again through a bit of like experience and kind of learning a little bit by trial by fire, at least in the product space. That's like really, really important and that's actually more important than ever and that's the kind of like the the seniority piece that we really uh looked for and and uh hope to kind of continuously cultivate through through that kind of experience yeah we hear this a lot obviously is like the the throughput

30:07Angela Jiang:that you can achieve now with the with the tools is such that actually the judgment around what to build and for whom and why and sequencing and how you how do you think about shipping that for a customer that means they'll actually get value out of it becomes in some senses even more important because the constraint is no longer usually technical. It's often about like, what are the product calls and how do I make those priorities?

30:28Katelyn Lesse:That's exactly right. I almost think it's like, you know, the demand for like really, really strategic thinking just like starts to skyrocket because execution is easier and easier than it's ever been.

30:37Angela Jiang:So maybe could you say a little bit more about that, Angela? How do you, in this world of like abundant capability, like how do you think about the prioritization of which direction you're going to go with on sort of platform?

30:48Katelyn Lesse:That's a great question. And I think, you know, there's a couple different dimensions I take a look at. I think one of them is always like about like reach. And to some extent, you know, these are kind of like very standard product principles. And so we still bring them into the platform, even with the pace of like the, you know, the technology changing so dramatically and users doing all sorts of crazy stuff on the platform. But reach is like really important. And one way we try to express that internally is just like, you know, go where users are. And so that's obviously very like just normal user-centric approach to things.

31:17Katelyn Lesse:But I think sometimes in the kind of age of like just like really rapid innovation, we tend to think maybe we kind of forget maybe some of the fundamentals. And I think they're actually just like honestly more important than ever. So one of the things is like on this kind of reach front or being where the user is, a lot of businesses and developers, they have their business like somewhere. And so like, you know, they would have built their stack with like AWS or GCP or Azure, or they have a specific stack that they really like to do. And we want to go and serve them. You know, I think going to them rather than having them come to us is like something that we care a lot about.

31:51Katelyn Lesse:And you can imagine people saying that about all sorts of products, but we like really, really try to make that kind of like a core like principle. Has that been hard to instill? Like, is it so counter to the way that technology has worked in the past that you're like, oh, no, we are... We really think about like democratizing the exponential. So we'll prioritize a lot of things that are like making it easier and easier for people just to get the right outcomes from what they're trying to do. You know, it's still like frustratingly still too hard to kind of like make an agent that just does what you want it to do.

32:18Katelyn Lesse:There's a lot of steering, a lot of like complexity around harness engineering, prop knobs and musicians. Like, I'm going to be honest, I don't even really fully understand harness engineering. Buzzwords to you. I mean, it's like, it's like very buzzwords. I mean, they're literally real things. Yeah, I'm nodding and I've got some sense of it. It's hard. Right. And like everyone kind of envisions a world closer to like it's like a great coworker that you love. And you just tell that person, hey, can you like grab that coffee for me? and instead of it being like, can you express that as a tool call?

32:43Katelyn Lesse:And you're like, that just goes and gets the coffee for you. And it's like, can I just tell you the outcome? Exactly. And so a lot of that stuff is about, you know, we'll really care about that because we want people to kind of experience kind of the magic of AI and the simpler we can do that, you know, the better as well. I think the last thing is just eventually as we, and we're kind of early in this part of the stage, but as businesses get more successful with building with AI and building like great products, like how can we support them better? And I think there's some ideas we have in that category, but we'll typically flow across these core three parameters in order to kind of consistently whittle down what we actually take our time to focus on.

33:18Katelyn Lesse:What are those parameters, remind me? Just going where the users actually are, democratizing that exponential, making that really simple, and then helping you grow eventually in one way or another. Yeah.

33:27Angela Jiang:I think that one of the things that is still mind-boggling because of the pace of the change is, I imagine for you as a company, so many organizations worry about the chasm, right? They worry that they've got like an exciting set of early adopters, but they are weird and unlike anyone else in the economy. And therefore, like overfitting your kind of product aspiration, your vision to that group of people means that you will have, you know, you'll have an adjustment when you want to go for early majority and all that sort of stuff. And because of the speed of change, and as you say, you've got a lot of people coming to you with all these universe of use cases.

34:00Angela Jiang:I imagine one of the hard things is actually just saying, like, do we maximize for the excitement of the like really, really early adopters, the people who understand to your point, Caitlin, like they really understand the potential within the technology and they want to push it to its greatest extent. And that's where your opportunity lies as well. Like you can be like, oh, I can do an MCP on MCP. Like that was learning and valuable for Anthropoc. But meanwhile, I think what I'm mostly hearing from you, Angela, is like, yeah, most people are also in their own environments and like it's taking a while for them to figure out exactly what AI can do for them and for their businesses.

34:33Angela Jiang:And like we've got to keep an eye on the opportunity there because that's probably where the bulk of the economy is going to be. I imagine this is just like a real tension that sits within the team. Do you feel that or am I imagining it? No, you're not. And I think that goes back to the idea of, you know, a developer platform can cater to the here, like, really customizable little building block primitives, right, versus here's a solution that's a little bit more out of the box. And I think, you know, Angela mentioned we launched Cloud Managed Agents recently, and that's the more out of the box solution.

35:04Angela Jiang:And when I say recently, I mean, like, a few weeks ago. And so... Eons ago in AI time. Exactly. But, and so it's been cool because people have started building on it, and this has been great. And I think it has enabled a lot of people who otherwise wouldn't have built certain things because it would have been harder and, you know, taken more time. But the majority of our business today, our platform business today, is companies who and people who have chosen to build on those very customizable primitive building blocks. And it's really interesting because, like, there's still endless work for us to do within that category of things, right.

35:38Angela Jiang:And so we are very much like spending our time across both like we want to make sure that we're continuing to build the building blocks the primitives as well as these kind of higher order abstraction things that you can do right out of the box a hack from that perspective is actually the higher order things that we've built for us is actually just like a collection of our own building block primitives for the most part and so it's nice because we can kind of build it that layer and then like piece it together for people in the way that we think largely makes sense but yeah I mean And I think the people who are really pushed, have pushed the boundaries and gotten us to where, you know, we are today and where they are today are looking to us to continue to build across that whole spectrum of solutions.

36:20Angela Jiang:And so we'll continue to do so. Yeah. And so much technical learning, I imagine, comes to your team, too. Like, it's, you sort of, you mentioned democratization in some sense, being able to make a platform that you allow the ingenuity of, you know, potentially billions of people to then just say, like, what if you went in that direction? What if you went in that direction? That's really cool. And actually, like maybe for those of us who have done product before, you know, the way in which we get product insights has changed, right? It used to be like you literally get like a product insights platform, SaaS platform.

36:50Angela Jiang:You plug it into your platform. It would look on and be like, Angela is trying to click this button and the button doesn't work and it doesn't do this. Like you must be getting product insights through a very, very different layer through the product. Like how does that surface within the team and how does that like flow through? I imagine you've got a bunch of agents that hoover that stuff up, try and summarize it, try and pick out the pieces. Am I getting even that part right? And then it sort of eventually makes its way to Angela's team and she figures out whether it's worth listening to or not and how she prioritizes.

37:19Angela Jiang:Like, is there even that, is that linearity even a part of the way you work? So I would say something that's actually pretty unique in this space is we don't have access and we don't like store or look at people's prompts. So when people are actually working with the platform and building agents and whatever they're doing on top of the platform, we don't have as much data as you might expect us to have that help us understand what they're building and how they're using the platform. And so it actually becomes more important than ever for us to build closer relationships with our customers. So is it just you see like a prompt didn't work?

37:54Angela Jiang:Like you see a kind of almost binary outcome success failure of a prompt? Is that what you're getting? Yeah, exactly. it's like, you know, someone made a request and like, you know, they sent a prompt, right? And some response came back out of the model that we sent back to that person on the other side of that API request. Like, and we can see obviously like volume of these things happening and token usage and those sorts of things, but not the actual like pure data of the prompts. And so what's interesting is there's a concept within, you know, the AI community that you might hear called evals.

38:25Angela Jiang:And evals is, it's really just tests. It sounds cooler though. It sounds cooler. But it's interesting because point being, the like AI, like LLM models, they're non deterministic, like you might send a prompt through and you might get back something that like isn't a different from regular code, where like, if you write code a certain way, and you get output, you will always get that same output, right? Right. And so evals is a way to kind of try to benchmark, like, here's what I'm actually looking for and here's what success looks like for my agent. And some of that is like, you know, very static analysis of like certain words came out of the thing.

38:59Angela Jiang:And some of it is like, actually, you throw another agent at that agent to like judge how it did. Right. And there's like a lot of different ways to do this. But the customers who we actually work most closely with as we're building new things or as we're releasing new models are the customers who have put a lot of work into their evals. Because for us, we can run all our own evals, right? We can like build agents, we can like run with these models, and we can see how well are these things performing. But it's not until we actually can like go work with all of our best customers and say, okay, now try this on your eval suite, right?

39:35Angela Jiang:And let us know how those things have performed, that we can actually evolve what we're doing and get better feedback and better data. So that's one thing, you know, I'd be curious how you and your team feel about this, but I think it's been unique and different for me than working on past platforms and past products where you kind of just have more access to data and understanding of how people are using the product and how well it's going for them. whereas here it's kind of like it's sensitive data right like we're not working with these prompts and so instead we kind of have to build the relationships we have to help our customers build out really great eval suites and then be able to share data back with us on how things are doing so that we can iterate and evolve yeah no i think that's definitely true um i think two

40:17Katelyn Lesse:other pieces i would just add to that is like uh you know it almost sounds a bit old school in a way but you know those kind of uh innovators that you were talking about about you know how do we especially these kind of like ones who are really just pushing the frontier of stuff, those are the ones that we do really try to spend a lot of time with. And yes, there's, you know, a broader market for kind of almost everyone else who could take something a little bit more off the shelf. But these kind of innovators and kind of like, you know, very, very sophisticated builders, we love to spend just tons of time with them to understand exactly what they're doing, how can we work with them, how can we improve things for them.

40:49Katelyn Lesse:And we actually kind of like learn a bit together. And that I do think is a bit unique, you know, in other areas. someone gives you feedback, you listen to it, and then you go fix it for them, make it better for them. In this kind of space, we're kind of like, and oftentimes, like kind of co-discovering, we'll share something that we've tried, they share something they've tried, and we're like, okay, like that could be very interesting, and then maybe we'll, you know, put that out as a best practice or something like that. And then in terms of like, I think like just broadly as a kind of like feedback system, something that's been kind of interesting about some of the way that we get feedback today is like, actually, everyone just like dumps everything into Slack, or puts it in Doc or like Google Drive, we just, we put it everywhere.

41:24Katelyn Lesse:I've actually been asked this question a lot about like, how do we manage context and our knowledge stores and all this stuff? And actually we like, we don't really like organize anything. We just keep track of it. We just like, it's all raw data. It's all raw data. It's all context. And actually we actually have Claude and some of the internal tools that we built do a lot of that heavy lifting. So it is actually easier as a PM, I think, to kind of get some of those insights. We will ask Claude oftentimes like, hey, you know, we like we set up this feature. It'd be great if you can kind of like just scan through Slack, scan through other things and tell me how people are, you know, liking it or not liking it or any kind of that feedback.

41:58Katelyn Lesse:And I think that's been really, really helpful as a way to kind of, you know, try to like synthesize what would normally probably take a very long time to do.

42:06Angela Jiang:Yeah, we were chatting about this in the context of a different business. But, you know, obviously the future is all about the quality of the context you can apply to these beautiful models, right? And so like, what are the things that are low context at the moment that sit in people's heads that ideally don't sit in people's heads, but like there's maybe no natural place for it to go. And comms is a great example of this. Like most people live in, you know, for those of us who use Slack or Slack-like products, to the extent any of them still exist, you know, mostly we feel deluged by like comms, right?

42:34Angela Jiang:But actually, you know, in the future world, the quality of the decisions we can take will be we need more comms, not less. The question is what sits on top of it that makes it manageable for us at like a human level, a human cognition level. And we were chatting about like, actually, maybe the future is more people say more things. It's just that it doesn't it doesn't surface to the level of I have to know about it. It might just be an agent picks this up. But then the interesting thing to me is like, in some ways, we're actually like right back at the level of like how much a human can extract a thing that's been in their brain and put it in the pool of public resource.

43:08Angela Jiang:And I feel like maybe the future is like what's going to do that most efficiently, right? Like what's going to read my thoughts most efficiently that allows other people to get that. So it's not me just like tapping, tapping away.

43:17Katelyn Lesse:That's right. And I think like also you don't have to like think through those abstractions as much, you know, like if you were to put something out kind of like, I don't know, I think sometimes about like the, when the printing press like first came around and people first started writing and, you know, people like, oh my God, there's going to just be a ridiculous proliferation of content that's out there. like 200 books got made and people were like oh my god how could i possibly like read 200 books um and in that world though people were very very thoughtful and like particular about like what would they what is worth putting into one of these 200 books that's out there and then you kind of fast forward you know people can just like tweet anything they send anything in slack all these things are just like i think just quicker more expressive we care a little bit less about you know how um how many like specific forms of abstraction we need to go do and so i think that like if we were to kind of project forward, I think context definitely is king and it's going to be super, super useful.

44:08Katelyn Lesse:And I think maybe the fluidity by which we do that just happens even more so.

44:12Angela Jiang:Yeah, super interesting. So I'm interested also for our audience, those of them that are early stage founders contemplating founding a company or building one at the moment, you see the future around the corner, as it were, you are seeing certainly on how we design technology, even if we don't know what the fully, what the outcomes are, you're seeing that future before others do because you're experimenting you're getting the the highest signal um you know signal on that what would you be optimizing for if you're an early stage founder right now because you can't predict what the future is going to be you can't predict necessarily what the model's capabilities will become what should you be focused really on maybe

44:50Katelyn Lesse:i'll start with you angela i think there's a couple things i think you know there's there's obviously a bunch of different models out there they're all doing very interesting things and and they're getting better and better. And I think like one thing I would do is probably spend quite a bit of time playing around with these models. And I think the less you have to fight it, the better probably it's ready for something. What do you mean fight it? So there's a lot of like, you know, there's been all sorts of ideas around like, you know, founders have looked at different domains and they're like, I can totally make this better.

45:19Katelyn Lesse:I can envision a product that, I was just talking to a founder earlier today who was envisioning like a health product that could really remember everything about their medical record, their preferences, all these things. And this was a couple years ago when this founder had started her company. And she was telling me how she was fighting so hard with the models. So there was so much scaffolding she had to make, so much complexity around just get this model to do this very specific thing of remembering something. I've also seen this with customer service, with some voice-related content a little bit a while ago.

45:52Katelyn Lesse:But that's an example of where maybe the model's just kind of maybe not good enough. Your intuition as a founder is probably still accurate, but there's probably still some really great problem idea that you have. But I think one key bit in this area is like you want to wait till the model is kind of like just good enough for you. And the biggest test of that is just you're not fighting it so much. So if you are telling it to, you know, you have an idea for something deeply personalized and you just try it out a little bit. And let's say it like just kind of like it's not perfect, but it kind of does it.

46:18Katelyn Lesse:That's probably a pretty good sign that like, you know, the technology and your problem spaces, they're intersecting at the right moment. And so it's worth it. And the reliability beneath all of that will improve in the time that it takes you to then. So if you build the product and the model just gets better in that direction, then you're riding with it. And then that's, I think, a great spot to be in versus in the other direction where if you find yourself constantly fighting over it, the idea is probably still good, potentially, but maybe the technology is like not quite there. And so even if you were to move on it, it's very easy for a competitor, you know, who maybe waited six months or something like that to just kind of like bump up so much faster.

46:52Katelyn Lesse:So I think that's a bit differentiated of this particular moment in time of like maybe how to kind of think about the technological angle of how you should consider whether or not you should start your company.

47:02Angela Jiang:Which, let's be plain, is really only knowable through like manually attempting this. Exactly. Within lots of different contexts and, you know, obviously getting agents to do it. But like there's no getting around the testing of the model's capabilities directly to know the answer to that question.

47:19Katelyn Lesse:Right. And I think there's a lot of alpha in that for founders. And so anyone who has kind of a bit of a builder mentality, and it's something that we might have all chatted a little bit around, was just like, you know, just being a builder and like not being, I think, afraid to kind of test it. It's okay if you're not technical. It's okay if you have no idea what you're doing. These tools are easier than ever to kind of experiment with. But kind of like, you know, playing around with it just really gives you a sense for whether your ideas has kind of like technical standing grounds to allow you to move at the speed with the models rather than kind of against them.

47:47Angela Jiang:And that feels particularly true if you're doing anything that's touching a regulated industry, right? Because you're already up against it with the kind of trust factor. That's exactly right. Yeah. Yeah. Like there's no space for like, oh, it gets it right 80 % of the time, you know? Yeah, exactly. That's just not sufficient. Exactly. Caitlin, do you have a similar or different view from that? No, I agree with that. And I think maybe just to add something a little bit different or in a slightly different category, I would say, I think a lot of people are building in infrastructure, like particularly like infrastructure that sits like, you know, around a model or the things that are like, if you pretend you had the ability to build agents that could be like extremely autonomous or extremely long running, and maybe they can use a computer, they can use all these different things, right?

48:31Angela Jiang:Like, what is the infrastructure that everybody would like need to be building on in order to make those things happen. I've seen some of, you know, the coolest startups in our space building stuff that are like pushing the boundaries of how will infrastructure actually keep up as like these agenda capabilities get better and better. And so, you know, I agree with everything Angela's saying, but one of the like really key, like I'm hearing ideas here and there, I'm like, oh, that's awesome. That's going to like really, really last for a long time tends to be at that sort of layer. Yeah, it's cool.

49:01Angela Jiang:because I think there's a lot of people who also feel like the kind of developer tools market has gone through like a lot of change. But actually what I'm hearing is like, no, it's actually as vibrant as ever. It just looks and the shape of it is different. The way in which it operates is different. I wanted to pick up on one theme that is sort of an extension of Anthropics capabilities. SaaS-based pricing is really hard in an environment where like Seek-based doesn't make sense. No one really knows what the future holds, but I'd love any insights you have on like what you're seeing some of your customers thinking about in the space of pricing products in the age of AI?

49:33Katelyn Lesse:I think, you know, we've definitely seen folks kind of like face the same kind of problem over and over again when it comes to kind of like seat-based pricing. And I think with that kind of like style of thing, we're starting to see founders, you know, really try to kind of move towards slightly more usage-based, but no one really loves usage-based, you know, billing. Value-based, even harder. Even harder. And I think there's obviously been mid-conversations about like outcome-based pricing as well, but it's very, very hard to do. And I think in the industries where it has kind of found a little bit of a foothold, it tends to be a little bit more of like a race to the bottom, which oftentimes when you build these kinds of products, you want to do to your point like value-based pricing.

50:12Katelyn Lesse:And so that's been kind of, I think, complex for a lot of people to kind of navigate. I'm personally quite bullish on the idea of like, maybe something closer to like the complexity of the task that you can kind of have. And I think that that might kind of combine at least something that's like slightly more intuitive to a user where, you know, as opposed to like a token is, you know, that's no one really, that's, it is a unit, but it's, it's not terribly usefully interpretable for a user. But at the same time, it scales with kind of the costs that come with, with agentic systems. Yeah. It's a good proxy for the token cost.

50:42Katelyn Lesse:Yeah, exactly. And I think that's the most important thing is like you, what you, for all these like pricing models that the people are experimenting with and who knows where, where, you know, people will really end up like gravitating towards. But I do think the, the kind of essential part is just making sure that like your pricing model is actually mapped to your cost. Turns out that like 1970s business books are probably still right on that thing. Yeah, you know, a lot of the foundational principles still still continue to hold. So I think that that continues to be probably like where a lot of these kind of like pricing innovations will gather ground.

51:11Angela Jiang:So Speed, you're building yourselves on this exponential that you're also hoping to help others do. What's something you're seeing? What's an emergent sort of innovation that you're seeing that you're really excited to explore in the next, you know, what remains of 2026?

51:25Katelyn Lesse:Something I'm really excited about is like the idea that we can kind of like expedite how agents can connect with more and more systems. Kayla and I are like very excited about connectivity, like in general, like the idea, like, you know, if you can build an agent, that's great, but it needs to like touch like the external systems. And we have MCPs and we, people are starting to build CLIs. What's a CLI? Command line interface. I was about to say computer. I haven't said the acronym for a long time. There's a lot of these ideas around, I think, like, how can you basically like build something like agents want so that they can kind of connect more to kind of these external systems?

51:59Katelyn Lesse:Like there's so many systems out there that are just like so old, like they don't even have APIs. They maybe have like a really ancient like UI that you need to go and click through. And in some regulated industries, you have to, you know, provide a record that you change that button by like five pixels. And so those things are like, you know, I think a lot of really important stuff happens at the other end of those kind of external systems. And there are things that I think I'm personally very excited about of how can we find ways to kind of automate a lot of that? There'd be an amazing world where I think there's some kind of old system.

52:32Katelyn Lesse:There's a lot of important data on the other end of it. And in a safe and secure way, how can we maybe use computer use and computer vision to kind of automate some of that? Maybe we can automatically turn that into a set of APIs for you and then maintain those APIs. And that way your agents are actually able to connect programmatically to all those pieces. And then that gives you a ton of connectivity that you don't have to think through, right? A systems administrator doesn't have to think through, like, they don't have to know what a CLI is or what an API is or what any of these things are.

52:57Katelyn Lesse:Claude was able to kind of just do the connection for them and do it safely and securely. Yeah, beautiful. Caitlin?

53:03Angela Jiang:Yeah, I think one of the things that I've seen that's most exciting, and our team has done a lot of this, and I think some of the most like AI pill that I pill teams that I've seen have done these sorts of things is a lot of teams are kind of building they're using the tooling to build their own solutions to problems that they need to work on. And a good example of this is a lot of engineering teams are building more like end to end developer agents, right, as compared to like, we've got excellent tools out there today, like cloud code and cursor and others that let you write code in an automated way.

53:35Angela Jiang:Like, great. Right. But the end to end lifecycle of a change is what am I even building? Then the code gets written. Then I need to test it against my own systems. And then I need to open a pull request. Right. Like there's a whole bunch of things you have to go through. And so, you know, like we've built our own with just like engineers being creative, being like, oh, I'm going to build agents that like do different parts of this or more parts of this. And we've seen like Stripe put out some blog posts around. They called it Minions, which is adorable, like their own platform for doing this sort of stuff internally.

54:06Angela Jiang:And Ramp actually did something similar. So a few of these companies have said, okay, cool, we want to build like more end to end platforms for how work gets done, just like purely out of engineers who are like, I'm going to push the boundaries of what all this these tools can do to solve my problems. And so I think what's exciting about that is like, we've spent a lot of time, I think, kind of trying to encourage teams like this is what the platform is like here to support you doing um especially with stuff like cloud managed agents like you can build like end-to-end solutions to problems right that like you know it's uh it would have taken you a lot of code and a lot of time and a lot of effort previously and now it's kind of like this customization layer that's like pretty thin on top of that sort of infrastructure and so um I think this this line of thinking this like okay fine now the code part can be kind of automated like what's the next part that i can automate and how do i build more autonomous and long-running agents that can do this type of work um pushing those boundaries has been like kind of the secret sauce of how our team has been able to move really quickly and how we're starting to see some other teams pick that up like in kind of the community and in the industry has been really exciting and so um i think that's just something that you know the teams that are going to be the most successful are going to be the ones pushing and building those sorts of solutions for themselves.

55:25Angela Jiang:I pick up really quickly on that one, Caitlin. It's so interesting to hear about. I feel like adoption of any new technology, and this seems also true of AI to some extent, is like it's not linear. Sometimes it's fits and bursts. The capability has been there for a little while. Even people who know what they're talking about have been talking about the capability. And it's like, you can do this. I can even show you. And then all of a sudden there's like a watershed moment and suddenly everyone's like, oh, that's what you meant. We can do that. Do you find that this is also true? And have you learned anything about those moments of disproportionate adoption?

55:58Angela Jiang:Is it just like a certain person in the developer community with a certain standing talks about it and then suddenly everyone picks it up? Or is it something else? Yeah, I think this has kind of happened in a few different ways. And to me, it's actually less about adoption, more just how people are using the tool. So like a really, really simple, easy example is you could pop open a coding agent like CloudCode and you can give it like a fix this bug sort of prompt. And it's not going to do a good job. And then you're like angrily like prompting and yelling at it until you get a good outcome. And you stop using please.

56:28Angela Jiang:Yeah, yeah, exactly. Don't waste your tokens on please. Or it's in all caps, right? Yeah, exactly. So there's there's that. Right. And then there's engineers who are kind of like, they've started to get used to the tools and say, okay, cool, I know that I'm going to get a better outcome if I go into plan mode and I create a plan and then I make sure that the agent knows or like, you know, cloud code or whatever knows, like, actually verify your own changes, like write tests and run those tests and like continuously iterate until you get to something that fully works, right? and you know then you go even further and you say and now actually I can fire off autonomous agents to work on these tasks like I don't even need to be a human in the loop for every single one of these I can just like make sure there's a good plan and then I can fire it off and it can run remotely and I can say okay great I'll get back to you later sort of thing and so there's this whole spectrum and I think that a lot of this is just like the willingness to be kind of uncomfortable and be pushing the tools to be used in a way that's not, you know, what's like easy and right in front of you and like really simple.

57:31Angela Jiang:It's like, why not try like, you know, throw an agent in a remote environment at some problem that previously would have been very hands-on and human in the loop trying to work on. And the way that these things spread is like, I think it's developer community and people sharing ideas and things like this. I think it's like every engineering team needs like somebody who is just like, so excited about these things and just like wants to try everything. And then they try things and they talk about it. And then their teammates kind of like, feel a little bit more comfortable trying things is like, Oh, my teammate over there was successful with that.

58:06Angela Jiang:And so I think those are the sorts of things that need to happen. The teams that I've seen struggle are the ones where it's like, they don't have that like spark person, right? Who's like, I want to push the boundaries, I want to try a thing. And then I want to show everyone what I did. Right. And so everyone's kind of stuck with the very basic, like somebody tells them you have to use coding agents now. And then they like pop open a tool and they're like, okay, I guess here we go. Right. And, um, their mindset is not to be in, in that boundary pushing. I think if you've got a team that's like that and is kind of struggling, like literally just go find and hire one person who's like, I'm so AI pilled and I want to push the boundaries of these things because it'll be a pretty big spark with the rest of the team.

58:44Angela Jiang:Yeah. I think it makes a huge difference. Yeah. Yeah. Final question for you both. And we ask this of everyone that comes on the pod, one version of this. What's something you've changed your mind about? Angela, I'm going to start with you.

58:55Katelyn Lesse:You know, I've changed my mind a bit about how much could be like truly autonomous. I think, you know, looking back like a year ago, I would say, you know, yeah, I could see maybe like, you know, this kind of very like boring, drudgerous like flow that I don't want to do. I can see that being automated. and I can see like some kind of portions of product development and just like the software build being automated and I think as I look a little bit into the future I'm actually a lot more I think bullish on how much actually can be done by agents and a little bit to my to my own surprise I think there's been portions where I'm like there's just no way that like the agent can kind of like handle this autonomously can't or for whatever reason don't want to or it's decided

59:41Angela Jiang:that it's a thing that shouldn't be. Yep. Interesting. Caitlin? Yeah, I think when I was first starting to get into all this stuff, the thing that was top of mind for me was, okay, the model capabilities are going to get better and better and better. They're on this exponential, right? And you start to think about how is the world going to change and how is work going to change and how are products going to change as a result of these model capabilities? And in my mind, the bottleneck was the model capabilities. Like the model capabilities had to get better in order for all these things to be unlocked.

1:00:11Angela Jiang:and I actually think more and more lately I'm realizing like there's actually just this massive gap between how good the models already are and what we could be accomplishing with them versus how people are actually making use of that and how they're actually using them and I think a lot of that is just because like the world is messy and and things aren't perfect and there's you know like people have different information or people have different motivations and and all these different things and so um I've started to think a lot more about like I'm I'm maybe even like less focused than it used to be on like how the world is going to change as a result of model capabilities getting better and more like we kind of have this like existential problem to solve where if we're actually going to get all these amazing benefits out of the technology that we want to get we have to figure out how to like close that gap of where you know the reality of how it's being used actually is versus the the power of the models and so um i think that like the bottleneck being actually kind of just like the reality of the messy world is is something that uh i've i've started to realize lately and it's been interesting and how would you characterize would you call it the imagination gap or the like uh overwhelmed gap like what do you think is the thing that defines that gap i do think a lot of it is is imagination it's like agency of humans right to be like i'm going to be someone who is going to just like try all the things and push the boundaries and do these sorts of things.

1:01:33Angela Jiang:And I think some of it, you know, is actually on us to help solve for the community. Like that's actually how we define the value of the platform that we build, right? Is to close that gap for a lot of people and just give them the tools to make it so that they can build things that follow that same exponential of model capabilities. So I think it's a few things, but I don't know. It's a lot of things actually, just because again, it's like the world is kind of messy and things aren't perfect. And so finding all those ways that we can just help people stay on that same exponential is what's really been important to us.

1:02:04Angela Jiang:Caitlin, Angela, thank you so much for being on Wild Hearts. Thank you for having us. Thanks for having us.

1:02:14Angela Jiang:Thank you so much for joining us for another episode of Wild Hearts. If you want to learn more from other ambitious people building, designing and creating the world that we all want to live in, then please hit the subscribe and follow button. this podcast is a labor of love from the blackbird team and day one the show is produced by camilla herring from blackbird our marketing genius is laura coford and our editors are from day one andy jones sanjay chabaria and georgia catalan thank you all so much for listening and i'll see you all next week