How to Build a Technical Strategy That Solves Business Problems | CircleCI CTO, Rob Zuber

23:27And so we're trying to measure the risk of taking that bet, right? And there's lots of ways to de-risk bets, right? Again, how can I learn a little more to sort of make a better decision? But if you treat it as that, this is an exploration to get new information, then it's a much more comfortable conversation, first of all. What did we learn? Oh, we learned that that was the incorrect hypothesis. Cool. No one's upset about that. No one's emotions are tied to it. We figured something out. That was our plan was to learn something. I just finished this book, Modern Software Engineering. And I think it's Dave Farley's book.

23:59He does a great job of talking about applying the scientific method, if you will, to building software. and all of science, like everything that we know came from guessing and then testing. I don't know, maybe it's this. Oh, can we get empirical evidence that that's true? Oh, actually, it's been proven wrong because this other thing came up. No one's like, I failed as a scientist. They're like, I did my job. I learned something really interesting about how the universe works or whatever. And if you think in that way, then all you're trying to do is get better information all the time instead of, again, tying everything to this was my idea and now it's a bad idea sort of thing like that does not allow you to move forward.

24:33It does bring up another risk, though, where some folks get so obsessed with this idea of being right the first time or like really nailing it. They spend too much time on that. Like, let me figure out my hypothesis. Let me strategize. Let me try to protect the future instead of saying, OK, let's get my hands dirty. Yeah. Make one hypothesis. Try it out. Fail. Learn from it. Have more context and information and go apply that. Yeah. We have tried to instill, and this is hard, I'll be honest, that even doing things that you know are the wrong direction, if you will, can be useful if they will give you new information.

25:07So in an incident handling situation, right? So let's say we're having a production incident. We would want to restore service to our customers first and foremost. That's the most important thing, right? And saying, what if we turn this thing off? Well, we're not 100 % sure. Or what if we roll this back? Let's take a really simple example. What if we roll that thing back? Well, we're not 100 % sure that that's the problem. But if we roll it back and it doesn't fix it, now we know it's not the problem. We don't have to debate it anymore, right? Instead of feeling like, oh, but we need to have the right answer in order to make a decision right here.

25:37Like even doing things, that's not necessarily the wrong direction, but doing things that we aren't confident in, if they will tell us something new and we can gather new information about the situation is a completely reasonable approach. And then if it does fix it and we don't understand why it fixed it, let's discuss it in the retro instead of while customers are not able to do their work sort of thing. Right. So like a that's like a little bit on the side about having clear priorities and what you value. But B saying if it gives us new information, then we're willing to try it. And, you know, we apply that across a lot of things where we say, oh, you know, we're not really sure if this is important or if this is important.

26:13Like, let's get rid of it and see what happens sort of thing. Right. Like particularly in this year that we've all been operating in from a macroeconomic perspective, there's a lot of, you know, do we need all these tools? Do we like can we be more efficient sort of thing? And you sort of, you know, you look around and see, well, someone's using this and someone's using this and whatever. Could we consolidate on one? And, you know, well, that would slow us down in this spot, but would it speed us up overall? Right. It's sort of like trying to zoom out a little bit and look at what's the big picture and will this drive us towards our goals?

26:42And that might feel a little backwards over here, but we can make that decision to sort of drive all of us forward. I think this aligns back to one of the first themes you mentioned around how to create great technical strategy and make sure you hold it forward, which is getting business context. Because what you're talking about is two things. One, it's getting technical context, you know, on an incident, on something else. Like this is something we already do or should do as a best practice. And so then if you apply that out to saying, OK, we also have to be business leaders at a certain level.

27:09Great. Then it's logical. you should try to get the business context from these communications, hypotheses, etc. And the other pieces, I would say, where you're thinking about efficiency of tools, thinking it through, in some ways, that's also business context. That's CFO context of saying like, we need to cut our burn rate, or we need to reduce our tool spend, or simply we need to manage less tools because the overhead of that is challenging for our platform team, or it's challenging for just organizational throughputs because it creates these communication barriers. and, you know, it creates information barriers.

27:43And so I'm hearing this theme for you where like you deeply value learning in context and I think it applies directly to your talk. And so I'd love to zoom back out a bit and say like, what are other things that you view as key to maintaining this crucial technical strategy direction? It depends on your level a little bit, but I would say for me personally, I even think of technical strategy as secondary because what it allows me to do is focus more of my energy on my first team, which is the executive of the company, right? When you're a CTO, your peers are not mostly technical leaders, right?

28:21They're the CFO and the CEO and the CRO and whatever. I don't necessarily think of it as just collecting context and putting that into technical implementation. Like my job is to solve the problems of the business. And if technology is helpful, I will apply technology. If my team or my organization or the engineering organization is helpful or how they're behaving needs to change or what they're focused on needs to change, then absolutely, like I will use that knowledge to drive that. But at the end of the day, I'm trying to create more alignment, whatever, so that folks are more effective and I don't need to be as involved so I can go focus on the things that really matter for the business.

29:05right? I mean, I am a salesperson for the company. I am a customer focused leader. I concern myself with the issues of legal and finance and everything, right? And we as a team look at all those problems and work to solve them together more than everyone focusing on their own department and then coming and reporting status, right? Like we don't deliver effectively as an organization until we as an executive are working on those problems together. And yes, we have specific skills. We have specific organizations. But as a CTO, if I don't understand in pretty decent intricacy, how our financial model works or what our legal concerns are, but all of those pieces matter.

29:50Yeah. And I can help the legal team make good decisions about how we should be structured globally, right? I can help the finance team make good decisions about what we should be spending in different areas or look at where their concerns are and say, oh, I think I know a way to solve that problem that may not even be coming from a technical perspective, but just from my understanding of the business and the market. That brings up two things. So one, I hear you talking about helping provide context, right? Like CFOs need context into how technical leaders are thinking too. Like you have a role in providing them that and helping share, oh, this is why we're thinking about the product this way.

30:28This is why we're engineering this this way. So I think that's, that's something that I would love to talk a bit about and understand how you approach that. Because yes, like absolutely what you're saying, like we need to get contacts from the rest of the business, but you're also at a role where now you start providing it. And I think a lot of folks who are maybe at that director level for the first time and are starting to have to do more of that, would love to understand how you approach any best practices. How do you then bring that context that you gather back to your team effectively? I think it's a little bit collaborative problem solving, right?

31:02Like where I see these things come together. And, you know, there was an interesting talk on AWS cost in the last, you know, couple of days. I'm sure cloud cost is exciting for everybody. There's a lot of situations where folks in engineering, for example, feel like finance provides them a set of, you know, a set of budgets. You can spend money on AWS, you can spend money on people, you can spend money on whatever. Let's say R &D, right? So I'm thinking like in accounting lines from a public statement, but R &D is going to spend a bucket of cash and try to drive the maximum outcome for the business.

31:38And I, as a technical leader, can say, actually if we spent less on people over here and more on third-party software or actually reduce this part of third-party software and put that into our cloud spend or whatever, we would have greater impact for the business, right? So that's not necessarily something that someone in a finance role is going to see. Like I need to help them see that, help them and learn from them. Like what is the difference if I hire a contractor versus a full-time person? Like what What does that mean for the company? Do I want to buy a three-year commitment to get an additional discount right now?

32:15Or is that too much risk for us as a business, right? Like all of those things are places where I have useful information and my CFO has useful information. And the best outcome is not going to be them telling me something or me telling them something, but us sitting down together and saying, here are our options. Let's figure this out. Let's understand the trade-offs. Yeah, exactly. And it's going to take both of us. And I think that's not something that has to happen only at an executive level, right? I think, again, at a director level, like every director has a finance partner or something like that, like someone in the finance org who's working with them, and they should be having the same conversation, not here's your cell in the spreadsheet, try to stay under this number.

32:51But this is what I'm trying to achieve. Here's what I think it would cost. Here's the impact that I believe we can have on the business. You know, are we in a position to spend that cash right now if it's going to return in 12 months? Or do I need to be spending cash that's going to return in six or in three? How can I think about structuring that? Like that's a conversation that I would want directors to be having. And if you're not having those conversations, you're not preparing yourself to then be a VP where you're definitely having them or a CTO or something else. If I'm a director or maybe even just like a senior engineering manager who wants to take a step to be a business leader, how should they be thinking about that level up process and how to grow?

33:27Yeah, it's a fantastic question. And I always struggle a little bit in these conversations because I've been a CTO for like 16 years by starting company. And so that's one option. If you go run your own business, you're going to learn all this stuff very, very fast. You know, you spend a lot of time with your investors and you'll spend, I mean, you will be the CFO at some point because you won't have one yet. And it gives you a very, very good perspective of what matters. Particularly when you're trying to find product market fit and trying to be super efficient and, you know, be nimble. Like that's a very good perspective.

33:56You don't have to do that. It's not for everyone. I mean, people are stressed out right now. I would I say it's like a very high stress thing to participate in those conversations. I mean, whatever level you're at, there's someone above you who's having a bigger conversation about that topic. And I would say I don't have a lot of people coming to me saying some of my folks. I mean, to be clear, I actually don't run the engineering organization. I have a sort of small group of very technical IC folks that work for me. But some of those folks are interested. Help me understand, you know, what the CFO is thinking about right now.

34:30And I'm happy to have that conversation because I have enough context about what the CFO is thinking about because we were very close to be able to share that with someone. And then if they said, I'd like to understand more or tell me about this. And I said, I don't have any answer to that question, but she would love to talk to you, right? Because that's helpful to our CFO as well. I mean, I'm not going to send 200 engineers to talk to the CFO, but people who are keenly interested in trying to get that growth or, you know, let's have a session. Maybe if all of you want to understand, like if I say enough of that interest, all of you want to understand, let's have a session and talk through the things that we're seeing and the things that, you know, our finance team is seeing and what really matters so that we can each get a little bit more context about what matters.

35:06I don't know if it's everyone, but I'd say folks are generally reluctant to ask around, Hey, you know, this, this is out of my wheelhouse. Therefore I shouldn't talk about it. But those things out of your wheelhouse are the things that are going to help you be a good business leader. And ultimately growing as an engineering leader is about growing as a business leader, because the more of the value and impact that you can to have around the business that you can bring, the more you're going to be able to move up through those roles. Like at some point, you're not about technology anymore. You know, everything you need to know other than the ability to learn new tech or understand the impact.

35:39And what you need to know now is how the business functions and how to be impactful. That alignment piece and bringing that alignment back to your team, as you brought up, is also a crucial skill set. Are there any other best practices that you had mentioned around helping share all the contacts you gathered or encouraging others to gather contacts themselves. I had this moment recently, which was, I'll just describe it, where I had received some questions about some other stuff related to how we were structuring teams and some sort of low-level stuff. But the signal that I got from that set of questions was that people didn't understand, you know, within parts of engineering, basically how CircleCI makes money.

36:17And I don't mean we charge customers and therefore we get paid and we make money, but literally like how we really go to market, right? What our growth and expansion model is inside of our customers, et cetera, because they think about the user and they think about the problems of that user, but they maybe don't see, you know, our go to market motion and the split between what happens with self-serve customers versus what happens with, you know, customers where we have a sales engagement, those sorts of things. And so I basically kind of drew a bunch of diagrams and did a talk at one of our R &D call hands, we call it whatever ridiculous name, but anyway, an all hands that's on a call.

36:52Yeah. And just walk through step by step. This is how people join. This is, you know, this is how, what it looks like when they're free. This is what they, how they pay us. This is what revenue recognition is. We're like very sort of accounting pointed terms, but that matters in terms of, you know, how we build features, honestly, like what it is, what kind of capabilities we're trying to give to our customers. So they'll be successful on the platform. You know, there might be big opportunities like that to say, okay, like Like just this is how our business functions from a purely sort of someone else's perspective, right?

37:22A finance perspective, a go-to-market perspective. And understanding that picture, I mean, maybe doesn't change everything that they do every day, but might change the way that they think about a very specific problem. And, you know, one thing that I often say is the number of decisions that your developers are making every day is you can't keep track. Every line of code in some fashion is a decision, right? I'm making a trade-off decision every time I hit the keyboard and you're not going to be involved in 99 % of those decisions. And so trying to give people that bigger picture understanding, which again, you can do at very large scale.

37:58Also at smaller scale, you have a whole management structure, right? So you want your directors to be really clued in so they can help the managers understand and also translate that down into context for that sort of more localized pursuit. just as a CTO telling everyone everything about what's happening in the business, probably a little overwhelming, take a really long time. And often ends up being in a place like, okay, I think I understand that, but I don't see how that applies to what I'm doing. And so using director, manager, senior manager, maybe in there, depending on what your structure is like, to localize those problems and add the context of the challenges of a particular team, right?

38:40So you can imagine, while we have certain teams where like how money flows through the system is really critical. Right. And then we have other teams who maybe aren't thinking about it, but that might impact, oh, if we, you know, if we built this feature in this way, that would help us grow, you know, free users, which is helpful to us in the long term. So how can we do that? Yeah, I think this scaling thing you're talking about is really important too. Like we think a lot about like as leaders, how do we scale ourselves and scale our organizations? And one of the things that really underlies a lot of the approaches you're taking here is like you clearly trust that your team will get contacts, be provided contacts by you and others, and then make good decisions.

39:17And there's a lot of research that shows that when engineers or any team members feel that trust, they perform better too. They understand what they need to do. They keep doing that learning process and they feel unable to do their best work. What's your approach to building that trust in your organization and making sure you have the right people in place? Yeah, it's a great question. I mean, that's an ongoing pursuit, obviously. And I mean, I'll always hedge, like we are not perfect, right? Like there are places, yeah, there are places where I think we do this well and there are places where I think we struggle and that can depend on people, you know, can depend on people involved.

39:56It can depend on the particular context. Like I think if you're a CircleCI engineer, understand, like we talked about money, Like that's not the core of what we do. Absolutely. We need to charge people and we need to make money and all that. But that's not how most of our developers, for example, think about the product. Right. Right. They use the product every day, unsurprisingly, to deliver their own software. And so if you're working on those features, then it's really easy to have context. Right. Because you have this like, oh, I know this would be easier if I did this. Right. And you can go talk to your PM and you can sort of share that locally.

40:29whereas you know what we don't necessarily see i won't go all the way into like payment systems but what we don't necessarily see is really large enterprises right the the challenges of really large enterprises are not as close to our challenges as individual developers because we're a smaller company right things like socks compliance right public companies and sort of separation of duties and things like that that we you know a lot of us earlier in our organization thought, nobody needs that, right? That's not a good way to deliver software versus, okay, so now we have a bigger picture. So yes, how do we take that and help folks see it, right?

41:08And a good part of, we talked about talking to customers. Obviously, we have PMs, but who gather that at a higher level, but really encouraging the discussion, right? Oh, that's interesting. Why are we doing that? Like, help us understand, give us some examples of customers, whatever that might be to understand this is why they're asking for it. This is how this works. And that works well in some cases and doesn't in others. But trying to get everyone to understand the customer and that particular customer problem in a way that they can come up with creative solutions. You know, it's one of the things that I've always loved about CI and CD, Agile, whatever.

41:45Like we talked earlier about, we're probably going to be wrong, right? And we're all about getting faster feedback. and one of the key elements of that fast feedback for me is that as an engineer, I am putting code in front of customers constantly. And so they might not be calling me and saying, hey, Rob, like I don't like this feature, but I can see the metrics moving. Oh, we put this thing out and yeah, everyone loves it. Or we put this thing out and no one's touched it. Like, what did we do wrong? How can we learn more? We do lots of user research, which is kind of in the middle and we have access to the recordings and sort of backroom access and stuff like that so that like individual engineers can go really hear how customers are thinking about the problem.

42:26And I think all of those things to get you that fast feedback directly from customers really connects you as an engineer with the problem, right? I mean, I grew up through waterfall and 12-month delivery of products that nobody cared about and nobody wanted by the time we got them in the market. And the first time I did continuous deployment, which was like 2011, I was terrified, right? Like deployment used to be seven gallons of coffee and four all nighters in a row to try to get the thing to work. And I was like, we're going to do that every day. That sounds awful. Right. But then we tried it and I was like, this is amazing.

43:00I will never not do this. Right. I will never stop delivering value to customers as quickly as I think about it. Here's my idea. Now it's in the hands of customers. Like there's a little typing involved, but that's going away apparently. And so, so then like, why would I stop doing that? Right. And the thing that I love about it is that I understand how the customer is responding. And that direct connection and direct feedback allows me, to your point, to be more engaged in the problem. More so say, okay, I understand how to do something for my customers. And I feel that, I don't know, call it a dopamine hit if you want.

43:33But I feel that positive energy towards delivery. And when it's negative feedback, great, I know how to fix it. Versus, okay, let's sit and talk about it for a while. And then this person is going to talk to that person who's going to talk to that person. So I think everything we've done around software delivery in the last 20 plus years has really got us to this place where we're connected to the customer in a way that's really engaging. Yeah, this is a generalism, but something we think a lot about is like merging devs or happy devs. You want to deploy code. You want that dopamine hit. You want to see how things worked and then learn and grow off it.

44:04We like problem solving. And I mean, frankly, every time we put out a podcast, I get excited too. I'm like, oh, great. Like we're putting something out there. Hopefully it has value. I think a lot of what you're saying is great advice for technical leaders at every level. You know, get business context, understand the customer, you know, talk to your peers, learn, learn, learn, learn, iterate, test hypotheses. And I would love to understand a little more in depth what advice you would have for people who want to go be technical founders. Because I know a lot of our audience is like, okay, I'm an engineering manager.

44:34I want to be a director or I'm a director. I want to be a VP. And I think you've given some great advice for them. But I know you also have this unique perspective as a multiple time founder who's had success doing it. It's going to be a ride. It's hard work. That's not really helpful. I mean, I think a lot of my approach as a leader comes from that, right? Like, so a lot of what I'm saying is how I would approach starting a business. Even more than in anything you're doing now, the faster you can get feedback about your idea, the more chances you're going to have to get it right. and to go to the sort of simplest extreme of that, like when I want to launch a new product, I'm going to start with a landing page, right?

45:18I'm thinking about this idea. If I put up five, 10 landing pages, which means just, hey, coming soon, put in your email if you want to be notified. Does anyone respond, right? Does anyone click on the ads that I put up? Does anyone put in their email? If no one cares about this thing, how much money am I going to invest in it, right? I think there's a good amount of aura put around like founders who have this amazing idea that no one believes in, but they have this tenacity and they go after it for years and then they succeed. That is an outlier. That is an extreme outlier. What most people do is pivot, pivot, pivot, pivot, pivot, like try to figure out there's something here.

45:57I know there's something here, but I need to get it into a shape that people are going to respond. And so when I talk about multiple landing pages, like if I phrase it in this way versus, okay, people are interested in this idea. But if I explain it this way versus this way, if I call it this versus this, does that drive differences, right? These are like$50 experiments, right? I talk to people who are thinking of founding companies and they're like, I'm going to mortgage my house and hire a team and do it. I'm like, please stop. like until you know that you have product market fit or something some signal that you're on the right track the cost of experimentation today is almost non-existent right like i can spin up a single host in a cloud i can run a lambda job to respond to requests i can doesn't pay for a subscription on i don't know if unbounce still exists but like a landing page site the the cost of experimentation should be like it shouldn't even show up on your personal monthly budget is what I would say, right?

46:51And so there's so many opportunities to learn and figure out where you have an opportunity. And then from there, don't lose that discipline is all I would say. Like, it doesn't matter how big you get. It doesn't matter how fast you're going. Like stick with doing the things that you have. Some evidence are going to drive value. And if you don't have evidence, find evidence, right? That same, like, how can I test the hypothesis that tells me if I'm going the right direction or not? Because the faster you adjust, the more time, right? Let's say you're funded at this point, right? You've got cash in the bank and you need to put that cash to maximum use.

47:28Like, are you going to spend it all building the first version of the product that no one likes? Or are you going to build 18 versions in that time, right? Because if you build 18, you got way more chances to get it right. And so that discipline, I think, is super critical. Keep making hypotheses or bets and getting more information and learning from it. And I mean, you brought this up or alluded to it earlier that, I mean, things change really fast, both in technology. I mean, yes, ChatTPT has been technically out for a while, but it went viral really fast. And now it's this normalized thing and LLMs are everywhere.

48:00And we're all talking about how can we apply an LLM within our technology? How should we be managing our data? Two years ago, not everyone was doing that. Or look at just what's happening in the business world broadly. I mean, a couple of years back, like the balance of how people were hiring was very different. there weren't massive layoffs that were happening maybe you could fully grow a business off of oh we're selling to small startups and scale ups and it's a lot harder now because money is a lot more expensive just looking at interest rates and things change fast we went from boom times to not so boom times and if you lose that learning that you keep on bringing up this through line that I hear from you it's so easy to be caught flat footed absolutely I think things are always going to change right changes the job, right?

48:50Like if you think about it, even if you go to the very, very micro, right? I actually did not study computer science. I have an engineering background and almost every other engineering discipline has way less flexibility, right? I design a bridge, I build the bridge. Like - It's gonna be around for a while. Yeah, exactly. I mean, unfortunately the new Oakland Bay Bridge, they had to retrofit, I think two or three times right after building it. But for the most part, like you're committed to a design by the time you start implementing it, right? So the process, the process of designing of like multiple engineers signing off and, you know, lots of research up front, et cetera, is that's how those things have to get done.

49:27But software is like, it's so pliable, right? So you can start from, you know, everyone builds their hello world, right? And that's it, initial commit, get, init, whatever. And then everything after that is a change. I change it, I change it, I change it, I change it, right? And so orienting even the software that you build at the smallest level to be prepared to change is the job, right? Because all you're doing after that is changing it. And so you want to make it easy to change. And so if you take that discipline and apply it to how you think about business, you're going to put yourself in a really good spot, right?

50:02Like we want to be well positioned to change. Doesn't mean we should change all the time, right? Confusing our customers, launching different products, whatever. Like that's not a great place to be. But there's always going to be shifts in the market. There's always going to be dynamics. Like we're going to go from cash is easy to come by to cash is hard to come by. Right. So we're changing the model of how we operate our business. We're going to go from AIML is nowhere to if you don't have it in your product, no one wants to talk to you. Right. So how fast can I make that change? Right. And if I am 100 % committed, all of my weight going in one direction and now I have to turn, that's way harder.

50:40And it's going to leave me exposed. when someone else is going to grab that idea and run with it in another direction. Actually, you brought up, I think, an important piece of advice to founders, which is just buy a.ai domain. That's the first thing you should do. After that, you know, it's a little easier. It does bring up something I want to ask you, which is what are the changes that you see coming next? Are there things that you have your mind on? You're like, you know, I'm anticipating this, whether, you know, broader market conditions or technological things that you see coming down the line that people should be thinking about.

51:11we're definitely in a time of uncertainty, right? Like ask anyone about economic, you know, future economic conditions are going to get a broad spectrum of answers. So, I mean, there you go. That's like your set of possible futures is quite wide. I'm not going to say this is the one I'm betting on. What I'm betting on is a high variability in there, right? So then your questions are, how do I hedge, right? Which I think is, I've been asking that question a lot more lately than I have in the past. And I think it's a really interesting question. I don't know if people are familiar with like financial hedging and I'm not going to talk about that model.

51:41I'll just say that's a mistake I've made before, right? Like thinking I had the potential future in mind and going all in and going jumping on it. And when it works, it's great. But when you're wrong, the risk is high. And I would say in those kinds of conditions, another thing that I would highlight, which I think is good for founders and for many folks is I can only think of it metaphorically, but knowing what your stop loss is, what is the point at which if this is not working and we can't prove it, we are going to call into question this plan. I love this expression, a Ulysses contract, and I'm not going to explain all the details, but it's basically like when I am of sound mind, I make the commitment to certain behaviors such that when we are in the moment, right?

52:23It's so easy to say, what if we just keep going? What if we just keep going? I like my favorite examples of this are all from like mountaineering disasters. And I'm sorry for bringing this up, but like a turnaround time where you say, if we are not in the final stretch by 2 p.m., we are going back down the mountain because then we're going to live to like do it another day. Yeah. And every mountaineering disaster story you read has people going past the time. Like, oh, we get the summit. Yeah. We're just, but we're so close, right? We're so close. Maybe we're oxygen deprived. And now we're like, right.

52:49Because you're making terrible decisions. Yeah. Right. Right. And we may not be at, you know, 8000 meters or whatever in our day jobs, but under stress. Right. Oh, my gosh. We've like burned a bunch of cash. And now we're feeling stressed that we have to make this work versus we've proven it doesn't work. How much time do we have to pivot or like what is our hedge? What's our other option? You know, all those things I think are good to frame early. There's this belief that founders are people who commit to this one idea and they just keep, you know, charging forward. Don't let that be your thinking.

53:23Absolutely, there are points where people will question and you have a vision and you need to pursue, but you need to decide like where that is. Because if you just think the first thing you thought of is going to be amazing, you know, you can burn tons of cash, tons of trust, tons of future opportunities by pursuing something sort of without checking in, I guess. Yeah, that check-in is crucial. Rob, I've really enjoyed this conversation. Do you have any closing thoughts you want to share with the audience about what's coming, advice or anything else you want to mention? Prepare yourself for change.

53:52I think the thing that I would want to land out of all of that is that approach makes things easier. I think we often end up in this spot and we're all under a lot of pressure right now. We're all sort of exhausted from just a few years of what's been a grind. Oh, man. And it feels like, you know, you're asking me to do something more maybe. But it generally pays off to put yourself in the situation where you are more adaptable. So I would take that extra time, like focus less on what's the thing I have to do right now and a little bit more on how do I set myself up, particularly as leaders, for a wide variety of outcomes in the next couple of years.

54:31I mean, we at CircleCI, unsurprisingly, trying to adapt and focusing on adapting to, you know, this whole shift in AI ML. How do we put capabilities in our product? Are we prepared for that change? How do we support other people trying to build those things? Like what does software development look like over the next couple of years? That's super fascinating. I'll just say, if anybody wants to talk to me about that, come find me. There's a broad spectrum of possible futures, probably more than we're used to at the moment. and so take the time to step back and look at how are we going to act in each of these situations and how do we set ourselves up to be successful across that broad spectrum.

55:04De-risk a bit and understand what could be coming because you'll be more prepared for it. I think that's a great insight. Do you want to talk about what CircleCI is doing in AI ML? I'm happy to dive in there a bit more. I would say that there's two things that we're very interested in. One, again, is how do we help people deliver software faster of any kind, right? We launched this error summarizer So you have a stack trace, you know, you can click a button and get an English sort of explanation of what the problem is and how to fix it. We're going to continue to iterate on tools like that. We've in the lab at least demoed self-healing pipelines.

55:37So if your build breaks, we'll just fix it for you because we have all the tests. We know what it looks like to be right. So we can generate code that is right according to your definition, even if your code is wrong. Right. That's a pretty cool spot. That's pretty bad. Yeah. And then on the other side, many of our customers are in the same situation, right? They're building AI-enabled capabilities into their products, and they're unfamiliar with how to test those, right? We're, as engineers, we're all used to determinism. Like, two plus two has been four for a really long time, and now it's like somewhere between 3.1 and 4.9.

56:07It's probably okay, right? And so helping folks just learn and understand how to do that effectively in their standard software practices. Like, we don't need to reinvent all of software delivery to deliver software that happens to be AI-enabled. And so including some additional capabilities around non-determinism, stuff like that. And then we're looking at all the other tools that we have and helping folks apply them. Like I'm putting out a new, I'm testing a new endpoint in a product, right? Does that look much different from I'm just rolling out a new version of something like canaries, blue-green, feature flags.

56:44Like we've built all the tools for these things over time. And software engineering has solved a lot of these problems, but we're at this interesting intersection of kind of two communities, folks who have been building AI ML in the lab and folks who have been delivering software that they expect to behave exactly the same every single time. And so we're bringing those pieces together to help those software engineers deliver with the tools that like, honestly, the AI ML folks have built all the tools. It's just that the software engineers don't necessarily know that they exist or how to use them.

57:12So putting that into your delivery pipeline so you can continue to deliver effectively. And I always tell this joke or whatever, I guess it's like self-deprecating. But when your CTO shows up and say, I don't really know what AI is, but we need it in the product by next week, right? And as a developer, you're like, what do I do? And the reality of the situation is so much of this stuff has been done for you and you have all the pieces. And so just helping people sort of see that path and continue to deliver effectively. So that's what we're focused on at the moment. I mean, it's an awesome opportunity, I think for everybody.

57:42People are building some very, very cool products. People are building tons of products that are not cool, but it's experimentation, right? It's like absolutely be hypothesis driven because this is brand new. We see tons of possibilities. Not all of those possibilities are going to play, but some of them are. So, you know, be quick, iterate fast. Don't go spend your time learning everything there is to know about ML. Send a request to a foundation model and put something in your product and see if it works. Right. And then when you know that it works, you can worry about how do I scale it? How do I optimize it?

58:13How do I deal with cost, et cetera? Fantastic. Thank you for all these great insights, Rob. I've really enjoyed our conversation. if you want insights from more engineering leaders just like Rob, make sure you tune in every week whether you're on YouTube or Spotify, Apple Podcasts, we're on all your podcast players and you can also check us out on Substack, devinrupted.substack.com we put articles out there every week along with information on the podcast and thanks for tuning in, hope you enjoyed the conversation thanks for coming on Rob. Thanks for having me it's great to be here