GitLab’s CEO on Why the Next Great Developer Might Not Write Code | Bill Staples

32:51How can we make, you know, the full life cycle of software engineering, AI driven? Because, you know, when we study where engineers spend time, it's only like a quarter of the time that they're actually authoring code. The rest of it is all that other stuff. And it's even less fun than authoring code. I just put it that way. So even more beneficial for them to be able to delegate mundane tasks, tedious things to an agent who can go do work on their behalf. So more people writing crappy code, yes. And that drives more demand for a software lifecycle to quality and secure and assess that. And that's where we're excited.

33:35you know i think to the 10x engineer um question that's absolutely what we're like when i think of what our um asian platform should do it should make more 10x engineers and for those you know gifted 10x engineers today should make them even even better um and uh you know you can imagine a world where as a software engineer, you're assigned a set of bugs in the code and you've got to check out those issues one at a time, check out the code, look at the code, debug, analyze what's going on, come up with a fix. And you do that one issue at a time. And sometimes an issue can take days, days and days to debug.

34:24imagine a world where as an engineer you can instance an agent in fact you can instance 10 agents you can parallelize the debugging and code fixing for 10 issues at a time on the same day in parallel you can have those agents go do all the tedious checking of the code and looking for issues they have all the context of issues that have been filed against this code block security scans that have been run, test passes that have been made, previous deployments, errors in production. They can pull all that context in without you having to do it manually and assess the code to look for the potential coding error and then come back to you and say, well, here's what I found stitching all of that context together.

35:13Here's what I think the code error is. Here's a code suggestion for how to fix it. Does it look right to you, human? And I, as an engineer, can paralyze that across 10 issues, can review the work of all the agents. And when I feel good about it, say, go ahead, merge that. You know, code change. That's like, that's so empowering for engineers. Like you would take what's super tedious and a lot of toil and you'd create something that like can elevate their impact in a significant way. That's what we're building. And what about option three? I think good is always in the eye of the beholder. And so what's good today, most of the time for software is good enough is what it really means.

36:02Like every software project I've been part of has bugs. It's just a matter of finding them and how bad they are. Every software project has usability issues. uh you know you try to design them perfectly and get them just right but there's always corner cases and problems um not every software project i've worked on is beautiful and elegant and amazing um it requires really skilled you know designers to do that and i think uh over time certainly ai will get better at those things but i think that just means the bar is going to go up. Like the standard of what we expect from software in terms of quality of design, quality of experience, quality of outcomes is just going to keep going up and up.

36:51And I don't think we're anywhere near exhausting the demand for software and the demand for high quality outcomes from our software. And so I don't think this is a anytime soon kind of we're at the limits of what software can do. And when you say agents, I hear a lot of different definitions these days for what people think of as agents. How do you think of it? I think of it as an AI driven process that is specialized in particular skills. So you build an agent typically focused on a particular domain and, and, um, give it the appropriate context to go exercise those skills and execute particular tasks with those skills.

37:47And then, you know, um, you can, agents can obviously talk to each other and, you know, work with each other, share those skills and then interact with humans to get input or prompt, you know, more prompt on the particular task they're executing. Internally at GitLab today, have you put part of the team on like, all right, what's our story here? How are we going to work on this? Who's building the agents? How much are we spending on that? Like, have you had to like, kind of, Oh, it's one of our biggest investments right now, actually. Yeah. Super awesome, awesome work going on. I see that a lot of, uh, these tech companies that have been, that were born pre our modern definition of, of AI, um, since the transformer, I'm curious, like, doesn't everybody want to be working on that?

38:41My observation is that many of these types of companies that have been around for a while are all scrambling to either integrate the technology into their product or internally, right? But on the product side, I have seen so many people that are like, oh, the core stuff is boring. I don't care that it drives most of our revenue. Like, I want to go work on the cutting edge. I see how much these AI folks are making. I hear you, Bill, on your earnings report, using the word agent and all these things. Like, I want to be a part of that. Like, you're telling me that's the future. Like, why, why, why wouldn't I want to work on the future?

39:22Like, does it, does it create any of that tension in the company? You know, I, I actually think that would be an awesome thing. The more people who want to explore AI and make it part of our platform, I think the better. Obviously we have other things that we're investing in and doing, you know, one of the things that's helped GitLab grow in recent years is our security capabilities, you know, SAS and DAST and dependency analysis and container scanning. And we do, you know, a lot to help our enterprise customers ensure that they're, you know, securing their software and their company and their customer data.

40:02And it takes a specialized skill set to be good. You know, For example, the code scanners often require people who are experts in a particular language that know what kinds of vulnerability patterns exist in particular languages. And so you imagine if you're that kind of expert, your skills are prized. You have a place to contribute at GitLab. Obviously, you're probably hired to do that. And there's always more work we can do to make those scanners, to make our security capabilities themselves better. And they're working on that. But you can also imagine a world where AI could help with some of that pattern recognition on security flaws, vulnerability detection, remediation.

40:47And so it's a chance for them both to continue to contribute, but also think about new approaches to security that are AI driven. And, you know, to some extent, same thing's true of core DevOps capabilities. Like CICD, the ability to integrate code, package, and deploy it is 10 years old. And there's opportunity to rethink what role should AI play in that. And so every team has a set of priorities. Sometimes those are core quality. Sometimes those are enhanced features for human engineers that we currently serve. And sometimes that's like, let's think through how can an agent help with these use cases so that the human engineer and the agent can work together, collaborate to get this done more quickly.

41:37This company is like a$10 billion market cap. And, you know, you've been a fancy executive for a long time. You were sure the chief product officer at New Relic before, like you come from a product and technology background before becoming the CEO. So, but I wonder, like, do you get insecure about the technology? Like people that are in the technology often tell me like how they feel like they're really struggling to keep up and they're in it every day. They're not like managing people and companies and earnings reports and all these things, you know, like, uh, how do you feel about keeping up?

42:17I have never seen the pace this high on innovation. It is hard to keep up. And again, why I'm really grateful for experts around me that are more dedicated to a particular function. And in this case, product and engineering experts that can go deep and understand every single day the next thing that's coming out and new approaches. So I learn a lot from them and I study a lot myself just to try and keep up. Study how? Read and watch lots of videos and podcasts. i'm in the middle of one right now they're just dense like uh like it's amazing and if you were to re-listen to some of these like six months ago they're like almost irrelevant and half the time completely wrong because someone's talking their own book about something that ended up not panning out i have found myself carving out 30 to 45 minutes every day whether it's at night or just during work to get into these LLMs.

43:20Like I'll just live in chat GPT to just like feel the contours of the technology and the limits of what I'm bumping into and what's working and what's not. And I haven't found another more effective way yet. I'm curious if maybe, maybe I should go to reading, but to like feel this thing. you know to like feel what these things are doing yeah and where their heads go you know and what happens when i shift models even a model in open ai let alone across different models and yeah that's fascinating it is interesting to see the different kind of personalities and the different approaches i think i have perplex perplexity claude um obviously chat gpt crock i think i have them all on my phone.

44:17Sometimes when I don't like an answer, I'll go to another one and I'll get, you know, get a better answer. Actually, it's super interesting to see. And they evolve every time there's a new model, you know, there's, there's like a different experience subtly and sometimes more dramatically. It is super fascinating. I'm curious, like as a guy who asked lots of questions as part of your job, do you ever turn it around and have the, you know, the AI ask you questions? One of the best prompts that I have seen that I stole from, I think it was Toby at Shopify. Do you know what I'm talking about? It was, uh, I read his memo recently, but not, okay.

44:59I think it was Toby from Shopify. I hope I'm not. The prompt was basically knowing everything you know about me. Oh, don't be shy. Like, what do you think would surprise me about myself. And it was a very scary, deep answer. Really? Very scary, deep. Well, it probably also has the context of all the things you say in these podcasts, right? Oh, I fed it all of the transcripts and everything. Yeah. But also with memory that's been introduced not that long ago. Right. Not even memory within a chat, but memory within the system. Yep. It has everything that I've ever asked it. It has all the context on something.

45:43And so then I start to ask it questions based on, on the memory. And sometimes the answers are like freaky. Yeah. You know, I have told in this case, chat CPT, like never bullshit me. Like I always want, don't pander to me. I I always want you to be very direct. And now before it gives me an answer to something, it says no BS or to be blunt with you. You know, like it just knows, which is funny. Isn't that amazing? I think the best advice I've seen is like, stop treating it like a tool and treat it like a collaborator. Right, like somebody you want to have as a lifelong collaborator. And so like giving it prompts, like, yeah, don't BS me.

46:32really help it like better, better collaborate with you, better interact. Right. But don't you think like to your, to your point, Bill, about you going across these different LLMs, like perplexity and anthropic and open AI and Grog, maybe we're just in a moment in time now, but a founder, a KP founder recently said this to me, like with memory and he's an open AI, chat GPT has become the stickiest piece of software I've ever used in my life. And it might be the stickiest piece of software like in history because it has so much context that now if I think about like, I have gotten some dissatisfying answers recently.

47:17I'm going to like do some deep work about like some health things that I'm working on. I just been kind of annoyed by some of the answers, I feel a little bit stuck because I can't port that memory into Anthropic. Yeah. You know what I mean? It's like it becomes a proxy for a person or a relationship, right? Like, would you ever dismiss a relationship that was valuable to you? Exactly. Which is also why now, interestingly, consensus view in Silicon Valley today is that, you know, So maybe OpenAI is going to run away with this thing because they have the lead. And when you have the lead, even if it's incremental, it's tough to switch out if you've invested all of this kind of like time in getting to know you.

48:08I don't know. I think it's still early days. I do think memory is an awesome new capability. And, you know, we're excited at GitLab to incorporate memory into our agent interactions because, you know, so much of software engineering, again, is like learning through experience and iteration. And like you'll go fix a bug and then you'll get code reviewed. Somebody else or a machine will look at it and realize, oh, you missed this or you got that wrong. And that's a learning moment. The same thing happens for agents. You know, they take a crack at a problem and a human looks at it and says, no, no, no, I want this change or that change.

48:47And today the LLM itself doesn't learn from that, right? It's trained elsewhere. But if you build memory into the service and it remembers that interaction and it reflects on that interaction, the next time it makes a bug fix, it can remember, oh, I got that part wrong. Did I get it wrong this time? But is that actually working well enough where you were like, you would say, go ship that in production? Because my experience, again, just like this is kind of simple Jubin playing around with this software, is that even the way that I like, if I give it a bunch of transcripts for the podcast, I have to be quite specific today with the things that I want it to remember.

49:30It doesn't just remember everything. I have to say, okay, make sure you remember the titles of all of the guests because that's who it is. And the sequencing of when it was. If I have time sequencing questions of something before 2022 and after. Or if I want to ask about, if I want to query around like tonality, right? I have to make sure that I tell it to memorize. It's not, it's still not, I would certainly not ship anything like this. if I had customers that were counting on it. We're not there yet. Yeah. See, that's why I think we're still in early days. ChatGPT and all the other tools like it are awesome.

50:12And they're, you know, powered by the latest LLMs as they get updated. And they're getting new features like memory. But this use case that I was just describing around, you know, reflection and learning can be somewhat domain specific. So for you, you're trying to like give it the specific prompts to say, hey, I need to remember guests and title and these things because ChatTPT as an application doesn't understand that, you know, that data format out of the box. You have to give it that. And the same thing is true of software engineering. Like we use the same LLMs to power, you know, GitLab Duo, but we have to build the specific use cases like memory and reflection on code reviews in order to give it that context in order for it to then apply it the next interaction.

51:09And so that's the work that we're doing. That's the work every company has to do on top of the LMs in order to make it domain-specific for a given task. Yeah. Okay. Agreed. but can I pull that thread a little bit further and explore this with you? Then are we going to have domain-specific everything? Yeah, I mean, I think that's, of course, there's a knowledge base, there's a level of competency that LLMs are growing at all the time. But the application of that knowledge,

51:47I think is really important and often domain specific. And so that's the opportunity for so many of the applications that exist today. By embracing AI, you get a super smart, you know, competent intelligence, but then you have to figure out how to apply it for a given domain. And unless the open AIs and anthropics of the world go build every application. Yeah. And so I think that is the opportunity at the application layer that so many of us are pursuing because we have the data, we have the context, we have the human interaction today. And by infusing AI and bring agents to the table, we can power our users even more specifically than just a generic LLM can through ChatGPT.

52:41And when stakeholders, like, let's just say you think analysts are some of the investors are some of the smartest folks that you know, but like these investors, I mean, no disrespect, have no idea what they're talking about when it comes to like the future of these technologies. And I guarantee you, I haven't listened into your last earnings report, but there's probably some pretty big questions about how are you thinking about agents and AI stuff? Like, how do you reconcile that? Yeah, they're super smart at what they do, which is, you know, they study companies and look at the numbers very deeply.

53:22They look at the, you know, competitive market. They look at, they go interview customers and understand what customers are saying about you, understand even what team members or employees in the company or those who've left say about the company and how it's doing. And so they're very good at all of that and then making judgments on what do we recommend? What do we invest in? How do we do our business? They're not, obviously, they're not innovators. They don't purport to be. They don't know where technology is going. So they're doing those calculations based on the information they gather. And that's what I find super, super engaging because they, they know a lot about the business and the company and what's going on around us.

54:13And that interaction is what I, what I find so exciting. Yeah. I don't, I don't look to them for technology. Fair enough. Fair enough. I'm curious, maybe on a more personal note, how do you organize your days before you were in an office, right? like now you're not in an office i'm just curious how do you think about structuring a day a week a month to-do lists goals like how do you organize your time one of the most um helpful i think leadership uh lessons i've learned uh is to create a create an operating system basically for myself and how I lead, how I run the company. And so it's pretty structured.

55:05I have, you know, basically it all starts with people like make sure I'm surrounding myself constantly by the right set of people. Like we talked about before, great human beings with integrity, high trust, and excellence at their domain. And then a ladder up from that is like strategy. What is our vision? What is our approach to this business in order to create long-term customer value and therefore investor return? And then it goes to process. What is the process by which I can measure the progress every single week of the business. And that cadence is, I found super important. Previously, I've had a quarterly cadence because quarterly boundaries are important for public companies especially.

55:59But when I have a quarterly cadence of setting priorities and inspecting, it's just not, especially in the pace of the world today, is not frequent enough. When you set a quarter priority, it's you check in the first month and everyone's like yay we're on track this is such an awesome priority we're gonna nail this one second month you check in oh we're off track a little bit we're not sure we're gonna make it you know some of this stuff might slip some of this stuff's not working out and then the third month it's like oh shoot we we failed you know we got some of the stuff done but most of it didn't get done like that is the rhythm that is the typical pattern And so I moved to a weekly cadence now.

56:38So every day of my week is planned. And what I do is I take the priorities of the company and break them down into basically initiatives with key results that we expect to drive. And I check in with each of those, about 15 of them every single week, to understand where we are, what progress we've made since last week. and how things are, you know, how things are working out. That's super important because I, you know, I think staying in the details of things and understanding as things evolve, if there's context the team needs, if there's coaching they need, if, you know, things are surprising them that you should be aware of.

57:26So you're not swooping in with like low context when things are blowing up and saying, oh, you're incompetent or this is totally screwed up and let's go change direction another time. It's so important. No one wants that from a leader, especially a CEO. So that's how I kind of stay in touch and every single day of my week is planned with those operating check-ins across the key initiatives of the company. I try and spend several weeks in the quarter as well with customers because I feel like that constant feedback and interaction with customers just keeps me with that customer mindset and speaking the voice of the customer, both to the team and to the investor community.

58:11So that's something I try and prioritize every quarter as well. And then, like I said, I manage my energy. So it looks different in terms of unstructured time every day. sometimes it'll be you know going deep on writing a piece of um you know content for the for the external or internal audiences or sometimes it means you know reading a bunch of things and getting getting sharper so your energy management is unstructured yeah well i'm constantly managing energy and that well i guess by design because you have to flex to what your energy is or isn't Yeah. Yeah. I mean, I try and create enough energy every day to do the structured work.

58:56And then when there's unstructured time, I decide, do I need a nap? Do I need to eat? Do I have the energy to go going on something else? Do you block off time for dinner with your wife? Yeah. Every day? Every day. Blocked? Yep. What time? Six o 'clock. For an hour? Minimum an hour. I try to make it two. That's nice. Yeah. Phones away. That's one of my bad habits. My wife always gives me a... The phone's there. I try to keep it away, but it's a terrible habit, isn't it? Yeah, it sure is. I don't know if you struggle with that, but yeah. I have just been like a do not disturb monster recently, where I became so attached to the buzzing in my pocket that it forget about like, you know, dinner with my partner or whatever, like it was constantly interrupting everything.

59:52And so I've gone full, do not disturb for hours, hours on end. It's been really helpful. And certainly at dinner, I try and I'm a pretty efficient eater. Some would say I, maybe a little too efficient. Uh, uh, but I try and keep my phone away for those good for you 30 45 minutes i gotta do better just just for peace of mind yeah and then you go back to work yeah usually eight o 'clock yeah 7 38 yep and then are you like uh do you stay up late no actually i usually try to um i usually try to go to sleep early because then i wake up early it does depend uh like 10 p.m i like to try to sleep yeah and then usually up sometimes as early as three sometimes five yeah six the latest depends on if i'm up working in the night you know if i've got the energy i'll i'll work at midnight or three in fact one of my teams from 20 years ago it's funny just uh texted me the other day um i can't remember what exactly what we were chatting about, but they're like, we were just talking about you and your 3 a.m.

1:01:04emails, how much we miss them. That was like 20 years ago. Like still, you still do that. Still do it when I, you know, when I've got something I'm passionate about. And then you'll just adjust your hours to later if you're up till three. Yeah. Or you'll just bear through it. I manage the energy. So yeah, if I, if I'm not feeling like I can bring myself to the game, I'll take a nap in the middle of the day or, you know, go to bed earlier that night. Yeah, there's really not too many rules for you as it relates to energy management. I mean, I've committed my life at this point to the company. And I love what I do.

1:01:41And so I do as much of it as I can. How much do you eat? Do you eat every meal?

1:01:49The worst part is being on the road traveling because then you're usually doing dinners with customers or team or whatever. and the hours and restaurants are always different. When I'm working at home, I try to have a good breakfast, moderate lunch, and then dinner with Shelly. But it's usually pretty light on the breakfast and lunch. It's tough on the road. It is really hard. It's tough on the road. It is. Yeah, that and the sleep too. Sleeping in a different bed every night. Sleeping is way worse, actually, I think, than eating. Because eating, you can at least just order what you want from the menu.

1:02:24Yeah. Well, when you're your age and you can eat about anything and not gain weight. It's so hard to eat healthy now. And finding a way to like get a sweat in. Yeah. Exercise is also tough. My experience has been that when I'm traveling, I can't tell if I don't sleep well because I'm in a bed and that environment, the temperature, I like it cold. I can't get cold enough in the room or if it's because when i'm traveling my mind is so in work like i have no time with my partner for dinner or whatever like i'm just it's all work and then i go home to the hotel and usually i've been in meetings all i go back to the i'm in meetings all day catching up on email till right like in bed yeah i rarely do yeah yeah and so then my brain just keeps going.

1:03:19So I don't know if it's, uh, if it's, if it's mental or physical, it's both, I think, but, and it's compounded worse. Yeah. Yeah. Yeah. It's, that's why you just got to work so hard that you're exhausted and sleep. That's right. Yeah. I appreciate you doing this. Yeah. My pleasure. Really fun. Really fun. I'm excited to see like your, you know, you're a few months in two months, no six months. Uh, yeah, December 5th. So So coming up, yeah, six months. Well, I'm excited to see how the next few quarters and years go. Thank you. Really exciting. Thank you. Are you hiring? Of course. What are you hiring for?

1:04:02Everything. And hiring anywhere. Yeah. That's the beautiful thing. I think we have team members in 65 countries right now. That's wild. And no offices. So everyone's working. Yeah. When you hear the word grit, what do you think of? I think it's becoming comfortable with uncomfortable things so you can take on hard problems. I think for me as well, what I aspire to do is that with a smile and make it look easy. I'm still working on that because I think to accomplish great things, you got to tackle hard problems. Sometimes I forget to smile. So I'm working on that part. Those kind of people inspire me.

1:04:53Bill, thank you. That's it for now. If you liked the episode, please leave us a review or go back into the archives where we've done more than 200 episodes with some fantastic folks. This podcast is a Kleiner Perkins production, and I'm Juven. Thanks for listening.